Compare commits

...

7 Commits

Author SHA1 Message Date
Patrick Buckley 8068ae105d chore: bump version to 1.4.0a2 2026-04-14 11:17:06 -07:00
renovate[bot] 6e99bb8b0b chore(deps): update dependency hls.js to v1.6.16 (#354)
* chore(deps): update dependency hls.js to v1.6.16

* chore: download vendored hls.js files + add hls to workflow detection loop

The wheel-completeness check failed on the Renovate bump because
vendor-js.yml only iterated katex/hljs/mermaid — so hls.js PRs
never got their files auto-downloaded. Adding hls to the loop so
future Renovate bumps are merge-ready without manual intervention.

Also running the update now to fix this specific PR.

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Patrick Buckley <buckleypm@gmail.com>
2026-04-14 11:15:36 -07:00
Patrick Buckley eb59cdefda feat: pass resolved capabilities through to providers, add server com… (#352)
* feat: pass resolved capabilities through to providers, add server compat layer

The LLMProvider protocol previously forced providers to re-derive
capabilities from static lookup tables, ignoring config overrides set
via the admin UI or config.toml (e.g. thinking_mode, token_param).
This adds an optional capabilities parameter to create_streaming and
create_completion so the session can pass its config-merged
ModelCapabilities through to providers.

On top of this, adds a server compatibility layer for local model
servers (vLLM, llama.cpp). Profiles suggest thinking mode and server
workarounds (skip_special_tokens for vLLM, reasoning_format for
llama.cpp) during model detection, with structured admin UI fields
for server type, thinking mode, and extra body params.

Verified against real vLLM (Gemma 4 31B) and llama.cpp (Gemma 4 E4B)
servers.

* fix: defensive copy in _finalize_extra_body, expose thinking_param in UI

Shallow-copy extra_params and its chat_template_kwargs in the provider
before _apply_thinking_mode mutates them, so callers that reuse the
same dict across models are safe.

Replace the hidden thinking_param input with a visible text field
that appears when thinking mode is enabled. Shows the default
"enable_thinking" and hints that Granite/DeepSeek use "thinking".

* fix: address Copilot review feedback on admin UI and server compat

- Preserve unrepresentable thinking_mode values (e.g. "adaptive") in
  raw capabilities JSON instead of silently dropping on edit round-trip
- Validate capabilities and extra body JSON are plain objects, not
  arrays or primitives
- Deep-merge chat_template_kwargs from extra_body instead of silently
  dropping, so operators can extend/override template kwargs

* fix: hide server compat section for non-local providers

The Server Compatibility fields (server type, thinking mode, extra
body) only apply to openai-compatible (local model servers). Hide
the entire section when the provider is openai, anthropic, or google.

* fix: normalize capsObj to plain object on edit load

Defend against DB rows where capabilities is a JSON literal null,
an array, or a primitive — previous code would crash on the
capsObj.server_compat / capsObj.thinking_mode reads. Same defensive
check also applied to the server_compat nested value.

* refactor: extract _isPlainObject helper for JSON type checks

Consolidates the null/array/typeof check that was inlined at three
different call sites into a single helper. Keeps the intent obvious
at each use site and avoids the awkward multi-condition ternary.
2026-04-14 11:05:51 -07:00
Patrick Buckley 06d7cf8896 chore: bump version to 1.4.0a1 2026-04-13 17:19:22 -07:00
Patrick Buckley 934cb075d6 feat: per-model sampling parameters (temperature, max_tokens, reasoni… (#350)
* feat: per-model sampling parameters (temperature, max_tokens, reasoning_effort)

Model sampling parameters were global-only settings applied uniformly to
all models. Different models have fundamentally different requirements
(o-series needs no temperature, Anthropic needs temp=1.0 with thinking,
local models may need different max_tokens). This adds per-model overrides
with global fallback so each model definition can specify its own defaults.

Migration 036 adds nullable temperature, max_tokens, reasoning_effort
columns to model_definitions. NULL inherits the global default from
ConfigStore. The session factory and /model switch command both resolve
per-model override → global fallback consistently.

The admin UI model create/edit modal now has dedicated form fields for
these parameters with client-side validation, a visual section divider,
and per-model override hints in the model table rows.

Removes vestigial model.name and model.context_window global settings
(now handled per-model by the model registry) with startup warnings for
existing config.toml users.

* fix: defensive parsing for config.toml per-model sampling params

Wrap temperature/max_tokens conversions in try/except with range
validation. Invalid values log a warning and fall back to None
(inherit global default) instead of aborting registry load.
2026-04-13 17:14:58 -07:00
Patrick Buckley a793d009fd fix: use gethostname() instead of getfqdn() for advertise URLs (#349)
* fix: use gethostname() instead of getfqdn() for advertise URLs

socket.getfqdn() does a reverse DNS lookup that often returns a
truncated hostname (e.g. "flat" instead of "flat-blck-io"). Use
gethostname() for advertise URLs in both server and console. For TLS
SANs, include both names so certs cover all variations.

* docs: clarify advertise URL comment re Docker/k8s
2026-04-13 14:52:12 -07:00
Patrick Buckley 2a05ba5915 fix: standardize database env vars on TURNSTONE_DB_* naming (#348)
* fix: standardize database env vars on TURNSTONE_DB_* naming

compose.yaml used DB_BACKEND/DATABASE_URL in .env which got mapped to
TURNSTONE_DB_BACKEND/TURNSTONE_DB_URL inside containers. Running bare-
metal required the TURNSTONE_ prefix, but docs didn't explain this.
Eliminate the indirection — use TURNSTONE_DB_BACKEND and TURNSTONE_DB_URL
everywhere (compose, .env, bare-metal, docs, bootstrap wizard).

* fix: update .env.example to use TURNSTONE_DB_* naming
2026-04-13 14:48:51 -07:00
48 changed files with 1604 additions and 121 deletions
+2 -2
View File
@@ -20,10 +20,10 @@ TURNSTONE_JWT_SECRET=changeme-to-32-bytes-of-hex
# -- Database ------------------------------------------------------------------
# Single-node default is SQLite (zero config). Set these for PostgreSQL:
# DB_BACKEND=postgresql
# TURNSTONE_DB_BACKEND=postgresql
# POSTGRES_USER=turnstone
# POSTGRES_PASSWORD=changeme
# DATABASE_URL=postgresql+psycopg://turnstone:changeme@postgres:5432/turnstone
# TURNSTONE_DB_URL=postgresql+psycopg://turnstone:changeme@postgres:5432/turnstone
# -- Ports ---------------------------------------------------------------------
# SERVER_PORT=8080
+1 -1
View File
@@ -48,7 +48,7 @@ jobs:
id: detect
run: |
updates=()
for lib in katex hljs mermaid; do
for lib in katex hljs mermaid hls; do
version=$(grep -oE "${lib}-[0-9.]+" pyproject.toml | head -1 | sed "s/${lib}-//")
[[ -z "$version" ]] && continue
[[ -d "turnstone/shared_static/${lib}-${version}" ]] && continue
+9
View File
@@ -53,6 +53,15 @@ pip install turnstone[console]
turnstone-console --port 8090
```
For PostgreSQL (recommended for production):
```bash
pip install turnstone[postgres]
export TURNSTONE_DB_BACKEND=postgresql
export TURNSTONE_DB_URL="postgresql+psycopg://user:pass@localhost:5432/turnstone"
turnstone-server --port 8080 --base-url http://localhost:8000/v1
```
### Docker
```bash
+9 -9
View File
@@ -9,7 +9,7 @@
# Usage:
# Infra only: docker compose up
# Single node: docker compose --profile production up
# Production (PG): DB_BACKEND=postgresql docker compose --profile production up
# Production (PG): TURNSTONE_DB_BACKEND=postgresql docker compose --profile production up
# 10-node cluster: docker compose --profile cluster up
# =============================================================================
@@ -94,8 +94,8 @@ services:
- TURNSTONE_JWT_SECRET=${TURNSTONE_JWT_SECRET:?Set TURNSTONE_JWT_SECRET in .env}
- MODEL=${MODEL:-}
- MCP_CONFIG=${MCP_CONFIG:-}
- TURNSTONE_DB_BACKEND=${DB_BACKEND:-sqlite}
- TURNSTONE_DB_URL=${DATABASE_URL:-}
- TURNSTONE_DB_BACKEND=${TURNSTONE_DB_BACKEND:-sqlite}
- TURNSTONE_DB_URL=${TURNSTONE_DB_URL:-}
- TURNSTONE_NODE_ID=${TURNSTONE_NODE_ID:-}
- TURNSTONE_ADVERTISE_URL=${TURNSTONE_ADVERTISE_URL:-http://server:8080}
extra_hosts:
@@ -131,8 +131,8 @@ services:
environment:
# Generate with: python -c "import secrets; print(secrets.token_hex(32))"
- TURNSTONE_JWT_SECRET=${TURNSTONE_JWT_SECRET:?Set TURNSTONE_JWT_SECRET in .env}
- TURNSTONE_DB_BACKEND=${DB_BACKEND:-sqlite}
- TURNSTONE_DB_URL=${DATABASE_URL:-}
- TURNSTONE_DB_BACKEND=${TURNSTONE_DB_BACKEND:-sqlite}
- TURNSTONE_DB_URL=${TURNSTONE_DB_URL:-}
- TURNSTONE_CONSOLE_URL=http://console:8090
networks:
- turnstone-net
@@ -165,8 +165,8 @@ services:
- TURNSTONE_DISCORD_GUILD=${TURNSTONE_DISCORD_GUILD:-0}
# Generate with: python -c "import secrets; print(secrets.token_hex(32))"
- TURNSTONE_JWT_SECRET=${TURNSTONE_JWT_SECRET:?Set TURNSTONE_JWT_SECRET in .env}
- TURNSTONE_DB_BACKEND=${DB_BACKEND:-postgresql}
- TURNSTONE_DB_URL=${DATABASE_URL:-postgresql+psycopg://${POSTGRES_USER:-turnstone}:${POSTGRES_PASSWORD:-turnstone}@postgres:5432/turnstone}
- TURNSTONE_DB_BACKEND=${TURNSTONE_DB_BACKEND:-postgresql}
- TURNSTONE_DB_URL=${TURNSTONE_DB_URL:-postgresql+psycopg://${POSTGRES_USER:-turnstone}:${POSTGRES_PASSWORD:-turnstone}@postgres:5432/turnstone}
- TURNSTONE_CHANNEL_ADVERTISE_URL=http://channel:8091
networks:
- turnstone-net
@@ -215,8 +215,8 @@ services:
TURNSTONE_JWT_SECRET: ${TURNSTONE_JWT_SECRET:?Set TURNSTONE_JWT_SECRET in .env}
MODEL: ${MODEL:-}
MCP_CONFIG: ${MCP_CONFIG:-}
TURNSTONE_DB_BACKEND: ${DB_BACKEND:-postgresql}
TURNSTONE_DB_URL: ${DATABASE_URL:-postgresql+psycopg://${POSTGRES_USER:-turnstone}:${POSTGRES_PASSWORD:?}@postgres:5432/turnstone}
TURNSTONE_DB_BACKEND: ${TURNSTONE_DB_BACKEND:-postgresql}
TURNSTONE_DB_URL: ${TURNSTONE_DB_URL:-postgresql+psycopg://${POSTGRES_USER:-turnstone}:${POSTGRES_PASSWORD:?}@postgres:5432/turnstone}
TURNSTONE_NODE_ID: node-1
TURNSTONE_ADVERTISE_URL: http://server-1:8080
extra_hosts: ["host.docker.internal:host-gateway"]
+30 -3
View File
@@ -696,6 +696,26 @@ agent_model = "claude"
Each `[models.*]` entry produces a `ModelConfig` with a `provider` field
(default: `"openai"`). Supported values: `"openai"`, `"anthropic"`, `"google"`,
and `"openai-compatible"`.
**Per-model sampling overrides:** Each model can specify `temperature`,
`max_tokens`, and `reasoning_effort` to override the global defaults from
ConfigStore. When unset (`NULL`), the global default is used.
```toml
[models.local]
base_url = "http://localhost:8000/v1"
model = "qwen3-32b"
temperature = 0.7
max_tokens = 8192
[models.o3]
base_url = "https://api.openai.com/v1"
api_key = "sk-..."
model = "o3"
reasoning_effort = "high"
# temperature omitted — uses global default
```
An optional `[models.*.capabilities]` sub-table overrides per-model
`ModelCapabilities` flags (useful for local models whose capabilities
cannot be detected programmatically):
@@ -709,9 +729,15 @@ model = "qwen-3.5-vl"
supports_vision = true
```
**Database model definitions:** On server entry points, models can also be
defined in the `model_definitions` table (admin Models tab). DB models support
the same per-model sampling overrides. Config.toml models override DB models
with the same alias in-memory (the DB rows are never modified).
**Lifecycle:**
1. `load_model_registry()` reads `[models.*]` sections from config.toml and
builds a `"default"` entry from CLI `--base-url`/`--model`/`--api-key` args
1. `load_model_registry()` loads DB model definitions (if storage available),
then overlays `[models.*]` from config.toml, then builds a `"default"` entry
from CLI `--base-url`/`--model`/`--api-key` args
2. The registry is passed to the session factory closure in both `cli.py` and
`server.py`; each workstream resolves its model on creation
3. `ModelRegistry.get_client()` lazily creates SDK client instances via
@@ -720,7 +746,8 @@ supports_vision = true
4. `ModelRegistry.get_provider()` lazily creates `LLMProvider` instances via
`create_provider()` (also cached and thread-safe)
5. `/model` command shows available models; `/model <alias>` switches the
active workstream's client, model, and context window
active workstream's client, model, context window, and per-model sampling
parameters
6. `_create_stream_with_retry()` tries the primary model, then each fallback
alias in order if the primary is unreachable
7. `_run_agent()` resolves `registry.agent_model` (if set) for plan/task
+4 -1
View File
@@ -295,7 +295,7 @@ class "ModelRegistry" as ModelReg {
--
Thread-safe lazy client + provider
creation. Loaded by load_model_registry()
from CLI args + [models.*] config.
from DB + [models.*] config + CLI args.
--
core/model_registry.py
}
@@ -306,6 +306,9 @@ class "ModelConfig" as ModelCfg <<frozen>> {
+ base_url: str
+ model: str
+ context_window: int
+ temperature: float | None
+ max_tokens: int | None
+ reasoning_effort: str | None
}
' Circuit breaker state
+2 -2
View File
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6471e611beebf647f3a191eb16588571a404cc52a43067883a2b6f06dd936376
size 594676
oid sha256:474b900448ec04d1117b48a2b55614524721b2f04ac4bda66170bd0a06aae0f2
size 624573
+5 -1
View File
@@ -83,11 +83,15 @@ Auth is always enabled. `TURNSTONE_JWT_SECRET` is required.
| Variable | Default | Description |
|----------|---------|-------------|
| `TURNSTONE_DB_BACKEND` | `sqlite` | Storage backend: `sqlite` or `postgresql` |
| `TURNSTONE_DB_URL` | — | Database URL (e.g. `postgresql://user:pass@db:5432/turnstone`). For SQLite, defaults to `/data/.turnstone.db` |
| `TURNSTONE_DB_URL` | — | Database URL (e.g. `postgresql+psycopg://user:pass@postgres:5432/turnstone`). For SQLite, defaults to `/data/.turnstone.db` |
| `TURNSTONE_DB_POOL_SIZE` | `2` | PostgreSQL connection pool size per process (default: 2 base + 3 overflow = 5 max) |
| `POSTGRES_USER` | `turnstone` | PostgreSQL container username (used in default `TURNSTONE_DB_URL` for cluster/channel) |
| `POSTGRES_PASSWORD` | — | PostgreSQL container password (required for production and cluster profiles) |
The database stores workstream history, user accounts, and API tokens. When using JWT auth, a database backend is required for user storage.
> **Upgrading from <1.3.0a4:** Earlier versions used `DB_BACKEND` and `DATABASE_URL` in `.env`, which `compose.yaml` mapped to the `TURNSTONE_`-prefixed names internally. These short aliases have been removed. Rename `DB_BACKEND` → `TURNSTONE_DB_BACKEND` and `DATABASE_URL` → `TURNSTONE_DB_URL` in your `.env` file.
> **Large clusters:** Each turnstone process maintains a small connection pool (5 max). At hundreds of nodes this adds up — use [PgBouncer](pgbouncer.md) in transaction pooling mode between turnstone and PostgreSQL.
> **First-time setup:** After deploying with auth enabled, create an initial admin user by running `turnstone-admin create-user` inside the container:
+1 -1
View File
@@ -67,7 +67,7 @@ services:
```
Then point turnstone services at PgBouncer instead of PostgreSQL
directly by changing the `DATABASE_URL` (or `TURNSTONE_DB_URL`):
directly by changing `TURNSTONE_DB_URL`:
```bash
# Before (direct)
+28 -3
View File
@@ -36,6 +36,31 @@ users to the admin Settings API.
---
## Per-Model Sampling Overrides
The global `model.temperature`, `model.max_tokens`, and `model.reasoning_effort`
settings serve as cluster-wide defaults. Individual models can override these
via per-model settings in the `model_definitions` table (admin Models tab).
Resolution order for sampling parameters:
| Priority | Source |
|----------|--------|
| 1 (highest) | Per-model override (set in Models tab) |
| 2 | Global default (set in Settings tab) |
| 3 | Registry default (code) |
When a per-model override is `NULL` (empty in the UI), the global default is
used. Switching models via `/model <alias>` re-resolves sampling parameters
from the new model's overrides or global defaults.
**Removed settings:** `model.name` and `model.context_window` have been removed
from ConfigStore. Model names and context windows are now configured per-model
in the Models tab. A startup warning is logged if these keys appear in
`config.toml`.
---
## Bootstrap vs ConfigStore
**Bootstrap settings** are required before storage is available (database
@@ -49,12 +74,12 @@ connection, Redis, auth secrets, server bind address). These stay in
| Auth | `[auth]` | config.toml / env |
| Console bind | `[console]` | config.toml / env |
**ConfigStore settings** (51 settings) are loaded from the database after
storage initialization:
**ConfigStore settings** are loaded from the database after storage
initialization:
| Section | Settings |
|---------|----------|
| `model` | name, temperature, max_tokens, reasoning_effort, context_window |
| `model` | default_alias, temperature, max_tokens, reasoning_effort |
| `session` | instructions, retention_days, compact_max_tokens, auto_compact_pct |
| `tools` | timeout, truncation, agent_max_turns, skip_permissions, search, search_threshold, search_max_results |
| `server` | workstream_idle_timeout, max_workstreams |
+2 -2
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "turnstone"
version = "1.3.0a3"
version = "1.4.0a2"
description = "Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."
readme = "README.md"
license = "BUSL-1.1"
@@ -80,7 +80,7 @@ include = [
"turnstone/shared_static/katex-0.16.45/**/*",
"turnstone/shared_static/hljs-11.11.1/**/*",
"turnstone/shared_static/mermaid-11.14.0/**/*",
"turnstone/shared_static/hls-1.6.15/**/*",
"turnstone/shared_static/hls-1.6.16/**/*",
"turnstone/sdk/py.typed",
"turnstone/deploy/*.yaml",
]
+5 -5
View File
@@ -87,8 +87,8 @@ class TestSetGetRoundTrip:
assert store.get("tools.skip_permissions") is False
def test_str(self, store):
store.set("model.name", "gpt-5")
assert store.get("model.name") == "gpt-5"
store.set("model.default_alias", "gpt5-prod")
assert store.get("model.default_alias") == "gpt5-prod"
# ---------------------------------------------------------------------------
@@ -165,10 +165,10 @@ class TestStoredKeys:
assert store.stored_keys() == frozenset()
store.set("tools.timeout", 30)
assert store.stored_keys() == frozenset({"tools.timeout"})
store.set("model.name", "gpt-5")
assert store.stored_keys() == frozenset({"tools.timeout", "model.name"})
store.set("model.default_alias", "gpt5-prod")
assert store.stored_keys() == frozenset({"tools.timeout", "model.default_alias"})
store.delete("tools.timeout")
assert store.stored_keys() == frozenset({"model.name"})
assert store.stored_keys() == frozenset({"model.default_alias"})
# ---------------------------------------------------------------------------
+51
View File
@@ -161,3 +161,54 @@ class TestModelDefinitionStorage:
assert m["capabilities"] == "{}"
assert m["enabled"] is True
assert m["created_by"] == ""
# Per-model sampling params default to None (use global default)
assert m["temperature"] is None
assert m["max_tokens"] is None
assert m["reasoning_effort"] is None
def test_create_with_sampling_params(self, db: SQLiteBackend) -> None:
did = _make_id()
db.create_model_definition(
definition_id=did,
alias="sampling",
model="gpt-5",
temperature=0.7,
max_tokens=8192,
reasoning_effort="high",
)
m = db.get_model_definition(did)
assert m is not None
assert m["temperature"] == 0.7
assert m["max_tokens"] == 8192
assert m["reasoning_effort"] == "high"
def test_create_with_zero_temperature(self, db: SQLiteBackend) -> None:
"""temperature=0.0 is a valid override, distinct from None."""
did = _make_id()
db.create_model_definition(
definition_id=did, alias="zero-temp", model="o3", temperature=0.0
)
m = db.get_model_definition(did)
assert m is not None
assert m["temperature"] == 0.0
def test_update_sampling_params(self, db: SQLiteBackend) -> None:
did = _make_id()
db.create_model_definition(definition_id=did, alias="upd-samp", model="gpt-5")
db.update_model_definition(did, temperature=1.2, max_tokens=4096, reasoning_effort="low")
m = db.get_model_definition(did)
assert m is not None
assert m["temperature"] == 1.2
assert m["max_tokens"] == 4096
assert m["reasoning_effort"] == "low"
def test_clear_sampling_params(self, db: SQLiteBackend) -> None:
"""Setting sampling params to None clears them back to global default."""
did = _make_id()
db.create_model_definition(
definition_id=did, alias="clear-samp", model="gpt-5", temperature=0.9
)
db.update_model_definition(did, temperature=None)
m = db.get_model_definition(did)
assert m is not None
assert m["temperature"] is None
+116
View File
@@ -51,6 +51,31 @@ class TestModelConfig:
cfg = ModelConfig(alias="test", base_url="http://x", api_key="sk-secret-key", model="m")
assert "sk-secret-key" not in repr(cfg)
def test_sampling_params_default_none(self) -> None:
cfg = ModelConfig(alias="x", base_url="x", api_key="x", model="x")
assert cfg.temperature is None
assert cfg.max_tokens is None
assert cfg.reasoning_effort is None
def test_sampling_params_set(self) -> None:
cfg = ModelConfig(
alias="x",
base_url="x",
api_key="x",
model="x",
temperature=0.7,
max_tokens=8192,
reasoning_effort="high",
)
assert cfg.temperature == 0.7
assert cfg.max_tokens == 8192
assert cfg.reasoning_effort == "high"
def test_zero_temperature_distinct_from_none(self) -> None:
cfg = ModelConfig(alias="x", base_url="x", api_key="x", model="x", temperature=0.0)
assert cfg.temperature == 0.0
assert cfg.temperature is not None
# ---------------------------------------------------------------------------
# ModelRegistry
@@ -483,6 +508,58 @@ class TestLoadModelRegistryWithDB:
reg = load_model_registry("http://x/v1", "x", "x", storage=storage)
assert reg.get_config("caps-model").capabilities == {"supports_vision": True}
def test_db_sampling_params_loaded(self) -> None:
"""Per-model sampling params from DB are carried in ModelConfig."""
storage = _MockStorage(
[
{
"alias": "hot-model",
"model": "m",
"provider": "openai",
"base_url": "",
"api_key": "",
"context_window": 32768,
"capabilities": "{}",
"enabled": True,
"temperature": 1.5,
"max_tokens": 4096,
"reasoning_effort": "high",
}
]
)
with patch("turnstone.core.model_registry.load_config", return_value={}):
reg = load_model_registry("http://x/v1", "x", "x", storage=storage)
cfg = reg.get_config("hot-model")
assert cfg.temperature == 1.5
assert cfg.max_tokens == 4096
assert cfg.reasoning_effort == "high"
def test_db_sampling_params_null_means_none(self) -> None:
"""NULL sampling params in DB map to None (use global default)."""
storage = _MockStorage(
[
{
"alias": "null-model",
"model": "m",
"provider": "openai",
"base_url": "",
"api_key": "",
"context_window": 32768,
"capabilities": "{}",
"enabled": True,
"temperature": None,
"max_tokens": None,
"reasoning_effort": None,
}
]
)
with patch("turnstone.core.model_registry.load_config", return_value={}):
reg = load_model_registry("http://x/v1", "x", "x", storage=storage)
cfg = reg.get_config("null-model")
assert cfg.temperature is None
assert cfg.max_tokens is None
assert cfg.reasoning_effort is None
def test_db_default_alias_not_clobbered(self) -> None:
"""DB model with alias='default' is not overwritten by CLI args."""
storage = _MockStorage(
@@ -688,6 +765,45 @@ class TestSessionModelCommand:
assert session.context_window == 64000
assert "Switched to" in session.ui.infos[-1]
def test_model_switch_applies_sampling_params(self) -> None:
reg = ModelRegistry(
models={
"default": ModelConfig("default", "x", "x", "default-model"),
"hot": ModelConfig(
"hot",
"y",
"y",
"hot-model",
temperature=1.5,
max_tokens=2048,
reasoning_effort="high",
),
},
default="default",
)
session = _make_session(registry=reg, model_alias="default")
assert session.temperature == 0.5 # initial global default
session.handle_command("/model hot")
assert session.temperature == 1.5
assert session.max_tokens == 2048
assert session.reasoning_effort == "high"
def test_model_switch_none_params_reverts_to_global(self) -> None:
"""Switching to a model with no overrides reverts to global defaults."""
reg = ModelRegistry(
models={
"hot": ModelConfig("hot", "x", "x", "hot-model", temperature=1.5),
"plain": ModelConfig("plain", "y", "y", "plain-model"),
},
default="hot",
)
session = _make_session(registry=reg, model_alias="hot")
session.temperature = 1.5 # as set by per-model override
# Without a config_store, fallback keeps current value (CLI sessions).
# With a config_store, it would revert to the global default.
session.handle_command("/model plain")
assert session.temperature == 1.5 # no config_store → keeps current
def test_model_switch_unknown_alias(self) -> None:
reg = ModelRegistry(
models={"default": ModelConfig("default", "x", "x", "test-model")},
+46
View File
@@ -152,6 +152,52 @@ class TestOpenAIProvider:
def test_provider_name(self) -> None:
assert self.provider.provider_name == "openai-compatible"
# -- _apply_thinking_mode -------------------------------------------------
def test_thinking_mode_none_does_nothing(self) -> None:
"""No thinking params injected when thinking_mode is 'none'."""
caps = ModelCapabilities(thinking_mode="none")
extra_body: dict[str, Any] = {"chat_template_kwargs": {"reasoning_effort": "medium"}}
OpenAIProvider._apply_thinking_mode(extra_body, caps)
assert "enable_thinking" not in extra_body["chat_template_kwargs"]
def test_thinking_mode_manual_injects_param(self) -> None:
"""Manual thinking mode injects enable_thinking into chat_template_kwargs."""
caps = ModelCapabilities(thinking_mode="manual")
extra_body: dict[str, Any] = {"chat_template_kwargs": {"reasoning_effort": "medium"}}
OpenAIProvider._apply_thinking_mode(extra_body, caps)
assert extra_body["chat_template_kwargs"]["enable_thinking"] is True
assert extra_body["chat_template_kwargs"]["reasoning_effort"] == "medium"
def test_thinking_mode_custom_param(self) -> None:
"""Custom thinking_param (e.g. Granite's 'thinking') is used."""
caps = ModelCapabilities(thinking_mode="manual", thinking_param="thinking")
extra_body: dict[str, Any] = {"chat_template_kwargs": {}}
OpenAIProvider._apply_thinking_mode(extra_body, caps)
assert extra_body["chat_template_kwargs"]["thinking"] is True
assert "enable_thinking" not in extra_body["chat_template_kwargs"]
def test_thinking_mode_does_not_override_explicit(self) -> None:
"""If operator explicitly set the param to False, provider respects it."""
caps = ModelCapabilities(thinking_mode="manual")
extra_body: dict[str, Any] = {"chat_template_kwargs": {"enable_thinking": False}}
OpenAIProvider._apply_thinking_mode(extra_body, caps)
assert extra_body["chat_template_kwargs"]["enable_thinking"] is False
def test_thinking_mode_creates_ctk_if_missing(self) -> None:
"""Creates chat_template_kwargs dict if not present in extra_body."""
caps = ModelCapabilities(thinking_mode="manual")
extra_body: dict[str, Any] = {}
OpenAIProvider._apply_thinking_mode(extra_body, caps)
assert extra_body["chat_template_kwargs"]["enable_thinking"] is True
def test_thinking_mode_adaptive(self) -> None:
"""Adaptive thinking mode also injects the param."""
caps = ModelCapabilities(thinking_mode="adaptive")
extra_body: dict[str, Any] = {"chat_template_kwargs": {}}
OpenAIProvider._apply_thinking_mode(extra_body, caps)
assert extra_body["chat_template_kwargs"]["enable_thinking"] is True
# -- _sanitize_messages ---------------------------------------------------
def test_sanitize_messages_none_content_no_tool_calls(self) -> None:
+273
View File
@@ -0,0 +1,273 @@
"""Tests for turnstone.core.server_compat — profile suggestion and merging."""
from __future__ import annotations
from typing import Any
from unittest.mock import MagicMock
from turnstone.core.providers._openai_chat import OpenAIChatCompletionsProvider
from turnstone.core.providers._protocol import ModelCapabilities
from turnstone.core.server_compat import merge_server_compat, suggest_profile
# ---------------------------------------------------------------------------
# suggest_profile
# ---------------------------------------------------------------------------
class TestSuggestProfile:
def test_vllm_gemma4(self) -> None:
p = suggest_profile("vllm", "google/gemma-4-31B-it")
assert p["capabilities"]["thinking_mode"] == "manual"
assert p["capabilities"]["thinking_param"] == "enable_thinking"
assert p["server_compat"]["extra_body"]["skip_special_tokens"] is False
def test_vllm_gemma3(self) -> None:
p = suggest_profile("vllm", "google/gemma-3-27b-it")
assert p["capabilities"]["thinking_mode"] == "manual"
def test_vllm_qwen3(self) -> None:
p = suggest_profile("vllm", "Qwen/Qwen3-8B")
assert p["capabilities"]["thinking_mode"] == "manual"
assert p["capabilities"]["thinking_param"] == "enable_thinking"
# Qwen doesn't need skip_special_tokens workaround
assert "extra_body" not in p.get("server_compat", {})
def test_vllm_qwq(self) -> None:
p = suggest_profile("vllm", "Qwen/QwQ-32B")
assert p["capabilities"]["thinking_mode"] == "manual"
def test_vllm_granite(self) -> None:
p = suggest_profile("vllm", "ibm-granite/granite-3.2-2b-instruct")
assert p["capabilities"]["thinking_param"] == "thinking"
def test_vllm_deepseek_r1(self) -> None:
p = suggest_profile("vllm", "deepseek-ai/DeepSeek-R1-Distill-Qwen-7B")
assert p["capabilities"]["thinking_param"] == "thinking"
def test_vllm_deepseek_v3_no_thinking(self) -> None:
"""DeepSeek-V3 is a chat model, not a reasoning model — no thinking profile."""
p = suggest_profile("vllm", "deepseek-ai/DeepSeek-V3-0324")
assert "capabilities" not in p
assert p["server_compat"]["server_type"] == "vllm"
def test_vllm_non_thinking_model(self) -> None:
p = suggest_profile("vllm", "meta-llama/Llama-3-70B-Instruct")
assert "capabilities" not in p
assert p["server_compat"]["server_type"] == "vllm"
def test_llama_cpp_non_thinking(self) -> None:
p = suggest_profile("llama.cpp", "some-model")
assert p["server_compat"]["server_type"] == "llama.cpp"
assert "capabilities" not in p
def test_llama_cpp_gemma_thinking(self) -> None:
"""llama.cpp with Gemma model gets thinking profile with reasoning_format."""
p = suggest_profile("llama.cpp", "gemma-4-E4B-it.gguf")
assert p["capabilities"]["thinking_mode"] == "manual"
assert p["server_compat"]["extra_body"]["reasoning_format"] == "auto"
def test_llama_cpp_qwen_thinking(self) -> None:
p = suggest_profile("llama.cpp", "Qwen3-8B-Q4_K_M.gguf")
assert p["capabilities"]["thinking_mode"] == "manual"
def test_sglang(self) -> None:
p = suggest_profile("sglang", "some-model")
assert p["server_compat"]["server_type"] == "sglang"
def test_unknown_server(self) -> None:
assert suggest_profile("unknown", "foo") == {}
def test_empty_inputs(self) -> None:
assert suggest_profile("", "") == {}
def test_openai_compatible_fallback(self) -> None:
"""Generic openai-compatible without a specific profile."""
assert suggest_profile("openai-compatible", "some-local-model") == {}
def test_case_insensitive_model_match(self) -> None:
"""Model matching should be case-insensitive."""
p = suggest_profile("vllm", "Google/GEMMA-4-31B-IT")
assert p["capabilities"]["thinking_mode"] == "manual"
def test_holo_requires_holo2(self) -> None:
"""Short 'holo' prefix shouldn't false-match; 'holo2' should match."""
p_short = suggest_profile("vllm", "some-org/hologram-7b")
assert "capabilities" not in p_short
p_long = suggest_profile("vllm", "some-org/Holo2-14B")
assert p_long["capabilities"]["thinking_mode"] == "manual"
def test_suggest_returns_deep_copy(self) -> None:
"""Mutating the returned profile should not affect future calls."""
p1 = suggest_profile("vllm", "google/gemma-4-31B-it")
p1["capabilities"]["thinking_mode"] = "none"
p2 = suggest_profile("vllm", "google/gemma-4-31B-it")
assert p2["capabilities"]["thinking_mode"] == "manual"
# ---------------------------------------------------------------------------
# merge_server_compat
# ---------------------------------------------------------------------------
class TestMergeServerCompat:
def test_empty_compat_returns_base_only(self) -> None:
base = {"reasoning_effort": "medium"}
result = merge_server_compat(base, {})
assert result == {"chat_template_kwargs": {"reasoning_effort": "medium"}}
def test_extra_body_merged_top_level(self) -> None:
base = {"reasoning_effort": "medium"}
compat = {"extra_body": {"skip_special_tokens": False}}
result = merge_server_compat(base, compat)
assert result["skip_special_tokens"] is False
assert "chat_template_kwargs" in result
def test_full_vllm_gemma_compat(self) -> None:
base = {"reasoning_effort": "medium"}
compat = {
"server_type": "vllm",
"extra_body": {"skip_special_tokens": False},
}
result = merge_server_compat(base, compat)
assert result == {
"chat_template_kwargs": {"reasoning_effort": "medium"},
"skip_special_tokens": False,
}
def test_extra_body_chat_template_kwargs_deep_merged(self) -> None:
"""chat_template_kwargs in extra_body is deep-merged, operator wins."""
base = {"reasoning_effort": "medium"}
compat = {
"extra_body": {
"chat_template_kwargs": {"custom_flag": True, "reasoning_effort": "high"},
"skip_special_tokens": False,
},
}
result = merge_server_compat(base, compat)
# Operator values win over base
assert result["chat_template_kwargs"]["custom_flag"] is True
assert result["chat_template_kwargs"]["reasoning_effort"] == "high"
assert result["skip_special_tokens"] is False
def test_extra_body_chat_template_kwargs_non_dict_ignored(self) -> None:
"""Non-dict chat_template_kwargs in extra_body is safely ignored."""
base = {"reasoning_effort": "medium"}
compat = {"extra_body": {"chat_template_kwargs": "bad"}}
result = merge_server_compat(base, compat)
assert result["chat_template_kwargs"] == {"reasoning_effort": "medium"}
def test_base_not_mutated(self) -> None:
base = {"reasoning_effort": "medium"}
compat = {"extra_body": {"skip_special_tokens": False}}
merge_server_compat(base, compat)
assert "skip_special_tokens" not in base
def test_non_dict_extra_body_ignored(self) -> None:
"""Gracefully handle malformed server_compat."""
base = {"reasoning_effort": "medium"}
result = merge_server_compat(base, {"extra_body": 42})
assert result == {"chat_template_kwargs": {"reasoning_effort": "medium"}}
# ---------------------------------------------------------------------------
# End-to-end: session merge + provider thinking mode
# ---------------------------------------------------------------------------
class TestEndToEndRequestShaping:
"""Compose both layers — session builds extra_params, provider applies thinking."""
def test_vllm_gemma_full_flow(self) -> None:
"""Session merges server workarounds, provider adds thinking param."""
caps = ModelCapabilities(thinking_mode="manual", thinking_param="enable_thinking")
base_ctk = {"reasoning_effort": "medium"}
server_compat = {
"server_type": "vllm",
"extra_body": {"skip_special_tokens": False},
}
# Step 1: session merges
extra_params = merge_server_compat(base_ctk, server_compat)
# Step 2: provider finalises
extra_body = dict(extra_params)
OpenAIChatCompletionsProvider._apply_thinking_mode(extra_body, caps)
assert extra_body == {
"chat_template_kwargs": {
"reasoning_effort": "medium",
"enable_thinking": True,
},
"skip_special_tokens": False,
}
def test_granite_thinking_key(self) -> None:
"""Granite uses 'thinking' instead of 'enable_thinking'."""
caps = ModelCapabilities(thinking_mode="manual", thinking_param="thinking")
extra_params = merge_server_compat({"reasoning_effort": "low"}, {})
extra_body = dict(extra_params)
OpenAIChatCompletionsProvider._apply_thinking_mode(extra_body, caps)
assert extra_body["chat_template_kwargs"]["thinking"] is True
assert "enable_thinking" not in extra_body["chat_template_kwargs"]
def test_non_thinking_model_no_injection(self) -> None:
"""Non-thinking model gets no thinking params."""
caps = ModelCapabilities() # thinking_mode="none"
extra_params = merge_server_compat({"reasoning_effort": "medium"}, {})
extra_body = dict(extra_params)
OpenAIChatCompletionsProvider._apply_thinking_mode(extra_body, caps)
assert extra_body == {"chat_template_kwargs": {"reasoning_effort": "medium"}}
# ---------------------------------------------------------------------------
# Probe integration: suggest_profile called from _detect_openai_compat
# ---------------------------------------------------------------------------
class TestProbeIntegration:
def test_detect_vllm_gemma_suggests_profile(self) -> None:
"""_detect_openai_compat returns suggested_capabilities and suggested_server_compat."""
from turnstone.core.model_registry import _detect_openai_compat
result: dict[str, Any] = {
"reachable": True,
"model_found": True,
"available_models": ["google/gemma-4-31B-it"],
"context_window": None,
"server_type": None,
"error": None,
}
model_obj = MagicMock()
model_obj.model_dump.return_value = {"owned_by": "vllm"}
_detect_openai_compat(
result, model_obj, "google/gemma-4-31B-it", "http://localhost:8000/v1"
)
assert result["server_type"] == "vllm"
assert result["suggested_capabilities"]["thinking_mode"] == "manual"
assert result["suggested_capabilities"]["thinking_param"] == "enable_thinking"
assert result["suggested_server_compat"]["extra_body"]["skip_special_tokens"] is False
def test_detect_non_thinking_no_suggested_capabilities(self) -> None:
"""Non-thinking vLLM model gets server_compat but no capabilities suggestion."""
from turnstone.core.model_registry import _detect_openai_compat
result: dict[str, Any] = {
"reachable": True,
"model_found": True,
"available_models": ["meta-llama/Llama-3-70B"],
"context_window": None,
"server_type": None,
"error": None,
}
model_obj = MagicMock()
model_obj.model_dump.return_value = {"owned_by": "vllm"}
_detect_openai_compat(
result, model_obj, "meta-llama/Llama-3-70B", "http://localhost:8000/v1"
)
assert result["server_type"] == "vllm"
assert "suggested_capabilities" not in result
assert result["suggested_server_compat"]["server_type"] == "vllm"
+82
View File
@@ -1080,3 +1080,85 @@ class TestProviderExtraParams:
openai_prov = create_provider("openai")
result = session._provider_extra_params(provider=openai_prov)
assert result is None
def test_server_compat_extra_body_merged(self, tmp_db):
"""server_compat.extra_body workarounds are merged into extra_params."""
from turnstone.core.model_registry import ModelConfig, ModelRegistry
session = self._session_with_provider("openai-compatible", tmp_db)
cfg = ModelConfig(
alias="test",
base_url="http://localhost:8000/v1",
api_key="none",
model="google/gemma-4-31B-it",
server_compat={
"extra_body": {"skip_special_tokens": False},
},
)
session._registry = ModelRegistry(models={"test": cfg}, default="test")
session._model_alias = "test"
result = session._provider_extra_params()
assert result is not None
assert result["chat_template_kwargs"]["reasoning_effort"] == "medium"
assert result["skip_special_tokens"] is False
def test_empty_server_compat_backwards_compatible(self, tmp_db):
"""Empty server_compat produces same output as before."""
session = self._session_with_provider("openai-compatible", tmp_db)
result = session._provider_extra_params()
assert result == {"chat_template_kwargs": {"reasoning_effort": "medium"}}
def test_server_compat_with_reasoning_effort_override(self, tmp_db):
"""reasoning_effort override works alongside server_compat."""
from turnstone.core.model_registry import ModelConfig, ModelRegistry
session = self._session_with_provider("openai-compatible", tmp_db)
cfg = ModelConfig(
alias="test",
base_url="http://localhost:8000/v1",
api_key="none",
model="google/gemma-4-31B-it",
server_compat={"extra_body": {"skip_special_tokens": False}},
)
session._registry = ModelRegistry(models={"test": cfg}, default="test")
session._model_alias = "test"
result = session._provider_extra_params(reasoning_effort="high")
assert result is not None
assert result["chat_template_kwargs"]["reasoning_effort"] == "high"
assert result["skip_special_tokens"] is False
def test_model_alias_resolves_target_compat(self, tmp_db):
"""model_alias parameter selects compat from the target, not the primary."""
from turnstone.core.model_registry import ModelConfig, ModelRegistry
session = self._session_with_provider("openai-compatible", tmp_db)
primary = ModelConfig(
alias="primary",
base_url="http://localhost:8000/v1",
api_key="none",
model="google/gemma-4-31B-it",
server_compat={"extra_body": {"skip_special_tokens": False}},
)
fallback = ModelConfig(
alias="fallback",
base_url="http://localhost:9000/v1",
api_key="none",
model="meta-llama/Llama-3-70B",
)
reg = ModelRegistry(
models={"primary": primary, "fallback": fallback},
default="primary",
fallback=["fallback"],
)
session._registry = reg
session._model_alias = "primary"
# Primary alias → gets Gemma workaround
result_primary = session._provider_extra_params()
assert result_primary is not None
assert result_primary["skip_special_tokens"] is False
# Fallback alias → no compat, just base kwargs
result_fallback = session._provider_extra_params(model_alias="fallback")
assert result_fallback == {"chat_template_kwargs": {"reasoning_effort": "medium"}}
assert "skip_special_tokens" not in result_fallback
+3 -3
View File
@@ -69,7 +69,7 @@ class TestValidateValueCoercion:
validate_value("tools.timeout", None)
def test_str(self):
assert validate_value("model.name", "gpt-5") == "gpt-5"
assert validate_value("model.default_alias", "gpt5-prod") == "gpt5-prod"
assert validate_value("session.instructions", "be nice") == "be nice"
@@ -143,10 +143,10 @@ class TestSerializeDeserialize:
def test_str_round_trip(self):
v = "hello world"
assert deserialize_value("model.name", serialize_value(v)) == v
assert deserialize_value("model.default_alias", serialize_value(v)) == v
def test_str_round_trip_empty(self):
assert deserialize_value("model.name", serialize_value("")) == ""
assert deserialize_value("model.default_alias", serialize_value("")) == ""
# ---------------------------------------------------------------------------
+1 -1
View File
@@ -1,3 +1,3 @@
"""turnstone - Multi-node AI orchestration platform with tool use, agent routing, and cluster simulation."""
__version__ = "1.3.0a3"
__version__ = "1.4.0a2"
+9
View File
@@ -812,6 +812,9 @@ class ModelDefinitionInfo(BaseModel):
context_window: int = 32768
capabilities: str = "{}"
enabled: bool = True
temperature: float | None = None
max_tokens: int | None = None
reasoning_effort: str | None = None
source: str = ""
created_by: str = ""
created: str = ""
@@ -827,6 +830,9 @@ class CreateModelDefinitionRequest(BaseModel):
context_window: int = 32768
capabilities: dict[str, Any] = Field(default_factory=dict)
enabled: bool = True
temperature: float | None = None
max_tokens: int | None = None
reasoning_effort: str | None = None
class UpdateModelDefinitionRequest(BaseModel):
@@ -838,6 +844,9 @@ class UpdateModelDefinitionRequest(BaseModel):
context_window: int | None = None
capabilities: dict[str, Any] | None = None
enabled: bool | None = None
temperature: float | None = None
max_tokens: int | None = None
reasoning_effort: str | None = None
class ListModelDefinitionsResponse(BaseModel):
+4 -3
View File
@@ -76,8 +76,9 @@ For commercial providers (OpenAI, Anthropic-via-proxy), use the real key.
- `TAVILY_API_KEY` Web search API key (optional)
### Database
- `DB_BACKEND` `sqlite` (default) or `postgresql`
- `DATABASE_URL` PostgreSQL connection string (production only)
- `TURNSTONE_DB_BACKEND` `sqlite` (default) or `postgresql`
- `TURNSTONE_DB_URL` PostgreSQL connection URL (production only), \
e.g. `postgresql+psycopg://turnstone:<password>@postgres:5432/turnstone`
- `POSTGRES_USER` PostgreSQL username (default: turnstone)
- `POSTGRES_PASSWORD` PostgreSQL password (required for production)
@@ -184,7 +185,7 @@ exact commands to run next (e.g., `docker compose --profile production up -d` th
- If `compose.yaml` is missing, call `write_compose` before anything else. \
The compose file uses pre-built images from ghcr.io no local Docker build is needed.
- If an existing .env is detected, summarize what's configured and ask what to change.
- The `DATABASE_URL` for docker compose internal networking uses the hostname `postgres` \
- The `TURNSTONE_DB_URL` for docker compose internal networking uses the hostname `postgres` \
(e.g., `postgresql+psycopg://turnstone:<password>@postgres:5432/turnstone`).
- For local LLM backends (vLLM, llama.cpp, Ollama, etc.), set `OPENAI_API_KEY=dummy` in the \
.env file local servers typically don't require authentication. The `LLM_BASE_URL` should \
+94 -2
View File
@@ -1318,8 +1318,11 @@ async def _lifespan(app: Starlette) -> AsyncGenerator[None, None]:
try:
if not tls_mgr.ca_initialized:
await tls_mgr.init_ca()
hostname = socket.getfqdn()
hostname = socket.gethostname()
fqdn = socket.getfqdn()
cert_hostnames = [hostname, "localhost", "127.0.0.1"]
if fqdn != hostname:
cert_hostnames.append(fqdn)
extra_sans = os.environ.get("TURNSTONE_TLS_SANS", "")
if extra_sans:
cert_hostnames.extend(s.strip() for s in extra_sans.split(",") if s.strip())
@@ -5232,6 +5235,9 @@ async def admin_import_mcp_config(request: Request) -> JSONResponse:
_MODEL_ALIAS_RE = re.compile(r"^[a-zA-Z0-9._-]+$")
_MODEL_PROVIDERS = frozenset({"openai", "anthropic", "openai-compatible", "google"})
_REASONING_EFFORT_CHOICES = frozenset(
{"", "none", "minimal", "low", "medium", "high", "xhigh", "max"}
)
# Keep in sync with turnstone.core.providers._google.GOOGLE_DEFAULT_BASE_URL
_PROVIDER_DEFAULT_URLS: dict[str, str] = {
"openai": "https://api.openai.com/v1",
@@ -5344,12 +5350,18 @@ async def admin_list_model_definitions(request: Request) -> JSONResponse:
model_name = ""
provider = "openai"
context_window = 0
cfg_temperature = None
cfg_max_tokens = None
cfg_reasoning_effort = None
for node_models in node_statuses.values():
nm = node_models.get(alias)
if nm:
model_name = nm.get("model", "")
provider = nm.get("provider", "openai")
context_window = nm.get("context_window", 0)
cfg_temperature = nm.get("temperature")
cfg_max_tokens = nm.get("max_tokens")
cfg_reasoning_effort = nm.get("reasoning_effort")
break
result.append(
{
@@ -5362,6 +5374,9 @@ async def admin_list_model_definitions(request: Request) -> JSONResponse:
"context_window": context_window,
"capabilities": "{}",
"enabled": True,
"temperature": cfg_temperature,
"max_tokens": cfg_max_tokens,
"reasoning_effort": cfg_reasoning_effort,
"source": "config",
"created_by": "",
"created": "",
@@ -5451,6 +5466,36 @@ async def admin_create_model_definition(request: Request) -> JSONResponse:
capabilities = json.dumps(caps) if isinstance(caps, dict) else "{}"
enabled = bool(body.get("enabled", True))
# Per-model sampling overrides (None = use global default)
temperature: float | None = None
if body.get("temperature") is not None:
try:
temperature = float(body["temperature"])
except (ValueError, TypeError):
return JSONResponse({"error": "temperature must be a number"}, status_code=400)
if not 0.0 <= temperature <= 2.0:
return JSONResponse(
{"error": "temperature must be between 0.0 and 2.0"}, status_code=400
)
max_tokens: int | None = None
if body.get("max_tokens") is not None:
try:
max_tokens = int(body["max_tokens"])
except (ValueError, TypeError):
return JSONResponse({"error": "max_tokens must be an integer"}, status_code=400)
if max_tokens < 1:
return JSONResponse({"error": "max_tokens must be >= 1"}, status_code=400)
reasoning_effort: str | None = None
if body.get("reasoning_effort") is not None:
reasoning_effort = str(body["reasoning_effort"]).strip()
if reasoning_effort and reasoning_effort not in _REASONING_EFFORT_CHOICES:
return JSONResponse(
{"error": f"Invalid reasoning_effort: {reasoning_effort!r}"},
status_code=400,
)
if not reasoning_effort:
reasoning_effort = None
storage.create_model_definition(
definition_id=definition_id,
alias=alias,
@@ -5462,6 +5507,9 @@ async def admin_create_model_definition(request: Request) -> JSONResponse:
capabilities=capabilities,
enabled=enabled,
created_by=audit_uid,
temperature=temperature,
max_tokens=max_tokens,
reasoning_effort=reasoning_effort,
)
record_audit(
@@ -5570,6 +5618,50 @@ async def admin_update_model_definition(request: Request) -> JSONResponse:
if "enabled" in body:
updates["enabled"] = bool(body["enabled"])
# Per-model sampling overrides — explicit null clears to "use global default"
if "temperature" in body:
raw_temp = body["temperature"]
if raw_temp is None:
updates["temperature"] = None
else:
try:
temp_val = float(raw_temp)
except (ValueError, TypeError):
return JSONResponse({"error": "temperature must be a number"}, status_code=400)
if not 0.0 <= temp_val <= 2.0:
return JSONResponse(
{"error": "temperature must be between 0.0 and 2.0"},
status_code=400,
)
updates["temperature"] = temp_val
if "max_tokens" in body:
raw_mt = body["max_tokens"]
if raw_mt is None:
updates["max_tokens"] = None
else:
try:
mt_val = int(raw_mt)
except (ValueError, TypeError):
return JSONResponse({"error": "max_tokens must be an integer"}, status_code=400)
if mt_val < 1:
return JSONResponse({"error": "max_tokens must be >= 1"}, status_code=400)
updates["max_tokens"] = mt_val
if "reasoning_effort" in body:
raw_re = body["reasoning_effort"]
if raw_re is None:
updates["reasoning_effort"] = None
else:
re_val = str(raw_re).strip()
if not re_val:
updates["reasoning_effort"] = None
elif re_val not in _REASONING_EFFORT_CHOICES:
return JSONResponse(
{"error": f"Invalid reasoning_effort: {re_val!r}"},
status_code=400,
)
else:
updates["reasoning_effort"] = re_val
if updates:
storage.update_model_definition(definition_id, **updates)
@@ -7933,7 +8025,7 @@ def main() -> None:
else:
_advertise_host = args.host
if _advertise_host in ("0.0.0.0", "::", ""):
_advertise_host = _socket.getfqdn()
_advertise_host = _socket.gethostname()
console_url = f"http://{_advertise_host}:{args.port}"
if auth_storage:
try:
+223 -7
View File
@@ -4447,6 +4447,24 @@ function _renderModels(items) {
colAlias.appendChild(document.createTextNode(" "));
colAlias.appendChild(defBadge);
}
// Per-model sampling override indicators
var overrides = [];
if (m.temperature != null) overrides.push("temp=" + m.temperature);
if (m.max_tokens != null) overrides.push("max_tok=" + m.max_tokens);
if (m.reasoning_effort != null)
overrides.push("effort=" + m.reasoning_effort);
if (overrides.length) {
var ovrSpan = document.createElement("span");
ovrSpan.className = "model-overrides-hint";
ovrSpan.textContent = overrides.join(", ");
ovrSpan.title = "Per-model overrides (override global defaults)";
ovrSpan.setAttribute(
"aria-label",
"Per-model overrides: " + overrides.join(", "),
);
colAlias.appendChild(document.createElement("br"));
colAlias.appendChild(ovrSpan);
}
row.appendChild(colAlias);
// Model ID
@@ -4578,6 +4596,19 @@ function _renderModels(items) {
});
}
function _isPlainObject(v) {
return v !== null && typeof v === "object" && !Array.isArray(v);
}
function _toggleThinkingParam() {
var mode = document.getElementById("model-thinking-mode").value;
var row = document.getElementById("model-thinking-param-row");
row.style.display = mode ? "" : "none";
// Set default when first enabling
var paramEl = document.getElementById("model-thinking-param");
if (mode && !paramEl.value) paramEl.value = "enable_thinking";
}
function showCreateModelModal() {
_modelCreateTrigger = document.activeElement;
var ov = document.getElementById("model-create-overlay");
@@ -4593,7 +4624,21 @@ function showCreateModelModal() {
document.getElementById("model-api-key").value = "";
document.getElementById("model-api-key").placeholder = "sk-...";
document.getElementById("model-ctx-window").value = "0";
document.getElementById("model-temperature").value = "";
document.getElementById("model-max-tokens").value = "";
document.getElementById("model-reasoning-effort").value = "";
document.getElementById("model-server-type").value = "";
document.getElementById("model-thinking-mode").value = "";
document.getElementById("model-thinking-param").value = "";
document.getElementById("model-thinking-param-row").style.display = "none";
document.getElementById("model-extra-body").value = "";
document.getElementById("model-capabilities").value = "";
// Clear validation error styling from prior submit attempts
["model-extra-body", "model-capabilities"].forEach(function (id) {
var el = document.getElementById(id);
el.removeAttribute("aria-invalid");
el.style.borderColor = "";
});
document.getElementById("model-enabled").checked = true;
document.getElementById("model-detect-result").style.display = "none";
document.getElementById("model-detect-btn").disabled = false;
@@ -4626,15 +4671,55 @@ function showEditModelModal(definitionId) {
"\u2022\u2022\u2022 (leave blank to keep existing)";
document.getElementById("model-ctx-window").value =
m.context_window != null ? m.context_window : 0;
// Parse capabilities JSON for display
var caps = m.capabilities || "{}";
document.getElementById("model-temperature").value =
m.temperature != null ? m.temperature : "";
document.getElementById("model-max-tokens").value =
m.max_tokens != null ? m.max_tokens : "";
document.getElementById("model-reasoning-effort").value =
m.reasoning_effort != null ? m.reasoning_effort : "";
// Parse capabilities JSON and extract server_compat for structured fields
var capsObj = {};
try {
caps = JSON.stringify(JSON.parse(caps), null, 2);
capsObj = JSON.parse(m.capabilities || "{}");
} catch (e) {
/* keep raw */
/* keep empty */
}
if (caps === "{}") caps = "";
document.getElementById("model-capabilities").value = caps;
// Defend against null/array/primitive values in the DB
if (!_isPlainObject(capsObj)) capsObj = {};
var sc = _isPlainObject(capsObj.server_compat)
? capsObj.server_compat
: {};
// Only extract thinking_mode into the dropdown when the UI can
// represent it ("manual" or ""). Values like "adaptive" (Anthropic-
// only) stay in the raw capabilities JSON so they aren't silently
// lost on save.
var tmVal = capsObj.thinking_mode || "";
var tmRepresentable = tmVal === "" || tmVal === "manual";
if (tmRepresentable) {
document.getElementById("model-thinking-mode").value = tmVal;
document.getElementById("model-thinking-param").value =
capsObj.thinking_param || "";
} else {
document.getElementById("model-thinking-mode").value = "";
document.getElementById("model-thinking-param").value = "";
}
_toggleThinkingParam();
// Server compat: server_type and extra_body workarounds
document.getElementById("model-server-type").value = sc.server_type || "";
var eb = sc.extra_body || {};
var ebText = JSON.stringify(eb, null, 2);
document.getElementById("model-extra-body").value =
ebText === "{}" ? "" : ebText;
// Remove structured fields from capabilities display — only delete
// thinking_mode/thinking_param when the UI successfully captured them.
delete capsObj.server_compat;
if (tmRepresentable) {
delete capsObj.thinking_mode;
delete capsObj.thinking_param;
}
var capsText = JSON.stringify(capsObj, null, 2);
document.getElementById("model-capabilities").value =
capsText === "{}" ? "" : capsText;
document.getElementById("model-enabled").checked = m.enabled !== false;
_applyProviderDefaults();
})
@@ -4667,15 +4752,65 @@ function submitCreateModel() {
return;
}
var capsText = document.getElementById("model-capabilities").value.trim();
var capsEl = document.getElementById("model-capabilities");
var capsText = capsEl.value.trim();
var caps = {};
capsEl.removeAttribute("aria-invalid");
capsEl.style.borderColor = "";
if (capsText) {
try {
caps = JSON.parse(capsText);
} catch (e) {
capsEl.setAttribute("aria-invalid", "true");
capsEl.style.borderColor = "var(--red)";
_showModelError("Invalid JSON in capabilities");
return;
}
if (!_isPlainObject(caps)) {
capsEl.setAttribute("aria-invalid", "true");
capsEl.style.borderColor = "var(--red)";
_showModelError(
"Capabilities must be a JSON object (not array or primitive)",
);
return;
}
}
// Thinking mode → capabilities (provider uses this to inject
// the correct chat_template_kwargs param automatically).
var thinkingMode = document.getElementById("model-thinking-mode").value;
if (thinkingMode) {
caps.thinking_mode = thinkingMode;
// Preserve thinking_param so Granite/DeepSeek "thinking" key
// isn't silently reverted to the default "enable_thinking".
var savedParam = document.getElementById("model-thinking-param").value;
if (savedParam) caps.thinking_param = savedParam;
}
// Build server_compat from structured fields
var serverCompat = {};
var serverType = document.getElementById("model-server-type").value;
if (serverType) serverCompat.server_type = serverType;
var ebEl = document.getElementById("model-extra-body");
var ebText = ebEl.value.trim();
ebEl.removeAttribute("aria-invalid");
ebEl.style.borderColor = "";
if (ebText) {
try {
var ebParsed = JSON.parse(ebText);
if (!_isPlainObject(ebParsed)) {
throw new Error("not an object");
}
serverCompat.extra_body = ebParsed;
} catch (e) {
ebEl.setAttribute("aria-invalid", "true");
ebEl.style.borderColor = "var(--red)";
_showModelError("Extra body params must be a JSON object");
return;
}
}
if (Object.keys(serverCompat).length > 0) {
caps.server_compat = serverCompat;
}
var form = {
@@ -4689,6 +4824,36 @@ function submitCreateModel() {
enabled: document.getElementById("model-enabled").checked,
};
// Per-model sampling overrides — null when empty (use global default)
var tempVal = document.getElementById("model-temperature").value.trim();
if (tempVal !== "") {
var t = parseFloat(tempVal);
if (isNaN(t) || t < 0 || t > 2) {
_showModelError("Temperature must be between 0 and 2");
return;
}
form.temperature = t;
} else {
form.temperature = null;
}
var mtVal = document.getElementById("model-max-tokens").value.trim();
if (mtVal !== "") {
var mt = parseInt(mtVal, 10);
if (isNaN(mt) || mt < 1) {
_showModelError("Max tokens must be at least 1");
return;
}
form.max_tokens = mt;
} else {
form.max_tokens = null;
}
var reVal = document.getElementById("model-reasoning-effort").value;
if (reVal !== "") {
form.reasoning_effort = reVal;
} else {
form.reasoning_effort = null;
}
var apiKey = document.getElementById("model-api-key").value;
if (apiKey) form.api_key = apiKey;
@@ -4838,6 +5003,52 @@ function detectModel() {
resultDiv.appendChild(
_detectResultLine("Server type: " + d.server_type),
);
// Auto-fill server type if not already set and value is a known option
var stEl = document.getElementById("model-server-type");
var stOpts = Array.from(stEl.options).map(function (o) {
return o.value;
});
if (!stEl.value && stOpts.indexOf(d.server_type) !== -1)
stEl.value = d.server_type;
}
// Auto-fill capabilities from suggested profile
if (d.suggested_capabilities) {
var sc2 = d.suggested_capabilities;
var tmEl = document.getElementById("model-thinking-mode");
if (!tmEl.value && sc2.thinking_mode) {
tmEl.value = sc2.thinking_mode;
}
if (sc2.thinking_param) {
var tpEl = document.getElementById("model-thinking-param");
if (!tpEl.value) tpEl.value = sc2.thinking_param;
}
_toggleThinkingParam();
}
// Auto-fill server compat from suggested profile
if (d.suggested_server_compat) {
var ssc = d.suggested_server_compat;
var stEl2 = document.getElementById("model-server-type");
var stOpts2 = Array.from(stEl2.options).map(function (o) {
return o.value;
});
if (
!stEl2.value &&
ssc.server_type &&
stOpts2.indexOf(ssc.server_type) !== -1
)
stEl2.value = ssc.server_type;
if (ssc.extra_body) {
var ebEl2 = document.getElementById("model-extra-body");
if (!ebEl2.value.trim()) {
var ebJson = JSON.stringify(ssc.extra_body, null, 2);
if (ebJson !== "{}") ebEl2.value = ebJson;
}
}
}
if (d.suggested_capabilities || d.suggested_server_compat) {
resultDiv.appendChild(
_detectResultLine("\u2713 Compatibility profile suggested", "green"),
);
}
resultDiv.style.borderColor = "var(--green)";
})
@@ -4931,6 +5142,11 @@ function _applyProviderDefaults() {
if (!def) return;
document.getElementById("model-base-url").placeholder = def.urlPlaceholder;
document.getElementById("model-name").placeholder = def.modelPlaceholder;
// Server compat section only applies to local model servers
var scSection = document.getElementById("model-server-compat-section");
if (scSection) {
scSection.style.display = provider === "openai-compatible" ? "" : "none";
}
}
/* Populate the model name datalist with known model prefixes for the
+36
View File
@@ -1551,6 +1551,42 @@ window.TURNSTONE_KB_SHORTCUTS = [
<input type="password" id="model-api-key" placeholder="sk-..." autocomplete="off">
<label for="model-ctx-window">Context Window <span style="font-weight:400;text-transform:none">(0 = auto-detect from model)</span></label>
<input type="number" id="model-ctx-window" value="0" min="0">
<div class="modal-section-divider" role="separator">Sampling Defaults</div>
<label for="model-temperature">Temperature <span style="font-weight:400;text-transform:none">(empty = use global default)</span></label>
<input type="number" id="model-temperature" placeholder="Global default" step="0.1" min="0" max="2">
<label for="model-max-tokens">Max Tokens <span style="font-weight:400;text-transform:none">(empty = use global default)</span></label>
<input type="number" id="model-max-tokens" placeholder="Global default" min="1">
<label for="model-reasoning-effort">Reasoning Effort <span style="font-weight:400;text-transform:none">(empty = use global default)</span></label>
<select id="model-reasoning-effort">
<option value="">Global default</option>
<option value="none">none</option>
<option value="minimal">minimal</option>
<option value="low">low</option>
<option value="medium">medium</option>
<option value="high">high</option>
<option value="xhigh">xhigh</option>
<option value="max">max</option>
</select>
<div id="model-server-compat-section" style="display:none">
<div class="modal-section-divider" role="separator">Server Compatibility</div>
<label for="model-server-type">Server Type <span style="font-weight:400;text-transform:none">(auto-detected or manual)</span></label>
<select id="model-server-type">
<option value="">Auto / Unknown</option>
<option value="vllm">vLLM</option>
<option value="llama.cpp">llama.cpp</option>
<option value="openai-compatible">Other OpenAI-compatible</option>
</select>
<label for="model-thinking-mode">Thinking Mode <span style="font-weight:400;text-transform:none">(reasoning / chain-of-thought)</span></label>
<select id="model-thinking-mode" onchange="_toggleThinkingParam()">
<option value="">None</option>
<option value="manual">Enabled</option>
</select>
<div id="model-thinking-param-row" style="display:none">
<label for="model-thinking-param" style="font-size:11px">Template param name <span style="font-weight:400;text-transform:none">(Granite/DeepSeek use "thinking")</span></label>
<input type="text" id="model-thinking-param" value="enable_thinking" placeholder="enable_thinking" style="font-family:var(--font-mono);font-size:11px"></div>
<label for="model-extra-body">Extra body params <span style="font-weight:400;text-transform:none">(JSON, merged into every request)</span></label>
<textarea id="model-extra-body" rows="2" placeholder='{"skip_special_tokens": false}' style="font-family:var(--font-mono);font-size:11px"></textarea>
</div>
<label for="model-capabilities">Capabilities <span style="font-weight:400;text-transform:none">(JSON)</span></label>
<textarea id="model-capabilities" rows="3" placeholder='{"supports_vision": true}' style="font-family:var(--font-mono);font-size:11px"></textarea>
<div style="display:flex;gap:20px;margin-top:14px">
+6
View File
@@ -2471,6 +2471,12 @@ h3.skill-spec-heading { font-size: inherit; margin-block: 0; }
.model-provider-google{color:var(--green);border-color:rgba(52,211,153,.2)}
.model-provider-compat{color:var(--fg-dim);border-color:var(--border-strong)}
/* Per-model override hints */
.model-overrides-hint{font-size:10px;color:var(--fg-dim);font-family:var(--font-mono);letter-spacing:.02em}
/* Modal section divider for field groups */
.modal-section-divider{font-family:var(--font-display);font-size:9px;font-weight:600;text-transform:uppercase;letter-spacing:.1em;color:var(--fg-dim);margin:16px 0 4px;padding-top:12px;border-top:1px solid var(--border)}
/* Model source badge */
.scope-db{color:var(--blue);border-color:rgba(56,189,248,.2)}
+18
View File
@@ -266,3 +266,21 @@ def warn_migrated_settings() -> None:
config_key,
key,
)
# Warn about removed settings whose config.toml keys are now ignored.
# model.name → use model definitions (Models tab); model.context_window
# → set per-model in the Models tab (context_window column).
removed_settings: dict[str, str] = {
"model.name": "Use model definitions in the Models tab instead.",
"model.context_window": "Set per-model in the Models tab instead.",
}
for key, guidance in removed_settings.items():
section, config_key = key.split(".", 1)
section_data = cfg.get(section, {})
if isinstance(section_data, dict) and config_key in section_data:
log.warning(
"config.toml [%s] %s has been removed and will be ignored. %s",
section,
config_key,
guidance,
)
+1 -1
View File
@@ -73,7 +73,7 @@ _EXPLICIT_SCRUB: frozenset[str] = frozenset(
"AZURE_CLIENT_SECRET",
"GCP_SERVICE_ACCOUNT_KEY",
"GOOGLE_APPLICATION_CREDENTIALS",
"DATABASE_URL",
"DATABASE_URL", # conventional name (Heroku, Railway, etc.) — kept for defence-in-depth
"TURNSTONE_DB_URL",
}
)
+73 -3
View File
@@ -35,6 +35,13 @@ class ModelConfig:
provider: str = "openai"
capabilities: dict[str, Any] = field(default_factory=dict)
source: str = "" # "config", "db", or "" (CLI default)
# Per-model sampling overrides (None = use global default from ConfigStore)
temperature: float | None = None
max_tokens: int | None = None
reasoning_effort: str | None = None
# Server compatibility settings for openai-compatible backends.
# Populated from capabilities["server_compat"] during load.
server_compat: dict[str, Any] = field(default_factory=dict)
# ---------------------------------------------------------------------------
@@ -262,12 +269,20 @@ def load_model_registry(
caps = parsed
except (_json.JSONDecodeError, TypeError):
pass # falls back to empty capabilities
# Extract server_compat from capabilities (namespaced key)
row_server_compat = caps.pop("server_compat", {})
if not isinstance(row_server_compat, dict):
row_server_compat = {}
row_base_url = _resolve_env_vars(row.get("base_url", ""))
row_provider = _resolve_openai_provider(row.get("provider", "openai"), row_base_url)
row_model = row["model"]
# 0 = auto-detect: inherit CLI-detected context_window,
# same fallback chain as config.toml models
row_ctx = row.get("context_window", 0) or context_window
# Per-model sampling overrides (None = use global default)
row_temperature = row.get("temperature")
row_max_tokens = row.get("max_tokens")
row_reasoning_effort = row.get("reasoning_effort")
configs[alias] = ModelConfig(
alias=alias,
base_url=row_base_url,
@@ -277,6 +292,12 @@ def load_model_registry(
provider=row_provider,
capabilities=caps,
source="db",
temperature=float(row_temperature) if row_temperature is not None else None,
max_tokens=int(row_max_tokens) if row_max_tokens is not None else None,
reasoning_effort=row_reasoning_effort
if row_reasoning_effort is not None
else None,
server_compat=row_server_compat,
)
except Exception:
log.warning("Failed to load model definitions from storage", exc_info=True)
@@ -290,6 +311,44 @@ def load_model_registry(
log.warning("Model entry '%s' has no model name, skipping", alias)
continue
entry_base_url = _resolve_env_vars(entry.get("base_url", base_url))
# Per-model sampling overrides from config.toml — invalid values
# are logged and treated as None (inherit global default).
entry_temp: float | None = None
entry_max_tokens: int | None = None
entry_effort: str | None = None
raw_temp = entry.get("temperature")
if raw_temp is not None:
try:
entry_temp = float(raw_temp)
if not 0.0 <= entry_temp <= 2.0:
log.warning(
"Model '%s' temperature %.2f out of range [0, 2], ignoring",
alias,
entry_temp,
)
entry_temp = None
except (ValueError, TypeError):
log.warning("Model '%s' has invalid temperature %r, ignoring", alias, raw_temp)
raw_mt = entry.get("max_tokens")
if raw_mt is not None:
try:
entry_max_tokens = int(raw_mt)
if entry_max_tokens < 1:
log.warning("Model '%s' max_tokens %d < 1, ignoring", alias, entry_max_tokens)
entry_max_tokens = None
except (ValueError, TypeError):
log.warning("Model '%s' has invalid max_tokens %r, ignoring", alias, raw_mt)
raw_effort = entry.get("reasoning_effort")
if raw_effort is not None:
entry_effort = str(raw_effort)
entry_caps = (
dict(entry.get("capabilities", {}))
if isinstance(entry.get("capabilities"), dict)
else {}
)
entry_server_compat = entry_caps.pop("server_compat", {})
if not isinstance(entry_server_compat, dict):
entry_server_compat = {}
configs[alias] = ModelConfig(
alias=alias,
base_url=entry_base_url,
@@ -297,10 +356,12 @@ def load_model_registry(
model=model_name,
context_window=entry.get("context_window", context_window),
provider=_resolve_openai_provider(entry.get("provider", "openai"), entry_base_url),
capabilities=entry.get("capabilities", {})
if isinstance(entry.get("capabilities"), dict)
else {},
capabilities=entry_caps,
source="config",
temperature=entry_temp,
max_tokens=entry_max_tokens,
reasoning_effort=entry_effort,
server_compat=entry_server_compat,
)
# 3. Ensure a "default" entry from CLI args (only if not already defined
@@ -596,3 +657,12 @@ def _detect_openai_compat(
result["server_type"] = "vllm"
else:
result["server_type"] = "openai-compatible"
# Suggest capabilities and server compat based on detected server_type
from turnstone.core.server_compat import suggest_profile
suggested = suggest_profile(result.get("server_type", ""), model_id)
if suggested.get("capabilities"):
result["suggested_capabilities"] = suggested["capabilities"]
if suggested.get("server_compat"):
result["suggested_server_compat"] = suggested["server_compat"]
+4 -2
View File
@@ -568,9 +568,10 @@ class AnthropicProvider:
extra_params: dict[str, Any] | None = None,
deferred_names: frozenset[str] | None = None,
cancel_ref: list[Any] | None = None,
capabilities: ModelCapabilities | None = None,
) -> Iterator[StreamChunk]:
_ensure_anthropic()
caps = self.get_capabilities(model)
caps = capabilities or self.get_capabilities(model)
system_prompt, converted_msgs = self._convert_messages(messages)
kwargs = self._build_thinking_and_kwargs(
caps,
@@ -771,9 +772,10 @@ class AnthropicProvider:
reasoning_effort: str = "medium",
extra_params: dict[str, Any] | None = None,
deferred_names: frozenset[str] | None = None,
capabilities: ModelCapabilities | None = None,
) -> CompletionResult:
_ensure_anthropic()
caps = self.get_capabilities(model)
caps = capabilities or self.get_capabilities(model)
system_prompt, converted_msgs = self._convert_messages(messages)
kwargs = self._build_thinking_and_kwargs(
caps,
+56 -6
View File
@@ -108,6 +108,52 @@ class OpenAIChatCompletionsProvider:
kwargs["web_search_options"] = {}
return tools
# -- thinking mode -------------------------------------------------------
@staticmethod
def _apply_thinking_mode(
extra_body: dict[str, Any],
caps: ModelCapabilities,
) -> None:
"""Inject thinking-mode params into *extra_body* based on capabilities.
When ``caps.thinking_mode`` is ``"manual"`` or ``"adaptive"``, sets
the model-family-specific key (``caps.thinking_param``, e.g.
``"enable_thinking"`` or ``"thinking"``) to ``True`` inside
``extra_body["chat_template_kwargs"]``.
Does nothing when thinking mode is ``"none"`` or the key is already
present (operator override via ``extra_body`` takes precedence).
"""
if caps.thinking_mode == "none":
return
ctk = extra_body.get("chat_template_kwargs")
if not isinstance(ctk, dict):
ctk = {}
extra_body["chat_template_kwargs"] = ctk
if caps.thinking_param not in ctk:
ctk[caps.thinking_param] = True
def _finalize_extra_body(
self,
extra_params: dict[str, Any] | None,
caps: ModelCapabilities,
) -> dict[str, Any] | None:
"""Build the final ``extra_body``, injecting thinking params if needed.
Returns ``None`` when the result would be empty (no extra_body needed).
Shallow-copies *extra_params* and its ``chat_template_kwargs`` so the
caller's dict is never mutated.
"""
eb: dict[str, Any] = {}
if extra_params:
eb = dict(extra_params)
ctk = eb.get("chat_template_kwargs")
if isinstance(ctk, dict):
eb["chat_template_kwargs"] = dict(ctk)
self._apply_thinking_mode(eb, caps)
return eb or None
# -- streaming -----------------------------------------------------------
def create_streaming(
@@ -123,8 +169,9 @@ class OpenAIChatCompletionsProvider:
extra_params: dict[str, Any] | None = None,
deferred_names: frozenset[str] | None = None,
cancel_ref: list[Any] | None = None,
capabilities: ModelCapabilities | None = None,
) -> Iterator[StreamChunk]:
caps = self.get_capabilities(model)
caps = capabilities or self.get_capabilities(model)
messages = self._prepare_messages(messages)
kwargs: dict[str, Any] = {
"model": model,
@@ -139,8 +186,9 @@ class OpenAIChatCompletionsProvider:
tools = apply_tool_search(caps, tools, deferred_names)
if tools:
kwargs["tools"] = tools
if extra_params:
kwargs["extra_body"] = extra_params
extra_body = self._finalize_extra_body(extra_params, caps)
if extra_body:
kwargs["extra_body"] = extra_body
log.debug(
"openai.chat.request",
@@ -250,8 +298,9 @@ class OpenAIChatCompletionsProvider:
reasoning_effort: str = "medium",
extra_params: dict[str, Any] | None = None,
deferred_names: frozenset[str] | None = None,
capabilities: ModelCapabilities | None = None,
) -> CompletionResult:
caps = self.get_capabilities(model)
caps = capabilities or self.get_capabilities(model)
messages = self._prepare_messages(messages)
kwargs: dict[str, Any] = {
"model": model,
@@ -265,8 +314,9 @@ class OpenAIChatCompletionsProvider:
tools = apply_tool_search(caps, tools, deferred_names)
if tools:
kwargs["tools"] = tools
if extra_params:
kwargs["extra_body"] = extra_params
extra_body = self._finalize_extra_body(extra_params, caps)
if extra_body:
kwargs["extra_body"] = extra_body
log.debug(
"openai.chat.request",
@@ -223,9 +223,10 @@ class OpenAIResponsesProvider:
temperature: float,
reasoning_effort: str,
deferred_names: frozenset[str] | None,
capabilities: ModelCapabilities | None = None,
) -> dict[str, Any]:
"""Build the kwargs dict for ``client.responses.create/stream``."""
caps = self.get_capabilities(model)
caps = capabilities or self.get_capabilities(model)
instructions, input_items = self._convert_messages(messages)
tools = apply_tool_search(caps, tools, deferred_names)
@@ -276,6 +277,7 @@ class OpenAIResponsesProvider:
extra_params: dict[str, Any] | None = None,
deferred_names: frozenset[str] | None = None,
cancel_ref: list[Any] | None = None,
capabilities: ModelCapabilities | None = None,
) -> Iterator[StreamChunk]:
if extra_params:
log.debug("openai.responses: extra_params ignored (not supported by Responses API)")
@@ -287,6 +289,7 @@ class OpenAIResponsesProvider:
temperature,
reasoning_effort,
deferred_names,
capabilities=capabilities,
)
kwargs["stream"] = True
@@ -455,6 +458,7 @@ class OpenAIResponsesProvider:
reasoning_effort: str = "medium",
extra_params: dict[str, Any] | None = None,
deferred_names: frozenset[str] | None = None,
capabilities: ModelCapabilities | None = None,
) -> CompletionResult:
if extra_params:
log.debug("openai.responses: extra_params ignored (not supported by Responses API)")
@@ -466,6 +470,7 @@ class OpenAIResponsesProvider:
temperature,
reasoning_effort,
deferred_names,
capabilities=capabilities,
)
log.debug(
+13
View File
@@ -74,6 +74,11 @@ class ModelCapabilities:
supports_tools: bool = True
token_param: str = "max_completion_tokens"
thinking_mode: str = "none" # "none" | "manual" | "adaptive"
# For openai-compatible servers: the chat_template_kwargs key that
# toggles thinking (e.g. "enable_thinking" for Gemma/Qwen,
# "thinking" for Granite/DeepSeek). Ignored when thinking_mode is
# "none" or by providers that handle thinking natively (Anthropic).
thinking_param: str = "enable_thinking"
supports_effort: bool = False
effort_levels: tuple[str, ...] = ()
reasoning_effort_values: tuple[str, ...] = ()
@@ -127,9 +132,16 @@ class LLMProvider(Protocol):
extra_params: dict[str, Any] | None = None,
deferred_names: frozenset[str] | None = None,
cancel_ref: list[Any] | None = None,
capabilities: ModelCapabilities | None = None,
) -> Iterator[StreamChunk]:
"""Create a streaming request, yielding normalized StreamChunks.
If *capabilities* is provided the provider uses it instead of
calling ``get_capabilities(model)`` internally. This lets the
session pass config-merged capabilities so that overrides from
the model registry (e.g. ``thinking_mode``, ``token_param``)
are respected.
If *cancel_ref* is provided the provider appends the underlying SDK
stream object (which has a ``.close()`` method) before yielding the
first chunk. The caller can then close it from another thread to
@@ -149,6 +161,7 @@ class LLMProvider(Protocol):
reasoning_effort: str = "medium",
extra_params: dict[str, Any] | None = None,
deferred_names: frozenset[str] | None = None,
capabilities: ModelCapabilities | None = None,
) -> CompletionResult:
"""Create a non-streaming request, returning a normalized result."""
...
+207
View File
@@ -0,0 +1,207 @@
"""Server compatibility profiles for OpenAI-compatible backends.
Different local model servers (vLLM, llama.cpp, SGLang) need different
request shaping. This module separates two concerns:
1. **Model capabilities** ``thinking_mode`` and ``thinking_param`` are
properties of the *model* (Gemma thinks, Llama doesn't). These go
into the ``capabilities`` dict and flow through ``ModelCapabilities``
so the provider can act on them (just like Anthropic's thinking mode).
2. **Server workarounds** ``extra_body`` overrides like
``skip_special_tokens=false`` are properties of the *server* (vLLM
bug workaround). These stay in ``server_compat`` and get merged
into the request's ``extra_body`` at call time.
Profiles are *suggestions* only. The admin UI auto-fills them on
Detect; the operator has final say, and the stored DB config is what
actually gets used at request time.
"""
from __future__ import annotations
import copy
from typing import Any
# ---------------------------------------------------------------------------
# Profile suggestions
# ---------------------------------------------------------------------------
# Each profile has two optional parts:
# "capabilities" — merged into the model's capabilities dict (thinking_mode etc.)
# "server_compat" — stored as server_compat (extra_body workarounds)
_PROFILES: dict[str, dict[str, Any]] = {
"vllm-gemma-thinking": {
"capabilities": {
"thinking_mode": "manual",
"thinking_param": "enable_thinking",
},
"server_compat": {
"server_type": "vllm",
# Workaround: vLLM strips special tokens before the Gemma4
# reasoning parser sees them. skip_special_tokens=false
# preserves <|channel> / <channel|> markers so reasoning
# content is extracted correctly.
"extra_body": {"skip_special_tokens": False},
},
},
"vllm-qwen-thinking": {
"capabilities": {
"thinking_mode": "manual",
"thinking_param": "enable_thinking",
},
"server_compat": {
"server_type": "vllm",
},
},
"vllm-granite-thinking": {
"capabilities": {
"thinking_mode": "manual",
"thinking_param": "thinking",
},
"server_compat": {
"server_type": "vllm",
},
},
"vllm-deepseek-thinking": {
"capabilities": {
"thinking_mode": "manual",
"thinking_param": "thinking",
},
"server_compat": {
"server_type": "vllm",
},
},
"vllm-holo-thinking": {
"capabilities": {
"thinking_mode": "manual",
"thinking_param": "enable_thinking",
},
"server_compat": {
"server_type": "vllm",
},
},
"vllm": {
"server_compat": {
"server_type": "vllm",
},
},
"llama.cpp": {
"server_compat": {
"server_type": "llama.cpp",
},
},
"llama.cpp-thinking": {
"capabilities": {
"thinking_mode": "manual",
"thinking_param": "enable_thinking",
},
"server_compat": {
"server_type": "llama.cpp",
# llama.cpp uses reasoning_format (top-level request param) to
# extract thinking into the reasoning_content response field.
# "auto" lets the server decide based on the model's template;
# "deepseek" forces extraction for all thinking models.
"extra_body": {"reasoning_format": "auto"},
},
},
"sglang": {
"server_compat": {
"server_type": "sglang",
},
},
}
# Model-family → profile key mapping. Checked in order; first match wins.
_VLLM_MODEL_PROFILES: list[tuple[str, str]] = [
("gemma-4", "vllm-gemma-thinking"),
("gemma-3", "vllm-gemma-thinking"),
("gemma4", "vllm-gemma-thinking"),
("gemma3", "vllm-gemma-thinking"),
("qwen3", "vllm-qwen-thinking"),
("qwq", "vllm-qwen-thinking"),
("granite-3", "vllm-granite-thinking"),
("granite3", "vllm-granite-thinking"),
("deepseek-r1", "vllm-deepseek-thinking"),
("holo2", "vllm-holo-thinking"),
]
# llama.cpp model-family → profile key mapping.
_LLAMA_CPP_MODEL_PROFILES: list[tuple[str, str]] = [
("gemma-4", "llama.cpp-thinking"),
("gemma-3", "llama.cpp-thinking"),
("gemma4", "llama.cpp-thinking"),
("gemma3", "llama.cpp-thinking"),
("qwen3", "llama.cpp-thinking"),
("qwq", "llama.cpp-thinking"),
("deepseek-r1", "llama.cpp-thinking"),
]
# ---------------------------------------------------------------------------
# Public API
# ---------------------------------------------------------------------------
def suggest_profile(server_type: str, model_id: str) -> dict[str, Any]:
"""Suggest capabilities and server compat based on server type and model.
Returns a dict with optional ``"capabilities"`` and ``"server_compat"``
keys. Empty dict when no special settings are needed.
"""
profile_key: str | None = None
model_lower = (model_id or "").lower()
if server_type == "vllm":
for substring, key in _VLLM_MODEL_PROFILES:
if substring in model_lower:
profile_key = key
break
if profile_key is None:
profile_key = "vllm"
elif server_type == "llama.cpp":
for substring, key in _LLAMA_CPP_MODEL_PROFILES:
if substring in model_lower:
profile_key = key
break
if profile_key is None:
profile_key = "llama.cpp"
elif server_type in _PROFILES:
profile_key = server_type
if profile_key is None:
return {}
return copy.deepcopy(_PROFILES[profile_key])
def merge_server_compat(
base_chat_template_kwargs: dict[str, Any],
server_compat: dict[str, Any],
) -> dict[str, Any]:
"""Build the ``extra_body`` dict by merging server compat into base kwargs.
*base_chat_template_kwargs* always contains at least ``reasoning_effort``.
*server_compat* comes from ``ModelConfig.server_compat``.
Note: thinking-mode params (``enable_thinking``, ``thinking``) are **not**
merged here the provider handles those via ``ModelCapabilities``.
This function only merges server workarounds from ``extra_body``.
Returns the complete dict to pass as ``extra_body`` to the OpenAI client.
"""
extra: dict[str, Any] = {"chat_template_kwargs": dict(base_chat_template_kwargs)}
# Merge top-level extra_body overrides (skip_special_tokens, etc.)
compat_eb = server_compat.get("extra_body")
if isinstance(compat_eb, dict):
for key, value in compat_eb.items():
if key == "chat_template_kwargs":
# Deep-merge: operator values in extra_body win over the
# base dict (which has reasoning_effort). This lets
# operators intentionally extend chat_template_kwargs.
if isinstance(value, dict):
extra["chat_template_kwargs"].update(value)
continue
extra[key] = value
return extra
+71 -6
View File
@@ -1448,21 +1448,50 @@ class ChatSession:
self,
reasoning_effort: str | None = None,
provider: LLMProvider | None = None,
model_alias: str | None = None,
) -> dict[str, Any] | None:
"""Build provider-specific extra parameters.
``chat_template_kwargs`` is only meaningful for local model servers
(``openai-compatible``). Commercial OpenAI rejects it as an unknown
parameter, and handles ``reasoning_effort`` natively.
Merges server workarounds (``skip_special_tokens``, etc.) from
``ModelConfig.server_compat`` into the request's ``extra_body``.
Thinking-mode params (``enable_thinking``) are handled separately
by the provider based on ``ModelCapabilities.thinking_mode``.
*model_alias* controls which model config supplies server compat
settings. When ``None``, defaults to the session's primary alias.
"""
from turnstone.core.server_compat import merge_server_compat
prov = provider or self._provider
if prov.provider_name == "openai-compatible":
kwargs = dict(self._chat_template_kwargs_base)
ctk_base = dict(self._chat_template_kwargs_base)
if reasoning_effort:
kwargs["reasoning_effort"] = reasoning_effort
return {"chat_template_kwargs": kwargs}
ctk_base["reasoning_effort"] = reasoning_effort
return merge_server_compat(
ctk_base,
self._get_server_compat(model_alias),
)
return None
def _get_server_compat(self, model_alias: str | None = None) -> dict[str, Any]:
"""Get server compatibility settings from a model config.
*model_alias* selects the config to read. Falls back to the
session's primary alias when ``None``.
"""
alias = model_alias or self._model_alias
if self._registry and alias:
try:
cfg = self._registry.get_config(alias)
return dict(cfg.server_compat)
except (ValueError, KeyError):
pass
return {}
def _utility_completion(
self,
messages: list[dict[str, Any]],
@@ -1488,6 +1517,7 @@ class ChatSession:
temperature=temperature,
reasoning_effort=reasoning_effort,
extra_params=self._provider_extra_params(reasoning_effort=reasoning_effort),
capabilities=caps,
)
# -- tool search helpers --------------------------------------------------
@@ -1622,8 +1652,16 @@ class ChatSession:
try:
fb_client, fb_model, _ = self._registry.resolve(alias)
fb_provider = self._registry.get_provider(alias)
fb_caps = self._resolve_capabilities(fb_provider, fb_model, alias)
self.ui.on_info(f"[Primary model failed, falling back to {alias}]")
result = self._try_stream(fb_client, fb_model, msgs, provider=fb_provider)
result = self._try_stream(
fb_client,
fb_model,
msgs,
provider=fb_provider,
capabilities=fb_caps,
model_alias=alias,
)
if fb_tracker:
fb_tracker.record_success()
return result
@@ -1639,6 +1677,8 @@ class ChatSession:
model: str,
msgs: list[dict[str, Any]],
provider: LLMProvider | None = None,
capabilities: ModelCapabilities | None = None,
model_alias: str | None = None,
) -> Iterator[StreamChunk]:
"""Attempt a streaming API call with retries on transient errors."""
prov = provider or self._provider
@@ -1670,9 +1710,12 @@ class ChatSession:
max_tokens=self.max_tokens,
temperature=self.temperature,
reasoning_effort=self.reasoning_effort,
extra_params=self._provider_extra_params(provider=prov),
extra_params=self._provider_extra_params(
provider=prov, model_alias=model_alias
),
deferred_names=self._get_deferred_names(),
cancel_ref=self._cancel_ref,
capabilities=capabilities or self._get_capabilities(prov, model),
)
except Exception as e:
ename = type(e).__name__
@@ -5381,10 +5424,12 @@ class ChatSession:
if not agent_caps.supports_web_search and not self._resolve_search_client():
tools = _without_tool(tools, "web_search")
# Build extra params for agent calls
# Build extra params for agent calls — resolve server compat from the
# agent's own model alias, not the session's primary model.
agent_extra = self._provider_extra_params(
reasoning_effort=reasoning_effort,
provider=agent_provider,
model_alias=agent_alias,
)
def _api_call(
@@ -5403,6 +5448,7 @@ class ChatSession:
temperature=self.temperature,
reasoning_effort=reasoning_effort or self.reasoning_effort,
extra_params=agent_extra,
capabilities=agent_caps,
)
except Exception as e:
ename = type(e).__name__
@@ -6924,6 +6970,25 @@ class ChatSession:
self.context_window = cfg.context_window
if not self._manual_tool_truncation:
self.tool_truncation = int(cfg.context_window * self._chars_per_token * 0.5)
# Apply per-model sampling overrides, falling back to global
# defaults — mirrors session_factory() resolution logic so
# switching away from a model with overrides doesn't leak them.
cs = self._config_store
self.temperature = (
cfg.temperature
if cfg.temperature is not None
else (cs.get("model.temperature") if cs else self.temperature)
)
self.max_tokens = (
cfg.max_tokens
if cfg.max_tokens is not None
else (cs.get("model.max_tokens") if cs else self.max_tokens)
)
self.reasoning_effort = (
cfg.reasoning_effort
if cfg.reasoning_effort is not None
else (cs.get("model.reasoning_effort") if cs else self.reasoning_effort)
)
self._init_system_messages()
self._save_config()
self.ui.on_info(f"Switched to {cyan(arg)}: {model_name}")
+14 -29
View File
@@ -35,14 +35,6 @@ def _build_registry() -> dict[str, SettingDef]:
"""Build the settings registry from declarative definitions."""
defs: list[SettingDef] = [
# -- model ----------------------------------------------------------
SettingDef(
"model.name",
"str",
"",
"Default model name (empty = use provider default)",
"model",
help="Which AI model to use for conversations. Leave empty to use the provider's default.",
),
SettingDef(
"model.default_alias",
"str",
@@ -58,45 +50,38 @@ def _build_registry() -> dict[str, SettingDef]:
"model.temperature",
"float",
0.5,
"Sampling temperature (ignored by models that don't support it, e.g. o-series)",
"Default sampling temperature (overridden by per-model settings)",
"model",
min_value=0.0,
max_value=2.0,
help="Controls randomness in responses. Lower values (0.0\u20130.3) give focused, "
"deterministic output; higher values (0.7\u20131.5) make responses more creative and varied.",
help="Default sampling temperature for models without a per-model override. "
"Controls randomness in responses. Lower values (0.0\u20130.3) give focused, "
"deterministic output; higher values (0.7\u20131.5) make responses more creative "
"and varied. Per-model overrides can be set in the Models tab.",
reference_url="https://arxiv.org/abs/1904.09751",
),
SettingDef(
"model.max_tokens",
"int",
32768,
"Max output tokens per response",
"Default max output tokens (overridden by per-model settings)",
"model",
min_value=1,
help="Upper limit on how long each response can be. One token is roughly 4 characters "
"of English text. Higher values allow longer responses but cost more.",
help="Default max output tokens for models without a per-model override. "
"Upper limit on how long each response can be. One token is roughly 4 characters "
"of English text. Per-model overrides can be set in the Models tab.",
),
SettingDef(
"model.reasoning_effort",
"str",
"medium",
"Reasoning effort level (only applies to models with reasoning support)",
"Default reasoning effort (overridden by per-model settings)",
"model",
choices=["", "none", "minimal", "low", "medium", "high", "xhigh", "max"],
help="How much internal \u2018thinking\u2019 the model does before responding. Higher effort "
"improves quality on complex tasks but is slower and uses more tokens. Not all models "
"support this \u2014 it is silently ignored when unsupported.",
),
SettingDef(
"model.context_window",
"int",
0,
"Context window size in tokens (0 = auto-detect from model)",
"model",
min_value=0,
help="How much conversation history the model can see at once, measured in tokens "
"(~4 characters each). Set to 0 to auto-detect from the model. Only override this "
"if auto-detection fails (common with local models).",
help="Default reasoning effort for models without a per-model override. "
"Controls how much internal \u2018thinking\u2019 the model does before responding. "
"Higher effort improves quality on complex tasks but is slower and uses more "
"tokens. Per-model overrides can be set in the Models tab.",
),
# -- session --------------------------------------------------------
SettingDef(
+6
View File
@@ -3084,6 +3084,9 @@ class PostgreSQLBackend:
capabilities: str = "{}",
enabled: bool = True,
created_by: str = "",
temperature: float | None = None,
max_tokens: int | None = None,
reasoning_effort: str | None = None,
) -> None:
from sqlalchemy.dialects import postgresql
@@ -3101,6 +3104,9 @@ class PostgreSQLBackend:
context_window=context_window,
capabilities=capabilities,
enabled=1 if enabled else 0,
temperature=temperature,
max_tokens=max_tokens,
reasoning_effort=reasoning_effort,
created_by=created_by,
created=now,
updated=now,
+3
View File
@@ -1070,6 +1070,9 @@ class StorageBackend(Protocol):
capabilities: str = "{}",
enabled: bool = True,
created_by: str = "",
temperature: float | None = None,
max_tokens: int | None = None,
reasoning_effort: str | None = None,
) -> None:
"""Create a model definition. No-op if definition_id already exists."""
...
+3
View File
@@ -587,6 +587,9 @@ model_definitions = sa.Table(
sa.Column("context_window", sa.Integer, nullable=False, server_default="32768"),
sa.Column("capabilities", sa.Text, nullable=False, server_default="{}"),
sa.Column("enabled", sa.Integer, nullable=False, server_default="1"),
sa.Column("temperature", sa.Float, nullable=True),
sa.Column("max_tokens", sa.Integer, nullable=True),
sa.Column("reasoning_effort", sa.Text, nullable=True),
sa.Column("created_by", sa.Text, nullable=False, server_default=""),
sa.Column("created", sa.Text, nullable=False),
sa.Column("updated", sa.Text, nullable=False),
+6
View File
@@ -3147,6 +3147,9 @@ class SQLiteBackend:
capabilities: str = "{}",
enabled: bool = True,
created_by: str = "",
temperature: float | None = None,
max_tokens: int | None = None,
reasoning_effort: str | None = None,
) -> None:
now = datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%S")
@@ -3163,6 +3166,9 @@ class SQLiteBackend:
"context_window": context_window,
"capabilities": capabilities,
"enabled": 1 if enabled else 0,
"temperature": temperature,
"max_tokens": max_tokens,
"reasoning_effort": reasoning_effort,
"created_by": created_by,
"created": now,
"updated": now,
+3
View File
@@ -106,6 +106,9 @@ MODEL_DEFINITION_MUTABLE = frozenset(
"context_window",
"capabilities",
"enabled",
"temperature",
"max_tokens",
"reasoning_effort",
}
)
PROMPT_POLICY_MUTABLE = frozenset({"name", "content", "tool_gate", "priority", "enabled"})
@@ -0,0 +1,32 @@
"""Add per-model sampling parameters to model_definitions.
Adds nullable temperature, max_tokens, and reasoning_effort columns
so each model can override the global defaults. NULL means "inherit
the cluster-wide setting from system_settings".
Revision ID: 036
Revises: 035
Create Date: 2026-04-13
"""
import sqlalchemy as sa
from alembic import op
revision = "036"
down_revision = "035"
branch_labels = None
depends_on = None
def upgrade() -> None:
with op.batch_alter_table("model_definitions") as batch:
batch.add_column(sa.Column("temperature", sa.Float, nullable=True))
batch.add_column(sa.Column("max_tokens", sa.Integer, nullable=True))
batch.add_column(sa.Column("reasoning_effort", sa.Text, nullable=True))
def downgrade() -> None:
with op.batch_alter_table("model_definitions") as batch:
batch.drop_column("reasoning_effort")
batch.drop_column("max_tokens")
batch.drop_column("temperature")
+7 -7
View File
@@ -9,7 +9,7 @@
# Infra only: docker compose up
# Single node: docker compose --profile production up
# Production (PG): docker compose --profile production up
# (set DB_BACKEND, DATABASE_URL, POSTGRES_PASSWORD in .env)
# (set TURNSTONE_DB_BACKEND, TURNSTONE_DB_URL, POSTGRES_PASSWORD in .env)
#
# Set TURNSTONE_IMAGE_TAG in .env to pin the image version (default: latest).
# =============================================================================
@@ -94,8 +94,8 @@ services:
- TURNSTONE_JWT_SECRET=${TURNSTONE_JWT_SECRET:?Set TURNSTONE_JWT_SECRET in .env}
- MODEL=${MODEL:-}
- MCP_CONFIG=${MCP_CONFIG:-}
- TURNSTONE_DB_BACKEND=${DB_BACKEND:-sqlite}
- TURNSTONE_DB_URL=${DATABASE_URL:-}
- TURNSTONE_DB_BACKEND=${TURNSTONE_DB_BACKEND:-sqlite}
- TURNSTONE_DB_URL=${TURNSTONE_DB_URL:-}
- TURNSTONE_NODE_ID=${TURNSTONE_NODE_ID:-}
- TURNSTONE_ADVERTISE_URL=${TURNSTONE_ADVERTISE_URL:-http://server:8080}
extra_hosts:
@@ -128,8 +128,8 @@ services:
environment:
# Generate with: python -c "import secrets; print(secrets.token_hex(32))"
- TURNSTONE_JWT_SECRET=${TURNSTONE_JWT_SECRET:?Set TURNSTONE_JWT_SECRET in .env}
- TURNSTONE_DB_BACKEND=${DB_BACKEND:-sqlite}
- TURNSTONE_DB_URL=${DATABASE_URL:-}
- TURNSTONE_DB_BACKEND=${TURNSTONE_DB_BACKEND:-sqlite}
- TURNSTONE_DB_URL=${TURNSTONE_DB_URL:-}
- TURNSTONE_CONSOLE_URL=http://console:8090
networks:
- turnstone-net
@@ -161,8 +161,8 @@ services:
- TURNSTONE_DISCORD_GUILD=${TURNSTONE_DISCORD_GUILD:-0}
# Generate with: python -c "import secrets; print(secrets.token_hex(32))"
- TURNSTONE_JWT_SECRET=${TURNSTONE_JWT_SECRET:?Set TURNSTONE_JWT_SECRET in .env}
- TURNSTONE_DB_BACKEND=${DB_BACKEND:-postgresql}
- TURNSTONE_DB_URL=${DATABASE_URL:-postgresql+psycopg://${POSTGRES_USER:-turnstone}:${POSTGRES_PASSWORD:-turnstone}@postgres:5432/turnstone}
- TURNSTONE_DB_BACKEND=${TURNSTONE_DB_BACKEND:-postgresql}
- TURNSTONE_DB_URL=${TURNSTONE_DB_URL:-postgresql+psycopg://${POSTGRES_USER:-turnstone}:${POSTGRES_PASSWORD:-turnstone}@postgres:5432/turnstone}
- TURNSTONE_CHANNEL_ADVERTISE_URL=http://channel:8091
networks:
- turnstone-net
+35 -16
View File
@@ -2774,6 +2774,9 @@ def internal_model_status(request: Request) -> JSONResponse:
"source": cfg.source,
"context_window": cfg.context_window,
"enabled": True,
"temperature": cfg.temperature,
"max_tokens": cfg.max_tokens,
"reasoning_effort": cfg.reasoning_effort,
}
return JSONResponse({"models": models})
@@ -3406,9 +3409,8 @@ def main() -> None:
client = create_client(provider_name, base_url=base_url, api_key=api_key)
cs_model = config_store.get("model.name")
cli_model = args.model
effective_model = cli_model or cs_model or None
effective_model = cli_model or None
if effective_model:
model = effective_model
detected_ctx = None
@@ -3422,13 +3424,10 @@ def main() -> None:
# entry and relies on DB / config.toml models instead.
model = ""
# Use detected context window, fall back to ConfigStore override or 32768
cfg_ctx = config_store.get("model.context_window")
# Use detected context window, fall back to 32768
if detected_ctx:
context_window = detected_ctx
log.info("Context window: %s (detected from backend)", f"{context_window:,}")
elif cfg_ctx: # 0 = auto-detect (no override)
context_window = cfg_ctx
else:
context_window = 32768
@@ -3595,15 +3594,32 @@ def main() -> None:
except Exception as e:
log.warning("Failed to resolve judge_model %r: %s", judge_model, e)
# Per-model sampling overrides take priority over global defaults
eff_temperature = (
r_cfg.temperature
if r_cfg.temperature is not None
else config_store.get("model.temperature")
)
eff_max_tokens = (
r_cfg.max_tokens
if r_cfg.max_tokens is not None
else config_store.get("model.max_tokens")
)
eff_reasoning_effort = (
r_cfg.reasoning_effort
if r_cfg.reasoning_effort is not None
else config_store.get("model.reasoning_effort")
)
return ChatSession(
client=r_client,
model=r_model,
ui=ui,
instructions=config_store.get("session.instructions") or None,
temperature=config_store.get("model.temperature"),
max_tokens=config_store.get("model.max_tokens"),
temperature=eff_temperature,
max_tokens=eff_max_tokens,
tool_timeout=config_store.get("tools.timeout"),
reasoning_effort=config_store.get("model.reasoning_effort"),
reasoning_effort=eff_reasoning_effort,
context_window=r_cfg.context_window,
compact_max_tokens=config_store.get("session.compact_max_tokens"),
auto_compact_pct=config_store.get("session.auto_compact_pct"),
@@ -3715,15 +3731,15 @@ def main() -> None:
cors_origins = parse_cors_origins()
# Construct advertise URL for service registration.
# In Docker/k8s, socket.gethostname() returns the container ID which
# isn't DNS-resolvable by other containers. Priority:
# 1. TURNSTONE_ADVERTISE_URL env var (explicit override)
# Construct advertise URL for service registration. Priority:
# 1. TURNSTONE_ADVERTISE_URL env var (required in Docker/k8s where
# gethostname() returns a container ID that peers can't resolve)
# 2. Explicit --host (not a wildcard bind address)
# 3. socket.getfqdn() (may work in k8s with proper DNS)
# 3. socket.gethostname() (bare-metal fallback; getfqdn() does
# reverse DNS which often truncates the hostname)
_advertise_url = os.environ.get("TURNSTONE_ADVERTISE_URL", "")
if not _advertise_url:
_advertise_host = args.host if args.host not in ("0.0.0.0", "::") else socket.getfqdn()
_advertise_host = args.host if args.host not in ("0.0.0.0", "::") else socket.gethostname()
_advertise_url = f"http://{_advertise_host}:{args.port}"
_skip_perms = config_store.get("tools.skip_permissions")
@@ -3793,8 +3809,11 @@ def main() -> None:
from turnstone.core.tls import TLSClient
hostname = socket.getfqdn()
hostname = socket.gethostname()
fqdn = socket.getfqdn()
hostnames = [hostname, "localhost", "127.0.0.1"]
if fqdn != hostname:
hostnames.append(fqdn)
# Only add bind host if it's a concrete address
if args.host not in ("0.0.0.0", "::", ""):
hostnames.append(args.host)
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+1 -1
View File
@@ -4504,7 +4504,7 @@ function _loadHls(callback) {
if (_hlsState === "loading") return;
_hlsState = "loading";
var script = document.createElement("script");
script.src = "/shared/hls-1.6.15/hls.min.js";
script.src = "/shared/hls-1.6.16/hls.min.js";
script.onload = function () {
_hlsState = "ready";
var q = _hlsQueue;
Generated
+1 -1
View File
@@ -2506,7 +2506,7 @@ wheels = [
[[package]]
name = "turnstone"
version = "1.3.0a3"
version = "1.4.0a2"
source = { editable = "." }
dependencies = [
{ name = "alembic" },