Adds a per-alias `auth_mode` on model definitions so a model backend can authenticate to an Entra-fronted gateway with a per-request minted token instead of one shared static API key, letting the gateway attribute calls to the actual user or to the app as a machine identity. - `static` (default, unchanged) sends the stored `api_key`. - `entra_obo` mints a per-user On-Behalf-Of token for `obo_audience` from the caller's captured refresh credential. - `entra_app` mints an app-identity token via the client-credentials grant, and covers userless turns that OBO cannot. Reuses the existing OBO grant legs, refresh-token rotation CAS, cluster advisory lock and the `mcp_user_tokens` mint-cache, keyed under synthetic `__model_obo__:<audience>` / `__model_app__:<audience>` rows. The token binds at the call site through `client.with_options(api_key=...)` so each SDK emits it on its own auth path rather than through header injection. Migration 068 adds `auth_mode` and `obo_audience`. Both are additive and existing rows default to `static`, so behaviour is unchanged unless an alias opts in. Operator controls: `model.auth_audience_allowlist` is an exact-match allow-list that gates which audiences may be configured and denies all by default, and changing a mode or audience requires `admin.mcp`. `model.auth_fail_closed` decides whether a failed mint may fall back to an explicitly configured static key. A delegated call with no user, or a dynamic alias with no real static key, always refuses. Two changes here apply regardless of whether any alias opts in: - Storage and app state are now wired into the console MCP client manager. This fixes per-user `oauth_user` / `oauth_obo` dispatch for coordinator-hosted sessions, which previously raised `RuntimeError` on first call because `set_app_state` was only ever called on the node. - Unattended watch restores and `--resume` resolve the persisted workstream owner instead of constructing the session under an empty principal. A workstream with no owner is now a permanent refusal rather than an anonymous, auto-approved run.
16 KiB
System Settings
See also: Settings Architecture diagram
The system settings feature provides database-backed configuration for server
nodes. Settings are stored in the system_settings table and managed through
the admin API or console Settings tab. This replaces config.toml for
non-bootstrap settings on server entry points, while the CLI continues to read
config.toml directly.
Overview
Settings follow a typed registry pattern: every storable setting has a
SettingDef entry in settings_registry.py with type, default, description,
validation constraints, and a restart_required flag. Unknown keys are rejected
at the API boundary.
At runtime, ConfigStore loads all settings from storage into an in-memory
cache. Reads are lock-free dict lookups on an immutable snapshot. Writes acquire
a lock, persist to storage, and swap the cache atomically.
Precedence
Settings resolution differs between entry points:
| Entry point | Chain |
|---|---|
Server (turnstone-server) |
CLI flag > ConfigStore > registry default |
CLI (turnstone) |
CLI flag > config.toml > argparse default |
The server's apply_config() ignores config.toml sections that overlap with
ConfigStore. A startup warning is logged for each overlapping key, directing
users to the admin Settings API.
Per-Model Sampling Overrides
The global model.temperature, model.max_tokens, and model.reasoning_effort
settings serve as cluster-wide defaults. Individual models can override these
via per-model settings in the model_definitions table (admin Models tab).
Resolution order for sampling parameters:
| Priority | Source |
|---|---|
| 1 (highest) | Per-model override (set in Models tab) |
| 2 | Global default (set in Settings tab) |
| 3 | Registry default (code) |
When a per-model override is NULL (empty in the UI), the global default is
used. Switching models via /model <alias> re-resolves sampling parameters
from the new model's overrides or global defaults.
Model backend authentication
Model definitions support three backend credential modes:
auth_mode |
Identity sent to the model gateway |
|---|---|
static |
The definition's stored api_key. |
entra_obo |
A caller-delegated Entra access token minted from that user's captured OIDC credential. |
entra_app |
A shared app-identity token minted with Turnstone's OIDC client credentials. |
Dynamic modes require an exact obo_audience resource App ID URI. Before an
admin can save one, an operator must add that literal audience to
model.auth_audience_allowlist (comma- or newline-separated). Wildcards and
base-URL host matching are intentionally unsupported. Changing dynamic auth,
its audience, or the gateway base_url also requires admin.mcp; service
tokens do not bypass this capability-escalation gate.
entra_app is supported only with [oidc] obo_grant_profile = "entra".
Judge, output-guard, perception, utility, and sub-agent lanes inherit the
session's effective user for entra_obo. The perception memo is partitioned by
that principal as well as alias and content hash, so a result authorized as one
user cannot be served to another. Scheduled and wake-driven work retains the
workstream owner even when no user is connected. Eval and optimizer lanes are
registry-less development tools and therefore do not use dynamic model
authentication.
entra_app is an explicit model-definition choice; Turnstone never changes a
failed or ownerless entra_obo call into a client-credentials grant. An
entra_obo call with no effective user always refuses. A dynamic alias without
a real static key also always refuses instead of issuing its SDK-construction
placeholder. When a real static key is explicitly configured, mint failures
may use it by default; set model.auth_fail_closed = true to prohibit even that
fallback. A refusal is not routed through the model fallback chain.
Dynamic token caches are encrypted in mcp_user_tokens, shared across nodes,
and memoized on each host. Unlinking a user's OIDC identity purges their
entra_obo rows and memo entries. entra_app rows belong to the shared
__app__ identity and are not user-deprovisioned; after client-credential
revocation, an already-minted app bearer remains usable until its recorded
expiry.
obo_audience is literal and capped at 2048 characters. Environment-variable
expansion is deliberately not applied, so the allow-list decision cannot vary
by node or expand beyond the persisted boundary.
Responses output controls (per-model)
Models whose capability table declares Responses output controls expose two additional fields in the Models create/edit shelf:
| Field | Stored capability | Values | Effect |
|---|---|---|---|
| Output verbosity | verbosity |
low, medium, high |
Controls answer length independently of reasoning effort. |
| Reasoning mode | reasoning_mode |
standard, pro |
Selects standard or higher-compute Pro execution without changing the model ID. |
An empty selection means provider default and omits the capability key. Known
GPT-5.6 models inherit support from the built-in table without persisting
redundant support flags. An OpenAI-compatible model pinned to the Responses API
can opt in with the supports_verbosity and supports_pro_mode capability
tiles. Chat Completions and non-Responses providers do not surface or submit
these controls.
Removed settings: model.name and model.context_window have been removed
from ConfigStore. Model names and context windows are now configured per-model
in the Models tab. A startup warning is logged if these keys appear in
config.toml.
Reasoning persistence (per-model)
Two boolean flags on model_definitions (migration 052) control how
reasoning text round-trips per model:
| Flag | Default | Effect |
|---|---|---|
surface_persisted_reasoning |
True |
Surface stored reasoning text on /history payloads so a page reload re-renders the reasoning bubble. Storage of reasoning bytes is independent of this flag — they ride in provider_data regardless. |
replay_reasoning_to_model |
False |
Send stored reasoning blocks back to the provider on subsequent turns. Capability-gated: only takes effect when the model's ModelCapabilities.supports_reasoning_replay is also True. Set on canonical OpenAI gpt-5*/o-series and Anthropic Claude entries; unknown / local-server models default to False so an operator who flips the flag on a model whose API doesn't understand reasoning replay silently no-ops rather than 400-ing. |
Edit both via the admin Models tab. See the architecture doc for the
provider-side mechanics (Anthropic thinking, OpenAI Responses
reasoning + include=["reasoning.encrypted_content"], synthetic
reasoning_text for Chat Completions / vLLM / llama.cpp / Gemini-compat).
Task agent overrides
task_agent sub-sessions resolve independently from the conversation model
so operators can pick a cheaper/faster model for autonomous loops:
| Setting | Purpose |
|---|---|
model.task_alias |
Alias used for task_agent sub-sessions. Falls back to [model].agent_model in config.toml, then the session's active model. |
model.task_effort |
Reasoning effort for task_agent. Empty string means "inherit from the session". |
Both are live-editable from the Settings tab and take effect on the next sub-agent invocation — no restart required.
Bootstrap vs ConfigStore
Bootstrap settings are required before storage is available (database
connection, Redis, auth secrets, server bind address). These stay in
config.toml and environment variables.
| Category | Section | Where |
|---|---|---|
| API credentials | [api] |
config.toml / env |
| Database | [database] |
config.toml / env |
| Auth | [auth] |
config.toml / env |
| Console bind | [console] |
config.toml / env |
ConfigStore settings are loaded from the database after storage initialization:
| Section | Settings |
|---|---|
model |
default_alias, auth_audience_allowlist, auth_fail_closed, temperature, max_tokens, reasoning_effort, task_alias, task_effort |
session |
instructions, retention_days, compact_max_tokens, auto_compact_pct |
tools |
timeout, truncation, agent_max_turns, skip_permissions, search, search_threshold, search_max_results |
server |
workstream_idle_timeout, max_workstreams |
cluster |
node_fan_out_limit, mcp_max_servers |
mcp |
config_path, registry_url |
ratelimit |
enabled, requests_per_second, burst, trusted_proxies |
health |
backend_probe_interval, backend_probe_timeout, circuit_breaker_threshold, circuit_breaker_cooldown |
judge |
enabled, model, provider, base_url, api_key, confidence_threshold, max_context_ratio, timeout, read_only_tools, output_guard, redact_secrets, cancel_on_approval |
interface |
close_tab_action, theme |
skills |
discovery_url |
memory |
relevance_k, fetch_limit, max_content, nudge_cooldown, nudges |
Settings are addressed by dotted key (e.g. memory.relevance_k). Each has a
declared type (int, float, str, bool), optional min_value/max_value
range, optional choices list, and an is_secret flag.
Storage
The system_settings table (migration 015) stores settings as JSON-encoded
values with a composite primary key of (key, node_id):
| Column | Type | Description |
|---|---|---|
key |
text | Dotted setting key (e.g. model.temperature) |
value |
text | JSON-encoded value |
node_id |
text | Node ID for per-node overrides (empty string = global) |
is_secret |
int | 1 if the setting contains secrets |
changed_by |
text | Username of last editor |
created |
text | ISO timestamp |
updated |
text | ISO timestamp |
Per-node overrides layer on top of global settings. When ConfigStore loads,
it fetches global settings first, then overlays per-node values.
Admin API
Four endpoints on the console server, all requiring the admin.settings
permission.
GET /v1/api/admin/settings
List all settings with their effective values, defaults, and metadata.
Response: 200
{
"settings": [
{
"key": "model.temperature",
"value": 0.7,
"source": "storage",
"type": "float",
"description": "Sampling temperature",
"section": "model",
"is_secret": false,
"node_id": "",
"changed_by": "admin",
"updated": "2026-03-14T10:00:00",
"restart_required": false
}
]
}
GET /v1/api/admin/settings/schema
Return the full registry catalog (all defined settings with metadata). Useful for building dynamic admin UIs.
Response: 200
{
"schema": [
{
"key": "model.temperature",
"type": "float",
"default": 0.5,
"description": "Sampling temperature",
"section": "model",
"is_secret": false,
"min_value": 0.0,
"max_value": 2.0,
"choices": null,
"restart_required": false
}
]
}
PUT /v1/api/admin/settings/{key}
Update a setting. The value is validated against the registry (type coercion,
range, choices). Secret settings (is_secret=true) cannot be written via the
API -- they must be configured via config.toml or environment variables.
Path parameters:
| Parameter | Type | Description |
|---|---|---|
key |
string | Dotted setting key (e.g. model.temperature) |
Request body:
{
"value": 0.7,
"node_id": ""
}
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
value |
any | yes | -- | New value (type-coerced against registry) |
node_id |
string | no | "" |
Node ID for per-node override |
Response (success): 200
{
"key": "model.temperature",
"value": 0.7,
"source": "storage",
"type": "float",
"description": "Sampling temperature",
"section": "model",
"is_secret": false,
"node_id": "",
"changed_by": "admin",
"updated": "",
"restart_required": false
}
Errors:
| Status | Condition |
|---|---|
| 400 | Unknown key, invalid value, type mismatch, out of range |
| 403 | Secret setting (must use config.toml or env) |
DELETE /v1/api/admin/settings/{key}
Reset a setting to its registry default by removing it from storage.
Path parameters:
| Parameter | Type | Description |
|---|---|---|
key |
string | Dotted setting key |
Query parameters:
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
node_id |
string | no | "" |
Node ID (empty = global) |
Response (success): 200
{"status": "ok", "key": "model.temperature", "default": 0.5}
Response (not found): 404
{"error": "Setting 'model.temperature' has no stored value"}
Secret Settings
Settings with is_secret=True (currently only judge.api_key) are blocked
from the write API with a 403 response. This prevents accidental exposure
through the admin UI or audit logs. Secret settings must be configured via
config.toml or environment variables.
The list endpoint masks secret values: stored secrets appear as "***"
rather than their actual value.
Hot Reload
ConfigStore caches all settings in memory for fast, lock-free reads. To
refresh the cache after external changes (e.g. direct database edits or
cluster-wide propagation):
POST /v1/api/_internal/config-reload
This triggers ConfigStore.reload(), which re-reads all settings from storage
and atomically swaps the cache. The version counter increments on every
reload.
Behavior after reload:
- New workstreams pick up updated values immediately (via
session_factory) - Existing sessions keep their frozen configuration (settings are captured at workstream creation time, not read on every turn)
- Settings marked
restart_required=Trueneed a server restart to take effect
Migration from config.toml
On startup, warn_migrated_settings() scans config.toml for keys that are
now managed by ConfigStore. Each overlap produces a warning:
WARNING config.toml [model] temperature is now managed via Settings API —
this value will be ignored. Use the admin Settings tab or
PUT /v1/api/admin/settings/model.temperature to configure.
To migrate:
- Note the values from
config.tomlfor sections that overlap with ConfigStore - Use
PUT /v1/api/admin/settings/{key}or the console Settings tab to set each value - Remove the migrated sections from
config.toml - Restart the server to verify no warnings
SDK
Python
from turnstone.sdk import TurnstoneConsole
with TurnstoneConsole("http://localhost:9090", token="tok_xxx") as admin:
# List all settings with effective values
result = admin.list_settings()
for s in result["settings"]:
print(f"{s['key']} = {s['value']} (source: {s['source']})")
# Get the schema catalog
schema = admin.get_settings_schema()
# Update a setting
admin.update_setting("model.temperature", value=0.7)
# Update with per-node override
admin.update_setting("model.temperature", value=0.3, node_id="node-2")
# Reset to default
admin.delete_setting("model.temperature")
TypeScript
import { TurnstoneConsole } from "@turnstone/sdk";
const admin = new TurnstoneConsole({
baseUrl: "http://localhost:9090",
token: "tok_xxx",
});
// List all settings
const result = await admin.listSettings();
for (const s of result.settings) {
console.log(`${s.key} = ${s.value} (source: ${s.source})`);
}
// Get schema catalog
const schema = await admin.getSettingsSchema();
// Update a setting
await admin.updateSetting("model.temperature", { value: 0.7 });
// Reset to default
await admin.deleteSetting("model.temperature");
Architecture
See Settings Architecture diagram for the full data flow covering server startup, admin API writes, hot reload, and settings precedence.