* fix: enforce the image generation flag on the legacy chat feature path
* fix: refresh the config store when image generation is disabled in admin settings
* fix: hide active feature pills when the feature is no longer available
Revoking a user's `features.memories` permission removed their access to the memories API and to the native function-calling memory tools, but their stored memories were still injected into the system context on the legacy function-calling path.
The branch in `process_chat_payload` only checked the client-supplied `features['memory']` flag plus the global `memories.system_context.enable` switch, with no user-permission check. `add_memory_context` did not compensate: it only checks `model_allows_memory`, which is a model capability rather than a permission, and the one call inside it that does check the permission (`query_memory`) has its 403 swallowed by a `try/except`, so `Memories.get_memories_by_user_id` and the neighbourhood scan still fed the system prompt.
Gate the branch with the same permission check the native path already performs in `get_builtin_tools`, matching the neighbouring `web_search` and `image_generation` branches.
Only the caller's own memories were injected into the caller's own context, so there was no cross-user exposure. The practical effect was that the permission toggle did not do what its name implies: an admin who revoked it still got memory content injected for that user.
`get_message_list` moves through `messages_map` by key but tracked each message's own `id` field, which the message body does not have to carry. Track the key instead.
With ENABLE_OTEL and ENABLE_OTEL_LOGS set, InterceptHandler builds the message once for loguru and then hands the same LogRecord to the OpenTelemetry handler, whose _translate calls record.getMessage() a second time. That used to be free, because the message was already a finished f-string with nothing to substitute. Now that log calls pass lazy %-args, the second call re-runs the whole interpolation, so every exported record is formatted twice.
The two getMessage() calls on a 78 kB retrieval record:
before 373.0 us
after 0.1 us
Stamping the built message back onto the record makes the second call a plain string return. msg and args are both in OpenTelemetry's _RESERVED_ATTRS, so neither ever reaches the exported attributes. The isinstance guard matters: _translate exports a non-str msg such as the dicts routers/audio.py logs as a typed body rather than a string, so those records are left untouched, and they have no %-args to format twice anyway. Body, attributes and severity were compared against LoggingHandler._translate for str, dict, list, int, None, exception and exc_info records.
Logging in through CyberArk Identity dies at the callback with "Unsupported {'app_id'} in header" and the user sees "The email or password provided is incorrect". Any provider that puts a vendor-specific parameter in the ID token header hits this; CAS was already patched by name, CyberArk is the next one.
Authlib 1.7 verifies ID tokens with joserfc, which rejects header parameters it does not recognise. The old fix registered `client_id` so CAS would work, which only ever fixes one provider at a time. This turns off the unknown-header rejection instead, so any private header parameter is ignored rather than fatal. Signature verification, the algorithm allowlist, `crit` handling and value validation of registered headers all still run, so nothing that actually protects the token is relaxed.
Fixes#28062
Chat creation and chat moves each carried their own copy of the same folder_id validation, resolving the folder and checking ownership and shared write access in slightly different ways. Both now call a single has_folder_write_access helper, which the chat-completions creation path uses as well, so ownership, inherited write grants and nonexistent or malformed ids behave identically everywhere a chat folder_id is set. The owner case also costs one query fewer than before.
The Responses API handler had a branch for response.output_item.done whose own comment said it was handled specifically below, but it never ran. The generic branch matching any response.*.done event came first in the chain and matched this event too, so it fell through and returned the accumulated output unchanged, leaving the dedicated branch below unreachable since the feature was added.
Moving the dedicated branch above the generic one makes the event apply. On a compliant stream this changes nothing, since response.completed replaces the whole output with the same data straight afterwards. It matters when a provider is less tidy: one that never sends response.content_part.added leaves the assistant's own reply unextractable from the next turn's context, and one that omits response.content_part.done drops the annotations that only arrive with the finished item. Both are repaired by honouring the event.
Worth knowing: the item replaces whatever the deltas accumulated, with no guard against a provider sending back less than it streamed. A reasoning item arriving without its content would therefore lose the reasoning body, which is the same shape of provider brokenness that #27800 already needed a guard for.
The generic response.*.delta branch could leave the streaming handler in two states that crash the caller. It bound its result only inside the guard that checks the target item exists, but returned that result outside the guard, so a delta arriving before its output item, or carrying an index past the end, raised UnboundLocalError. Separately, an event name with only two dot-separated parts failed the length check and fell off the end of the branch, so the function returned None and both call sites raised TypeError unpacking it.
Where the response is streamed to a browser both crashes were swallowed at debug level and cost a chunk. On the direct API path there is no handler between here and the server, so the caller kept its 200 while the body was cut short with no [DONE], and the outlet filters never ran.
The return now sits inside the guard with a branch-level fallback that hands back the accumulated output untouched, which is what the sibling done branch and every other skip path in this function already do. Deltas whose item exists behave exactly as before.
Dropping an orphan delta is deliberate rather than synthesizing the missing item: response.output_item.added appends without regard to output_index, so a placeholder would be duplicated when the real item arrives, and a fabricated function_call would have no name or call id.
Raising GLOBAL_LOG_LEVEL to WARNING buys quieter output but not less work: 241 INFO call sites interpolate their payload into an f-string before the logging call gets to drop it. The heaviest is get_doc, which logs every chunk id and metadata dict in a collection, so on the full-context retrieval path that is the entire knowledge base, once per chat request.
That one line at WARNING, CPython 3.12:
| knowledge base | payload | before | after |
| -------------- | ------- | -------- | ------- |
| top-k of 3 | 1.2 kB | 3.8 us | 0.07 us |
| 500 chunks | 201 kB | 583.6 us | 0.08 us |
| 5000 chunks | 2.0 MB | 5.8 ms | 0.15 us |
The lazy form log.info('query_doc:result %s %s', result.ids, result.metadatas) hands the payload to record.getMessage(), which the InterceptHandler only reaches once a record has passed the level check. Output at INFO is byte-identical. Two sites that already built their message eagerly, one str concat and one % operator, move to the same lazy form.
The `response.completed` handler replaced the accumulated output with the terminal event's `output` whenever that key was present, guarded only by `is not None`. An empty array satisfies that guard, so a provider that finishes the stream with `"output": []` wiped everything collected from `response.output_item.added`, `response.output_text.delta` and `response.output_item.done`.
The assistant message was then persisted with `output: []` and empty content, which shows up as a reply that renders correctly while streaming and disappears the moment the stream ends.
Fall back to the accumulated output when the terminal array is empty. A spec-compliant `response.completed` still wins, since a populated array is truthy, and when nothing was streamed the accumulated output is empty too, so the fallback cannot invent content.
Fixes#27789
GLOBAL_LOG_LEVEL defaults to INFO, so every log.debug(...) in the backend is discarded, but the message is built first: 187 call sites interpolate their payload into an f-string before the logging call runs, so the work happens on every request and the result is thrown away. The worst one sits in process_chat_payload and stringifies the whole request body, full conversation history included, once per chat completion.
That one line with DEBUG disabled, CPython 3.12:
| conversation | payload | before | after |
| ------------ | ------- | -------- | ------- |
| 4 messages | 1.2 kB | 3.4 us | 0.07 us |
| 20 messages | 17 kB | 24.8 us | 0.07 us |
| 60 messages | 123 kB | 216.6 us | 0.07 us |
The lazy form log.debug('form_data: %s', form_data) hands the payload to record.getMessage(), which the InterceptHandler only reaches once a record has passed the level check. With DEBUG enabled the emitted lines are byte-identical, f'{x=}' sites included: those map to %r. MistralLoader._debug_log callers get the same treatment, since that wrapper already forwards *args.
`get_permissions` deep-copies the default permission tree with a `json.loads(json.dumps(...))` round trip before merging group permissions into it. It runs on signin, signup, the permissions endpoint, OAuth, and the chat-completion middleware.
It now goes through `JSONCodec`, which selects orjson when `ENABLE_ORJSON` is set. The intermediate string never leaves the expression, so neither the escaping nor the separator differences between the two backends are observable; only the resulting object is used.
`default_permissions` always originates from `Config.get('user.permissions')`, a SQLAlchemy `JSON` column, so the tree is JSON-native by construction and the round trip is exact.
Note for anyone tempted to simplify this to `copy.deepcopy`: measured on the real `DEFAULT_USER_PERMISSIONS` shape over 200k iterations, `deepcopy` takes 3.51s against 1.45s for the stdlib round trip and 0.40s for orjson. The round trip is the fast option, not a workaround.
With `ENABLE_ORJSON` unset, which is the default, `JSONCodec` is stdlib `json` and this call site behaves exactly as before.
The code interpreter parses every message from the Jupyter kernel websocket with stdlib `json`, in a loop that runs for the duration of an execution. Messages carrying large stdout or a base64 image payload are the expensive ones.
It now goes through `JSONCodec`, which selects orjson when `ENABLE_ORJSON` is set. Every consumer of the parsed message reads strings only: `content.text`, `content.data['text/plain']` and `['image/png']`, `content.traceback`, and `content.execution_state`. Jupyter renders large integers into `text/plain` as strings rather than JSON numbers, so no numeric round trip is involved.
The one-shot `execute_request` message this module sends keeps stdlib `json`; it is a small fixed-shape dict sent once per execution.
With `ENABLE_ORJSON` unset, which is the default, `JSONCodec` is stdlib `json` and this call site behaves exactly as before.
`request.app.state.MODELS` is a `RedisDict` when Redis is configured. Unpacking it with `{**pool}` makes Python call `keys()` and then `__getitem__` once per key, which is one HKEYS plus one HGET per model, issued sequentially through a synchronous client. At 200 models that is 201 blocking Redis round trips per call.
`RedisDict.items()` is a single HGETALL, so `dict(pool.items())` fetches the same data in one round trip. `utils/chat.py:184` already does exactly this and carries a comment explaining why; these ten call sites were missed.
They are on the direct-connection branch of the task endpoints (title, tags, follow-up, autocomplete, query generation and the rest), of `chat_completed`, and of context compaction, so they run for background tasks fired on ordinary chat turns.
Behaviour is unchanged. The merged mapping is identical, the explicitly added direct model still overrides any pool entry with the same id, and when Redis is not configured the pool is a plain dict where `dict(d.items())` and `{**d}` are equivalent.
It also closes a race. `RedisDict.set` writes with HSET and then HDELs the stale keys, so a key returned by HKEYS could be deleted before its HGET arrived, raising `KeyError` out of the dict literal and failing the request mid model refresh. The old path could likewise observe a mix of pre- and post-refresh entries. HGETALL is atomic, so the caller now always sees one coherent snapshot.
orjson emits U+2028, U+2029 and U+0085 raw, where stdlib `json.dumps` escapes them under its default `ensure_ascii=True`. Python treats all three as line boundaries, so with `ENABLE_ORJSON` set, one of them inside model output splits a `data: {...}` SSE frame in half. Both halves then fail to parse and the delta is dropped with no error.
`utils/middleware.py` reassembles frames with `splitlines()`, so an affected response silently loses content on the direct API path. External clients are exposed as well: httpx's `LineDecoder` reimplements the same line-boundary semantics, so any SDK reading the OpenAI-compatible stream through `aiter_lines` breaks on a raw separator.
The three characters are escaped on the way out of `ORJSONCodec.dumps`. That restores parity with stdlib and fixes every reader at once, rather than patching one consumer and leaving external clients broken. They are the complete set: of the ten code points `splitlines()` treats as boundaries, the other seven are below U+0020, where JSON already forces an escape.
The membership guard is load bearing. Calling `translate` unconditionally costs roughly 1.5 us on a typical SSE chunk against 0.115 us for the serialization it wraps, so it would spend more than orjson saves. The three scans cost about 0.04 us.
Payloads containing none of the three are returned unchanged, byte for byte. With `ENABLE_ORJSON` unset, which is the default, none of this code runs.
U+2028 and U+2029 are common in text extracted from PDFs and word processor documents, so the realistic trigger is a model quoting an uploaded file back to the user.
Swap the JSON encoder/decoder used across the backend from stdlib json to
orjson when ENABLE_ORJSON is set — HTTP request bodies, JSONResponse
bodies, upstream provider responses, SSE chunks, and socket.io/Redis
payloads.
The flag defaults to off, in which case the app uses stdlib json and
engineio's codec verbatim, so default behaviour is unchanged.
- json_codec exports JSONCodec (stdlib json or the orjson codec) and
SOCKETIO_JSON (engineio's codec or the orjson codec); call sites import
JSONCodec and stay implementation-agnostic
- apply_orjson_http_json() is a no-op when the flag is off, leaving
starlette's Request.json / JSONResponse.render untouched
- the orjson codec falls back to the stdlib for inputs orjson rejects
(non-str dict keys, ints beyond 64 bits, NaN literals)
- orjson is imported only when the flag is on
- FastAPI(default_response_class=...) is deliberately not used: an
explicit default disables the Pydantic direct-to-bytes fast path for
response_model routes
* fix: fetch terminal system prompt per request with TTL cache
The system prompt was only fetched once in set_terminal_servers (startup
or connection save) with a 3s timeout, using a synthetic 'system' user.
That snapshot silently stays empty when the fetch races a cold-started
orchestrator instance, and goes stale when instances are reprovisioned
with a changed OPEN_TERMINAL_SYSTEM_PROMPT — recovering only after a
restart or a manual connection re-save.
- Fetch /system during get_terminal_tools with the user's own
credentials via a central TTL-cached method (5 min per server+user;
failures cached 60s so a dead instance doesn't stall every request),
falling back to the cached snapshot.
- Raise the fetch timeout from 3s to 30s so cold-provisioned instances
can answer.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KjnQJNKozp47vTB13pRyYs
* refac: fetch the terminal system prompt per request without a cache
Drop the module-level TTL cache and fetch the system prompt directly in
get_terminal_tools, gathered with the existing uncached per-request cwd
fetch that already follows this pattern. The fetch uses the user's own
credentials and falls back to the set_terminal_servers snapshot, so a
cold or unreachable instance degrades to the previous behaviour instead
of needing an error cache. Also restore the 3s timeout: on the request
path a 30s wait would stall chat completions, and a cold instance is
covered by the snapshot fallback until it warms up.
---------
Co-authored-by: Claude <noreply@anthropic.com>
Timothy Jaeryang Baek