The Responses API handler had a branch for response.output_item.done whose own comment said it was handled specifically below, but it never ran. The generic branch matching any response.*.done event came first in the chain and matched this event too, so it fell through and returned the accumulated output unchanged, leaving the dedicated branch below unreachable since the feature was added.
Moving the dedicated branch above the generic one makes the event apply. On a compliant stream this changes nothing, since response.completed replaces the whole output with the same data straight afterwards. It matters when a provider is less tidy: one that never sends response.content_part.added leaves the assistant's own reply unextractable from the next turn's context, and one that omits response.content_part.done drops the annotations that only arrive with the finished item. Both are repaired by honouring the event.
Worth knowing: the item replaces whatever the deltas accumulated, with no guard against a provider sending back less than it streamed. A reasoning item arriving without its content would therefore lose the reasoning body, which is the same shape of provider brokenness that #27800 already needed a guard for.
An interactive prompt raised by __event_call__ was meant to come back as an error dictionary when it timed out. It never did: sio.call raises socketio.exceptions.TimeoutError, which does not inherit from the builtin TimeoutError the handler was catching, so the exception escaped into plugin code instead. Because that exception carries no message, the call sites that wrap plugin calls in except Exception as e turned it into an empty string, so a timed-out prompt looked like an empty answer rather than a failure, and the error branches written for it were dead.
The handler now catches socketio's class alongside the builtin, so a timeout returns the intended error dictionary and a plugin can tell the two apart.
The session eviction that sat inside that handler is removed rather than switched on. It had never executed, and it is wrong in both directions: it compares the pool entry by value, which the heartbeat rewrites every thirty seconds, so it would usually not fire, and when it did fire on a short timeout it would evict a live tab whose user had simply not answered yet, with nothing to restore the entry short of a reload. Genuinely dead sessions are already reaped on missed heartbeats by periodic_session_pool_cleanup.
WEBSOCKET_EVENT_CALLER_TIMEOUT is unset by default, which means no timeout at all, so this only affects deployments that set it.
The generic response.*.delta branch could leave the streaming handler in two states that crash the caller. It bound its result only inside the guard that checks the target item exists, but returned that result outside the guard, so a delta arriving before its output item, or carrying an index past the end, raised UnboundLocalError. Separately, an event name with only two dot-separated parts failed the length check and fell off the end of the branch, so the function returned None and both call sites raised TypeError unpacking it.
Where the response is streamed to a browser both crashes were swallowed at debug level and cost a chunk. On the direct API path there is no handler between here and the server, so the caller kept its 200 while the body was cut short with no [DONE], and the outlet filters never ran.
The return now sits inside the guard with a branch-level fallback that hands back the accumulated output untouched, which is what the sibling done branch and every other skip path in this function already do. Deltas whose item exists behave exactly as before.
Dropping an orphan delta is deliberate rather than synthesizing the missing item: response.output_item.added appends without regard to output_index, so a placeholder would be duplicated when the real item arrives, and a fabricated function_call would have no name or call id.
Raising GLOBAL_LOG_LEVEL to WARNING buys quieter output but not less work: 241 INFO call sites interpolate their payload into an f-string before the logging call gets to drop it. The heaviest is get_doc, which logs every chunk id and metadata dict in a collection, so on the full-context retrieval path that is the entire knowledge base, once per chat request.
That one line at WARNING, CPython 3.12:
| knowledge base | payload | before | after |
| -------------- | ------- | -------- | ------- |
| top-k of 3 | 1.2 kB | 3.8 us | 0.07 us |
| 500 chunks | 201 kB | 583.6 us | 0.08 us |
| 5000 chunks | 2.0 MB | 5.8 ms | 0.15 us |
The lazy form log.info('query_doc:result %s %s', result.ids, result.metadatas) hands the payload to record.getMessage(), which the InterceptHandler only reaches once a record has passed the level check. Output at INFO is byte-identical. Two sites that already built their message eagerly, one str concat and one % operator, move to the same lazy form.
`backend/open_webui/models/chats.py` imports `json`, but the module contains no `json.` references. The only remaining matches in the file are SQL function names such as `json_each` and `json_typeof` inside `text()` strings, which are unrelated to the import.
Noticed while profiling the chat read/write path: the import made it look as though the module serialized locally, when all of that happens in `utils/misc.py:sanitize_data_for_db`.
One-line deletion, no behaviour change.
The `response.completed` handler replaced the accumulated output with the terminal event's `output` whenever that key was present, guarded only by `is not None`. An empty array satisfies that guard, so a provider that finishes the stream with `"output": []` wiped everything collected from `response.output_item.added`, `response.output_text.delta` and `response.output_item.done`.
The assistant message was then persisted with `output: []` and empty content, which shows up as a reply that renders correctly while streaming and disappears the moment the stream ends.
Fall back to the accumulated output when the terminal array is empty. A spec-compliant `response.completed` still wins, since a populated array is truthy, and when nothing was streamed the accumulated output is empty too, so the fallback cannot invent content.
Fixes#27789
Permission checks are the most repeated database work in a request, and every one of them asks the same question: which groups is this user in. Today that question cannot use an index.
`group_member` has only its primary key and a `(group_id, user_id)` unique constraint. That constraint leads on `group_id`, so a lookup by `user_id` has to walk the entire membership table, every time. `Groups.get_groups_by_member_id` sits under `has_permission`, `has_access`, `check_model_access` and the `AccessGrants` fallbacks, so an ordinary chat completion pays that walk several times before the model is even called, and the admin user list pays it once per row.
The cost scales with total memberships across all users rather than with the size of any one user's, so it stays invisible on a small instance and then arrives all at once on a large one.
Measured on SQLite, timing the real join from `get_groups_by_member_id`:
| memberships | before | after |
|---|---|---|
| 5,000 | 0.04 ms | 0.03 ms |
| 50,000 | 0.10 ms | 0.04 ms |
| 200,000 | 1.33 ms | 0.04 ms |
| 500,000 | 2.94 ms | 0.04 ms |
The after column is flat because the lookup becomes a seek instead of a scan. Concretely: on a deployment with 500k memberships, say 10,000 users in 50 groups each, one chat completion currently spends roughly 15 ms of database time answering the same question over and over. Afterwards it is under 0.2 ms. On a small install you will not be able to measure the difference, and that is fine, the point is that the curve stops bending.
The index is `(user_id, group_id)`. The trailing column makes those lookups index-only, since `group_id` is the column they select. Queries that lead on `group_id`, such as `get_group_user_ids_by_id` and the `chat_messages` subqueries, are already served by the existing unique constraint and are unaffected.
What to expect when the migration runs: on PostgreSQL this is a plain `CREATE INDEX`, which takes a SHARE lock, so reads continue while writes to `group_member` block until it completes. The table holds one row per membership, so expect sub-second even on the numbers above. `CONCURRENTLY` cannot be used here because the migration runner wraps the upgrade in a transaction, and it is not warranted at this table size.
GLOBAL_LOG_LEVEL defaults to INFO, so every log.debug(...) in the backend is discarded, but the message is built first: 187 call sites interpolate their payload into an f-string before the logging call runs, so the work happens on every request and the result is thrown away. The worst one sits in process_chat_payload and stringifies the whole request body, full conversation history included, once per chat completion.
That one line with DEBUG disabled, CPython 3.12:
| conversation | payload | before | after |
| ------------ | ------- | -------- | ------- |
| 4 messages | 1.2 kB | 3.4 us | 0.07 us |
| 20 messages | 17 kB | 24.8 us | 0.07 us |
| 60 messages | 123 kB | 216.6 us | 0.07 us |
The lazy form log.debug('form_data: %s', form_data) hands the payload to record.getMessage(), which the InterceptHandler only reaches once a record has passed the level check. With DEBUG enabled the emitted lines are byte-identical, f'{x=}' sites included: those map to %r. MistralLoader._debug_log callers get the same treatment, since that wrapper already forwards *args.
`get_permissions` deep-copies the default permission tree with a `json.loads(json.dumps(...))` round trip before merging group permissions into it. It runs on signin, signup, the permissions endpoint, OAuth, and the chat-completion middleware.
It now goes through `JSONCodec`, which selects orjson when `ENABLE_ORJSON` is set. The intermediate string never leaves the expression, so neither the escaping nor the separator differences between the two backends are observable; only the resulting object is used.
`default_permissions` always originates from `Config.get('user.permissions')`, a SQLAlchemy `JSON` column, so the tree is JSON-native by construction and the round trip is exact.
Note for anyone tempted to simplify this to `copy.deepcopy`: measured on the real `DEFAULT_USER_PERMISSIONS` shape over 200k iterations, `deepcopy` takes 3.51s against 1.45s for the stdlib round trip and 0.40s for orjson. The round trip is the fast option, not a workaround.
With `ENABLE_ORJSON` unset, which is the default, `JSONCodec` is stdlib `json` and this call site behaves exactly as before.
The code interpreter parses every message from the Jupyter kernel websocket with stdlib `json`, in a loop that runs for the duration of an execution. Messages carrying large stdout or a base64 image payload are the expensive ones.
It now goes through `JSONCodec`, which selects orjson when `ENABLE_ORJSON` is set. Every consumer of the parsed message reads strings only: `content.text`, `content.data['text/plain']` and `['image/png']`, `content.traceback`, and `content.execution_state`. Jupyter renders large integers into `text/plain` as strings rather than JSON numbers, so no numeric round trip is involved.
The one-shot `execute_request` message this module sends keeps stdlib `json`; it is a small fixed-shape dict sent once per execution.
With `ENABLE_ORJSON` unset, which is the default, `JSONCodec` is stdlib `json` and this call site behaves exactly as before.
`_json_to_metadata` parses the metadata of every result row returned by search and get. It now goes through `JSONCodec`, which selects orjson when `ENABLE_ORJSON` is set.
The text it parses is produced by Oracle's own `JSON_SERIALIZE`, and the column is a native `JSON` type, so the database normalises whatever was written and the reader never depends on the writer's escaping.
The matching `_metadata_to_json` write deliberately keeps stdlib `json`: it passes `default=self._decimal_handler`, orjson accepts none of stdlib's keyword arguments, and dropping the handler would turn a currently successful insert of a `Decimal` into a hard failure. The read side has no such constraint.
With `ENABLE_ORJSON` unset, which is the default, `JSONCodec` is stdlib `json` and this call site behaves exactly as before.
`routers/ollama.py` serializes the outbound body with stdlib `json` on six inference paths: `/api/chat`, the OpenAI-compatible completions and chat completions proxies, embeddings, the Anthropic messages proxy, and responses. All six carry a full conversation or an embedding batch.
They now go through `JSONCodec`, which selects orjson when `ENABLE_ORJSON` is set. Every one is passed to `send_request`, which hands it to aiohttp as `data=`; aiohttp encodes `str` as UTF-8 and derives `Content-Length` from the encoded bytes. None is hashed, cached, length-measured or persisted.
Admin model management keeps stdlib: `/api/unload`, `/api/pull`, `/api/delete` and `/api/show` serialize fixed one- or two-key dicts, as do the blob download and upload progress events and the error frames. Codec dispatch on those costs about what it saves.
With `ENABLE_ORJSON` unset, which is the default, `JSONCodec` is stdlib `json` and this call site behaves exactly as before.
`passthrough_anthropic_messages` in `main.py` serializes the full request payload with stdlib `json` before sending it upstream. It is the largest single serialization on that path, since the body carries the whole conversation.
It now goes through `JSONCodec`, which selects orjson when `ENABLE_ORJSON` is set. The result is passed to aiohttp as `data=`, which encodes `str` as UTF-8 and sets `Content-Length` from the encoded bytes. The payload originates from a parsed request dict, so it holds only JSON-native types, and the serialized string is never hashed, compared or persisted.
The remaining stdlib `json` calls in this module are left alone: two are a debug log line and a fixed Ollama unload payload, and one parses an upstream error body.
With `ENABLE_ORJSON` unset, which is the default, `JSONCodec` is stdlib `json` and this call site behaves exactly as before.
`request.app.state.MODELS` is a `RedisDict` when Redis is configured. Unpacking it with `{**pool}` makes Python call `keys()` and then `__getitem__` once per key, which is one HKEYS plus one HGET per model, issued sequentially through a synchronous client. At 200 models that is 201 blocking Redis round trips per call.
`RedisDict.items()` is a single HGETALL, so `dict(pool.items())` fetches the same data in one round trip. `utils/chat.py:184` already does exactly this and carries a comment explaining why; these ten call sites were missed.
They are on the direct-connection branch of the task endpoints (title, tags, follow-up, autocomplete, query generation and the rest), of `chat_completed`, and of context compaction, so they run for background tasks fired on ordinary chat turns.
Behaviour is unchanged. The merged mapping is identical, the explicitly added direct model still overrides any pool entry with the same id, and when Redis is not configured the pool is a plain dict where `dict(d.items())` and `{**d}` are equivalent.
It also closes a race. `RedisDict.set` writes with HSET and then HDELs the stale keys, so a key returned by HKEYS could be deleted before its HGET arrived, raising `KeyError` out of the dict literal and failing the request mid model refresh. The old path could likewise observe a mix of pre- and post-refresh entries. HGETALL is atomic, so the caller now always sees one coherent snapshot.
Fourteen modules import `json` without using it. Ruff flags every one with F401, and a word-boundary search for `json` in each file matches only the import line itself, including inside strings, comments and annotations.
Two exclusions, both deliberate. Migration files are left alone: the import is equally dead there, but those files are frozen history and not worth the churn. `models/chats.py` has the same dead import and is handled in its own change, so it is skipped here to avoid two changes touching the same line.
No behaviour change.
orjson emits U+2028, U+2029 and U+0085 raw, where stdlib `json.dumps` escapes them under its default `ensure_ascii=True`. Python treats all three as line boundaries, so with `ENABLE_ORJSON` set, one of them inside model output splits a `data: {...}` SSE frame in half. Both halves then fail to parse and the delta is dropped with no error.
`utils/middleware.py` reassembles frames with `splitlines()`, so an affected response silently loses content on the direct API path. External clients are exposed as well: httpx's `LineDecoder` reimplements the same line-boundary semantics, so any SDK reading the OpenAI-compatible stream through `aiter_lines` breaks on a raw separator.
The three characters are escaped on the way out of `ORJSONCodec.dumps`. That restores parity with stdlib and fixes every reader at once, rather than patching one consumer and leaving external clients broken. They are the complete set: of the ten code points `splitlines()` treats as boundaries, the other seven are below U+0020, where JSON already forces an escape.
The membership guard is load bearing. Calling `translate` unconditionally costs roughly 1.5 us on a typical SSE chunk against 0.115 us for the serialization it wraps, so it would spend more than orjson saves. The three scans cost about 0.04 us.
Payloads containing none of the three are returned unchanged, byte for byte. With `ENABLE_ORJSON` unset, which is the default, none of this code runs.
U+2028 and U+2029 are common in text extracted from PDFs and word processor documents, so the realistic trigger is a model quoting an uploaded file back to the user.
New **pt-BR** translations for items introduced in the latest releases, plus a consistency/quality pass across existing strings (grammar, tone, capitalization, pluralization). Placeholders and hotkeys preserved. No logic changes.
The Valkey backend serializes chunk metadata on every insert and parses it back on every result row in `get` and `query`. Both directions now go through `JSONCodec`, which selects orjson when `ENABLE_ORJSON` is set.
The stored `metadata_json` field is never matched against as text. `_build_filter_expression` only emits TAG predicates, and the TAG fields are `id`, `hash`, `file_id`, `source` and `knowledge_base_id`; `metadata_json` appears only as a return field that is immediately re-parsed. So rows written with escaped non-ASCII and rows written raw are indistinguishable to every reader, and no migration is needed.
`process_metadata` already stringifies datetimes and strips null bytes and lone surrogates before the write, so the two backends cannot disagree about what is serializable here.
Both read `except` clauses widen from `(json.JSONDecodeError, TypeError)` to `(ValueError, TypeError)`. The codec falls back to engineio's codec, which installs `parse_int=_safe_int` and raises a bare `ValueError` for integer literals longer than 100 characters; the narrower clause would have let that escape and abort a search instead of yielding empty metadata. `json.JSONDecodeError` is a `ValueError` subclass, so this is a strict superset. That removes the module's last use of stdlib `json`, so the import goes with it.
With `ENABLE_ORJSON` unset, which is the default, `JSONCodec` is stdlib `json` and this call site behaves exactly as before.
RAG vector search runs in a thread pool, but then calls `future.result()` on the event loop thread, so the whole worker freezes until every collection answers. Every other user's token stream stops for that long. It's the default retrieval path.
Now `asyncio.gather` over `asyncio.to_thread`, matching what `routers/retrieval.py:2779` already does for the same call.
Measured with 3 queries across 4 collections, 60 ms search, and a second request wanting a turn every 5 ms:
| | before | after |
|---|---|---|
| RAG call | 62.0 ms | 61.2 ms |
| other request's turns | 0 | 7 |
| its worst stall | 62.5 ms | 16.0 ms |
Same results, same order, same `(result, error)` contract. Cancellation now lands mid-search instead of after every thread finishes. Threads move from an unbounded per-call pool to the loop's bounded shared one.
Timothy Jaeryang Baek