The streaming handler's content accumulator is a closure cell (declared nonlocal in stream_body_handler), and CPython's in-place string append optimization only applies to plain local variables (STORE_FAST), never to cell variables (STORE_DEREF). The content += value form introduced in #27231 therefore still allocates and copies the full accumulated string on every delta, exactly like the f-string it replaced; whether that copy is cheap or expensive is up to the allocator, and measurements swing accordingly (6 to 83 ms of pure copying for a 400 KB response on Python 3.12, against 0.3 ms for this patch).
Accumulate the deltas in a list instead and join once at the single read site (publish_chat_finished_event at stream end). List append is amortized O(1) with no dependence on reference counts, bytecode specialization or allocator behaviour, so accumulation is O(n) by construction. The non-str fallback keeps the previous f-string coercion semantics.
Verified end to end against a mock SSE upstream: a streamed chat with a think-tag block plus 40 content deltas produces output items, message text, reasoning text and usage identical to current dev, with no errors in the server log.
The branch that normalizes plain JSON error lines from streaming upstreams (lines without the SSE data: prefix) called Chats.upsert_message_to_chat_by_id_and_message_id without await. The coroutine was never executed, so the error was never written to the chat and Python emitted a "coroutine was never awaited" RuntimeWarning instead. The frontend still received the error event, but after a reload the message showed no trace of the failure.
The parallel error-persist branch further down the same handler already awaits the call; this aligns the two.
Several spots in the chat pipeline issued sequential single-key config
SELECTs, or fetched the same key twice back-to-back, on every request:
- chat_completion_tools_handler: task model default/external and the
tools prompt template were four sequential Config round trips (the
template was fetched twice). One batched Config.get_many now serves
all of them.
- chat_completion_files_handler: the six RAG settings (top_k,
top_k_reranker, relevance_threshold, hybrid_bm25_weight,
enable_hybrid_search, full_context) were six sequential round trips
inside the retrieval call. Batched into one get_many.
- Voice and code-interpreter prompt templates were each fetched twice
within one conditional; fetch once and reuse. The code-interpreter
engine was likewise fetched twice per execution.
- Skill resolution fetched the accessible-skills list, kept only the
ids, then re-fetched each mentioned skill by id (N+1). Reuse the
rows from the access query.
Value semantics are identical: get_many applies the same defaults as
the individual gets, and the pre-existing truthiness/empty-string
checks on templates are preserved exactly.
Claude-Session: https://claude.ai/code/session_01MHg5zs1VBjvRWQ54qHpfYD
Co-authored-by: Claude <noreply@anthropic.com>
The middleware code-interpreter path defines `restricted_import` as `async def` and assigns it to `builtins.__import__`, which Python's import machinery calls synchronously. Calling an async function returns a coroutine without running its body, so the blocklist check never executes and `_real_import` is never called. When `CODE_INTERPRETER_BLOCKED_MODULES` is set, blocked modules are therefore not blocked, and every subsequent import inside the interpreter binds a dangling coroutine instead of the module, breaking legitimate imports as well.
Define the hook as a regular `def`, matching the working implementation in `tools/builtin.py`. A blocked top-level import now raises `ImportError`, and all other imports pass through to the real importer.
The streaming handler rebuilt the full accumulated response with
`content = f'{content}{value}'` on every content delta — a complete
string copy per chunk, making accumulation O(n^2) over the response
length. Use in-place `content += value` for the (universal) str case,
which CPython extends in place, keeping accumulation O(n); the
f-string fallback is preserved for non-str values so coercion
behavior is unchanged.
In the ENABLE_REALTIME_CHAT_SAVE branch, full_output() — which
concatenates the entire accumulated output — was called twice per
chunk (once for the DB upsert, once for the emitted delta). Compute
it once and reuse.
Claude-Session: https://claude.ai/code/session_01MHg5zs1VBjvRWQ54qHpfYD
Co-authored-by: Claude <noreply@anthropic.com>
Timothy Jaeryang Baek