GLOBAL_LOG_LEVEL defaults to INFO, so every log.debug(...) in the backend is discarded, but the message is built first: 187 call sites interpolate their payload into an f-string before the logging call runs, so the work happens on every request and the result is thrown away. The worst one sits in process_chat_payload and stringifies the whole request body, full conversation history included, once per chat completion.
That one line with DEBUG disabled, CPython 3.12:
| conversation | payload | before | after |
| ------------ | ------- | -------- | ------- |
| 4 messages | 1.2 kB | 3.4 us | 0.07 us |
| 20 messages | 17 kB | 24.8 us | 0.07 us |
| 60 messages | 123 kB | 216.6 us | 0.07 us |
The lazy form log.debug('form_data: %s', form_data) hands the payload to record.getMessage(), which the InterceptHandler only reaches once a record has passed the level check. With DEBUG enabled the emitted lines are byte-identical, f'{x=}' sites included: those map to %r. MistralLoader._debug_log callers get the same treatment, since that wrapper already forwards *args.
validate_url() calls socket.getaddrinfo() for SSRF protection, which
blocks the event loop for 100-700ms per DNS lookup. This affects:
- Image generation (get_image_data) — every external image URL
- Image editing (load_url_image) — every external image URL
- OAuth profile pictures (_process_picture_url) — every login
- Webhook delivery (post_webhook) — every notification
- Image base64 conversion (get_image_base64_from_url) — chat images
Wrap all 5 async call sites in asyncio.to_thread() so DNS resolution
runs in the thread pool. The event loop remains free to serve other
requests during the lookup.
Benchmark (3 domains, 3 trials averaged):
- BEFORE: max jitter 479ms, 1 blocked ping per trial
- AFTER: max jitter 1ms, 0 blocked pings (324x improvement)
Co-authored-by: Tim Baek <tim@openwebui.com>
* fix: block private-IP webhook URLs to close SSRF on caller-controlled URL
post_webhook(url, ...) in utils/webhook.py forwards the URL straight to
aiohttp.ClientSession.post with no SSRF gate. The URL is caller-controlled
on two surfaces:
- User notification settings under ENABLE_USER_WEBHOOKS=true — any
authenticated user can set the URL their notifications POST to.
- Automation notification triggers (calendar alerts, etc.).
Without a gate, the URL can target cloud metadata (169.254.169.254 /
fd00:ec2::254), localhost-bound services, RFC1918 internal hosts, or any
other private address reachable from the server process. Blind SSRF — no
response body returned to the caller — but enough to enumerate internal
services via response timing / status codes, and on cloud deployments
enough to issue requests against IMDSv1 if available.
Call validate_url() at the top of post_webhook. The function blocks
private/reserved IPs when ENABLE_RAG_LOCAL_WEB_FETCH is False (the
default), is the project's chosen SSRF gate, and is already applied to
the equivalent fetch surfaces (retrieval, image-load, OAuth profile
picture). Operators who legitimately need to webhook to private IPs
(internal monitoring, self-hosted Slack alternatives, etc.) can set
ENABLE_RAG_LOCAL_WEB_FETCH=True — same opt-out as the other gated
surfaces.
Scope intentionally limited to webhooks. The OAuth discovery and
external reranker paths cwanglab also flagged are admin-configured with
intentional private-IP defaults (reranker defaults to
http://localhost:8080/v1/rerank) and are out of scope per Rule 9 — the
admin owns the URL choice and the operator opt-out exists for them too.
Reported by cwanglab in GHSA-5x9f-85cg-w3hf (cluster canonical with six
closed siblings: g36v-23gj-j69x, 6j8f-h58v-xgmw, xpwv-52pm-p8hj,
v9gp-hv2c-9qv8, fw7w-jrw7-p3v9, x7xq-74rg-m8mf).
Co-authored-by: cwanglab <cwanglab@users.noreply.github.com>
* fix: also pass allow_redirects=False on webhook post_webhook session.post
Companion to the previous commit. validate_url() only validates the
initial URL; aiohttp's default allow_redirects=True would still follow
a 302 to a private-IP target. Same redirect-bypass class as the rh5x
cluster's five call sites, sixth call site to receive the same gate.
Co-authored-by: cwanglab <cwanglab@users.noreply.github.com>
---------
Co-authored-by: cwanglab <cwanglab@users.noreply.github.com>
json.loads(event_data.get("user", {})) crashes with TypeError when
the "user" key is absent because the default value {} is a dict, not
a JSON string. json.loads expects str/bytes, not dict.
Also handle the case where "user" is already a dict (not serialized
JSON) to make the webhook more robust.
Co-authored-by: gambletan <ethanchang32@gmail.com>
Classic298