39 Commits

Author SHA1 Message Date
Classic298 ac8af4996c perf: index group_member on (user_id, group_id) (#27822)
Permission checks are the most repeated database work in a request, and every one of them asks the same question: which groups is this user in. Today that question cannot use an index.

`group_member` has only its primary key and a `(group_id, user_id)` unique constraint. That constraint leads on `group_id`, so a lookup by `user_id` has to walk the entire membership table, every time. `Groups.get_groups_by_member_id` sits under `has_permission`, `has_access`, `check_model_access` and the `AccessGrants` fallbacks, so an ordinary chat completion pays that walk several times before the model is even called, and the admin user list pays it once per row.

The cost scales with total memberships across all users rather than with the size of any one user's, so it stays invisible on a small instance and then arrives all at once on a large one.

Measured on SQLite, timing the real join from `get_groups_by_member_id`:

| memberships | before | after |
|---|---|---|
| 5,000 | 0.04 ms | 0.03 ms |
| 50,000 | 0.10 ms | 0.04 ms |
| 200,000 | 1.33 ms | 0.04 ms |
| 500,000 | 2.94 ms | 0.04 ms |

The after column is flat because the lookup becomes a seek instead of a scan. Concretely: on a deployment with 500k memberships, say 10,000 users in 50 groups each, one chat completion currently spends roughly 15 ms of database time answering the same question over and over. Afterwards it is under 0.2 ms. On a small install you will not be able to measure the difference, and that is fine, the point is that the curve stops bending.

The index is `(user_id, group_id)`. The trailing column makes those lookups index-only, since `group_id` is the column they select. Queries that lead on `group_id`, such as `get_group_user_ids_by_id` and the `chat_messages` subqueries, are already served by the existing unique constraint and are unaffected.

What to expect when the migration runs: on PostgreSQL this is a plain `CREATE INDEX`, which takes a SHARE lock, so reads continue while writes to `group_member` block until it completes. The table holds one row per membership, so expect sub-second even on the numbers above. `CONCURRENTLY` cannot be used here because the migration runner wraps the upgrade in a transaction, and it is not warranted at this table size.
2026-07-31 19:09:15 -05:00
Classic298 6be11d4fc9 chore: remove dead json imports (#27815)
Fourteen modules import `json` without using it. Ruff flags every one with F401, and a word-boundary search for `json` in each file matches only the import line itself, including inside strings, comments and annotations.

Two exclusions, both deliberate. Migration files are left alone: the import is equally dead there, but those files are frozen history and not worth the churn. `models/chats.py` has the same dead import and is handled in its own change, so it is skipped here to avoid two changes touching the same line.

No behaviour change.
2026-07-31 17:25:40 -04:00
Timothy Jaeryang Baek 6d0295588e refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
Timothy Jaeryang Baek 25898116ea chore: format 2026-04-12 18:12:59 -05:00
Timothy Jaeryang Baek 27169124f2 refac: async db 2026-04-12 14:22:11 -05:00
Timothy Jaeryang Baek 968462609f refac 2026-03-24 18:05:19 -05:00
Timothy Jaeryang Baek ade617efa8 refac 2026-03-24 04:49:48 -05:00
Timothy Jaeryang Baek de3317e26b refac 2026-03-17 17:58:01 -05:00
Timothy Jaeryang Baek 87d33f6e18 refac 2026-02-25 14:52:41 -06:00
Timothy Jaeryang Baek 538501c88d refac 2026-02-24 15:19:49 -06:00
Timothy Jaeryang Baek e5e39be90f refac 2026-02-16 13:14:40 -06:00
Timothy Jaeryang Baek 09dc28df1e chore: format 2026-02-16 00:43:32 -06:00
Timothy Jaeryang Baek 33308022f0 refac 2026-02-15 23:57:40 -06:00
Timothy Jaeryang Baek 96c07f44a8 refac 2026-02-11 16:45:47 -06:00
Timothy Jaeryang Baek f376d4f378 chore: format 2026-02-11 16:24:11 -06:00
Timothy Jaeryang Baek f7406ff576 refac 2026-02-09 13:28:14 -06:00
Timothy Jaeryang Baek 3ae44d11a5 refac 2026-02-08 22:37:13 -06:00
Timothy Jaeryang Baek b147616080 refac 2026-02-08 21:32:12 -06:00
Classic298 68e257849d perf: optimize database queries in functions, feedbacks, and groups (#21019) 2026-01-29 21:44:50 +04:00
Timothy Jaeryang Baek de0cbb9073 refac 2026-01-12 21:56:02 +04:00
Timothy Jaeryang Baek 5a075a2c83 fix: members only groups 2026-01-12 21:53:41 +04:00
Classic298 732d9b484d fix: resolve N+1 query pattern in users endpoint (#20427)
## Summary

Fixed N+1 query pattern in the `/api/v1/users` endpoint where groups were being fetched for each user individually.

### Problem

The `GET /api/v1/users` endpoint called `Groups.get_groups_by_member_id()` for each user, resulting in:
- 1 query for users
- N queries for groups (one per user)

### Solution

Added a new `Groups.get_groups_by_member_ids()` method that fetches groups for multiple users in a single query using SQL `IN` clause and `JOIN`.

### Changes

- **[groups.py](open_webui/models/groups.py)**: Added `get_groups_by_member_ids()` method
- **[users.py](open_webui/routers/users.py)**: Updated endpoint to use bulk method

### Result

- Before: 1 + N queries
- After: 2 queries total (1 for users, 1 for all groups)
2026-01-06 21:26:14 +04:00
Timothy Jaeryang Baek ca514cd3ed refac: group share to settings 2026-01-05 05:32:56 +04:00
Timothy Jaeryang Baek 2041ab483e refac/enh: db session sharing 2025-12-28 22:00:44 +04:00
Classic298 823b9a6dd9 chore/perf: Remove old SRC level log env vars with no impact (#20045)
* Update openai.py

* Update env.py

* Merge pull request open-webui#19030 from open-webui/dev (#119)

Co-authored-by: Tim Baek <tim@openwebui.com>
Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Tim Baek <tim@openwebui.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-12-20 08:16:14 -05:00
Timothy Jaeryang Baek c1d760692f refac: db group 2025-11-28 22:48:58 -05:00
Timothy Jaeryang Baek baa1e07aec refac 2025-11-25 04:37:58 -05:00
Timothy Jaeryang Baek 7be750bcbb feat/enh: group share setting 2025-11-20 19:12:56 -05:00
Timothy Jaeryang Baek bc576782d7 refac: group members backend 2025-11-17 05:09:06 -05:00
Timothy Jaeryang Baek edf7a3d142 refac 2025-08-26 22:05:27 +04:00
Timothy Jaeryang Baek 8b425a6295 refac 2025-08-26 21:56:52 +04:00
Timothy Jaeryang Baek 9634df4347 refac/enh: group add/remove users endpoints 2025-07-17 01:50:37 +04:00
Timothy Jaeryang Baek 371bdd7afa refac 2025-06-11 20:40:19 +04:00
Timothy Jaeryang Baek cce5f024bd feat: WEBUI_AUTH_TRUSTED_GROUPS_HEADER 2025-05-24 23:17:12 +04:00
Timothy Jaeryang Baek 45f4bc18f8 refac: access controls 2025-01-20 23:20:47 -08:00
Timothy Jaeryang Baek aa442f694b enh: validate user id before saving group 2025-01-20 23:09:55 -08:00
Timothy Jaeryang Baek 217e3a13c8 feat: Add ability to change permissions on group creation API 2025-01-17 12:03:24 -08:00
Timothy Jaeryang Baek d701b69e05 enh: channel notification 2024-12-25 00:53:25 -07:00
Timothy Jaeryang Baek d3d161f723 wip 2024-12-10 00:54:13 -08:00