42 Commits

Author SHA1 Message Date
Classic298 6be11d4fc9 chore: remove dead json imports (#27815)
Fourteen modules import `json` without using it. Ruff flags every one with F401, and a word-boundary search for `json` in each file matches only the import line itself, including inside strings, comments and annotations.

Two exclusions, both deliberate. Migration files are left alone: the import is equally dead there, but those files are frozen history and not worth the churn. `models/chats.py` has the same dead import and is handled in its own change, so it is skipped here to avoid two changes touching the same line.

No behaviour change.
2026-07-31 17:25:40 -04:00
Timothy Jaeryang Baek f1409266fe refac 2026-07-27 02:44:21 -04:00
G30 771540f3de feat: sortable title and updated at columns on the knowledge workspace page (#27457) 2026-07-26 18:19:31 -04:00
Classic298 310ae91302 perf: make tools defer_content real and batch tool and knowledge access filters (#27387)
Tools.get_tools(defer_content=True) contained the literal dead statement "stmt = stmt": the deferral was a no-op, so every tools listing loaded the full Python source of every tool (five caller sites pass defer_content=True expecting the optimization: the tools list endpoints and the user and group permission overviews). The listing now selects every column except content, and ToolModel.content becomes optional to represent deferred rows; router projections are content-less response models, so nothing downstream reads the source on these paths.

On top of that, get_tools_by_user_id issued one grant query per non-owned tool and Knowledges.get_knowledge_bases_by_user_id did the same per knowledge base (the latter also sits inside per-file access checks). Both now resolve grants for all non-owned rows in a single get_accessible_resource_ids call, the same batch helper the model listing already uses.

Benchmark (real SQLite DB):

| metric | before | after |
| --- | --- | --- |
| tools listing, 33 tools x ~200 KB source | 5.13 ms | 3.18 ms |
| grant queries per accessible-tools call, N non-owned tools | N | 1 |
| grant queries per accessible-KBs call, N non-owned KBs | N | 1 |

The listing row scales with source size; on Postgres the deferral additionally avoids shipping every tool's source over the wire per listing, and each removed grant query was a real round trip.

Functionally verified: deferred listings match full listings field for field with content None, grants included and router projections working; access filtering returns exactly owned plus granted tools and knowledge bases and nothing for strangers; full (non-deferred) reads still carry the source.
2026-07-23 17:51:07 -05:00
Classic298 5b518cbe43 perf: list knowledge base file metadata without loading extracted file contents (#27386)
Knowledges.get_file_metadatas_by_id fetched full File rows, whose data column carries the entire extracted text of each document, validated each into a FileModel and then threw everything except id, hash, meta and the timestamps away. The function backs every knowledge base detail view and runs again after every file add or remove (eight call sites in the knowledge router), so rendering a filename list for a 50-file knowledge base parsed tens of megabytes of JSON per request.

The listing now selects exactly the five columns the response needs, joined through KnowledgeFile, mirroring the column-only helper that already existed in the files model for id-based lookups.

Benchmark (real SQLite DB, 50 files with ~200 KB extracted text each):

| metric | before | after |
| --- | --- | --- |
| KB file metadata listing | 14.2 ms | 1.20 ms |

The gap widens linearly with file size and count since extracted contents no longer get read, parsed or validated at all.

Functionally verified: output matches the old implementation field for field on all 50 files and an unknown knowledge base still returns an empty list.
2026-07-23 17:50:55 -05:00
Timothy Jaeryang Baek e5e2cd7876 refac 2026-07-17 04:48:21 -04:00
Timothy Jaeryang Baek ab84bbf08c refac 2026-06-29 04:19:33 -05:00
Timothy Jaeryang Baek 46c1d6591b refac 2026-06-29 01:41:43 -05:00
Timothy Jaeryang Baek 15c7e37438 refac 2026-06-23 23:13:32 +02:00
Timothy Jaeryang Baek 591e0aafa1 refac 2026-05-28 17:24:33 -05:00
Timothy Jaeryang Baek 1527eb6e01 refac 2026-05-20 01:46:59 +04:00
Timothy Jaeryang Baek 4e78b355ef refac 2026-05-14 02:00:23 +09:00
Timothy Jaeryang Baek d74ee34d91 refac 2026-05-14 01:49:34 +09:00
Timothy Jaeryang Baek c2cbc47ca7 feat: knowledge directory 2026-05-13 22:37:53 +09:00
Timothy Jaeryang Baek 6d0295588e refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
Timothy Jaeryang Baek 11e076817a refac 2026-05-09 07:34:46 +09:00
Timothy Jaeryang Baek 25898116ea chore: format 2026-04-12 18:12:59 -05:00
Timothy Jaeryang Baek 27169124f2 refac: async db 2026-04-12 14:22:11 -05:00
Timothy Jaeryang Baek ade617efa8 refac 2026-03-24 04:49:48 -05:00
Timothy Jaeryang Baek de3317e26b refac 2026-03-17 17:58:01 -05:00
Timothy Jaeryang Baek fcff9c3afd refac 2026-03-01 13:20:55 -06:00
Timothy Jaeryang Baek 631e30e22d refac 2026-02-21 15:35:34 -06:00
Timothy Jaeryang Baek 8c485b260f refac 2026-02-19 16:53:21 -06:00
Timothy Jaeryang Baek f376d4f378 chore: format 2026-02-11 16:24:11 -06:00
Timothy Jaeryang Baek 0f78451c2b refac 2026-02-08 21:24:20 -06:00
Timothy Jaeryang Baek a9a0ce6bea refac 2026-01-22 03:09:04 +04:00
G30 e9926694c3 fix: add username search support to workspace and admin pages (#20780)
This fix restores and extends the username/email search functionality across workspace pages that was originally added in PR #14002. The issue was that:

1. The backend search functions for Models and Knowledge only searched `User.name` and `User.email`, but not `User.username`

2. The Functions admin page lacked user search entirely

Changes made:

Added User.username to backend search conditions for Models and Knowledge pages
Added complete user search (name, email, username) to the Functions admin page client-side filter
2026-01-19 13:42:33 +04:00
Timothy Jaeryang Baek 700349064d chore: format 2026-01-08 01:55:56 +04:00
G30 8712446cc1 fix: resolve undefined knowledge base name in file reference tooltip (#20333) 2026-01-03 18:13:19 +04:00
Timothy Jaeryang Baek 2041ab483e refac/enh: db session sharing 2025-12-28 22:00:44 +04:00
Classic298 823b9a6dd9 chore/perf: Remove old SRC level log env vars with no impact (#20045)
* Update openai.py

* Update env.py

* Merge pull request open-webui#19030 from open-webui/dev (#119)

Co-authored-by: Tim Baek <tim@openwebui.com>
Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Tim Baek <tim@openwebui.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-12-20 08:16:14 -05:00
Timothy Jaeryang Baek 4d9a51ba33 refac 2025-12-11 00:11:12 -05:00
Timothy Jaeryang Baek ceae3d48e6 enh/refac: kb pagination 2025-12-10 23:19:19 -05:00
Timothy Jaeryang Baek c15201620d refac: kb files 2025-12-10 15:48:27 -05:00
Timothy Jaeryang Baek 94a8439105 feat/enh: kb file pagination 2025-12-10 00:53:41 -05:00
Timothy Jaeryang Baek e301d1962e refac/perf: has_access_to_file optimization 2025-12-02 11:11:17 -05:00
Timothy Jaeryang Baek d19023288e feat/enh: kb files db migration 2025-12-02 10:53:32 -05:00
Timothy Jaeryang Baek 40e40d1ddd enh/sec: verify folder data integrity
#17182
2025-09-07 01:04:56 +04:00
Tim Jaeryang Baek 472b71f331 Merge pull request #17166 from sihyeonn/perf/sh-model-layer
perf: fix N+1 query issues in user group access control validation
2025-09-03 13:36:01 +04:00
Sihyeon Jang eff06538a6 perf: fix N+1 query issues in user group access control validation
- Pre-fetch user group IDs in get_*_by_user_id methods across models layer
- Pass user_group_ids to has_access to avoid repeated group queries
- Reduce query count from 1+N to 1+1 pattern for access control validation
- Apply consistent optimization across knowledge, models, notes, prompts, and tools

Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
2025-09-03 05:56:48 +09:00
Sihyeon Jang f588655f7f perf: fix N+1 query issue in get_knowledge_bases method
- Replace individual user queries with batch fetching
- Use single query to fetch all required users at once
- Implement O(1) user lookup with dictionary mapping
- Reduce query count from 1+N to 1+1 pattern

Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
2025-09-03 05:29:47 +09:00
Timothy Jaeryang Baek d3d161f723 wip 2024-12-10 00:54:13 -08:00