146 Commits

Author SHA1 Message Date
Timothy Jaeryang Baek bd8378f643 refac 2026-08-10 23:22:08 -06:00
Classic298 ac8af4996c perf: index group_member on (user_id, group_id) (#27822)
Permission checks are the most repeated database work in a request, and every one of them asks the same question: which groups is this user in. Today that question cannot use an index.

`group_member` has only its primary key and a `(group_id, user_id)` unique constraint. That constraint leads on `group_id`, so a lookup by `user_id` has to walk the entire membership table, every time. `Groups.get_groups_by_member_id` sits under `has_permission`, `has_access`, `check_model_access` and the `AccessGrants` fallbacks, so an ordinary chat completion pays that walk several times before the model is even called, and the admin user list pays it once per row.

The cost scales with total memberships across all users rather than with the size of any one user's, so it stays invisible on a small instance and then arrives all at once on a large one.

Measured on SQLite, timing the real join from `get_groups_by_member_id`:

| memberships | before | after |
|---|---|---|
| 5,000 | 0.04 ms | 0.03 ms |
| 50,000 | 0.10 ms | 0.04 ms |
| 200,000 | 1.33 ms | 0.04 ms |
| 500,000 | 2.94 ms | 0.04 ms |

The after column is flat because the lookup becomes a seek instead of a scan. Concretely: on a deployment with 500k memberships, say 10,000 users in 50 groups each, one chat completion currently spends roughly 15 ms of database time answering the same question over and over. Afterwards it is under 0.2 ms. On a small install you will not be able to measure the difference, and that is fine, the point is that the curve stops bending.

The index is `(user_id, group_id)`. The trailing column makes those lookups index-only, since `group_id` is the column they select. Queries that lead on `group_id`, such as `get_group_user_ids_by_id` and the `chat_messages` subqueries, are already served by the existing unique constraint and are unaffected.

What to expect when the migration runs: on PostgreSQL this is a plain `CREATE INDEX`, which takes a SHARE lock, so reads continue while writes to `group_member` block until it completes. The table holds one row per membership, so expect sub-second even on the numbers above. `CONCURRENTLY` cannot be used here because the migration runner wraps the upgrade in a transaction, and it is not warranted at this table size.
2026-07-31 19:09:15 -05:00
Timothy Jaeryang Baek 965ef909d7 chore: format 2026-07-27 04:46:45 -04:00
Timothy Jaeryang Baek 50e050e195 refac
Co-Authored-By: djedi-knight <943190+djedi-knight@users.noreply.github.com>
2026-07-27 04:43:51 -04:00
Timothy Jaeryang Baek 20647bd2d5 chore: format 2026-07-27 00:12:47 -04:00
Timothy Jaeryang Baek f798d05586 refac 2026-07-26 19:34:41 -04:00
Timothy Jaeryang Baek 28bdcb063b refac 2026-07-24 01:54:36 -04:00
Timothy Jaeryang Baek 212eec408c refac 2026-07-24 01:44:30 -04:00
Timothy Jaeryang Baek bef8ae4b2f refac 2026-07-24 00:40:42 -04:00
Timothy Jaeryang Baek ca11bd90a7 chore: format 2026-07-23 13:41:16 -04:00
Timothy Jaeryang Baek 060d5da473 refac 2026-07-23 03:01:01 -04:00
Timothy Jaeryang Baek cf887b68ea refac 2026-07-23 02:54:56 -04:00
Timothy Jaeryang Baek 4d27bfff92 refac 2026-07-16 01:42:24 -04:00
Timothy Jaeryang Baek 90eca2ac25 refac 2026-07-01 03:37:35 -05:00
Timothy Jaeryang Baek c416c6cad6 refac 2026-07-01 02:53:42 -05:00
Timothy Jaeryang Baek 517cd8d102 refac 2026-06-29 13:03:14 -05:00
Timothy Jaeryang Baek c0c6c2181a refac 2026-06-29 12:40:20 -05:00
Timothy Jaeryang Baek 655afbe90b refac 2026-06-29 11:59:50 -05:00
Timothy Jaeryang Baek 70e4ffcc65 refac 2026-06-29 05:49:49 -05:00
Timothy Jaeryang Baek 124c7a3283 refac 2026-06-29 00:21:37 -05:00
Timothy Jaeryang Baek 2560533c1a refac 2026-06-29 00:18:40 -05:00
Timothy Jaeryang Baek 3f0c0e0a0d refac 2026-06-19 00:16:06 +02:00
Timothy Jaeryang Baek 21f9e5295b refac 2026-06-18 10:47:25 +02:00
Timothy Jaeryang Baek 5cdcdbaeec refac 2026-06-17 02:52:35 +02:00
Timothy Jaeryang Baek 6fce92aa12 chore: format 2026-06-01 13:56:55 -07:00
Timothy Jaeryang Baek d8b5b9fa79 refac 2026-05-21 15:29:49 +04:00
Algorithm5838 ae48838b04 fix: tag composite pk in migration (#24722) 2026-05-15 09:31:27 +09:00
Timothy Jaeryang Baek 9263b7568e refac
Co-Authored-By: Classic298 <27028174+Classic298@users.noreply.github.com>
2026-05-14 14:12:42 +09:00
Timothy Jaeryang Baek 73d2065227 fix: legacy peewee tables fk 2026-05-14 14:06:46 +09:00
Timothy Jaeryang Baek d7cfc1e46a refac 2026-05-14 14:06:32 +09:00
Timothy Jaeryang Baek 9717ada92f refac 2026-05-14 14:05:28 +09:00
Timothy Jaeryang Baek 9a8969ca93 refac 2026-05-14 13:49:50 +09:00
Timothy Jaeryang Baek 2e1b671e8d refac 2026-05-14 13:49:15 +09:00
Timothy Jaeryang Baek db2b3d7fd8 refac 2026-05-14 13:46:54 +09:00
Timothy Jaeryang Baek 1004dad274 refac 2026-05-14 13:46:36 +09:00
Timothy Jaeryang Baek ee3b14233a refac 2026-05-14 13:46:23 +09:00
Timothy Jaeryang Baek dc0f8ae6f2 refac 2026-05-14 13:46:08 +09:00
Timothy Jaeryang Baek 1b9d22e324 refac 2026-05-14 13:45:59 +09:00
Timothy Jaeryang Baek 98d3b23085 refac 2026-05-14 13:45:39 +09:00
Timothy Jaeryang Baek 6b1df94bf9 refac 2026-05-14 13:45:31 +09:00
Timothy Jaeryang Baek 95840e307a refac 2026-05-14 13:45:21 +09:00
Timothy Jaeryang Baek 6df09a4039 refac 2026-05-14 13:45:12 +09:00
Timothy Jaeryang Baek 459b1c3fda refac 2026-05-14 13:45:05 +09:00
Timothy Jaeryang Baek bd9f82d5a6 refac 2026-05-14 13:44:50 +09:00
Timothy Jaeryang Baek f0e88dadc8 refac 2026-05-14 13:08:53 +09:00
Timothy Jaeryang Baek 81f611fb73 refac 2026-05-14 03:06:37 +09:00
Timothy Jaeryang Baek c2cbc47ca7 feat: knowledge directory 2026-05-13 22:37:53 +09:00
Timothy Jaeryang Baek 6d0295588e refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
Timothy Jaeryang Baek a59c967d7e refac: modernize imports, standardize type hints and docstrings 2026-05-12 06:30:38 +09:00
Timothy Jaeryang Baek 2c2d06c31b refac: deprecate peewee migration layer
The Alembic init migration (7e5b5dc7342b) already creates the
equivalent schema. Peewee migrations are no longer needed for
any version >= 0.3.6.
2026-05-12 03:45:59 +09:00