Commit Graph

475 Commits

Author SHA1 Message Date
Timothy Jaeryang Baek b02397e460 feat: WEB_LOADER_TIMEOUT 2025-12-08 11:49:27 -05:00
Timothy Jaeryang Baek 7e6b8a9a71 refac/fix: docling auth 2025-12-06 08:06:42 -05:00
Timothy Jaeryang Baek 9a65ed2260 chore: format 2025-12-02 16:06:57 -05:00
Classic298 b29fdc2a0c Update milvus_multitenancy.py (#19695) 2025-12-02 15:38:06 -05:00
Classic298 12f237ff80 fix: Update milvus.py (#19602)
* Update milvus.py

* Update milvus.py

* Update milvus.py

* Update milvus.py

* Update milvus.py

---------

Co-authored-by: Tim Baek <tim@openwebui.com>
2025-12-02 15:30:31 -05:00
Henne a7e614ca4c feat: Adds document intelligence model configuration (#19692)
* Adds document intelligence model configuration

Enables the configuration of the Document Intelligence model to be used by the RAG pipeline.

This allows users to specify the model they want to use for document processing, providing flexibility and control over the extraction process.

* Added Titel to Document Intelligence Model Config

Added Titel to Document Intelligence Model Config
2025-12-02 14:41:09 -05:00
Timothy Jaeryang Baek d19023288e feat/enh: kb files db migration 2025-12-02 10:53:32 -05:00
Poccia 6e531679f4 fix/adjust web search to properly block domains (#19670)
Co-authored-by: Tim Baek <tim@openwebui.com>
2025-12-02 04:17:32 -05:00
Classic298 0a14196afb Update milvus_multitenancy.py (#19680) 2025-12-02 03:57:14 -05:00
Timothy Jaeryang Baek 743199f2d0 feat/enh: tool server function name filter list 2025-11-25 02:31:34 -05:00
Timothy Jaeryang Baek 2328dc284e feat/enh: async embedding processing setting
Co-Authored-By: Classic298 <27028174+Classic298@users.noreply.github.com>
2025-11-25 01:55:43 -05:00
Timothy Jaeryang Baek 9c19d0abd4 refac/breaking: docling params 2025-11-24 16:01:13 -05:00
Timothy Jaeryang Baek 06f0bfd9f5 fix 2025-11-24 05:58:22 -05:00
Timothy Jaeryang Baek 662a1fac47 fix: hybrid search 2025-11-24 05:52:18 -05:00
Timothy Jaeryang Baek 48d1e67e79 chore: format 2025-11-23 20:15:52 -05:00
Classic298 99f0fe7f32 chore: user header forward minimize code changes throughout codebase (#19392)
* Update external.py

* remove unused imports

* Update ollama.py

* Update ollama.py

* Update ollama.py

* Update openai.py
2025-11-23 16:48:03 -05:00
Classic298 60dbde7e19 chore (#19389) 2025-11-23 04:40:05 -05:00
Timothy Jaeryang Baek f9c96d03ad refac 2025-11-22 22:57:27 -05:00
Timothy Jaeryang Baek 9bfc414d26 refac 2025-11-22 21:33:14 -05:00
Classic298 902c6cfbea perf: 50x performance improvement for external embeddings (#19296)
* Update utils.py (#77)

Co-authored-by: Claude <noreply@anthropic.com>

* refactor: address code review feedback for embedding performance improvements (#92)

Co-authored-by: Claude <noreply@anthropic.com>

* fix: prevent sentence transformers from blocking async event loop (#95)

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-22 20:54:59 -05:00
Diwakar b8728064d8 feat: add support for Weaviate vector database (#14747) 2025-11-20 19:23:46 -05:00
Timothy Jaeryang Baek a1d09eae95 chore: format 2025-11-19 03:23:33 -05:00
Seth Argyle 720af637e6 fix: Use get_index() instead of list_indexes() in has_collection() to… (#19238)
* fix: Use get_index() instead of list_indexes() in has_collection() to handle pagination

Fixes #19233

  Replace list_indexes() pagination scan with direct get_index() lookup
  in has_collection() method. The previous implementation only checked
  the first ~1,000 indexes due to unhandled pagination, causing RAG
  queries to fail for indexes beyond the first page.

  Benefits:
  - Handles buckets with any number of indexes (no pagination needed)
  - ~8x faster (0.19s vs 1.53s in testing)
  - Proper exception handling for ResourceNotFoundException
  - Scales to millions of indexes

* Update s3vector.py

Unneeded exception handling removed to match original OWUI code
2025-11-19 00:19:10 -05:00
Jacob Leksan 07ef295a77 feat: Adding file metadata to hybrid search (#19095)
* Added metadata to hybrid search

* And config and env plus refac

* consistency

---------

Co-authored-by: Tim Baek <tim@openwebui.com>
2025-11-18 15:29:07 -05:00
Timothy Jaeryang Baek 2af4c4b3c7 refac 2025-11-18 04:42:09 -05:00
Timothy Jaeryang Baek 02238d3113 feat/security: Add SSRF protection with configurable blocklist
Co-Authored-By: Classic298 <27028174+Classic298@users.noreply.github.com>
2025-11-18 04:40:55 -05:00
lazariv 6cdb13d5cb feat: pgvector hnsw index type (#19158)
* Adding hnsw index type for pgvector, allowing vector dimensions larger than 2000

* remove some variable assignments

* Make USE_HALFVEC variable configurable

* Simplify USE_HALFVEC handling

* Raise runtime error if the index requires rebuilt

---------

Co-authored-by: Moritz <moritz.mueller2@tu-dresden.de>
2025-11-18 04:14:43 -05:00
Timothy Jaeryang Baek ee10f372a0 refac/enh: web search domain allow/block filter 2025-11-16 13:52:09 -05:00
Sang Lê 64747f7f79 Add Azure Search (#19104)
Co-authored-by: Tim Baek <tim@openwebui.com>
2025-11-13 19:12:34 -05:00
Classic298 ad17d35ac4 feat: Add custom API endpoint and user info headers for Perplexity Search (#31) (#19147)
Co-authored-by: Claude <noreply@anthropic.com>
2025-11-12 22:53:54 -05:00
Timothy Jaeryang Baek 4673e120c4 refac/fix: mineru params
breaking change
2025-11-11 00:30:11 -05:00
Timothy Jaeryang Baek bc739de024 refac: rerank 2025-11-09 21:33:50 -05:00
Tim Baek 284764e178 Merge pull request #19025 from krishna-medapati/fix-hybrid-search-17046-clean
fix: Handle AttributeError in hybrid search with reranking (#17046)
2025-11-09 20:43:52 -05:00
Timothy Jaeryang Baek 25c7f101f2 enh: optionally add user headers external websearch
Co-Authored-By: Classic298 <27028174+Classic298@users.noreply.github.com>
2025-11-09 16:09:29 -05:00
krishna-medapati 684324ae9e fix: Handle AttributeError in hybrid search with reranking (#17046)
- Split attribute existence checks from document content checks
- Added hasattr() check for metadatas attribute
- Prevents AttributeError when collection_result is missing attributes
- Maintains all original validation logic

Fixes #17046
2025-11-07 23:31:11 +05:30
Timothy Jaeryang Baek 7faf19dad9 refac 2025-11-06 15:21:06 -05:00
Timothy Jaeryang Baek c38f878e1e fix: firecrawl import 2025-11-06 15:19:08 -05:00
Timothy Jaeryang Baek e2b9942648 feat: Optionally forward user headers to external document loader
Co-Authored-By: Classic298 <27028174+Classic298@users.noreply.github.com>
2025-11-06 00:05:46 -05:00
Timothy Jaeryang Baek 415b93c7c3 enh: configurable mistral ocr base url 2025-11-05 23:25:51 -05:00
Timothy Jaeryang Baek a4fd26b478 enh/fix: google pse referer header 2025-11-04 13:50:07 -05:00
Timothy Jaeryang Baek d14329b285 refac 2025-11-02 18:47:41 -05:00
Timothy Jaeryang Baek d98c539d89 refac 2025-11-01 18:48:11 -04:00
Tim Baek c787070dc9 Merge pull request #18765 from mkhludnev/patch-2
fix: Don't missguide Tika with mime-type
2025-10-31 14:18:43 -07:00
Mikhail Khludnev 24aeec9120 Don't missguide Tika with mime-type
Fix #18683 
* Tika is smart enough to detect content type. 
* Windows browsers just misguides Tika providing `application/ms-word` for .rtf files.
2025-10-31 23:12:23 +03:00
Omar Aburub 3bcf9a442a fix: make SSL verification async 2025-10-29 16:14:53 +03:00
Wang Weixuan 5e17882488 fix: use trusted env in web search loader
Signed-off-by: Wang Weixuan <wangweixvan@gmail.com>
2025-10-28 04:58:00 +08:00
Timothy Jaeryang Baek a70bc52c34 chore: format 2025-10-26 19:33:39 -07:00
Timothy Jaeryang Baek 8197844ff7 refac 2025-10-26 17:22:23 -07:00
wei840222 7a3f4d85f6 refactor: replace requests with Firecrawl SDK in search and requests Firecrawl SDK in scrape rather than langchain_community FireCrawlLoader 2025-10-26 15:05:35 +08:00
Ivan Ostanin 7d29991fa5 fix: pass youtube_proxy as a GenericProxyConfig type object 2025-10-19 02:01:27 +02:00