mirror of
https://github.com/openclaw/openclaw.git
synced 2026-08-21 01:51:39 -06:00
Onur Solmaz
c2de3206d4
* feat(llama-cpp): add external server provider * feat(llama-cpp): document external server setup * refactor(llama-cpp): harden external provider boundaries * fix(llama-cpp): support external structured output * fix(llama-cpp): isolate replacement endpoint credentials * test(llama-cpp): register external live shard * fix(llama-cpp): preserve explicit endpoint authorization * fix(llama-cpp): clear disabled inline credentials * fix(llama-cpp): preserve external local service configs * test(llama-cpp): cover retained external configs * test(llama-cpp): cover authorization precedence
41 lines
1.2 KiB
Markdown
41 lines
1.2 KiB
Markdown
---
|
|
summary: "Managed and external llama.cpp servers for GGUF chat and embeddings."
|
|
read_when:
|
|
- You are installing, configuring, or auditing the llama-cpp plugin
|
|
title: "Llama Cpp plugin"
|
|
---
|
|
|
|
# Llama Cpp plugin
|
|
|
|
Managed and external llama.cpp servers for GGUF chat and embeddings.
|
|
|
|
## Distribution
|
|
|
|
- Package: `@openclaw/llama-cpp-provider`
|
|
- Install route: npm; ClawHub
|
|
|
|
## Surface
|
|
|
|
providers: `llama-cpp`, `llama-server`; contracts: `embeddingProviders`
|
|
|
|
<!-- openclaw-plugin-reference:manual-start -->
|
|
|
|
## Default text model
|
|
|
|
During interactive setup, OpenClaw installs a pinned, verified `llama-server`
|
|
and offers Gemma 4 E4B IT Q4_K_M as an approximately 5.0 GB download. The model
|
|
offer requires at least 16 GiB of total RAM. Existing cached models are still
|
|
detected on smaller machines.
|
|
|
|
To use another model, set `params.modelPath` to any custom GGUF. Custom models
|
|
are not subject to the bundled-download RAM requirement. On machines below the
|
|
requirement, you can also run a smaller model through Ollama or LM Studio, or
|
|
choose a cloud provider.
|
|
|
|
<!-- openclaw-plugin-reference:manual-end -->
|
|
|
|
## Related docs
|
|
|
|
- [llama-server](/providers/llama-server)
|
|
- [llama-cpp](/plugins/llama-cpp)
|