mirror of
https://github.com/openclaw/openclaw.git
synced 2026-08-14 06:33:09 -06:00
Peter Steinberger
1348387076
Move llama.cpp chat and local embeddings onto a verified externally managed llama-server runtime. Remove the in-process native runtime, forked embedding workers, and node-llama-cpp dependency while preserving guided setup, local GGUF models, tool-capable agent runs, diagnostics, and operator docs.
40 lines
1.1 KiB
Markdown
40 lines
1.1 KiB
Markdown
---
|
|
summary: "Managed local llama.cpp server for GGUF chat and embeddings."
|
|
read_when:
|
|
- You are installing, configuring, or auditing the llama-cpp plugin
|
|
title: "Llama Cpp plugin"
|
|
---
|
|
|
|
# Llama Cpp plugin
|
|
|
|
Managed local llama.cpp server for GGUF chat and embeddings.
|
|
|
|
## Distribution
|
|
|
|
- Package: `@openclaw/llama-cpp-provider`
|
|
- Install route: npm; ClawHub
|
|
|
|
## Surface
|
|
|
|
providers: `llama-cpp`; contracts: `embeddingProviders`
|
|
|
|
<!-- openclaw-plugin-reference:manual-start -->
|
|
|
|
## Default text model
|
|
|
|
During interactive setup, OpenClaw installs a pinned, verified `llama-server`
|
|
and offers Gemma 4 E4B IT Q4_K_M as an approximately 5.0 GB download. The model
|
|
offer requires at least 16 GiB of total RAM. Existing cached models are still
|
|
detected on smaller machines.
|
|
|
|
To use another model, set `params.modelPath` to any custom GGUF. Custom models
|
|
are not subject to the bundled-download RAM requirement. On machines below the
|
|
requirement, you can also run a smaller model through Ollama or LM Studio, or
|
|
choose a cloud provider.
|
|
|
|
<!-- openclaw-plugin-reference:manual-end -->
|
|
|
|
## Related docs
|
|
|
|
- [llama-cpp](/plugins/llama-cpp)
|