mirror of
https://github.com/openclaw/openclaw.git
synced 2026-08-27 04:47:03 -06:00
docs(memory): explain llama.cpp runtime diagnostics
This commit is contained in:
@@ -47,6 +47,11 @@ model. `local.modelCacheDir` overrides where downloaded models are cached
|
||||
(default: `~/.node-llama-cpp/models`), and `local.contextSize` accepts an
|
||||
integer or `"auto"`.
|
||||
|
||||
When `local.contextSize` is numeric, the provider also gives that requirement
|
||||
to node-llama-cpp's automatic GPU-layer placement. This lets node-llama-cpp fit
|
||||
the model and embedding context together while retaining its memory-safety
|
||||
checks. With `"auto"`, node-llama-cpp keeps its normal automatic placement.
|
||||
|
||||
## Native Runtime
|
||||
|
||||
Use Node 24 for the smoothest native install path. Source checkouts using
|
||||
@@ -57,6 +62,18 @@ pnpm approve-builds
|
||||
pnpm rebuild node-llama-cpp
|
||||
```
|
||||
|
||||
## Runtime diagnostics
|
||||
|
||||
Run `openclaw memory status --deep` after the provider has loaded to inspect
|
||||
the selected backend and build, device names, GPU offloaded layers, requested
|
||||
context size, and the last observed VRAM or unified-memory snapshot. The VRAM
|
||||
values include an observation timestamp because passive status reads do not
|
||||
reload the model or poll the device.
|
||||
|
||||
The same last-known facts can appear in `openclaw doctor` when the running
|
||||
Gateway has already used the local provider. A normal status or doctor command
|
||||
does not load a model just to collect diagnostics.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
If `node-llama-cpp` is missing or fails to load, OpenClaw reports the failure
|
||||
|
||||
@@ -292,6 +292,8 @@ Use `provider: "openai-compatible"` for a generic OpenAI-compatible
|
||||
openclaw memory index --force --agent main
|
||||
```
|
||||
|
||||
Numeric `local.contextSize` values also inform node-llama-cpp's automatic GPU-layer placement so model weights and the requested embedding context are fitted together. `openclaw memory status --deep` reports last-known llama.cpp backend, device, offload, requested-context, and timestamped memory facts after the runtime has loaded; passive status does not load a model.
|
||||
|
||||
Set `provider: "local"` explicitly for local GGUF embeddings. `hf:` and HTTP(S) model references are supported for explicit local configs (via node-llama-cpp's model resolution), but they do not change the default provider.
|
||||
|
||||
</Accordion>
|
||||
|
||||
Reference in New Issue
Block a user