* [AI] fix(diagnostics-otel): keep telemetry exporting across in-process restarts
Owned diagnostics-otel generations previously registered their OpenTelemetry
providers globally and only shut the providers down on stop. The pinned
sdk-node registers globals without override and never unregisters them, so a
second in-process generation (config-watcher reload, OPENCLAW_NO_RESPAWN=1,
containers) kept exporting through the first, already-shutdown providers and
telemetry silently stopped.
Switch owned mode to private BasicTracerProvider + MeterProvider instances
whose handles are injected directly into the existing recorder runtime, and
keep the preloaded (OPENCLAW_OTEL_PRELOADED=1) mode on the host's global
providers without ever registering or replacing globals. Resource detection
honors the pinned NodeSDK OTEL_NODE_RESOURCE_DETECTORS contract (unset
defaults to env+process+host; none/subset/all are respected), and the unused
@opentelemetry/sdk-node dependency is removed with a minimal lockfile update.
Adds a real-SDK two-generation restart regression, real-SDK resource-detector
selection coverage, migrates the unit mocks to the provider lifecycle, and
updates the OpenTelemetry docs.
Fixes#119997
Co-Authored-By: glm-5.2 <noreply@anthropic.com>
* [AI] test(diagnostics-otel): restore OpenTelemetry globals after restart test
The restart regression disables global context, metrics, propagation, and
trace APIs and forces OPENCLAW_OTEL_PRELOADED=0, but its cleanup only reset
diagnostic events, so a later test in the same worker could inherit no-op
providers. Snapshot the prior global registrations and preloaded env at
module load, and re-register/restore them in afterEach, matching the
existing exporter-health integration test pattern.
Co-Authored-By: glm-5.2 <noreply@anthropic.com>
* [AI] fix(diagnostics-otel): reconcile private providers with OTEL_SDK_DISABLED
Rebase onto main after #119961 (honor OTEL_SDK_DISABLED) changed the
disabled admission path to return before SDK construction. Restore the
removed getBooleanFromEnv import, drop the stale ownedNodeSdkDisabled
guard, and update disabled-mode tests to the new semantics (all routes
off, disabled runtime registered) plus the private-provider signal
path for the integration suite.
Related to #119997
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(diagnostics-otel): isolate restartable provider generations
* test(diagnostics-otel): prove same-pid generation routing
* test(diagnostics-otel): use supported watcher reload mode
* test(diagnostics-otel): satisfy ownership proof gates
* test(diagnostics-otel): use managed proof cleanup
* test(diagnostics-otel): verify injected trace ancestry
---------
Co-authored-by: glm-5.2 <noreply@anthropic.com>
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
Emit usage once at the shared plugin completion boundary and expose host-derived plugin identity only on OTel usage spans. Preserve aggregate metric labels and cover the redacted Gateway path end to end.
Refs #98968
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
Co-authored-by: ZengWen-DT <ceng.wen@xydigit.com>
* fix(diagnostics-otel): harness.run root span adopts diagnostic trace context
Fall back to internalOrTrustedParentContext when there is no active
tracked parent, so the openclaw.harness.run root span (and its whole
OTel subtree) adopts the diagnostic traceId instead of minting a new
OTel root. This lets spans propagated via the diagnostic traceparent
(e.g. a2a-gateway a2a.call / downstream a2a.task) share one trace with
the run internals.
* fix(diagnostics-otel): propagate exported span context
Co-authored-by: HughTang <44694613+HughTang@users.noreply.github.com>
* fix(diagnostics): break propagation import cycle
* fix(diagnostics): preserve trace propagation contracts
---------
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
Keep shared endpoint routing signal-aware while preserving explicit signal-specific URLs exactly. Cover the resolver through mocked constructors and real OTLP/HTTP trace, metric, and log exporters.
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
Diagnostics exporters are plugin services that only ever start inside the
Gateway process, so embedded `openclaw agent --local` turns emitted
diagnostic events in the CLI process with no subscriber: zero spans,
metrics, or logs reached the OTLP collector while identical Gateway-
dispatched turns exported fine.
The CLI now starts the `diagnostics-otel` service around embedded agent
runs via a scoped, non-activating plugin load that honors the same
plugin-enablement config as the Gateway, and flushes before exit: drain
the async diagnostic-event queue, then shut the SDK down (force flush).
Each step is bounded separately (5s drain, 10s flush) so an unreachable
collector cannot hold the CLI open, and a stalled drain cannot consume
the flush window and discard telemetry that was already buffered.
When OTel is configured but the plugin is absent, the run now says so
instead of exporting nothing with nothing explaining why.
`diagnostics-prometheus` stays Gateway-only: it is a pull-based scrape
server that is useless in a short-lived process and would race the
Gateway for its port. No new config surface; configs without OTel enabled
short-circuit before any plugin load.
Co-authored-by: Alex Knight <15041791+amknight@users.noreply.github.com>
* feat(gateway): advertise chat attachment limits on hello-ok
Clients had no way to learn the gateway attachment ceilings, so external
clients hardcoded guesses that drifted from server enforcement. Publish the
two unconditional decoded-size ceilings on hello-ok policy.attachments from
one shared resolver so advertised values cannot drift from the parser.
MIME acceptance and per-message counts stay server-side: they depend on the
entrypoint, the resolved model, and payload sniffing, so they cannot be stated
once per connection.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 17d6c355-8948-4b48-a936-e08b1c8806ef
* feat(gateway): advertise chat attachment limits on hello-ok
---------
Co-authored-by: Omar Shahine <10343873+omarshahine@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: clawsweeper <274271284+clawsweeper[bot]@users.noreply.github.com>
Copilot-Session: 17d6c355-8948-4b48-a936-e08b1c8806ef
* docs(gateway): document llamacpp toolSchemaProfile for custom llama.cpp endpoints
Custom openai-completions providers pointed at a llama.cpp / llama-server
endpoint bypass the GBNF-safe schema normalization that built-in llama-cpp,
ollama, and lmstudio providers apply automatically. The toolSchemaProfile:
"llamacpp" switch activates it, but the supported value was never named in
the docs, so users hit llama.cpp 400 'failed to parse grammar' on tools with
large minLength/maxLength (e.g. cron trigger.script, maxLength 65536).
- Expand the toolSchemaProfile row to name the values recognized on a model's
compat (llamacpp, gemini) and when llamacpp must be set explicitly;
cross-reference the unsupportedToolSchemaKeywords escape hatch.
- Add a 'Local models (llama.cpp / llama-server)' provider example showing
the required compat.toolSchemaProfile: "llamacpp" on a custom endpoint.
Related to #117070
Co-Authored-By: iCodeMate <noreply@anthropic.com>
* docs(gateway): describe llamacpp profile as stripping, not clamping
Address review feedback on #117120: the fallback sentence claimed the
llamacpp profile "also clamps" constraints, but cleanSchemaForLlamacppGbnf
only removes `pattern` outright and drops `maxLength` at or above the
2000-repetition threshold (LLAMACPP_GBNF_MAX_REPETITION_THRESHOLD). It
never clamps a value, and small bounds like `minLength: 1` are preserved
(confirmed by the projection test at src/plugin-sdk/provider-tools.test.ts).
Reword the escape-hatch sentence to describe the implemented stripping
behavior and contrast it with the unconditional keyword-list drops, so
operators do not expect a preserved bounded constraint.
No code or behavior changes; documentation only.
* docs(gateway): scope llamacpp profile description to its actual transformations
Address follow-up review on #117120: the toolSchemaProfile table row
described `llamacpp` as "rewriting tool schemas into the JSON Schema
subset llama.cpp can compile to GBNF," which overstates what
cleanSchemaForLlamacppGbnf does. The cleaner only removes `pattern`
outright and drops `maxLength` at or above the 2000-repetition threshold
(LLAMACPP_GBNF_MAX_REPETITION_THRESHOLD); it leaves every other keyword
and `minLength` untouched, and those are not guaranteed GBNF-safe.
Reword the row to state the exact transformation contract and explicitly
note it is a targeted compatibility cleaner, not a blanket GBNF-safety
guarantee. No code or behavior changes; documentation only.
Co-Authored-By: iCodeMate <noreply@anthropic.com>
* docs(gateway): tighten llama.cpp profile contract
* docs(gateway): order local model examples
---------
Co-authored-by: iCodeMate <noreply@anthropic.com>
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
* docs(config-channels): replace legacy allow:true with enabled:true for Discord, Slack, and Google Chat config examples
* docs(config-channels): repair channel configuration examples
Make the complete Discord, Slack, and Google Chat setup examples valid and keep Slack channel routing on stable IDs.
Co-authored-by: 赵旺0668001248 <zhao.wang1@xydigit.com>
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
The gateway.reload documentation advertised four settings the shipped
build rejects. gateway.reload is a strictObject accepting only `mode`,
and `mode` is a union of "off" | "hybrid", so a config written from the
docs fails validation and the gateway fails closed on the unknown keys.
Retired but still documented:
- `debounceMs` and `deferralTimeoutMs` (RETIRED_TUNING_PATHS; stripped by
`openclaw doctor --fix` via stripRetiredTuningKnobs)
- `mode: "restart"` and `mode: "hot"` (mapped to "hybrid" by
`openclaw doctor --fix`)
Correct all four docs pages that still describe them, and say what
replaced them: debounce and active-work deferral now run behind built-in
defaults rather than config keys.
Refs #116973
* docs(automation): rename scheduled-tasks feature wording to Automations
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WhJ8EiMXue6ADLmHfb7FL6
* docs: regenerate docs map and add Automations glossary entries
* docs(templates): follow renamed automations-vs-heartbeat anchor
* docs(automation): fix markdown formatting drift
* docs(automation): teach the canonical automations tool and sync the copied heartbeat default
Review follow-ups: normal instructions use the automations tool with cron as
an explicit compatibility alias; every verbatim copy of the default heartbeat
prompt matches the new shipped text from the strings PR.
---------
Co-authored-by: Omar Shahine <10343873+omarshahine@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Peter Steinberger