* refactor: consolidate coercion ownership
Centralize four canonical coercion helpers, migrate exact core and plugin duplicates through narrow Plugin SDK facades, and enforce declaration and plugin-normalization ownership boundaries.
The sweep adds eight focused SDK exports while deleting more production and tooling code than it adds. User-visible behavior is unchanged except for safer equivalent object and UI parsing at existing boundaries.
* fix: guard integer option ownership
Register resolveIntegerOption with the canonical function owner and extend the declaration-guard fixture so future local duplicates fail validation.
* fix: keep integer helpers on numeric facade
Remove the unshipped duplicate string-coerce exports and route every affected plugin consumer through the existing number-runtime contract.
* fix: point numeric coercion to number runtime
Make boundary and declaration diagnostics recommend the canonical numeric facade, with failing-before coverage for both guidance paths.
* [AI] fix(diagnostics-otel): keep telemetry exporting across in-process restarts
Owned diagnostics-otel generations previously registered their OpenTelemetry
providers globally and only shut the providers down on stop. The pinned
sdk-node registers globals without override and never unregisters them, so a
second in-process generation (config-watcher reload, OPENCLAW_NO_RESPAWN=1,
containers) kept exporting through the first, already-shutdown providers and
telemetry silently stopped.
Switch owned mode to private BasicTracerProvider + MeterProvider instances
whose handles are injected directly into the existing recorder runtime, and
keep the preloaded (OPENCLAW_OTEL_PRELOADED=1) mode on the host's global
providers without ever registering or replacing globals. Resource detection
honors the pinned NodeSDK OTEL_NODE_RESOURCE_DETECTORS contract (unset
defaults to env+process+host; none/subset/all are respected), and the unused
@opentelemetry/sdk-node dependency is removed with a minimal lockfile update.
Adds a real-SDK two-generation restart regression, real-SDK resource-detector
selection coverage, migrates the unit mocks to the provider lifecycle, and
updates the OpenTelemetry docs.
Fixes#119997
Co-Authored-By: glm-5.2 <noreply@anthropic.com>
* [AI] test(diagnostics-otel): restore OpenTelemetry globals after restart test
The restart regression disables global context, metrics, propagation, and
trace APIs and forces OPENCLAW_OTEL_PRELOADED=0, but its cleanup only reset
diagnostic events, so a later test in the same worker could inherit no-op
providers. Snapshot the prior global registrations and preloaded env at
module load, and re-register/restore them in afterEach, matching the
existing exporter-health integration test pattern.
Co-Authored-By: glm-5.2 <noreply@anthropic.com>
* [AI] fix(diagnostics-otel): reconcile private providers with OTEL_SDK_DISABLED
Rebase onto main after #119961 (honor OTEL_SDK_DISABLED) changed the
disabled admission path to return before SDK construction. Restore the
removed getBooleanFromEnv import, drop the stale ownedNodeSdkDisabled
guard, and update disabled-mode tests to the new semantics (all routes
off, disabled runtime registered) plus the private-provider signal
path for the integration suite.
Related to #119997
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(diagnostics-otel): isolate restartable provider generations
* test(diagnostics-otel): prove same-pid generation routing
* test(diagnostics-otel): use supported watcher reload mode
* test(diagnostics-otel): satisfy ownership proof gates
* test(diagnostics-otel): use managed proof cleanup
* test(diagnostics-otel): verify injected trace ancestry
---------
Co-authored-by: glm-5.2 <noreply@anthropic.com>
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
Emit usage once at the shared plugin completion boundary and expose host-derived plugin identity only on OTel usage spans. Preserve aggregate metric labels and cover the redacted Gateway path end to end.
Refs #98968
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
Co-authored-by: ZengWen-DT <ceng.wen@xydigit.com>
* fix(diagnostics-otel): harness.run root span adopts diagnostic trace context
Fall back to internalOrTrustedParentContext when there is no active
tracked parent, so the openclaw.harness.run root span (and its whole
OTel subtree) adopts the diagnostic traceId instead of minting a new
OTel root. This lets spans propagated via the diagnostic traceparent
(e.g. a2a-gateway a2a.call / downstream a2a.task) share one trace with
the run internals.
* fix(diagnostics-otel): propagate exported span context
Co-authored-by: HughTang <44694613+HughTang@users.noreply.github.com>
* fix(diagnostics): break propagation import cycle
* fix(diagnostics): preserve trace propagation contracts
---------
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
Keep shared endpoint routing signal-aware while preserving explicit signal-specific URLs exactly. Cover the resolver through mocked constructors and real OTLP/HTTP trace, metric, and log exporters.
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
* fix(diagnostics-otel): keep parent span links across long turns
Retained trusted span contexts translate a completed lifecycle span's
diagnostic span id to the real OTel span context. That table was evicted
5s after run.completed, so children of a longer turn missed their parent
and OTel minted a fresh trace id for each one, silently splitting a turn
into single-span traces.
Retention is now identity-based and lives for the service lifetime,
bounded by MAX_RETAINED_TRUSTED_SPAN_CONTEXTS, which removes the timing
race and the drain/timeout cleanup subsystem.
Also nests openclaw.exec under its run: the exec event now carries a
trusted trace context and the recorder resolves the ambient run scope.
* fix(diagnostics-otel): key retained span contexts by diagnostic trace id
Retained lifecycle span contexts were stored under the span's OTel trace id
but looked up with the event's diagnostic trace id. Those are different id
spaces, so every post-completion parent lookup missed and each straggler
span started a brand-new trace.
The mocked suite could not catch this: its tracer reports the same trace id
the test feeds in, collapsing both id spaces into one value. Adds a boundary
test that runs the real OTel SDK and OTLP/protobuf exporter against a local
receiver and asserts on exported span bytes.
* fix(diagnostics-otel): harden span-linking contract after review
Review findings on the parent-span linking fix:
- completeTrackedLifecycleSpan now takes the DiagnosticTraceContext instead of a
bare trace id beside a span id. Two adjacent hex strings let a transposition
compile clean and silently reinstate split traces; DiagnosticTraceContext is
the only shape an OTel SpanContext cannot satisfy (traceFlags string vs number).
- internalOrTrustedTraceContext composes normalizedTrustedTraceContext rather
than spelling the trust predicate a third time.
- Corrected comments that claimed OTel ids never equal diagnostic ids. Spans
parented from an upstream traceparent do adopt the diagnostic trace id; the
removed guard only ever misfired on root lifecycle spans.
- Exec comment now states the real parent: the openclaw harness opens no run
scope, so exec nests under openclaw.harness.run.
Test gaps closed:
- The core emitter switch was untested; reverting it kept every test green.
The pty-fallback test now asserts the emitted metadata.
- The boundary test covered only run.completed of three lifecycle owners; it now
covers harness.run.completed and message.processed too, each with a straggler.
- Dropped the service-context cast for the typed helper, which caught a bogus
context.assembled field, and moved teardown into afterEach so a failed
assertion cannot leak the real NodeSDK across files.
- Scoped the one-trace assertion to this turn's spans; faked Date so a lazy
Date.now expiry cannot pass.
Docs: exec and tool spans now listed as staying on an upstream request trace.
* fix(diagnostics-otel): keep late children on the trace on every terminator
Native review found three correctness gaps in the previous commit:
- Exec spans could be exported naming a parent span id nothing ever emits. The
ambient lookup fell through to a remote-parent fallback built from diagnostic
ids, which is a regression from the previous clean-root behavior and breaks
waterfalls and parent-id-keyed backends. Exec now resolves only spans this
process exported and stays a root on a miss; message spans keep the remote
fallback because their context can come from an inbound traceparent.
- harness.run.error never retained its span context, so an aborted turn still
split. For the openclaw harness that span is the only ancestor a late child
has, and aborted turns emit no run.completed.
- completeTrackedLifecycleSpan no longer takes a redundant spanId.
The boundary test now runs through the OPENCLAW_OTEL_PRELOADED seam with an
in-memory exporter and releases the global tracer provider in teardown. A
NodeSDK cannot be unregistered, so the old version would have made any later
real-SDK test silently export nothing. That also deletes the hand-rolled OTLP
protobuf decoder. Added guards for the aborted-turn path and for exec staying
parentless; both are mutation-verified.
* fix(diagnostics-otel): age retained parent contexts out after 10 minutes
Retention had no time bound at all, so a command backgrounded by a turn could
finish hours later and still attach to a parent that ended long before. Backends
derive trace latency from the span envelope, so a 30s turn rendered as hours with
a child extending past its parent.
The horizon is checked lazily on lookup rather than on a timer: a timer firing
mid-turn is exactly what silently split traces before. Past the horizon a
straggler starts its own trace, which only loses causality for a span that would
have skewed its parent's duration anyway.
The two bounds now pin each other - shrinking the horizon to the old 5s window
fails the original regression tests, and removing it fails the new horizon test.
* fix(diagnostics-otel): retain parent contexts until capacity
* build(deps): remove npm shrinkwrap; mirror pnpm lock into transient package locks
npm 12 removed shrinkwrap (command + tarball/root loading). Delete all 82
committed npm-shrinkwrap.json files and stop publishing lockfiles; keep
pnpm-lock.yaml as the single reviewed dependency boundary. The generator
becomes scripts/generate-npm-package-lock.mjs and feeds plugin bundling via
a transient package-lock.json + npm ci (works on npm 11 and 12). Tarball
validation treats the published 2026.7.2 beta train as a shrinkwrap
transition; self-update npm detection now uses install topology instead of
the shipped shrinkwrap.
* fix(deps): repair lint, deadcode, and test-type lanes for the npm 12 migration
- sort integrity comparisons with an explicit comparator (oxlint)
- keep resolveBunGlobalNodeModules module-local (knip unused-export gate)
- model npm pack --json as npm<=11 array / npm 12 name-keyed object
- default calver destructuring in the tarball test fixture
Model-call spans wrap outbound provider inference requests, so classify them as CLIENT independently of the optional GenAI naming convention. Generic tool spans remain unchanged because their shared events can represent local work.
Release note: operators filtering dashboards or alerts on span kind will see model-call spans move from INTERNAL to CLIENT.
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* improve(diagnostics): surface run startup phases as session-correlated events
The embedded runner reports ordered startup milestones (workspace, auth,
context_engine, ..., model_call_started) through the private
onExecutionPhase callback, so they reach typing indicators and cron
watchdogs but never the diagnostic bus. External status surfaces that
already consume run.attempt / run.progress / model.call.* cannot see
where a turn is during startup — exactly the window that matters when a
run hangs before its first model call.
Wrap params.onExecutionPhase once at the runner entry so every phase
transition also emits a session-correlated run.execution_phase event
(runId, sessionId, sessionKey, closed EmbeddedAgentExecutionPhase +
provider/model/tool context). The event rides the async diagnostic lane
like the other high-frequency lifecycle events, so ordering against
model.call.* is preserved and no subscriber work runs on the hot path.
The wrapper tracks session rotation via onSessionIdChanged so
post-compaction phases carry the current session id. Stability records
project the milestone into their dedicated phase/provider/model fields.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(diagnostics): preserve trusted phase observability
* chore(plugin-sdk): refresh diagnostic API baseline
* fix(diagnostics): publish execution phases to plugins
* refactor(logging): keep public diagnostic recorder path
* fix(plugin-sdk): keep deprecated budget exact
* fix(plugin-sdk): refresh diagnostic surface budget
* fix(diagnostics): preserve execution phase coverage
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
* fix(extensions): make indexed access explicit across channel plugins
Transport-payload-safe burn-down: malformed Telegram/Discord/QQ/LINE
and sibling channel input keeps existing skip paths; no synthesized
fields, no new throws in delivery loops. Zalo escape sentinels preserve
literal matches instead of undefined replacements.
* fix(extensions): make indexed access explicit across provider and memory plugins
Stream and model iteration, tool-block guards, capture guards, and
sparse accumulators; singleton model reads carry named invariants.
* fix(extensions): make indexed access explicit across tooling plugins, flip the extensions lane
Remaining plugins (oc-path, qa-lab, browser, logbook, and siblings) plus
the tsconfig.extensions.json flag flip. Cleanup: logbook sampleFrames
NaN index at max=1, QA retry clamp at non-positive attempts, dead Canvas
probe and OpenShell no-op slice removed, twitch test setup leak excluded
from the prod lane.
* refactor(plugin-sdk): expose expectDefined via a focused SDK subpath
Extensions imported @openclaw/normalization-core directly, crossing the
external-plugin packaging boundary (it only worked because the runtime
builder bundles undeclared workspace helpers). expect-runtime joins the
canonical entrypoints JSON, generated exports, API baseline, docs, and
subpath contract test; all 78 extension imports now use the SDK seam.
Two scanner-shaped locals renamed for review-bundle hygiene.
* chore(plugin-sdk): raise surface budgets for the expect-runtime subpath
One new entrypoint with one callable export, added intentionally as the
packaging-honest seam for extension invariant helpers.
* improve(diagnostics-otel): tune duration/context histogram bucket boundaries
The openclaw.run.duration_ms, openclaw.harness.duration_ms, and
openclaw.context.tokens histograms used the SDK default bucket
boundaries. The default duration buckets top out at 10s, so agent runs —
which routinely take minutes — all collapse into the +Inf overflow
bucket, making p95/p99 latency meaningless without collector-side
reconfiguration.
Add explicit_bucket_boundaries advice via named constants, matching the
existing GEN_AI_*_BUCKETS convention in this file:
- AGENT_DURATION_MS_BUCKETS (1s … 1h) for the run and harness duration
histograms, which share the same range.
- CONTEXT_TOKENS_BUCKETS on a token scale (1k … 2M). openclaw.context.tokens
records context-window limit/used token counts (e.g. 128000), so the
boundaries must cover real context sizes rather than small integers.
Adds a test asserting the three histograms advertise the expected
boundaries and that run/harness share the same set.
* fix(diagnostics-otel): preserve prior histogram buckets
---------
Co-authored-by: Harry Chen <zhiling.chen@binance.com>
Co-authored-by: Vincent Koc <25068+vincentkoc@users.noreply.github.com>
* fix: show exported tool results in trace viewers
* fix(diagnostics-otel): emit semconv response key and execute_tool identity so trace viewers show tool results
---------
Co-authored-by: Alex Knight <15041791+amknight@users.noreply.github.com>
* fix(diagnostics-otel): surface error message on run/harness error spans
Errored openclaw.run / openclaw.harness.run spans only carried a
low-cardinality errorCategory (or a hardcoded "error"), so trace UIs
showed "outcome error" with no message. Thread the redacted error
message through run.completed / harness.run.completed / harness.run.error
onto an openclaw.error span attribute + span status, mirroring
recordWebhookError. The raw message stays off metric attrs to preserve
cardinality; support-bundle redaction covers the new error field by name.
* fix(diagnostics-otel): harden run failure telemetry
Co-authored-by: Alex Knight <aknight@atlassian.com>
* test(diagnostics-otel): assert wire-level redaction
* chore(changelog): defer release note to release process
---------
Co-authored-by: Peter Steinberger <steipete@gmail.com>
Vincent Koc