Commit Graph

17 Commits

Author SHA1 Message Date
Peter Steinberger 234df15a6d chore: refresh dependencies after seven-day cooldown (#128414)
* build(deps): refresh dependencies after cooldown

Apply dependency, toolchain, action, image, and exact tool updates released by the inclusive 2026-08-16 seven-day cutoff. Adapt owner boundaries for the resulting CUA, logging, Teams, Markdown, native, and test-harness contract changes while retaining versions blocked by upstream compatibility constraints.

* fix(ui): align markdown renderer env typing

* fix(deps): align postcss and mistral peer contracts

* fix(deps): repair refreshed dependency contracts

* fix(deps): retain tslog startup budget

* fix(ci): verify Android tools with SHA-256

* fix(ci): fence Android SDK cache version
2026-08-24 03:01:54 -07:00
Jesse Merhi 0e8faacd71 fix(scripts): build heap ignores its systemd memory budget and takes the full default (#123979)
* fix(scripts): size the tsdown heap from the build's own cgroup budget

The build heap probe only read the cgroup root (/sys/fs/cgroup/memory.max and
the v1 equivalent). Those files exist only when the process runs in a
namespaced container cgroup; under systemd the budget lives on the process's
own slice, and the v2 root carries no limit at all. So every systemd-managed
build found no limit, fell back to /proc/meminfo MemTotal, and took the full
12288 MB default heap regardless of its actual budget.

Observed on a 15.4 GiB host: openclaw-main-update.service ran tsdown with
NODE_OPTIONS=--max-old-space-size=12288 while its user@999.service slice was
bounded at 5 GiB, reaching 3.2 GB RSS and 6.25 GB peak before the host began
OOM-killing unrelated services.

Resolve the limit from /proc/self/cgroup and walk that chain instead, reading
memory.high alongside memory.max (memory.high throttles reclaim rather than
failing allocation, so a heap above it stalls the build instead of OOM-ing),
and take the tightest bound found. Root paths stay as the container fallback,
and an explicitly injected path list still disables detection.

Co-authored-by: jesse-merhi <79823012+jesse-merhi@users.noreply.github.com>

* fix(scripts): resolve the build heap budget from the v1 memory controller too

The slice walk only accepted the unified 0:: record, so a legacy or hybrid
systemd host fell back to the root probe and kept taking host memory. One
resolver now walks both hierarchies leaf-to-root, which makes the static root
list its own depth-0 case and removes it.

Co-authored-by: jesse-merhi <79823012+jesse-merhi@users.noreply.github.com>

* fix(scripts): read cgroup controller mounts instead of assuming their paths

v1 controllers can be co-mounted at the cgroup root, where memory.limit_in_bytes
sits under the slice with no per-controller directory, so the hardcoded
/sys/fs/cgroup/memory probe missed the budget and the build took the full
12288MB default. Mount points now come from mountinfo.

Co-authored-by: jesse-merhi <79823012+jesse-merhi@users.noreply.github.com>

* fix(scripts): translate cgroup records through the mount root

mountinfo field 4 is the subtree a cgroupfs mount exposes. Under a container
mount the /proc/self/cgroup record stays host-absolute, so walking it verbatim
probed paths below the visible mount and the build fell back to host memory.
Records now translate through the mount root before the walk.

Co-authored-by: jesse-merhi <79823012+jesse-merhi@users.noreply.github.com>

* fix(scripts): skip cgroup mounts that cannot represent this process

Falling back to the mount root for a record outside the mount's subtree sized
the build from an unrelated cgroup: an inherited namespace clamped the heap to
the 2048MB floor from a foreign 1GiB limit. Non-representable mounts are now
skipped, and the blind root probe only runs when no memory record exists.

Co-authored-by: jesse-merhi <79823012+jesse-merhi@users.noreply.github.com>

* fix(scripts): keep every cgroup mount view, not just the last one seen

One hierarchy can be visible through several mounts and only some expose a
subtree containing this process. Retaining only the last view dropped the
budget whenever a non-representable bind view came later, sending the build
back to host MemTotal.

Co-authored-by: jesse-merhi <79823012+jesse-merhi@users.noreply.github.com>

* fix(scripts): decode octal-escaped mountinfo paths before matching cgroups

ClawSweeper P2 on 7e64ad61f7: the cgroup resolver compared mountinfo's mount
root and mount point verbatim. The kernel escapes space, tab, newline, and
backslash in those two fields, so any cgroup mounted under such a path never
matched, the bounded slice was missed, and heap sizing silently fell back to
host memory.

Decode both fields before matching. The decoder lives in scripts/lib beside the
other shared script helpers rather than inline, so the scripts program has one
copy rather than a new ad hoc one.

Regression test fails pre-fix: a v2 mount at "/sys/fs/cgroup\040dir" with a
5 GiB memory.high yields --max-old-space-size=12288 (host fallback) before the
fix and 4352 after.

Follow-up, deliberately not bundled here: src/infra/sqlite-wal.ts,
src/commands/doctor-state-integrity.ts, and src/plugins/bundled-source-overlays.ts
each carry their own private copy of this same decoder. Consolidating all four
into @openclaw/normalization-core is the right end state, but it touches a
shared package plus three core modules and belongs in its own reviewable change.

Co-authored-by: jesse-merhi <79823012+jesse-merhi@users.noreply.github.com>

* fix(scripts): resolve cgroup-namespace-relative records to their mount

ClawSweeper P1 on d6fe49dd3f: inside a cgroup namespace /proc/self/cgroup
reports the namespace root ("0::/") while mountinfo field 4 stays the host
subtree the cgroupfs was mounted from ("/docker/<id>"). relativeCgroupPath then
found no prefix match and returned null; because a memory record had already
been seen, the root probe was skipped and the build fell back to host MemTotal.
A constrained container therefore missed its own budget entirely.

That namespace root is exactly what the mount exposes at its mount point, so it
resolves to "/" rather than failing closed.

Regression test fails pre-fix: a "0::/" record against a /docker/2f1a9c mount
root with a 5 GiB memory.max yields --max-old-space-size=12288 before the fix
and 4352 after.

Co-authored-by: jesse-merhi <79823012+jesse-merhi@users.noreply.github.com>

* fix(scripts): reject inherited cgroup mount views instead of guessing

ClawSweeper P1 on b4d200c5d2: the previous commit resolved a namespace-relative
record against any mount root, including the inherited views cgroup_namespaces(7)
documents, whose field-4 root reads "/..". Which cgroup such a view exposes is not
derivable from mountinfo, so probing it can size the build from an unrelated
cgroup's limit.

Reject non-canonical mount roots outright. An undecidable view now falls back to
host sizing, which is current main's behavior, rather than silently adopting the
wrong budget.

Regression test covers the "/.." inherited mount: it must yield host MemTotal
sizing, not the 5 GiB limit sitting behind that mount.

Co-authored-by: jesse-merhi <79823012+jesse-merhi@users.noreply.github.com>

* fix(scripts): fail closed on namespace-root records against non-root mounts

ClawSweeper P1 on 731d3bbc8e: a "0::/" record does not prove that a mount
rooted at some other subtree exposes this process's cgroup. Resolving that pair
could cap the build heap from an unrelated cgroup's limit.

Return no mapping for it. An undecidable pair now falls back to host sizing,
which is current main's behavior, so the failure mode is a missed optimisation
rather than a wrong budget. The "/.." inherited-mount rejection stays; this
covers the broader ambiguous mapping it did not.

The namespace-relative test is repointed accordingly: an unrelated mounted
subtree must yield host sizing, not that subtree's limit.

Net production change: none (4 lines swapped).

Co-authored-by: jesse-merhi <79823012+jesse-merhi@users.noreply.github.com>

* fix(build): cap tsdown heap to the real budget and refuse hosts that cannot build

The 2048MB floor was applied on top of a discovered cgroup limit, so a small
container was handed a heap larger than it could honour. Measured in real
cgroups, that does not OOM-kill, it thrashes: a 1500MiB container sat pinned at
its ceiling for 10 minutes with oom_kill at 0, never finished the second of
eleven invocations, and starved every other process on the host.

Cap to the discovered budget, then refuse up front when that budget cannot hold
the build. The threshold is the whole-build peak, not a single pass: a full
eleven-invocation build peaks at 4730MiB, so a 5GiB slice completes while 4GiB
and 2816MiB slices are both killed partway through the third invocation.

The refusal runs before any output is cleaned, so a host that cannot rebuild
does not also lose the build it has.

* fix(build): harden tsdown heap admission

* fix(build): guard the default tsdown plan

* fix(build): preserve runtime-only Docker builds

* fix(build): admit only declaration cache misses

* fix(build): scope heap admission to real budgets

* fix(build): guard direct unified declarations

* fix(build): guard the canonical tsdown config

* fix(build): satisfy cache planning lint

* fix(gateway): release empty orphan leases

* fix(build): cap cgroup budget by host memory

* fix(build): serialize the canonical tsdown config

* test(build): freeze host memory fixtures

* fix(build): honor cgroup v1 soft limits

* fix(build): respect cgroup v1 hierarchy mode

* fix(build): admit unified runtime plans

* fix(build): admit every unified runtime path

* fix(build): collect repeated tsdown filters

* fix(build): ignore cgroup v1 soft limits

* fix(build): use explicit heap override as opt-in

* refactor(build): simplify memory admission

* fix(build): harden constrained build recovery

* fix(ci): prebuild runtime before real CLI shards

* fix(build): honor runtime-only runner environment

* fix(ci): satisfy tooling shard lint
2026-08-24 14:18:48 +10:00
Peter Steinberger 9a947e4bb8 fix(sessions): flatten Markdown in session list previews (#125113)
* fix(sessions): flatten Markdown in session list previews

Session-list previews were extracted verbatim from the last transcript
message, so raw Markdown leaked into every surface that renders the
subtitle as plain text — Control UI sidebar, TUI picker, native session
lists, and the sessions_list tool. A finished session read as
"Landed [PR #124879](https://github.com/...)".

Flatten lastMessagePreview at the Gateway producer, inside the
watermark-validated title-field cache, so the cost is amortized and no
consumer re-implements stripping. The flattener is the regex chain that
already existed privately in the Control UI narration line; it moves to
@openclaw/normalization-core/markdown-plain-text and both surfaces now
share one implementation. Session titles keep their own normalization
and are unchanged.

Also let an unread final observer digest outrank the raw last reply in
the sidebar subtitle, so the utility model's headline wins the slot it
was written for. The finalDigestUnread gate is untouched, so an
already-read digest still falls back to the flattened preview.

* fix(ci): register Markdown preview module

* fix(sessions): preserve literal preview punctuation

* fix(sessions): flatten previews before truncation

* fix(test): adapt session projection callback
2026-08-17 00:40:16 -07:00
Peter Steinberger c23d66e3b5 refactor: consolidate coercion ownership (#122692)
* refactor: consolidate coercion ownership

* test: align shard check with weighted planning

* chore: refresh plugin SDK API baseline
2026-08-12 09:25:28 -07:00
Peter Steinberger 1220a7609a refactor: consolidate promise-like guards (#121861)
* feat(normalization-core): add promise-like guard

* refactor: consolidate promise-like guards

* fix(normalization-core): keep isPromiseLike non-throwing on hostile then getters

ClawSweeper finding on #121861: the diagnostics-path local guard caught throwing
then getters; the canonical guard must classify, never throw.

* test(normalization-core): annotate intentional hostile-thenable fixture
2026-08-10 22:50:48 -07:00
Peter Steinberger 33ea3e16e9 refactor: consolidate core micro-helpers (#117825)
* refactor: centralize stable stringification

* refactor: reuse canonical record coercion

* refactor: reuse safe JSON parsing in cron storage

* refactor: centralize environment truthiness

* fix: enforce model scan and block reply timeouts

* refactor: consolidate signal-aware sleep helper

* fix: preserve plugin SDK sleep contract

* test: satisfy model scan timeout lint
2026-08-01 23:10:46 -07:00
Vincent Koc b3ce00b480 refactor(schema): share config validation runtime (#116622) 2026-07-31 12:40:48 +08:00
loong 7bdd1f646b fix(memory-host-sdk): resolve stable execPath for worker fork to survive Homebrew Node upgrades (#99318)
* fix(memory): survive Homebrew Node upgrades

Co-authored-by: 袁龙辉0668001277 <yuan.longhui@xydigit.com>

* test(whatsapp): isolate last-route coverage

* fix(deepinfra): preserve offline model compatibility

* test(memory): isolate migration cleanup lifecycle

* test: stabilize aggregate extension gates

* ci: retrigger pull request workflow

* test: preserve inherited Node options

* test(cli): tolerate cold hosted startup

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-29 06:49:02 -04:00
qingminlong a01319b27f fix: account for CJK text in compaction estimates (issue #103930) (#114386)
* fix: account for CJK text in compaction estimates

* fix: share CJK compaction estimator

* fix: format CJK compaction estimator

* fix: make CJK regexes lint-safe

Co-authored-by: qingminlong <qing.minlong@xydigit.com>

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-07-27 08:51:44 -04:00
Peter Steinberger f0c43dcf72 feat(ui): format international phone numbers for display (#112400)
* feat(ui): format phone numbers for display

* ci: track normalization package exports in Knip
2026-07-21 17:10:25 -07:00
Peter Steinberger 2aa4ff0825 refactor: consolidate duplicate helper exports (#106016)
* refactor(normalization): consolidate string helpers

* refactor(normalization): consolidate agent ids

* refactor(paths): consolidate user path resolution

* refactor(gateway): preserve transcript helper facade

* fix(normalization): align helper build aliases

* fix(memory): keep helper import line neutral

* chore(plugin-sdk): align consolidated helper surface

* refactor(normalization): reuse lowercase string helper

* refactor(normalization): reuse record guard

* refactor(normalization): share surrogate boundary helper

* refactor(agents): share token estimate constant

* refactor(memory): distinguish standalone helper contracts

* refactor(normalization): share error cause formatter

* refactor(config): distinguish eligibility predicates

* refactor(plugins): share registry state symbol

* refactor(cli): reuse outbound dependency adapter

* refactor(cron): distinguish positive duration parser

* fix(gateway): keep transcript helper private

* fix(paths): preserve public helper status

* refactor(channels): reuse registry normalizer

* refactor(agents): reuse agent core runtime facade

* refactor(auth): reuse locked profile upsert

* refactor(records): distinguish trap-safe guard

* refactor(migrations): distinguish sync directory helper

* test(plugins): drop stale duration alias expectations

* fix(channels): remove stale registry import

* chore(plugin-sdk): refresh helper API baseline

* fix(memory): keep renamed helpers private

* fix(plugins): keep registry state key private

* fix(plugin-sdk): tighten wildcard surface budget

* chore(plugin-sdk): refresh rebased helper API baseline
2026-07-13 03:22:00 -07:00
Peter Steinberger 6df498c3cd refactor: canonicalize binary operation results (#106049)
* refactor(normalization-core): promote shared result type

* refactor(agent-core): reuse canonical error coercion

* refactor: discriminate internal operation results

* docs: establish shared result convention

* fix(agent-core): preserve error coercion facade

* build: derive package entries from exports

* fix(plugins): resolve shared result subpath

* fix(ci): align Result integration guards

* build: refresh plugin SDK API baseline

* fix(agent-core): preserve toError compatibility
2026-07-13 00:39:43 -07:00
Peter Steinberger e686693732 feat(tooling): adopt noUncheckedIndexedAccess via a strict-ratchet lane (phase 1) (#104577)
* feat(tooling): add noUncheckedIndexedAccess strict-ratchet lane

* fix(packages): make indexed access explicit across ratchet packages

Burns down all 65 noUncheckedIndexedAccess errors in the seven ratchet
packages with behavior-identical restructuring (iteration, charAt/slice,
regex-group guards, validated hextet tuple). net-policy invariant
violations now throw instead of failing open. Adds
@openclaw/normalization-core/expect (expectDefined/first/last) with
subpath export and tests. memory-host-sdk fixes kept but the package
stays out of the lane: it re-exports core src/** so the flag would apply
transitively to all of core.

* fix(lint): keep .oxlintrc.json strict-JSON parseable for extension lint wrappers
2026-07-11 10:26:12 -07:00
Peter Steinberger 062f88e3e3 refactor: extract reusable AI runtime package (#99059)
* refactor: extract reusable AI runtime package

* refactor: complete AI provider relocation

* refactor: keep llm core internal

* refactor(ai): make @openclaw/ai self-contained with host policy ports

Move pure transport helpers (tool projections, strict-schema normalization,
prompt-cache boundary, stream guards, anthropic/openai compat, request
activity) from src into packages/ai; move utf16-slice into
normalization-core. Inject host policy (guarded fetch, redaction,
strict-tool defaults, diagnostics logging) through AiTransportHost with
inert library defaults installed by src/llm/stream.ts. Narrow the public
barrel to instance-scoped createApiRegistry/createLlmRuntime; the
process-default runtime moves behind internal/ and
registerBuiltInApiProviders takes an explicit registry. Delete the
src/llm/api-registry re-export facade.

* fix(ai): teach node, jiti, and vite resolvers the @openclaw/ai and utf16-slice subpaths

The workspace alias tables in root-alias.cjs, plugin-sdk-native-resolver,
sdk-alias, the shared vitest config, and the Control UI vite config only
knew @openclaw/llm-core; Node-side plugin loading resolved @openclaw/ai
through the pnpm symlink to the unbuilt dist (checks-node-compact CI
failures), and the Control UI build broke on the new
normalization-core/utf16-slice subpath.

* chore(ui): drop leftover service-worker debug logging

* build(release): ship @openclaw/ai with its own shrinkwrap and honest dependency set

packages/ai declares only its six real runtime deps (kysely, chalk, json5,
tslog, zod, fs-safe, and proxyline were never imported); orphaned root deps
removed. generate-npm-shrinkwrap now treats publishable packages/* like
publishable plugins so the AI tarball pins its transitive tree even though
workspace deps are omitted from the root shrinkwrap. knip learns the
package entry points; the tsdown dts neverBundle option moves to its
documented deps.dts home; the README documents the no-semver internal/*
contract and host ports.

* docs(ai): add minimal external-consumer example app

examples/ai-chat consumes only the public @openclaw/ai surface (built dist
via the workspace link): isolated runtime, built-in provider registration,
one streamed completion. Supports Anthropic/OpenAI via env keys and a
keyless local Ollama target; live-verified against Ollama.

* docs(ai): document the @openclaw/ai package and workspace shrinkwrap boundary

* chore(check): include examples/ in duplicate-scan targets

* fix: emit normalization package subpaths

* fix: complete AI package boundary artifacts

* fix: align AI package boundary contracts

* fix(ci): stabilize package release contracts

* test: align documentation contract checks

* test: keep cron docs guard aligned

* test: align restored docs contract guards

* test: follow upstream docs contracts

* docs: drop superseded talk wording
2026-07-05 01:56:40 -04:00
Peter Steinberger 49cc59b1e8 refactor: consolidate markdown code fences, error coercion, and byte-identical helper pairs (#99932)
* refactor(shared): add markdown code span/fence helpers and migrate seven call sites

* refactor(normalization-core): add canonical toErrorObject and migrate six copies

* refactor: consolidate byte-identical helper pairs onto owner modules

* refactor: reuse canonical path and token helpers in session tools and credentials

* refactor(packages): dedupe compaction summarization tail and session dir parsing

* fix(security): keep missing extensions dir silent in shared plugin dir lister

* refactor: reuse media-core chunk reader and shared dreaming session key helper

* fix(plugins): register error-coercion subpath in sdk alias table

* chore(plugin-sdk): re-pin callable export count after rebase

* chore(plugin-sdk): re-pin callable export count after rebase
2026-07-04 08:40:41 -04:00
Dallin Romney febc70036f refactor: consolidate exact boolean coercion (#99750)
* refactor: consolidate exact boolean parsing

* test: fix normalization subpath resolution

* fix: resolve normalization boolean subpath
2026-07-03 19:42:06 -07:00
Peter Steinberger 00d8d7ead0 refactor: extract normalization core package
Extract shared normalization/coercion helpers into private @openclaw/normalization-core workspace package while preserving existing plugin SDK helper subpaths.\n\nAlso keeps direct normalization-core imports internal, wires UI/build/loader resolution, and replaces the slow PR network CodeQL lane with a fast added-line boundary scan while retaining full CodeQL for scheduled/manual runs.\n\nVerification: local moved tests, plugin SDK boundary tests, extension loader tests, agents-support shard, UI build/test, build artifacts, lint, workflow guards, autoreview, and GitHub CI passed on PR head 963d893715.
2026-05-31 01:33:00 +01:00