perf(ci): serve node_modules snapshots from O(1) protected sticky disks (#109752)

* perf(ci): serve node_modules snapshots from O(1) protected sticky disks

Every Blacksmith mount of the dependency sticky disk currently 429s:
the v2 key minted one backing disk per PR and per manifest hash, which
saturated Blacksmith's installation-wide 1000-sticky-disk budget (run
29559333389, job checks-node-compact-small-15: "Sticky disk limit
exceeded"). Same-repo shards then fall back to a cold, storeless pnpm
install (~40s) on every run - worse than the ~22s actions/cache path
the sticky rollout replaced.

Re-key the snapshot to one stable disk per node-version and move the
install inputs into a runtime fingerprint marker:

- Key is now `<repo>-node-deps-bind-v3-<node-version>`; dependency
  changes refresh the disk in place instead of allocating a new one.
- The fingerprint (manifest hashFiles set + node version + lockfile
  mode) is evaluated on the bind step, before the mount lands, so the
  '**/package.json' glob cannot sweep snapshot-internal manifests.
- Consumers mount read-only (commit: false); on fingerprint match they
  restore importer archives and skip pnpm install entirely, on mismatch
  they install against the clone's on-disk store (warm store, no
  actions/cache download) without capturing.
- Writers are trusted non-PR jobs only: build-artifacts on canonical
  pushes plus the scheduled vitest-cache-warm run as a deadline writer
  (main pushes cancel each other under merge traffic and would starve
  the snapshot). commit: on-change keeps warm no-op runs from
  committing.
- Roll the consumer flag out to every Linux Blacksmith lane that
  installs dependencies (build-artifacts, check-shard,
  check-additional-shard, check-docs, checks-ui, control-ui-i18n,
  native-i18n, qa-smoke-ci-profile, checks-fast-core, both contract
  shards) with the same fork/dispatch gates as the nondist shard.

Guards now enumerate the consumer set, pin the O(1) key shape, enforce
single-writer commit expressions, and cover the fingerprint marker in
the importer capture/restore helper.

Verified locally: importer archive capture 0.96s/228K and restore
0.07s against the real 2.0GB hoisted tree; a snapshot untarred into a
fresh workspace resolves modules, runs bin shims, `pnpm exec`, and a
Vitest suite (one-time pnpm reconcile only when the absolute path
changes, which Blacksmith's fixed /home/runner/_work layout avoids).

* fix(ci): commit dependency snapshots explicitly, not via on-change heuristic

stickydisk's on-change mode compares allocated disk bytes with a 4KB
threshold, so a fingerprint refresh whose reinstall keeps usage stable
(metadata-only manifest edits, same-sized dependency swaps) could be
silently discarded, stranding every consumer on a stale marker and a
permanent reinstall path. Writers now commit explicitly, mirroring the
Vitest transform disk's rationale; warm no-op writer runs re-commit an
identical snapshot, which is cheap and safe. Also document that the
non-PR commit gate binds cooperating code only - the enforced trust
boundary stays the fork/dispatch runner gate, matching the protected
node-compile disk's posture.

* test(ci): guard sticky consumers against writerless node-version key splits

Reviewer follow-up: the snapshot key is partitioned by node-version and
both writers (build-artifacts, vitest-cache-warm) rely on the action
default, so a consumer pinning any other version would split onto a
key nobody seeds and silently regress to permanently cold installs.
Pin the action default to 24.x and assert every sticky consumer
resolves to that same key segment.

* test: narrow sticky-consumer step.with before indexing

The filter's runtime narrowing did not carry into the map callback, so
step.with indexing failed check-test-types (TS18048). Collect a narrowed
stepWith instead.
This commit is contained in:
Peter Steinberger
2026-07-17 00:38:50 -07:00
committed by GitHub
parent d26951ae9a
commit f69cb2f75e
5 changed files with 233 additions and 39 deletions
+56 -12
View File
@@ -35,6 +35,14 @@ inputs:
alongside.
required: false
default: "false"
save-sticky-disk:
description: >
Whether this job may commit the shared dependency snapshot. One
designated job per node-version key; only honored outside pull_request
events so feature-branch installs can never publish a protected
snapshot.
required: false
default: "false"
runtime-cache-sticky-disk:
description: Whether runtime caches should use Blacksmith sticky disks instead of actions/cache.
required: false
@@ -116,14 +124,25 @@ runs:
if: inputs.sticky-disk == 'true'
uses: useblacksmith/stickydisk@5b350170ae4ef55b536b548ef5f5896e76a6b54f # v1.4.0
with:
# The key covers every dependency and lifecycle input. PR scope keeps
# feature snapshots separate from protected pushes. The v2 marker lets
# exact warm hits trust the snapshotted stock layout without rescanning.
key: ${{ github.repository }}-node-deps-bind-v2-${{ github.event_name == 'pull_request' && format('pr-{0}', github.event.pull_request.number) || 'protected' }}-${{ inputs.node-version }}-frozen-${{ inputs.frozen-lockfile }}-${{ hashFiles('**/package.json', 'pnpm-lock.yaml', 'pnpm-workspace.yaml', '.npmrc', '.pnpmfile.cjs', 'pnpmfile.cjs', '.github/actions/setup-node-env/sticky-importers.sh', 'scripts/postinstall-bundled-plugins.mjs', 'scripts/preinstall-package-manager-warning.mjs', 'scripts/prepare-git-hooks.mjs') }}
# One stable disk per node-version. The v2 per-PR/per-manifest-hash
# keys minted a new backing disk for every PR and dependency change
# and saturated Blacksmith's installation-wide sticky-disk budget,
# 429-failing every mount. Install inputs live in the runtime
# fingerprint marker below instead of the key, so dependency changes
# refresh this disk in place rather than allocating a new one.
key: ${{ github.repository }}-node-deps-bind-v3-${{ inputs.node-version }}
path: /var/tmp/openclaw-node-deps
# v1.4.0 skips commit after failed/cancelled steps or ambiguous failure
# detection, so an incomplete first hydration cannot seed this key.
commit: if-missing
# Single semantic writer: only the designated trusted-push job may
# commit, so pull_request clones stay read-only. Like every sticky
# disk here, this gate binds cooperating code, not hostile code: the
# enforced trust boundary is the fork/dispatch runner gate in ci.yml,
# and same-repo PR authors already hold repository write access.
# Explicit true (not on-change) because the allocated-byte heuristic
# can miss a fingerprint refresh whose reinstall keeps disk usage
# stable, permanently stranding consumers on a stale marker. v1.4.0
# skips commit after failed/cancelled steps, so a broken install
# cannot seed this key.
commit: ${{ inputs.save-sticky-disk == 'true' && github.event_name != 'pull_request' && 'true' || 'false' }}
- name: Mount protected Vitest transform seed
if: inputs.vitest-fs-cache == 'true' && (inputs.sticky-disk == 'true' || inputs.runtime-cache-sticky-disk == 'true') && github.event_name == 'pull_request' && runner.os != 'Windows'
@@ -254,6 +273,12 @@ runs:
- name: Bind sticky node_modules into workspace
if: inputs.sticky-disk == 'true'
shell: bash
env:
# Evaluated before this step's bind mount populates node_modules, so
# the '**/package.json' glob only sees checkout manifests. Recomputing
# this hash any later would sweep snapshot-internal package.json files
# into the fingerprint and permanently miss the warm path.
DEPS_FINGERPRINT: node-${{ inputs.node-version }}-frozen-${{ inputs.frozen-lockfile }}-${{ hashFiles('**/package.json', 'pnpm-lock.yaml', 'pnpm-workspace.yaml', '.npmrc', '.pnpmfile.cjs', 'pnpmfile.cjs', '.github/actions/setup-node-env/sticky-importers.sh', 'scripts/postinstall-bundled-plugins.mjs', 'scripts/preinstall-package-manager-warning.mjs', 'scripts/prepare-git-hooks.mjs') }}
run: |
set -euo pipefail
sticky_root=/var/tmp/openclaw-node-deps
@@ -271,6 +296,7 @@ runs:
findmnt --target "$workspace_modules"
# zizmor: ignore[github-env] static trusted path defined in this composite action.
echo "PNPM_CONFIG_STORE_DIR=$sticky_store" >> "$GITHUB_ENV"
echo "OPENCLAW_STICKY_DEPS_FINGERPRINT=$DEPS_FINGERPRINT" >> "$GITHUB_ENV"
# pnpm exec may reconcile a restored workspace before nested builds.
# Sticky jobs already have every build tool, so invoke their Node entrypoints directly.
echo "OPENCLAW_BUILD_ALL_NO_PNPM=1" >> "$GITHUB_ENV"
@@ -309,6 +335,7 @@ runs:
FROZEN_LOCKFILE: ${{ inputs.frozen-lockfile }}
STICKY_DISK: ${{ inputs.sticky-disk }}
STICKY_ROOT: /var/tmp/openclaw-node-deps
STICKY_WRITER: ${{ inputs.save-sticky-disk == 'true' && github.event_name != 'pull_request' && 'true' || 'false' }}
run: |
set -euo pipefail
export PATH="$NODE_BIN:$PATH"
@@ -353,19 +380,36 @@ runs:
ln -sfn . "$PNPM_CONFIG_MODULES_DIR/node_modules"
export NODE_PATH="$PNPM_CONFIG_MODULES_DIR${NODE_PATH:+:$NODE_PATH}"
fi
sticky_ready_marker="$STICKY_ROOT/.install-complete-v2"
if [ "$STICKY_DISK" = "true" ] && [ -f "$sticky_ready_marker" ]; then
sticky_marker="$STICKY_ROOT/.openclaw-deps-fingerprint"
sticky_fingerprint=""
if [ "$STICKY_DISK" = "true" ] && [ -f "$sticky_marker" ]; then
sticky_fingerprint="$(<"$sticky_marker")"
fi
if [ "$STICKY_DISK" = "true" ] && [ -n "$sticky_fingerprint" ] &&
[ "$sticky_fingerprint" = "${OPENCLAW_STICKY_DEPS_FINGERPRINT:?}" ]; then
bash "$GITHUB_ACTION_PATH/sticky-importers.sh" restore "$STICKY_ROOT" "$GITHUB_WORKSPACE"
echo "Sticky dependency snapshot is ready; skipping pnpm install"
echo "Sticky dependency snapshot matches the install fingerprint; skipping pnpm install"
else
if [ "$STICKY_DISK" = "true" ] && [ -n "$sticky_fingerprint" ]; then
echo "Sticky dependency snapshot is stale (disk: $sticky_fingerprint, want: $OPENCLAW_STICKY_DEPS_FINGERPRINT); reinstalling"
fi
# A stale marker must not survive a mid-install failure: the commit
# heuristics skip failed steps, but a wrong marker plus a partial
# tree would poison every consumer if one ever slipped through.
if [ "$STICKY_DISK" = "true" ] && [ "$STICKY_WRITER" = "true" ]; then
rm -f "$sticky_marker"
fi
pnpm "${install_args[@]}" || pnpm "${install_args[@]}"
if [ -n "${PNPM_CONFIG_MODULES_DIR:-}" ]; then
rm -rf node_modules
ln -sfn "$PNPM_CONFIG_MODULES_DIR" node_modules
ln -sfn . "$PNPM_CONFIG_MODULES_DIR/node_modules"
fi
if [ "$STICKY_DISK" = "true" ]; then
bash "$GITHUB_ACTION_PATH/sticky-importers.sh" capture "$STICKY_ROOT" "$GITHUB_WORKSPACE"
# Only the designated trusted writer captures importer archives and
# publishes the fingerprint; read-only clones are discarded at job
# end, so capturing there would only burn shard wall clock.
if [ "$STICKY_DISK" = "true" ] && [ "$STICKY_WRITER" = "true" ]; then
bash "$GITHUB_ACTION_PATH/sticky-importers.sh" capture "$STICKY_ROOT" "$GITHUB_WORKSPACE" "$OPENCLAW_STICKY_DEPS_FINGERPRINT"
fi
fi
@@ -5,10 +5,11 @@ mode="${1:?mode is required}"
sticky_root="${2:?sticky root is required}"
workspace="${3:?workspace is required}"
archive="$sticky_root/importer-node-modules.tar"
marker="$sticky_root/.install-complete-v2"
marker="$sticky_root/.openclaw-deps-fingerprint"
case "$mode" in
capture)
fingerprint="${4:?fingerprint is required}"
mkdir -p "$sticky_root"
list_file="$(mktemp)"
temp_archive="$archive.tmp.$$"
@@ -22,7 +23,9 @@ case "$mode" in
tar --create --file "$temp_archive" --null --files-from "$list_file"
)
mv "$temp_archive" "$archive"
touch "$marker"
# The marker lands last: a fingerprint is only ever visible next to the
# importer archive it describes, so consumers cannot restore a torn pair.
printf '%s\n' "$fingerprint" > "$marker"
;;
restore)
if [[ ! -f "$archive" ]]; then
+59
View File
@@ -1016,6 +1016,15 @@ jobs:
install-bun: "false"
node-compile-cache: "true"
node-compile-cache-scope: "build"
# Blacksmith same-repo runs clone dependencies from a sticky disk.
# Fork PRs must keep actions/cache: sticky snapshots are writable,
# repository-global state and must never be produced by fork code.
sticky-disk: ${{ github.event_name != 'workflow_dispatch' && github.repository == 'openclaw/openclaw' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == 'openclaw/openclaw') && 'true' || 'false' }}
use-actions-cache: ${{ github.event_name != 'workflow_dispatch' && github.repository == 'openclaw/openclaw' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == 'openclaw/openclaw') && 'false' || 'true' }}
# Designated dependency-snapshot writer: exactly one trusted-push job
# per node-version key refreshes the protected disk; every other
# sticky consumer mounts read-only.
save-sticky-disk: "true"
runtime-cache-sticky-disk: ${{ github.event_name != 'workflow_dispatch' && github.repository == 'openclaw/openclaw' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == 'openclaw/openclaw') && 'true' || 'false' }}
save-node-compile-cache: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' && 'true' || 'false' }}
@@ -1227,6 +1236,11 @@ jobs:
uses: ./.github/actions/setup-node-env
with:
install-bun: "false"
# Blacksmith same-repo runs clone dependencies from a sticky disk.
# Fork PRs must keep actions/cache: sticky snapshots are writable,
# repository-global state and must never be produced by fork code.
sticky-disk: ${{ github.event_name != 'workflow_dispatch' && github.repository == 'openclaw/openclaw' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == 'openclaw/openclaw') && 'true' || 'false' }}
use-actions-cache: ${{ github.event_name != 'workflow_dispatch' && github.repository == 'openclaw/openclaw' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == 'openclaw/openclaw') && 'false' || 'true' }}
- name: Check native app i18n inventory
run: pnpm native:i18n:check
@@ -1254,6 +1268,11 @@ jobs:
with:
node-version: "24.x"
install-bun: "false"
# Blacksmith same-repo runs clone dependencies from a sticky disk.
# Fork PRs must keep actions/cache: sticky snapshots are writable,
# repository-global state and must never be produced by fork code.
sticky-disk: ${{ github.event_name != 'workflow_dispatch' && github.repository == 'openclaw/openclaw' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == 'openclaw/openclaw') && 'true' || 'false' }}
use-actions-cache: ${{ github.event_name != 'workflow_dispatch' && github.repository == 'openclaw/openclaw' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == 'openclaw/openclaw') && 'false' || 'true' }}
- name: Install Playwright Chromium
run: |
@@ -1298,6 +1317,11 @@ jobs:
with:
node-version: "24.x"
install-bun: "false"
# Blacksmith same-repo runs clone dependencies from a sticky disk.
# Fork PRs must keep actions/cache: sticky snapshots are writable,
# repository-global state and must never be produced by fork code.
sticky-disk: ${{ github.event_name != 'workflow_dispatch' && github.repository == 'openclaw/openclaw' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == 'openclaw/openclaw') && 'true' || 'false' }}
use-actions-cache: ${{ github.event_name != 'workflow_dispatch' && github.repository == 'openclaw/openclaw' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == 'openclaw/openclaw') && 'false' || 'true' }}
- name: Check Control UI locale parity
run: pnpm ui:i18n:check
@@ -1371,6 +1395,11 @@ jobs:
uses: ./.github/actions/setup-node-env
with:
install-bun: ${{ matrix.task == 'bun-launcher' && 'true' || 'false' }}
# Blacksmith same-repo runs clone dependencies from a sticky disk.
# Fork PRs must keep actions/cache: sticky snapshots are writable,
# repository-global state and must never be produced by fork code.
sticky-disk: ${{ github.event_name != 'workflow_dispatch' && github.repository == 'openclaw/openclaw' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == 'openclaw/openclaw') && 'true' || 'false' }}
use-actions-cache: ${{ github.event_name != 'workflow_dispatch' && github.repository == 'openclaw/openclaw' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == 'openclaw/openclaw') && 'false' || 'true' }}
- name: Run ${{ matrix.task }} (${{ matrix.runtime }})
env:
@@ -1500,6 +1529,11 @@ jobs:
uses: ./.github/actions/setup-node-env
with:
install-bun: "false"
# Blacksmith same-repo runs clone dependencies from a sticky disk.
# Fork PRs must keep actions/cache: sticky snapshots are writable,
# repository-global state and must never be produced by fork code.
sticky-disk: ${{ github.event_name != 'workflow_dispatch' && github.repository == 'openclaw/openclaw' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == 'openclaw/openclaw') && 'true' || 'false' }}
use-actions-cache: ${{ github.event_name != 'workflow_dispatch' && github.repository == 'openclaw/openclaw' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == 'openclaw/openclaw') && 'false' || 'true' }}
- name: Set up Blacksmith Docker layer cache
if: ${{ matrix.docker_cache == true && github.event_name != 'workflow_dispatch' && github.repository == 'openclaw/openclaw' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == 'openclaw/openclaw') }}
@@ -1671,6 +1705,11 @@ jobs:
uses: ./.github/actions/setup-node-env
with:
install-bun: "false"
# Blacksmith same-repo runs clone dependencies from a sticky disk.
# Fork PRs must keep actions/cache: sticky snapshots are writable,
# repository-global state and must never be produced by fork code.
sticky-disk: ${{ github.event_name != 'workflow_dispatch' && github.repository == 'openclaw/openclaw' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == 'openclaw/openclaw') && 'true' || 'false' }}
use-actions-cache: ${{ github.event_name != 'workflow_dispatch' && github.repository == 'openclaw/openclaw' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == 'openclaw/openclaw') && 'false' || 'true' }}
- name: Run plugin contract shard
env:
@@ -1709,6 +1748,11 @@ jobs:
uses: ./.github/actions/setup-node-env
with:
install-bun: "false"
# Blacksmith same-repo runs clone dependencies from a sticky disk.
# Fork PRs must keep actions/cache: sticky snapshots are writable,
# repository-global state and must never be produced by fork code.
sticky-disk: ${{ github.event_name != 'workflow_dispatch' && github.repository == 'openclaw/openclaw' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == 'openclaw/openclaw') && 'true' || 'false' }}
use-actions-cache: ${{ github.event_name != 'workflow_dispatch' && github.repository == 'openclaw/openclaw' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == 'openclaw/openclaw') && 'false' || 'true' }}
- name: Run channel contract shard
env:
@@ -1925,6 +1969,11 @@ jobs:
uses: ./.github/actions/setup-node-env
with:
install-bun: "false"
# Blacksmith same-repo runs clone dependencies from a sticky disk.
# Fork PRs must keep actions/cache: sticky snapshots are writable,
# repository-global state and must never be produced by fork code.
sticky-disk: ${{ github.event_name != 'workflow_dispatch' && github.repository == 'openclaw/openclaw' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == 'openclaw/openclaw') && 'true' || 'false' }}
use-actions-cache: ${{ github.event_name != 'workflow_dispatch' && github.repository == 'openclaw/openclaw' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == 'openclaw/openclaw') && 'false' || 'true' }}
# check-lint's shard runner rebuilds the same plugin-sdk boundary
# artifacts the boundary lane snapshots (~72s cold); restore them from
@@ -2143,6 +2192,11 @@ jobs:
uses: ./.github/actions/setup-node-env
with:
install-bun: "false"
# Blacksmith same-repo runs clone dependencies from a sticky disk.
# Fork PRs must keep actions/cache: sticky snapshots are writable,
# repository-global state and must never be produced by fork code.
sticky-disk: ${{ github.event_name != 'workflow_dispatch' && github.repository == 'openclaw/openclaw' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == 'openclaw/openclaw') && 'true' || 'false' }}
use-actions-cache: ${{ github.event_name != 'workflow_dispatch' && github.repository == 'openclaw/openclaw' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == 'openclaw/openclaw') && 'false' || 'true' }}
# Same-repo runs carry boundary artifacts on a Blacksmith sticky disk:
# the GitHub cache below is evicted so quickly under the repo quota that
@@ -2382,6 +2436,11 @@ jobs:
uses: ./.github/actions/setup-node-env
with:
install-bun: "false"
# Blacksmith same-repo runs clone dependencies from a sticky disk.
# Fork PRs must keep actions/cache: sticky snapshots are writable,
# repository-global state and must never be produced by fork code.
sticky-disk: ${{ github.event_name != 'workflow_dispatch' && github.repository == 'openclaw/openclaw' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == 'openclaw/openclaw') && 'true' || 'false' }}
use-actions-cache: ${{ github.event_name != 'workflow_dispatch' && github.repository == 'openclaw/openclaw' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == 'openclaw/openclaw') && 'false' || 'true' }}
- name: Check formatting
if: needs.preflight.outputs.run_format_check == 'true'
+5
View File
@@ -30,6 +30,11 @@ jobs:
node-compile-cache-scope: "test"
runtime-cache-sticky-disk: "true"
save-node-compile-cache: "true"
# Daily deadline writer for the protected dependency snapshot:
# canonical main pushes cancel each other under merge traffic, so
# build-artifacts commits can starve; this scheduled run cannot be
# superseded and keeps the snapshot within a day of main.
save-sticky-disk: "true"
save-vitest-fs-cache: "true"
sticky-disk: "true"
use-actions-cache: "false"