Files
openclaw/src/logging/subsystem.test.ts
T
Peter Steinberger e2a112a556 feat(onboard): guided CLI onboarding with live AI verification and classic fallback (#101880)
* feat(onboard): guided CLI onboarding with live AI verification and classic fallback

Interactive `openclaw onboard` (and bare `openclaw` on a fresh install) now
runs a guided flow with macOS-app parity: detect existing AI access, live-test
candidates with a real completion before persisting anything, walk down the
ladder on failure with mapped reasons, and offer verified manual API-key entry
from installed provider manifests (masked input). In-flow escapes: classic
wizard, Crestodian chat, skip-AI. Classic wizard gains an optional post-auth
live verification step. `--classic`, `--modern`, and `--non-interactive`
contracts unchanged. Docs corrected for post-#99935 routing.

Closes #101851

* improve(onboard): quiet probe diagnostics in wizard TTY, carry risk ack into classic escape

Candidate live-tests during guided setup are probes: rename their run id and
lane to the existing probe conventions (logging/subsystem.ts console
suppression, command-queue quiet probe lanes) so expected failures stop
leaking raw diagnostics into the Clack UI; file diagnostics unchanged. The
classic-wizard escape now passes the already-collected risk acknowledgement
through instead of re-prompting in the same session.

* fix(onboard): quiet the session-derived setup-inference probe lane too

The live-test run enqueues on two lanes: the explicit probe lane and one
derived from its temp session key. Extend the shared quiet-probe predicate to
cover the derived lane so a failing candidate cannot leak lane-task
diagnostics into the wizard TTY.

* improve(onboard): suppress subsystem console output during wizard live tests

Provider-transport subsystem loggers (model-fetch start/response, transport
errors) carry no run id, so probe suppression cannot catch them and a failing
candidate printed raw log lines into the Clack TTY. Reuse the TUI console
subsystem-filter seam via a finally-safe scoped helper around guided
activation and the classic live-verify; file logging is unchanged and the
gateway (macOS app) surface is unaffected.

* fix(onboard): never auto-replace a configured model when its live check fails

The re-run verification probe executes outside the configured workspace (setup
never runs workspace plugins), so a workspace-backed current model can fail
the check while working fine in the agent. Stop the auto ladder on an
existing-model failure and hand the decision to the manual stage instead of
silently persisting a different candidate as the default. Docs note the
fail-safe and the workspace caveat.

* feat(onboard): two-way switching between Crestodian chat and the menu wizards

From the chat, `open setup wizard`, `open classic wizard`, and `open channel
wizard for <channel>` hand off to the guided flow, the classic wizard, or the
masked `channels add` wizard after the chat TUI tears down (mirrors the
open-tui handoff; gateway surface gets a text pointer instead). The hosted
channel wizard no longer dead-ends at sensitive steps — it offers the switch
and remembers the channel. New read-only `channel info <channel>` operation
and ring-zero action surface label, blurb, configured state, and the real
docs URL from channel-setup discovery so the assistant can explain Slack or
Telegram prerequisites instead of guessing; both prompts instruct it to use
them. `channels add --channel <id>` now preselects the channel. Docs cover
the interchangeable flows.

* fix(onboard): avoid param reassignment in open-setup handoff

* improve(onboard): separate ask-about vs connect intent in channel prompt guidance

Live test showed the agent detouring an explicit connect request through
channel_info because the guidance said to consult it first. Both prompts now
distinguish asking about a channel (channel info + docs link) from asking to
connect (connect right away).

* fix(channels): mark channel token entry as sensitive input

The shared single-token prompt lacked sensitive:true, so terminal wizards
echoed pasted channel tokens and the Crestodian chat bridge (which refuses
plain-text secrets based on this flag) hosted the Telegram token step in
visible chat. Found live-testing the chat-to-wizard switch; pre-existing on
main but load-bearing for the masked-wizard contract this PR documents.

* fix(onboard): restore terminal state around the guided flow's TUI launch

Mirror the classic finalize handoff so the chat TUI never inherits the wizard
prompter's raw/paused terminal state on the default first-run path.

* fix(channels): type the token prompter mock for the sensitive-flag assertion

* fix(gateway): map the TUI-only open-setup action to none for app clients

Engine-side surface gating already prevents open-setup replies on the gateway
surface; this keeps the client-visible action enum stable even if that gate
ever regresses. (Reviewed with the switching round; missed in its commit.)

* docs: regenerate docs map for onboarding page changes
2026-07-09 12:40:55 +01:00

316 lines
12 KiB
TypeScript

// Subsystem logger tests cover per-subsystem log routing and filtering.
import fs from "node:fs";
import path from "node:path";
import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest";
import { setConsoleSubsystemFilter, shouldLogSubsystemToConsole } from "./console.js";
import { createSuiteLogPathTracker } from "./log-test-helpers.js";
import { resetLogger, setLoggerOverride } from "./logger.js";
import { loggingState } from "./state.js";
import { createSubsystemLogger } from "./subsystem.js";
const logPathTracker = createSuiteLogPathTracker("openclaw-subsystem-log-");
function installConsoleMethodSpy(method: "log" | "warn" | "error") {
const spy = vi.fn();
loggingState.rawConsole = {
log: method === "log" ? spy : vi.fn(),
info: vi.fn(),
warn: method === "warn" ? spy : vi.fn(),
error: method === "error" ? spy : vi.fn(),
};
return spy;
}
function firstMockArgAsString(mock: { mock: { calls: readonly unknown[][] } }): string {
const [call] = mock.mock.calls;
if (!call) {
throw new Error("expected console mock call");
}
return String(call[0]);
}
beforeAll(async () => {
await logPathTracker.setup();
});
afterEach(() => {
setConsoleSubsystemFilter(null);
setLoggerOverride(null);
loggingState.rawConsole = null;
resetLogger();
vi.unstubAllEnvs();
vi.useRealTimers();
});
afterAll(async () => {
await logPathTracker.cleanup();
});
describe("createSubsystemLogger().isEnabled", () => {
it("returns true for any/file when only file logging would emit", () => {
setLoggerOverride({ level: "debug", consoleLevel: "silent" });
const log = createSubsystemLogger("agent/embedded");
expect(log.isEnabled("debug")).toBe(true);
expect(log.isEnabled("debug", "file")).toBe(true);
expect(log.isEnabled("debug", "console")).toBe(false);
});
it("returns true for any/console when only console logging would emit", () => {
setLoggerOverride({ level: "silent", consoleLevel: "debug" });
const log = createSubsystemLogger("agent/embedded");
expect(log.isEnabled("debug")).toBe(true);
expect(log.isEnabled("debug", "console")).toBe(true);
expect(log.isEnabled("debug", "file")).toBe(false);
});
it("uses threshold ordering for non-equal console levels", () => {
setLoggerOverride({ level: "silent", consoleLevel: "fatal" });
const fatalOnly = createSubsystemLogger("agent/embedded");
expect(fatalOnly.isEnabled("error", "console")).toBe(false);
expect(fatalOnly.isEnabled("fatal", "console")).toBe(true);
setLoggerOverride({ level: "silent", consoleLevel: "trace" });
const traceLogger = createSubsystemLogger("agent/embedded");
expect(traceLogger.isEnabled("debug", "console")).toBe(true);
});
it("never treats silent as an emittable console level", () => {
setLoggerOverride({ level: "silent", consoleLevel: "info" });
const log = createSubsystemLogger("agent/embedded");
expect(log.isEnabled("silent", "console")).toBe(false);
});
it("returns false when neither console nor file logging would emit", () => {
setLoggerOverride({ level: "silent", consoleLevel: "silent" });
const log = createSubsystemLogger("agent/embedded");
expect(log.isEnabled("debug")).toBe(false);
expect(log.isEnabled("debug", "console")).toBe(false);
expect(log.isEnabled("debug", "file")).toBe(false);
});
it("honors console subsystem filters for console target", () => {
setLoggerOverride({ level: "silent", consoleLevel: "info" });
setConsoleSubsystemFilter(["gateway"]);
const log = createSubsystemLogger("agent/embedded");
expect(log.isEnabled("info", "console")).toBe(false);
});
it("does not apply console subsystem filters to file target", () => {
setLoggerOverride({ level: "info", consoleLevel: "silent" });
setConsoleSubsystemFilter(["gateway"]);
const log = createSubsystemLogger("agent/embedded");
expect(log.isEnabled("info", "file")).toBe(true);
expect(log.isEnabled("info")).toBe(true);
});
it("treats missing subsystem labels as non-matches when filters are active", () => {
setConsoleSubsystemFilter(["gateway"]);
expect(shouldLogSubsystemToConsole(undefined as unknown as string)).toBe(false);
});
it("disables console logging when a malformed subsystem logger checks enablement", () => {
setLoggerOverride({ level: "silent", consoleLevel: "info" });
setConsoleSubsystemFilter(["gateway"]);
const log = createSubsystemLogger(undefined as unknown as string);
expect(log.isEnabled("info", "console")).toBe(false);
});
it("falls back to an unknown subsystem label when a malformed logger emits", () => {
setLoggerOverride({ level: "silent", consoleLevel: "warn" });
const warn = installConsoleMethodSpy("warn");
const log = createSubsystemLogger(undefined as unknown as string);
log.warn("missing subsystem label");
expect(warn).toHaveBeenCalledTimes(1);
expect(firstMockArgAsString(warn)).toContain("[unknown]");
});
it("suppresses probe warnings for embedded subsystems based on structured run metadata", () => {
setLoggerOverride({ level: "silent", consoleLevel: "warn" });
const warn = installConsoleMethodSpy("warn");
const log = createSubsystemLogger("agent/embedded").child("failover");
log.warn("embedded run failover decision", {
runId: "probe-test-run",
consoleMessage: "embedded run failover decision",
});
expect(warn).not.toHaveBeenCalled();
});
it("keeps setup-inference probe warnings in the file log while suppressing console", () => {
const file = logPathTracker.nextPath();
setLoggerOverride({ level: "warn", consoleLevel: "warn", file });
const warn = installConsoleMethodSpy("warn");
const log = createSubsystemLogger("agent/embedded");
log.warn("embedded run failover decision", {
runId: "probe-setup-inference-test-run",
provider: "openai",
consoleMessage: "embedded run failover decision: provider=openai error=Authentication failed",
});
log.warn("embedded run agent end", {
runId: "probe-setup-inference-test-run",
provider: "openai",
consoleMessage: "embedded run agent end: provider=openai error=Authentication failed",
});
expect(warn).not.toHaveBeenCalled();
const fileLog = fs.readFileSync(file, "utf8");
expect(fileLog).toContain("embedded run failover decision");
expect(fileLog).toContain("embedded run agent end");
expect(fileLog).toContain('"provider":"openai"');
});
it("does not suppress probe errors for embedded subsystems", () => {
setLoggerOverride({ level: "silent", consoleLevel: "error" });
const error = installConsoleMethodSpy("error");
const log = createSubsystemLogger("agent/embedded").child("failover");
log.error("embedded run failover decision", {
runId: "probe-test-run",
consoleMessage: "embedded run failover decision",
});
expect(error).toHaveBeenCalledTimes(1);
});
it("suppresses probe warnings for model-fallback child subsystems based on structured run metadata", () => {
setLoggerOverride({ level: "silent", consoleLevel: "warn" });
const warn = installConsoleMethodSpy("warn");
const log = createSubsystemLogger("model-fallback").child("decision");
log.warn("model fallback decision", {
runId: "probe-test-run",
consoleMessage: "model fallback decision",
});
expect(warn).not.toHaveBeenCalled();
});
it("does not suppress probe errors for model-fallback child subsystems", () => {
setLoggerOverride({ level: "silent", consoleLevel: "error" });
const error = installConsoleMethodSpy("error");
const log = createSubsystemLogger("model-fallback").child("decision");
log.error("model fallback decision", {
runId: "probe-test-run",
consoleMessage: "model fallback decision",
});
expect(error).toHaveBeenCalledTimes(1);
});
it("still emits non-probe warnings for embedded subsystems", () => {
setLoggerOverride({ level: "silent", consoleLevel: "warn" });
const warn = installConsoleMethodSpy("warn");
const log = createSubsystemLogger("agent/embedded").child("auth-profiles");
log.warn("auth profile failure state updated", {
runId: "run-123",
consoleMessage: "auth profile failure state updated",
});
expect(warn).toHaveBeenCalledTimes(1);
});
it("still emits non-probe model-fallback child warnings", () => {
setLoggerOverride({ level: "silent", consoleLevel: "warn" });
const warn = installConsoleMethodSpy("warn");
const log = createSubsystemLogger("model-fallback").child("decision");
log.warn("model fallback decision", {
runId: "run-123",
consoleMessage: "model fallback decision",
});
expect(warn).toHaveBeenCalledTimes(1);
});
it("redacts sensitive tokens at the console sink so subsystem writes do not leak secrets (#73284)", () => {
setLoggerOverride({ level: "silent", consoleLevel: "warn" });
const warn = installConsoleMethodSpy("warn");
const log = createSubsystemLogger("gateway");
const secret = "sk-supersecretvaluefortest12345";
log.warn(`token=${secret}`);
expect(warn).toHaveBeenCalledTimes(1);
const written = firstMockArgAsString(warn);
expect(written).not.toContain(secret);
expect(written).toMatch(/sk-sup…2345|\*\*\*/);
});
it("redacts Bearer tokens on subsystem error console writes", () => {
setLoggerOverride({ level: "silent", consoleLevel: "error" });
const error = installConsoleMethodSpy("error");
const log = createSubsystemLogger("gateway").child("auth");
const bearer = "Bearer abcdefghijklmnopqrstuvwxyz";
log.error(`Authorization failed: ${bearer}`);
expect(error).toHaveBeenCalledTimes(1);
const written = firstMockArgAsString(error);
expect(written).not.toContain("abcdefghijklmnopqrstuvwxyz");
expect(written).toContain("Bearer ");
});
it("redacts before colorizing subsystem console messages so ANSI reset codes survive", () => {
vi.stubEnv("FORCE_COLOR", "1");
setLoggerOverride({ level: "silent", consoleLevel: "info" });
const logSpy = installConsoleMethodSpy("log");
const log = createSubsystemLogger("gateway/auth");
const secret = "sk-abcdefghijklmnopqrstuvwxyz123456";
log.info(`provider API_KEY=${secret}`);
expect(logSpy).toHaveBeenCalledTimes(1);
const written = firstMockArgAsString(logSpy);
expect(written).not.toContain(secret);
expect(written).toContain("API_KEY=***");
expect(written.endsWith("\u001B[39m")).toBe(true);
});
it("redacts sensitive tokens from raw subsystem console output", () => {
setLoggerOverride({ level: "silent", consoleLevel: "info" });
const logSpy = installConsoleMethodSpy("log");
const log = createSubsystemLogger("gateway/auth");
const secret = "sk-rawtokenabcdefghijklmnopqrstuvwxyz123456";
log.raw(`raw token ${secret}`);
expect(logSpy).toHaveBeenCalledTimes(1);
const written = firstMockArgAsString(logSpy);
expect(written).not.toContain(secret);
expect(written).toContain("sk-raw…3456");
});
it("keeps long-lived subsystem loggers on the current-day rolling file", () => {
const logDir = path.dirname(logPathTracker.nextPath());
const firstDay = path.join(logDir, "openclaw-2026-01-01.log");
const secondDay = path.join(logDir, "openclaw-2026-01-02.log");
vi.useFakeTimers();
vi.setSystemTime(new Date("2026-01-01T08:00:00Z"));
setLoggerOverride({ level: "info", consoleLevel: "silent", file: firstDay });
const log = createSubsystemLogger("diagnostics");
log.info("first day subsystem log");
vi.setSystemTime(new Date("2026-01-02T08:00:00Z"));
log.info("second day subsystem log");
expect(fs.readFileSync(firstDay, "utf8")).toContain("first day subsystem log");
expect(fs.readFileSync(secondDay, "utf8")).toContain("second day subsystem log");
expect(fs.readFileSync(firstDay, "utf8")).not.toContain("second day subsystem log");
});
});