feat(secrets): egress-time credential injection with process-local sentinels (#102009)

* feat(secrets): resolve SecretRef model credentials at egress via process-local sentinels

SecretRef-managed model-provider credentials now travel as opaque
oc-sent-v1 sentinels through auth storage, stream options, and SDK
config; the guarded model fetch injects real values into headers and
URLs immediately before the SSRF-guarded send and fails closed on
unknown sentinels. packages/ai adapters converge on the host guarded
fetch where the SDK supports custom fetch and unwrap at construction
where it does not. Resolved values (and their percent-encoded forms)
register for exact-value log redaction. Kill switch:
OPENCLAW_SECRET_SENTINELS=off. Also fixes a pre-existing unhandled
rejection race in capNonOkResponseBodyLazily (pipeThrough writer leak).

* test(plugin-sdk): update public surface budget
This commit is contained in:
Peter Steinberger
2026-07-08 12:56:41 +01:00
committed by GitHub
parent 83ebbcb3ac
commit 4bf70be01a
67 changed files with 2959 additions and 234 deletions
+1
View File
@@ -23,6 +23,7 @@ Docs: https://docs.openclaw.ai
### Fixes
- **SecretRef model credentials:** keep resolved provider secrets behind process-local sentinels through auth storage, stream setup, SDK configuration, and managed local-provider probing, then inject plaintext only at the final network or provider-plugin boundary while retaining exact-value log redaction. (#102008, #102009)
- **Lean local model shell access:** keep `exec` directly visible beside the default structured Tool Search controls so coding-tuned local models can use their shell fallback instead of searching for missing domain tools. (#87587) Thanks @vincentkoc.
- **OAuth refresh contention diagnostics:** keep local lock paths out of user-facing refresh failures and avoid duplicate failure prefixes while preserving structured provider and profile classification. (#83383) Thanks @vincentkoc.
- **Exec approval prompts:** keep background-disabled fallback warnings out of pending gateway/node approvals and show them only after a command actually runs in the foreground. (#78184) Thanks @vincentkoc.
+1
View File
@@ -3710,6 +3710,7 @@ Do not edit it by hand; run `pnpm docs:map:gen`.
- Route: /gateway/secrets
- Headings:
- H2: Runtime model
- H2: Egress-time injection (sentinels)
- H2: Agent-access boundary
- H2: Active-surface filtering
- H2: Gateway auth surface diagnostics
+16 -1
View File
@@ -24,10 +24,25 @@ Plaintext credentials remain agent-readable if they sit in files the agent can i
- Startup fails fast when an effectively active SecretRef cannot be resolved.
- Reload is an atomic swap: full success, or keep the last-known-good snapshot.
- Policy violations (for example an OAuth-mode auth profile combined with SecretRef input) fail activation before the runtime swap.
- Runtime requests read only the active in-memory snapshot. Outbound delivery paths (Discord reply/thread delivery, Telegram action sends) also read that snapshot and do not re-resolve refs per send.
- Runtime requests read only the active in-memory snapshot. Model-provider SecretRef credentials pass through auth storage and stream options as process-local sentinels until egress. Outbound delivery paths (Discord reply/thread delivery, Telegram action sends) also read that snapshot and do not re-resolve refs per send.
This keeps secret-provider outages off hot request paths.
## Egress-time injection (sentinels)
For model-provider credentials backed by SecretRefs, OpenClaw mints an opaque, process-local sentinel during model-auth resolution. Auth storage, stream options, SDK configuration, logs, error objects, and most runtime introspection therefore see a value such as `oc-sent-v1-...`, not the provider credential. The guarded model fetch and managed local-provider health probes replace known sentinels in URL and header values immediately before each request leaves the process.
Unknown sentinel-shaped values fail closed before network activity. OpenClaw refuses to send the request rather than forwarding an unresolved sentinel to a provider. Resolved secret values are also registered for exact-value log redaction as a defense in depth measure.
Provider adapters use the latest injection point their SDK supports:
- SDKs with a custom fetch option receive OpenClaw's guarded fetch, so the SDK retains the sentinel.
- SDKs without a custom fetch option unwrap the sentinel immediately before client construction. Plugin-owned provider streams and agent harnesses unwrap at the final core-owned handoff because those transports do not share OpenClaw's guarded fetch.
Sentinels reduce plaintext exposure across the model-call chain, but they are not process isolation. The real value still exists in same-process memory and appears at the final adapter boundary. Plain environment credentials that are not configured through SecretRefs remain plaintext and are outside this mechanism.
Set `OPENCLAW_SECRET_SENTINELS=off` (also accepts `0` or `false`, case-insensitive) to disable sentinel minting during incident response or compatibility troubleshooting. The kill switch does not disable exact-value redaction registration.
## Agent-access boundary
SecretRefs stop credentials from being persisted in config and generated model files, but they are not a process-isolation boundary. A plaintext credential left on disk in a path the agent can read is still readable via file or shell tools, bypassing API-level redaction.
+22
View File
@@ -22,6 +22,8 @@ export interface AiTransportHost {
timeoutMs?: number,
options?: { sanitizeSse?: boolean },
): typeof fetch | undefined;
/** Resolves host-owned process-local secret sentinel substrings immediately before egress. */
resolveSecretSentinel(value: string): string;
/** Redacts secrets inside structured tool-result payloads. */
redactSecrets<T>(value: T): T;
/** Redacts secret-bearing text in tool payload strings. */
@@ -46,6 +48,7 @@ export interface AiTransportHost {
const inertAiTransportHost: AiTransportHost = {
buildModelFetch: () => undefined,
resolveSecretSentinel: (value) => value,
redactSecrets: (value) => value,
redactToolPayloadText: (text) => text,
resolveOpenAIStrictToolSetting: (_model, options) =>
@@ -64,3 +67,22 @@ export function configureAiTransportHost(host: Partial<AiTransportHost>): void {
export function getAiTransportHost(): AiTransportHost {
return activeAiTransportHost;
}
/** Resolves sentinel substrings in custom headers at a no-fetch adapter boundary. */
export function resolveAiTransportHeaderSentinels(
headers: Record<string, string> | undefined,
): Record<string, string> | undefined {
if (!headers) {
return undefined;
}
const host = getAiTransportHost();
let resolvedHeaders: Record<string, string> | undefined;
for (const [name, value] of Object.entries(headers)) {
const resolved = host.resolveSecretSentinel(value);
if (resolved !== value) {
resolvedHeaders ??= { ...headers };
resolvedHeaders[name] = resolved;
}
}
return resolvedHeaders ?? headers;
}
@@ -151,6 +151,35 @@ describe("Anthropic provider", () => {
expect(config.defaultHeaders?.["x-api-key"]).toBeUndefined();
});
it("keeps sentinel-backed Foundry Authorization headers on bearer routing", async () => {
const sentinel = "oc-sent-v1-0123456789abcdef01234567";
configureAiTransportHost({
buildModelFetch: () => async () => new Response(null, { status: 500 }),
resolveSecretSentinel: (value) => value.replaceAll(sentinel, "Bearer entra-access-token"),
});
const model = makeAnthropicModel({
provider: "microsoft-foundry",
baseUrl: "https://example.services.ai.azure.com/anthropic",
headers: { Authorization: sentinel },
});
streamAnthropic(
model,
{ messages: [{ role: "user", content: "hello", timestamp: 1 }] },
{
apiKey: sentinel,
},
);
await vi.waitFor(() => expect(anthropicMockState.configs).toHaveLength(1));
const config = anthropicMockState.configs[0] as {
apiKey?: string | null;
authToken?: string | null;
};
expect(config.apiKey).toBeNull();
expect(config.authToken).toBe(sentinel);
});
it("keeps Microsoft Foundry API-key profiles on Anthropic API key auth", async () => {
const model = makeAnthropicModel({
provider: "microsoft-foundry",
+10 -3
View File
@@ -9,7 +9,7 @@ import type {
TextBlockParam,
} from "@anthropic-ai/sdk/resources/messages.js";
import { getEnvApiKey } from "../env-api-keys.js";
import { getAiTransportHost } from "../host.js";
import { getAiTransportHost, resolveAiTransportHeaderSentinels } from "../host.js";
import { calculateCost, clampThinkingLevel } from "../model-utils.js";
import type {
AnthropicMessagesCompat,
@@ -1111,7 +1111,8 @@ export const streamSimpleAnthropic: StreamFunction<
};
function isOAuthToken(apiKey: string): boolean {
return apiKey.includes("sk-ant-oat");
// Inspect the host-resolved shape only for auth routing; the SDK still receives the sentinel.
return getAiTransportHost().resolveSecretSentinel(apiKey).includes("sk-ant-oat");
}
function isAnthropicPublicEndpoint(baseUrl: string | undefined): boolean {
@@ -1161,6 +1162,7 @@ function createClient(
/^kimi(?:-|$)/.test(model.provider) && thinkingEnabled
? { sanitizeSse: false as const }
: undefined;
// Anthropic supports custom fetch, so sentinels stay opaque until guarded egress.
const fetch = getAiTransportHost().buildModelFetch(model, undefined, fetchOptions);
if (model.provider === "cloudflare-ai-gateway") {
@@ -1208,7 +1210,12 @@ function createClient(
return { client, isOAuthToken: false, serverSideFallback: false };
}
if (usesFoundryBearerAuth(model)) {
if (
usesFoundryBearerAuth({
...model,
headers: resolveAiTransportHeaderSentinels(model.headers),
})
) {
const client = new Anthropic({
apiKey: null,
authToken: apiKey,
@@ -200,6 +200,7 @@ function createClient(
}
const { baseUrl, apiVersion } = resolveAzureConfig(model, options);
// Both OpenAI clients support custom fetch, so sentinels stay opaque until guarded egress.
const guardedFetch = getAiTransportHost().buildModelFetch({ ...model, baseUrl });
if (isOpenAICompatibleAzureResponsesBaseUrl(baseUrl)) {
+2
View File
@@ -1,6 +1,8 @@
// Cloudflare provider metadata describes Cloudflare-hosted model capabilities.
import type { Model } from "../types.js";
// This module owns URL metadata only; Anthropic/OpenAI adapters inject guarded fetch.
export function isCloudflareProvider(provider: string): boolean {
return provider === "cloudflare-workers-ai" || provider === "cloudflare-ai-gateway";
}
@@ -0,0 +1,124 @@
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { configureAiTransportHost } from "../host.js";
import type { Context, Model } from "../types.js";
const googleMockState = vi.hoisted(() => ({ configs: [] as unknown[] }));
vi.mock("@google/genai", () => ({
GoogleGenAI: class MockGoogleGenAI {
models = {
generateContentStream: vi.fn(() => {
throw new Error("stop after constructor");
}),
};
constructor(config: unknown) {
googleMockState.configs.push(config);
}
},
ResourceScope: { COLLECTION: "COLLECTION" },
ThinkingLevel: {
THINKING_LEVEL_UNSPECIFIED: "THINKING_LEVEL_UNSPECIFIED",
MINIMAL: "MINIMAL",
LOW: "LOW",
MEDIUM: "MEDIUM",
HIGH: "HIGH",
},
}));
import { streamGoogleVertex } from "./google-vertex.js";
import { streamGoogle } from "./google.js";
const context = {
messages: [{ role: "user", content: "hello", timestamp: 0 }],
} satisfies Context;
const sentinel = "oc-sent-v1-0123456789abcdef01234567";
function googleModel(): Model<"google-generative-ai"> {
return {
id: "gemini-3-flash-preview",
name: "Gemini 3 Flash Preview",
api: "google-generative-ai",
provider: "google",
baseUrl: "https://generativelanguage.googleapis.com/v1beta",
reasoning: true,
input: ["text"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 1_000_000,
maxTokens: 8192,
};
}
function vertexModel(): Model<"google-vertex"> {
return {
...googleModel(),
api: "google-vertex",
provider: "google-vertex",
baseUrl: "https://us-central1-aiplatform.googleapis.com/v1",
};
}
describe("Google SDK construction auth", () => {
beforeEach(() => {
googleMockState.configs = [];
});
afterEach(() => {
configureAiTransportHost({});
});
it("unwraps Google API-key sentinels immediately before client construction", async () => {
const buildModelFetch = vi.fn();
configureAiTransportHost({
buildModelFetch,
resolveSecretSentinel: (value) => value.replaceAll(sentinel, "google-construction-secret"),
});
const result = await streamGoogle(
{
...googleModel(),
headers: { Authorization: `Bearer ${sentinel}` },
},
context,
{ apiKey: sentinel },
).result();
expect(result.stopReason).toBe("error");
expect(googleMockState.configs[0]).toMatchObject({
apiKey: "google-construction-secret",
httpOptions: { headers: { Authorization: "Bearer google-construction-secret" } },
});
expect(JSON.stringify(googleMockState.configs[0])).not.toContain(sentinel);
expect(buildModelFetch).not.toHaveBeenCalled();
});
it("unwraps Vertex API-key sentinels immediately before client construction", async () => {
const buildModelFetch = vi.fn();
configureAiTransportHost({
buildModelFetch,
resolveSecretSentinel: (value) => value.replaceAll(sentinel, "vertex-construction-secret"),
});
const result = await streamGoogleVertex(
{
...vertexModel(),
headers: { "X-Provider-Token": sentinel },
},
context,
{
apiKey: sentinel,
project: "demo-project",
location: "us-central1",
},
).result();
expect(result.stopReason).toBe("error");
expect(googleMockState.configs[0]).toMatchObject({
apiKey: "vertex-construction-secret",
vertexai: true,
httpOptions: { headers: { "X-Provider-Token": "vertex-construction-secret" } },
});
expect(JSON.stringify(googleMockState.configs[0])).not.toContain(sentinel);
expect(buildModelFetch).not.toHaveBeenCalled();
});
});
+8 -2
View File
@@ -6,6 +6,7 @@ import {
ResourceScope,
ThinkingLevel as VertexThinkingLevel,
} from "@google/genai";
import { getAiTransportHost, resolveAiTransportHeaderSentinels } from "../host.js";
import type { Context, Model, SimpleStreamOptions, StreamFunction } from "../types.js";
import { AssistantMessageEventStream } from "../utils/event-stream.js";
import type { GoogleThinkingLevel } from "./google-shared.js";
@@ -97,9 +98,11 @@ function createClientWithApiKey(
apiKey: string,
optionsHeaders?: Record<string, string>,
): GoogleGenAI {
// @google/genai exposes RequestInit options but no custom fetch; unwrap at construction.
const resolvedApiKey = getAiTransportHost().resolveSecretSentinel(apiKey);
return new GoogleGenAI({
vertexai: true,
apiKey,
apiKey: resolvedApiKey,
apiVersion: API_VERSION,
httpOptions: buildHttpOptions(model, optionsHeaders),
});
@@ -120,7 +123,10 @@ function buildHttpOptions(
}
if (model.headers || optionsHeaders) {
httpOptions.headers = { ...model.headers, ...optionsHeaders };
httpOptions.headers = resolveAiTransportHeaderSentinels({
...model.headers,
...optionsHeaders,
});
}
return Object.keys(httpOptions).length > 0 ? httpOptions : undefined;
+8 -2
View File
@@ -1,6 +1,7 @@
// Google provider adapts Gemini streams and tools to the agent runtime.
import { type GenerateContentParameters, GoogleGenAI } from "@google/genai";
import { getEnvApiKey } from "../env-api-keys.js";
import { getAiTransportHost, resolveAiTransportHeaderSentinels } from "../host.js";
import type { Context, Model, SimpleStreamOptions, StreamFunction } from "../types.js";
import { AssistantMessageEventStream } from "../utils/event-stream.js";
import {
@@ -74,11 +75,16 @@ function createClient(
httpOptions.apiVersion = ""; // baseUrl already includes version path, don't append
}
if (model.headers || optionsHeaders) {
httpOptions.headers = { ...model.headers, ...optionsHeaders };
httpOptions.headers = resolveAiTransportHeaderSentinels({
...model.headers,
...optionsHeaders,
});
}
// @google/genai exposes RequestInit options but no custom fetch; unwrap at construction.
const resolvedApiKey = apiKey ? getAiTransportHost().resolveSecretSentinel(apiKey) : undefined;
return new GoogleGenAI({
apiKey,
apiKey: resolvedApiKey,
httpOptions: Object.keys(httpOptions).length > 0 ? httpOptions : undefined,
});
}
@@ -128,7 +128,6 @@ describe("Mistral bounded-stream-read real wire proof (loopback http.createServe
// parser (`EventStream`) would see when a streaming body exceeds 16 MiB.
describe("Mistral bounded-stream-read direct (synthetic ReadableStream)", () => {
it("caps an oversized synthetic ReadableStream at 16 MiB", async () => {
const fetcher = createBoundedMistralFetcher(MAX);
const CHUNK = 1024 * 1024;
let sent = 0;
const synthetic = new ReadableStream<Uint8Array>({
@@ -147,38 +146,23 @@ describe("Mistral bounded-stream-read direct (synthetic ReadableStream)", () =>
status: 200,
headers: { "content-type": "application/octet-stream" },
});
const fetcher = createBoundedMistralFetcher(MAX, async () => syntheticResponse);
let captured: Error | undefined;
const wrapped = await fetcher("http://unused.invalid/");
try {
// Replace the fetcher's internal `fetch` call by exercising the
// post-fetchResponse code path directly: build a `Wrapped`
// that re-enters `fetcher` as if a real fetch returned our
// synthetic Response, by patching the global fetch.
const originalFetch = globalThis.fetch;
globalThis.fetch = (() => Promise.resolve(syntheticResponse)) as typeof globalThis.fetch;
try {
const wrapped = await fetcher("http://unused.invalid/");
try {
await readAllChunks(wrapped.body);
} catch (err) {
captured = err as Error;
}
} finally {
globalThis.fetch = originalFetch;
}
expect(captured).toBeInstanceOf(Error);
const match = (captured as Error).message.match(
/mistral: stream body exceeds \d+ bytes \(got (\d+)\)/,
);
expect(match).not.toBeNull();
const got = Number(match![1]);
// Synthetic stream chunks are exactly 1 MiB aligned, so cap+1 reads
// give exactly cap + 1 MiB = 16 MiB + 1 MiB = 17 825 792 bytes.
expect(got).toBe(16777216 + CHUNK);
} finally {
// Best-effort cleanup if the test threw mid-flight.
// No intervals to clear for this test; the synthetic stream closes
// automatically when `sent >= 18`.
await readAllChunks(wrapped.body);
} catch (err) {
captured = err as Error;
}
expect(captured).toBeInstanceOf(Error);
const match = (captured as Error).message.match(
/mistral: stream body exceeds \d+ bytes \(got (\d+)\)/,
);
expect(match).not.toBeNull();
const got = Number(match![1]);
// Synthetic stream chunks are exactly 1 MiB aligned, so cap+1 reads
// give exactly cap + 1 MiB = 16 MiB + 1 MiB = 17 825 792 bytes.
expect(got).toBe(16777216 + CHUNK);
});
});
+22
View File
@@ -5,6 +5,7 @@ import type { Context, Model } from "../types.js";
import { SYSTEM_PROMPT_CACHE_BOUNDARY } from "../utils/system-prompt-cache-boundary.js";
const mistralMockState = vi.hoisted(() => ({
configs: [] as unknown[],
payloads: [] as unknown[],
}));
@@ -19,6 +20,10 @@ vi.mock("@mistralai/mistralai", async () => {
return {
...actual,
Mistral: class MockMistral {
constructor(config: unknown) {
mistralMockState.configs.push(config);
}
chat = {
stream: vi.fn(async (payload: unknown) => {
mistralMockState.payloads.push(payload);
@@ -70,6 +75,7 @@ function makeUnreadableParameterTool() {
describe("Mistral provider", () => {
beforeEach(() => {
mistralMockState.configs = [];
mistralMockState.payloads = [];
});
@@ -89,6 +95,22 @@ describe("Mistral provider", () => {
expect((mistralMockState.payloads[0] as { stop?: unknown }).stop).toEqual(["STOP"]);
});
it("routes the Mistral HTTPClient through the host guarded fetch", async () => {
const hostFetch = vi.fn<typeof fetch>(async () => new Response("guarded"));
configureAiTransportHost({ buildModelFetch: () => hostFetch });
await streamMistral(makeMistralModel(), context, { apiKey: "sentinel-key" }).result();
const config = mistralMockState.configs[0] as {
apiKey?: string;
httpClient?: { request(request: Request): Promise<Response> };
};
expect(config.apiKey).toBe("sentinel-key");
const response = await config.httpClient?.request(new Request("https://api.mistral.ai/chat"));
expect(await response?.text()).toBe("guarded");
expect(hostFetch).toHaveBeenCalledTimes(1);
});
it("uses reasoning effort for Mistral Medium 3.5", async () => {
const stream = streamSimpleMistral(
{
+10 -2
View File
@@ -8,6 +8,7 @@ import type {
FunctionTool,
} from "@mistralai/mistralai/models/components";
import { getEnvApiKey } from "../env-api-keys.js";
import { getAiTransportHost } from "../host.js";
import { calculateCost, clampThinkingLevel } from "../model-utils.js";
import type {
AssistantMessage,
@@ -57,9 +58,10 @@ const MISTRAL_STREAM_BODY_MAX_BYTES = 16 * 1024 * 1024;
*/
export function createBoundedMistralFetcher(
maxBytes: number = MISTRAL_STREAM_BODY_MAX_BYTES,
upstreamFetch: Fetcher = fetch,
): Fetcher {
return async (input, init) => {
const response = init == null ? await fetch(input) : await fetch(input, init);
const response = init == null ? await upstreamFetch(input) : await upstreamFetch(input, init);
if (!response.body || typeof response.body.getReader !== "function") {
return response;
}
@@ -139,7 +141,13 @@ export const streamMistral: StreamFunction<"mistral-conversations", MistralOptio
// `httpClient` is passed, `ClientSDK.#httpClient` is set from it and
// every `chat.stream` / `complete` call routes through
// `HTTPClient.request` → `this.fetcher(req)`).
httpClient: new HTTPClient({ fetcher: createBoundedMistralFetcher() }),
// Mistral accepts HTTPClient.fetcher, so compose guarded egress with the byte cap.
httpClient: new HTTPClient({
fetcher: createBoundedMistralFetcher(
MISTRAL_STREAM_BODY_MAX_BYTES,
getAiTransportHost().buildModelFetch(model) ?? fetch,
),
}),
});
const normalizeMistralToolCallId = createMistralToolCallIdNormalizer();
@@ -3,6 +3,7 @@ import { zstdDecompressSync } from "node:zlib";
// ChatGPT Responses provider tests cover stream handling and timeout behavior.
import { MAX_TIMER_TIMEOUT_MS } from "@openclaw/normalization-core/number-coercion";
import { afterEach, describe, expect, it, vi } from "vitest";
import { configureAiTransportHost } from "../host.js";
import type { Context, Model } from "../types.js";
import { SYSTEM_PROMPT_CACHE_BOUNDARY } from "../utils/system-prompt-cache-boundary.js";
import {
@@ -104,6 +105,7 @@ describe("streamOpenAICodexResponses transport", () => {
vi.unstubAllGlobals();
vi.useRealTimers();
resetOpenAICodexWebSocketDebugStats();
configureAiTransportHost({});
});
const model = {
@@ -123,6 +125,41 @@ describe("streamOpenAICodexResponses transport", () => {
messages: [{ role: "user", content: "hi", timestamp: 1 }],
} satisfies Context;
it("unwraps sentinels before constructing ChatGPT SSE auth headers", async () => {
const realToken = createJwt({
"https://api.openai.com/auth": { chatgpt_account_id: "acct-sentinel" },
});
const sentinel = "oc-sent-v1-0123456789abcdef01234567";
configureAiTransportHost({
resolveSecretSentinel: (value) => value.replaceAll(sentinel, realToken),
});
let authorization: string | null = null;
let providerToken: string | null = null;
vi.stubGlobal(
"fetch",
vi.fn(async (_input, init) => {
const headers = new Headers(init?.headers);
authorization = headers.get("authorization");
providerToken = headers.get("x-provider-token");
return completedSseResponse();
}),
);
const result = await streamOpenAICodexResponses(
{ ...model, headers: { "X-Provider-Token": `Bearer ${sentinel}` } },
context,
{
apiKey: sentinel,
transport: "sse",
},
).result();
expect(result.stopReason).toBe("stop");
expect(authorization).toBe(`Bearer ${realToken}`);
expect(authorization).not.toContain(sentinel);
expect(providerToken).toBe(`Bearer ${realToken}`);
});
it("builds the first Node request with an OS-specific user agent", async () => {
vi.resetModules();
const freshProvider = await import("./openai-chatgpt-responses.js");
@@ -31,6 +31,7 @@ import {
clampTimerTimeoutMs,
} from "@openclaw/normalization-core/number-coercion";
import { getEnvApiKey } from "../env-api-keys.js";
import { getAiTransportHost, resolveAiTransportHeaderSentinels } from "../host.js";
import { registerSessionResourceCleanup } from "../session-resources.js";
import type {
Api,
@@ -264,10 +265,14 @@ export const streamOpenAICodexResponses: StreamFunction<
};
try {
const apiKey = options?.apiKey || getEnvApiKey(model.provider) || "";
if (!apiKey) {
const unresolvedApiKey = options?.apiKey || getEnvApiKey(model.provider) || "";
if (!unresolvedApiKey) {
throw new Error(`No API key for provider: ${model.provider}`);
}
// WebSocket auth has no fetch seam; unwrap immediately before request construction.
const apiKey = getAiTransportHost().resolveSecretSentinel(unresolvedApiKey);
const modelHeaders = resolveAiTransportHeaderSentinels(model.headers);
const optionHeaders = resolveAiTransportHeaderSentinels(options?.headers);
const accountId = extractOpenAICodexAccountId(apiKey);
let body = buildRequestBody(model, context, options);
@@ -281,15 +286,15 @@ export const streamOpenAICodexResponses: StreamFunction<
// see the SSE-path session_id addition in buildOpenAIClientHeaders (agents/openai-transport-stream.ts).
const websocketRequestId = options?.sessionId || createCodexRequestId();
const sseHeaders = buildSSEHeaders(
model.headers,
options?.headers,
modelHeaders,
optionHeaders,
accountId,
apiKey,
options?.sessionId,
);
const websocketHeaders = buildWebSocketHeaders(
model.headers,
options?.headers,
modelHeaders,
optionHeaders,
accountId,
apiKey,
websocketRequestId,
@@ -642,6 +642,7 @@ function createClient(
baseURL: isCloudflareProvider(model.provider) ? resolveCloudflareBaseUrl(model) : model.baseUrl,
dangerouslyAllowBrowser: true,
defaultHeaders,
// OpenAI supports custom fetch, so sentinels stay opaque until guarded egress.
fetch: getAiTransportHost().buildModelFetch(model),
});
}
@@ -0,0 +1,86 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import { configureAiTransportHost } from "../host.js";
import type { Context, Model } from "../types.js";
const openAiMockState = vi.hoisted(() => ({ configs: [] as unknown[] }));
vi.mock("openai", () => ({
default: class MockOpenAI {
responses = {
create: vi.fn(() => {
throw new Error("stop after constructor");
}),
};
constructor(config: unknown) {
openAiMockState.configs.push(config);
}
},
}));
import { streamOpenAIResponses } from "./openai-responses.js";
const context = {
messages: [{ role: "user", content: "hello", timestamp: 0 }],
} satisfies Context;
function model(overrides: Partial<Model<"openai-responses">> = {}) {
return {
id: "gpt-5.5",
name: "GPT-5.5",
api: "openai-responses",
provider: "openai",
baseUrl: "https://api.openai.com/v1",
reasoning: true,
input: ["text"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 200_000,
maxTokens: 8192,
...overrides,
} satisfies Model<"openai-responses">;
}
describe("OpenAI Responses provider", () => {
afterEach(() => {
openAiMockState.configs = [];
configureAiTransportHost({});
});
it("constructs the SDK client with the host guarded fetch", async () => {
const hostFetch: typeof fetch = async () => new Response(null, { status: 500 });
configureAiTransportHost({ buildModelFetch: () => hostFetch });
const result = await streamOpenAIResponses(model(), context, {
apiKey: "sentinel-key",
}).result();
expect(result.stopReason).toBe("error");
expect(openAiMockState.configs).toHaveLength(1);
expect((openAiMockState.configs[0] as { fetch?: unknown }).fetch).toBe(hostFetch);
});
it("keeps Cloudflare composed upstream auth opaque in SDK headers", async () => {
const hostFetch: typeof fetch = async () => new Response(null, { status: 500 });
configureAiTransportHost({ buildModelFetch: () => hostFetch });
await streamOpenAIResponses(
model({
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/account/gateway/openai",
}),
context,
{ apiKey: "oc-sent-v1-0123456789abcdef01234567" },
).result();
const config = openAiMockState.configs[0] as {
apiKey?: string;
defaultHeaders?: Record<string, string | null>;
fetch?: unknown;
};
expect(config.apiKey).toBe("oc-sent-v1-0123456789abcdef01234567");
expect(config.defaultHeaders?.["cf-aig-authorization"]).toBe(
"Bearer oc-sent-v1-0123456789abcdef01234567",
);
expect(config.fetch).toBe(hostFetch);
});
});
@@ -2,6 +2,7 @@
import OpenAI from "openai";
import type { ResponseCreateParamsStreaming } from "openai/resources/responses/responses.js";
import { getEnvApiKey } from "../env-api-keys.js";
import { getAiTransportHost } from "../host.js";
import type {
CacheRetention,
Context,
@@ -172,6 +173,8 @@ function createClient(
baseURL: isCloudflareProvider(model.provider) ? resolveCloudflareBaseUrl(model) : model.baseUrl,
dangerouslyAllowBrowser: true,
defaultHeaders,
// OpenAI supports custom fetch, so sentinels stay opaque until guarded egress.
fetch: getAiTransportHost().buildModelFetch(model),
});
}
@@ -93,6 +93,7 @@ function createLazyRegistration<TApi extends Api, TOptions extends StreamOptions
}
const registerBuiltIns: RegisterBuiltIn[] = [
// Registration is transport-free; each lazy adapter owns its fetch or construction unwrap.
createLazyRegistration(
"anthropic-messages",
() => import("./anthropic.js"),
+2 -2
View File
@@ -195,12 +195,12 @@ export function readPluginSdkSurfaceBudgets(env = process.env) {
),
publicExports: readPluginSdkSurfaceBudgetEnv(
"OPENCLAW_PLUGIN_SDK_MAX_PUBLIC_EXPORTS",
10463,
10464,
env,
),
publicFunctionExports: readPluginSdkSurfaceBudgetEnv(
"OPENCLAW_PLUGIN_SDK_MAX_PUBLIC_FUNCTION_EXPORTS",
5220,
5221,
env,
),
publicDeprecatedExports: readPluginSdkSurfaceBudgetEnv(
+10 -2
View File
@@ -62,6 +62,7 @@ import type {
ThinkingLevel,
} from "../llm/types.js";
import "../llm/ai-transport-host.js";
import { looksLikeSecretSentinel, resolveSecretSentinel } from "../secrets/sentinel.js";
import { MALFORMED_STREAMING_FRAGMENT_ERROR_MESSAGE } from "../shared/assistant-error-format.js";
import {
applyAnthropicPayloadPolicyToParams,
@@ -70,6 +71,7 @@ import {
import { buildCopilotDynamicHeaders, hasCopilotVisionInput } from "./copilot-dynamic-headers.js";
import { parseJsonObjectPreservingUnsafeIntegers } from "./json-unsafe-integers.js";
import { resolveProviderEndpoint } from "./provider-attribution.js";
import { unwrapModelHeaderSentinelsForProviderEgress } from "./provider-secret-egress.js";
import { buildGuardedModelFetch } from "./provider-transport-fetch.js";
import type { StreamFn } from "./runtime/index.js";
import { transformTransportMessages } from "./transport-message-transform.js";
@@ -288,7 +290,9 @@ function adjustMaxTokensForThinking(params: {
}
function isAnthropicOAuthToken(apiKey: string): boolean {
return apiKey.includes("sk-ant-oat");
// Auth routing may inspect the real shape, but guarded fetch still receives the sentinel.
const resolved = looksLikeSecretSentinel(apiKey) ? resolveSecretSentinel(apiKey) : apiKey;
return (resolved ?? apiKey).includes("sk-ant-oat");
}
function isDirectAnthropicModel(model: Pick<AnthropicTransportModel, "provider" | "baseUrl">) {
@@ -910,7 +914,11 @@ function createAnthropicTransportClient(params: {
isOAuthToken: false,
};
}
if (usesFoundryBearerAuth(model)) {
if (
usesFoundryBearerAuth(
unwrapModelHeaderSentinelsForProviderEgress(model, "Anthropic Foundry auth routing"),
)
) {
const betaFeatures = needsInterleavedBeta ? ["interleaved-thinking-2025-05-14"] : [];
return {
client: createAnthropicMessagesClient({
+1
View File
@@ -73,6 +73,7 @@ vi.mock("./embedded-agent-runner/model.js", () => ({
}));
vi.mock("./model-auth.js", () => ({
applySecretRefHeaderSentinels: (model: unknown) => model,
ensureAuthProfileStore: (...args: unknown[]) => ensureAuthProfileStoreMock(...args),
ensureAuthProfileStoreWithoutExternalProfiles: (...args: unknown[]) =>
ensureAuthProfileStoreWithoutExternalProfilesMock(...args),
+24 -13
View File
@@ -44,6 +44,7 @@ import {
import {
ensureAuthProfileStore,
ensureAuthProfileStoreWithoutExternalProfiles,
applySecretRefHeaderSentinels,
getApiKeyForModel,
requireApiKey,
} from "./model-auth.js";
@@ -54,6 +55,10 @@ import {
import { ensureOpenClawModelsJson } from "./models-config.js";
import { listOpenAIAuthProfileProvidersForAgentRuntime } from "./openai-routing.js";
import { applyPreparedRuntimeAuthToModel } from "./provider-request-config.js";
import {
protectPreparedProviderRuntimeAuth,
unwrapSecretSentinelsForProviderEgress,
} from "./provider-secret-egress.js";
import { registerProviderStreamForModel } from "./provider-stream.js";
import { stripToolResultDetails } from "./session-transcript-repair.js";
import { resolveAgentTimeoutMs } from "./timeout.js";
@@ -721,6 +726,7 @@ export async function runBtwSideQuestion(
profileId: effectiveAuthProfileId,
...(authStore ? { store: authStore } : {}),
agentDir: params.agentDir,
secretSentinels: true,
});
const resolvedAuthProfileId = apiKeyInfo.profileId ?? effectiveAuthProfileId;
let runtimeModel = model;
@@ -729,29 +735,34 @@ export async function runBtwSideQuestion(
? undefined
: requireApiKey(apiKeyInfo, model.provider);
if (apiKey) {
const preparedAuth = await prepareProviderRuntimeAuth({
const preparedAuth = protectPreparedProviderRuntimeAuth({
sourceApiKey: apiKey,
provider: model.provider,
config: params.cfg,
workspaceDir,
env: process.env,
context: {
preparedAuth: await prepareProviderRuntimeAuth({
provider: model.provider,
config: params.cfg,
agentDir: params.agentDir,
workspaceDir,
env: process.env,
provider: model.provider,
modelId: model.id,
model,
apiKey,
authMode: apiKeyInfo.mode,
profileId: resolvedAuthProfileId,
},
context: {
config: params.cfg,
agentDir: params.agentDir,
workspaceDir,
env: process.env,
provider: model.provider,
modelId: model.id,
model,
apiKey: unwrapSecretSentinelsForProviderEgress(apiKey, "provider runtime auth exchange"),
authMode: apiKeyInfo.mode,
profileId: resolvedAuthProfileId,
},
}),
});
runtimeModel = applyPreparedRuntimeAuthToModel(runtimeModel, preparedAuth);
if (preparedAuth?.apiKey) {
apiKey = preparedAuth.apiKey;
}
}
runtimeModel = applySecretRefHeaderSentinels(runtimeModel, params.cfg);
// Use the provider's own stream fn so providers like Ollama (which build
// `/api/chat` or `/v1/chat/completions` paths based on api mode) construct
+25 -13
View File
@@ -106,6 +106,10 @@ import { ensureOpenClawModelsJson } from "../models-config.js";
import { wrapStreamFnTextTransforms } from "../plugin-text-transforms.js";
import { resolveAgentPromptSurfaceForSessionKey } from "../prompt-surface.js";
import { applyPreparedRuntimeAuthToModel } from "../provider-request-config.js";
import {
protectPreparedProviderRuntimeAuth,
unwrapSecretSentinelsForProviderEgress,
} from "../provider-secret-egress.js";
import { registerProviderStreamForModel } from "../provider-stream.js";
import {
applyAgentRunSessionTargetIdentity,
@@ -687,6 +691,7 @@ async function compactEmbeddedAgentSessionDirectOnce(
profileId: authProfileId,
agentDir,
workspaceDir: resolvedWorkspace,
secretSentinels: true,
});
if (!apiKeyInfo.apiKey) {
@@ -694,23 +699,30 @@ async function compactEmbeddedAgentSessionDirectOnce(
throw new MissingProviderAuthError(runtimeModel.provider, apiKeyInfo);
}
} else {
const preparedAuth = await prepareProviderRuntimeAuth({
const preparedAuth = protectPreparedProviderRuntimeAuth({
sourceApiKey: apiKeyInfo.apiKey,
provider: runtimeModel.provider,
config: params.config,
workspaceDir: resolvedWorkspace,
env: process.env,
context: {
preparedAuth: await prepareProviderRuntimeAuth({
provider: runtimeModel.provider,
config: params.config,
agentDir,
workspaceDir: resolvedWorkspace,
env: process.env,
provider: runtimeModel.provider,
modelId,
model: runtimeModel,
apiKey: apiKeyInfo.apiKey,
authMode: apiKeyInfo.mode,
profileId: apiKeyInfo.profileId,
},
context: {
config: params.config,
agentDir,
workspaceDir: resolvedWorkspace,
env: process.env,
provider: runtimeModel.provider,
modelId,
model: runtimeModel,
apiKey: unwrapSecretSentinelsForProviderEgress(
apiKeyInfo.apiKey,
"provider runtime auth exchange",
),
authMode: apiKeyInfo.mode,
profileId: apiKeyInfo.profileId,
},
}),
});
runtimeModel = applyPreparedRuntimeAuthToModel(runtimeModel, preparedAuth);
const runtimeApiKey = preparedAuth?.apiKey ?? apiKeyInfo.apiKey;
@@ -3,6 +3,8 @@ import crypto from "node:crypto";
import type { StreamFn } from "openclaw/plugin-sdk/agent-core";
import type { Model } from "openclaw/plugin-sdk/llm";
import { describe, expect, it, vi } from "vitest";
import { isSecretValueRegisteredForRedaction } from "../../logging/secret-redaction-registry.js";
import { mintSecretSentinel, resolveSecretSentinel } from "../../secrets/sentinel.js";
import { prepareGooglePromptCacheStreamFn } from "./google-prompt-cache.js";
import { EmbeddedAttemptSessionTakeoverError } from "./run/attempt.session-lock.js";
@@ -149,6 +151,7 @@ function streamOptions(streamFn: { mock: { calls: unknown[][] } }, callIndex = 0
}
function preparePromptCacheStream(params: {
apiKey?: string;
fetchMock: ReturnType<typeof vi.fn>;
now: number;
sessionManager: TestGooglePromptCacheSessionManager;
@@ -158,7 +161,7 @@ function preparePromptCacheStream(params: {
// tests can focus on cache lifecycle behavior.
return prepareGooglePromptCacheStreamFn(
{
apiKey: "gemini-api-key",
apiKey: params.apiKey ?? "gemini-api-key",
extraParams: { cacheRetention: "long" },
model: makeGoogleModel(),
modelId: "gemini-3.1-pro-preview",
@@ -175,6 +178,70 @@ function preparePromptCacheStream(params: {
}
describe("google prompt cache", () => {
it("parses sentinel-backed OAuth JSON before guarded cache egress", async () => {
const fetchMock = createCacheFetchMock({
name: "cachedContents/oauth-cache",
expireTime: new Date(2_000_000).toISOString(),
});
const { streamFn } = createCapturingStreamFn();
const oauthJson = JSON.stringify({ token: "google-oauth-token", projectId: "demo" });
const sentinel = mintSecretSentinel(oauthJson, { label: "model-auth:google" });
const wrapped = await preparePromptCacheStream({
apiKey: sentinel,
fetchMock,
now: 1_000_000,
sessionManager: makeSessionManager([]),
streamFn,
});
await Promise.resolve(
wrapped?.(
makeGoogleModel(),
{ systemPrompt: "Follow policy.", messages: [] } as never,
{} as never,
),
);
const headers = fetchInit(fetchMock).headers as Record<string, string>;
expect(resolveSecretSentinel(headers.Authorization)).toBe("Bearer google-oauth-token");
expect(headers["x-goog-api-key"]).toBeUndefined();
expect(headers["Content-Type"]).toBe("application/json");
});
it("registers parsed OAuth headers when sentinels are disabled", async () => {
vi.stubEnv("OPENCLAW_SECRET_SENTINELS", "off");
const fetchMock = createCacheFetchMock({
name: "cachedContents/oauth-cache",
expireTime: new Date(2_000_000).toISOString(),
});
const { streamFn } = createCapturingStreamFn();
const oauthJson = JSON.stringify({ token: "google-kill-switch-token", projectId: "demo" });
const apiKey = mintSecretSentinel(oauthJson, { label: "model-auth:google" });
try {
const wrapped = await preparePromptCacheStream({
apiKey,
fetchMock,
now: 1_000_000,
sessionManager: makeSessionManager([]),
streamFn,
});
await Promise.resolve(
wrapped?.(
makeGoogleModel(),
{ systemPrompt: "Follow policy.", messages: [] } as never,
{} as never,
),
);
const headers = fetchInit(fetchMock).headers as Record<string, string>;
expect(headers.Authorization).toBe("Bearer google-kill-switch-token");
expect(isSecretValueRegisteredForRedaction(headers.Authorization)).toBe(true);
} finally {
vi.unstubAllEnvs();
}
});
it("creates cached content from the system prompt and strips that prompt from live requests", async () => {
// Cached system prompts should move out of live request context and into the
// cachedContent option to avoid paying prompt tokens repeatedly.
@@ -14,6 +14,12 @@ import { normalizeGoogleApiBaseUrl } from "../../infra/google-api-base-url.js";
import { readResponseWithLimit } from "../../infra/http-body.js";
import { streamWithPayloadPatch } from "../../llm/providers/stream-wrappers/stream-payload-utils.js";
import type { Model } from "../../llm/types.js";
import { isSecretValueRegisteredForRedaction } from "../../logging/secret-redaction-registry.js";
import {
looksLikeSecretSentinel,
mintSecretSentinel,
resolveSecretSentinel,
} from "../../secrets/sentinel.js";
import { resolveProviderRequestHeaders } from "../provider-request-config.js";
import { buildGuardedModelFetch } from "../provider-transport-fetch.js";
import type { StreamFn } from "../runtime/index.js";
@@ -296,13 +302,54 @@ async function readGooglePromptCacheJson<T>(response: Response): Promise<T> {
return JSON.parse(buffer.toString("utf8")) as T;
}
function resolveGooglePromptCacheAuthHeaders(params: {
apiKey: string;
provider: string;
}): Record<string, string> {
if (!looksLikeSecretSentinel(params.apiKey)) {
const headers = parseGeminiAuth(params.apiKey).headers;
if (!isSecretValueRegisteredForRedaction(params.apiKey)) {
return headers;
}
return Object.fromEntries(
Object.entries(headers).map(([name, value]) => [
name,
name.toLowerCase() === "authorization" || name.toLowerCase() === "x-goog-api-key"
? mintSecretSentinel(value, { label: `model-auth:${params.provider}` })
: value,
]),
);
}
const resolved = resolveSecretSentinel(params.apiKey);
if (resolved === undefined) {
throw new Error(
`Secret sentinel ${params.apiKey} is not registered in this process; refusing Google prompt-cache auth`,
);
}
return Object.fromEntries(
Object.entries(parseGeminiAuth(resolved).headers).map(([name, value]) => {
const isCredentialHeader =
name.toLowerCase() === "authorization" || name.toLowerCase() === "x-goog-api-key";
return [
name,
isCredentialHeader
? mintSecretSentinel(value, { label: `model-auth:${params.provider}` })
: value,
];
}),
);
}
function buildGooglePromptCacheHeaders(params: {
apiKey: string;
baseUrl: string;
headers?: Record<string, string>;
model: GooglePromptCacheModel;
}): Record<string, string> | undefined {
const authHeaders = parseGeminiAuth(params.apiKey).headers;
const authHeaders = resolveGooglePromptCacheAuthHeaders({
apiKey: params.apiKey,
provider: params.model.provider,
});
return (
resolveProviderRequestHeaders({
provider: params.model.provider,
@@ -1,6 +1,12 @@
// Coverage for embedded run auth initialization and runtime credential refresh.
import type { Model } from "openclaw/plugin-sdk/llm";
import { beforeEach, describe, expect, it, vi, type Mock } from "vitest";
import { isSecretValueRegisteredForRedaction } from "../../../logging/secret-redaction-registry.js";
import {
looksLikeSecretSentinel,
mintSecretSentinel,
resolveSecretSentinel,
} from "../../../secrets/sentinel.js";
import type { AuthProfileStore } from "../../auth-profiles.js";
import { FailoverError } from "../../failover-error.js";
import type { RuntimeAuthState } from "./helpers.js";
@@ -209,6 +215,87 @@ describe("createEmbeddedRunAuthController", () => {
expect(harness.runtimeAuthState?.profileId).toBe("default");
});
it("unwraps a sentinel for runtime auth exchange but keeps auth storage opaque", async () => {
const harness = createMutableAuthControllerHarness();
const setRuntimeApiKey = vi.fn<(provider: string, apiKey: string) => void>();
const secret = "runtime-exchange-source-secret";
const sentinel = mintSecretSentinel(secret, { label: "model-auth:custom-openai" });
mocks.getApiKeyForModel.mockResolvedValue({
apiKey: sentinel,
mode: "api-key",
source: "profile:custom-openai:default",
});
mocks.prepareProviderRuntimeAuth.mockResolvedValue({
apiKey: "runtime-exchange-token",
request: {
auth: {
mode: "header",
headerName: "api-key",
value: "runtime-header-token",
},
},
});
const controller = createMutableEmbeddedRunAuthController({ harness, setRuntimeApiKey });
await controller.initializeAuthProfile();
expect(mocks.getApiKeyForModel).toHaveBeenCalledWith(
expect.objectContaining({ secretSentinels: true }),
);
expect(mocks.prepareProviderRuntimeAuth).toHaveBeenCalledWith(
expect.objectContaining({ context: expect.objectContaining({ apiKey: secret }) }),
);
const storedApiKey = setRuntimeApiKey.mock.calls[0]?.[1];
expect(storedApiKey && looksLikeSecretSentinel(storedApiKey)).toBe(true);
expect(storedApiKey && resolveSecretSentinel(storedApiKey)).toBe("runtime-exchange-token");
const storedHeader = harness.runtimeModel.headers?.["api-key"];
expect(storedHeader && looksLikeSecretSentinel(storedHeader)).toBe(true);
expect(storedHeader && resolveSecretSentinel(storedHeader)).toBe("runtime-header-token");
});
it("preserves an empty runtime-auth result for fallback validation", async () => {
const harness = createMutableAuthControllerHarness();
const setRuntimeApiKey = vi.fn<(provider: string, apiKey: string) => void>();
const sentinel = mintSecretSentinel("runtime-source-secret", {
label: "model-auth:custom-openai",
});
mocks.getApiKeyForModel.mockResolvedValue({
apiKey: sentinel,
mode: "api-key",
source: "profile:custom-openai:default",
});
mocks.prepareProviderRuntimeAuth.mockResolvedValue({ apiKey: "" });
const controller = createMutableEmbeddedRunAuthController({ harness, setRuntimeApiKey });
await controller.initializeAuthProfile();
expect(setRuntimeApiKey).toHaveBeenCalledWith("custom-openai", sentinel);
});
it("registers exchanged credentials when sentinels are disabled", async () => {
vi.stubEnv("OPENCLAW_SECRET_SENTINELS", "off");
const harness = createMutableAuthControllerHarness();
const setRuntimeApiKey = vi.fn<(provider: string, apiKey: string) => void>();
const source = mintSecretSentinel("kill-switch-source-secret", {
label: "model-auth:custom-openai",
});
mocks.getApiKeyForModel.mockResolvedValue({
apiKey: source,
mode: "api-key",
source: "profile:custom-openai:default",
});
mocks.prepareProviderRuntimeAuth.mockResolvedValue({ apiKey: "kill-switch-runtime-token" });
try {
const controller = createMutableEmbeddedRunAuthController({ harness, setRuntimeApiKey });
await controller.initializeAuthProfile();
expect(setRuntimeApiKey).toHaveBeenCalledWith("custom-openai", "kill-switch-runtime-token");
expect(isSecretValueRegisteredForRedaction("kill-switch-runtime-token")).toBe(true);
} finally {
vi.unstubAllEnvs();
}
});
it("includes the checked credential source when an api key is missing", async () => {
const harness = createMutableAuthControllerHarness();
const setRuntimeApiKey = vi.fn<(provider: string, apiKey: string) => void>();
@@ -28,6 +28,10 @@ import {
applyPreparedRuntimeAuthToModel,
type ModelProviderRequestTransportOverrides,
} from "../../provider-request-config.js";
import {
protectPreparedProviderRuntimeAuth,
unwrapSecretSentinelsForProviderEgress,
} from "../../provider-secret-egress.js";
import { clampRuntimeAuthRefreshDelayMs } from "../../runtime-auth-refresh.js";
import {
RUNTIME_AUTH_REFRESH_MARGIN_MS,
@@ -117,8 +121,8 @@ export function createEmbeddedRunAuthController(params: {
apiKey: string;
authMode: string;
profileId?: string;
}) =>
prepareProviderRuntimeAuth({
}) => {
const preparedAuth = await prepareProviderRuntimeAuth({
provider: prepareParams.runtimeModel.provider,
config: params.config,
workspaceDir: params.workspaceDir,
@@ -131,11 +135,20 @@ export function createEmbeddedRunAuthController(params: {
provider: prepareParams.runtimeModel.provider,
modelId: params.getModelId(),
model: prepareParams.runtimeModel,
apiKey: prepareParams.apiKey,
apiKey: unwrapSecretSentinelsForProviderEgress(
prepareParams.apiKey,
"provider runtime auth exchange",
),
authMode: prepareParams.authMode,
profileId: prepareParams.profileId,
},
});
return protectPreparedProviderRuntimeAuth({
sourceApiKey: prepareParams.apiKey,
provider: prepareParams.runtimeModel.provider,
preparedAuth,
});
};
const clearRuntimeAuthRefreshTimer = () => {
const runtimeAuthState = params.getRuntimeAuthState();
@@ -373,6 +386,7 @@ export function createEmbeddedRunAuthController(params: {
agentDir: params.agentDir,
workspaceDir: params.workspaceDir,
lockedProfile: candidate != null && candidate === params.lockedProfileId,
secretSentinels: true,
});
};
@@ -5,6 +5,7 @@ import { SYSTEM_PROMPT_CACHE_BOUNDARY } from "@openclaw/ai/internal/shared";
import type { StreamFn } from "openclaw/plugin-sdk/agent-core";
import { afterEach, describe, expect, it, vi } from "vitest";
import { streamSimple } from "../../llm/stream.js";
import { mintSecretSentinel } from "../../secrets/sentinel.js";
import * as providerTransportStream from "../provider-transport-stream.js";
import {
testing,
@@ -124,6 +125,39 @@ describe("describeEmbeddedAgentStreamStrategy", () => {
});
describe("resolveEmbeddedAgentStreamFn", () => {
it("preserves sentinels for registered provider streams", async () => {
const secret = "plugin-stream-secret";
const sentinel = mintSecretSentinel(secret, { label: "model-auth:plugin" });
const providerStreamFn = vi.fn(async (model, _context, options) => ({ model, options }));
const model = {
api: "plugin-api",
provider: "plugin",
id: "plugin-model",
headers: { Authorization: `Bearer ${sentinel}` },
} as never;
const streamFn = resolveEmbeddedAgentStreamFn({
currentStreamFn: undefined,
providerStreamFn: providerStreamFn as never,
sessionId: "session-1",
model,
resolvedApiKey: sentinel,
});
const result = await expectStreamResultRecord(
streamFn(model, {} as never, {
headers: { "X-Managed": `Bearer ${sentinel}` },
}),
"plugin stream result",
);
expect(requireRecord(result.model, "plugin model").headers).toEqual({
Authorization: `Bearer ${sentinel}`,
});
expect(requireRecord(result.options, "plugin options").apiKey).toBe(sentinel);
expect(requireRecord(result.options, "plugin options").headers).toEqual({
"X-Managed": `Bearer ${sentinel}`,
});
});
it("prefers the resolved run api key over a later authStorage lookup", async () => {
const authStorage = {
getApiKey: vi.fn(async () => "storage-key"),
@@ -271,9 +271,10 @@ function wrapEmbeddedAgentStreamFn(
resolvedApiKey,
authStorage,
});
const selectedApiKey = apiKey ?? options?.apiKey;
return inner(m, transformContext(context), {
...mergeRunSignal(options),
apiKey: apiKey ?? options?.apiKey,
apiKey: selectedApiKey,
});
};
}
+28 -8
View File
@@ -1,3 +1,4 @@
import type { Model } from "openclaw/plugin-sdk/llm";
/**
* Routes compaction through selected native agent harnesses when supported.
*/
@@ -5,14 +6,17 @@ import { formatErrorMessage } from "../../infra/errors.js";
import { createSubsystemLogger } from "../../logging/subsystem.js";
import { parseAgentSessionKey } from "../../routing/session-key.js";
import { resolveUserPath } from "../../utils.js";
import type { Model } from "openclaw/plugin-sdk/llm";
import { isDefaultAgentRuntimeId, normalizeOptionalAgentRuntimeId } from "../agent-runtime-id.js";
import { resolveAgentDir, resolveSessionAgentIds } from "../agent-scope.js";
import type { CompactEmbeddedAgentSessionParams } from "../embedded-agent-runner/compact.types.js";
import { resolveModelAsync } from "../embedded-agent-runner/model.js";
import type { EmbeddedAgentCompactResult } from "../embedded-agent-runner/types.js";
import { getApiKeyForModel } from "../model-auth.js";
import { applySecretRefHeaderSentinels, getApiKeyForModel } from "../model-auth.js";
import { isCliRuntimeAliasForProvider, isCliRuntimeProvider } from "../model-runtime-aliases.js";
import {
unwrapModelHeaderSentinelsForProviderEgress,
unwrapSecretSentinelsForProviderEgress,
} from "../provider-secret-egress.js";
import { resolveAgentHarnessPolicy as resolveConfiguredAgentHarnessPolicy } from "./policy.js";
import { selectAgentHarness } from "./selection.js";
import type {
@@ -84,26 +88,28 @@ async function resolveHarnessCompactApiKey(params: {
if (!model) {
return existing ? { apiKey: existing } : {};
}
const runtimeModel = applySecretRefHeaderSentinels(model, compactParams.config);
if (existing) {
return { apiKey: existing, runtimeModel: model };
return { apiKey: existing, runtimeModel };
}
try {
const apiKeyInfo = await getApiKeyForModel({
model,
model: runtimeModel,
cfg: compactParams.config,
profileId: authProfileId,
agentDir,
workspaceDir,
secretSentinels: true,
});
return {
apiKey: apiKeyInfo.apiKey?.trim() || undefined,
runtimeModel: model,
runtimeModel,
};
} catch (err) {
log.debug("agent harness compaction credential lookup failed", {
error: formatErrorMessage(err),
});
return { runtimeModel: model };
return { runtimeModel };
}
}
@@ -195,8 +201,22 @@ export async function maybeCompactAgentHarnessSession(
resolvedApiKey || runtimeModel
? {
...compactParams,
...(resolvedApiKey ? { resolvedApiKey } : {}),
...(runtimeModel ? { runtimeModel } : {}),
...(resolvedApiKey
? {
resolvedApiKey: unwrapSecretSentinelsForProviderEgress(
resolvedApiKey,
"plugin harness compaction handoff",
),
}
: {}),
...(runtimeModel
? {
runtimeModel: unwrapModelHeaderSentinelsForProviderEgress(
runtimeModel,
"plugin harness compaction handoff",
),
}
: {}),
}
: compactParams;
if (shouldCompactAfterContextEngine) {
+33
View File
@@ -4,6 +4,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import type { OpenClawConfig } from "../../config/config.js";
import { OPENCLAW_EMBEDDED_CONTEXT_ENGINE_HOST } from "../../context-engine/host-compat.js";
import type { ContextEngine } from "../../context-engine/types.js";
import { mintSecretSentinel } from "../../secrets/sentinel.js";
import { testing as cliBackendsTesting } from "../cli-backends.js";
import type {
EmbeddedRunAttemptParams,
@@ -58,6 +59,7 @@ vi.mock("./builtin-openclaw.js", () => ({
}),
}));
vi.mock("../model-auth.js", () => ({
applySecretRefHeaderSentinels: (model: unknown) => model,
getApiKeyForModel: compactAuthMocks.getApiKeyForModel,
}));
vi.mock("../embedded-agent-runner/model.js", () => ({
@@ -290,6 +292,37 @@ function agentModelRuntimeConfig(
}
describe("runAgentHarnessAttempt", () => {
it("unwraps sentinels only at the plugin harness handoff", async () => {
const pluginRunAttempt = vi.fn<AgentHarness["runAttempt"]>(async () =>
createAttemptResult("codex"),
);
registerAgentHarness(
{
id: "codex",
label: "Codex",
supports: () => ({ supported: true, priority: 100 }),
runAttempt: pluginRunAttempt,
},
{ ownerPluginId: "codex" },
);
const secret = "plugin-provider-secret";
const sentinel = mintSecretSentinel(secret, { label: "model-auth:codex" });
const params = createAttemptParams(providerRuntimeConfig("codex", "codex"));
params.resolvedApiKey = sentinel;
params.model = {
...params.model,
headers: { Authorization: `Bearer ${sentinel}`, "X-Optional": null } as never,
};
await runAgentHarnessAttempt(params);
const handedOff = pluginRunAttempt.mock.calls[0]?.[0];
expect(handedOff?.resolvedApiKey).toBe(secret);
expect(handedOff?.model.headers?.Authorization).toBe(`Bearer ${secret}`);
expect(handedOff?.model.headers?.["X-Optional"]).toBeNull();
expect(params.resolvedApiKey).toBe(sentinel);
});
it("fails when a forced plugin harness is unavailable and fallback is omitted", async () => {
process.env.OPENCLAW_AGENT_RUNTIME = "codex";
+21 -2
View File
@@ -21,6 +21,10 @@ import type {
EmbeddedRunAttemptResult,
} from "../embedded-agent-runner/run/types.js";
import { isCliRuntimeAliasForProvider } from "../model-runtime-aliases.js";
import {
unwrapModelHeaderSentinelsForProviderEgress,
unwrapSecretSentinelsForProviderEgress,
} from "../provider-secret-egress.js";
import { resolveSandboxRuntimeStatus } from "../sandbox/runtime-status.js";
import { expandToolGroups, mergeAlsoAllowPolicy, normalizeToolName } from "../tool-policy.js";
import { createOpenClawAgentHarness } from "./builtin-openclaw.js";
@@ -372,8 +376,7 @@ export async function runAgentHarnessAttempt(
agentHarnessRuntimeOverride: params.agentHarnessRuntimeOverride,
});
const harness = selection.harness;
const attemptParams =
harness.id === "openclaw" ? params : applyPluginHarnessDenyAllToolPolicy(params);
const attemptParams = harness.id === "openclaw" ? params : preparePluginHarnessParams(params);
logAgentHarnessSelection(selection, {
provider: params.provider,
modelId: params.modelId,
@@ -398,6 +401,22 @@ export async function runAgentHarnessAttempt(
}
}
function preparePluginHarnessParams(params: EmbeddedRunAttemptParams): EmbeddedRunAttemptParams {
const boundary = "plugin harness handoff";
const resolvedApiKey = params.resolvedApiKey
? unwrapSecretSentinelsForProviderEgress(params.resolvedApiKey, boundary)
: params.resolvedApiKey;
const model = unwrapModelHeaderSentinelsForProviderEgress(params.model, boundary);
if (model === params.model && resolvedApiKey === params.resolvedApiKey) {
return applyPluginHarnessDenyAllToolPolicy(params);
}
return applyPluginHarnessDenyAllToolPolicy({
...params,
model,
resolvedApiKey,
});
}
function applyPluginHarnessDenyAllToolPolicy(
params: EmbeddedRunAttemptParams,
): EmbeddedRunAttemptParams {
+1
View File
@@ -66,6 +66,7 @@ describe("model auth markers", () => {
it("recognizes explicit non-secret markers", () => {
withEnv(cleanPluginManifestEnv(), () => {
expect(isNonSecretApiKeyMarker(NON_ENV_SECRETREF_MARKER)).toBe(true);
expect(isNonSecretApiKeyMarker("secretref-env:OPENAI_API_KEY")).toBe(true);
expect(isNonSecretApiKeyMarker(resolveOAuthApiKeyMarker("chutes"))).toBe(true);
expect(isNonSecretApiKeyMarker("ollama-local")).toBe(true);
expect(isNonSecretApiKeyMarker("lmstudio-local")).toBe(true);
+2 -1
View File
@@ -133,7 +133,8 @@ export function isNonSecretApiKeyMarker(
const isKnownMarker =
isOAuthApiKeyMarker(trimmed) ||
listKnownNonSecretApiKeyMarkers().includes(trimmed) ||
isAwsSdkAuthMarker(trimmed);
isAwsSdkAuthMarker(trimmed) ||
isSecretRefHeaderValueMarker(trimmed);
if (isKnownMarker) {
return true;
}
+334 -11
View File
@@ -9,6 +9,10 @@ import {
GCP_VERTEX_CREDENTIALS_MARKER,
NON_ENV_SECRETREF_MARKER,
} from "./model-auth-markers.js";
import {
attachModelProviderRequestTransport,
getModelProviderRequestTransport,
} from "./provider-request-config.js";
vi.mock("../plugins/plugin-registry.js", () => ({
loadPluginRegistrySnapshotWithMetadata: () => ({
@@ -146,6 +150,7 @@ vi.mock("../plugins/provider-runtime.js", async () => {
let applyAuthHeaderOverride: typeof import("./model-auth.js").applyAuthHeaderOverride;
let applyLocalNoAuthHeaderOverride: typeof import("./model-auth.js").applyLocalNoAuthHeaderOverride;
let applySecretRefHeaderSentinels: typeof import("./model-auth.js").applySecretRefHeaderSentinels;
let createRuntimeProviderAuthLookup: typeof import("./model-auth.js").createRuntimeProviderAuthLookup;
let formatMissingAuthError: typeof import("./model-auth.js").formatMissingAuthError;
let hasAvailableAuthForProvider: typeof import("./model-auth.js").hasAvailableAuthForProvider;
@@ -161,14 +166,18 @@ let resolveUsableCustomProviderApiKey: typeof import("./model-auth.js").resolveU
let cliCredentials: typeof import("./cli-credentials.js");
let clearRuntimeConfigSnapshot: typeof import("../config/config.js").clearRuntimeConfigSnapshot;
let setRuntimeConfigSnapshot: typeof import("../config/config.js").setRuntimeConfigSnapshot;
let looksLikeSecretSentinel: typeof import("../secrets/sentinel.js").looksLikeSecretSentinel;
let resolveSecretSentinel: typeof import("../secrets/sentinel.js").resolveSecretSentinel;
beforeAll(async () => {
vi.resetModules();
({ clearRuntimeConfigSnapshot, setRuntimeConfigSnapshot } = await import("../config/config.js"));
({ looksLikeSecretSentinel, resolveSecretSentinel } = await import("../secrets/sentinel.js"));
cliCredentials = await import("./cli-credentials.js");
({
applyAuthHeaderOverride,
applyLocalNoAuthHeaderOverride,
applySecretRefHeaderSentinels,
createRuntimeProviderAuthLookup,
formatMissingAuthError,
hasAvailableAuthForProvider,
@@ -287,6 +296,21 @@ function expectAuthFields(
}
}
function expectSecretSentinelAuth(
auth: Awaited<ReturnType<typeof resolveApiKeyForProvider>>,
expected: { value: string; source: string; mode: "api-key" | "oauth" },
) {
const apiKey = auth.apiKey;
expect(apiKey).toBeDefined();
if (!apiKey) {
throw new Error("expected model auth API key");
}
expect(looksLikeSecretSentinel(apiKey)).toBe(true);
expect(resolveSecretSentinel(apiKey)).toBe(expected.value);
expect(auth.source).toBe(expected.source);
expect(auth.mode).toBe(expected.mode);
}
describe("resolveAwsSdkEnvVarName", () => {
it("prefers bearer token over access keys and profile", () => {
const env = {
@@ -538,6 +562,7 @@ describe("resolveUsableCustomProviderApiKey", () => {
},
},
provider: "custom",
secretSentinels: true,
});
expect(resolved?.apiKey).toBe("sk-from-env");
expect(resolved?.source).toContain("OPENAI_API_KEY");
@@ -571,8 +596,10 @@ describe("resolveUsableCustomProviderApiKey", () => {
},
},
provider: "custom",
secretSentinels: true,
});
expect(resolved?.apiKey).toBe("sk-secretref-env");
expect(looksLikeSecretSentinel(resolved?.apiKey ?? "")).toBe(true);
expect(resolveSecretSentinel(resolved?.apiKey ?? "")).toBe("sk-secretref-env");
expect(resolved?.source).toContain("OPENAI_API_KEY");
} finally {
if (previous === undefined) {
@@ -583,6 +610,43 @@ describe("resolveUsableCustomProviderApiKey", () => {
}
});
it("sentinelizes config env SecretRefs on env-first provider resolution", async () => {
const previous = process.env.OPENAI_API_KEY;
process.env.OPENAI_API_KEY = "sk-secretref-env-first"; // pragma: allowlist secret
try {
const resolved = await resolveApiKeyForProvider({
cfg: {
models: {
providers: {
custom: {
baseUrl: "https://example.com/v1",
apiKey: {
source: "env",
provider: "default",
id: "OPENAI_API_KEY",
},
models: [],
},
},
},
},
provider: "custom",
credentialPrecedence: "env-first",
secretSentinels: true,
store: { version: 1, profiles: {} },
});
expect(looksLikeSecretSentinel(resolved.apiKey ?? "")).toBe(true);
expect(resolveSecretSentinel(resolved.apiKey ?? "")).toBe("sk-secretref-env-first");
} finally {
if (previous === undefined) {
delete process.env.OPENAI_API_KEY;
} else {
process.env.OPENAI_API_KEY = previous;
}
}
});
it("resolves env SecretRefs with unknown env IDs from process env for custom providers", () => {
const previous = process.env.MY_CUSTOM_KEY;
process.env.MY_CUSTOM_KEY = "sk-custom-secretref-env"; // pragma: allowlist secret
@@ -604,8 +668,10 @@ describe("resolveUsableCustomProviderApiKey", () => {
},
},
provider: "custom",
secretSentinels: true,
});
expect(resolved?.apiKey).toBe("sk-custom-secretref-env");
expect(looksLikeSecretSentinel(resolved?.apiKey ?? "")).toBe(true);
expect(resolveSecretSentinel(resolved?.apiKey ?? "")).toBe("sk-custom-secretref-env");
expect(resolved?.source).toContain("MY_CUSTOM_KEY");
} finally {
if (previous === undefined) {
@@ -634,8 +700,10 @@ describe("resolveUsableCustomProviderApiKey", () => {
},
},
provider: "bailian",
secretSentinels: true,
});
expect(resolved?.apiKey).toBe("sk-bailian-env");
expect(looksLikeSecretSentinel(resolved?.apiKey ?? "")).toBe(true);
expect(resolveSecretSentinel(resolved?.apiKey ?? "")).toBe("sk-bailian-env");
expect(resolved?.source).toContain("BAILIAN_API_KEY");
} finally {
if (previous === undefined) {
@@ -812,6 +880,68 @@ describe("resolveUsableCustomProviderApiKey", () => {
});
describe("resolveApiKeyForProvider", () => {
it("keeps plain environment credentials as plaintext", async () => {
const resolved = await withEnv("OPENAI_API_KEY", "sk-plain-env-key", () =>
resolveApiKeyForProvider({
provider: "openai",
store: { version: 1, profiles: {} },
}),
);
expect(resolved.apiKey).toBe("sk-plain-env-key");
expect(looksLikeSecretSentinel(resolved.apiKey ?? "")).toBe(false);
});
it("sentinelizes credentials resolved from auth-profile SecretRefs", async () => {
const profileId = "openai:secretref";
const resolved = await withEnv("OPENAI_PROFILE_SECRET", "sk-profile-secretref", () =>
resolveApiKeyForProvider({
provider: "openai",
profileId,
secretSentinels: true,
store: {
version: 1,
profiles: {
[profileId]: {
type: "api_key",
provider: "openai",
keyRef: { source: "env", provider: "default", id: "OPENAI_PROFILE_SECRET" },
},
},
},
}),
);
expectSecretSentinelAuth(resolved, {
value: "sk-profile-secretref",
source: `profile:${profileId}`,
mode: "api-key",
});
});
it("keeps SecretRef profile credentials request-ready outside model sentinel mode", async () => {
const profileId = "openai:non-model";
const resolved = await withEnv("OPENAI_NON_MODEL_SECRET", "sk-non-model-secret", () =>
resolveApiKeyForProvider({
provider: "openai",
profileId,
store: {
version: 1,
profiles: {
[profileId]: {
type: "api_key",
provider: "openai",
keyRef: { source: "env", provider: "default", id: "OPENAI_NON_MODEL_SECRET" },
},
},
},
}),
);
expect(resolved.apiKey).toBe("sk-non-model-secret");
expect(looksLikeSecretSentinel(resolved.apiKey ?? "")).toBe(false);
});
it("reuses plugin fallback auth without a models.providers entry", async () => {
const resolved = await withoutEnv("PLUGIN_WEB_API_KEY", () =>
resolveApiKeyForProvider({
@@ -873,12 +1003,13 @@ describe("resolveApiKeyForProvider", () => {
resolveApiKeyForProvider({
provider: "plugin-web",
cfg: sourceConfig,
secretSentinels: true,
store: { version: 1, profiles: {} },
}),
);
expectAuthFields(resolved, {
apiKey: "plugin-web-runtime-key",
expectSecretSentinelAuth(resolved, {
value: "plugin-web-runtime-key",
source: "plugins.entries.plugin-web.config.webSearch.apiKey",
mode: "api-key",
});
@@ -889,6 +1020,10 @@ describe("resolveApiKeyForProvider", () => {
name: "generated marker",
apiKey: NON_ENV_SECRETREF_MARKER,
},
{
name: "legacy env marker",
apiKey: "secretref-env:CLIPROXY_API_KEY",
},
{
name: "file SecretRef",
apiKey: { source: "file", provider: "vault", id: "/cliproxy/api-key" } as const,
@@ -921,11 +1056,12 @@ describe("resolveApiKeyForProvider", () => {
const resolved = await resolveApiKeyForProvider({
provider: "cliproxyapi",
cfg: sourceConfig,
secretSentinels: true,
store: { version: 1, profiles: {} },
});
expectAuthFields(resolved, {
apiKey: "sk-runtime-cliproxy",
expectSecretSentinelAuth(resolved, {
value: "sk-runtime-cliproxy",
source: "models.providers.cliproxyapi",
mode: "api-key",
});
@@ -945,6 +1081,68 @@ describe("resolveApiKeyForProvider", () => {
).toBe(true);
});
it("preserves SecretRef provenance for resolved runtime config clones", async () => {
const sourceConfig = {
models: {
providers: {
cliproxyapi: {
api: "openai-responses" as const,
apiKey: { source: "file", provider: "vault", id: "/cliproxy/api-key" } as const,
baseUrl: "https://cliproxy.example/v1",
models: [],
},
},
},
};
const runtimeConfig = {
models: {
providers: {
cliproxyapi: {
...sourceConfig.models.providers.cliproxyapi,
apiKey: "sk-runtime-clone", // pragma: allowlist secret
},
},
},
};
setRuntimeConfigSnapshot(runtimeConfig, sourceConfig);
const resolved = await resolveApiKeyForProvider({
provider: "cliproxyapi",
cfg: structuredClone(runtimeConfig),
secretSentinels: true,
store: { version: 1, profiles: {} },
});
expectSecretSentinelAuth(resolved, {
value: "sk-runtime-clone",
source: "models.providers.cliproxyapi",
mode: "api-key",
});
const preferred = await resolveApiKeyForProvider({
provider: "cliproxyapi",
cfg: structuredClone(runtimeConfig),
preferredProfile: "cliproxyapi:preferred",
credentialPrecedence: "profile-first",
secretSentinels: true,
store: {
version: 1,
profiles: {
"cliproxyapi:preferred": {
type: "api_key",
provider: "cliproxyapi",
key: "sk-preferred-profile", // pragma: allowlist secret
},
},
},
});
expectAuthFields(preferred, {
apiKey: "sk-preferred-profile",
source: "profile:cliproxyapi:preferred",
mode: "api-key",
});
});
it("does not treat a custom provider managed SecretRef marker as auth without a runtime snapshot", async () => {
const sourceConfig = {
models: {
@@ -1161,6 +1359,7 @@ describe("resolveApiKeyForProvider", () => {
const resolved = await resolveApiKeyForProvider({
provider: "cliproxyapi",
cfg: sourceConfig,
secretSentinels: true,
store: {
version: 1,
profiles: {
@@ -1173,8 +1372,8 @@ describe("resolveApiKeyForProvider", () => {
},
});
expectAuthFields(resolved, {
apiKey: "sk-runtime-cliproxy",
expectSecretSentinelAuth(resolved, {
value: "sk-runtime-cliproxy",
source: "models.providers.cliproxyapi",
mode: "api-key",
});
@@ -1385,11 +1584,12 @@ describe("resolveApiKeyForProvider synthetic local auth for custom providers
const auth = await resolveApiKeyForProvider({
provider: "ollama-gpu1",
cfg: sourceConfig,
secretSentinels: true,
store: { version: 1, profiles: {} },
});
expectAuthFields(auth, {
apiKey: "sk-runtime-ollama",
expectSecretSentinelAuth(auth, {
value: "sk-runtime-ollama",
source: "models.providers.ollama-gpu1",
mode: "api-key",
});
@@ -1749,6 +1949,129 @@ describe("applyAuthHeaderOverride", () => {
});
});
it("sentinelizes SecretRef-managed provider headers from the runtime snapshot", () => {
const sourceConfig = {
models: {
providers: {
google: {
baseUrl: "https://generativelanguage.googleapis.com/v1beta/openai",
api: "openai-completions" as const,
headers: {
Authorization: "secretref-env:GOOGLE_AUTH_TOKEN",
"X-Managed": NON_ENV_SECRETREF_MARKER,
},
models: [],
},
},
},
};
const runtimeConfig = {
models: {
providers: {
google: {
...sourceConfig.models.providers.google,
headers: {
Authorization: "Bearer runtime-google-secret",
"X-Managed": "runtime-managed-secret",
},
},
},
},
};
setRuntimeConfigSnapshot(runtimeConfig, sourceConfig);
const result = applyAuthHeaderOverride(
{
...baseModel,
headers: {
Authorization: "Bearer runtime-google-secret",
"X-Managed": "runtime-managed-secret",
"X-Plain": "visible",
},
},
null,
sourceConfig,
);
expect(looksLikeSecretSentinel(result.headers?.Authorization ?? "")).toBe(true);
expect(resolveSecretSentinel(result.headers?.Authorization ?? "")).toBe(
"Bearer runtime-google-secret",
);
expect(looksLikeSecretSentinel(result.headers?.["X-Managed"] ?? "")).toBe(true);
expect(resolveSecretSentinel(result.headers?.["X-Managed"] ?? "")).toBe(
"runtime-managed-secret",
);
expect(result.headers?.["X-Plain"]).toBe("visible");
});
it("sentinelizes SecretRef-managed request headers and composed auth", () => {
const sourceConfig = {
models: {
providers: {
google: {
baseUrl: "https://generativelanguage.googleapis.com/v1beta/openai",
api: "openai-completions" as const,
request: {
headers: { "X-Managed": NON_ENV_SECRETREF_MARKER },
auth: {
mode: "authorization-bearer" as const,
token: "secretref-env:GOOGLE_BEARER_TOKEN",
},
},
models: [],
},
},
},
};
const runtimeConfig = {
models: {
providers: {
google: {
...sourceConfig.models.providers.google,
request: {
headers: { "X-Managed": "runtime-managed-secret" },
auth: {
mode: "authorization-bearer" as const,
token: "runtime-bearer-secret",
},
},
},
},
},
};
setRuntimeConfigSnapshot(runtimeConfig, sourceConfig);
const result = applySecretRefHeaderSentinels(
attachModelProviderRequestTransport(
{
...baseModel,
headers: {
"X-Managed": "runtime-managed-secret",
Authorization: "Bearer runtime-bearer-secret",
},
},
runtimeConfig.models.providers.google.request,
),
sourceConfig,
);
const managedSentinel = result.headers?.["X-Managed"] ?? "";
expect(looksLikeSecretSentinel(managedSentinel)).toBe(true);
expect(resolveSecretSentinel(managedSentinel)).toBe("runtime-managed-secret");
const bearerSentinel = result.headers?.Authorization?.slice("Bearer ".length) ?? "";
expect(looksLikeSecretSentinel(bearerSentinel)).toBe(true);
expect(resolveSecretSentinel(bearerSentinel)).toBe("runtime-bearer-secret");
const request = getModelProviderRequestTransport(result);
const requestHeaderSentinel = request?.headers?.["X-Managed"] ?? "";
expect(looksLikeSecretSentinel(requestHeaderSentinel)).toBe(true);
expect(resolveSecretSentinel(requestHeaderSentinel)).toBe("runtime-managed-secret");
expect(request?.auth?.mode).toBe("authorization-bearer");
const requestTokenSentinel =
request?.auth?.mode === "authorization-bearer" ? request.auth.token : "";
expect(looksLikeSecretSentinel(requestTokenSentinel)).toBe(true);
expect(resolveSecretSentinel(requestTokenSentinel)).toBe("runtime-bearer-secret");
});
it("returns model unchanged when authHeader is not set", () => {
const result = applyAuthHeaderOverride(
baseModel,
+344 -32
View File
@@ -13,6 +13,7 @@ import { formatCliCommand } from "../cli/command-format.js";
import {
getRuntimeConfigSnapshot,
getRuntimeConfigSourceSnapshot,
hashRuntimeConfigValue,
selectApplicableRuntimeConfig,
} from "../config/config.js";
import type { ModelProviderAuthMode, ModelProviderConfig } from "../config/types.js";
@@ -30,6 +31,7 @@ import {
import { resolveOwningPluginIdsForProviderRef } from "../plugins/providers.js";
import { resolveRuntimeSyntheticAuthProviderRefState } from "../plugins/synthetic-auth.runtime.js";
import { resolveDefaultSecretProviderAlias } from "../secrets/ref-contract.js";
import { mintSecretSentinel } from "../secrets/sentinel.js";
import { normalizeOptionalSecretInput } from "../utils/normalize-secret-input.js";
import { resolveDefaultAgentDir } from "./agent-scope-config.js";
import {
@@ -56,10 +58,16 @@ import {
CUSTOM_LOCAL_AUTH_MARKER,
isKnownEnvApiKeyMarker,
isNonSecretApiKeyMarker,
isSecretRefHeaderValueMarker,
NON_ENV_SECRETREF_MARKER,
SECRETREF_ENV_HEADER_MARKER_PREFIX,
} from "./model-auth-markers.js";
import { ProviderAuthError, type ResolvedProviderAuth } from "./model-auth-runtime-shared.js";
import { normalizeProviderId } from "./model-selection.js";
import {
attachModelProviderRequestTransport,
getModelProviderRequestTransport,
} from "./provider-request-config.js";
export {
ensureAuthProfileStore,
@@ -78,6 +86,25 @@ export {
export type { ResolvedProviderAuth } from "./model-auth-runtime-shared.js";
export type ProviderCredentialPrecedence = "profile-first" | "env-first";
function sentinelizeSecretRefProfileApiKey(params: {
apiKey: string;
enabled?: boolean;
profileId: string;
provider: string;
store: AuthProfileStore;
}): string {
const credential = params.store.profiles[params.profileId];
const ref =
credential?.type === "api_key"
? coerceSecretRef(credential.keyRef)
: credential?.type === "token"
? coerceSecretRef(credential.tokenRef)
: null;
return ref && params.enabled
? mintSecretSentinel(params.apiKey, { label: `model-auth:${params.provider}` })
: params.apiKey;
}
/** Precomputed provider-auth lookup tables reused during one runtime turn. */
export type RuntimeProviderAuthLookup = {
envApiKey: Pick<
@@ -272,6 +299,7 @@ export function resolveUsableCustomProviderApiKey(params: {
cfg: OpenClawConfig | undefined;
provider: string;
env?: NodeJS.ProcessEnv;
secretSentinels?: boolean;
}): ResolvedCustomProviderApiKey | null {
const customProviderConfig = resolveProviderConfig(params.cfg, params.provider);
const apiKeyRef = coerceSecretRef(customProviderConfig?.apiKey);
@@ -298,7 +326,9 @@ export function resolveUsableCustomProviderApiKey(params: {
}
const applied = new Set(getShellEnvAppliedKeys());
return {
apiKey: envValue,
apiKey: params.secretSentinels
? mintSecretSentinel(envValue, { label: `model-auth:${params.provider}` })
: envValue,
source: resolveEnvSourceLabel({
applied,
envVars: [envVarName],
@@ -550,6 +580,7 @@ export async function resolveProviderEntryApiKeyBinding(params: {
provider: string;
store: AuthProfileStore;
agentDir?: string;
secretSentinels?: boolean;
}): Promise<ProviderEntryApiKeyBindingResolution> {
const reference = resolveProviderEntryApiKeyProfileReference(params);
if (reference.kind === "none" || reference.kind === "marker") {
@@ -575,7 +606,13 @@ export async function resolveProviderEntryApiKeyBinding(params: {
return {
kind: "profile-resolved",
auth: {
apiKey: resolved.apiKey,
apiKey: sentinelizeSecretRefProfileApiKey({
apiKey: resolved.apiKey,
enabled: params.secretSentinels,
profileId: resolvedProfileId,
provider: params.provider,
store: params.store,
}),
profileId: resolvedProfileId,
source: `profile:${resolvedProfileId}`,
mode: resolved.profileType ? profileTypeToAuthMode(resolved.profileType) : reference.mode,
@@ -659,16 +696,68 @@ function isManagedSecretRefApiKeyMarker(apiKey: string | undefined): boolean {
return apiKey?.trim() === NON_ENV_SECRETREF_MARKER;
}
function hasManagedSecretRefProviderApiKey(
cfg: OpenClawConfig | undefined,
provider: string,
): boolean {
function hasSecretRefProviderApiKey(cfg: OpenClawConfig | undefined, provider: string): boolean {
const apiKey = resolveProviderConfig(cfg, provider)?.apiKey;
const ref = coerceSecretRef(apiKey);
if (ref) {
return ref.source !== "env";
if (coerceSecretRef(apiKey)) {
return true;
}
return typeof apiKey === "string" && isManagedSecretRefApiKeyMarker(apiKey);
return (
typeof apiKey === "string" &&
(isManagedSecretRefApiKeyMarker(apiKey) ||
apiKey.trim().startsWith(SECRETREF_ENV_HEADER_MARKER_PREFIX))
);
}
function providerConfigMatchesRuntimeSnapshot(params: {
inputConfig: OpenClawConfig | undefined;
runtimeConfig: OpenClawConfig | null;
provider: string;
}): boolean {
const inputProvider = resolveProviderConfig(params.inputConfig, params.provider);
const runtimeProvider = resolveProviderConfig(params.runtimeConfig ?? undefined, params.provider);
if (!inputProvider || !runtimeProvider) {
return false;
}
const toComparableConfig = (providerConfig: ModelProviderConfig): OpenClawConfig => ({
models: { providers: { [params.provider]: providerConfig } },
});
return (
hashRuntimeConfigValue(toComparableConfig(inputProvider)) ===
hashRuntimeConfigValue(toComparableConfig(runtimeProvider))
);
}
function sentinelizeConfigSecretRefEnvApiKey(params: {
apiKey: string;
source: string;
cfg: OpenClawConfig | undefined;
provider: string;
enabled?: boolean;
}): string {
if (!params.enabled) {
return params.apiKey;
}
const runtimeConfig = getRuntimeConfigSnapshot();
const runtimeSourceConfig = getRuntimeConfigSourceSnapshot();
const sourceConfig = providerConfigMatchesRuntimeSnapshot({
inputConfig: params.cfg,
runtimeConfig,
provider: params.provider,
})
? (runtimeSourceConfig ?? params.cfg)
: params.cfg;
const configured = resolveProviderConfig(sourceConfig, params.provider)?.apiKey;
const ref = coerceSecretRef(configured);
const envId =
ref?.source === "env"
? ref.id
: typeof configured === "string" &&
configured.trim().startsWith(SECRETREF_ENV_HEADER_MARKER_PREFIX)
? configured.trim().slice(SECRETREF_ENV_HEADER_MARKER_PREFIX.length)
: undefined;
return envId && params.source.includes(envId)
? mintSecretSentinel(params.apiKey, { label: `model-auth:${params.provider}` })
: params.apiKey;
}
function resolveLiteralProviderConfigApiKeyAuth(params: {
@@ -691,10 +780,8 @@ function resolveLiteralProviderConfigApiKeyAuth(params: {
function resolveManagedSecretRefRuntimeProviderAuth(params: {
cfg: OpenClawConfig | undefined;
provider: string;
secretSentinels?: boolean;
}): ResolvedProviderAuth | undefined {
if (!hasManagedSecretRefProviderApiKey(params.cfg, params.provider)) {
return undefined;
}
const runtimeConfig = getRuntimeConfigSnapshot();
const runtimeSourceConfig = getRuntimeConfigSourceSnapshot();
if (params.cfg && params.cfg !== runtimeConfig && !runtimeSourceConfig) {
@@ -705,13 +792,35 @@ function resolveManagedSecretRefRuntimeProviderAuth(params: {
runtimeConfig,
runtimeSourceConfig,
});
if (!runtimeConfig || applicableConfig !== runtimeConfig) {
const usesRuntimeProvider =
applicableConfig === runtimeConfig ||
providerConfigMatchesRuntimeSnapshot({
inputConfig: params.cfg,
runtimeConfig,
provider: params.provider,
});
const sourceConfig = usesRuntimeProvider ? (runtimeSourceConfig ?? undefined) : params.cfg;
if (!hasSecretRefProviderApiKey(sourceConfig, params.provider)) {
return undefined;
}
return resolveLiteralProviderConfigApiKeyAuth({
if (!runtimeConfig || !usesRuntimeProvider) {
return undefined;
}
const resolved = resolveLiteralProviderConfigApiKeyAuth({
cfg: runtimeConfig,
provider: params.provider,
});
if (!resolved?.apiKey) {
return undefined;
}
return {
...resolved,
apiKey: params.secretSentinels
? mintSecretSentinel(resolved.apiKey, {
label: `model-auth:${params.provider}`,
})
: resolved.apiKey,
};
}
/** True when a custom local provider can use a synthetic no-auth placeholder. */
@@ -845,12 +954,13 @@ function resolveProviderSyntheticRuntimeAuth(params: {
cfg: OpenClawConfig | undefined;
provider: string;
modelApi?: string;
secretSentinels?: boolean;
}): SyntheticProviderAuthResolution {
const runtimeAuth = resolveManagedSecretRefRuntimeProviderAuth(params);
if (runtimeAuth) {
return { auth: runtimeAuth };
}
if (hasManagedSecretRefProviderApiKey(params.cfg, params.provider)) {
if (hasSecretRefProviderApiKey(params.cfg, params.provider)) {
return { blockedOnManagedSecretRef: true };
}
@@ -891,7 +1001,14 @@ function resolveProviderSyntheticRuntimeAuth(params: {
return { blockedOnManagedSecretRef: true };
}
return {
auth: runtimePluginAuth,
auth: {
...runtimePluginAuth,
apiKey: params.secretSentinels
? mintSecretSentinel(runtimeApiKey, {
label: `model-auth:${params.provider}`,
})
: runtimeApiKey,
},
};
}
@@ -899,6 +1016,7 @@ function resolveSyntheticLocalProviderAuth(params: {
cfg: OpenClawConfig | undefined;
provider: string;
modelApi?: string;
secretSentinels?: boolean;
}): ResolvedProviderAuth | null {
const syntheticProviderAuth = resolveProviderSyntheticRuntimeAuth(params);
if (syntheticProviderAuth.auth) {
@@ -1021,6 +1139,8 @@ export async function resolveApiKeyForProvider(params: {
forceRefresh?: boolean;
credentialPrecedence?: ProviderCredentialPrecedence;
modelApi?: string;
/** Keep SecretRef-backed model credentials opaque until a sentinel-aware transport boundary. */
secretSentinels?: boolean;
}): Promise<ResolvedProviderAuth> {
const { provider, cfg, profileId, preferredProfile } = params;
const agentDir = params.agentDir?.trim() || (cfg ? resolveDefaultAgentDir(cfg) : undefined);
@@ -1062,7 +1182,13 @@ export async function resolveApiKeyForProvider(params: {
const resolvedProfileId = resolved.profileId ?? profileId;
const mode = resolved.profileType ?? store.profiles[resolvedProfileId]?.type;
const result: ResolvedProviderAuth = {
apiKey: resolved.apiKey,
apiKey: sentinelizeSecretRefProfileApiKey({
apiKey: resolved.apiKey,
enabled: params.secretSentinels,
profileId: resolvedProfileId,
provider,
store,
}),
profileId: resolvedProfileId,
source: `profile:${resolvedProfileId}`,
mode: mode ? profileTypeToAuthMode(mode) : "api-key",
@@ -1147,7 +1273,13 @@ export async function resolveApiKeyForProvider(params: {
return resolveApiKeyForProvider({ ...params, credentialPrecedence: "profile-first" });
}
return {
apiKey: envResolved.apiKey,
apiKey: sentinelizeConfigSecretRefEnvApiKey({
apiKey: envResolved.apiKey,
source: envResolved.source,
cfg,
provider,
enabled: params.secretSentinels,
}),
source: envResolved.source,
mode: resolvedMode,
};
@@ -1168,6 +1300,7 @@ export async function resolveApiKeyForProvider(params: {
provider,
store: scopedStore,
agentDir,
secretSentinels: params.secretSentinels,
});
if (providerEntryBinding.kind === "profile-resolved") {
assertAuthModeAllowedForModel({
@@ -1201,11 +1334,19 @@ export async function resolveApiKeyForProvider(params: {
}
if (shouldPreferExplicitConfigApiKeyAuth(cfg, provider)) {
const runtimeCustomKey = resolveManagedSecretRefRuntimeProviderAuth({ cfg, provider });
const runtimeCustomKey = resolveManagedSecretRefRuntimeProviderAuth({
cfg,
provider,
secretSentinels: params.secretSentinels,
});
if (runtimeCustomKey) {
return runtimeCustomKey;
}
const customKey = resolveUsableCustomProviderApiKey({ cfg, provider });
const customKey = resolveUsableCustomProviderApiKey({
cfg,
provider,
secretSentinels: params.secretSentinels,
});
if (customKey) {
return {
apiKey: customKey.apiKey,
@@ -1215,7 +1356,11 @@ export async function resolveApiKeyForProvider(params: {
}
}
const providerConfig = resolveProviderConfig(cfg, provider);
const configuredLocalKey = resolveUsableCustomProviderApiKey({ cfg, provider });
const configuredLocalKey = resolveUsableCustomProviderApiKey({
cfg,
provider,
secretSentinels: params.secretSentinels,
});
if (configuredLocalKey && isNonSecretApiKeyMarker(configuredLocalKey.apiKey)) {
return {
apiKey: configuredLocalKey.apiKey,
@@ -1284,7 +1429,13 @@ export async function resolveApiKeyForProvider(params: {
? profileTypeToAuthMode(mode)
: "api-key";
const result: ResolvedProviderAuth = {
apiKey: resolved.apiKey,
apiKey: sentinelizeSecretRefProfileApiKey({
apiKey: resolved.apiKey,
enabled: params.secretSentinels,
profileId: resolvedProfileId,
provider,
store,
}),
profileId: resolvedProfileId,
source: `profile:${resolvedProfileId}`,
mode: resolvedMode,
@@ -1341,7 +1492,13 @@ export async function resolveApiKeyForProvider(params: {
})
) {
const result: ResolvedProviderAuth = {
apiKey: envResolved.apiKey,
apiKey: sentinelizeConfigSecretRefEnvApiKey({
apiKey: envResolved.apiKey,
source: envResolved.source,
cfg,
provider,
enabled: params.secretSentinels,
}),
source: envResolved.source,
mode: resolvedMode,
};
@@ -1349,7 +1506,20 @@ export async function resolveApiKeyForProvider(params: {
}
}
const customKey = resolveUsableCustomProviderApiKey({ cfg, provider });
const managedRuntimeAuth = resolveManagedSecretRefRuntimeProviderAuth({
cfg,
provider,
secretSentinels: params.secretSentinels,
});
if (managedRuntimeAuth) {
return managedRuntimeAuth;
}
const customKey = resolveUsableCustomProviderApiKey({
cfg,
provider,
secretSentinels: params.secretSentinels,
});
if (customKey) {
const result = { apiKey: customKey.apiKey, source: customKey.source, mode: "api-key" as const };
return result;
@@ -1363,6 +1533,7 @@ export async function resolveApiKeyForProvider(params: {
cfg,
provider,
modelApi: params.modelApi,
secretSentinels: params.secretSentinels,
});
if (syntheticLocalAuth) {
return syntheticLocalAuth;
@@ -1579,6 +1750,7 @@ export async function getApiKeyForModel(params: {
workspaceDir?: string;
lockedProfile?: boolean;
credentialPrecedence?: ProviderCredentialPrecedence;
secretSentinels?: boolean;
}): Promise<ResolvedProviderAuth> {
return resolveApiKeyForProvider({
provider: params.model.provider,
@@ -1591,6 +1763,7 @@ export async function getApiKeyForModel(params: {
lockedProfile: params.lockedProfile,
credentialPrecedence: params.credentialPrecedence,
modelApi: params.model.api,
secretSentinels: params.secretSentinels,
});
}
@@ -1617,6 +1790,144 @@ export function applyLocalNoAuthHeaderOverride<T extends Model>(
};
}
export function applySecretRefHeaderSentinels<T extends Model>(
model: T,
cfg: OpenClawConfig | undefined,
): T {
if (!model.headers) {
return model;
}
const runtimeConfig = getRuntimeConfigSnapshot();
const runtimeSourceConfig = getRuntimeConfigSourceSnapshot();
const applicableConfig = selectApplicableRuntimeConfig({
inputConfig: cfg,
runtimeConfig,
runtimeSourceConfig,
});
const usesRuntimeProvider =
applicableConfig === runtimeConfig ||
providerConfigMatchesRuntimeSnapshot({
inputConfig: cfg,
runtimeConfig,
provider: model.provider,
});
if (!runtimeConfig || !runtimeSourceConfig || !usesRuntimeProvider) {
return model;
}
const sourceProvider = resolveProviderConfig(runtimeSourceConfig, model.provider);
const runtimeProvider = resolveProviderConfig(runtimeConfig, model.provider);
const replacements = new Map<string, { value: string; replacement: string }>();
const isManagedSecret = (value: unknown) =>
coerceSecretRef(value) !== null ||
(typeof value === "string" && isSecretRefHeaderValueMarker(value));
const addReplacement = (name: string, value: string, replacement?: string) => {
replacements.set(name.trim().toLowerCase(), {
value,
replacement:
replacement ?? mintSecretSentinel(value, { label: `model-auth:${model.provider}` }),
});
};
for (const [name, sourceValue] of Object.entries(sourceProvider?.headers ?? {})) {
if (!isManagedSecret(sourceValue)) {
continue;
}
const value = normalizeOptionalSecretInput(runtimeProvider?.headers?.[name]);
if (value) {
addReplacement(name, value);
}
}
for (const [name, sourceValue] of Object.entries(sourceProvider?.request?.headers ?? {})) {
if (!isManagedSecret(sourceValue)) {
continue;
}
const value = normalizeOptionalSecretInput(runtimeProvider?.request?.headers?.[name]);
if (value) {
addReplacement(name, value);
}
}
const sourceAuth = sourceProvider?.request?.auth;
const runtimeAuth = runtimeProvider?.request?.auth;
const attachedRequest = getModelProviderRequestTransport(model);
let protectedRequest = attachedRequest;
let protectedRequestHeaders: Record<string, string> | undefined;
for (const [name, sourceValue] of Object.entries(sourceProvider?.request?.headers ?? {})) {
if (!isManagedSecret(sourceValue)) {
continue;
}
const value = normalizeOptionalSecretInput(runtimeProvider?.request?.headers?.[name]);
if (!value || attachedRequest?.headers?.[name] !== value) {
continue;
}
protectedRequestHeaders ??= { ...attachedRequest.headers };
protectedRequestHeaders[name] = mintSecretSentinel(value, {
label: `model-auth:${model.provider}`,
});
}
if (protectedRequestHeaders && attachedRequest) {
protectedRequest = { ...attachedRequest, headers: protectedRequestHeaders };
}
if (
sourceAuth?.mode === "authorization-bearer" &&
runtimeAuth?.mode === "authorization-bearer" &&
isManagedSecret(sourceAuth.token)
) {
const token = normalizeOptionalSecretInput(runtimeAuth.token)?.trim();
if (token) {
if (attachedRequest?.auth?.mode === "authorization-bearer") {
protectedRequest = {
...protectedRequest,
auth: {
...attachedRequest.auth,
token: mintSecretSentinel(token, { label: `model-auth:${model.provider}` }),
},
};
}
addReplacement(
"Authorization",
`Bearer ${token}`,
`Bearer ${mintSecretSentinel(token, { label: `model-auth:${model.provider}` })}`,
);
}
} else if (
sourceAuth?.mode === "header" &&
runtimeAuth?.mode === "header" &&
isManagedSecret(sourceAuth.value)
) {
const value = normalizeOptionalSecretInput(runtimeAuth.value)?.trim();
const headerName = runtimeAuth.headerName.trim();
const prefix = runtimeAuth.prefix?.trim() ?? "";
if (headerName && value) {
if (attachedRequest?.auth?.mode === "header") {
protectedRequest = {
...protectedRequest,
auth: {
...attachedRequest.auth,
value: mintSecretSentinel(value, { label: `model-auth:${model.provider}` }),
},
};
}
addReplacement(
headerName,
`${prefix}${value}`,
`${prefix}${mintSecretSentinel(value, { label: `model-auth:${model.provider}` })}`,
);
}
}
let headers: Record<string, string> | undefined;
for (const [name, value] of Object.entries(model.headers)) {
const replacement = replacements.get(name.trim().toLowerCase());
if (replacement?.value !== value) {
continue;
}
headers ??= { ...model.headers };
headers[name] = replacement.replacement;
}
const protectedModel = headers ? { ...model, headers } : model;
return protectedRequest && protectedRequest !== attachedRequest
? attachModelProviderRequestTransport(protectedModel, protectedRequest)
: protectedModel;
}
/**
* When the provider config sets `authHeader: true`, inject an explicit
* `Authorization: Bearer <apiKey>` header into the model so downstream SDKs
@@ -1632,23 +1943,24 @@ export function applyAuthHeaderOverride<T extends Model>(
auth: ResolvedProviderAuth | null | undefined,
cfg: OpenClawConfig | undefined,
): T {
const sentinelModel = applySecretRefHeaderSentinels(model, cfg);
if (!auth?.apiKey) {
return model;
return sentinelModel;
}
// Reject synthetic marker values that are not real credentials.
if (isNonSecretApiKeyMarker(auth.apiKey)) {
return model;
return sentinelModel;
}
const providerConfig = resolveProviderConfig(cfg, model.provider);
const providerConfig = resolveProviderConfig(cfg, sentinelModel.provider);
if (!providerConfig?.authHeader) {
return model;
return sentinelModel;
}
// Strip any existing authorization header (case-insensitive) before
// injecting the canonical one so we don't produce a comma-joined value.
const headers: Record<string, string> = {};
if (model.headers) {
for (const [key, value] of Object.entries(model.headers)) {
if (sentinelModel.headers) {
for (const [key, value] of Object.entries(sentinelModel.headers)) {
if (normalizeOptionalLowercaseString(key) !== "authorization") {
headers[key] = value;
}
@@ -1657,7 +1969,7 @@ export function applyAuthHeaderOverride<T extends Model>(
headers.Authorization = `Bearer ${auth.apiKey}`;
return {
...model,
...sentinelModel,
headers,
};
}
+27 -1
View File
@@ -6,6 +6,7 @@ import path from "node:path";
import { MAX_TIMER_TIMEOUT_MS } from "@openclaw/normalization-core/number-coercion";
import type { Model } from "openclaw/plugin-sdk/llm";
import { afterEach, describe, expect, it, vi } from "vitest";
import { mintSecretSentinel } from "../secrets/sentinel.js";
import { killPidIfAlive, readPidFile, waitForPidToExit } from "../test-utils/process-tree.js";
import {
attachModelProviderLocalService,
@@ -167,8 +168,9 @@ describe("provider local service", () => {
},
);
const sentinel = mintSecretSentinel("health-secret", { label: "local-health-probe" });
const lease = await ensureModelProviderLocalService(model, {
Authorization: "Bearer health-secret",
Authorization: `Bearer ${sentinel}`,
"X-Tenant": "acme",
});
@@ -187,6 +189,30 @@ describe("provider local service", () => {
await waitForProbeFailure(healthUrl);
});
it("rejects unknown sentinels before starting a local service", async () => {
const port = await freePort();
const unknown = "oc-sent-v1-fedcba987654321001234567";
const model = attachModelProviderLocalService(
{
id: "demo",
provider: "local-unknown-auth",
api: "openai-completions",
baseUrl: `http://127.0.0.1:${port}/v1`,
} as unknown as Model<"openai-completions">,
{
command: process.execPath,
args: ["--version"],
readyTimeoutMs: 1_000,
},
);
await expect(
ensureModelProviderLocalService(model, { Authorization: `Bearer ${unknown}` }),
).rejects.toThrow(
`Secret sentinel ${unknown} is not registered in this process; refusing to probe local model provider health`,
);
});
it("cancels local service health probe response bodies", async () => {
let socketClosed = false;
const sockets = new Set<net.Socket>();
+8 -1
View File
@@ -17,6 +17,7 @@ import {
signalChildProcessTree,
shouldDetachChildForProcessTree,
} from "../process/child-process-tree.js";
import { unwrapHeadersInitSentinelsForProviderEgress } from "./provider-secret-egress.js";
const log = createSubsystemLogger("provider-local-service");
const DEFAULT_READY_TIMEOUT_MS = 120_000;
@@ -222,6 +223,12 @@ async function probeHealth(
signal?: AbortSignal | null,
): Promise<boolean> {
throwIfAborted(signal);
// Local-service orchestration retains sentinel headers across retries. Only
// the actual health request may materialize credentials.
const egressHeaders = unwrapHeadersInitSentinelsForProviderEgress(
headers,
"to probe local model provider health",
);
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), DEFAULT_PROBE_TIMEOUT_MS);
timeout.unref?.();
@@ -229,7 +236,7 @@ async function probeHealth(
signal?.addEventListener("abort", onAbort, { once: true });
let response: Response | undefined;
try {
response = await fetch(url, { headers, signal: controller.signal });
response = await fetch(url, { headers: egressHeaders, signal: controller.signal });
return response.ok;
} catch {
if (signal?.aborted) {
+91
View File
@@ -0,0 +1,91 @@
import { describe, expect, it } from "vitest";
import { mintSecretSentinel } from "../secrets/sentinel.js";
import {
attachModelProviderRequestTransport,
getModelProviderRequestTransport,
} from "./provider-request-config.js";
import { unwrapModelHeaderSentinelsForProviderEgress } from "./provider-secret-egress.js";
describe("unwrapModelHeaderSentinelsForProviderEgress", () => {
it("unwraps sentinels in visible headers and attached request transport overrides", () => {
const headerSecret = "egress-visible-header-secret";
const bearerSecret = "egress-runtime-bearer-secret";
const overrideHeaderSecret = "egress-override-header-secret";
const model = attachModelProviderRequestTransport(
{
id: "test-model",
headers: {
"x-api-key": mintSecretSentinel(headerSecret, { label: "egress-test:visible" }),
},
},
{
headers: {
"x-extra": mintSecretSentinel(overrideHeaderSecret, { label: "egress-test:override" }),
},
auth: {
mode: "authorization-bearer",
token: mintSecretSentinel(bearerSecret, { label: "egress-test:bearer" }),
},
},
);
const unwrapped = unwrapModelHeaderSentinelsForProviderEgress(model, "egress test");
expect(unwrapped.headers?.["x-api-key"]).toBe(headerSecret);
const request = getModelProviderRequestTransport(unwrapped);
expect(request?.headers?.["x-extra"]).toBe(overrideHeaderSecret);
expect(request?.auth).toEqual({ mode: "authorization-bearer", token: bearerSecret });
// Original model stays sentineled: unwrap must not mutate shared state.
expect(model.headers["x-api-key"]).not.toBe(headerSecret);
expect(getModelProviderRequestTransport(model)?.auth).not.toEqual(request?.auth);
});
it("unwraps header-mode auth values in attached request transport overrides", () => {
const headerAuthSecret = "egress-header-auth-secret";
const model = attachModelProviderRequestTransport(
{ id: "test-model", headers: undefined },
{
auth: {
mode: "header",
headerName: "x-goog-api-key",
value: mintSecretSentinel(headerAuthSecret, { label: "egress-test:header-auth" }),
},
},
);
const request = getModelProviderRequestTransport(
unwrapModelHeaderSentinelsForProviderEgress(model, "egress test"),
);
expect(request?.auth).toEqual({
mode: "header",
headerName: "x-goog-api-key",
value: headerAuthSecret,
});
});
it("returns the same model instance when nothing is sentineled", () => {
const model = attachModelProviderRequestTransport(
{ id: "test-model", headers: { "x-plain": "plain-value" } },
{ auth: { mode: "provider-default" } },
);
expect(unwrapModelHeaderSentinelsForProviderEgress(model, "egress test")).toBe(model);
});
it("rejects unknown sentinel-shaped values in attached overrides", () => {
const model = attachModelProviderRequestTransport(
{ id: "test-model", headers: undefined },
{
auth: {
mode: "authorization-bearer",
token: "oc-sent-v1-00000000000000000000dead",
},
},
);
expect(() => unwrapModelHeaderSentinelsForProviderEgress(model, "egress test")).toThrow(
/not registered in this process/,
);
});
});
+184
View File
@@ -0,0 +1,184 @@
import { isSecretValueRegisteredForRedaction } from "../logging/secret-redaction-registry.js";
import {
looksLikeSecretSentinel,
mintSecretSentinel,
swapSecretSentinelsInText,
} from "../secrets/sentinel.js";
import {
attachModelProviderRequestTransport,
getModelProviderRequestTransport,
type ModelProviderRequestTransportOverrides,
} from "./provider-request-config.js";
type PreparedProviderRuntimeAuth = {
apiKey: string;
baseUrl?: string;
request?: ModelProviderRequestTransportOverrides;
expiresAt?: number;
};
function protectRuntimeAuthValue(params: {
value: string;
provider: string;
label: string;
}): string {
if (!params.value) {
return params.value;
}
return looksLikeSecretSentinel(params.value)
? params.value
: mintSecretSentinel(params.value, {
label: `model-auth:${params.provider}:${params.label}`,
});
}
/** Re-sentinels credentials returned by a provider auth exchange. */
export function protectPreparedProviderRuntimeAuth(params: {
sourceApiKey: string;
provider: string;
preparedAuth: PreparedProviderRuntimeAuth | null | undefined;
}): PreparedProviderRuntimeAuth | undefined {
const { preparedAuth } = params;
if (!preparedAuth) {
return undefined;
}
if (
!looksLikeSecretSentinel(params.sourceApiKey) &&
!isSecretValueRegisteredForRedaction(params.sourceApiKey)
) {
return preparedAuth;
}
const protect = (value: string, label: string) =>
protectRuntimeAuthValue({ value, provider: params.provider, label });
const request = preparedAuth.request;
const headers = request?.headers
? Object.fromEntries(
Object.entries(request.headers).map(([name, value]) => [
name,
protect(value, `runtime-header:${name.toLowerCase()}`),
]),
)
: undefined;
const auth = request?.auth;
const protectedAuth =
auth?.mode === "authorization-bearer"
? { ...auth, token: protect(auth.token, "runtime-bearer") }
: auth?.mode === "header"
? {
...auth,
value: protect(auth.value, `runtime-auth-header:${auth.headerName.toLowerCase()}`),
}
: auth;
return {
...preparedAuth,
apiKey: protect(preparedAuth.apiKey, "runtime-api-key"),
...(request
? {
request: {
...request,
...(headers ? { headers } : {}),
...(protectedAuth ? { auth: protectedAuth } : {}),
},
}
: {}),
};
}
export function unwrapSecretSentinelsForProviderEgress(value: string, boundary: string): string {
const swapped = swapSecretSentinelsInText(value);
const unknown = swapped.unknown[0];
if (unknown) {
throw new Error(
`Secret sentinel ${unknown} is not registered in this process; refusing ${boundary}`,
);
}
return swapped.text;
}
export function unwrapHeaderSentinelsForProviderEgress<T extends Record<string, unknown>>(
input: T,
boundary: string,
): T {
let headers: Record<string, unknown> | undefined;
for (const [name, value] of Object.entries(input)) {
if (typeof value !== "string") {
continue;
}
const resolved = unwrapSecretSentinelsForProviderEgress(value, boundary);
if (resolved !== value) {
headers ??= { ...input };
headers[name] = resolved;
}
}
return headers ? (headers as T) : input;
}
export function unwrapHeadersInitSentinelsForProviderEgress(
input: HeadersInit | undefined,
boundary: string,
): HeadersInit | undefined {
if (!input) {
return input;
}
const headers = new Headers(input);
let changed = false;
for (const [name, value] of headers) {
const resolved = unwrapSecretSentinelsForProviderEgress(value, boundary);
if (resolved !== value) {
headers.set(name, resolved);
changed = true;
}
}
return changed ? headers : input;
}
function unwrapRequestTransportSentinelsForProviderEgress(
request: ModelProviderRequestTransportOverrides | undefined,
boundary: string,
): ModelProviderRequestTransportOverrides | undefined {
if (!request) {
return request;
}
const headers = request.headers
? unwrapHeaderSentinelsForProviderEgress(request.headers, boundary)
: request.headers;
let auth = request.auth;
if (auth?.mode === "authorization-bearer") {
const token = unwrapSecretSentinelsForProviderEgress(auth.token, boundary);
if (token !== auth.token) {
auth = { ...auth, token };
}
} else if (auth?.mode === "header") {
const value = unwrapSecretSentinelsForProviderEgress(auth.value, boundary);
if (value !== auth.value) {
auth = { ...auth, value };
}
}
if (headers === request.headers && auth === request.auth) {
return request;
}
return {
...request,
...(headers ? { headers } : {}),
...(auth ? { auth } : {}),
};
}
export function unwrapModelHeaderSentinelsForProviderEgress<
T extends { headers?: Record<string, unknown> },
>(model: T, boundary: string): T {
// Plugin transports read both visible headers and the symbol-attached request
// overrides; both can carry sentinels minted by protectPreparedProviderRuntimeAuth.
const headers = model.headers
? unwrapHeaderSentinelsForProviderEgress(model.headers, boundary)
: model.headers;
const request = getModelProviderRequestTransport(model);
const unwrappedRequest = unwrapRequestTransportSentinelsForProviderEgress(request, boundary);
if (headers === model.headers && unwrappedRequest === request) {
return model;
}
const next = headers === model.headers ? ({ ...model } as T) : ({ ...model, headers } as T);
return unwrappedRequest === request
? next
: attachModelProviderRequestTransport(next, unwrappedRequest);
}
+60 -19
View File
@@ -7,6 +7,11 @@ import type { OpenClawConfig } from "../config/types.openclaw.js";
import type { Api, Model } from "../llm/types.js";
import { resolveProviderStreamFn } from "../plugins/provider-runtime.js";
import { ensureCustomApiRegistered } from "./custom-api-registry.js";
import {
unwrapHeaderSentinelsForProviderEgress,
unwrapModelHeaderSentinelsForProviderEgress,
unwrapSecretSentinelsForProviderEgress,
} from "./provider-secret-egress.js";
import { createTransportAwareStreamFnForModel } from "./provider-transport-stream.js";
import type { StreamFn } from "./runtime/index.js";
@@ -19,28 +24,43 @@ export function registerProviderStreamForModel<TApi extends Api>(params: {
env?: NodeJS.ProcessEnv;
allowRuntimePluginLoad?: boolean;
}): StreamFn | undefined {
const streamFn =
resolveProviderStreamFn({
provider: params.model.provider,
// Plugin stream factories may capture model headers, so construction is the
// last safe boundary for providers that do not expose the host fetch seam.
const pluginModel = unwrapModelHeaderSentinelsForProviderEgress(
params.model,
"plugin provider stream construction",
);
const providerStreamFn = resolveProviderStreamFn({
provider: params.model.provider,
config: params.cfg,
workspaceDir: params.workspaceDir,
env: params.env,
allowRuntimePluginLoad: params.allowRuntimePluginLoad,
context: {
config: params.cfg,
workspaceDir: params.workspaceDir,
env: params.env,
allowRuntimePluginLoad: params.allowRuntimePluginLoad,
context: {
config: params.cfg,
agentDir: params.agentDir,
workspaceDir: params.workspaceDir,
provider: params.model.provider,
modelId: params.model.id,
model: params.model,
},
}) ??
createTransportAwareStreamFnForModel(params.model, {
cfg: params.cfg,
agentDir: params.agentDir,
workspaceDir: params.workspaceDir,
env: params.env,
});
provider: params.model.provider,
modelId: params.model.id,
model: pluginModel,
},
});
const transportFallback = providerStreamFn
? undefined
: createTransportAwareStreamFnForModel(
params.model.api === "google-generative-ai" ? pluginModel : params.model,
{
cfg: params.cfg,
agentDir: params.agentDir,
workspaceDir: params.workspaceDir,
env: params.env,
},
);
const streamFn = providerStreamFn
? wrapPluginProviderStream(providerStreamFn)
: transportFallback && params.model.api === "google-generative-ai"
? wrapPluginProviderStream(transportFallback)
: transportFallback;
if (!streamFn) {
return undefined;
}
@@ -49,3 +69,24 @@ export function registerProviderStreamForModel<TApi extends Api>(params: {
ensureCustomApiRegistered(params.model.api, streamFn);
return streamFn;
}
function wrapPluginProviderStream(streamFn: StreamFn): StreamFn {
const boundary = "plugin provider stream handoff";
return (model, context, options) => {
const apiKey = options?.apiKey
? unwrapSecretSentinelsForProviderEgress(options.apiKey, boundary)
: options?.apiKey;
const headers = options?.headers
? unwrapHeaderSentinelsForProviderEgress(options.headers, boundary)
: options?.headers;
const resolvedOptions =
apiKey === options?.apiKey && headers === options?.headers
? options
: { ...options, apiKey, headers };
return streamFn(
unwrapModelHeaderSentinelsForProviderEgress(model, boundary),
context,
resolvedOptions,
);
};
}
@@ -0,0 +1,56 @@
import { createServer } from "node:http";
import type { AddressInfo } from "node:net";
import type { Model } from "openclaw/plugin-sdk/llm";
import { afterEach, describe, expect, it } from "vitest";
import { redactSensitiveText } from "../logging/redact.js";
import { resetSecretRedactionRegistryForTest } from "../logging/secret-redaction-registry.js";
import { mintSecretSentinel } from "../secrets/sentinel.js";
import { buildGuardedModelFetch } from "./provider-transport-fetch.js";
describe("guarded model fetch secret sentinel integration", () => {
afterEach(() => {
resetSecretRedactionRegistryForTest();
});
it("injects the real header only at local HTTP egress and redacts the resolved value", async () => {
let receivedAuthorization: string | undefined;
const server = createServer((request, response) => {
receivedAuthorization = request.headers.authorization;
response.writeHead(200, { "content-type": "application/json" });
response.end('{"ok":true}');
});
await new Promise<void>((resolve, reject) => {
server.once("error", reject);
server.listen(0, "127.0.0.1", resolve);
});
try {
const port = (server.address() as AddressInfo).port;
const baseUrl = `http://127.0.0.1:${port}/v1`;
const model = {
id: "integration-model",
provider: "sentinel-integration",
api: "openai-responses",
baseUrl,
} as unknown as Model<"openai-responses">;
const secret = "integration-provider-secret";
const sentinel = mintSecretSentinel(secret, { label: "model-auth:integration" });
const response = await buildGuardedModelFetch(model)(`${baseUrl}/responses`, {
method: "POST",
headers: { Authorization: `Bearer ${sentinel}` },
body: "{}",
});
await response.text();
expect(receivedAuthorization).toBe(`Bearer ${secret}`);
expect(redactSensitiveText(`upstream used ${secret}`, { mode: "off" })).toBe(
"upstream used integr…cret",
);
} finally {
await new Promise<void>((resolve) => {
server.close(() => resolve());
});
}
});
});
+146
View File
@@ -3,6 +3,7 @@ import { MAX_TIMER_TIMEOUT_MS } from "@openclaw/normalization-core/number-coerci
import { Stream } from "openai/streaming";
import type { Model } from "openclaw/plugin-sdk/llm";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { mintSecretSentinel } from "../secrets/sentinel.js";
import { buildGuardedModelFetch } from "./provider-transport-fetch.js";
type ProviderRequestPolicyConfigMockResult = {
@@ -177,6 +178,110 @@ describe("buildGuardedModelFetch", () => {
delete process.env.OPENCLAW_SDK_RETRY_MAX_WAIT_SECONDS;
});
function sentinelModel(): Model<"openai-responses"> {
return {
id: "gpt-5.5",
provider: "openai",
api: "openai-responses",
baseUrl: "https://api.openai.com/v1",
} as unknown as Model<"openai-responses">;
}
it("swaps sentinels in Request-form headers", async () => {
const sentinel = mintSecretSentinel("request-form-secret", { label: "request-form" });
const request = new Request("https://api.openai.com/v1/responses", {
method: "POST",
headers: { Authorization: `Bearer ${sentinel}` },
});
const response = await buildGuardedModelFetch(sentinelModel())(request);
await response.text();
const headers = new Headers((latestGuardedFetchParams().init as RequestInit).headers);
expect(headers.get("authorization")).toBe("Bearer request-form-secret");
});
it("swaps sentinels in record init headers", async () => {
const recordSentinel = mintSecretSentinel("record-header-secret", { label: "record-header" });
const response = await buildGuardedModelFetch(sentinelModel())(
"https://api.openai.com/v1/responses",
{
headers: { "x-api-key": recordSentinel },
},
);
await response.text();
expect(
new Headers((latestGuardedFetchParams().init as RequestInit).headers).get("x-api-key"),
).toBe("record-header-secret");
expect(
new Headers(ensureModelProviderLocalServiceMock.mock.calls[0]?.[1] as HeadersInit).get(
"x-api-key",
),
).toBe(recordSentinel);
});
it("swaps sentinels in tuple init headers", async () => {
const tupleSentinel = mintSecretSentinel("tuple-header-secret", { label: "tuple-header" });
const response = await buildGuardedModelFetch(sentinelModel())(
"https://api.openai.com/v1/responses",
{
headers: [["x-api-key", tupleSentinel]],
},
);
await response.text();
expect(
new Headers((latestGuardedFetchParams().init as RequestInit).headers).get("x-api-key"),
).toBe("tuple-header-secret");
});
it("swaps sentinels in Headers init and composed Cloudflare auth values", async () => {
const sentinel = mintSecretSentinel("cloudflare-upstream-secret", { label: "cloudflare" });
const response = await buildGuardedModelFetch(sentinelModel())(
"https://api.openai.com/v1/responses",
{
headers: new Headers({ "cf-aig-authorization": `Bearer ${sentinel}` }),
},
);
await response.text();
const headers = new Headers((latestGuardedFetchParams().init as RequestInit).headers);
expect(headers.get("cf-aig-authorization")).toBe("Bearer cloudflare-upstream-secret");
});
it("swaps sentinels in URL query parameters", async () => {
const sentinel = mintSecretSentinel("gemini&scope=two+#%", { label: "gemini-query" });
const response = await buildGuardedModelFetch(sentinelModel())(
`https://api.openai.com/v1/responses?key=${sentinel}`,
);
await response.text();
expect(latestGuardedFetchParams().url).toBe(
"https://api.openai.com/v1/responses?key=gemini%26scope%3Dtwo%2B%23%25",
);
});
it("rejects unknown sentinel-shaped values before guarded fetch", async () => {
const unknown = "oc-sent-v1-fedcba987654321001234567";
await expect(
buildGuardedModelFetch(sentinelModel())("https://api.openai.com/v1/responses", {
headers: { Authorization: `Bearer ${unknown}` },
}),
).rejects.toThrow(
`Secret sentinel ${unknown} is not registered in this process; refusing to send request`,
);
expect(fetchWithSsrFGuardMock).not.toHaveBeenCalled();
});
it("keeps the no-sentinel fast path request init untouched", async () => {
const init: RequestInit = { headers: { Authorization: "Bearer plain-env-key" } };
const response = await buildGuardedModelFetch(sentinelModel())(
"https://api.openai.com/v1/responses",
init,
);
await response.text();
expect(latestGuardedFetchParams().init).toStrictEqual(init);
});
it("pushes provider capture metadata into the shared guarded fetch seam", async () => {
const model = {
id: "gpt-5.4",
@@ -1592,6 +1697,47 @@ describe("buildGuardedModelFetch", () => {
expect(text.length).toBeLessThan(OVER_LIMIT);
});
it("returns a capped body before guarded cleanup finishes", async () => {
const OVER_LIMIT = 100 * 1024;
let finishRelease!: () => void;
const releasePending = new Promise<void>((resolve) => {
finishRelease = resolve;
});
const release = vi.fn(() => releasePending);
fetchWithSsrFGuardMock.mockResolvedValue({
response: new Response(new Uint8Array(OVER_LIMIT), {
status: 429,
statusText: "Too Many Requests",
}),
finalUrl: "https://custom-azure.openai.azure.com/openai/v1/responses",
release,
});
const model = {
id: "gpt-5.5",
provider: "azure",
api: "azure-openai-responses",
baseUrl: "https://custom-azure.openai.azure.com/openai/v1",
} as unknown as Model<"azure-openai-responses">;
const response = await buildGuardedModelFetch(model)(
"https://custom-azure.openai.azure.com/openai/v1/responses",
{ method: "POST" },
);
const timeout = Symbol("timeout");
let timeoutHandle: ReturnType<typeof setTimeout> | undefined;
const result = await Promise.race([
response.text(),
new Promise<typeof timeout>((resolve) => {
timeoutHandle = setTimeout(() => resolve(timeout), 100);
}),
]);
clearTimeout(timeoutHandle);
finishRelease();
expect(result).not.toBe(timeout);
expect(release).toHaveBeenCalledTimes(1);
});
it("preserves SDK ability to cancel retryable non-OK responses before reading body", async () => {
// Regression: a non-OK body wrapper must be lazy. The OpenAI SDK may decide
// to cancel a 429/5xx response and retry before reading the body. If the
+109 -21
View File
@@ -29,6 +29,11 @@ import {
import type { Model } from "../llm/types.js";
import { createSubsystemLogger } from "../logging/subsystem.js";
import { resolveDebugProxySettings } from "../proxy-capture/env.js";
import {
resolveSecretSentinel,
SECRET_SENTINEL_PATTERN,
swapSecretSentinelsInText,
} from "../secrets/sentinel.js";
import { emitModelTransportDebug } from "./model-transport-debug.js";
import { formatModelTransportDebugUrl } from "./model-transport-url.js";
import { ProviderHttpError, readResponseTextLimited } from "./provider-http-errors.js";
@@ -103,25 +108,42 @@ function capNonOkResponseBodyLazily(response: Response, maxBytes: number): Respo
if (!source) {
return response;
}
let reader: ReadableStreamDefaultReader<Uint8Array> | undefined;
let total = 0;
const capped = source.pipeThrough(
new TransformStream<Uint8Array, Uint8Array>({
transform(chunk, controller) {
const nextTotal = total + chunk.byteLength;
if (nextTotal > maxBytes) {
const remaining = maxBytes - total;
if (remaining > 0) {
controller.enqueue(chunk.subarray(0, remaining));
}
total = maxBytes;
controller.terminate();
// Own the reader: Node can leak an internal pipeThrough writer rejection when
// downstream cancellation races the cap terminating the transform.
const capped = new ReadableStream<Uint8Array>({
start() {
reader = source.getReader();
},
async pull(controller) {
try {
const chunk = await reader?.read();
if (!chunk || chunk.done) {
controller.close();
return;
}
total = nextTotal;
controller.enqueue(chunk);
},
}),
);
const remaining = maxBytes - total;
if (chunk.value.byteLength > remaining) {
if (remaining > 0) {
controller.enqueue(chunk.value.subarray(0, remaining));
}
total = maxBytes;
controller.close();
void reader?.cancel().catch(() => undefined);
return;
}
total += chunk.value.byteLength;
controller.enqueue(chunk.value);
} catch (error) {
controller.error(error);
void reader?.cancel(error).catch(() => undefined);
}
},
async cancel(reason) {
await reader?.cancel(reason).catch(() => undefined);
},
});
return new Response(capped, response);
}
@@ -743,6 +765,63 @@ export function resolveProviderTransportSsrFPolicy(params: {
);
}
function headersContainSecretSentinel(headers: HeadersInit | undefined): boolean {
if (!headers) {
return false;
}
for (const value of new Headers(headers).values()) {
if (value.includes("oc-sent-v1-")) {
return true;
}
}
return false;
}
function swapSecretSentinelsInUrl(url: string): { text: string; unknown: string[] } {
if (!url.includes("oc-sent-v1-")) {
return { text: url, unknown: [] };
}
const unknown = new Set<string>();
const text = url.replace(new RegExp(SECRET_SENTINEL_PATTERN.source, "g"), (sentinel) => {
const value = resolveSecretSentinel(sentinel);
if (value === undefined) {
unknown.add(sentinel);
return sentinel;
}
// Sentinels are URL-safe placeholders. Encode the real bytes so query/path structure is stable.
return encodeURIComponent(value);
});
return { text, unknown: [...unknown] };
}
function swapSecretSentinelsForEgress(params: { url: string; headers?: HeadersInit }): {
url: string;
headers?: Headers;
} {
if (!params.url.includes("oc-sent-v1-") && !headersContainSecretSentinel(params.headers)) {
return { url: params.url };
}
const urlSwap = swapSecretSentinelsInUrl(params.url);
const headers = params.headers ? new Headers(params.headers) : undefined;
const unknown = new Set(urlSwap.unknown);
if (headers) {
for (const [name, value] of headers.entries()) {
const swapped = swapSecretSentinelsInText(value);
headers.set(name, swapped.text);
for (const sentinel of swapped.unknown) {
unknown.add(sentinel);
}
}
}
const unresolved = unknown.values().next().value;
if (unresolved) {
throw new Error(
`Secret sentinel ${unresolved} is not registered in this process; refusing to send request`,
);
}
return { url: urlSwap.text, ...(headers ? { headers } : {}) };
}
export function buildGuardedModelFetch(
model: Model,
timeoutMs?: number,
@@ -772,7 +851,7 @@ export function buildGuardedModelFetch(
return async (input, init) => {
let localServiceLease: ProviderLocalServiceLease | undefined;
const request = input instanceof Request ? new Request(input, init) : undefined;
const url =
const rawUrl =
request?.url ??
(input instanceof URL
? input.toString()
@@ -781,6 +860,12 @@ export function buildGuardedModelFetch(
: (() => {
throw new Error("Unsupported fetch input for transport-aware model request");
})());
const rawHeaders = request?.headers ?? init?.headers;
const swappedEgress = swapSecretSentinelsForEgress({
url: rawUrl,
headers: rawHeaders,
});
const url = swappedEgress.url;
const policy = resolveProviderTransportSsrFPolicy({
baseUrl: model.baseUrl,
url,
@@ -796,13 +881,15 @@ export function buildGuardedModelFetch(
request &&
({
method: request.method,
headers: request.headers,
headers: swappedEgress.headers ?? request.headers,
body: request.body ?? undefined,
redirect: request.redirect,
signal: request.signal,
...(request.body ? ({ duplex: "half" } as const) : {}),
} satisfies RequestInit & { duplex?: "half" });
const baseInit = requestInit ?? init;
const baseInit =
requestInit ??
(swappedEgress.headers && init ? { ...init, headers: swappedEgress.headers } : init);
const synthesizeJsonAsSse = await requestBodyHasStreamTrue(request, baseInit);
const baseSignal = baseInit?.signal ?? undefined;
const localServiceSignal = buildModelRequestSignal(baseSignal, requestTimeoutMs);
@@ -830,14 +917,15 @@ export function buildGuardedModelFetch(
emitModelTransportDebug(
log,
`[model-fetch] start provider=${model.provider} api=${model.api} model=${model.id} ` +
`method=${baseInit?.method ?? "GET"} url=${formatModelTransportDebugUrl(url)} timeoutMs=${requestTimeoutMs} ` +
// Log the pre-swap URL: the swapped URL can carry an injected credential in its path.
`method=${baseInit?.method ?? "GET"} url=${formatModelTransportDebugUrl(rawUrl)} timeoutMs=${requestTimeoutMs} ` +
`proxy=${dispatcherPolicy ? "configured" : useEnvProxy ? "env" : "none"} ` +
`policy=${policy ? "custom" : "default"}`,
);
try {
localServiceLease = await ensureModelProviderLocalService(
model,
baseInit?.headers,
rawHeaders,
localServiceSignal,
);
result = await fetchWithSsrFGuard(
@@ -3,6 +3,11 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import type { OpenClawConfig } from "../config/types.openclaw.js";
import type { Model } from "../llm/types.js";
import {
looksLikeSecretSentinel,
mintSecretSentinel,
resolveSecretSentinel,
} from "../secrets/sentinel.js";
// Hoisted mocks keep Vitest module replacement stable while the implementation
// under test imports auth, model resolution, and transport helpers at module load.
@@ -32,6 +37,7 @@ vi.mock("./simple-completion-transport.js", () => ({
}));
vi.mock("./model-auth.js", () => ({
applySecretRefHeaderSentinels: (model: unknown) => model,
formatMissingAuthError: vi.fn(
(auth: { source: string; mode: string }, provider: string) =>
`No API key resolved for provider "${provider}" (auth mode: ${auth.mode}, checked: ${auth.source}).`,
@@ -281,6 +287,34 @@ describe("prepareSimpleCompletionModel", () => {
expect(result.auth.apiKey).not.toBe("ghu_original_github_token");
});
it("keeps an exchanged Copilot token opaque when its source is a sentinel", async () => {
const sourceSecret = "github-source-secret";
const sourceSentinel = mintSecretSentinel(sourceSecret, {
label: "model-auth:github-copilot",
});
hoisted.resolveModelMock.mockReturnValueOnce({
model: { provider: "github-copilot", id: "gpt-4.1" },
authStorage: { setRuntimeApiKey: hoisted.setRuntimeApiKeyMock },
modelRegistry: {},
});
hoisted.getApiKeyForModelMock.mockResolvedValueOnce({
apiKey: sourceSentinel,
source: "profile:github-copilot:default",
mode: "token",
});
const result = await prepareSimpleCompletionModel({
cfg: undefined,
provider: "github-copilot",
modelId: "gpt-4.1",
});
expect(hoisted.resolveCopilotApiTokenMock).toHaveBeenCalledWith({ githubToken: sourceSecret });
expectPreparedModelResult(result);
expect(looksLikeSecretSentinel(result.auth.apiKey ?? "")).toBe(true);
expect(resolveSecretSentinel(result.auth.apiKey ?? "")).toBe("copilot-runtime-token");
});
it("applies exchanged copilot baseUrl to returned model", async () => {
hoisted.resolveModelMock.mockReturnValueOnce({
model: {
+44 -16
View File
@@ -25,6 +25,7 @@ import { DEFAULT_PROVIDER } from "./defaults.js";
import { resolveModel, resolveModelAsync } from "./embedded-agent-runner/model.js";
import { resolveAgentHarnessPolicy } from "./harness/policy.js";
import {
applySecretRefHeaderSentinels,
applyLocalNoAuthHeaderOverride,
formatMissingAuthError,
getApiKeyForModel,
@@ -38,6 +39,10 @@ import {
} from "./model-selection.js";
import { OPENAI_PROVIDER_ID, isOpenAIProvider } from "./openai-routing.js";
import { applyPreparedRuntimeAuthToModel } from "./provider-request-config.js";
import {
protectPreparedProviderRuntimeAuth,
unwrapSecretSentinelsForProviderEgress,
} from "./provider-secret-egress.js";
import { prepareModelForSimpleCompletion } from "./simple-completion-transport.js";
type SimpleCompletionAuthStorage = {
@@ -168,30 +173,49 @@ async function setRuntimeApiKeyForCompletion(params: {
if (params.model.provider === "github-copilot") {
const { resolveCopilotApiToken } = await import("../plugin-sdk/provider-auth.js");
const copilotToken = await resolveCopilotApiToken({
githubToken: params.apiKey,
githubToken: unwrapSecretSentinelsForProviderEgress(
params.apiKey,
"GitHub Copilot runtime auth exchange",
),
});
params.authStorage.setRuntimeApiKey(params.model.provider, copilotToken.token);
const protectedAuth = protectPreparedProviderRuntimeAuth({
sourceApiKey: params.apiKey,
provider: params.model.provider,
preparedAuth: {
apiKey: copilotToken.token,
baseUrl: copilotToken.baseUrl,
},
});
const runtimeApiKey = protectedAuth?.apiKey ?? copilotToken.token;
params.authStorage.setRuntimeApiKey(params.model.provider, runtimeApiKey);
return {
apiKey: copilotToken.token,
apiKey: runtimeApiKey,
model: { ...params.model, baseUrl: copilotToken.baseUrl },
};
}
const preparedAuth = await prepareProviderRuntimeAuth({
const preparedAuth = protectPreparedProviderRuntimeAuth({
sourceApiKey: params.apiKey,
provider: params.model.provider,
config: params.cfg,
workspaceDir: params.workspaceDir,
env: process.env,
context: {
preparedAuth: await prepareProviderRuntimeAuth({
provider: params.model.provider,
config: params.cfg,
workspaceDir: params.workspaceDir,
env: process.env,
provider: params.model.provider,
modelId: params.model.id,
model: params.model,
apiKey: params.apiKey,
authMode: params.authMode,
profileId: params.profileId,
},
context: {
config: params.cfg,
workspaceDir: params.workspaceDir,
env: process.env,
provider: params.model.provider,
modelId: params.model.id,
model: params.model,
apiKey: unwrapSecretSentinelsForProviderEgress(
params.apiKey,
"provider runtime auth exchange",
),
authMode: params.authMode,
profileId: params.profileId,
},
}),
});
const runtimeApiKey = preparedAuth?.apiKey?.trim() || params.apiKey;
params.authStorage.setRuntimeApiKey(params.model.provider, runtimeApiKey);
@@ -255,6 +279,7 @@ export async function prepareSimpleCompletionModel(params: {
agentDir: params.agentDir,
profileId: params.profileId,
preferredProfile: params.preferredProfile,
secretSentinels: true,
});
} catch (err) {
return {
@@ -297,7 +322,10 @@ export async function prepareSimpleCompletionModel(params: {
};
return {
model: applyLocalNoAuthHeaderOverride(resolvedModel, resolvedAuth),
model: applySecretRefHeaderSentinels(
applyLocalNoAuthHeaderOverride(resolvedModel, resolvedAuth),
params.cfg,
),
auth: resolvedAuth,
};
}
+24 -4
View File
@@ -5,6 +5,8 @@ import type { Model } from "openclaw/plugin-sdk/llm";
import { afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
import type { OpenClawConfig } from "../config/config.js";
import { createMoonshotThinkingWrapper } from "../llm/providers/stream-wrappers/moonshot-thinking.js";
import { mintSecretSentinel } from "../secrets/sentinel.js";
import type { StreamFn } from "./runtime/index.js";
const createAnthropicVertexStreamFnForModel = vi.fn();
const ensureCustomApiRegistered = vi.fn();
@@ -16,6 +18,7 @@ const createTransportAwareStreamFnForModel = vi.fn();
const prepareTransportAwareSimpleModel = vi.fn();
const resolveTransportAwareSimpleApi = vi.fn();
const prepareGoogleSimpleCompletionModel = vi.fn((model: unknown) => model);
const pluginStreamFn = vi.fn(() => "plugin-stream-result" as never);
vi.mock("./anthropic-vertex-stream.js", () => ({
createAnthropicVertexStreamFnForModel,
@@ -62,6 +65,7 @@ describe("prepareModelForSimpleCompletion", () => {
createAnthropicVertexStreamFnForModel.mockReset();
ensureCustomApiRegistered.mockReset();
resolveProviderStreamFn.mockReset();
pluginStreamFn.mockClear();
wrapProviderSimpleCompletionStreamFn.mockReset();
buildTransportAwareSimpleStreamFn.mockReset();
createOpenClawTransportStreamFnForModel.mockReset();
@@ -70,7 +74,7 @@ describe("prepareModelForSimpleCompletion", () => {
resolveTransportAwareSimpleApi.mockReset();
prepareGoogleSimpleCompletionModel.mockReset();
createAnthropicVertexStreamFnForModel.mockReturnValue("vertex-stream");
resolveProviderStreamFn.mockReturnValue("ollama-stream");
resolveProviderStreamFn.mockReturnValue(pluginStreamFn);
wrapProviderSimpleCompletionStreamFn.mockReturnValue(undefined);
buildTransportAwareSimpleStreamFn.mockReturnValue(undefined);
createOpenClawTransportStreamFnForModel.mockReturnValue(undefined);
@@ -142,6 +146,8 @@ describe("prepareModelForSimpleCompletion", () => {
});
it("registers the configured Ollama transport and keeps the original api", () => {
const secret = "ollama-provider-secret";
const sentinel = mintSecretSentinel(secret, { label: "model-auth:ollama" });
const model: Model<"ollama"> = {
id: "llama3",
name: "Llama 3",
@@ -153,7 +159,7 @@ describe("prepareModelForSimpleCompletion", () => {
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 8192,
maxTokens: 4096,
headers: {},
headers: { Authorization: `Bearer ${sentinel}` },
};
const cfg: OpenClawConfig = {
models: {
@@ -183,8 +189,22 @@ describe("prepareModelForSimpleCompletion", () => {
expect(request.config).toBe(cfg);
expect(request.context?.provider).toBe("ollama");
expect(request.context?.modelId).toBe("llama3");
expect(request.context?.model).toBe(model);
expect(ensureCustomApiRegistered).toHaveBeenCalledWith("ollama", "ollama-stream");
expect(request.context?.model).toEqual({
...model,
headers: { Authorization: `Bearer ${secret}` },
});
expect(ensureCustomApiRegistered).toHaveBeenCalledWith("ollama", expect.any(Function));
const registeredStream = ensureCustomApiRegistered.mock.calls[0]?.[1] as StreamFn;
void registeredStream(
{ ...model, headers: { Authorization: `Bearer ${sentinel}` } } as never,
{} as never,
{ apiKey: sentinel, headers: { "X-Managed": `Bearer ${sentinel}` } } as never,
);
expect(pluginStreamFn).toHaveBeenCalledWith(
expect.objectContaining({ headers: { Authorization: `Bearer ${secret}` } }),
{},
{ apiKey: secret, headers: { "X-Managed": `Bearer ${secret}` } },
);
expect(result).toBe(model);
});
+1
View File
@@ -672,6 +672,7 @@ export async function resolveModelRuntimeApiKey(params: {
model: params.model,
cfg: params.cfg,
agentDir: params.agentDir,
secretSentinels: true,
});
// Bedrock's runtime client owns AWS credential-chain resolution. Keep the
// empty sentinel out of auth storage and pass it through to the stream.
+4 -1
View File
@@ -119,7 +119,10 @@ const musicGenerateBackgroundMocks = vi.hoisted(() => ({
}),
}));
vi.mock("../../config/config.js", () => configMocks);
vi.mock("../../config/config.js", async (importOriginal) => ({
...(await importOriginal<typeof import("../../config/config.js")>()),
...configMocks,
}));
vi.mock("../../media/store.js", () => mediaStoreMocks);
vi.mock("../../media/web-media.js", async () => {
const actual = await vi.importActual<typeof import("../../media/web-media.js")>(
@@ -1,6 +1,7 @@
// Native PDF provider tests cover direct Anthropic and Gemini request shapes,
// base URL handling, and bounded API error reporting.
import { afterEach, describe, expect, it, vi } from "vitest";
import { mintSecretSentinel } from "../../secrets/sentinel.js";
import * as pdfNativeProviders from "./pdf-native-providers.js";
vi.mock("../../plugins/provider-runtime.js", () => ({
@@ -112,6 +113,29 @@ describe("native PDF provider API calls", () => {
expect(body.messages[0].content[1].type).toBe("text");
});
it("unwraps sentinel-backed native PDF headers only at the request handoff", async () => {
const apiKey = mintSecretSentinel("native-pdf-api-secret", {
label: "model-auth:anthropic",
});
const managedHeader = mintSecretSentinel("native-pdf-managed-secret", {
label: "model-auth:anthropic",
});
const fetchMock = mockFetchResponse(
jsonResponse({ content: [{ type: "text", text: "Analysis" }] }),
);
await pdfNativeProviders.anthropicAnalyzePdf(
makeAnthropicAnalyzeParams({
apiKey,
requestConfig: { headers: { "X-Managed": `Bearer ${managedHeader}` } },
}),
);
const [, opts] = firstFetchCall(fetchMock) as [string, { headers: Headers }];
expect(opts.headers.get("x-api-key")).toBe("native-pdf-api-secret");
expect(opts.headers.get("X-Managed")).toBe("Bearer native-pdf-managed-secret");
});
it("anthropicAnalyzePdf honors ANTHROPIC_BASE_URL when no base URL is configured", async () => {
vi.stubEnv("ANTHROPIC_BASE_URL", "https://anthropic-pdf-proxy.example/v1");
const fetchMock = mockFetchResponse(
+9 -1
View File
@@ -14,6 +14,7 @@ import { isRecord } from "../../utils.js";
import { normalizeSecretInput } from "../../utils/normalize-secret-input.js";
import { resolveAnthropicMessagesUrl } from "../anthropic-transport-stream.js";
import type { ModelProviderRequestTransportOverrides } from "../provider-request-config.js";
import { unwrapSecretSentinelsForProviderEgress } from "../provider-secret-egress.js";
import { resolveProviderTransportSsrFPolicy } from "../provider-transport-fetch.js";
type PdfInput = {
@@ -43,9 +44,16 @@ type NativePdfJsonRequest = {
};
async function postNativePdfJson(params: NativePdfJsonRequest): Promise<Record<string, unknown>> {
const headers = new Headers(params.headers);
for (const [name, value] of headers.entries()) {
headers.set(
name,
unwrapSecretSentinelsForProviderEgress(value, `${params.failureLabel} header handoff`),
);
}
const { response, release } = await postJsonRequest({
url: params.url,
headers: params.headers,
headers,
body: params.body,
timeoutMs: NATIVE_PDF_PROVIDER_FETCH_TIMEOUT_MS,
fetchFn: fetch,
+3
View File
@@ -377,6 +377,9 @@ describe("createPdfTool", () => {
const [, loadOptions] = firstMockCall(loadSpy, "loadWebMediaRaw");
expectFields(loadOptions, { maxBytes: 524_288 });
expect(modelAuth.getApiKeyForModel).toHaveBeenCalledWith(
expect.objectContaining({ secretSentinels: true }),
);
});
});
+5 -1
View File
@@ -19,6 +19,7 @@ import { extractPdfContent, type PdfExtractedContent } from "../../media/pdf-ext
import { loadWebMediaRaw } from "../../media/web-media.js";
import { resolveUserPath } from "../../utils.js";
import type { AuthProfileStore } from "../auth-profiles/types.js";
import { applySecretRefHeaderSentinels } from "../model-auth.js";
import { getModelProviderRequestTransport } from "../provider-request-config.js";
import { registerProviderStreamForModel } from "../provider-stream.js";
import { optionalFiniteNumberSchema } from "../schema/typebox.js";
@@ -177,7 +178,10 @@ async function runPdfPrompt(params: {
cfg: effectiveCfg,
modelOverride: params.modelOverride,
run: async (provider, modelId) => {
const model = resolveModelFromRegistry({ modelRegistry, provider, modelId });
const model = applySecretRefHeaderSentinels(
resolveModelFromRegistry({ modelRegistry, provider, modelId }),
effectiveCfg,
);
const apiKey = await resolveModelRuntimeApiKey({
model,
cfg: effectiveCfg,
+11
View File
@@ -6,6 +6,7 @@ import { resolveOpenAIStrictToolSetting } from "../agents/openai-strict-tool-set
import { buildGuardedModelFetch } from "../agents/provider-transport-fetch.js";
import { redactSecrets, redactToolPayloadText } from "../logging/redact.js";
import { createSubsystemLogger } from "../logging/subsystem.js";
import { swapSecretSentinelsInText } from "../secrets/sentinel.js";
const transportLogBySubsystem = new Map<string, ReturnType<typeof createSubsystemLogger>>();
@@ -20,6 +21,16 @@ function transportLog(subsystem: string): ReturnType<typeof createSubsystemLogge
configureAiTransportHost({
buildModelFetch: buildGuardedModelFetch,
resolveSecretSentinel: (value) => {
const swapped = swapSecretSentinelsInText(value);
const unknown = swapped.unknown[0];
if (unknown) {
throw new Error(
`Secret sentinel ${unknown} is not registered in this process; refusing to construct provider client`,
);
}
return swapped.text;
},
redactSecrets,
redactToolPayloadText,
resolveOpenAIStrictToolSetting,
+69
View File
@@ -4,6 +4,7 @@ import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import { withEnv } from "../test-utils/env.js";
import { withFullContextToolPayloadRedaction } from "./redact-internal.js";
import {
getDefaultRedactPatterns,
redactSecrets,
@@ -11,8 +12,13 @@ import {
redactSensitiveLines,
redactSensitiveText,
redactToolDetail,
redactToolPayloadTextWithConfig,
resolveRedactOptions,
} from "./redact.js";
import {
registerSecretValueForRedaction,
resetSecretRedactionRegistryForTest,
} from "./secret-redaction-registry.js";
const defaults = getDefaultRedactPatterns();
let tempDirs: string[] = [];
@@ -26,12 +32,75 @@ function writeConfig(source: string): string {
}
afterEach(() => {
resetSecretRedactionRegistryForTest();
for (const dir of tempDirs) {
fs.rmSync(dir, { force: true, recursive: true });
}
tempDirs = [];
});
describe("registered exact secret values", () => {
it("masks registered values in text and nested structured data", () => {
const secret = "registered-exact-secret";
registerSecretValueForRedaction(secret);
expect(redactSensitiveText(`before ${secret} after`, { mode: "off" })).toBe(
"before regist…cret after",
);
expect(redactSecrets({ detail: `before ${secret} after` })).toEqual({
detail: "before regist…cret after",
});
expect(
redactToolPayloadTextWithConfig(
`full context ${secret}`,
withFullContextToolPayloadRedaction(undefined),
),
).toBe("full context regist…cret");
});
it("ignores values shorter than six characters", () => {
registerSecretValueForRedaction("abcde");
expect(redactSensitiveText("value abcde", { mode: "off" })).toBe("value abcde");
expect(redactSecrets({ detail: "abcde" })).toEqual({ detail: "abcde" });
});
it("masks the percent-encoded form of registered values", () => {
const secret = "path/token with+reserved%chars";
registerSecretValueForRedaction(secret);
const encoded = encodeURIComponent(secret);
expect(redactSensitiveText(`url path ${encoded}`, { mode: "off" })).not.toContain(encoded);
expect(redactSensitiveText(`raw ${secret}`, { mode: "off" })).not.toContain(secret);
});
it("evicts the oldest value after 512 registrations", () => {
const first = "exact-registry-value-000";
registerSecretValueForRedaction(first);
for (let index = 1; index <= 512; index += 1) {
registerSecretValueForRedaction(`exact-registry-value-${index.toString().padStart(3, "0")}`);
}
const last = "exact-registry-value-512";
expect(redactSensitiveText(first, { mode: "off" })).toBe(first);
expect(redactSensitiveText(last, { mode: "off" })).toBe("exact-…-512");
});
it("refreshes duplicate registration recency before eviction", () => {
const first = "exact-registry-refresh-000";
const second = "exact-registry-refresh-001";
for (let index = 0; index < 512; index += 1) {
registerSecretValueForRedaction(
`exact-registry-refresh-${index.toString().padStart(3, "0")}`,
);
}
registerSecretValueForRedaction(first);
registerSecretValueForRedaction("exact-registry-refresh-512");
expect(redactSensitiveText(first, { mode: "off" })).not.toContain(first);
expect(redactSensitiveText(second, { mode: "off" })).toBe(second);
});
});
describe("redactSensitiveText", () => {
it("masks env assignments while keeping the key", () => {
const input = "OPENAI_API_KEY=sk-1234567890abcdef";
+20 -14
View File
@@ -4,6 +4,7 @@ import { compileConfigRegex } from "../security/config-regex.js";
import { readLoggingConfig } from "./config.js";
import { replacePatternBounded } from "./redact-bounded.js";
import { isFullContextToolPayloadRedaction } from "./redact-internal.js";
import { redactRegisteredSecretValues } from "./secret-redaction-registry.js";
export type RedactSensitiveMode = "off" | "tools";
export type RedactPattern = string | RegExp;
@@ -987,18 +988,21 @@ export function redactSensitiveText(text: string, options?: RedactOptions): stri
if (!text) {
return text;
}
const exactRedacted = redactRegisteredSecretValues(text, maskToken);
const resolvedOptions = options ?? resolveConfigRedaction();
if (normalizeMode(resolvedOptions.mode) === "off") {
return text;
return exactRedacted;
}
if (!resolvedOptions.patterns?.length && !couldMatchDefaultRedactPatterns(text)) {
return text;
if (!resolvedOptions.patterns?.length && !couldMatchDefaultRedactPatterns(exactRedacted)) {
return exactRedacted;
}
const resolved = resolveRedactOptions(resolvedOptions);
if (!resolved.patterns.length) {
return text;
return exactRedacted;
}
return redactText(text, resolved.patterns, { redactFormBodies: resolved.redactFormBodies });
return redactText(exactRedacted, resolved.patterns, {
redactFormBodies: resolved.redactFormBodies,
});
}
export function redactToolDetail(detail: string): string {
@@ -1030,9 +1034,10 @@ export function redactToolPayloadTextWithConfig(
if (!text) {
return text;
}
const exactRedacted = redactRegisteredSecretValues(text, maskToken);
if (isFullContextToolPayloadRedaction(loggingConfig)) {
const resolved = resolveRedactOptions(resolveToolPayloadRedaction(loggingConfig));
return redactText(text, resolved.patterns, {
return redactText(exactRedacted, resolved.patterns, {
fullContext: true,
redactFormBodies: resolved.redactFormBodies,
});
@@ -1050,11 +1055,12 @@ function redactSensitiveFieldValueWithOptions(
options: RedactOptions,
path: readonly string[] = [key],
): string {
const exactRedacted = redactRegisteredSecretValues(value, maskToken);
const resolved = resolveRedactOptions(options);
if (resolved.mode === "off") {
return value;
return exactRedacted;
}
const redacted = redactText(value, resolved.patterns, {
const redacted = redactText(exactRedacted, resolved.patterns, {
redactFormBodies: resolved.redactFormBodies,
});
const shouldRedactAppPassword = redacted !== value || STRUCTURED_APP_PASSWORD_FIELD_RE.test(key);
@@ -1073,17 +1079,17 @@ function redactSensitiveFieldValueWithOptions(
}
if (
normalizedStructuredKey === "session" &&
STRUCTURED_INTERNAL_SOURCE_PATH_VALUE_RE.test(value)
STRUCTURED_INTERNAL_SOURCE_PATH_VALUE_RE.test(exactRedacted)
) {
return value;
return exactRedacted;
}
if (isSensitiveFieldKey(key)) {
if (isShellReferenceToKey(key, value)) {
return value;
if (isShellReferenceToKey(key, exactRedacted)) {
return exactRedacted;
}
return maskToken(value);
return maskToken(exactRedacted);
}
return value;
return exactRedacted;
}
export function redactSensitiveFieldValue(
+82
View File
@@ -0,0 +1,82 @@
const MIN_SECRET_VALUE_LENGTH = 6;
const MAX_SECRET_VALUES = 512;
const registeredValues = new Map<string, true>();
let compiledMatcher: RegExp | undefined;
let firstChars = new Set<string>();
function escapeRegExp(value: string): string {
return value.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
}
function rebuildProbe(): void {
firstChars = new Set([...registeredValues.keys()].map((value) => value[0]));
compiledMatcher = undefined;
}
function registerOneSecretValue(value: string): void {
if (registeredValues.delete(value)) {
registeredValues.set(value, true);
return;
}
registeredValues.set(value, true);
if (registeredValues.size > MAX_SECRET_VALUES) {
const oldest = registeredValues.keys().next().value;
if (oldest !== undefined) {
registeredValues.delete(oldest);
}
}
rebuildProbe();
}
/** Registers one resolved secret for exact-value log redaction. */
export function registerSecretValueForRedaction(value: string): void {
if (value.length < MIN_SECRET_VALUE_LENGTH) {
return;
}
registerOneSecretValue(value);
// URL egress percent-encodes injected values; redact that surface form too.
const encoded = encodeURIComponent(value);
if (encoded !== value) {
registerOneSecretValue(encoded);
}
}
/** Returns whether a value has SecretRef provenance in the process registry. */
export function isSecretValueRegisteredForRedaction(value: string): boolean {
return registeredValues.has(value);
}
/** Replaces registered exact values while preserving the caller's mask convention. */
export function redactRegisteredSecretValues(
text: string,
mask: (value: string) => string,
): string {
if (!text || registeredValues.size === 0) {
return text;
}
let couldMatch = false;
for (const firstChar of firstChars) {
if (text.includes(firstChar)) {
couldMatch = true;
break;
}
}
if (!couldMatch) {
return text;
}
compiledMatcher ??= new RegExp(
[...registeredValues.keys()]
.toSorted((left, right) => right.length - left.length)
.map(escapeRegExp)
.join("|"),
"g",
);
return text.replace(compiledMatcher, (value) => mask(value));
}
/** Test-only reset for process-global redaction state. */
export function resetSecretRedactionRegistryForTest(): void {
registeredValues.clear();
rebuildProbe();
}
+97 -1
View File
@@ -3,6 +3,11 @@
import path from "node:path";
import { MAX_TIMER_TIMEOUT_MS } from "@openclaw/normalization-core/number-coercion";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import {
looksLikeSecretSentinel,
mintSecretSentinel,
resolveSecretSentinel,
} from "../secrets/sentinel.js";
const hoisted = vi.hoisted(() => ({
completeMock: vi.fn(),
@@ -26,6 +31,7 @@ const hoisted = vi.hoisted(() => ({
resolveModelAsyncMock: vi.fn(),
resolveModelWithRegistryMock: vi.fn(),
resolveCopilotApiTokenMock: vi.fn(),
unwrapSecretSentinelsForProviderEgressMock: vi.fn((value: string) => value),
}));
const {
completeMock,
@@ -41,6 +47,7 @@ const {
resolveModelAsyncMock,
resolveModelWithRegistryMock,
resolveCopilotApiTokenMock,
unwrapSecretSentinelsForProviderEgressMock,
} = hoisted;
type ResolveModelWithRegistryTestParams = {
@@ -98,6 +105,7 @@ vi.mock("../agents/models-config.js", async () => ({
}));
vi.mock("../agents/model-auth.js", () => ({
applySecretRefHeaderSentinels: (model: unknown) => model,
getApiKeyForModel: getApiKeyForModelMock,
resolveApiKeyForProvider: resolveApiKeyForProviderMock,
requireApiKey: requireApiKeyMock,
@@ -107,6 +115,13 @@ vi.mock("../agents/provider-stream.js", () => ({
registerProviderStreamForModel: registerProviderStreamForModelMock,
}));
vi.mock("../agents/provider-secret-egress.js", async () => ({
...(await vi.importActual<typeof import("../agents/provider-secret-egress.js")>(
"../agents/provider-secret-egress.js",
)),
unwrapSecretSentinelsForProviderEgress: unwrapSecretSentinelsForProviderEgressMock,
}));
vi.mock("../agents/agent-model-discovery.js", () => ({
discoverAuthStorage: () => ({
setRuntimeApiKey: setRuntimeApiKeyMock,
@@ -248,6 +263,36 @@ describe("describeImageWithModel", () => {
expect(completeMock).not.toHaveBeenCalled();
});
it("unwraps a sentinel only at the direct MiniMax VLM handoff", async () => {
getApiKeyForModelMock.mockResolvedValueOnce({
apiKey: "oc-sent-v1-0123456789abcdef01234567",
source: "test",
mode: "api-key",
});
unwrapSecretSentinelsForProviderEgressMock.mockReturnValueOnce("resolved-minimax-secret");
await describeImageWithModel({
cfg: {},
agentDir: "/tmp/openclaw-agent",
provider: "minimax-portal",
model: "MiniMax-VL-01",
buffer: Buffer.from("png-bytes"),
fileName: "image.png",
mime: "image/png",
timeoutMs: 1000,
});
expect(unwrapSecretSentinelsForProviderEgressMock).toHaveBeenCalledWith(
"oc-sent-v1-0123456789abcdef01234567",
"MiniMax VLM request",
);
const [, fetchOptionsValue] = requireFirstMockCall(fetchMock, "fetch");
const fetchOptions = requireRecord(fetchOptionsValue, "fetch options");
expect(fetchOptions.headers).toMatchObject({
Authorization: "Bearer resolved-minimax-secret",
});
});
it("uses generic completion for non-canonical minimax-portal image models", async () => {
discoverModelsMock.mockReturnValue({
find: vi.fn(() => ({
@@ -1386,7 +1431,7 @@ describe("describeImageWithModel", () => {
timestamp: Date.now(),
content: [{ type: "text", text: "A solid red square." }],
};
const providerStreamFn = vi.fn(() => ({
const providerStreamFn = vi.fn((_model: unknown, _context: unknown, _options: unknown) => ({
result: vi.fn(async () => providerStreamResult),
}));
registerProviderStreamForModelMock.mockReturnValueOnce(providerStreamFn);
@@ -1439,6 +1484,57 @@ describe("describeImageWithModel", () => {
expect(contentTypes).toContain("image");
});
it("keeps an exchanged Copilot image token opaque for sentinel-backed auth", async () => {
const sourceSecret = "copilot-image-source-secret";
const sourceSentinel = mintSecretSentinel(sourceSecret, {
label: "model-auth:github-copilot",
});
getApiKeyForModelMock.mockResolvedValueOnce({
apiKey: sourceSentinel,
source: "test",
mode: "token",
});
unwrapSecretSentinelsForProviderEgressMock.mockReturnValueOnce(sourceSecret);
const providerStreamFn = vi.fn((_model: unknown, _context: unknown, _options: unknown) => ({
result: vi.fn(async () => ({
role: "assistant",
api: "openai-completions",
provider: "github-copilot",
model: "gpt-4.1",
stopReason: "stop",
timestamp: Date.now(),
content: [{ type: "text", text: "ok" }],
})),
}));
registerProviderStreamForModelMock.mockReturnValueOnce(providerStreamFn);
discoverModelsMock.mockReturnValue({
find: vi.fn(() => ({
provider: "github-copilot",
id: "gpt-4.1",
input: ["text", "image"],
api: "openai-completions",
})),
});
await describeImageWithModel({
cfg: {},
agentDir: "/tmp/openclaw-agent",
provider: "github-copilot",
model: "gpt-4.1",
buffer: Buffer.from("png-bytes"),
fileName: "image.png",
mime: "image/png",
timeoutMs: 1000,
});
expect(resolveCopilotApiTokenMock).toHaveBeenCalledWith({ githubToken: sourceSecret });
const storedToken = setRuntimeApiKeyMock.mock.calls[0]?.[1] as string;
expect(looksLikeSecretSentinel(storedToken)).toBe(true);
expect(resolveSecretSentinel(storedToken)).toBe("copilot-api-token");
const streamOptions = providerStreamFn.mock.calls[0]?.[2] as { apiKey?: string };
expect(streamOptions.apiKey).toBe(storedToken);
});
it("fails github-copilot image runtime setup when token exchange fails", async () => {
discoverModelsMock.mockReturnValue({
find: vi.fn(() => ({
+23 -6
View File
@@ -5,6 +5,7 @@ import { isRecord } from "@openclaw/normalization-core/record-coerce";
import { resolveModelAsync } from "../agents/embedded-agent-runner/model.js";
import { isMinimaxVlmModel, minimaxUnderstandImage } from "../agents/minimax-vlm.js";
import {
applySecretRefHeaderSentinels,
getApiKeyForModel,
requireApiKey,
resolveApiKeyForProvider,
@@ -12,6 +13,10 @@ import {
import { normalizeModelRef } from "../agents/model-selection.js";
import { ensureOpenClawModelsJson } from "../agents/models-config.js";
import { resolveProviderRequestCapabilities } from "../agents/provider-attribution.js";
import {
protectPreparedProviderRuntimeAuth,
unwrapSecretSentinelsForProviderEgress,
} from "../agents/provider-secret-egress.js";
import { registerProviderStreamForModel } from "../agents/provider-stream.js";
import {
coerceImageAssistantText,
@@ -234,6 +239,7 @@ async function prepareResolvedImageRuntime(
profileId: params.profile,
preferredProfile: params.preferredProfile,
store: params.authStore,
secretSentinels: true,
});
// Bedrock's runtime client owns AWS credential-chain resolution. Keep the
// empty sentinel out of auth storage and pass it through to the stream.
@@ -242,7 +248,7 @@ async function prepareResolvedImageRuntime(
apiKeyInfo.mode === "aws-sdk" &&
model.api === "bedrock-converse-stream"
) {
return { apiKey: "", model };
return { apiKey: "", model: applySecretRefHeaderSentinels(model, params.cfg) };
}
let apiKey = requireApiKey(apiKeyInfo, model.provider);
// Image tool bypasses prepareRuntimeAuth — exchange OAuth token for
@@ -250,16 +256,24 @@ async function prepareResolvedImageRuntime(
// matches what runtime chat requests send.
if (model.provider === "github-copilot") {
const copilotToken = await resolveCopilotApiToken({
githubToken: apiKey,
githubToken: unwrapSecretSentinelsForProviderEgress(
apiKey,
"GitHub Copilot image-auth exchange",
),
});
apiKey = copilotToken.token;
const runtimeBaseUrl = copilotToken.baseUrl?.trim();
const protectedAuth = protectPreparedProviderRuntimeAuth({
sourceApiKey: apiKey,
provider: model.provider,
preparedAuth: { apiKey: copilotToken.token, baseUrl: copilotToken.baseUrl },
});
apiKey = protectedAuth?.apiKey ?? copilotToken.token;
const runtimeBaseUrl = protectedAuth?.baseUrl?.trim();
if (runtimeBaseUrl) {
model = { ...model, baseUrl: runtimeBaseUrl };
}
}
authStorage.setRuntimeApiKey(model.provider, apiKey);
return { apiKey, model };
return { apiKey, model: applySecretRefHeaderSentinels(model, params.cfg) };
}
function buildImageContext(
@@ -333,13 +347,15 @@ async function describeImagesWithMinimax(params: {
images: Array<{ buffer: Buffer; mime?: string }>;
}): Promise<ImagesDescriptionResult> {
const responses: string[] = [];
// MiniMax VLM owns a direct fetch path, so unwrap only at this final handoff.
const apiKey = unwrapSecretSentinelsForProviderEgress(params.apiKey, "MiniMax VLM request");
for (const [index, image] of params.images.entries()) {
const prompt =
params.images.length > 1
? `${params.prompt}\n\nDescribe image ${index + 1} of ${params.images.length} independently.`
: params.prompt;
const text = await minimaxUnderstandImage({
apiKey: params.apiKey,
apiKey,
provider: params.provider,
prompt,
imageDataUrl: `data:${image.mime ?? "image/jpeg"};base64,${image.buffer.toString("base64")}`,
@@ -425,6 +441,7 @@ async function resolveMinimaxVlmFallbackRuntime(params: {
const auth = await resolveApiKeyForProvider({
provider: authProvider,
cfg: params.cfg,
secretSentinels: true,
profileId: params.profile,
preferredProfile: params.preferredProfile,
agentDir: params.agentDir,
+77 -1
View File
@@ -1,5 +1,9 @@
// Proxy capture runtime tests cover session creation and capture lifecycle.
import { beforeEach, describe, expect, it } from "vitest";
import { beforeEach, describe, expect, it, vi } from "vitest";
import {
registerSecretValueForRedaction,
resetSecretRedactionRegistryForTest,
} from "../logging/secret-redaction-registry.js";
import type { DebugProxySettings } from "./env.js";
import {
captureHttpExchange,
@@ -94,6 +98,7 @@ describe("debug proxy runtime", () => {
finalizeDebugProxyCapture(settings, deps);
events.length = 0;
calls.length = 0;
resetSecretRedactionRegistryForTest();
fetchTarget.fetch = async () => new Response("{}", { status: 200 });
});
@@ -196,6 +201,77 @@ describe("debug proxy runtime", () => {
});
});
it("redacts registered exact values in custom headers and URL queries", async () => {
const secret = "capture-managed-secret";
const pathSecret = "capture/path secret";
registerSecretValueForRedaction(secret);
registerSecretValueForRedaction(pathSecret);
captureHttpExchange(
{
url: `https://api.example.com/models/${encodeURIComponent(pathSecret)}?key=${encodeURIComponent(secret)}`,
method: "GET",
requestHeaders: { "X-Managed": `Bearer ${secret}` },
response: new Response("{}", { status: 200 }),
},
settings,
deps,
);
await new Promise((resolve) => {
setImmediate(resolve);
});
const request = events.find((event) => event.kind === "request");
expect(request?.path).toBe("/models/%5BREDACTED%5D?key=%5BREDACTED%5D");
expect(JSON.parse(String(request?.headersJson))).toStrictEqual({
"X-Managed": "Bearer [REDACTED]",
});
});
it("redacts registered values from failed global-fetch capture events", async () => {
const secret = "capture-failure/secret";
registerSecretValueForRedaction(secret);
fetchTarget.fetch = vi.fn(async () => {
throw new Error(`request failed for ${secret}`);
}) as typeof fetch;
initializeDebugProxyCapture("test", settings, deps);
await expect(
fetchTarget.fetch(`https://api.example.com/models/${encodeURIComponent(secret)}`),
).rejects.toThrow("request failed");
const event = events.find((candidate) => candidate.kind === "error");
expect(event?.path).toBe("/models/%5BREDACTED%5D");
expect(event?.errorText).toBe("request failed for [REDACTED]");
});
it("keeps capture URLs valid when the full URL is a registered secret", () => {
const secretUrl = "https://signed.example/v1/callback";
registerSecretValueForRedaction(secretUrl);
captureHttpExchange(
{
url: secretUrl,
method: "GET",
response: new Response("{}", { status: 200 }),
},
settings,
deps,
);
const request = events.find((candidate) => candidate.kind === "request");
expect(request?.host).toBe("redacted.invalid");
expect(request?.path).toBe("/%5BREDACTED%5D");
});
it("does not fail capture on malformed percent escapes", async () => {
registerSecretValueForRedaction("capture-secret");
initializeDebugProxyCapture("test", settings, deps);
await expect(fetchTarget.fetch("https://api.example.com/x#%")).resolves.toBeInstanceOf(
Response,
);
});
it("skips capturing the body when Content-Length exceeds the cap", async () => {
initializeDebugProxyCapture("test", settings, deps);
captureHttpExchange(
+81 -10
View File
@@ -2,6 +2,7 @@
import { randomUUID } from "node:crypto";
import { URL } from "node:url";
import { normalizeRequestInitHeadersForFetch } from "../infra/fetch-headers.js";
import { redactRegisteredSecretValues } from "../logging/secret-redaction-registry.js";
import { resolveDebugProxySettings, type DebugProxySettings } from "./env.js";
import {
closeDebugProxyCaptureStore,
@@ -197,11 +198,79 @@ function redactedCaptureHeaders(
for (const [name, value] of entries) {
// Header names are matched exactly and by sensitive fragments because
// providers use many token/key naming variants.
redacted[name] = isSensitiveCaptureHeaderName(name) ? REDACTED_CAPTURE_HEADER_VALUE : value;
redacted[name] = isSensitiveCaptureHeaderName(name)
? REDACTED_CAPTURE_HEADER_VALUE
: redactRegisteredSecretValues(value, () => REDACTED_CAPTURE_HEADER_VALUE);
}
return redacted;
}
function redactCaptureUrl(rawUrl: string): string {
let url: URL;
try {
url = new URL(rawUrl);
} catch {
return "https://redacted.invalid/%5BREDACTED%5D";
}
const redactComponent = (value: string) =>
redactRegisteredSecretValues(value, () => REDACTED_CAPTURE_HEADER_VALUE);
const decodeComponent = (value: string) => {
try {
return decodeURIComponent(value);
} catch {
return value;
}
};
if (redactComponent(url.hostname) !== url.hostname) {
url.hostname = "redacted.invalid";
}
for (const key of ["username", "password"] as const) {
const decoded = decodeComponent(url[key]);
const redacted = redactComponent(decoded);
if (redacted !== decoded) {
url[key] = redacted;
}
}
url.pathname = url.pathname
.split("/")
.map((segment) => {
try {
const decoded = decodeURIComponent(segment);
const redacted = redactComponent(decoded);
return redacted === decoded ? segment : encodeURIComponent(redacted);
} catch {
return segment;
}
})
.join("/");
const searchParams = new URLSearchParams();
let searchChanged = false;
for (const [name, value] of url.searchParams.entries()) {
const redactedName = redactComponent(name);
const redactedValue = redactComponent(value);
searchParams.append(redactedName, redactedValue);
if (redactedName !== name || redactedValue !== value) {
searchChanged = true;
}
}
if (searchChanged) {
url.search = searchParams.toString();
}
const decodedHash = decodeComponent(url.hash.slice(1));
const redactedHash = redactComponent(decodedHash);
if (redactedHash !== decodedHash) {
url.hash = redactedHash;
}
const serialized = url.toString();
return redactComponent(serialized) === serialized
? serialized
: `${url.protocol}//redacted.invalid/%5BREDACTED%5D`;
}
function redactCaptureText(value: string): string {
return redactRegisteredSecretValues(value, () => REDACTED_CAPTURE_HEADER_VALUE);
}
function createHttpCaptureEventBase(params: {
settings: DebugProxySettings;
rawUrl: string;
@@ -285,13 +354,14 @@ function installDebugProxyGlobalFetchPatch(
} catch (error) {
if (url && /^https?:/i.test(url)) {
const store = runtime.getStore();
const parsed = new URL(url);
const captureUrl = redactCaptureUrl(url);
const parsed = new URL(captureUrl);
store.recordEvent({
sessionId: settings.sessionId,
ts: Date.now(),
sourceScope: "openclaw",
sourceProcess: settings.sourceProcess,
protocol: protocolFromUrl(url),
protocol: protocolFromUrl(captureUrl),
direction: "local",
kind: "error",
flowId: randomUUID(),
@@ -303,7 +373,7 @@ function installDebugProxyGlobalFetchPatch(
"GET",
host: parsed.host,
path: `${parsed.pathname}${parsed.search}`,
errorText: error instanceof Error ? error.message : String(error),
errorText: redactCaptureText(error instanceof Error ? error.message : String(error)),
metaJson: runtime.safeJsonString({ captureOrigin: "global-fetch" }),
});
}
@@ -389,7 +459,8 @@ export function captureHttpExchange(
const runtime = resolveRuntimeDeps(deps);
const store = runtime.getStore();
const flowId = params.flowId ?? randomUUID();
const url = new URL(params.url);
const captureUrl = redactCaptureUrl(params.url);
const url = new URL(captureUrl);
const requestBody =
typeof params.requestBody === "string" || Buffer.isBuffer(params.requestBody)
? params.requestBody
@@ -404,7 +475,7 @@ export function captureHttpExchange(
store.recordEvent({
...createHttpCaptureEventBase({
settings,
rawUrl: params.url,
rawUrl: captureUrl,
url,
transport: params.transport,
direction: "outbound",
@@ -427,7 +498,7 @@ export function captureHttpExchange(
store.recordEvent({
...createHttpCaptureEventBase({
settings,
rawUrl: params.url,
rawUrl: captureUrl,
url,
transport: params.transport,
direction: "inbound",
@@ -485,7 +556,7 @@ export function captureHttpExchange(
store.recordEvent({
...createHttpCaptureEventBase({
settings,
rawUrl: params.url,
rawUrl: captureUrl,
url,
transport: params.transport,
direction: "inbound",
@@ -504,7 +575,7 @@ export function captureHttpExchange(
store.recordEvent({
...createHttpCaptureEventBase({
settings,
rawUrl: params.url,
rawUrl: captureUrl,
url,
transport: params.transport,
direction: "local",
@@ -512,7 +583,7 @@ export function captureHttpExchange(
flowId,
method: params.method,
}),
errorText: error instanceof Error ? error.message : String(error),
errorText: redactCaptureText(error instanceof Error ? error.message : String(error)),
});
});
}
+23 -1
View File
@@ -1,5 +1,7 @@
/** Tests runtime SecretRef resolution across core config and auth-profile surfaces. */
import { describe, expect, it } from "vitest";
import { afterEach, describe, expect, it } from "vitest";
import { redactSensitiveText } from "../logging/redact.js";
import { resetSecretRedactionRegistryForTest } from "../logging/secret-redaction-registry.js";
import { asConfig, setupSecretsRuntimeSnapshotTestHooks } from "./runtime.test-support.ts";
const EMPTY_LOADABLE_PLUGIN_ORIGINS = new Map();
@@ -12,6 +14,10 @@ const CODEX_APP_SERVER_TOKEN_REF = {
id: "CODEX_APP_SERVER_TOKEN",
} as const;
afterEach(() => {
resetSecretRedactionRegistryForTest();
});
function expectWarning(
snapshot: Awaited<ReturnType<typeof prepareSecretsRuntimeSnapshot>>,
expected: { code: string; path: string },
@@ -25,6 +31,22 @@ function expectWarning(
}
describe("secrets runtime snapshot", () => {
it("registers every resolved value for exact redaction", async () => {
const secret = "runtime-registration-secret";
await prepareSecretsRuntimeSnapshot({
config: asConfig({
talk: {
apiKey: { source: "env", provider: "default", id: "TALK_API_KEY" },
},
}),
env: { TALK_API_KEY: secret },
includeAuthStoreRefs: false,
loadablePluginOrigins: EMPTY_LOADABLE_PLUGIN_ORIGINS,
});
expect(redactSensitiveText(`resolved ${secret}`, { mode: "off" })).toBe("resolved runtim…cret");
});
it("resolves sandbox ssh secret refs for active ssh backends", async () => {
const snapshot = await prepareSecretsRuntimeSnapshot({
config: asConfig({
+6
View File
@@ -9,6 +9,7 @@ import {
} from "../agents/auth-profiles.js";
import type { AuthProfileStore } from "../agents/auth-profiles/types.js";
import type { OpenClawConfig } from "../config/types.openclaw.js";
import { registerSecretValueForRedaction } from "../logging/secret-redaction-registry.js";
import type { PluginManifestRegistry } from "../plugins/manifest-registry.js";
import type { PluginMetadataSnapshot } from "../plugins/plugin-metadata-snapshot.js";
import type { PluginOrigin } from "../plugins/plugin-origin.types.js";
@@ -220,6 +221,11 @@ export async function prepareSecretsRuntimeSnapshot(params: {
cache: context.cache,
manifestRegistry: context.manifestRegistry,
});
for (const value of resolved.values()) {
if (typeof value === "string") {
registerSecretValueForRedaction(value);
}
}
applyResolvedAssignments({
assignments: context.assignments,
resolved,
+86
View File
@@ -0,0 +1,86 @@
import { afterEach, describe, expect, it } from "vitest";
import { redactSensitiveText } from "../logging/redact.js";
import { resetSecretRedactionRegistryForTest } from "../logging/secret-redaction-registry.js";
import {
looksLikeSecretSentinel,
mintSecretSentinel,
resolveSecretSentinel,
SECRET_SENTINEL_PATTERN,
swapSecretSentinelsInText,
} from "./sentinel.js";
describe("secret sentinels", () => {
afterEach(() => {
delete process.env.OPENCLAW_SECRET_SENTINELS;
resetSecretRedactionRegistryForTest();
});
it("mints, recognizes, resolves, and reuses sentinels by value and label", () => {
const first = mintSecretSentinel("provider-secret-value", { label: "model-auth:openai" });
const repeated = mintSecretSentinel("provider-secret-value", { label: "model-auth:openai" });
const otherLabel = mintSecretSentinel("provider-secret-value", { label: "model-auth:other" });
expect(first).toMatch(/^oc-sent-v1-[0-9a-f]{24}$/);
expect(first.match(SECRET_SENTINEL_PATTERN)).toEqual([first]);
expect(looksLikeSecretSentinel(first)).toBe(true);
expect(resolveSecretSentinel(first)).toBe("provider-secret-value");
expect(repeated).toBe(first);
expect(otherLabel).not.toBe(first);
});
it("swaps repeated and composed sentinel substrings", () => {
const first = mintSecretSentinel("first-secret-value", { label: "model-auth:openai" });
const second = mintSecretSentinel("second-secret-value", { label: "model-auth:cloudflare" });
expect(
swapSecretSentinelsInText(`Bearer ${first}; cf-aig-authorization=Bearer ${second}; ${first}`),
).toEqual({
text: "Bearer first-secret-value; cf-aig-authorization=Bearer second-secret-value; first-secret-value",
unknown: [],
});
});
it("reports unknown sentinel-shaped values without replacing them", () => {
const unknown = "oc-sent-v1-0123456789abcdef01234567";
expect(swapSecretSentinelsInText(`Bearer ${unknown}`)).toEqual({
text: `Bearer ${unknown}`,
unknown: [unknown],
});
});
it("treats sentinel-shaped bytes inside resolved values as opaque", () => {
const secret = "prefix-oc-sent-v1-0123456789abcdef01234567";
const sentinel = mintSecretSentinel(secret, { label: "nested-shape" });
expect(swapSecretSentinelsInText(`Bearer ${sentinel}`)).toEqual({
text: `Bearer ${secret}`,
unknown: [],
});
});
it.each(["off", " OFF ", "0", "false", "False"])(
"returns plaintext when the kill switch is %s",
(value) => {
process.env.OPENCLAW_SECRET_SENTINELS = value;
expect(mintSecretSentinel("kill-switch-secret", { label: "model-auth:test" })).toBe(
"kill-switch-secret",
);
},
);
it("registers minted values for exact redaction across registry eviction", () => {
const first = "sentinel-registry-value-000";
const firstSentinel = mintSecretSentinel(first, { label: "model-auth:0" });
for (let index = 1; index <= 512; index += 1) {
mintSecretSentinel(`sentinel-registry-value-${index.toString().padStart(3, "0")}`, {
label: `model-auth:${index}`,
});
}
const last = "sentinel-registry-value-512";
expect(redactSensitiveText(first, { mode: "tools", patterns: [] })).toBe(first);
expect(redactSensitiveText(last, { mode: "tools", patterns: [] })).not.toContain(last);
expect(resolveSecretSentinel(firstSentinel)).toBe(first);
expect(redactSensitiveText(first, { mode: "tools", patterns: [] })).not.toContain(first);
});
});
+67
View File
@@ -0,0 +1,67 @@
import { randomBytes } from "node:crypto";
import { registerSecretValueForRedaction } from "../logging/secret-redaction-registry.js";
const SECRET_SENTINEL_PREFIX = "oc-sent-v1-";
const SECRET_SENTINEL_SOURCE = `${SECRET_SENTINEL_PREFIX}[0-9a-f]{24}`;
export const SECRET_SENTINEL_PATTERN = new RegExp(SECRET_SENTINEL_SOURCE, "g");
const valuesBySentinel = new Map<string, string>();
const sentinelsByValueAndLabel = new Map<string, Map<string, string>>();
function secretSentinelsEnabled(env: NodeJS.ProcessEnv = process.env): boolean {
const configured = env.OPENCLAW_SECRET_SENTINELS?.trim().toLowerCase();
return configured !== "off" && configured !== "0" && configured !== "false";
}
export function looksLikeSecretSentinel(value: string): boolean {
return new RegExp(`^${SECRET_SENTINEL_SOURCE}$`).test(value);
}
/** Mints one stable process-local sentinel for a secret value and label. */
export function mintSecretSentinel(value: string, meta: { label: string }): string {
registerSecretValueForRedaction(value);
if (!secretSentinelsEnabled()) {
return value;
}
const byLabel = sentinelsByValueAndLabel.get(value) ?? new Map<string, string>();
const existing = byLabel.get(meta.label);
if (existing) {
return existing;
}
let sentinel: string;
do {
sentinel = `${SECRET_SENTINEL_PREFIX}${randomBytes(12).toString("hex")}`;
} while (valuesBySentinel.has(sentinel));
byLabel.set(meta.label, sentinel);
sentinelsByValueAndLabel.set(value, byLabel);
valuesBySentinel.set(sentinel, value);
return sentinel;
}
/** Resolves a process-local sentinel without exposing the registry itself. */
export function resolveSecretSentinel(sentinel: string): string | undefined {
const value = valuesBySentinel.get(sentinel);
if (value !== undefined) {
// Refresh the bounded redaction registry whenever a live sentinel is used.
registerSecretValueForRedaction(value);
}
return value;
}
/** Swaps every known sentinel substring and reports unknown sentinel-shaped values. */
export function swapSecretSentinelsInText(text: string): { text: string; unknown: string[] } {
if (!text.includes(SECRET_SENTINEL_PREFIX)) {
return { text, unknown: [] };
}
const unknown = new Set<string>();
const swapped = text.replace(new RegExp(SECRET_SENTINEL_SOURCE, "g"), (sentinel) => {
const value = resolveSecretSentinel(sentinel);
if (value === undefined) {
unknown.add(sentinel);
return sentinel;
}
return value;
});
return { text: swapped, unknown: [...unknown] };
}