mirror of
https://github.com/openclaw/openclaw.git
synced 2026-08-12 21:53:00 -06:00
feat(secrets): egress-time credential injection with process-local sentinels (#102009)
* feat(secrets): resolve SecretRef model credentials at egress via process-local sentinels SecretRef-managed model-provider credentials now travel as opaque oc-sent-v1 sentinels through auth storage, stream options, and SDK config; the guarded model fetch injects real values into headers and URLs immediately before the SSRF-guarded send and fails closed on unknown sentinels. packages/ai adapters converge on the host guarded fetch where the SDK supports custom fetch and unwrap at construction where it does not. Resolved values (and their percent-encoded forms) register for exact-value log redaction. Kill switch: OPENCLAW_SECRET_SENTINELS=off. Also fixes a pre-existing unhandled rejection race in capNonOkResponseBodyLazily (pipeThrough writer leak). * test(plugin-sdk): update public surface budget
This commit is contained in:
committed by
GitHub
parent
83ebbcb3ac
commit
4bf70be01a
@@ -23,6 +23,7 @@ Docs: https://docs.openclaw.ai
|
||||
|
||||
### Fixes
|
||||
|
||||
- **SecretRef model credentials:** keep resolved provider secrets behind process-local sentinels through auth storage, stream setup, SDK configuration, and managed local-provider probing, then inject plaintext only at the final network or provider-plugin boundary while retaining exact-value log redaction. (#102008, #102009)
|
||||
- **Lean local model shell access:** keep `exec` directly visible beside the default structured Tool Search controls so coding-tuned local models can use their shell fallback instead of searching for missing domain tools. (#87587) Thanks @vincentkoc.
|
||||
- **OAuth refresh contention diagnostics:** keep local lock paths out of user-facing refresh failures and avoid duplicate failure prefixes while preserving structured provider and profile classification. (#83383) Thanks @vincentkoc.
|
||||
- **Exec approval prompts:** keep background-disabled fallback warnings out of pending gateway/node approvals and show them only after a command actually runs in the foreground. (#78184) Thanks @vincentkoc.
|
||||
|
||||
@@ -3710,6 +3710,7 @@ Do not edit it by hand; run `pnpm docs:map:gen`.
|
||||
- Route: /gateway/secrets
|
||||
- Headings:
|
||||
- H2: Runtime model
|
||||
- H2: Egress-time injection (sentinels)
|
||||
- H2: Agent-access boundary
|
||||
- H2: Active-surface filtering
|
||||
- H2: Gateway auth surface diagnostics
|
||||
|
||||
+16
-1
@@ -24,10 +24,25 @@ Plaintext credentials remain agent-readable if they sit in files the agent can i
|
||||
- Startup fails fast when an effectively active SecretRef cannot be resolved.
|
||||
- Reload is an atomic swap: full success, or keep the last-known-good snapshot.
|
||||
- Policy violations (for example an OAuth-mode auth profile combined with SecretRef input) fail activation before the runtime swap.
|
||||
- Runtime requests read only the active in-memory snapshot. Outbound delivery paths (Discord reply/thread delivery, Telegram action sends) also read that snapshot and do not re-resolve refs per send.
|
||||
- Runtime requests read only the active in-memory snapshot. Model-provider SecretRef credentials pass through auth storage and stream options as process-local sentinels until egress. Outbound delivery paths (Discord reply/thread delivery, Telegram action sends) also read that snapshot and do not re-resolve refs per send.
|
||||
|
||||
This keeps secret-provider outages off hot request paths.
|
||||
|
||||
## Egress-time injection (sentinels)
|
||||
|
||||
For model-provider credentials backed by SecretRefs, OpenClaw mints an opaque, process-local sentinel during model-auth resolution. Auth storage, stream options, SDK configuration, logs, error objects, and most runtime introspection therefore see a value such as `oc-sent-v1-...`, not the provider credential. The guarded model fetch and managed local-provider health probes replace known sentinels in URL and header values immediately before each request leaves the process.
|
||||
|
||||
Unknown sentinel-shaped values fail closed before network activity. OpenClaw refuses to send the request rather than forwarding an unresolved sentinel to a provider. Resolved secret values are also registered for exact-value log redaction as a defense in depth measure.
|
||||
|
||||
Provider adapters use the latest injection point their SDK supports:
|
||||
|
||||
- SDKs with a custom fetch option receive OpenClaw's guarded fetch, so the SDK retains the sentinel.
|
||||
- SDKs without a custom fetch option unwrap the sentinel immediately before client construction. Plugin-owned provider streams and agent harnesses unwrap at the final core-owned handoff because those transports do not share OpenClaw's guarded fetch.
|
||||
|
||||
Sentinels reduce plaintext exposure across the model-call chain, but they are not process isolation. The real value still exists in same-process memory and appears at the final adapter boundary. Plain environment credentials that are not configured through SecretRefs remain plaintext and are outside this mechanism.
|
||||
|
||||
Set `OPENCLAW_SECRET_SENTINELS=off` (also accepts `0` or `false`, case-insensitive) to disable sentinel minting during incident response or compatibility troubleshooting. The kill switch does not disable exact-value redaction registration.
|
||||
|
||||
## Agent-access boundary
|
||||
|
||||
SecretRefs stop credentials from being persisted in config and generated model files, but they are not a process-isolation boundary. A plaintext credential left on disk in a path the agent can read is still readable via file or shell tools, bypassing API-level redaction.
|
||||
|
||||
@@ -22,6 +22,8 @@ export interface AiTransportHost {
|
||||
timeoutMs?: number,
|
||||
options?: { sanitizeSse?: boolean },
|
||||
): typeof fetch | undefined;
|
||||
/** Resolves host-owned process-local secret sentinel substrings immediately before egress. */
|
||||
resolveSecretSentinel(value: string): string;
|
||||
/** Redacts secrets inside structured tool-result payloads. */
|
||||
redactSecrets<T>(value: T): T;
|
||||
/** Redacts secret-bearing text in tool payload strings. */
|
||||
@@ -46,6 +48,7 @@ export interface AiTransportHost {
|
||||
|
||||
const inertAiTransportHost: AiTransportHost = {
|
||||
buildModelFetch: () => undefined,
|
||||
resolveSecretSentinel: (value) => value,
|
||||
redactSecrets: (value) => value,
|
||||
redactToolPayloadText: (text) => text,
|
||||
resolveOpenAIStrictToolSetting: (_model, options) =>
|
||||
@@ -64,3 +67,22 @@ export function configureAiTransportHost(host: Partial<AiTransportHost>): void {
|
||||
export function getAiTransportHost(): AiTransportHost {
|
||||
return activeAiTransportHost;
|
||||
}
|
||||
|
||||
/** Resolves sentinel substrings in custom headers at a no-fetch adapter boundary. */
|
||||
export function resolveAiTransportHeaderSentinels(
|
||||
headers: Record<string, string> | undefined,
|
||||
): Record<string, string> | undefined {
|
||||
if (!headers) {
|
||||
return undefined;
|
||||
}
|
||||
const host = getAiTransportHost();
|
||||
let resolvedHeaders: Record<string, string> | undefined;
|
||||
for (const [name, value] of Object.entries(headers)) {
|
||||
const resolved = host.resolveSecretSentinel(value);
|
||||
if (resolved !== value) {
|
||||
resolvedHeaders ??= { ...headers };
|
||||
resolvedHeaders[name] = resolved;
|
||||
}
|
||||
}
|
||||
return resolvedHeaders ?? headers;
|
||||
}
|
||||
|
||||
@@ -151,6 +151,35 @@ describe("Anthropic provider", () => {
|
||||
expect(config.defaultHeaders?.["x-api-key"]).toBeUndefined();
|
||||
});
|
||||
|
||||
it("keeps sentinel-backed Foundry Authorization headers on bearer routing", async () => {
|
||||
const sentinel = "oc-sent-v1-0123456789abcdef01234567";
|
||||
configureAiTransportHost({
|
||||
buildModelFetch: () => async () => new Response(null, { status: 500 }),
|
||||
resolveSecretSentinel: (value) => value.replaceAll(sentinel, "Bearer entra-access-token"),
|
||||
});
|
||||
const model = makeAnthropicModel({
|
||||
provider: "microsoft-foundry",
|
||||
baseUrl: "https://example.services.ai.azure.com/anthropic",
|
||||
headers: { Authorization: sentinel },
|
||||
});
|
||||
|
||||
streamAnthropic(
|
||||
model,
|
||||
{ messages: [{ role: "user", content: "hello", timestamp: 1 }] },
|
||||
{
|
||||
apiKey: sentinel,
|
||||
},
|
||||
);
|
||||
|
||||
await vi.waitFor(() => expect(anthropicMockState.configs).toHaveLength(1));
|
||||
const config = anthropicMockState.configs[0] as {
|
||||
apiKey?: string | null;
|
||||
authToken?: string | null;
|
||||
};
|
||||
expect(config.apiKey).toBeNull();
|
||||
expect(config.authToken).toBe(sentinel);
|
||||
});
|
||||
|
||||
it("keeps Microsoft Foundry API-key profiles on Anthropic API key auth", async () => {
|
||||
const model = makeAnthropicModel({
|
||||
provider: "microsoft-foundry",
|
||||
|
||||
@@ -9,7 +9,7 @@ import type {
|
||||
TextBlockParam,
|
||||
} from "@anthropic-ai/sdk/resources/messages.js";
|
||||
import { getEnvApiKey } from "../env-api-keys.js";
|
||||
import { getAiTransportHost } from "../host.js";
|
||||
import { getAiTransportHost, resolveAiTransportHeaderSentinels } from "../host.js";
|
||||
import { calculateCost, clampThinkingLevel } from "../model-utils.js";
|
||||
import type {
|
||||
AnthropicMessagesCompat,
|
||||
@@ -1111,7 +1111,8 @@ export const streamSimpleAnthropic: StreamFunction<
|
||||
};
|
||||
|
||||
function isOAuthToken(apiKey: string): boolean {
|
||||
return apiKey.includes("sk-ant-oat");
|
||||
// Inspect the host-resolved shape only for auth routing; the SDK still receives the sentinel.
|
||||
return getAiTransportHost().resolveSecretSentinel(apiKey).includes("sk-ant-oat");
|
||||
}
|
||||
|
||||
function isAnthropicPublicEndpoint(baseUrl: string | undefined): boolean {
|
||||
@@ -1161,6 +1162,7 @@ function createClient(
|
||||
/^kimi(?:-|$)/.test(model.provider) && thinkingEnabled
|
||||
? { sanitizeSse: false as const }
|
||||
: undefined;
|
||||
// Anthropic supports custom fetch, so sentinels stay opaque until guarded egress.
|
||||
const fetch = getAiTransportHost().buildModelFetch(model, undefined, fetchOptions);
|
||||
|
||||
if (model.provider === "cloudflare-ai-gateway") {
|
||||
@@ -1208,7 +1210,12 @@ function createClient(
|
||||
return { client, isOAuthToken: false, serverSideFallback: false };
|
||||
}
|
||||
|
||||
if (usesFoundryBearerAuth(model)) {
|
||||
if (
|
||||
usesFoundryBearerAuth({
|
||||
...model,
|
||||
headers: resolveAiTransportHeaderSentinels(model.headers),
|
||||
})
|
||||
) {
|
||||
const client = new Anthropic({
|
||||
apiKey: null,
|
||||
authToken: apiKey,
|
||||
|
||||
@@ -200,6 +200,7 @@ function createClient(
|
||||
}
|
||||
|
||||
const { baseUrl, apiVersion } = resolveAzureConfig(model, options);
|
||||
// Both OpenAI clients support custom fetch, so sentinels stay opaque until guarded egress.
|
||||
const guardedFetch = getAiTransportHost().buildModelFetch({ ...model, baseUrl });
|
||||
|
||||
if (isOpenAICompatibleAzureResponsesBaseUrl(baseUrl)) {
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
// Cloudflare provider metadata describes Cloudflare-hosted model capabilities.
|
||||
import type { Model } from "../types.js";
|
||||
|
||||
// This module owns URL metadata only; Anthropic/OpenAI adapters inject guarded fetch.
|
||||
|
||||
export function isCloudflareProvider(provider: string): boolean {
|
||||
return provider === "cloudflare-workers-ai" || provider === "cloudflare-ai-gateway";
|
||||
}
|
||||
|
||||
@@ -0,0 +1,124 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { configureAiTransportHost } from "../host.js";
|
||||
import type { Context, Model } from "../types.js";
|
||||
|
||||
const googleMockState = vi.hoisted(() => ({ configs: [] as unknown[] }));
|
||||
|
||||
vi.mock("@google/genai", () => ({
|
||||
GoogleGenAI: class MockGoogleGenAI {
|
||||
models = {
|
||||
generateContentStream: vi.fn(() => {
|
||||
throw new Error("stop after constructor");
|
||||
}),
|
||||
};
|
||||
|
||||
constructor(config: unknown) {
|
||||
googleMockState.configs.push(config);
|
||||
}
|
||||
},
|
||||
ResourceScope: { COLLECTION: "COLLECTION" },
|
||||
ThinkingLevel: {
|
||||
THINKING_LEVEL_UNSPECIFIED: "THINKING_LEVEL_UNSPECIFIED",
|
||||
MINIMAL: "MINIMAL",
|
||||
LOW: "LOW",
|
||||
MEDIUM: "MEDIUM",
|
||||
HIGH: "HIGH",
|
||||
},
|
||||
}));
|
||||
|
||||
import { streamGoogleVertex } from "./google-vertex.js";
|
||||
import { streamGoogle } from "./google.js";
|
||||
|
||||
const context = {
|
||||
messages: [{ role: "user", content: "hello", timestamp: 0 }],
|
||||
} satisfies Context;
|
||||
const sentinel = "oc-sent-v1-0123456789abcdef01234567";
|
||||
|
||||
function googleModel(): Model<"google-generative-ai"> {
|
||||
return {
|
||||
id: "gemini-3-flash-preview",
|
||||
name: "Gemini 3 Flash Preview",
|
||||
api: "google-generative-ai",
|
||||
provider: "google",
|
||||
baseUrl: "https://generativelanguage.googleapis.com/v1beta",
|
||||
reasoning: true,
|
||||
input: ["text"],
|
||||
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
|
||||
contextWindow: 1_000_000,
|
||||
maxTokens: 8192,
|
||||
};
|
||||
}
|
||||
|
||||
function vertexModel(): Model<"google-vertex"> {
|
||||
return {
|
||||
...googleModel(),
|
||||
api: "google-vertex",
|
||||
provider: "google-vertex",
|
||||
baseUrl: "https://us-central1-aiplatform.googleapis.com/v1",
|
||||
};
|
||||
}
|
||||
|
||||
describe("Google SDK construction auth", () => {
|
||||
beforeEach(() => {
|
||||
googleMockState.configs = [];
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
configureAiTransportHost({});
|
||||
});
|
||||
|
||||
it("unwraps Google API-key sentinels immediately before client construction", async () => {
|
||||
const buildModelFetch = vi.fn();
|
||||
configureAiTransportHost({
|
||||
buildModelFetch,
|
||||
resolveSecretSentinel: (value) => value.replaceAll(sentinel, "google-construction-secret"),
|
||||
});
|
||||
|
||||
const result = await streamGoogle(
|
||||
{
|
||||
...googleModel(),
|
||||
headers: { Authorization: `Bearer ${sentinel}` },
|
||||
},
|
||||
context,
|
||||
{ apiKey: sentinel },
|
||||
).result();
|
||||
|
||||
expect(result.stopReason).toBe("error");
|
||||
expect(googleMockState.configs[0]).toMatchObject({
|
||||
apiKey: "google-construction-secret",
|
||||
httpOptions: { headers: { Authorization: "Bearer google-construction-secret" } },
|
||||
});
|
||||
expect(JSON.stringify(googleMockState.configs[0])).not.toContain(sentinel);
|
||||
expect(buildModelFetch).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("unwraps Vertex API-key sentinels immediately before client construction", async () => {
|
||||
const buildModelFetch = vi.fn();
|
||||
configureAiTransportHost({
|
||||
buildModelFetch,
|
||||
resolveSecretSentinel: (value) => value.replaceAll(sentinel, "vertex-construction-secret"),
|
||||
});
|
||||
|
||||
const result = await streamGoogleVertex(
|
||||
{
|
||||
...vertexModel(),
|
||||
headers: { "X-Provider-Token": sentinel },
|
||||
},
|
||||
context,
|
||||
{
|
||||
apiKey: sentinel,
|
||||
project: "demo-project",
|
||||
location: "us-central1",
|
||||
},
|
||||
).result();
|
||||
|
||||
expect(result.stopReason).toBe("error");
|
||||
expect(googleMockState.configs[0]).toMatchObject({
|
||||
apiKey: "vertex-construction-secret",
|
||||
vertexai: true,
|
||||
httpOptions: { headers: { "X-Provider-Token": "vertex-construction-secret" } },
|
||||
});
|
||||
expect(JSON.stringify(googleMockState.configs[0])).not.toContain(sentinel);
|
||||
expect(buildModelFetch).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -6,6 +6,7 @@ import {
|
||||
ResourceScope,
|
||||
ThinkingLevel as VertexThinkingLevel,
|
||||
} from "@google/genai";
|
||||
import { getAiTransportHost, resolveAiTransportHeaderSentinels } from "../host.js";
|
||||
import type { Context, Model, SimpleStreamOptions, StreamFunction } from "../types.js";
|
||||
import { AssistantMessageEventStream } from "../utils/event-stream.js";
|
||||
import type { GoogleThinkingLevel } from "./google-shared.js";
|
||||
@@ -97,9 +98,11 @@ function createClientWithApiKey(
|
||||
apiKey: string,
|
||||
optionsHeaders?: Record<string, string>,
|
||||
): GoogleGenAI {
|
||||
// @google/genai exposes RequestInit options but no custom fetch; unwrap at construction.
|
||||
const resolvedApiKey = getAiTransportHost().resolveSecretSentinel(apiKey);
|
||||
return new GoogleGenAI({
|
||||
vertexai: true,
|
||||
apiKey,
|
||||
apiKey: resolvedApiKey,
|
||||
apiVersion: API_VERSION,
|
||||
httpOptions: buildHttpOptions(model, optionsHeaders),
|
||||
});
|
||||
@@ -120,7 +123,10 @@ function buildHttpOptions(
|
||||
}
|
||||
|
||||
if (model.headers || optionsHeaders) {
|
||||
httpOptions.headers = { ...model.headers, ...optionsHeaders };
|
||||
httpOptions.headers = resolveAiTransportHeaderSentinels({
|
||||
...model.headers,
|
||||
...optionsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
return Object.keys(httpOptions).length > 0 ? httpOptions : undefined;
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
// Google provider adapts Gemini streams and tools to the agent runtime.
|
||||
import { type GenerateContentParameters, GoogleGenAI } from "@google/genai";
|
||||
import { getEnvApiKey } from "../env-api-keys.js";
|
||||
import { getAiTransportHost, resolveAiTransportHeaderSentinels } from "../host.js";
|
||||
import type { Context, Model, SimpleStreamOptions, StreamFunction } from "../types.js";
|
||||
import { AssistantMessageEventStream } from "../utils/event-stream.js";
|
||||
import {
|
||||
@@ -74,11 +75,16 @@ function createClient(
|
||||
httpOptions.apiVersion = ""; // baseUrl already includes version path, don't append
|
||||
}
|
||||
if (model.headers || optionsHeaders) {
|
||||
httpOptions.headers = { ...model.headers, ...optionsHeaders };
|
||||
httpOptions.headers = resolveAiTransportHeaderSentinels({
|
||||
...model.headers,
|
||||
...optionsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
// @google/genai exposes RequestInit options but no custom fetch; unwrap at construction.
|
||||
const resolvedApiKey = apiKey ? getAiTransportHost().resolveSecretSentinel(apiKey) : undefined;
|
||||
return new GoogleGenAI({
|
||||
apiKey,
|
||||
apiKey: resolvedApiKey,
|
||||
httpOptions: Object.keys(httpOptions).length > 0 ? httpOptions : undefined,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -128,7 +128,6 @@ describe("Mistral bounded-stream-read real wire proof (loopback http.createServe
|
||||
// parser (`EventStream`) would see when a streaming body exceeds 16 MiB.
|
||||
describe("Mistral bounded-stream-read direct (synthetic ReadableStream)", () => {
|
||||
it("caps an oversized synthetic ReadableStream at 16 MiB", async () => {
|
||||
const fetcher = createBoundedMistralFetcher(MAX);
|
||||
const CHUNK = 1024 * 1024;
|
||||
let sent = 0;
|
||||
const synthetic = new ReadableStream<Uint8Array>({
|
||||
@@ -147,38 +146,23 @@ describe("Mistral bounded-stream-read direct (synthetic ReadableStream)", () =>
|
||||
status: 200,
|
||||
headers: { "content-type": "application/octet-stream" },
|
||||
});
|
||||
const fetcher = createBoundedMistralFetcher(MAX, async () => syntheticResponse);
|
||||
|
||||
let captured: Error | undefined;
|
||||
const wrapped = await fetcher("http://unused.invalid/");
|
||||
try {
|
||||
// Replace the fetcher's internal `fetch` call by exercising the
|
||||
// post-fetchResponse code path directly: build a `Wrapped`
|
||||
// that re-enters `fetcher` as if a real fetch returned our
|
||||
// synthetic Response, by patching the global fetch.
|
||||
const originalFetch = globalThis.fetch;
|
||||
globalThis.fetch = (() => Promise.resolve(syntheticResponse)) as typeof globalThis.fetch;
|
||||
try {
|
||||
const wrapped = await fetcher("http://unused.invalid/");
|
||||
try {
|
||||
await readAllChunks(wrapped.body);
|
||||
} catch (err) {
|
||||
captured = err as Error;
|
||||
}
|
||||
} finally {
|
||||
globalThis.fetch = originalFetch;
|
||||
}
|
||||
expect(captured).toBeInstanceOf(Error);
|
||||
const match = (captured as Error).message.match(
|
||||
/mistral: stream body exceeds \d+ bytes \(got (\d+)\)/,
|
||||
);
|
||||
expect(match).not.toBeNull();
|
||||
const got = Number(match![1]);
|
||||
// Synthetic stream chunks are exactly 1 MiB aligned, so cap+1 reads
|
||||
// give exactly cap + 1 MiB = 16 MiB + 1 MiB = 17 825 792 bytes.
|
||||
expect(got).toBe(16777216 + CHUNK);
|
||||
} finally {
|
||||
// Best-effort cleanup if the test threw mid-flight.
|
||||
// No intervals to clear for this test; the synthetic stream closes
|
||||
// automatically when `sent >= 18`.
|
||||
await readAllChunks(wrapped.body);
|
||||
} catch (err) {
|
||||
captured = err as Error;
|
||||
}
|
||||
expect(captured).toBeInstanceOf(Error);
|
||||
const match = (captured as Error).message.match(
|
||||
/mistral: stream body exceeds \d+ bytes \(got (\d+)\)/,
|
||||
);
|
||||
expect(match).not.toBeNull();
|
||||
const got = Number(match![1]);
|
||||
// Synthetic stream chunks are exactly 1 MiB aligned, so cap+1 reads
|
||||
// give exactly cap + 1 MiB = 16 MiB + 1 MiB = 17 825 792 bytes.
|
||||
expect(got).toBe(16777216 + CHUNK);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -5,6 +5,7 @@ import type { Context, Model } from "../types.js";
|
||||
import { SYSTEM_PROMPT_CACHE_BOUNDARY } from "../utils/system-prompt-cache-boundary.js";
|
||||
|
||||
const mistralMockState = vi.hoisted(() => ({
|
||||
configs: [] as unknown[],
|
||||
payloads: [] as unknown[],
|
||||
}));
|
||||
|
||||
@@ -19,6 +20,10 @@ vi.mock("@mistralai/mistralai", async () => {
|
||||
return {
|
||||
...actual,
|
||||
Mistral: class MockMistral {
|
||||
constructor(config: unknown) {
|
||||
mistralMockState.configs.push(config);
|
||||
}
|
||||
|
||||
chat = {
|
||||
stream: vi.fn(async (payload: unknown) => {
|
||||
mistralMockState.payloads.push(payload);
|
||||
@@ -70,6 +75,7 @@ function makeUnreadableParameterTool() {
|
||||
|
||||
describe("Mistral provider", () => {
|
||||
beforeEach(() => {
|
||||
mistralMockState.configs = [];
|
||||
mistralMockState.payloads = [];
|
||||
});
|
||||
|
||||
@@ -89,6 +95,22 @@ describe("Mistral provider", () => {
|
||||
expect((mistralMockState.payloads[0] as { stop?: unknown }).stop).toEqual(["STOP"]);
|
||||
});
|
||||
|
||||
it("routes the Mistral HTTPClient through the host guarded fetch", async () => {
|
||||
const hostFetch = vi.fn<typeof fetch>(async () => new Response("guarded"));
|
||||
configureAiTransportHost({ buildModelFetch: () => hostFetch });
|
||||
|
||||
await streamMistral(makeMistralModel(), context, { apiKey: "sentinel-key" }).result();
|
||||
|
||||
const config = mistralMockState.configs[0] as {
|
||||
apiKey?: string;
|
||||
httpClient?: { request(request: Request): Promise<Response> };
|
||||
};
|
||||
expect(config.apiKey).toBe("sentinel-key");
|
||||
const response = await config.httpClient?.request(new Request("https://api.mistral.ai/chat"));
|
||||
expect(await response?.text()).toBe("guarded");
|
||||
expect(hostFetch).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("uses reasoning effort for Mistral Medium 3.5", async () => {
|
||||
const stream = streamSimpleMistral(
|
||||
{
|
||||
|
||||
@@ -8,6 +8,7 @@ import type {
|
||||
FunctionTool,
|
||||
} from "@mistralai/mistralai/models/components";
|
||||
import { getEnvApiKey } from "../env-api-keys.js";
|
||||
import { getAiTransportHost } from "../host.js";
|
||||
import { calculateCost, clampThinkingLevel } from "../model-utils.js";
|
||||
import type {
|
||||
AssistantMessage,
|
||||
@@ -57,9 +58,10 @@ const MISTRAL_STREAM_BODY_MAX_BYTES = 16 * 1024 * 1024;
|
||||
*/
|
||||
export function createBoundedMistralFetcher(
|
||||
maxBytes: number = MISTRAL_STREAM_BODY_MAX_BYTES,
|
||||
upstreamFetch: Fetcher = fetch,
|
||||
): Fetcher {
|
||||
return async (input, init) => {
|
||||
const response = init == null ? await fetch(input) : await fetch(input, init);
|
||||
const response = init == null ? await upstreamFetch(input) : await upstreamFetch(input, init);
|
||||
if (!response.body || typeof response.body.getReader !== "function") {
|
||||
return response;
|
||||
}
|
||||
@@ -139,7 +141,13 @@ export const streamMistral: StreamFunction<"mistral-conversations", MistralOptio
|
||||
// `httpClient` is passed, `ClientSDK.#httpClient` is set from it and
|
||||
// every `chat.stream` / `complete` call routes through
|
||||
// `HTTPClient.request` → `this.fetcher(req)`).
|
||||
httpClient: new HTTPClient({ fetcher: createBoundedMistralFetcher() }),
|
||||
// Mistral accepts HTTPClient.fetcher, so compose guarded egress with the byte cap.
|
||||
httpClient: new HTTPClient({
|
||||
fetcher: createBoundedMistralFetcher(
|
||||
MISTRAL_STREAM_BODY_MAX_BYTES,
|
||||
getAiTransportHost().buildModelFetch(model) ?? fetch,
|
||||
),
|
||||
}),
|
||||
});
|
||||
|
||||
const normalizeMistralToolCallId = createMistralToolCallIdNormalizer();
|
||||
|
||||
@@ -3,6 +3,7 @@ import { zstdDecompressSync } from "node:zlib";
|
||||
// ChatGPT Responses provider tests cover stream handling and timeout behavior.
|
||||
import { MAX_TIMER_TIMEOUT_MS } from "@openclaw/normalization-core/number-coercion";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { configureAiTransportHost } from "../host.js";
|
||||
import type { Context, Model } from "../types.js";
|
||||
import { SYSTEM_PROMPT_CACHE_BOUNDARY } from "../utils/system-prompt-cache-boundary.js";
|
||||
import {
|
||||
@@ -104,6 +105,7 @@ describe("streamOpenAICodexResponses transport", () => {
|
||||
vi.unstubAllGlobals();
|
||||
vi.useRealTimers();
|
||||
resetOpenAICodexWebSocketDebugStats();
|
||||
configureAiTransportHost({});
|
||||
});
|
||||
|
||||
const model = {
|
||||
@@ -123,6 +125,41 @@ describe("streamOpenAICodexResponses transport", () => {
|
||||
messages: [{ role: "user", content: "hi", timestamp: 1 }],
|
||||
} satisfies Context;
|
||||
|
||||
it("unwraps sentinels before constructing ChatGPT SSE auth headers", async () => {
|
||||
const realToken = createJwt({
|
||||
"https://api.openai.com/auth": { chatgpt_account_id: "acct-sentinel" },
|
||||
});
|
||||
const sentinel = "oc-sent-v1-0123456789abcdef01234567";
|
||||
configureAiTransportHost({
|
||||
resolveSecretSentinel: (value) => value.replaceAll(sentinel, realToken),
|
||||
});
|
||||
let authorization: string | null = null;
|
||||
let providerToken: string | null = null;
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
vi.fn(async (_input, init) => {
|
||||
const headers = new Headers(init?.headers);
|
||||
authorization = headers.get("authorization");
|
||||
providerToken = headers.get("x-provider-token");
|
||||
return completedSseResponse();
|
||||
}),
|
||||
);
|
||||
|
||||
const result = await streamOpenAICodexResponses(
|
||||
{ ...model, headers: { "X-Provider-Token": `Bearer ${sentinel}` } },
|
||||
context,
|
||||
{
|
||||
apiKey: sentinel,
|
||||
transport: "sse",
|
||||
},
|
||||
).result();
|
||||
|
||||
expect(result.stopReason).toBe("stop");
|
||||
expect(authorization).toBe(`Bearer ${realToken}`);
|
||||
expect(authorization).not.toContain(sentinel);
|
||||
expect(providerToken).toBe(`Bearer ${realToken}`);
|
||||
});
|
||||
|
||||
it("builds the first Node request with an OS-specific user agent", async () => {
|
||||
vi.resetModules();
|
||||
const freshProvider = await import("./openai-chatgpt-responses.js");
|
||||
|
||||
@@ -31,6 +31,7 @@ import {
|
||||
clampTimerTimeoutMs,
|
||||
} from "@openclaw/normalization-core/number-coercion";
|
||||
import { getEnvApiKey } from "../env-api-keys.js";
|
||||
import { getAiTransportHost, resolveAiTransportHeaderSentinels } from "../host.js";
|
||||
import { registerSessionResourceCleanup } from "../session-resources.js";
|
||||
import type {
|
||||
Api,
|
||||
@@ -264,10 +265,14 @@ export const streamOpenAICodexResponses: StreamFunction<
|
||||
};
|
||||
|
||||
try {
|
||||
const apiKey = options?.apiKey || getEnvApiKey(model.provider) || "";
|
||||
if (!apiKey) {
|
||||
const unresolvedApiKey = options?.apiKey || getEnvApiKey(model.provider) || "";
|
||||
if (!unresolvedApiKey) {
|
||||
throw new Error(`No API key for provider: ${model.provider}`);
|
||||
}
|
||||
// WebSocket auth has no fetch seam; unwrap immediately before request construction.
|
||||
const apiKey = getAiTransportHost().resolveSecretSentinel(unresolvedApiKey);
|
||||
const modelHeaders = resolveAiTransportHeaderSentinels(model.headers);
|
||||
const optionHeaders = resolveAiTransportHeaderSentinels(options?.headers);
|
||||
|
||||
const accountId = extractOpenAICodexAccountId(apiKey);
|
||||
let body = buildRequestBody(model, context, options);
|
||||
@@ -281,15 +286,15 @@ export const streamOpenAICodexResponses: StreamFunction<
|
||||
// see the SSE-path session_id addition in buildOpenAIClientHeaders (agents/openai-transport-stream.ts).
|
||||
const websocketRequestId = options?.sessionId || createCodexRequestId();
|
||||
const sseHeaders = buildSSEHeaders(
|
||||
model.headers,
|
||||
options?.headers,
|
||||
modelHeaders,
|
||||
optionHeaders,
|
||||
accountId,
|
||||
apiKey,
|
||||
options?.sessionId,
|
||||
);
|
||||
const websocketHeaders = buildWebSocketHeaders(
|
||||
model.headers,
|
||||
options?.headers,
|
||||
modelHeaders,
|
||||
optionHeaders,
|
||||
accountId,
|
||||
apiKey,
|
||||
websocketRequestId,
|
||||
|
||||
@@ -642,6 +642,7 @@ function createClient(
|
||||
baseURL: isCloudflareProvider(model.provider) ? resolveCloudflareBaseUrl(model) : model.baseUrl,
|
||||
dangerouslyAllowBrowser: true,
|
||||
defaultHeaders,
|
||||
// OpenAI supports custom fetch, so sentinels stay opaque until guarded egress.
|
||||
fetch: getAiTransportHost().buildModelFetch(model),
|
||||
});
|
||||
}
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { configureAiTransportHost } from "../host.js";
|
||||
import type { Context, Model } from "../types.js";
|
||||
|
||||
const openAiMockState = vi.hoisted(() => ({ configs: [] as unknown[] }));
|
||||
|
||||
vi.mock("openai", () => ({
|
||||
default: class MockOpenAI {
|
||||
responses = {
|
||||
create: vi.fn(() => {
|
||||
throw new Error("stop after constructor");
|
||||
}),
|
||||
};
|
||||
|
||||
constructor(config: unknown) {
|
||||
openAiMockState.configs.push(config);
|
||||
}
|
||||
},
|
||||
}));
|
||||
|
||||
import { streamOpenAIResponses } from "./openai-responses.js";
|
||||
|
||||
const context = {
|
||||
messages: [{ role: "user", content: "hello", timestamp: 0 }],
|
||||
} satisfies Context;
|
||||
|
||||
function model(overrides: Partial<Model<"openai-responses">> = {}) {
|
||||
return {
|
||||
id: "gpt-5.5",
|
||||
name: "GPT-5.5",
|
||||
api: "openai-responses",
|
||||
provider: "openai",
|
||||
baseUrl: "https://api.openai.com/v1",
|
||||
reasoning: true,
|
||||
input: ["text"],
|
||||
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
|
||||
contextWindow: 200_000,
|
||||
maxTokens: 8192,
|
||||
...overrides,
|
||||
} satisfies Model<"openai-responses">;
|
||||
}
|
||||
|
||||
describe("OpenAI Responses provider", () => {
|
||||
afterEach(() => {
|
||||
openAiMockState.configs = [];
|
||||
configureAiTransportHost({});
|
||||
});
|
||||
|
||||
it("constructs the SDK client with the host guarded fetch", async () => {
|
||||
const hostFetch: typeof fetch = async () => new Response(null, { status: 500 });
|
||||
configureAiTransportHost({ buildModelFetch: () => hostFetch });
|
||||
|
||||
const result = await streamOpenAIResponses(model(), context, {
|
||||
apiKey: "sentinel-key",
|
||||
}).result();
|
||||
|
||||
expect(result.stopReason).toBe("error");
|
||||
expect(openAiMockState.configs).toHaveLength(1);
|
||||
expect((openAiMockState.configs[0] as { fetch?: unknown }).fetch).toBe(hostFetch);
|
||||
});
|
||||
|
||||
it("keeps Cloudflare composed upstream auth opaque in SDK headers", async () => {
|
||||
const hostFetch: typeof fetch = async () => new Response(null, { status: 500 });
|
||||
configureAiTransportHost({ buildModelFetch: () => hostFetch });
|
||||
|
||||
await streamOpenAIResponses(
|
||||
model({
|
||||
provider: "cloudflare-ai-gateway",
|
||||
baseUrl: "https://gateway.ai.cloudflare.com/v1/account/gateway/openai",
|
||||
}),
|
||||
context,
|
||||
{ apiKey: "oc-sent-v1-0123456789abcdef01234567" },
|
||||
).result();
|
||||
|
||||
const config = openAiMockState.configs[0] as {
|
||||
apiKey?: string;
|
||||
defaultHeaders?: Record<string, string | null>;
|
||||
fetch?: unknown;
|
||||
};
|
||||
expect(config.apiKey).toBe("oc-sent-v1-0123456789abcdef01234567");
|
||||
expect(config.defaultHeaders?.["cf-aig-authorization"]).toBe(
|
||||
"Bearer oc-sent-v1-0123456789abcdef01234567",
|
||||
);
|
||||
expect(config.fetch).toBe(hostFetch);
|
||||
});
|
||||
});
|
||||
@@ -2,6 +2,7 @@
|
||||
import OpenAI from "openai";
|
||||
import type { ResponseCreateParamsStreaming } from "openai/resources/responses/responses.js";
|
||||
import { getEnvApiKey } from "../env-api-keys.js";
|
||||
import { getAiTransportHost } from "../host.js";
|
||||
import type {
|
||||
CacheRetention,
|
||||
Context,
|
||||
@@ -172,6 +173,8 @@ function createClient(
|
||||
baseURL: isCloudflareProvider(model.provider) ? resolveCloudflareBaseUrl(model) : model.baseUrl,
|
||||
dangerouslyAllowBrowser: true,
|
||||
defaultHeaders,
|
||||
// OpenAI supports custom fetch, so sentinels stay opaque until guarded egress.
|
||||
fetch: getAiTransportHost().buildModelFetch(model),
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
@@ -93,6 +93,7 @@ function createLazyRegistration<TApi extends Api, TOptions extends StreamOptions
|
||||
}
|
||||
|
||||
const registerBuiltIns: RegisterBuiltIn[] = [
|
||||
// Registration is transport-free; each lazy adapter owns its fetch or construction unwrap.
|
||||
createLazyRegistration(
|
||||
"anthropic-messages",
|
||||
() => import("./anthropic.js"),
|
||||
|
||||
@@ -195,12 +195,12 @@ export function readPluginSdkSurfaceBudgets(env = process.env) {
|
||||
),
|
||||
publicExports: readPluginSdkSurfaceBudgetEnv(
|
||||
"OPENCLAW_PLUGIN_SDK_MAX_PUBLIC_EXPORTS",
|
||||
10463,
|
||||
10464,
|
||||
env,
|
||||
),
|
||||
publicFunctionExports: readPluginSdkSurfaceBudgetEnv(
|
||||
"OPENCLAW_PLUGIN_SDK_MAX_PUBLIC_FUNCTION_EXPORTS",
|
||||
5220,
|
||||
5221,
|
||||
env,
|
||||
),
|
||||
publicDeprecatedExports: readPluginSdkSurfaceBudgetEnv(
|
||||
|
||||
@@ -62,6 +62,7 @@ import type {
|
||||
ThinkingLevel,
|
||||
} from "../llm/types.js";
|
||||
import "../llm/ai-transport-host.js";
|
||||
import { looksLikeSecretSentinel, resolveSecretSentinel } from "../secrets/sentinel.js";
|
||||
import { MALFORMED_STREAMING_FRAGMENT_ERROR_MESSAGE } from "../shared/assistant-error-format.js";
|
||||
import {
|
||||
applyAnthropicPayloadPolicyToParams,
|
||||
@@ -70,6 +71,7 @@ import {
|
||||
import { buildCopilotDynamicHeaders, hasCopilotVisionInput } from "./copilot-dynamic-headers.js";
|
||||
import { parseJsonObjectPreservingUnsafeIntegers } from "./json-unsafe-integers.js";
|
||||
import { resolveProviderEndpoint } from "./provider-attribution.js";
|
||||
import { unwrapModelHeaderSentinelsForProviderEgress } from "./provider-secret-egress.js";
|
||||
import { buildGuardedModelFetch } from "./provider-transport-fetch.js";
|
||||
import type { StreamFn } from "./runtime/index.js";
|
||||
import { transformTransportMessages } from "./transport-message-transform.js";
|
||||
@@ -288,7 +290,9 @@ function adjustMaxTokensForThinking(params: {
|
||||
}
|
||||
|
||||
function isAnthropicOAuthToken(apiKey: string): boolean {
|
||||
return apiKey.includes("sk-ant-oat");
|
||||
// Auth routing may inspect the real shape, but guarded fetch still receives the sentinel.
|
||||
const resolved = looksLikeSecretSentinel(apiKey) ? resolveSecretSentinel(apiKey) : apiKey;
|
||||
return (resolved ?? apiKey).includes("sk-ant-oat");
|
||||
}
|
||||
|
||||
function isDirectAnthropicModel(model: Pick<AnthropicTransportModel, "provider" | "baseUrl">) {
|
||||
@@ -910,7 +914,11 @@ function createAnthropicTransportClient(params: {
|
||||
isOAuthToken: false,
|
||||
};
|
||||
}
|
||||
if (usesFoundryBearerAuth(model)) {
|
||||
if (
|
||||
usesFoundryBearerAuth(
|
||||
unwrapModelHeaderSentinelsForProviderEgress(model, "Anthropic Foundry auth routing"),
|
||||
)
|
||||
) {
|
||||
const betaFeatures = needsInterleavedBeta ? ["interleaved-thinking-2025-05-14"] : [];
|
||||
return {
|
||||
client: createAnthropicMessagesClient({
|
||||
|
||||
@@ -73,6 +73,7 @@ vi.mock("./embedded-agent-runner/model.js", () => ({
|
||||
}));
|
||||
|
||||
vi.mock("./model-auth.js", () => ({
|
||||
applySecretRefHeaderSentinels: (model: unknown) => model,
|
||||
ensureAuthProfileStore: (...args: unknown[]) => ensureAuthProfileStoreMock(...args),
|
||||
ensureAuthProfileStoreWithoutExternalProfiles: (...args: unknown[]) =>
|
||||
ensureAuthProfileStoreWithoutExternalProfilesMock(...args),
|
||||
|
||||
+24
-13
@@ -44,6 +44,7 @@ import {
|
||||
import {
|
||||
ensureAuthProfileStore,
|
||||
ensureAuthProfileStoreWithoutExternalProfiles,
|
||||
applySecretRefHeaderSentinels,
|
||||
getApiKeyForModel,
|
||||
requireApiKey,
|
||||
} from "./model-auth.js";
|
||||
@@ -54,6 +55,10 @@ import {
|
||||
import { ensureOpenClawModelsJson } from "./models-config.js";
|
||||
import { listOpenAIAuthProfileProvidersForAgentRuntime } from "./openai-routing.js";
|
||||
import { applyPreparedRuntimeAuthToModel } from "./provider-request-config.js";
|
||||
import {
|
||||
protectPreparedProviderRuntimeAuth,
|
||||
unwrapSecretSentinelsForProviderEgress,
|
||||
} from "./provider-secret-egress.js";
|
||||
import { registerProviderStreamForModel } from "./provider-stream.js";
|
||||
import { stripToolResultDetails } from "./session-transcript-repair.js";
|
||||
import { resolveAgentTimeoutMs } from "./timeout.js";
|
||||
@@ -721,6 +726,7 @@ export async function runBtwSideQuestion(
|
||||
profileId: effectiveAuthProfileId,
|
||||
...(authStore ? { store: authStore } : {}),
|
||||
agentDir: params.agentDir,
|
||||
secretSentinels: true,
|
||||
});
|
||||
const resolvedAuthProfileId = apiKeyInfo.profileId ?? effectiveAuthProfileId;
|
||||
let runtimeModel = model;
|
||||
@@ -729,29 +735,34 @@ export async function runBtwSideQuestion(
|
||||
? undefined
|
||||
: requireApiKey(apiKeyInfo, model.provider);
|
||||
if (apiKey) {
|
||||
const preparedAuth = await prepareProviderRuntimeAuth({
|
||||
const preparedAuth = protectPreparedProviderRuntimeAuth({
|
||||
sourceApiKey: apiKey,
|
||||
provider: model.provider,
|
||||
config: params.cfg,
|
||||
workspaceDir,
|
||||
env: process.env,
|
||||
context: {
|
||||
preparedAuth: await prepareProviderRuntimeAuth({
|
||||
provider: model.provider,
|
||||
config: params.cfg,
|
||||
agentDir: params.agentDir,
|
||||
workspaceDir,
|
||||
env: process.env,
|
||||
provider: model.provider,
|
||||
modelId: model.id,
|
||||
model,
|
||||
apiKey,
|
||||
authMode: apiKeyInfo.mode,
|
||||
profileId: resolvedAuthProfileId,
|
||||
},
|
||||
context: {
|
||||
config: params.cfg,
|
||||
agentDir: params.agentDir,
|
||||
workspaceDir,
|
||||
env: process.env,
|
||||
provider: model.provider,
|
||||
modelId: model.id,
|
||||
model,
|
||||
apiKey: unwrapSecretSentinelsForProviderEgress(apiKey, "provider runtime auth exchange"),
|
||||
authMode: apiKeyInfo.mode,
|
||||
profileId: resolvedAuthProfileId,
|
||||
},
|
||||
}),
|
||||
});
|
||||
runtimeModel = applyPreparedRuntimeAuthToModel(runtimeModel, preparedAuth);
|
||||
if (preparedAuth?.apiKey) {
|
||||
apiKey = preparedAuth.apiKey;
|
||||
}
|
||||
}
|
||||
runtimeModel = applySecretRefHeaderSentinels(runtimeModel, params.cfg);
|
||||
|
||||
// Use the provider's own stream fn so providers like Ollama (which build
|
||||
// `/api/chat` or `/v1/chat/completions` paths based on api mode) construct
|
||||
|
||||
@@ -106,6 +106,10 @@ import { ensureOpenClawModelsJson } from "../models-config.js";
|
||||
import { wrapStreamFnTextTransforms } from "../plugin-text-transforms.js";
|
||||
import { resolveAgentPromptSurfaceForSessionKey } from "../prompt-surface.js";
|
||||
import { applyPreparedRuntimeAuthToModel } from "../provider-request-config.js";
|
||||
import {
|
||||
protectPreparedProviderRuntimeAuth,
|
||||
unwrapSecretSentinelsForProviderEgress,
|
||||
} from "../provider-secret-egress.js";
|
||||
import { registerProviderStreamForModel } from "../provider-stream.js";
|
||||
import {
|
||||
applyAgentRunSessionTargetIdentity,
|
||||
@@ -687,6 +691,7 @@ async function compactEmbeddedAgentSessionDirectOnce(
|
||||
profileId: authProfileId,
|
||||
agentDir,
|
||||
workspaceDir: resolvedWorkspace,
|
||||
secretSentinels: true,
|
||||
});
|
||||
|
||||
if (!apiKeyInfo.apiKey) {
|
||||
@@ -694,23 +699,30 @@ async function compactEmbeddedAgentSessionDirectOnce(
|
||||
throw new MissingProviderAuthError(runtimeModel.provider, apiKeyInfo);
|
||||
}
|
||||
} else {
|
||||
const preparedAuth = await prepareProviderRuntimeAuth({
|
||||
const preparedAuth = protectPreparedProviderRuntimeAuth({
|
||||
sourceApiKey: apiKeyInfo.apiKey,
|
||||
provider: runtimeModel.provider,
|
||||
config: params.config,
|
||||
workspaceDir: resolvedWorkspace,
|
||||
env: process.env,
|
||||
context: {
|
||||
preparedAuth: await prepareProviderRuntimeAuth({
|
||||
provider: runtimeModel.provider,
|
||||
config: params.config,
|
||||
agentDir,
|
||||
workspaceDir: resolvedWorkspace,
|
||||
env: process.env,
|
||||
provider: runtimeModel.provider,
|
||||
modelId,
|
||||
model: runtimeModel,
|
||||
apiKey: apiKeyInfo.apiKey,
|
||||
authMode: apiKeyInfo.mode,
|
||||
profileId: apiKeyInfo.profileId,
|
||||
},
|
||||
context: {
|
||||
config: params.config,
|
||||
agentDir,
|
||||
workspaceDir: resolvedWorkspace,
|
||||
env: process.env,
|
||||
provider: runtimeModel.provider,
|
||||
modelId,
|
||||
model: runtimeModel,
|
||||
apiKey: unwrapSecretSentinelsForProviderEgress(
|
||||
apiKeyInfo.apiKey,
|
||||
"provider runtime auth exchange",
|
||||
),
|
||||
authMode: apiKeyInfo.mode,
|
||||
profileId: apiKeyInfo.profileId,
|
||||
},
|
||||
}),
|
||||
});
|
||||
runtimeModel = applyPreparedRuntimeAuthToModel(runtimeModel, preparedAuth);
|
||||
const runtimeApiKey = preparedAuth?.apiKey ?? apiKeyInfo.apiKey;
|
||||
|
||||
@@ -3,6 +3,8 @@ import crypto from "node:crypto";
|
||||
import type { StreamFn } from "openclaw/plugin-sdk/agent-core";
|
||||
import type { Model } from "openclaw/plugin-sdk/llm";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { isSecretValueRegisteredForRedaction } from "../../logging/secret-redaction-registry.js";
|
||||
import { mintSecretSentinel, resolveSecretSentinel } from "../../secrets/sentinel.js";
|
||||
import { prepareGooglePromptCacheStreamFn } from "./google-prompt-cache.js";
|
||||
import { EmbeddedAttemptSessionTakeoverError } from "./run/attempt.session-lock.js";
|
||||
|
||||
@@ -149,6 +151,7 @@ function streamOptions(streamFn: { mock: { calls: unknown[][] } }, callIndex = 0
|
||||
}
|
||||
|
||||
function preparePromptCacheStream(params: {
|
||||
apiKey?: string;
|
||||
fetchMock: ReturnType<typeof vi.fn>;
|
||||
now: number;
|
||||
sessionManager: TestGooglePromptCacheSessionManager;
|
||||
@@ -158,7 +161,7 @@ function preparePromptCacheStream(params: {
|
||||
// tests can focus on cache lifecycle behavior.
|
||||
return prepareGooglePromptCacheStreamFn(
|
||||
{
|
||||
apiKey: "gemini-api-key",
|
||||
apiKey: params.apiKey ?? "gemini-api-key",
|
||||
extraParams: { cacheRetention: "long" },
|
||||
model: makeGoogleModel(),
|
||||
modelId: "gemini-3.1-pro-preview",
|
||||
@@ -175,6 +178,70 @@ function preparePromptCacheStream(params: {
|
||||
}
|
||||
|
||||
describe("google prompt cache", () => {
|
||||
it("parses sentinel-backed OAuth JSON before guarded cache egress", async () => {
|
||||
const fetchMock = createCacheFetchMock({
|
||||
name: "cachedContents/oauth-cache",
|
||||
expireTime: new Date(2_000_000).toISOString(),
|
||||
});
|
||||
const { streamFn } = createCapturingStreamFn();
|
||||
const oauthJson = JSON.stringify({ token: "google-oauth-token", projectId: "demo" });
|
||||
const sentinel = mintSecretSentinel(oauthJson, { label: "model-auth:google" });
|
||||
const wrapped = await preparePromptCacheStream({
|
||||
apiKey: sentinel,
|
||||
fetchMock,
|
||||
now: 1_000_000,
|
||||
sessionManager: makeSessionManager([]),
|
||||
streamFn,
|
||||
});
|
||||
|
||||
await Promise.resolve(
|
||||
wrapped?.(
|
||||
makeGoogleModel(),
|
||||
{ systemPrompt: "Follow policy.", messages: [] } as never,
|
||||
{} as never,
|
||||
),
|
||||
);
|
||||
|
||||
const headers = fetchInit(fetchMock).headers as Record<string, string>;
|
||||
expect(resolveSecretSentinel(headers.Authorization)).toBe("Bearer google-oauth-token");
|
||||
expect(headers["x-goog-api-key"]).toBeUndefined();
|
||||
expect(headers["Content-Type"]).toBe("application/json");
|
||||
});
|
||||
|
||||
it("registers parsed OAuth headers when sentinels are disabled", async () => {
|
||||
vi.stubEnv("OPENCLAW_SECRET_SENTINELS", "off");
|
||||
const fetchMock = createCacheFetchMock({
|
||||
name: "cachedContents/oauth-cache",
|
||||
expireTime: new Date(2_000_000).toISOString(),
|
||||
});
|
||||
const { streamFn } = createCapturingStreamFn();
|
||||
const oauthJson = JSON.stringify({ token: "google-kill-switch-token", projectId: "demo" });
|
||||
const apiKey = mintSecretSentinel(oauthJson, { label: "model-auth:google" });
|
||||
|
||||
try {
|
||||
const wrapped = await preparePromptCacheStream({
|
||||
apiKey,
|
||||
fetchMock,
|
||||
now: 1_000_000,
|
||||
sessionManager: makeSessionManager([]),
|
||||
streamFn,
|
||||
});
|
||||
await Promise.resolve(
|
||||
wrapped?.(
|
||||
makeGoogleModel(),
|
||||
{ systemPrompt: "Follow policy.", messages: [] } as never,
|
||||
{} as never,
|
||||
),
|
||||
);
|
||||
|
||||
const headers = fetchInit(fetchMock).headers as Record<string, string>;
|
||||
expect(headers.Authorization).toBe("Bearer google-kill-switch-token");
|
||||
expect(isSecretValueRegisteredForRedaction(headers.Authorization)).toBe(true);
|
||||
} finally {
|
||||
vi.unstubAllEnvs();
|
||||
}
|
||||
});
|
||||
|
||||
it("creates cached content from the system prompt and strips that prompt from live requests", async () => {
|
||||
// Cached system prompts should move out of live request context and into the
|
||||
// cachedContent option to avoid paying prompt tokens repeatedly.
|
||||
|
||||
@@ -14,6 +14,12 @@ import { normalizeGoogleApiBaseUrl } from "../../infra/google-api-base-url.js";
|
||||
import { readResponseWithLimit } from "../../infra/http-body.js";
|
||||
import { streamWithPayloadPatch } from "../../llm/providers/stream-wrappers/stream-payload-utils.js";
|
||||
import type { Model } from "../../llm/types.js";
|
||||
import { isSecretValueRegisteredForRedaction } from "../../logging/secret-redaction-registry.js";
|
||||
import {
|
||||
looksLikeSecretSentinel,
|
||||
mintSecretSentinel,
|
||||
resolveSecretSentinel,
|
||||
} from "../../secrets/sentinel.js";
|
||||
import { resolveProviderRequestHeaders } from "../provider-request-config.js";
|
||||
import { buildGuardedModelFetch } from "../provider-transport-fetch.js";
|
||||
import type { StreamFn } from "../runtime/index.js";
|
||||
@@ -296,13 +302,54 @@ async function readGooglePromptCacheJson<T>(response: Response): Promise<T> {
|
||||
return JSON.parse(buffer.toString("utf8")) as T;
|
||||
}
|
||||
|
||||
function resolveGooglePromptCacheAuthHeaders(params: {
|
||||
apiKey: string;
|
||||
provider: string;
|
||||
}): Record<string, string> {
|
||||
if (!looksLikeSecretSentinel(params.apiKey)) {
|
||||
const headers = parseGeminiAuth(params.apiKey).headers;
|
||||
if (!isSecretValueRegisteredForRedaction(params.apiKey)) {
|
||||
return headers;
|
||||
}
|
||||
return Object.fromEntries(
|
||||
Object.entries(headers).map(([name, value]) => [
|
||||
name,
|
||||
name.toLowerCase() === "authorization" || name.toLowerCase() === "x-goog-api-key"
|
||||
? mintSecretSentinel(value, { label: `model-auth:${params.provider}` })
|
||||
: value,
|
||||
]),
|
||||
);
|
||||
}
|
||||
const resolved = resolveSecretSentinel(params.apiKey);
|
||||
if (resolved === undefined) {
|
||||
throw new Error(
|
||||
`Secret sentinel ${params.apiKey} is not registered in this process; refusing Google prompt-cache auth`,
|
||||
);
|
||||
}
|
||||
return Object.fromEntries(
|
||||
Object.entries(parseGeminiAuth(resolved).headers).map(([name, value]) => {
|
||||
const isCredentialHeader =
|
||||
name.toLowerCase() === "authorization" || name.toLowerCase() === "x-goog-api-key";
|
||||
return [
|
||||
name,
|
||||
isCredentialHeader
|
||||
? mintSecretSentinel(value, { label: `model-auth:${params.provider}` })
|
||||
: value,
|
||||
];
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
function buildGooglePromptCacheHeaders(params: {
|
||||
apiKey: string;
|
||||
baseUrl: string;
|
||||
headers?: Record<string, string>;
|
||||
model: GooglePromptCacheModel;
|
||||
}): Record<string, string> | undefined {
|
||||
const authHeaders = parseGeminiAuth(params.apiKey).headers;
|
||||
const authHeaders = resolveGooglePromptCacheAuthHeaders({
|
||||
apiKey: params.apiKey,
|
||||
provider: params.model.provider,
|
||||
});
|
||||
return (
|
||||
resolveProviderRequestHeaders({
|
||||
provider: params.model.provider,
|
||||
|
||||
@@ -1,6 +1,12 @@
|
||||
// Coverage for embedded run auth initialization and runtime credential refresh.
|
||||
import type { Model } from "openclaw/plugin-sdk/llm";
|
||||
import { beforeEach, describe, expect, it, vi, type Mock } from "vitest";
|
||||
import { isSecretValueRegisteredForRedaction } from "../../../logging/secret-redaction-registry.js";
|
||||
import {
|
||||
looksLikeSecretSentinel,
|
||||
mintSecretSentinel,
|
||||
resolveSecretSentinel,
|
||||
} from "../../../secrets/sentinel.js";
|
||||
import type { AuthProfileStore } from "../../auth-profiles.js";
|
||||
import { FailoverError } from "../../failover-error.js";
|
||||
import type { RuntimeAuthState } from "./helpers.js";
|
||||
@@ -209,6 +215,87 @@ describe("createEmbeddedRunAuthController", () => {
|
||||
expect(harness.runtimeAuthState?.profileId).toBe("default");
|
||||
});
|
||||
|
||||
it("unwraps a sentinel for runtime auth exchange but keeps auth storage opaque", async () => {
|
||||
const harness = createMutableAuthControllerHarness();
|
||||
const setRuntimeApiKey = vi.fn<(provider: string, apiKey: string) => void>();
|
||||
const secret = "runtime-exchange-source-secret";
|
||||
const sentinel = mintSecretSentinel(secret, { label: "model-auth:custom-openai" });
|
||||
mocks.getApiKeyForModel.mockResolvedValue({
|
||||
apiKey: sentinel,
|
||||
mode: "api-key",
|
||||
source: "profile:custom-openai:default",
|
||||
});
|
||||
mocks.prepareProviderRuntimeAuth.mockResolvedValue({
|
||||
apiKey: "runtime-exchange-token",
|
||||
request: {
|
||||
auth: {
|
||||
mode: "header",
|
||||
headerName: "api-key",
|
||||
value: "runtime-header-token",
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
const controller = createMutableEmbeddedRunAuthController({ harness, setRuntimeApiKey });
|
||||
await controller.initializeAuthProfile();
|
||||
|
||||
expect(mocks.getApiKeyForModel).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ secretSentinels: true }),
|
||||
);
|
||||
expect(mocks.prepareProviderRuntimeAuth).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ context: expect.objectContaining({ apiKey: secret }) }),
|
||||
);
|
||||
const storedApiKey = setRuntimeApiKey.mock.calls[0]?.[1];
|
||||
expect(storedApiKey && looksLikeSecretSentinel(storedApiKey)).toBe(true);
|
||||
expect(storedApiKey && resolveSecretSentinel(storedApiKey)).toBe("runtime-exchange-token");
|
||||
const storedHeader = harness.runtimeModel.headers?.["api-key"];
|
||||
expect(storedHeader && looksLikeSecretSentinel(storedHeader)).toBe(true);
|
||||
expect(storedHeader && resolveSecretSentinel(storedHeader)).toBe("runtime-header-token");
|
||||
});
|
||||
|
||||
it("preserves an empty runtime-auth result for fallback validation", async () => {
|
||||
const harness = createMutableAuthControllerHarness();
|
||||
const setRuntimeApiKey = vi.fn<(provider: string, apiKey: string) => void>();
|
||||
const sentinel = mintSecretSentinel("runtime-source-secret", {
|
||||
label: "model-auth:custom-openai",
|
||||
});
|
||||
mocks.getApiKeyForModel.mockResolvedValue({
|
||||
apiKey: sentinel,
|
||||
mode: "api-key",
|
||||
source: "profile:custom-openai:default",
|
||||
});
|
||||
mocks.prepareProviderRuntimeAuth.mockResolvedValue({ apiKey: "" });
|
||||
|
||||
const controller = createMutableEmbeddedRunAuthController({ harness, setRuntimeApiKey });
|
||||
await controller.initializeAuthProfile();
|
||||
|
||||
expect(setRuntimeApiKey).toHaveBeenCalledWith("custom-openai", sentinel);
|
||||
});
|
||||
|
||||
it("registers exchanged credentials when sentinels are disabled", async () => {
|
||||
vi.stubEnv("OPENCLAW_SECRET_SENTINELS", "off");
|
||||
const harness = createMutableAuthControllerHarness();
|
||||
const setRuntimeApiKey = vi.fn<(provider: string, apiKey: string) => void>();
|
||||
const source = mintSecretSentinel("kill-switch-source-secret", {
|
||||
label: "model-auth:custom-openai",
|
||||
});
|
||||
mocks.getApiKeyForModel.mockResolvedValue({
|
||||
apiKey: source,
|
||||
mode: "api-key",
|
||||
source: "profile:custom-openai:default",
|
||||
});
|
||||
mocks.prepareProviderRuntimeAuth.mockResolvedValue({ apiKey: "kill-switch-runtime-token" });
|
||||
|
||||
try {
|
||||
const controller = createMutableEmbeddedRunAuthController({ harness, setRuntimeApiKey });
|
||||
await controller.initializeAuthProfile();
|
||||
expect(setRuntimeApiKey).toHaveBeenCalledWith("custom-openai", "kill-switch-runtime-token");
|
||||
expect(isSecretValueRegisteredForRedaction("kill-switch-runtime-token")).toBe(true);
|
||||
} finally {
|
||||
vi.unstubAllEnvs();
|
||||
}
|
||||
});
|
||||
|
||||
it("includes the checked credential source when an api key is missing", async () => {
|
||||
const harness = createMutableAuthControllerHarness();
|
||||
const setRuntimeApiKey = vi.fn<(provider: string, apiKey: string) => void>();
|
||||
|
||||
@@ -28,6 +28,10 @@ import {
|
||||
applyPreparedRuntimeAuthToModel,
|
||||
type ModelProviderRequestTransportOverrides,
|
||||
} from "../../provider-request-config.js";
|
||||
import {
|
||||
protectPreparedProviderRuntimeAuth,
|
||||
unwrapSecretSentinelsForProviderEgress,
|
||||
} from "../../provider-secret-egress.js";
|
||||
import { clampRuntimeAuthRefreshDelayMs } from "../../runtime-auth-refresh.js";
|
||||
import {
|
||||
RUNTIME_AUTH_REFRESH_MARGIN_MS,
|
||||
@@ -117,8 +121,8 @@ export function createEmbeddedRunAuthController(params: {
|
||||
apiKey: string;
|
||||
authMode: string;
|
||||
profileId?: string;
|
||||
}) =>
|
||||
prepareProviderRuntimeAuth({
|
||||
}) => {
|
||||
const preparedAuth = await prepareProviderRuntimeAuth({
|
||||
provider: prepareParams.runtimeModel.provider,
|
||||
config: params.config,
|
||||
workspaceDir: params.workspaceDir,
|
||||
@@ -131,11 +135,20 @@ export function createEmbeddedRunAuthController(params: {
|
||||
provider: prepareParams.runtimeModel.provider,
|
||||
modelId: params.getModelId(),
|
||||
model: prepareParams.runtimeModel,
|
||||
apiKey: prepareParams.apiKey,
|
||||
apiKey: unwrapSecretSentinelsForProviderEgress(
|
||||
prepareParams.apiKey,
|
||||
"provider runtime auth exchange",
|
||||
),
|
||||
authMode: prepareParams.authMode,
|
||||
profileId: prepareParams.profileId,
|
||||
},
|
||||
});
|
||||
return protectPreparedProviderRuntimeAuth({
|
||||
sourceApiKey: prepareParams.apiKey,
|
||||
provider: prepareParams.runtimeModel.provider,
|
||||
preparedAuth,
|
||||
});
|
||||
};
|
||||
|
||||
const clearRuntimeAuthRefreshTimer = () => {
|
||||
const runtimeAuthState = params.getRuntimeAuthState();
|
||||
@@ -373,6 +386,7 @@ export function createEmbeddedRunAuthController(params: {
|
||||
agentDir: params.agentDir,
|
||||
workspaceDir: params.workspaceDir,
|
||||
lockedProfile: candidate != null && candidate === params.lockedProfileId,
|
||||
secretSentinels: true,
|
||||
});
|
||||
};
|
||||
|
||||
|
||||
@@ -5,6 +5,7 @@ import { SYSTEM_PROMPT_CACHE_BOUNDARY } from "@openclaw/ai/internal/shared";
|
||||
import type { StreamFn } from "openclaw/plugin-sdk/agent-core";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { streamSimple } from "../../llm/stream.js";
|
||||
import { mintSecretSentinel } from "../../secrets/sentinel.js";
|
||||
import * as providerTransportStream from "../provider-transport-stream.js";
|
||||
import {
|
||||
testing,
|
||||
@@ -124,6 +125,39 @@ describe("describeEmbeddedAgentStreamStrategy", () => {
|
||||
});
|
||||
|
||||
describe("resolveEmbeddedAgentStreamFn", () => {
|
||||
it("preserves sentinels for registered provider streams", async () => {
|
||||
const secret = "plugin-stream-secret";
|
||||
const sentinel = mintSecretSentinel(secret, { label: "model-auth:plugin" });
|
||||
const providerStreamFn = vi.fn(async (model, _context, options) => ({ model, options }));
|
||||
const model = {
|
||||
api: "plugin-api",
|
||||
provider: "plugin",
|
||||
id: "plugin-model",
|
||||
headers: { Authorization: `Bearer ${sentinel}` },
|
||||
} as never;
|
||||
const streamFn = resolveEmbeddedAgentStreamFn({
|
||||
currentStreamFn: undefined,
|
||||
providerStreamFn: providerStreamFn as never,
|
||||
sessionId: "session-1",
|
||||
model,
|
||||
resolvedApiKey: sentinel,
|
||||
});
|
||||
|
||||
const result = await expectStreamResultRecord(
|
||||
streamFn(model, {} as never, {
|
||||
headers: { "X-Managed": `Bearer ${sentinel}` },
|
||||
}),
|
||||
"plugin stream result",
|
||||
);
|
||||
expect(requireRecord(result.model, "plugin model").headers).toEqual({
|
||||
Authorization: `Bearer ${sentinel}`,
|
||||
});
|
||||
expect(requireRecord(result.options, "plugin options").apiKey).toBe(sentinel);
|
||||
expect(requireRecord(result.options, "plugin options").headers).toEqual({
|
||||
"X-Managed": `Bearer ${sentinel}`,
|
||||
});
|
||||
});
|
||||
|
||||
it("prefers the resolved run api key over a later authStorage lookup", async () => {
|
||||
const authStorage = {
|
||||
getApiKey: vi.fn(async () => "storage-key"),
|
||||
|
||||
@@ -271,9 +271,10 @@ function wrapEmbeddedAgentStreamFn(
|
||||
resolvedApiKey,
|
||||
authStorage,
|
||||
});
|
||||
const selectedApiKey = apiKey ?? options?.apiKey;
|
||||
return inner(m, transformContext(context), {
|
||||
...mergeRunSignal(options),
|
||||
apiKey: apiKey ?? options?.apiKey,
|
||||
apiKey: selectedApiKey,
|
||||
});
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import type { Model } from "openclaw/plugin-sdk/llm";
|
||||
/**
|
||||
* Routes compaction through selected native agent harnesses when supported.
|
||||
*/
|
||||
@@ -5,14 +6,17 @@ import { formatErrorMessage } from "../../infra/errors.js";
|
||||
import { createSubsystemLogger } from "../../logging/subsystem.js";
|
||||
import { parseAgentSessionKey } from "../../routing/session-key.js";
|
||||
import { resolveUserPath } from "../../utils.js";
|
||||
import type { Model } from "openclaw/plugin-sdk/llm";
|
||||
import { isDefaultAgentRuntimeId, normalizeOptionalAgentRuntimeId } from "../agent-runtime-id.js";
|
||||
import { resolveAgentDir, resolveSessionAgentIds } from "../agent-scope.js";
|
||||
import type { CompactEmbeddedAgentSessionParams } from "../embedded-agent-runner/compact.types.js";
|
||||
import { resolveModelAsync } from "../embedded-agent-runner/model.js";
|
||||
import type { EmbeddedAgentCompactResult } from "../embedded-agent-runner/types.js";
|
||||
import { getApiKeyForModel } from "../model-auth.js";
|
||||
import { applySecretRefHeaderSentinels, getApiKeyForModel } from "../model-auth.js";
|
||||
import { isCliRuntimeAliasForProvider, isCliRuntimeProvider } from "../model-runtime-aliases.js";
|
||||
import {
|
||||
unwrapModelHeaderSentinelsForProviderEgress,
|
||||
unwrapSecretSentinelsForProviderEgress,
|
||||
} from "../provider-secret-egress.js";
|
||||
import { resolveAgentHarnessPolicy as resolveConfiguredAgentHarnessPolicy } from "./policy.js";
|
||||
import { selectAgentHarness } from "./selection.js";
|
||||
import type {
|
||||
@@ -84,26 +88,28 @@ async function resolveHarnessCompactApiKey(params: {
|
||||
if (!model) {
|
||||
return existing ? { apiKey: existing } : {};
|
||||
}
|
||||
const runtimeModel = applySecretRefHeaderSentinels(model, compactParams.config);
|
||||
if (existing) {
|
||||
return { apiKey: existing, runtimeModel: model };
|
||||
return { apiKey: existing, runtimeModel };
|
||||
}
|
||||
try {
|
||||
const apiKeyInfo = await getApiKeyForModel({
|
||||
model,
|
||||
model: runtimeModel,
|
||||
cfg: compactParams.config,
|
||||
profileId: authProfileId,
|
||||
agentDir,
|
||||
workspaceDir,
|
||||
secretSentinels: true,
|
||||
});
|
||||
return {
|
||||
apiKey: apiKeyInfo.apiKey?.trim() || undefined,
|
||||
runtimeModel: model,
|
||||
runtimeModel,
|
||||
};
|
||||
} catch (err) {
|
||||
log.debug("agent harness compaction credential lookup failed", {
|
||||
error: formatErrorMessage(err),
|
||||
});
|
||||
return { runtimeModel: model };
|
||||
return { runtimeModel };
|
||||
}
|
||||
}
|
||||
|
||||
@@ -195,8 +201,22 @@ export async function maybeCompactAgentHarnessSession(
|
||||
resolvedApiKey || runtimeModel
|
||||
? {
|
||||
...compactParams,
|
||||
...(resolvedApiKey ? { resolvedApiKey } : {}),
|
||||
...(runtimeModel ? { runtimeModel } : {}),
|
||||
...(resolvedApiKey
|
||||
? {
|
||||
resolvedApiKey: unwrapSecretSentinelsForProviderEgress(
|
||||
resolvedApiKey,
|
||||
"plugin harness compaction handoff",
|
||||
),
|
||||
}
|
||||
: {}),
|
||||
...(runtimeModel
|
||||
? {
|
||||
runtimeModel: unwrapModelHeaderSentinelsForProviderEgress(
|
||||
runtimeModel,
|
||||
"plugin harness compaction handoff",
|
||||
),
|
||||
}
|
||||
: {}),
|
||||
}
|
||||
: compactParams;
|
||||
if (shouldCompactAfterContextEngine) {
|
||||
|
||||
@@ -4,6 +4,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import type { OpenClawConfig } from "../../config/config.js";
|
||||
import { OPENCLAW_EMBEDDED_CONTEXT_ENGINE_HOST } from "../../context-engine/host-compat.js";
|
||||
import type { ContextEngine } from "../../context-engine/types.js";
|
||||
import { mintSecretSentinel } from "../../secrets/sentinel.js";
|
||||
import { testing as cliBackendsTesting } from "../cli-backends.js";
|
||||
import type {
|
||||
EmbeddedRunAttemptParams,
|
||||
@@ -58,6 +59,7 @@ vi.mock("./builtin-openclaw.js", () => ({
|
||||
}),
|
||||
}));
|
||||
vi.mock("../model-auth.js", () => ({
|
||||
applySecretRefHeaderSentinels: (model: unknown) => model,
|
||||
getApiKeyForModel: compactAuthMocks.getApiKeyForModel,
|
||||
}));
|
||||
vi.mock("../embedded-agent-runner/model.js", () => ({
|
||||
@@ -290,6 +292,37 @@ function agentModelRuntimeConfig(
|
||||
}
|
||||
|
||||
describe("runAgentHarnessAttempt", () => {
|
||||
it("unwraps sentinels only at the plugin harness handoff", async () => {
|
||||
const pluginRunAttempt = vi.fn<AgentHarness["runAttempt"]>(async () =>
|
||||
createAttemptResult("codex"),
|
||||
);
|
||||
registerAgentHarness(
|
||||
{
|
||||
id: "codex",
|
||||
label: "Codex",
|
||||
supports: () => ({ supported: true, priority: 100 }),
|
||||
runAttempt: pluginRunAttempt,
|
||||
},
|
||||
{ ownerPluginId: "codex" },
|
||||
);
|
||||
const secret = "plugin-provider-secret";
|
||||
const sentinel = mintSecretSentinel(secret, { label: "model-auth:codex" });
|
||||
const params = createAttemptParams(providerRuntimeConfig("codex", "codex"));
|
||||
params.resolvedApiKey = sentinel;
|
||||
params.model = {
|
||||
...params.model,
|
||||
headers: { Authorization: `Bearer ${sentinel}`, "X-Optional": null } as never,
|
||||
};
|
||||
|
||||
await runAgentHarnessAttempt(params);
|
||||
|
||||
const handedOff = pluginRunAttempt.mock.calls[0]?.[0];
|
||||
expect(handedOff?.resolvedApiKey).toBe(secret);
|
||||
expect(handedOff?.model.headers?.Authorization).toBe(`Bearer ${secret}`);
|
||||
expect(handedOff?.model.headers?.["X-Optional"]).toBeNull();
|
||||
expect(params.resolvedApiKey).toBe(sentinel);
|
||||
});
|
||||
|
||||
it("fails when a forced plugin harness is unavailable and fallback is omitted", async () => {
|
||||
process.env.OPENCLAW_AGENT_RUNTIME = "codex";
|
||||
|
||||
|
||||
@@ -21,6 +21,10 @@ import type {
|
||||
EmbeddedRunAttemptResult,
|
||||
} from "../embedded-agent-runner/run/types.js";
|
||||
import { isCliRuntimeAliasForProvider } from "../model-runtime-aliases.js";
|
||||
import {
|
||||
unwrapModelHeaderSentinelsForProviderEgress,
|
||||
unwrapSecretSentinelsForProviderEgress,
|
||||
} from "../provider-secret-egress.js";
|
||||
import { resolveSandboxRuntimeStatus } from "../sandbox/runtime-status.js";
|
||||
import { expandToolGroups, mergeAlsoAllowPolicy, normalizeToolName } from "../tool-policy.js";
|
||||
import { createOpenClawAgentHarness } from "./builtin-openclaw.js";
|
||||
@@ -372,8 +376,7 @@ export async function runAgentHarnessAttempt(
|
||||
agentHarnessRuntimeOverride: params.agentHarnessRuntimeOverride,
|
||||
});
|
||||
const harness = selection.harness;
|
||||
const attemptParams =
|
||||
harness.id === "openclaw" ? params : applyPluginHarnessDenyAllToolPolicy(params);
|
||||
const attemptParams = harness.id === "openclaw" ? params : preparePluginHarnessParams(params);
|
||||
logAgentHarnessSelection(selection, {
|
||||
provider: params.provider,
|
||||
modelId: params.modelId,
|
||||
@@ -398,6 +401,22 @@ export async function runAgentHarnessAttempt(
|
||||
}
|
||||
}
|
||||
|
||||
function preparePluginHarnessParams(params: EmbeddedRunAttemptParams): EmbeddedRunAttemptParams {
|
||||
const boundary = "plugin harness handoff";
|
||||
const resolvedApiKey = params.resolvedApiKey
|
||||
? unwrapSecretSentinelsForProviderEgress(params.resolvedApiKey, boundary)
|
||||
: params.resolvedApiKey;
|
||||
const model = unwrapModelHeaderSentinelsForProviderEgress(params.model, boundary);
|
||||
if (model === params.model && resolvedApiKey === params.resolvedApiKey) {
|
||||
return applyPluginHarnessDenyAllToolPolicy(params);
|
||||
}
|
||||
return applyPluginHarnessDenyAllToolPolicy({
|
||||
...params,
|
||||
model,
|
||||
resolvedApiKey,
|
||||
});
|
||||
}
|
||||
|
||||
function applyPluginHarnessDenyAllToolPolicy(
|
||||
params: EmbeddedRunAttemptParams,
|
||||
): EmbeddedRunAttemptParams {
|
||||
|
||||
@@ -66,6 +66,7 @@ describe("model auth markers", () => {
|
||||
it("recognizes explicit non-secret markers", () => {
|
||||
withEnv(cleanPluginManifestEnv(), () => {
|
||||
expect(isNonSecretApiKeyMarker(NON_ENV_SECRETREF_MARKER)).toBe(true);
|
||||
expect(isNonSecretApiKeyMarker("secretref-env:OPENAI_API_KEY")).toBe(true);
|
||||
expect(isNonSecretApiKeyMarker(resolveOAuthApiKeyMarker("chutes"))).toBe(true);
|
||||
expect(isNonSecretApiKeyMarker("ollama-local")).toBe(true);
|
||||
expect(isNonSecretApiKeyMarker("lmstudio-local")).toBe(true);
|
||||
|
||||
@@ -133,7 +133,8 @@ export function isNonSecretApiKeyMarker(
|
||||
const isKnownMarker =
|
||||
isOAuthApiKeyMarker(trimmed) ||
|
||||
listKnownNonSecretApiKeyMarkers().includes(trimmed) ||
|
||||
isAwsSdkAuthMarker(trimmed);
|
||||
isAwsSdkAuthMarker(trimmed) ||
|
||||
isSecretRefHeaderValueMarker(trimmed);
|
||||
if (isKnownMarker) {
|
||||
return true;
|
||||
}
|
||||
|
||||
+334
-11
@@ -9,6 +9,10 @@ import {
|
||||
GCP_VERTEX_CREDENTIALS_MARKER,
|
||||
NON_ENV_SECRETREF_MARKER,
|
||||
} from "./model-auth-markers.js";
|
||||
import {
|
||||
attachModelProviderRequestTransport,
|
||||
getModelProviderRequestTransport,
|
||||
} from "./provider-request-config.js";
|
||||
|
||||
vi.mock("../plugins/plugin-registry.js", () => ({
|
||||
loadPluginRegistrySnapshotWithMetadata: () => ({
|
||||
@@ -146,6 +150,7 @@ vi.mock("../plugins/provider-runtime.js", async () => {
|
||||
|
||||
let applyAuthHeaderOverride: typeof import("./model-auth.js").applyAuthHeaderOverride;
|
||||
let applyLocalNoAuthHeaderOverride: typeof import("./model-auth.js").applyLocalNoAuthHeaderOverride;
|
||||
let applySecretRefHeaderSentinels: typeof import("./model-auth.js").applySecretRefHeaderSentinels;
|
||||
let createRuntimeProviderAuthLookup: typeof import("./model-auth.js").createRuntimeProviderAuthLookup;
|
||||
let formatMissingAuthError: typeof import("./model-auth.js").formatMissingAuthError;
|
||||
let hasAvailableAuthForProvider: typeof import("./model-auth.js").hasAvailableAuthForProvider;
|
||||
@@ -161,14 +166,18 @@ let resolveUsableCustomProviderApiKey: typeof import("./model-auth.js").resolveU
|
||||
let cliCredentials: typeof import("./cli-credentials.js");
|
||||
let clearRuntimeConfigSnapshot: typeof import("../config/config.js").clearRuntimeConfigSnapshot;
|
||||
let setRuntimeConfigSnapshot: typeof import("../config/config.js").setRuntimeConfigSnapshot;
|
||||
let looksLikeSecretSentinel: typeof import("../secrets/sentinel.js").looksLikeSecretSentinel;
|
||||
let resolveSecretSentinel: typeof import("../secrets/sentinel.js").resolveSecretSentinel;
|
||||
|
||||
beforeAll(async () => {
|
||||
vi.resetModules();
|
||||
({ clearRuntimeConfigSnapshot, setRuntimeConfigSnapshot } = await import("../config/config.js"));
|
||||
({ looksLikeSecretSentinel, resolveSecretSentinel } = await import("../secrets/sentinel.js"));
|
||||
cliCredentials = await import("./cli-credentials.js");
|
||||
({
|
||||
applyAuthHeaderOverride,
|
||||
applyLocalNoAuthHeaderOverride,
|
||||
applySecretRefHeaderSentinels,
|
||||
createRuntimeProviderAuthLookup,
|
||||
formatMissingAuthError,
|
||||
hasAvailableAuthForProvider,
|
||||
@@ -287,6 +296,21 @@ function expectAuthFields(
|
||||
}
|
||||
}
|
||||
|
||||
function expectSecretSentinelAuth(
|
||||
auth: Awaited<ReturnType<typeof resolveApiKeyForProvider>>,
|
||||
expected: { value: string; source: string; mode: "api-key" | "oauth" },
|
||||
) {
|
||||
const apiKey = auth.apiKey;
|
||||
expect(apiKey).toBeDefined();
|
||||
if (!apiKey) {
|
||||
throw new Error("expected model auth API key");
|
||||
}
|
||||
expect(looksLikeSecretSentinel(apiKey)).toBe(true);
|
||||
expect(resolveSecretSentinel(apiKey)).toBe(expected.value);
|
||||
expect(auth.source).toBe(expected.source);
|
||||
expect(auth.mode).toBe(expected.mode);
|
||||
}
|
||||
|
||||
describe("resolveAwsSdkEnvVarName", () => {
|
||||
it("prefers bearer token over access keys and profile", () => {
|
||||
const env = {
|
||||
@@ -538,6 +562,7 @@ describe("resolveUsableCustomProviderApiKey", () => {
|
||||
},
|
||||
},
|
||||
provider: "custom",
|
||||
secretSentinels: true,
|
||||
});
|
||||
expect(resolved?.apiKey).toBe("sk-from-env");
|
||||
expect(resolved?.source).toContain("OPENAI_API_KEY");
|
||||
@@ -571,8 +596,10 @@ describe("resolveUsableCustomProviderApiKey", () => {
|
||||
},
|
||||
},
|
||||
provider: "custom",
|
||||
secretSentinels: true,
|
||||
});
|
||||
expect(resolved?.apiKey).toBe("sk-secretref-env");
|
||||
expect(looksLikeSecretSentinel(resolved?.apiKey ?? "")).toBe(true);
|
||||
expect(resolveSecretSentinel(resolved?.apiKey ?? "")).toBe("sk-secretref-env");
|
||||
expect(resolved?.source).toContain("OPENAI_API_KEY");
|
||||
} finally {
|
||||
if (previous === undefined) {
|
||||
@@ -583,6 +610,43 @@ describe("resolveUsableCustomProviderApiKey", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it("sentinelizes config env SecretRefs on env-first provider resolution", async () => {
|
||||
const previous = process.env.OPENAI_API_KEY;
|
||||
process.env.OPENAI_API_KEY = "sk-secretref-env-first"; // pragma: allowlist secret
|
||||
try {
|
||||
const resolved = await resolveApiKeyForProvider({
|
||||
cfg: {
|
||||
models: {
|
||||
providers: {
|
||||
custom: {
|
||||
baseUrl: "https://example.com/v1",
|
||||
apiKey: {
|
||||
source: "env",
|
||||
provider: "default",
|
||||
id: "OPENAI_API_KEY",
|
||||
},
|
||||
models: [],
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
provider: "custom",
|
||||
credentialPrecedence: "env-first",
|
||||
secretSentinels: true,
|
||||
store: { version: 1, profiles: {} },
|
||||
});
|
||||
|
||||
expect(looksLikeSecretSentinel(resolved.apiKey ?? "")).toBe(true);
|
||||
expect(resolveSecretSentinel(resolved.apiKey ?? "")).toBe("sk-secretref-env-first");
|
||||
} finally {
|
||||
if (previous === undefined) {
|
||||
delete process.env.OPENAI_API_KEY;
|
||||
} else {
|
||||
process.env.OPENAI_API_KEY = previous;
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it("resolves env SecretRefs with unknown env IDs from process env for custom providers", () => {
|
||||
const previous = process.env.MY_CUSTOM_KEY;
|
||||
process.env.MY_CUSTOM_KEY = "sk-custom-secretref-env"; // pragma: allowlist secret
|
||||
@@ -604,8 +668,10 @@ describe("resolveUsableCustomProviderApiKey", () => {
|
||||
},
|
||||
},
|
||||
provider: "custom",
|
||||
secretSentinels: true,
|
||||
});
|
||||
expect(resolved?.apiKey).toBe("sk-custom-secretref-env");
|
||||
expect(looksLikeSecretSentinel(resolved?.apiKey ?? "")).toBe(true);
|
||||
expect(resolveSecretSentinel(resolved?.apiKey ?? "")).toBe("sk-custom-secretref-env");
|
||||
expect(resolved?.source).toContain("MY_CUSTOM_KEY");
|
||||
} finally {
|
||||
if (previous === undefined) {
|
||||
@@ -634,8 +700,10 @@ describe("resolveUsableCustomProviderApiKey", () => {
|
||||
},
|
||||
},
|
||||
provider: "bailian",
|
||||
secretSentinels: true,
|
||||
});
|
||||
expect(resolved?.apiKey).toBe("sk-bailian-env");
|
||||
expect(looksLikeSecretSentinel(resolved?.apiKey ?? "")).toBe(true);
|
||||
expect(resolveSecretSentinel(resolved?.apiKey ?? "")).toBe("sk-bailian-env");
|
||||
expect(resolved?.source).toContain("BAILIAN_API_KEY");
|
||||
} finally {
|
||||
if (previous === undefined) {
|
||||
@@ -812,6 +880,68 @@ describe("resolveUsableCustomProviderApiKey", () => {
|
||||
});
|
||||
|
||||
describe("resolveApiKeyForProvider", () => {
|
||||
it("keeps plain environment credentials as plaintext", async () => {
|
||||
const resolved = await withEnv("OPENAI_API_KEY", "sk-plain-env-key", () =>
|
||||
resolveApiKeyForProvider({
|
||||
provider: "openai",
|
||||
store: { version: 1, profiles: {} },
|
||||
}),
|
||||
);
|
||||
|
||||
expect(resolved.apiKey).toBe("sk-plain-env-key");
|
||||
expect(looksLikeSecretSentinel(resolved.apiKey ?? "")).toBe(false);
|
||||
});
|
||||
|
||||
it("sentinelizes credentials resolved from auth-profile SecretRefs", async () => {
|
||||
const profileId = "openai:secretref";
|
||||
const resolved = await withEnv("OPENAI_PROFILE_SECRET", "sk-profile-secretref", () =>
|
||||
resolveApiKeyForProvider({
|
||||
provider: "openai",
|
||||
profileId,
|
||||
secretSentinels: true,
|
||||
store: {
|
||||
version: 1,
|
||||
profiles: {
|
||||
[profileId]: {
|
||||
type: "api_key",
|
||||
provider: "openai",
|
||||
keyRef: { source: "env", provider: "default", id: "OPENAI_PROFILE_SECRET" },
|
||||
},
|
||||
},
|
||||
},
|
||||
}),
|
||||
);
|
||||
|
||||
expectSecretSentinelAuth(resolved, {
|
||||
value: "sk-profile-secretref",
|
||||
source: `profile:${profileId}`,
|
||||
mode: "api-key",
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps SecretRef profile credentials request-ready outside model sentinel mode", async () => {
|
||||
const profileId = "openai:non-model";
|
||||
const resolved = await withEnv("OPENAI_NON_MODEL_SECRET", "sk-non-model-secret", () =>
|
||||
resolveApiKeyForProvider({
|
||||
provider: "openai",
|
||||
profileId,
|
||||
store: {
|
||||
version: 1,
|
||||
profiles: {
|
||||
[profileId]: {
|
||||
type: "api_key",
|
||||
provider: "openai",
|
||||
keyRef: { source: "env", provider: "default", id: "OPENAI_NON_MODEL_SECRET" },
|
||||
},
|
||||
},
|
||||
},
|
||||
}),
|
||||
);
|
||||
|
||||
expect(resolved.apiKey).toBe("sk-non-model-secret");
|
||||
expect(looksLikeSecretSentinel(resolved.apiKey ?? "")).toBe(false);
|
||||
});
|
||||
|
||||
it("reuses plugin fallback auth without a models.providers entry", async () => {
|
||||
const resolved = await withoutEnv("PLUGIN_WEB_API_KEY", () =>
|
||||
resolveApiKeyForProvider({
|
||||
@@ -873,12 +1003,13 @@ describe("resolveApiKeyForProvider", () => {
|
||||
resolveApiKeyForProvider({
|
||||
provider: "plugin-web",
|
||||
cfg: sourceConfig,
|
||||
secretSentinels: true,
|
||||
store: { version: 1, profiles: {} },
|
||||
}),
|
||||
);
|
||||
|
||||
expectAuthFields(resolved, {
|
||||
apiKey: "plugin-web-runtime-key",
|
||||
expectSecretSentinelAuth(resolved, {
|
||||
value: "plugin-web-runtime-key",
|
||||
source: "plugins.entries.plugin-web.config.webSearch.apiKey",
|
||||
mode: "api-key",
|
||||
});
|
||||
@@ -889,6 +1020,10 @@ describe("resolveApiKeyForProvider", () => {
|
||||
name: "generated marker",
|
||||
apiKey: NON_ENV_SECRETREF_MARKER,
|
||||
},
|
||||
{
|
||||
name: "legacy env marker",
|
||||
apiKey: "secretref-env:CLIPROXY_API_KEY",
|
||||
},
|
||||
{
|
||||
name: "file SecretRef",
|
||||
apiKey: { source: "file", provider: "vault", id: "/cliproxy/api-key" } as const,
|
||||
@@ -921,11 +1056,12 @@ describe("resolveApiKeyForProvider", () => {
|
||||
const resolved = await resolveApiKeyForProvider({
|
||||
provider: "cliproxyapi",
|
||||
cfg: sourceConfig,
|
||||
secretSentinels: true,
|
||||
store: { version: 1, profiles: {} },
|
||||
});
|
||||
|
||||
expectAuthFields(resolved, {
|
||||
apiKey: "sk-runtime-cliproxy",
|
||||
expectSecretSentinelAuth(resolved, {
|
||||
value: "sk-runtime-cliproxy",
|
||||
source: "models.providers.cliproxyapi",
|
||||
mode: "api-key",
|
||||
});
|
||||
@@ -945,6 +1081,68 @@ describe("resolveApiKeyForProvider", () => {
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("preserves SecretRef provenance for resolved runtime config clones", async () => {
|
||||
const sourceConfig = {
|
||||
models: {
|
||||
providers: {
|
||||
cliproxyapi: {
|
||||
api: "openai-responses" as const,
|
||||
apiKey: { source: "file", provider: "vault", id: "/cliproxy/api-key" } as const,
|
||||
baseUrl: "https://cliproxy.example/v1",
|
||||
models: [],
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
const runtimeConfig = {
|
||||
models: {
|
||||
providers: {
|
||||
cliproxyapi: {
|
||||
...sourceConfig.models.providers.cliproxyapi,
|
||||
apiKey: "sk-runtime-clone", // pragma: allowlist secret
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
setRuntimeConfigSnapshot(runtimeConfig, sourceConfig);
|
||||
|
||||
const resolved = await resolveApiKeyForProvider({
|
||||
provider: "cliproxyapi",
|
||||
cfg: structuredClone(runtimeConfig),
|
||||
secretSentinels: true,
|
||||
store: { version: 1, profiles: {} },
|
||||
});
|
||||
|
||||
expectSecretSentinelAuth(resolved, {
|
||||
value: "sk-runtime-clone",
|
||||
source: "models.providers.cliproxyapi",
|
||||
mode: "api-key",
|
||||
});
|
||||
|
||||
const preferred = await resolveApiKeyForProvider({
|
||||
provider: "cliproxyapi",
|
||||
cfg: structuredClone(runtimeConfig),
|
||||
preferredProfile: "cliproxyapi:preferred",
|
||||
credentialPrecedence: "profile-first",
|
||||
secretSentinels: true,
|
||||
store: {
|
||||
version: 1,
|
||||
profiles: {
|
||||
"cliproxyapi:preferred": {
|
||||
type: "api_key",
|
||||
provider: "cliproxyapi",
|
||||
key: "sk-preferred-profile", // pragma: allowlist secret
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
expectAuthFields(preferred, {
|
||||
apiKey: "sk-preferred-profile",
|
||||
source: "profile:cliproxyapi:preferred",
|
||||
mode: "api-key",
|
||||
});
|
||||
});
|
||||
|
||||
it("does not treat a custom provider managed SecretRef marker as auth without a runtime snapshot", async () => {
|
||||
const sourceConfig = {
|
||||
models: {
|
||||
@@ -1161,6 +1359,7 @@ describe("resolveApiKeyForProvider", () => {
|
||||
const resolved = await resolveApiKeyForProvider({
|
||||
provider: "cliproxyapi",
|
||||
cfg: sourceConfig,
|
||||
secretSentinels: true,
|
||||
store: {
|
||||
version: 1,
|
||||
profiles: {
|
||||
@@ -1173,8 +1372,8 @@ describe("resolveApiKeyForProvider", () => {
|
||||
},
|
||||
});
|
||||
|
||||
expectAuthFields(resolved, {
|
||||
apiKey: "sk-runtime-cliproxy",
|
||||
expectSecretSentinelAuth(resolved, {
|
||||
value: "sk-runtime-cliproxy",
|
||||
source: "models.providers.cliproxyapi",
|
||||
mode: "api-key",
|
||||
});
|
||||
@@ -1385,11 +1584,12 @@ describe("resolveApiKeyForProvider – synthetic local auth for custom providers
|
||||
const auth = await resolveApiKeyForProvider({
|
||||
provider: "ollama-gpu1",
|
||||
cfg: sourceConfig,
|
||||
secretSentinels: true,
|
||||
store: { version: 1, profiles: {} },
|
||||
});
|
||||
|
||||
expectAuthFields(auth, {
|
||||
apiKey: "sk-runtime-ollama",
|
||||
expectSecretSentinelAuth(auth, {
|
||||
value: "sk-runtime-ollama",
|
||||
source: "models.providers.ollama-gpu1",
|
||||
mode: "api-key",
|
||||
});
|
||||
@@ -1749,6 +1949,129 @@ describe("applyAuthHeaderOverride", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("sentinelizes SecretRef-managed provider headers from the runtime snapshot", () => {
|
||||
const sourceConfig = {
|
||||
models: {
|
||||
providers: {
|
||||
google: {
|
||||
baseUrl: "https://generativelanguage.googleapis.com/v1beta/openai",
|
||||
api: "openai-completions" as const,
|
||||
headers: {
|
||||
Authorization: "secretref-env:GOOGLE_AUTH_TOKEN",
|
||||
"X-Managed": NON_ENV_SECRETREF_MARKER,
|
||||
},
|
||||
models: [],
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
const runtimeConfig = {
|
||||
models: {
|
||||
providers: {
|
||||
google: {
|
||||
...sourceConfig.models.providers.google,
|
||||
headers: {
|
||||
Authorization: "Bearer runtime-google-secret",
|
||||
"X-Managed": "runtime-managed-secret",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
setRuntimeConfigSnapshot(runtimeConfig, sourceConfig);
|
||||
|
||||
const result = applyAuthHeaderOverride(
|
||||
{
|
||||
...baseModel,
|
||||
headers: {
|
||||
Authorization: "Bearer runtime-google-secret",
|
||||
"X-Managed": "runtime-managed-secret",
|
||||
"X-Plain": "visible",
|
||||
},
|
||||
},
|
||||
null,
|
||||
sourceConfig,
|
||||
);
|
||||
|
||||
expect(looksLikeSecretSentinel(result.headers?.Authorization ?? "")).toBe(true);
|
||||
expect(resolveSecretSentinel(result.headers?.Authorization ?? "")).toBe(
|
||||
"Bearer runtime-google-secret",
|
||||
);
|
||||
expect(looksLikeSecretSentinel(result.headers?.["X-Managed"] ?? "")).toBe(true);
|
||||
expect(resolveSecretSentinel(result.headers?.["X-Managed"] ?? "")).toBe(
|
||||
"runtime-managed-secret",
|
||||
);
|
||||
expect(result.headers?.["X-Plain"]).toBe("visible");
|
||||
});
|
||||
|
||||
it("sentinelizes SecretRef-managed request headers and composed auth", () => {
|
||||
const sourceConfig = {
|
||||
models: {
|
||||
providers: {
|
||||
google: {
|
||||
baseUrl: "https://generativelanguage.googleapis.com/v1beta/openai",
|
||||
api: "openai-completions" as const,
|
||||
request: {
|
||||
headers: { "X-Managed": NON_ENV_SECRETREF_MARKER },
|
||||
auth: {
|
||||
mode: "authorization-bearer" as const,
|
||||
token: "secretref-env:GOOGLE_BEARER_TOKEN",
|
||||
},
|
||||
},
|
||||
models: [],
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
const runtimeConfig = {
|
||||
models: {
|
||||
providers: {
|
||||
google: {
|
||||
...sourceConfig.models.providers.google,
|
||||
request: {
|
||||
headers: { "X-Managed": "runtime-managed-secret" },
|
||||
auth: {
|
||||
mode: "authorization-bearer" as const,
|
||||
token: "runtime-bearer-secret",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
setRuntimeConfigSnapshot(runtimeConfig, sourceConfig);
|
||||
|
||||
const result = applySecretRefHeaderSentinels(
|
||||
attachModelProviderRequestTransport(
|
||||
{
|
||||
...baseModel,
|
||||
headers: {
|
||||
"X-Managed": "runtime-managed-secret",
|
||||
Authorization: "Bearer runtime-bearer-secret",
|
||||
},
|
||||
},
|
||||
runtimeConfig.models.providers.google.request,
|
||||
),
|
||||
sourceConfig,
|
||||
);
|
||||
|
||||
const managedSentinel = result.headers?.["X-Managed"] ?? "";
|
||||
expect(looksLikeSecretSentinel(managedSentinel)).toBe(true);
|
||||
expect(resolveSecretSentinel(managedSentinel)).toBe("runtime-managed-secret");
|
||||
const bearerSentinel = result.headers?.Authorization?.slice("Bearer ".length) ?? "";
|
||||
expect(looksLikeSecretSentinel(bearerSentinel)).toBe(true);
|
||||
expect(resolveSecretSentinel(bearerSentinel)).toBe("runtime-bearer-secret");
|
||||
const request = getModelProviderRequestTransport(result);
|
||||
const requestHeaderSentinel = request?.headers?.["X-Managed"] ?? "";
|
||||
expect(looksLikeSecretSentinel(requestHeaderSentinel)).toBe(true);
|
||||
expect(resolveSecretSentinel(requestHeaderSentinel)).toBe("runtime-managed-secret");
|
||||
expect(request?.auth?.mode).toBe("authorization-bearer");
|
||||
const requestTokenSentinel =
|
||||
request?.auth?.mode === "authorization-bearer" ? request.auth.token : "";
|
||||
expect(looksLikeSecretSentinel(requestTokenSentinel)).toBe(true);
|
||||
expect(resolveSecretSentinel(requestTokenSentinel)).toBe("runtime-bearer-secret");
|
||||
});
|
||||
|
||||
it("returns model unchanged when authHeader is not set", () => {
|
||||
const result = applyAuthHeaderOverride(
|
||||
baseModel,
|
||||
|
||||
+344
-32
@@ -13,6 +13,7 @@ import { formatCliCommand } from "../cli/command-format.js";
|
||||
import {
|
||||
getRuntimeConfigSnapshot,
|
||||
getRuntimeConfigSourceSnapshot,
|
||||
hashRuntimeConfigValue,
|
||||
selectApplicableRuntimeConfig,
|
||||
} from "../config/config.js";
|
||||
import type { ModelProviderAuthMode, ModelProviderConfig } from "../config/types.js";
|
||||
@@ -30,6 +31,7 @@ import {
|
||||
import { resolveOwningPluginIdsForProviderRef } from "../plugins/providers.js";
|
||||
import { resolveRuntimeSyntheticAuthProviderRefState } from "../plugins/synthetic-auth.runtime.js";
|
||||
import { resolveDefaultSecretProviderAlias } from "../secrets/ref-contract.js";
|
||||
import { mintSecretSentinel } from "../secrets/sentinel.js";
|
||||
import { normalizeOptionalSecretInput } from "../utils/normalize-secret-input.js";
|
||||
import { resolveDefaultAgentDir } from "./agent-scope-config.js";
|
||||
import {
|
||||
@@ -56,10 +58,16 @@ import {
|
||||
CUSTOM_LOCAL_AUTH_MARKER,
|
||||
isKnownEnvApiKeyMarker,
|
||||
isNonSecretApiKeyMarker,
|
||||
isSecretRefHeaderValueMarker,
|
||||
NON_ENV_SECRETREF_MARKER,
|
||||
SECRETREF_ENV_HEADER_MARKER_PREFIX,
|
||||
} from "./model-auth-markers.js";
|
||||
import { ProviderAuthError, type ResolvedProviderAuth } from "./model-auth-runtime-shared.js";
|
||||
import { normalizeProviderId } from "./model-selection.js";
|
||||
import {
|
||||
attachModelProviderRequestTransport,
|
||||
getModelProviderRequestTransport,
|
||||
} from "./provider-request-config.js";
|
||||
|
||||
export {
|
||||
ensureAuthProfileStore,
|
||||
@@ -78,6 +86,25 @@ export {
|
||||
export type { ResolvedProviderAuth } from "./model-auth-runtime-shared.js";
|
||||
export type ProviderCredentialPrecedence = "profile-first" | "env-first";
|
||||
|
||||
function sentinelizeSecretRefProfileApiKey(params: {
|
||||
apiKey: string;
|
||||
enabled?: boolean;
|
||||
profileId: string;
|
||||
provider: string;
|
||||
store: AuthProfileStore;
|
||||
}): string {
|
||||
const credential = params.store.profiles[params.profileId];
|
||||
const ref =
|
||||
credential?.type === "api_key"
|
||||
? coerceSecretRef(credential.keyRef)
|
||||
: credential?.type === "token"
|
||||
? coerceSecretRef(credential.tokenRef)
|
||||
: null;
|
||||
return ref && params.enabled
|
||||
? mintSecretSentinel(params.apiKey, { label: `model-auth:${params.provider}` })
|
||||
: params.apiKey;
|
||||
}
|
||||
|
||||
/** Precomputed provider-auth lookup tables reused during one runtime turn. */
|
||||
export type RuntimeProviderAuthLookup = {
|
||||
envApiKey: Pick<
|
||||
@@ -272,6 +299,7 @@ export function resolveUsableCustomProviderApiKey(params: {
|
||||
cfg: OpenClawConfig | undefined;
|
||||
provider: string;
|
||||
env?: NodeJS.ProcessEnv;
|
||||
secretSentinels?: boolean;
|
||||
}): ResolvedCustomProviderApiKey | null {
|
||||
const customProviderConfig = resolveProviderConfig(params.cfg, params.provider);
|
||||
const apiKeyRef = coerceSecretRef(customProviderConfig?.apiKey);
|
||||
@@ -298,7 +326,9 @@ export function resolveUsableCustomProviderApiKey(params: {
|
||||
}
|
||||
const applied = new Set(getShellEnvAppliedKeys());
|
||||
return {
|
||||
apiKey: envValue,
|
||||
apiKey: params.secretSentinels
|
||||
? mintSecretSentinel(envValue, { label: `model-auth:${params.provider}` })
|
||||
: envValue,
|
||||
source: resolveEnvSourceLabel({
|
||||
applied,
|
||||
envVars: [envVarName],
|
||||
@@ -550,6 +580,7 @@ export async function resolveProviderEntryApiKeyBinding(params: {
|
||||
provider: string;
|
||||
store: AuthProfileStore;
|
||||
agentDir?: string;
|
||||
secretSentinels?: boolean;
|
||||
}): Promise<ProviderEntryApiKeyBindingResolution> {
|
||||
const reference = resolveProviderEntryApiKeyProfileReference(params);
|
||||
if (reference.kind === "none" || reference.kind === "marker") {
|
||||
@@ -575,7 +606,13 @@ export async function resolveProviderEntryApiKeyBinding(params: {
|
||||
return {
|
||||
kind: "profile-resolved",
|
||||
auth: {
|
||||
apiKey: resolved.apiKey,
|
||||
apiKey: sentinelizeSecretRefProfileApiKey({
|
||||
apiKey: resolved.apiKey,
|
||||
enabled: params.secretSentinels,
|
||||
profileId: resolvedProfileId,
|
||||
provider: params.provider,
|
||||
store: params.store,
|
||||
}),
|
||||
profileId: resolvedProfileId,
|
||||
source: `profile:${resolvedProfileId}`,
|
||||
mode: resolved.profileType ? profileTypeToAuthMode(resolved.profileType) : reference.mode,
|
||||
@@ -659,16 +696,68 @@ function isManagedSecretRefApiKeyMarker(apiKey: string | undefined): boolean {
|
||||
return apiKey?.trim() === NON_ENV_SECRETREF_MARKER;
|
||||
}
|
||||
|
||||
function hasManagedSecretRefProviderApiKey(
|
||||
cfg: OpenClawConfig | undefined,
|
||||
provider: string,
|
||||
): boolean {
|
||||
function hasSecretRefProviderApiKey(cfg: OpenClawConfig | undefined, provider: string): boolean {
|
||||
const apiKey = resolveProviderConfig(cfg, provider)?.apiKey;
|
||||
const ref = coerceSecretRef(apiKey);
|
||||
if (ref) {
|
||||
return ref.source !== "env";
|
||||
if (coerceSecretRef(apiKey)) {
|
||||
return true;
|
||||
}
|
||||
return typeof apiKey === "string" && isManagedSecretRefApiKeyMarker(apiKey);
|
||||
return (
|
||||
typeof apiKey === "string" &&
|
||||
(isManagedSecretRefApiKeyMarker(apiKey) ||
|
||||
apiKey.trim().startsWith(SECRETREF_ENV_HEADER_MARKER_PREFIX))
|
||||
);
|
||||
}
|
||||
|
||||
function providerConfigMatchesRuntimeSnapshot(params: {
|
||||
inputConfig: OpenClawConfig | undefined;
|
||||
runtimeConfig: OpenClawConfig | null;
|
||||
provider: string;
|
||||
}): boolean {
|
||||
const inputProvider = resolveProviderConfig(params.inputConfig, params.provider);
|
||||
const runtimeProvider = resolveProviderConfig(params.runtimeConfig ?? undefined, params.provider);
|
||||
if (!inputProvider || !runtimeProvider) {
|
||||
return false;
|
||||
}
|
||||
const toComparableConfig = (providerConfig: ModelProviderConfig): OpenClawConfig => ({
|
||||
models: { providers: { [params.provider]: providerConfig } },
|
||||
});
|
||||
return (
|
||||
hashRuntimeConfigValue(toComparableConfig(inputProvider)) ===
|
||||
hashRuntimeConfigValue(toComparableConfig(runtimeProvider))
|
||||
);
|
||||
}
|
||||
|
||||
function sentinelizeConfigSecretRefEnvApiKey(params: {
|
||||
apiKey: string;
|
||||
source: string;
|
||||
cfg: OpenClawConfig | undefined;
|
||||
provider: string;
|
||||
enabled?: boolean;
|
||||
}): string {
|
||||
if (!params.enabled) {
|
||||
return params.apiKey;
|
||||
}
|
||||
const runtimeConfig = getRuntimeConfigSnapshot();
|
||||
const runtimeSourceConfig = getRuntimeConfigSourceSnapshot();
|
||||
const sourceConfig = providerConfigMatchesRuntimeSnapshot({
|
||||
inputConfig: params.cfg,
|
||||
runtimeConfig,
|
||||
provider: params.provider,
|
||||
})
|
||||
? (runtimeSourceConfig ?? params.cfg)
|
||||
: params.cfg;
|
||||
const configured = resolveProviderConfig(sourceConfig, params.provider)?.apiKey;
|
||||
const ref = coerceSecretRef(configured);
|
||||
const envId =
|
||||
ref?.source === "env"
|
||||
? ref.id
|
||||
: typeof configured === "string" &&
|
||||
configured.trim().startsWith(SECRETREF_ENV_HEADER_MARKER_PREFIX)
|
||||
? configured.trim().slice(SECRETREF_ENV_HEADER_MARKER_PREFIX.length)
|
||||
: undefined;
|
||||
return envId && params.source.includes(envId)
|
||||
? mintSecretSentinel(params.apiKey, { label: `model-auth:${params.provider}` })
|
||||
: params.apiKey;
|
||||
}
|
||||
|
||||
function resolveLiteralProviderConfigApiKeyAuth(params: {
|
||||
@@ -691,10 +780,8 @@ function resolveLiteralProviderConfigApiKeyAuth(params: {
|
||||
function resolveManagedSecretRefRuntimeProviderAuth(params: {
|
||||
cfg: OpenClawConfig | undefined;
|
||||
provider: string;
|
||||
secretSentinels?: boolean;
|
||||
}): ResolvedProviderAuth | undefined {
|
||||
if (!hasManagedSecretRefProviderApiKey(params.cfg, params.provider)) {
|
||||
return undefined;
|
||||
}
|
||||
const runtimeConfig = getRuntimeConfigSnapshot();
|
||||
const runtimeSourceConfig = getRuntimeConfigSourceSnapshot();
|
||||
if (params.cfg && params.cfg !== runtimeConfig && !runtimeSourceConfig) {
|
||||
@@ -705,13 +792,35 @@ function resolveManagedSecretRefRuntimeProviderAuth(params: {
|
||||
runtimeConfig,
|
||||
runtimeSourceConfig,
|
||||
});
|
||||
if (!runtimeConfig || applicableConfig !== runtimeConfig) {
|
||||
const usesRuntimeProvider =
|
||||
applicableConfig === runtimeConfig ||
|
||||
providerConfigMatchesRuntimeSnapshot({
|
||||
inputConfig: params.cfg,
|
||||
runtimeConfig,
|
||||
provider: params.provider,
|
||||
});
|
||||
const sourceConfig = usesRuntimeProvider ? (runtimeSourceConfig ?? undefined) : params.cfg;
|
||||
if (!hasSecretRefProviderApiKey(sourceConfig, params.provider)) {
|
||||
return undefined;
|
||||
}
|
||||
return resolveLiteralProviderConfigApiKeyAuth({
|
||||
if (!runtimeConfig || !usesRuntimeProvider) {
|
||||
return undefined;
|
||||
}
|
||||
const resolved = resolveLiteralProviderConfigApiKeyAuth({
|
||||
cfg: runtimeConfig,
|
||||
provider: params.provider,
|
||||
});
|
||||
if (!resolved?.apiKey) {
|
||||
return undefined;
|
||||
}
|
||||
return {
|
||||
...resolved,
|
||||
apiKey: params.secretSentinels
|
||||
? mintSecretSentinel(resolved.apiKey, {
|
||||
label: `model-auth:${params.provider}`,
|
||||
})
|
||||
: resolved.apiKey,
|
||||
};
|
||||
}
|
||||
|
||||
/** True when a custom local provider can use a synthetic no-auth placeholder. */
|
||||
@@ -845,12 +954,13 @@ function resolveProviderSyntheticRuntimeAuth(params: {
|
||||
cfg: OpenClawConfig | undefined;
|
||||
provider: string;
|
||||
modelApi?: string;
|
||||
secretSentinels?: boolean;
|
||||
}): SyntheticProviderAuthResolution {
|
||||
const runtimeAuth = resolveManagedSecretRefRuntimeProviderAuth(params);
|
||||
if (runtimeAuth) {
|
||||
return { auth: runtimeAuth };
|
||||
}
|
||||
if (hasManagedSecretRefProviderApiKey(params.cfg, params.provider)) {
|
||||
if (hasSecretRefProviderApiKey(params.cfg, params.provider)) {
|
||||
return { blockedOnManagedSecretRef: true };
|
||||
}
|
||||
|
||||
@@ -891,7 +1001,14 @@ function resolveProviderSyntheticRuntimeAuth(params: {
|
||||
return { blockedOnManagedSecretRef: true };
|
||||
}
|
||||
return {
|
||||
auth: runtimePluginAuth,
|
||||
auth: {
|
||||
...runtimePluginAuth,
|
||||
apiKey: params.secretSentinels
|
||||
? mintSecretSentinel(runtimeApiKey, {
|
||||
label: `model-auth:${params.provider}`,
|
||||
})
|
||||
: runtimeApiKey,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
@@ -899,6 +1016,7 @@ function resolveSyntheticLocalProviderAuth(params: {
|
||||
cfg: OpenClawConfig | undefined;
|
||||
provider: string;
|
||||
modelApi?: string;
|
||||
secretSentinels?: boolean;
|
||||
}): ResolvedProviderAuth | null {
|
||||
const syntheticProviderAuth = resolveProviderSyntheticRuntimeAuth(params);
|
||||
if (syntheticProviderAuth.auth) {
|
||||
@@ -1021,6 +1139,8 @@ export async function resolveApiKeyForProvider(params: {
|
||||
forceRefresh?: boolean;
|
||||
credentialPrecedence?: ProviderCredentialPrecedence;
|
||||
modelApi?: string;
|
||||
/** Keep SecretRef-backed model credentials opaque until a sentinel-aware transport boundary. */
|
||||
secretSentinels?: boolean;
|
||||
}): Promise<ResolvedProviderAuth> {
|
||||
const { provider, cfg, profileId, preferredProfile } = params;
|
||||
const agentDir = params.agentDir?.trim() || (cfg ? resolveDefaultAgentDir(cfg) : undefined);
|
||||
@@ -1062,7 +1182,13 @@ export async function resolveApiKeyForProvider(params: {
|
||||
const resolvedProfileId = resolved.profileId ?? profileId;
|
||||
const mode = resolved.profileType ?? store.profiles[resolvedProfileId]?.type;
|
||||
const result: ResolvedProviderAuth = {
|
||||
apiKey: resolved.apiKey,
|
||||
apiKey: sentinelizeSecretRefProfileApiKey({
|
||||
apiKey: resolved.apiKey,
|
||||
enabled: params.secretSentinels,
|
||||
profileId: resolvedProfileId,
|
||||
provider,
|
||||
store,
|
||||
}),
|
||||
profileId: resolvedProfileId,
|
||||
source: `profile:${resolvedProfileId}`,
|
||||
mode: mode ? profileTypeToAuthMode(mode) : "api-key",
|
||||
@@ -1147,7 +1273,13 @@ export async function resolveApiKeyForProvider(params: {
|
||||
return resolveApiKeyForProvider({ ...params, credentialPrecedence: "profile-first" });
|
||||
}
|
||||
return {
|
||||
apiKey: envResolved.apiKey,
|
||||
apiKey: sentinelizeConfigSecretRefEnvApiKey({
|
||||
apiKey: envResolved.apiKey,
|
||||
source: envResolved.source,
|
||||
cfg,
|
||||
provider,
|
||||
enabled: params.secretSentinels,
|
||||
}),
|
||||
source: envResolved.source,
|
||||
mode: resolvedMode,
|
||||
};
|
||||
@@ -1168,6 +1300,7 @@ export async function resolveApiKeyForProvider(params: {
|
||||
provider,
|
||||
store: scopedStore,
|
||||
agentDir,
|
||||
secretSentinels: params.secretSentinels,
|
||||
});
|
||||
if (providerEntryBinding.kind === "profile-resolved") {
|
||||
assertAuthModeAllowedForModel({
|
||||
@@ -1201,11 +1334,19 @@ export async function resolveApiKeyForProvider(params: {
|
||||
}
|
||||
|
||||
if (shouldPreferExplicitConfigApiKeyAuth(cfg, provider)) {
|
||||
const runtimeCustomKey = resolveManagedSecretRefRuntimeProviderAuth({ cfg, provider });
|
||||
const runtimeCustomKey = resolveManagedSecretRefRuntimeProviderAuth({
|
||||
cfg,
|
||||
provider,
|
||||
secretSentinels: params.secretSentinels,
|
||||
});
|
||||
if (runtimeCustomKey) {
|
||||
return runtimeCustomKey;
|
||||
}
|
||||
const customKey = resolveUsableCustomProviderApiKey({ cfg, provider });
|
||||
const customKey = resolveUsableCustomProviderApiKey({
|
||||
cfg,
|
||||
provider,
|
||||
secretSentinels: params.secretSentinels,
|
||||
});
|
||||
if (customKey) {
|
||||
return {
|
||||
apiKey: customKey.apiKey,
|
||||
@@ -1215,7 +1356,11 @@ export async function resolveApiKeyForProvider(params: {
|
||||
}
|
||||
}
|
||||
const providerConfig = resolveProviderConfig(cfg, provider);
|
||||
const configuredLocalKey = resolveUsableCustomProviderApiKey({ cfg, provider });
|
||||
const configuredLocalKey = resolveUsableCustomProviderApiKey({
|
||||
cfg,
|
||||
provider,
|
||||
secretSentinels: params.secretSentinels,
|
||||
});
|
||||
if (configuredLocalKey && isNonSecretApiKeyMarker(configuredLocalKey.apiKey)) {
|
||||
return {
|
||||
apiKey: configuredLocalKey.apiKey,
|
||||
@@ -1284,7 +1429,13 @@ export async function resolveApiKeyForProvider(params: {
|
||||
? profileTypeToAuthMode(mode)
|
||||
: "api-key";
|
||||
const result: ResolvedProviderAuth = {
|
||||
apiKey: resolved.apiKey,
|
||||
apiKey: sentinelizeSecretRefProfileApiKey({
|
||||
apiKey: resolved.apiKey,
|
||||
enabled: params.secretSentinels,
|
||||
profileId: resolvedProfileId,
|
||||
provider,
|
||||
store,
|
||||
}),
|
||||
profileId: resolvedProfileId,
|
||||
source: `profile:${resolvedProfileId}`,
|
||||
mode: resolvedMode,
|
||||
@@ -1341,7 +1492,13 @@ export async function resolveApiKeyForProvider(params: {
|
||||
})
|
||||
) {
|
||||
const result: ResolvedProviderAuth = {
|
||||
apiKey: envResolved.apiKey,
|
||||
apiKey: sentinelizeConfigSecretRefEnvApiKey({
|
||||
apiKey: envResolved.apiKey,
|
||||
source: envResolved.source,
|
||||
cfg,
|
||||
provider,
|
||||
enabled: params.secretSentinels,
|
||||
}),
|
||||
source: envResolved.source,
|
||||
mode: resolvedMode,
|
||||
};
|
||||
@@ -1349,7 +1506,20 @@ export async function resolveApiKeyForProvider(params: {
|
||||
}
|
||||
}
|
||||
|
||||
const customKey = resolveUsableCustomProviderApiKey({ cfg, provider });
|
||||
const managedRuntimeAuth = resolveManagedSecretRefRuntimeProviderAuth({
|
||||
cfg,
|
||||
provider,
|
||||
secretSentinels: params.secretSentinels,
|
||||
});
|
||||
if (managedRuntimeAuth) {
|
||||
return managedRuntimeAuth;
|
||||
}
|
||||
|
||||
const customKey = resolveUsableCustomProviderApiKey({
|
||||
cfg,
|
||||
provider,
|
||||
secretSentinels: params.secretSentinels,
|
||||
});
|
||||
if (customKey) {
|
||||
const result = { apiKey: customKey.apiKey, source: customKey.source, mode: "api-key" as const };
|
||||
return result;
|
||||
@@ -1363,6 +1533,7 @@ export async function resolveApiKeyForProvider(params: {
|
||||
cfg,
|
||||
provider,
|
||||
modelApi: params.modelApi,
|
||||
secretSentinels: params.secretSentinels,
|
||||
});
|
||||
if (syntheticLocalAuth) {
|
||||
return syntheticLocalAuth;
|
||||
@@ -1579,6 +1750,7 @@ export async function getApiKeyForModel(params: {
|
||||
workspaceDir?: string;
|
||||
lockedProfile?: boolean;
|
||||
credentialPrecedence?: ProviderCredentialPrecedence;
|
||||
secretSentinels?: boolean;
|
||||
}): Promise<ResolvedProviderAuth> {
|
||||
return resolveApiKeyForProvider({
|
||||
provider: params.model.provider,
|
||||
@@ -1591,6 +1763,7 @@ export async function getApiKeyForModel(params: {
|
||||
lockedProfile: params.lockedProfile,
|
||||
credentialPrecedence: params.credentialPrecedence,
|
||||
modelApi: params.model.api,
|
||||
secretSentinels: params.secretSentinels,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -1617,6 +1790,144 @@ export function applyLocalNoAuthHeaderOverride<T extends Model>(
|
||||
};
|
||||
}
|
||||
|
||||
export function applySecretRefHeaderSentinels<T extends Model>(
|
||||
model: T,
|
||||
cfg: OpenClawConfig | undefined,
|
||||
): T {
|
||||
if (!model.headers) {
|
||||
return model;
|
||||
}
|
||||
const runtimeConfig = getRuntimeConfigSnapshot();
|
||||
const runtimeSourceConfig = getRuntimeConfigSourceSnapshot();
|
||||
const applicableConfig = selectApplicableRuntimeConfig({
|
||||
inputConfig: cfg,
|
||||
runtimeConfig,
|
||||
runtimeSourceConfig,
|
||||
});
|
||||
const usesRuntimeProvider =
|
||||
applicableConfig === runtimeConfig ||
|
||||
providerConfigMatchesRuntimeSnapshot({
|
||||
inputConfig: cfg,
|
||||
runtimeConfig,
|
||||
provider: model.provider,
|
||||
});
|
||||
if (!runtimeConfig || !runtimeSourceConfig || !usesRuntimeProvider) {
|
||||
return model;
|
||||
}
|
||||
const sourceProvider = resolveProviderConfig(runtimeSourceConfig, model.provider);
|
||||
const runtimeProvider = resolveProviderConfig(runtimeConfig, model.provider);
|
||||
const replacements = new Map<string, { value: string; replacement: string }>();
|
||||
const isManagedSecret = (value: unknown) =>
|
||||
coerceSecretRef(value) !== null ||
|
||||
(typeof value === "string" && isSecretRefHeaderValueMarker(value));
|
||||
const addReplacement = (name: string, value: string, replacement?: string) => {
|
||||
replacements.set(name.trim().toLowerCase(), {
|
||||
value,
|
||||
replacement:
|
||||
replacement ?? mintSecretSentinel(value, { label: `model-auth:${model.provider}` }),
|
||||
});
|
||||
};
|
||||
for (const [name, sourceValue] of Object.entries(sourceProvider?.headers ?? {})) {
|
||||
if (!isManagedSecret(sourceValue)) {
|
||||
continue;
|
||||
}
|
||||
const value = normalizeOptionalSecretInput(runtimeProvider?.headers?.[name]);
|
||||
if (value) {
|
||||
addReplacement(name, value);
|
||||
}
|
||||
}
|
||||
for (const [name, sourceValue] of Object.entries(sourceProvider?.request?.headers ?? {})) {
|
||||
if (!isManagedSecret(sourceValue)) {
|
||||
continue;
|
||||
}
|
||||
const value = normalizeOptionalSecretInput(runtimeProvider?.request?.headers?.[name]);
|
||||
if (value) {
|
||||
addReplacement(name, value);
|
||||
}
|
||||
}
|
||||
const sourceAuth = sourceProvider?.request?.auth;
|
||||
const runtimeAuth = runtimeProvider?.request?.auth;
|
||||
const attachedRequest = getModelProviderRequestTransport(model);
|
||||
let protectedRequest = attachedRequest;
|
||||
let protectedRequestHeaders: Record<string, string> | undefined;
|
||||
for (const [name, sourceValue] of Object.entries(sourceProvider?.request?.headers ?? {})) {
|
||||
if (!isManagedSecret(sourceValue)) {
|
||||
continue;
|
||||
}
|
||||
const value = normalizeOptionalSecretInput(runtimeProvider?.request?.headers?.[name]);
|
||||
if (!value || attachedRequest?.headers?.[name] !== value) {
|
||||
continue;
|
||||
}
|
||||
protectedRequestHeaders ??= { ...attachedRequest.headers };
|
||||
protectedRequestHeaders[name] = mintSecretSentinel(value, {
|
||||
label: `model-auth:${model.provider}`,
|
||||
});
|
||||
}
|
||||
if (protectedRequestHeaders && attachedRequest) {
|
||||
protectedRequest = { ...attachedRequest, headers: protectedRequestHeaders };
|
||||
}
|
||||
if (
|
||||
sourceAuth?.mode === "authorization-bearer" &&
|
||||
runtimeAuth?.mode === "authorization-bearer" &&
|
||||
isManagedSecret(sourceAuth.token)
|
||||
) {
|
||||
const token = normalizeOptionalSecretInput(runtimeAuth.token)?.trim();
|
||||
if (token) {
|
||||
if (attachedRequest?.auth?.mode === "authorization-bearer") {
|
||||
protectedRequest = {
|
||||
...protectedRequest,
|
||||
auth: {
|
||||
...attachedRequest.auth,
|
||||
token: mintSecretSentinel(token, { label: `model-auth:${model.provider}` }),
|
||||
},
|
||||
};
|
||||
}
|
||||
addReplacement(
|
||||
"Authorization",
|
||||
`Bearer ${token}`,
|
||||
`Bearer ${mintSecretSentinel(token, { label: `model-auth:${model.provider}` })}`,
|
||||
);
|
||||
}
|
||||
} else if (
|
||||
sourceAuth?.mode === "header" &&
|
||||
runtimeAuth?.mode === "header" &&
|
||||
isManagedSecret(sourceAuth.value)
|
||||
) {
|
||||
const value = normalizeOptionalSecretInput(runtimeAuth.value)?.trim();
|
||||
const headerName = runtimeAuth.headerName.trim();
|
||||
const prefix = runtimeAuth.prefix?.trim() ?? "";
|
||||
if (headerName && value) {
|
||||
if (attachedRequest?.auth?.mode === "header") {
|
||||
protectedRequest = {
|
||||
...protectedRequest,
|
||||
auth: {
|
||||
...attachedRequest.auth,
|
||||
value: mintSecretSentinel(value, { label: `model-auth:${model.provider}` }),
|
||||
},
|
||||
};
|
||||
}
|
||||
addReplacement(
|
||||
headerName,
|
||||
`${prefix}${value}`,
|
||||
`${prefix}${mintSecretSentinel(value, { label: `model-auth:${model.provider}` })}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
let headers: Record<string, string> | undefined;
|
||||
for (const [name, value] of Object.entries(model.headers)) {
|
||||
const replacement = replacements.get(name.trim().toLowerCase());
|
||||
if (replacement?.value !== value) {
|
||||
continue;
|
||||
}
|
||||
headers ??= { ...model.headers };
|
||||
headers[name] = replacement.replacement;
|
||||
}
|
||||
const protectedModel = headers ? { ...model, headers } : model;
|
||||
return protectedRequest && protectedRequest !== attachedRequest
|
||||
? attachModelProviderRequestTransport(protectedModel, protectedRequest)
|
||||
: protectedModel;
|
||||
}
|
||||
|
||||
/**
|
||||
* When the provider config sets `authHeader: true`, inject an explicit
|
||||
* `Authorization: Bearer <apiKey>` header into the model so downstream SDKs
|
||||
@@ -1632,23 +1943,24 @@ export function applyAuthHeaderOverride<T extends Model>(
|
||||
auth: ResolvedProviderAuth | null | undefined,
|
||||
cfg: OpenClawConfig | undefined,
|
||||
): T {
|
||||
const sentinelModel = applySecretRefHeaderSentinels(model, cfg);
|
||||
if (!auth?.apiKey) {
|
||||
return model;
|
||||
return sentinelModel;
|
||||
}
|
||||
// Reject synthetic marker values that are not real credentials.
|
||||
if (isNonSecretApiKeyMarker(auth.apiKey)) {
|
||||
return model;
|
||||
return sentinelModel;
|
||||
}
|
||||
const providerConfig = resolveProviderConfig(cfg, model.provider);
|
||||
const providerConfig = resolveProviderConfig(cfg, sentinelModel.provider);
|
||||
if (!providerConfig?.authHeader) {
|
||||
return model;
|
||||
return sentinelModel;
|
||||
}
|
||||
|
||||
// Strip any existing authorization header (case-insensitive) before
|
||||
// injecting the canonical one so we don't produce a comma-joined value.
|
||||
const headers: Record<string, string> = {};
|
||||
if (model.headers) {
|
||||
for (const [key, value] of Object.entries(model.headers)) {
|
||||
if (sentinelModel.headers) {
|
||||
for (const [key, value] of Object.entries(sentinelModel.headers)) {
|
||||
if (normalizeOptionalLowercaseString(key) !== "authorization") {
|
||||
headers[key] = value;
|
||||
}
|
||||
@@ -1657,7 +1969,7 @@ export function applyAuthHeaderOverride<T extends Model>(
|
||||
headers.Authorization = `Bearer ${auth.apiKey}`;
|
||||
|
||||
return {
|
||||
...model,
|
||||
...sentinelModel,
|
||||
headers,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -6,6 +6,7 @@ import path from "node:path";
|
||||
import { MAX_TIMER_TIMEOUT_MS } from "@openclaw/normalization-core/number-coercion";
|
||||
import type { Model } from "openclaw/plugin-sdk/llm";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { mintSecretSentinel } from "../secrets/sentinel.js";
|
||||
import { killPidIfAlive, readPidFile, waitForPidToExit } from "../test-utils/process-tree.js";
|
||||
import {
|
||||
attachModelProviderLocalService,
|
||||
@@ -167,8 +168,9 @@ describe("provider local service", () => {
|
||||
},
|
||||
);
|
||||
|
||||
const sentinel = mintSecretSentinel("health-secret", { label: "local-health-probe" });
|
||||
const lease = await ensureModelProviderLocalService(model, {
|
||||
Authorization: "Bearer health-secret",
|
||||
Authorization: `Bearer ${sentinel}`,
|
||||
"X-Tenant": "acme",
|
||||
});
|
||||
|
||||
@@ -187,6 +189,30 @@ describe("provider local service", () => {
|
||||
await waitForProbeFailure(healthUrl);
|
||||
});
|
||||
|
||||
it("rejects unknown sentinels before starting a local service", async () => {
|
||||
const port = await freePort();
|
||||
const unknown = "oc-sent-v1-fedcba987654321001234567";
|
||||
const model = attachModelProviderLocalService(
|
||||
{
|
||||
id: "demo",
|
||||
provider: "local-unknown-auth",
|
||||
api: "openai-completions",
|
||||
baseUrl: `http://127.0.0.1:${port}/v1`,
|
||||
} as unknown as Model<"openai-completions">,
|
||||
{
|
||||
command: process.execPath,
|
||||
args: ["--version"],
|
||||
readyTimeoutMs: 1_000,
|
||||
},
|
||||
);
|
||||
|
||||
await expect(
|
||||
ensureModelProviderLocalService(model, { Authorization: `Bearer ${unknown}` }),
|
||||
).rejects.toThrow(
|
||||
`Secret sentinel ${unknown} is not registered in this process; refusing to probe local model provider health`,
|
||||
);
|
||||
});
|
||||
|
||||
it("cancels local service health probe response bodies", async () => {
|
||||
let socketClosed = false;
|
||||
const sockets = new Set<net.Socket>();
|
||||
|
||||
@@ -17,6 +17,7 @@ import {
|
||||
signalChildProcessTree,
|
||||
shouldDetachChildForProcessTree,
|
||||
} from "../process/child-process-tree.js";
|
||||
import { unwrapHeadersInitSentinelsForProviderEgress } from "./provider-secret-egress.js";
|
||||
|
||||
const log = createSubsystemLogger("provider-local-service");
|
||||
const DEFAULT_READY_TIMEOUT_MS = 120_000;
|
||||
@@ -222,6 +223,12 @@ async function probeHealth(
|
||||
signal?: AbortSignal | null,
|
||||
): Promise<boolean> {
|
||||
throwIfAborted(signal);
|
||||
// Local-service orchestration retains sentinel headers across retries. Only
|
||||
// the actual health request may materialize credentials.
|
||||
const egressHeaders = unwrapHeadersInitSentinelsForProviderEgress(
|
||||
headers,
|
||||
"to probe local model provider health",
|
||||
);
|
||||
const controller = new AbortController();
|
||||
const timeout = setTimeout(() => controller.abort(), DEFAULT_PROBE_TIMEOUT_MS);
|
||||
timeout.unref?.();
|
||||
@@ -229,7 +236,7 @@ async function probeHealth(
|
||||
signal?.addEventListener("abort", onAbort, { once: true });
|
||||
let response: Response | undefined;
|
||||
try {
|
||||
response = await fetch(url, { headers, signal: controller.signal });
|
||||
response = await fetch(url, { headers: egressHeaders, signal: controller.signal });
|
||||
return response.ok;
|
||||
} catch {
|
||||
if (signal?.aborted) {
|
||||
|
||||
@@ -0,0 +1,91 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { mintSecretSentinel } from "../secrets/sentinel.js";
|
||||
import {
|
||||
attachModelProviderRequestTransport,
|
||||
getModelProviderRequestTransport,
|
||||
} from "./provider-request-config.js";
|
||||
import { unwrapModelHeaderSentinelsForProviderEgress } from "./provider-secret-egress.js";
|
||||
|
||||
describe("unwrapModelHeaderSentinelsForProviderEgress", () => {
|
||||
it("unwraps sentinels in visible headers and attached request transport overrides", () => {
|
||||
const headerSecret = "egress-visible-header-secret";
|
||||
const bearerSecret = "egress-runtime-bearer-secret";
|
||||
const overrideHeaderSecret = "egress-override-header-secret";
|
||||
const model = attachModelProviderRequestTransport(
|
||||
{
|
||||
id: "test-model",
|
||||
headers: {
|
||||
"x-api-key": mintSecretSentinel(headerSecret, { label: "egress-test:visible" }),
|
||||
},
|
||||
},
|
||||
{
|
||||
headers: {
|
||||
"x-extra": mintSecretSentinel(overrideHeaderSecret, { label: "egress-test:override" }),
|
||||
},
|
||||
auth: {
|
||||
mode: "authorization-bearer",
|
||||
token: mintSecretSentinel(bearerSecret, { label: "egress-test:bearer" }),
|
||||
},
|
||||
},
|
||||
);
|
||||
|
||||
const unwrapped = unwrapModelHeaderSentinelsForProviderEgress(model, "egress test");
|
||||
|
||||
expect(unwrapped.headers?.["x-api-key"]).toBe(headerSecret);
|
||||
const request = getModelProviderRequestTransport(unwrapped);
|
||||
expect(request?.headers?.["x-extra"]).toBe(overrideHeaderSecret);
|
||||
expect(request?.auth).toEqual({ mode: "authorization-bearer", token: bearerSecret });
|
||||
// Original model stays sentineled: unwrap must not mutate shared state.
|
||||
expect(model.headers["x-api-key"]).not.toBe(headerSecret);
|
||||
expect(getModelProviderRequestTransport(model)?.auth).not.toEqual(request?.auth);
|
||||
});
|
||||
|
||||
it("unwraps header-mode auth values in attached request transport overrides", () => {
|
||||
const headerAuthSecret = "egress-header-auth-secret";
|
||||
const model = attachModelProviderRequestTransport(
|
||||
{ id: "test-model", headers: undefined },
|
||||
{
|
||||
auth: {
|
||||
mode: "header",
|
||||
headerName: "x-goog-api-key",
|
||||
value: mintSecretSentinel(headerAuthSecret, { label: "egress-test:header-auth" }),
|
||||
},
|
||||
},
|
||||
);
|
||||
|
||||
const request = getModelProviderRequestTransport(
|
||||
unwrapModelHeaderSentinelsForProviderEgress(model, "egress test"),
|
||||
);
|
||||
|
||||
expect(request?.auth).toEqual({
|
||||
mode: "header",
|
||||
headerName: "x-goog-api-key",
|
||||
value: headerAuthSecret,
|
||||
});
|
||||
});
|
||||
|
||||
it("returns the same model instance when nothing is sentineled", () => {
|
||||
const model = attachModelProviderRequestTransport(
|
||||
{ id: "test-model", headers: { "x-plain": "plain-value" } },
|
||||
{ auth: { mode: "provider-default" } },
|
||||
);
|
||||
|
||||
expect(unwrapModelHeaderSentinelsForProviderEgress(model, "egress test")).toBe(model);
|
||||
});
|
||||
|
||||
it("rejects unknown sentinel-shaped values in attached overrides", () => {
|
||||
const model = attachModelProviderRequestTransport(
|
||||
{ id: "test-model", headers: undefined },
|
||||
{
|
||||
auth: {
|
||||
mode: "authorization-bearer",
|
||||
token: "oc-sent-v1-00000000000000000000dead",
|
||||
},
|
||||
},
|
||||
);
|
||||
|
||||
expect(() => unwrapModelHeaderSentinelsForProviderEgress(model, "egress test")).toThrow(
|
||||
/not registered in this process/,
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,184 @@
|
||||
import { isSecretValueRegisteredForRedaction } from "../logging/secret-redaction-registry.js";
|
||||
import {
|
||||
looksLikeSecretSentinel,
|
||||
mintSecretSentinel,
|
||||
swapSecretSentinelsInText,
|
||||
} from "../secrets/sentinel.js";
|
||||
import {
|
||||
attachModelProviderRequestTransport,
|
||||
getModelProviderRequestTransport,
|
||||
type ModelProviderRequestTransportOverrides,
|
||||
} from "./provider-request-config.js";
|
||||
|
||||
type PreparedProviderRuntimeAuth = {
|
||||
apiKey: string;
|
||||
baseUrl?: string;
|
||||
request?: ModelProviderRequestTransportOverrides;
|
||||
expiresAt?: number;
|
||||
};
|
||||
|
||||
function protectRuntimeAuthValue(params: {
|
||||
value: string;
|
||||
provider: string;
|
||||
label: string;
|
||||
}): string {
|
||||
if (!params.value) {
|
||||
return params.value;
|
||||
}
|
||||
return looksLikeSecretSentinel(params.value)
|
||||
? params.value
|
||||
: mintSecretSentinel(params.value, {
|
||||
label: `model-auth:${params.provider}:${params.label}`,
|
||||
});
|
||||
}
|
||||
|
||||
/** Re-sentinels credentials returned by a provider auth exchange. */
|
||||
export function protectPreparedProviderRuntimeAuth(params: {
|
||||
sourceApiKey: string;
|
||||
provider: string;
|
||||
preparedAuth: PreparedProviderRuntimeAuth | null | undefined;
|
||||
}): PreparedProviderRuntimeAuth | undefined {
|
||||
const { preparedAuth } = params;
|
||||
if (!preparedAuth) {
|
||||
return undefined;
|
||||
}
|
||||
if (
|
||||
!looksLikeSecretSentinel(params.sourceApiKey) &&
|
||||
!isSecretValueRegisteredForRedaction(params.sourceApiKey)
|
||||
) {
|
||||
return preparedAuth;
|
||||
}
|
||||
const protect = (value: string, label: string) =>
|
||||
protectRuntimeAuthValue({ value, provider: params.provider, label });
|
||||
const request = preparedAuth.request;
|
||||
const headers = request?.headers
|
||||
? Object.fromEntries(
|
||||
Object.entries(request.headers).map(([name, value]) => [
|
||||
name,
|
||||
protect(value, `runtime-header:${name.toLowerCase()}`),
|
||||
]),
|
||||
)
|
||||
: undefined;
|
||||
const auth = request?.auth;
|
||||
const protectedAuth =
|
||||
auth?.mode === "authorization-bearer"
|
||||
? { ...auth, token: protect(auth.token, "runtime-bearer") }
|
||||
: auth?.mode === "header"
|
||||
? {
|
||||
...auth,
|
||||
value: protect(auth.value, `runtime-auth-header:${auth.headerName.toLowerCase()}`),
|
||||
}
|
||||
: auth;
|
||||
return {
|
||||
...preparedAuth,
|
||||
apiKey: protect(preparedAuth.apiKey, "runtime-api-key"),
|
||||
...(request
|
||||
? {
|
||||
request: {
|
||||
...request,
|
||||
...(headers ? { headers } : {}),
|
||||
...(protectedAuth ? { auth: protectedAuth } : {}),
|
||||
},
|
||||
}
|
||||
: {}),
|
||||
};
|
||||
}
|
||||
|
||||
export function unwrapSecretSentinelsForProviderEgress(value: string, boundary: string): string {
|
||||
const swapped = swapSecretSentinelsInText(value);
|
||||
const unknown = swapped.unknown[0];
|
||||
if (unknown) {
|
||||
throw new Error(
|
||||
`Secret sentinel ${unknown} is not registered in this process; refusing ${boundary}`,
|
||||
);
|
||||
}
|
||||
return swapped.text;
|
||||
}
|
||||
|
||||
export function unwrapHeaderSentinelsForProviderEgress<T extends Record<string, unknown>>(
|
||||
input: T,
|
||||
boundary: string,
|
||||
): T {
|
||||
let headers: Record<string, unknown> | undefined;
|
||||
for (const [name, value] of Object.entries(input)) {
|
||||
if (typeof value !== "string") {
|
||||
continue;
|
||||
}
|
||||
const resolved = unwrapSecretSentinelsForProviderEgress(value, boundary);
|
||||
if (resolved !== value) {
|
||||
headers ??= { ...input };
|
||||
headers[name] = resolved;
|
||||
}
|
||||
}
|
||||
return headers ? (headers as T) : input;
|
||||
}
|
||||
|
||||
export function unwrapHeadersInitSentinelsForProviderEgress(
|
||||
input: HeadersInit | undefined,
|
||||
boundary: string,
|
||||
): HeadersInit | undefined {
|
||||
if (!input) {
|
||||
return input;
|
||||
}
|
||||
const headers = new Headers(input);
|
||||
let changed = false;
|
||||
for (const [name, value] of headers) {
|
||||
const resolved = unwrapSecretSentinelsForProviderEgress(value, boundary);
|
||||
if (resolved !== value) {
|
||||
headers.set(name, resolved);
|
||||
changed = true;
|
||||
}
|
||||
}
|
||||
return changed ? headers : input;
|
||||
}
|
||||
|
||||
function unwrapRequestTransportSentinelsForProviderEgress(
|
||||
request: ModelProviderRequestTransportOverrides | undefined,
|
||||
boundary: string,
|
||||
): ModelProviderRequestTransportOverrides | undefined {
|
||||
if (!request) {
|
||||
return request;
|
||||
}
|
||||
const headers = request.headers
|
||||
? unwrapHeaderSentinelsForProviderEgress(request.headers, boundary)
|
||||
: request.headers;
|
||||
let auth = request.auth;
|
||||
if (auth?.mode === "authorization-bearer") {
|
||||
const token = unwrapSecretSentinelsForProviderEgress(auth.token, boundary);
|
||||
if (token !== auth.token) {
|
||||
auth = { ...auth, token };
|
||||
}
|
||||
} else if (auth?.mode === "header") {
|
||||
const value = unwrapSecretSentinelsForProviderEgress(auth.value, boundary);
|
||||
if (value !== auth.value) {
|
||||
auth = { ...auth, value };
|
||||
}
|
||||
}
|
||||
if (headers === request.headers && auth === request.auth) {
|
||||
return request;
|
||||
}
|
||||
return {
|
||||
...request,
|
||||
...(headers ? { headers } : {}),
|
||||
...(auth ? { auth } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
export function unwrapModelHeaderSentinelsForProviderEgress<
|
||||
T extends { headers?: Record<string, unknown> },
|
||||
>(model: T, boundary: string): T {
|
||||
// Plugin transports read both visible headers and the symbol-attached request
|
||||
// overrides; both can carry sentinels minted by protectPreparedProviderRuntimeAuth.
|
||||
const headers = model.headers
|
||||
? unwrapHeaderSentinelsForProviderEgress(model.headers, boundary)
|
||||
: model.headers;
|
||||
const request = getModelProviderRequestTransport(model);
|
||||
const unwrappedRequest = unwrapRequestTransportSentinelsForProviderEgress(request, boundary);
|
||||
if (headers === model.headers && unwrappedRequest === request) {
|
||||
return model;
|
||||
}
|
||||
const next = headers === model.headers ? ({ ...model } as T) : ({ ...model, headers } as T);
|
||||
return unwrappedRequest === request
|
||||
? next
|
||||
: attachModelProviderRequestTransport(next, unwrappedRequest);
|
||||
}
|
||||
@@ -7,6 +7,11 @@ import type { OpenClawConfig } from "../config/types.openclaw.js";
|
||||
import type { Api, Model } from "../llm/types.js";
|
||||
import { resolveProviderStreamFn } from "../plugins/provider-runtime.js";
|
||||
import { ensureCustomApiRegistered } from "./custom-api-registry.js";
|
||||
import {
|
||||
unwrapHeaderSentinelsForProviderEgress,
|
||||
unwrapModelHeaderSentinelsForProviderEgress,
|
||||
unwrapSecretSentinelsForProviderEgress,
|
||||
} from "./provider-secret-egress.js";
|
||||
import { createTransportAwareStreamFnForModel } from "./provider-transport-stream.js";
|
||||
import type { StreamFn } from "./runtime/index.js";
|
||||
|
||||
@@ -19,28 +24,43 @@ export function registerProviderStreamForModel<TApi extends Api>(params: {
|
||||
env?: NodeJS.ProcessEnv;
|
||||
allowRuntimePluginLoad?: boolean;
|
||||
}): StreamFn | undefined {
|
||||
const streamFn =
|
||||
resolveProviderStreamFn({
|
||||
provider: params.model.provider,
|
||||
// Plugin stream factories may capture model headers, so construction is the
|
||||
// last safe boundary for providers that do not expose the host fetch seam.
|
||||
const pluginModel = unwrapModelHeaderSentinelsForProviderEgress(
|
||||
params.model,
|
||||
"plugin provider stream construction",
|
||||
);
|
||||
const providerStreamFn = resolveProviderStreamFn({
|
||||
provider: params.model.provider,
|
||||
config: params.cfg,
|
||||
workspaceDir: params.workspaceDir,
|
||||
env: params.env,
|
||||
allowRuntimePluginLoad: params.allowRuntimePluginLoad,
|
||||
context: {
|
||||
config: params.cfg,
|
||||
workspaceDir: params.workspaceDir,
|
||||
env: params.env,
|
||||
allowRuntimePluginLoad: params.allowRuntimePluginLoad,
|
||||
context: {
|
||||
config: params.cfg,
|
||||
agentDir: params.agentDir,
|
||||
workspaceDir: params.workspaceDir,
|
||||
provider: params.model.provider,
|
||||
modelId: params.model.id,
|
||||
model: params.model,
|
||||
},
|
||||
}) ??
|
||||
createTransportAwareStreamFnForModel(params.model, {
|
||||
cfg: params.cfg,
|
||||
agentDir: params.agentDir,
|
||||
workspaceDir: params.workspaceDir,
|
||||
env: params.env,
|
||||
});
|
||||
provider: params.model.provider,
|
||||
modelId: params.model.id,
|
||||
model: pluginModel,
|
||||
},
|
||||
});
|
||||
const transportFallback = providerStreamFn
|
||||
? undefined
|
||||
: createTransportAwareStreamFnForModel(
|
||||
params.model.api === "google-generative-ai" ? pluginModel : params.model,
|
||||
{
|
||||
cfg: params.cfg,
|
||||
agentDir: params.agentDir,
|
||||
workspaceDir: params.workspaceDir,
|
||||
env: params.env,
|
||||
},
|
||||
);
|
||||
const streamFn = providerStreamFn
|
||||
? wrapPluginProviderStream(providerStreamFn)
|
||||
: transportFallback && params.model.api === "google-generative-ai"
|
||||
? wrapPluginProviderStream(transportFallback)
|
||||
: transportFallback;
|
||||
if (!streamFn) {
|
||||
return undefined;
|
||||
}
|
||||
@@ -49,3 +69,24 @@ export function registerProviderStreamForModel<TApi extends Api>(params: {
|
||||
ensureCustomApiRegistered(params.model.api, streamFn);
|
||||
return streamFn;
|
||||
}
|
||||
|
||||
function wrapPluginProviderStream(streamFn: StreamFn): StreamFn {
|
||||
const boundary = "plugin provider stream handoff";
|
||||
return (model, context, options) => {
|
||||
const apiKey = options?.apiKey
|
||||
? unwrapSecretSentinelsForProviderEgress(options.apiKey, boundary)
|
||||
: options?.apiKey;
|
||||
const headers = options?.headers
|
||||
? unwrapHeaderSentinelsForProviderEgress(options.headers, boundary)
|
||||
: options?.headers;
|
||||
const resolvedOptions =
|
||||
apiKey === options?.apiKey && headers === options?.headers
|
||||
? options
|
||||
: { ...options, apiKey, headers };
|
||||
return streamFn(
|
||||
unwrapModelHeaderSentinelsForProviderEgress(model, boundary),
|
||||
context,
|
||||
resolvedOptions,
|
||||
);
|
||||
};
|
||||
}
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
import { createServer } from "node:http";
|
||||
import type { AddressInfo } from "node:net";
|
||||
import type { Model } from "openclaw/plugin-sdk/llm";
|
||||
import { afterEach, describe, expect, it } from "vitest";
|
||||
import { redactSensitiveText } from "../logging/redact.js";
|
||||
import { resetSecretRedactionRegistryForTest } from "../logging/secret-redaction-registry.js";
|
||||
import { mintSecretSentinel } from "../secrets/sentinel.js";
|
||||
import { buildGuardedModelFetch } from "./provider-transport-fetch.js";
|
||||
|
||||
describe("guarded model fetch secret sentinel integration", () => {
|
||||
afterEach(() => {
|
||||
resetSecretRedactionRegistryForTest();
|
||||
});
|
||||
|
||||
it("injects the real header only at local HTTP egress and redacts the resolved value", async () => {
|
||||
let receivedAuthorization: string | undefined;
|
||||
const server = createServer((request, response) => {
|
||||
receivedAuthorization = request.headers.authorization;
|
||||
response.writeHead(200, { "content-type": "application/json" });
|
||||
response.end('{"ok":true}');
|
||||
});
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
server.once("error", reject);
|
||||
server.listen(0, "127.0.0.1", resolve);
|
||||
});
|
||||
|
||||
try {
|
||||
const port = (server.address() as AddressInfo).port;
|
||||
const baseUrl = `http://127.0.0.1:${port}/v1`;
|
||||
const model = {
|
||||
id: "integration-model",
|
||||
provider: "sentinel-integration",
|
||||
api: "openai-responses",
|
||||
baseUrl,
|
||||
} as unknown as Model<"openai-responses">;
|
||||
const secret = "integration-provider-secret";
|
||||
const sentinel = mintSecretSentinel(secret, { label: "model-auth:integration" });
|
||||
|
||||
const response = await buildGuardedModelFetch(model)(`${baseUrl}/responses`, {
|
||||
method: "POST",
|
||||
headers: { Authorization: `Bearer ${sentinel}` },
|
||||
body: "{}",
|
||||
});
|
||||
await response.text();
|
||||
|
||||
expect(receivedAuthorization).toBe(`Bearer ${secret}`);
|
||||
expect(redactSensitiveText(`upstream used ${secret}`, { mode: "off" })).toBe(
|
||||
"upstream used integr…cret",
|
||||
);
|
||||
} finally {
|
||||
await new Promise<void>((resolve) => {
|
||||
server.close(() => resolve());
|
||||
});
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -3,6 +3,7 @@ import { MAX_TIMER_TIMEOUT_MS } from "@openclaw/normalization-core/number-coerci
|
||||
import { Stream } from "openai/streaming";
|
||||
import type { Model } from "openclaw/plugin-sdk/llm";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { mintSecretSentinel } from "../secrets/sentinel.js";
|
||||
import { buildGuardedModelFetch } from "./provider-transport-fetch.js";
|
||||
|
||||
type ProviderRequestPolicyConfigMockResult = {
|
||||
@@ -177,6 +178,110 @@ describe("buildGuardedModelFetch", () => {
|
||||
delete process.env.OPENCLAW_SDK_RETRY_MAX_WAIT_SECONDS;
|
||||
});
|
||||
|
||||
function sentinelModel(): Model<"openai-responses"> {
|
||||
return {
|
||||
id: "gpt-5.5",
|
||||
provider: "openai",
|
||||
api: "openai-responses",
|
||||
baseUrl: "https://api.openai.com/v1",
|
||||
} as unknown as Model<"openai-responses">;
|
||||
}
|
||||
|
||||
it("swaps sentinels in Request-form headers", async () => {
|
||||
const sentinel = mintSecretSentinel("request-form-secret", { label: "request-form" });
|
||||
const request = new Request("https://api.openai.com/v1/responses", {
|
||||
method: "POST",
|
||||
headers: { Authorization: `Bearer ${sentinel}` },
|
||||
});
|
||||
|
||||
const response = await buildGuardedModelFetch(sentinelModel())(request);
|
||||
await response.text();
|
||||
|
||||
const headers = new Headers((latestGuardedFetchParams().init as RequestInit).headers);
|
||||
expect(headers.get("authorization")).toBe("Bearer request-form-secret");
|
||||
});
|
||||
|
||||
it("swaps sentinels in record init headers", async () => {
|
||||
const recordSentinel = mintSecretSentinel("record-header-secret", { label: "record-header" });
|
||||
const response = await buildGuardedModelFetch(sentinelModel())(
|
||||
"https://api.openai.com/v1/responses",
|
||||
{
|
||||
headers: { "x-api-key": recordSentinel },
|
||||
},
|
||||
);
|
||||
await response.text();
|
||||
expect(
|
||||
new Headers((latestGuardedFetchParams().init as RequestInit).headers).get("x-api-key"),
|
||||
).toBe("record-header-secret");
|
||||
expect(
|
||||
new Headers(ensureModelProviderLocalServiceMock.mock.calls[0]?.[1] as HeadersInit).get(
|
||||
"x-api-key",
|
||||
),
|
||||
).toBe(recordSentinel);
|
||||
});
|
||||
|
||||
it("swaps sentinels in tuple init headers", async () => {
|
||||
const tupleSentinel = mintSecretSentinel("tuple-header-secret", { label: "tuple-header" });
|
||||
const response = await buildGuardedModelFetch(sentinelModel())(
|
||||
"https://api.openai.com/v1/responses",
|
||||
{
|
||||
headers: [["x-api-key", tupleSentinel]],
|
||||
},
|
||||
);
|
||||
await response.text();
|
||||
expect(
|
||||
new Headers((latestGuardedFetchParams().init as RequestInit).headers).get("x-api-key"),
|
||||
).toBe("tuple-header-secret");
|
||||
});
|
||||
|
||||
it("swaps sentinels in Headers init and composed Cloudflare auth values", async () => {
|
||||
const sentinel = mintSecretSentinel("cloudflare-upstream-secret", { label: "cloudflare" });
|
||||
const response = await buildGuardedModelFetch(sentinelModel())(
|
||||
"https://api.openai.com/v1/responses",
|
||||
{
|
||||
headers: new Headers({ "cf-aig-authorization": `Bearer ${sentinel}` }),
|
||||
},
|
||||
);
|
||||
await response.text();
|
||||
|
||||
const headers = new Headers((latestGuardedFetchParams().init as RequestInit).headers);
|
||||
expect(headers.get("cf-aig-authorization")).toBe("Bearer cloudflare-upstream-secret");
|
||||
});
|
||||
|
||||
it("swaps sentinels in URL query parameters", async () => {
|
||||
const sentinel = mintSecretSentinel("gemini&scope=two+#%", { label: "gemini-query" });
|
||||
const response = await buildGuardedModelFetch(sentinelModel())(
|
||||
`https://api.openai.com/v1/responses?key=${sentinel}`,
|
||||
);
|
||||
await response.text();
|
||||
|
||||
expect(latestGuardedFetchParams().url).toBe(
|
||||
"https://api.openai.com/v1/responses?key=gemini%26scope%3Dtwo%2B%23%25",
|
||||
);
|
||||
});
|
||||
|
||||
it("rejects unknown sentinel-shaped values before guarded fetch", async () => {
|
||||
const unknown = "oc-sent-v1-fedcba987654321001234567";
|
||||
await expect(
|
||||
buildGuardedModelFetch(sentinelModel())("https://api.openai.com/v1/responses", {
|
||||
headers: { Authorization: `Bearer ${unknown}` },
|
||||
}),
|
||||
).rejects.toThrow(
|
||||
`Secret sentinel ${unknown} is not registered in this process; refusing to send request`,
|
||||
);
|
||||
expect(fetchWithSsrFGuardMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("keeps the no-sentinel fast path request init untouched", async () => {
|
||||
const init: RequestInit = { headers: { Authorization: "Bearer plain-env-key" } };
|
||||
const response = await buildGuardedModelFetch(sentinelModel())(
|
||||
"https://api.openai.com/v1/responses",
|
||||
init,
|
||||
);
|
||||
await response.text();
|
||||
expect(latestGuardedFetchParams().init).toStrictEqual(init);
|
||||
});
|
||||
|
||||
it("pushes provider capture metadata into the shared guarded fetch seam", async () => {
|
||||
const model = {
|
||||
id: "gpt-5.4",
|
||||
@@ -1592,6 +1697,47 @@ describe("buildGuardedModelFetch", () => {
|
||||
expect(text.length).toBeLessThan(OVER_LIMIT);
|
||||
});
|
||||
|
||||
it("returns a capped body before guarded cleanup finishes", async () => {
|
||||
const OVER_LIMIT = 100 * 1024;
|
||||
let finishRelease!: () => void;
|
||||
const releasePending = new Promise<void>((resolve) => {
|
||||
finishRelease = resolve;
|
||||
});
|
||||
const release = vi.fn(() => releasePending);
|
||||
fetchWithSsrFGuardMock.mockResolvedValue({
|
||||
response: new Response(new Uint8Array(OVER_LIMIT), {
|
||||
status: 429,
|
||||
statusText: "Too Many Requests",
|
||||
}),
|
||||
finalUrl: "https://custom-azure.openai.azure.com/openai/v1/responses",
|
||||
release,
|
||||
});
|
||||
const model = {
|
||||
id: "gpt-5.5",
|
||||
provider: "azure",
|
||||
api: "azure-openai-responses",
|
||||
baseUrl: "https://custom-azure.openai.azure.com/openai/v1",
|
||||
} as unknown as Model<"azure-openai-responses">;
|
||||
|
||||
const response = await buildGuardedModelFetch(model)(
|
||||
"https://custom-azure.openai.azure.com/openai/v1/responses",
|
||||
{ method: "POST" },
|
||||
);
|
||||
const timeout = Symbol("timeout");
|
||||
let timeoutHandle: ReturnType<typeof setTimeout> | undefined;
|
||||
const result = await Promise.race([
|
||||
response.text(),
|
||||
new Promise<typeof timeout>((resolve) => {
|
||||
timeoutHandle = setTimeout(() => resolve(timeout), 100);
|
||||
}),
|
||||
]);
|
||||
clearTimeout(timeoutHandle);
|
||||
finishRelease();
|
||||
|
||||
expect(result).not.toBe(timeout);
|
||||
expect(release).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("preserves SDK ability to cancel retryable non-OK responses before reading body", async () => {
|
||||
// Regression: a non-OK body wrapper must be lazy. The OpenAI SDK may decide
|
||||
// to cancel a 429/5xx response and retry before reading the body. If the
|
||||
|
||||
@@ -29,6 +29,11 @@ import {
|
||||
import type { Model } from "../llm/types.js";
|
||||
import { createSubsystemLogger } from "../logging/subsystem.js";
|
||||
import { resolveDebugProxySettings } from "../proxy-capture/env.js";
|
||||
import {
|
||||
resolveSecretSentinel,
|
||||
SECRET_SENTINEL_PATTERN,
|
||||
swapSecretSentinelsInText,
|
||||
} from "../secrets/sentinel.js";
|
||||
import { emitModelTransportDebug } from "./model-transport-debug.js";
|
||||
import { formatModelTransportDebugUrl } from "./model-transport-url.js";
|
||||
import { ProviderHttpError, readResponseTextLimited } from "./provider-http-errors.js";
|
||||
@@ -103,25 +108,42 @@ function capNonOkResponseBodyLazily(response: Response, maxBytes: number): Respo
|
||||
if (!source) {
|
||||
return response;
|
||||
}
|
||||
let reader: ReadableStreamDefaultReader<Uint8Array> | undefined;
|
||||
let total = 0;
|
||||
const capped = source.pipeThrough(
|
||||
new TransformStream<Uint8Array, Uint8Array>({
|
||||
transform(chunk, controller) {
|
||||
const nextTotal = total + chunk.byteLength;
|
||||
if (nextTotal > maxBytes) {
|
||||
const remaining = maxBytes - total;
|
||||
if (remaining > 0) {
|
||||
controller.enqueue(chunk.subarray(0, remaining));
|
||||
}
|
||||
total = maxBytes;
|
||||
controller.terminate();
|
||||
// Own the reader: Node can leak an internal pipeThrough writer rejection when
|
||||
// downstream cancellation races the cap terminating the transform.
|
||||
const capped = new ReadableStream<Uint8Array>({
|
||||
start() {
|
||||
reader = source.getReader();
|
||||
},
|
||||
async pull(controller) {
|
||||
try {
|
||||
const chunk = await reader?.read();
|
||||
if (!chunk || chunk.done) {
|
||||
controller.close();
|
||||
return;
|
||||
}
|
||||
total = nextTotal;
|
||||
controller.enqueue(chunk);
|
||||
},
|
||||
}),
|
||||
);
|
||||
const remaining = maxBytes - total;
|
||||
if (chunk.value.byteLength > remaining) {
|
||||
if (remaining > 0) {
|
||||
controller.enqueue(chunk.value.subarray(0, remaining));
|
||||
}
|
||||
total = maxBytes;
|
||||
controller.close();
|
||||
void reader?.cancel().catch(() => undefined);
|
||||
return;
|
||||
}
|
||||
total += chunk.value.byteLength;
|
||||
controller.enqueue(chunk.value);
|
||||
} catch (error) {
|
||||
controller.error(error);
|
||||
void reader?.cancel(error).catch(() => undefined);
|
||||
}
|
||||
},
|
||||
async cancel(reason) {
|
||||
await reader?.cancel(reason).catch(() => undefined);
|
||||
},
|
||||
});
|
||||
return new Response(capped, response);
|
||||
}
|
||||
|
||||
@@ -743,6 +765,63 @@ export function resolveProviderTransportSsrFPolicy(params: {
|
||||
);
|
||||
}
|
||||
|
||||
function headersContainSecretSentinel(headers: HeadersInit | undefined): boolean {
|
||||
if (!headers) {
|
||||
return false;
|
||||
}
|
||||
for (const value of new Headers(headers).values()) {
|
||||
if (value.includes("oc-sent-v1-")) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
function swapSecretSentinelsInUrl(url: string): { text: string; unknown: string[] } {
|
||||
if (!url.includes("oc-sent-v1-")) {
|
||||
return { text: url, unknown: [] };
|
||||
}
|
||||
const unknown = new Set<string>();
|
||||
const text = url.replace(new RegExp(SECRET_SENTINEL_PATTERN.source, "g"), (sentinel) => {
|
||||
const value = resolveSecretSentinel(sentinel);
|
||||
if (value === undefined) {
|
||||
unknown.add(sentinel);
|
||||
return sentinel;
|
||||
}
|
||||
// Sentinels are URL-safe placeholders. Encode the real bytes so query/path structure is stable.
|
||||
return encodeURIComponent(value);
|
||||
});
|
||||
return { text, unknown: [...unknown] };
|
||||
}
|
||||
|
||||
function swapSecretSentinelsForEgress(params: { url: string; headers?: HeadersInit }): {
|
||||
url: string;
|
||||
headers?: Headers;
|
||||
} {
|
||||
if (!params.url.includes("oc-sent-v1-") && !headersContainSecretSentinel(params.headers)) {
|
||||
return { url: params.url };
|
||||
}
|
||||
const urlSwap = swapSecretSentinelsInUrl(params.url);
|
||||
const headers = params.headers ? new Headers(params.headers) : undefined;
|
||||
const unknown = new Set(urlSwap.unknown);
|
||||
if (headers) {
|
||||
for (const [name, value] of headers.entries()) {
|
||||
const swapped = swapSecretSentinelsInText(value);
|
||||
headers.set(name, swapped.text);
|
||||
for (const sentinel of swapped.unknown) {
|
||||
unknown.add(sentinel);
|
||||
}
|
||||
}
|
||||
}
|
||||
const unresolved = unknown.values().next().value;
|
||||
if (unresolved) {
|
||||
throw new Error(
|
||||
`Secret sentinel ${unresolved} is not registered in this process; refusing to send request`,
|
||||
);
|
||||
}
|
||||
return { url: urlSwap.text, ...(headers ? { headers } : {}) };
|
||||
}
|
||||
|
||||
export function buildGuardedModelFetch(
|
||||
model: Model,
|
||||
timeoutMs?: number,
|
||||
@@ -772,7 +851,7 @@ export function buildGuardedModelFetch(
|
||||
return async (input, init) => {
|
||||
let localServiceLease: ProviderLocalServiceLease | undefined;
|
||||
const request = input instanceof Request ? new Request(input, init) : undefined;
|
||||
const url =
|
||||
const rawUrl =
|
||||
request?.url ??
|
||||
(input instanceof URL
|
||||
? input.toString()
|
||||
@@ -781,6 +860,12 @@ export function buildGuardedModelFetch(
|
||||
: (() => {
|
||||
throw new Error("Unsupported fetch input for transport-aware model request");
|
||||
})());
|
||||
const rawHeaders = request?.headers ?? init?.headers;
|
||||
const swappedEgress = swapSecretSentinelsForEgress({
|
||||
url: rawUrl,
|
||||
headers: rawHeaders,
|
||||
});
|
||||
const url = swappedEgress.url;
|
||||
const policy = resolveProviderTransportSsrFPolicy({
|
||||
baseUrl: model.baseUrl,
|
||||
url,
|
||||
@@ -796,13 +881,15 @@ export function buildGuardedModelFetch(
|
||||
request &&
|
||||
({
|
||||
method: request.method,
|
||||
headers: request.headers,
|
||||
headers: swappedEgress.headers ?? request.headers,
|
||||
body: request.body ?? undefined,
|
||||
redirect: request.redirect,
|
||||
signal: request.signal,
|
||||
...(request.body ? ({ duplex: "half" } as const) : {}),
|
||||
} satisfies RequestInit & { duplex?: "half" });
|
||||
const baseInit = requestInit ?? init;
|
||||
const baseInit =
|
||||
requestInit ??
|
||||
(swappedEgress.headers && init ? { ...init, headers: swappedEgress.headers } : init);
|
||||
const synthesizeJsonAsSse = await requestBodyHasStreamTrue(request, baseInit);
|
||||
const baseSignal = baseInit?.signal ?? undefined;
|
||||
const localServiceSignal = buildModelRequestSignal(baseSignal, requestTimeoutMs);
|
||||
@@ -830,14 +917,15 @@ export function buildGuardedModelFetch(
|
||||
emitModelTransportDebug(
|
||||
log,
|
||||
`[model-fetch] start provider=${model.provider} api=${model.api} model=${model.id} ` +
|
||||
`method=${baseInit?.method ?? "GET"} url=${formatModelTransportDebugUrl(url)} timeoutMs=${requestTimeoutMs} ` +
|
||||
// Log the pre-swap URL: the swapped URL can carry an injected credential in its path.
|
||||
`method=${baseInit?.method ?? "GET"} url=${formatModelTransportDebugUrl(rawUrl)} timeoutMs=${requestTimeoutMs} ` +
|
||||
`proxy=${dispatcherPolicy ? "configured" : useEnvProxy ? "env" : "none"} ` +
|
||||
`policy=${policy ? "custom" : "default"}`,
|
||||
);
|
||||
try {
|
||||
localServiceLease = await ensureModelProviderLocalService(
|
||||
model,
|
||||
baseInit?.headers,
|
||||
rawHeaders,
|
||||
localServiceSignal,
|
||||
);
|
||||
result = await fetchWithSsrFGuard(
|
||||
|
||||
@@ -3,6 +3,11 @@
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import type { OpenClawConfig } from "../config/types.openclaw.js";
|
||||
import type { Model } from "../llm/types.js";
|
||||
import {
|
||||
looksLikeSecretSentinel,
|
||||
mintSecretSentinel,
|
||||
resolveSecretSentinel,
|
||||
} from "../secrets/sentinel.js";
|
||||
|
||||
// Hoisted mocks keep Vitest module replacement stable while the implementation
|
||||
// under test imports auth, model resolution, and transport helpers at module load.
|
||||
@@ -32,6 +37,7 @@ vi.mock("./simple-completion-transport.js", () => ({
|
||||
}));
|
||||
|
||||
vi.mock("./model-auth.js", () => ({
|
||||
applySecretRefHeaderSentinels: (model: unknown) => model,
|
||||
formatMissingAuthError: vi.fn(
|
||||
(auth: { source: string; mode: string }, provider: string) =>
|
||||
`No API key resolved for provider "${provider}" (auth mode: ${auth.mode}, checked: ${auth.source}).`,
|
||||
@@ -281,6 +287,34 @@ describe("prepareSimpleCompletionModel", () => {
|
||||
expect(result.auth.apiKey).not.toBe("ghu_original_github_token");
|
||||
});
|
||||
|
||||
it("keeps an exchanged Copilot token opaque when its source is a sentinel", async () => {
|
||||
const sourceSecret = "github-source-secret";
|
||||
const sourceSentinel = mintSecretSentinel(sourceSecret, {
|
||||
label: "model-auth:github-copilot",
|
||||
});
|
||||
hoisted.resolveModelMock.mockReturnValueOnce({
|
||||
model: { provider: "github-copilot", id: "gpt-4.1" },
|
||||
authStorage: { setRuntimeApiKey: hoisted.setRuntimeApiKeyMock },
|
||||
modelRegistry: {},
|
||||
});
|
||||
hoisted.getApiKeyForModelMock.mockResolvedValueOnce({
|
||||
apiKey: sourceSentinel,
|
||||
source: "profile:github-copilot:default",
|
||||
mode: "token",
|
||||
});
|
||||
|
||||
const result = await prepareSimpleCompletionModel({
|
||||
cfg: undefined,
|
||||
provider: "github-copilot",
|
||||
modelId: "gpt-4.1",
|
||||
});
|
||||
|
||||
expect(hoisted.resolveCopilotApiTokenMock).toHaveBeenCalledWith({ githubToken: sourceSecret });
|
||||
expectPreparedModelResult(result);
|
||||
expect(looksLikeSecretSentinel(result.auth.apiKey ?? "")).toBe(true);
|
||||
expect(resolveSecretSentinel(result.auth.apiKey ?? "")).toBe("copilot-runtime-token");
|
||||
});
|
||||
|
||||
it("applies exchanged copilot baseUrl to returned model", async () => {
|
||||
hoisted.resolveModelMock.mockReturnValueOnce({
|
||||
model: {
|
||||
|
||||
@@ -25,6 +25,7 @@ import { DEFAULT_PROVIDER } from "./defaults.js";
|
||||
import { resolveModel, resolveModelAsync } from "./embedded-agent-runner/model.js";
|
||||
import { resolveAgentHarnessPolicy } from "./harness/policy.js";
|
||||
import {
|
||||
applySecretRefHeaderSentinels,
|
||||
applyLocalNoAuthHeaderOverride,
|
||||
formatMissingAuthError,
|
||||
getApiKeyForModel,
|
||||
@@ -38,6 +39,10 @@ import {
|
||||
} from "./model-selection.js";
|
||||
import { OPENAI_PROVIDER_ID, isOpenAIProvider } from "./openai-routing.js";
|
||||
import { applyPreparedRuntimeAuthToModel } from "./provider-request-config.js";
|
||||
import {
|
||||
protectPreparedProviderRuntimeAuth,
|
||||
unwrapSecretSentinelsForProviderEgress,
|
||||
} from "./provider-secret-egress.js";
|
||||
import { prepareModelForSimpleCompletion } from "./simple-completion-transport.js";
|
||||
|
||||
type SimpleCompletionAuthStorage = {
|
||||
@@ -168,30 +173,49 @@ async function setRuntimeApiKeyForCompletion(params: {
|
||||
if (params.model.provider === "github-copilot") {
|
||||
const { resolveCopilotApiToken } = await import("../plugin-sdk/provider-auth.js");
|
||||
const copilotToken = await resolveCopilotApiToken({
|
||||
githubToken: params.apiKey,
|
||||
githubToken: unwrapSecretSentinelsForProviderEgress(
|
||||
params.apiKey,
|
||||
"GitHub Copilot runtime auth exchange",
|
||||
),
|
||||
});
|
||||
params.authStorage.setRuntimeApiKey(params.model.provider, copilotToken.token);
|
||||
const protectedAuth = protectPreparedProviderRuntimeAuth({
|
||||
sourceApiKey: params.apiKey,
|
||||
provider: params.model.provider,
|
||||
preparedAuth: {
|
||||
apiKey: copilotToken.token,
|
||||
baseUrl: copilotToken.baseUrl,
|
||||
},
|
||||
});
|
||||
const runtimeApiKey = protectedAuth?.apiKey ?? copilotToken.token;
|
||||
params.authStorage.setRuntimeApiKey(params.model.provider, runtimeApiKey);
|
||||
return {
|
||||
apiKey: copilotToken.token,
|
||||
apiKey: runtimeApiKey,
|
||||
model: { ...params.model, baseUrl: copilotToken.baseUrl },
|
||||
};
|
||||
}
|
||||
const preparedAuth = await prepareProviderRuntimeAuth({
|
||||
const preparedAuth = protectPreparedProviderRuntimeAuth({
|
||||
sourceApiKey: params.apiKey,
|
||||
provider: params.model.provider,
|
||||
config: params.cfg,
|
||||
workspaceDir: params.workspaceDir,
|
||||
env: process.env,
|
||||
context: {
|
||||
preparedAuth: await prepareProviderRuntimeAuth({
|
||||
provider: params.model.provider,
|
||||
config: params.cfg,
|
||||
workspaceDir: params.workspaceDir,
|
||||
env: process.env,
|
||||
provider: params.model.provider,
|
||||
modelId: params.model.id,
|
||||
model: params.model,
|
||||
apiKey: params.apiKey,
|
||||
authMode: params.authMode,
|
||||
profileId: params.profileId,
|
||||
},
|
||||
context: {
|
||||
config: params.cfg,
|
||||
workspaceDir: params.workspaceDir,
|
||||
env: process.env,
|
||||
provider: params.model.provider,
|
||||
modelId: params.model.id,
|
||||
model: params.model,
|
||||
apiKey: unwrapSecretSentinelsForProviderEgress(
|
||||
params.apiKey,
|
||||
"provider runtime auth exchange",
|
||||
),
|
||||
authMode: params.authMode,
|
||||
profileId: params.profileId,
|
||||
},
|
||||
}),
|
||||
});
|
||||
const runtimeApiKey = preparedAuth?.apiKey?.trim() || params.apiKey;
|
||||
params.authStorage.setRuntimeApiKey(params.model.provider, runtimeApiKey);
|
||||
@@ -255,6 +279,7 @@ export async function prepareSimpleCompletionModel(params: {
|
||||
agentDir: params.agentDir,
|
||||
profileId: params.profileId,
|
||||
preferredProfile: params.preferredProfile,
|
||||
secretSentinels: true,
|
||||
});
|
||||
} catch (err) {
|
||||
return {
|
||||
@@ -297,7 +322,10 @@ export async function prepareSimpleCompletionModel(params: {
|
||||
};
|
||||
|
||||
return {
|
||||
model: applyLocalNoAuthHeaderOverride(resolvedModel, resolvedAuth),
|
||||
model: applySecretRefHeaderSentinels(
|
||||
applyLocalNoAuthHeaderOverride(resolvedModel, resolvedAuth),
|
||||
params.cfg,
|
||||
),
|
||||
auth: resolvedAuth,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -5,6 +5,8 @@ import type { Model } from "openclaw/plugin-sdk/llm";
|
||||
import { afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import type { OpenClawConfig } from "../config/config.js";
|
||||
import { createMoonshotThinkingWrapper } from "../llm/providers/stream-wrappers/moonshot-thinking.js";
|
||||
import { mintSecretSentinel } from "../secrets/sentinel.js";
|
||||
import type { StreamFn } from "./runtime/index.js";
|
||||
|
||||
const createAnthropicVertexStreamFnForModel = vi.fn();
|
||||
const ensureCustomApiRegistered = vi.fn();
|
||||
@@ -16,6 +18,7 @@ const createTransportAwareStreamFnForModel = vi.fn();
|
||||
const prepareTransportAwareSimpleModel = vi.fn();
|
||||
const resolveTransportAwareSimpleApi = vi.fn();
|
||||
const prepareGoogleSimpleCompletionModel = vi.fn((model: unknown) => model);
|
||||
const pluginStreamFn = vi.fn(() => "plugin-stream-result" as never);
|
||||
|
||||
vi.mock("./anthropic-vertex-stream.js", () => ({
|
||||
createAnthropicVertexStreamFnForModel,
|
||||
@@ -62,6 +65,7 @@ describe("prepareModelForSimpleCompletion", () => {
|
||||
createAnthropicVertexStreamFnForModel.mockReset();
|
||||
ensureCustomApiRegistered.mockReset();
|
||||
resolveProviderStreamFn.mockReset();
|
||||
pluginStreamFn.mockClear();
|
||||
wrapProviderSimpleCompletionStreamFn.mockReset();
|
||||
buildTransportAwareSimpleStreamFn.mockReset();
|
||||
createOpenClawTransportStreamFnForModel.mockReset();
|
||||
@@ -70,7 +74,7 @@ describe("prepareModelForSimpleCompletion", () => {
|
||||
resolveTransportAwareSimpleApi.mockReset();
|
||||
prepareGoogleSimpleCompletionModel.mockReset();
|
||||
createAnthropicVertexStreamFnForModel.mockReturnValue("vertex-stream");
|
||||
resolveProviderStreamFn.mockReturnValue("ollama-stream");
|
||||
resolveProviderStreamFn.mockReturnValue(pluginStreamFn);
|
||||
wrapProviderSimpleCompletionStreamFn.mockReturnValue(undefined);
|
||||
buildTransportAwareSimpleStreamFn.mockReturnValue(undefined);
|
||||
createOpenClawTransportStreamFnForModel.mockReturnValue(undefined);
|
||||
@@ -142,6 +146,8 @@ describe("prepareModelForSimpleCompletion", () => {
|
||||
});
|
||||
|
||||
it("registers the configured Ollama transport and keeps the original api", () => {
|
||||
const secret = "ollama-provider-secret";
|
||||
const sentinel = mintSecretSentinel(secret, { label: "model-auth:ollama" });
|
||||
const model: Model<"ollama"> = {
|
||||
id: "llama3",
|
||||
name: "Llama 3",
|
||||
@@ -153,7 +159,7 @@ describe("prepareModelForSimpleCompletion", () => {
|
||||
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
|
||||
contextWindow: 8192,
|
||||
maxTokens: 4096,
|
||||
headers: {},
|
||||
headers: { Authorization: `Bearer ${sentinel}` },
|
||||
};
|
||||
const cfg: OpenClawConfig = {
|
||||
models: {
|
||||
@@ -183,8 +189,22 @@ describe("prepareModelForSimpleCompletion", () => {
|
||||
expect(request.config).toBe(cfg);
|
||||
expect(request.context?.provider).toBe("ollama");
|
||||
expect(request.context?.modelId).toBe("llama3");
|
||||
expect(request.context?.model).toBe(model);
|
||||
expect(ensureCustomApiRegistered).toHaveBeenCalledWith("ollama", "ollama-stream");
|
||||
expect(request.context?.model).toEqual({
|
||||
...model,
|
||||
headers: { Authorization: `Bearer ${secret}` },
|
||||
});
|
||||
expect(ensureCustomApiRegistered).toHaveBeenCalledWith("ollama", expect.any(Function));
|
||||
const registeredStream = ensureCustomApiRegistered.mock.calls[0]?.[1] as StreamFn;
|
||||
void registeredStream(
|
||||
{ ...model, headers: { Authorization: `Bearer ${sentinel}` } } as never,
|
||||
{} as never,
|
||||
{ apiKey: sentinel, headers: { "X-Managed": `Bearer ${sentinel}` } } as never,
|
||||
);
|
||||
expect(pluginStreamFn).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ headers: { Authorization: `Bearer ${secret}` } }),
|
||||
{},
|
||||
{ apiKey: secret, headers: { "X-Managed": `Bearer ${secret}` } },
|
||||
);
|
||||
expect(result).toBe(model);
|
||||
});
|
||||
|
||||
|
||||
@@ -672,6 +672,7 @@ export async function resolveModelRuntimeApiKey(params: {
|
||||
model: params.model,
|
||||
cfg: params.cfg,
|
||||
agentDir: params.agentDir,
|
||||
secretSentinels: true,
|
||||
});
|
||||
// Bedrock's runtime client owns AWS credential-chain resolution. Keep the
|
||||
// empty sentinel out of auth storage and pass it through to the stream.
|
||||
|
||||
@@ -119,7 +119,10 @@ const musicGenerateBackgroundMocks = vi.hoisted(() => ({
|
||||
}),
|
||||
}));
|
||||
|
||||
vi.mock("../../config/config.js", () => configMocks);
|
||||
vi.mock("../../config/config.js", async (importOriginal) => ({
|
||||
...(await importOriginal<typeof import("../../config/config.js")>()),
|
||||
...configMocks,
|
||||
}));
|
||||
vi.mock("../../media/store.js", () => mediaStoreMocks);
|
||||
vi.mock("../../media/web-media.js", async () => {
|
||||
const actual = await vi.importActual<typeof import("../../media/web-media.js")>(
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
// Native PDF provider tests cover direct Anthropic and Gemini request shapes,
|
||||
// base URL handling, and bounded API error reporting.
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { mintSecretSentinel } from "../../secrets/sentinel.js";
|
||||
import * as pdfNativeProviders from "./pdf-native-providers.js";
|
||||
|
||||
vi.mock("../../plugins/provider-runtime.js", () => ({
|
||||
@@ -112,6 +113,29 @@ describe("native PDF provider API calls", () => {
|
||||
expect(body.messages[0].content[1].type).toBe("text");
|
||||
});
|
||||
|
||||
it("unwraps sentinel-backed native PDF headers only at the request handoff", async () => {
|
||||
const apiKey = mintSecretSentinel("native-pdf-api-secret", {
|
||||
label: "model-auth:anthropic",
|
||||
});
|
||||
const managedHeader = mintSecretSentinel("native-pdf-managed-secret", {
|
||||
label: "model-auth:anthropic",
|
||||
});
|
||||
const fetchMock = mockFetchResponse(
|
||||
jsonResponse({ content: [{ type: "text", text: "Analysis" }] }),
|
||||
);
|
||||
|
||||
await pdfNativeProviders.anthropicAnalyzePdf(
|
||||
makeAnthropicAnalyzeParams({
|
||||
apiKey,
|
||||
requestConfig: { headers: { "X-Managed": `Bearer ${managedHeader}` } },
|
||||
}),
|
||||
);
|
||||
|
||||
const [, opts] = firstFetchCall(fetchMock) as [string, { headers: Headers }];
|
||||
expect(opts.headers.get("x-api-key")).toBe("native-pdf-api-secret");
|
||||
expect(opts.headers.get("X-Managed")).toBe("Bearer native-pdf-managed-secret");
|
||||
});
|
||||
|
||||
it("anthropicAnalyzePdf honors ANTHROPIC_BASE_URL when no base URL is configured", async () => {
|
||||
vi.stubEnv("ANTHROPIC_BASE_URL", "https://anthropic-pdf-proxy.example/v1");
|
||||
const fetchMock = mockFetchResponse(
|
||||
|
||||
@@ -14,6 +14,7 @@ import { isRecord } from "../../utils.js";
|
||||
import { normalizeSecretInput } from "../../utils/normalize-secret-input.js";
|
||||
import { resolveAnthropicMessagesUrl } from "../anthropic-transport-stream.js";
|
||||
import type { ModelProviderRequestTransportOverrides } from "../provider-request-config.js";
|
||||
import { unwrapSecretSentinelsForProviderEgress } from "../provider-secret-egress.js";
|
||||
import { resolveProviderTransportSsrFPolicy } from "../provider-transport-fetch.js";
|
||||
|
||||
type PdfInput = {
|
||||
@@ -43,9 +44,16 @@ type NativePdfJsonRequest = {
|
||||
};
|
||||
|
||||
async function postNativePdfJson(params: NativePdfJsonRequest): Promise<Record<string, unknown>> {
|
||||
const headers = new Headers(params.headers);
|
||||
for (const [name, value] of headers.entries()) {
|
||||
headers.set(
|
||||
name,
|
||||
unwrapSecretSentinelsForProviderEgress(value, `${params.failureLabel} header handoff`),
|
||||
);
|
||||
}
|
||||
const { response, release } = await postJsonRequest({
|
||||
url: params.url,
|
||||
headers: params.headers,
|
||||
headers,
|
||||
body: params.body,
|
||||
timeoutMs: NATIVE_PDF_PROVIDER_FETCH_TIMEOUT_MS,
|
||||
fetchFn: fetch,
|
||||
|
||||
@@ -377,6 +377,9 @@ describe("createPdfTool", () => {
|
||||
|
||||
const [, loadOptions] = firstMockCall(loadSpy, "loadWebMediaRaw");
|
||||
expectFields(loadOptions, { maxBytes: 524_288 });
|
||||
expect(modelAuth.getApiKeyForModel).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ secretSentinels: true }),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -19,6 +19,7 @@ import { extractPdfContent, type PdfExtractedContent } from "../../media/pdf-ext
|
||||
import { loadWebMediaRaw } from "../../media/web-media.js";
|
||||
import { resolveUserPath } from "../../utils.js";
|
||||
import type { AuthProfileStore } from "../auth-profiles/types.js";
|
||||
import { applySecretRefHeaderSentinels } from "../model-auth.js";
|
||||
import { getModelProviderRequestTransport } from "../provider-request-config.js";
|
||||
import { registerProviderStreamForModel } from "../provider-stream.js";
|
||||
import { optionalFiniteNumberSchema } from "../schema/typebox.js";
|
||||
@@ -177,7 +178,10 @@ async function runPdfPrompt(params: {
|
||||
cfg: effectiveCfg,
|
||||
modelOverride: params.modelOverride,
|
||||
run: async (provider, modelId) => {
|
||||
const model = resolveModelFromRegistry({ modelRegistry, provider, modelId });
|
||||
const model = applySecretRefHeaderSentinels(
|
||||
resolveModelFromRegistry({ modelRegistry, provider, modelId }),
|
||||
effectiveCfg,
|
||||
);
|
||||
const apiKey = await resolveModelRuntimeApiKey({
|
||||
model,
|
||||
cfg: effectiveCfg,
|
||||
|
||||
@@ -6,6 +6,7 @@ import { resolveOpenAIStrictToolSetting } from "../agents/openai-strict-tool-set
|
||||
import { buildGuardedModelFetch } from "../agents/provider-transport-fetch.js";
|
||||
import { redactSecrets, redactToolPayloadText } from "../logging/redact.js";
|
||||
import { createSubsystemLogger } from "../logging/subsystem.js";
|
||||
import { swapSecretSentinelsInText } from "../secrets/sentinel.js";
|
||||
|
||||
const transportLogBySubsystem = new Map<string, ReturnType<typeof createSubsystemLogger>>();
|
||||
|
||||
@@ -20,6 +21,16 @@ function transportLog(subsystem: string): ReturnType<typeof createSubsystemLogge
|
||||
|
||||
configureAiTransportHost({
|
||||
buildModelFetch: buildGuardedModelFetch,
|
||||
resolveSecretSentinel: (value) => {
|
||||
const swapped = swapSecretSentinelsInText(value);
|
||||
const unknown = swapped.unknown[0];
|
||||
if (unknown) {
|
||||
throw new Error(
|
||||
`Secret sentinel ${unknown} is not registered in this process; refusing to construct provider client`,
|
||||
);
|
||||
}
|
||||
return swapped.text;
|
||||
},
|
||||
redactSecrets,
|
||||
redactToolPayloadText,
|
||||
resolveOpenAIStrictToolSetting,
|
||||
|
||||
@@ -4,6 +4,7 @@ import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { afterEach, describe, expect, it } from "vitest";
|
||||
import { withEnv } from "../test-utils/env.js";
|
||||
import { withFullContextToolPayloadRedaction } from "./redact-internal.js";
|
||||
import {
|
||||
getDefaultRedactPatterns,
|
||||
redactSecrets,
|
||||
@@ -11,8 +12,13 @@ import {
|
||||
redactSensitiveLines,
|
||||
redactSensitiveText,
|
||||
redactToolDetail,
|
||||
redactToolPayloadTextWithConfig,
|
||||
resolveRedactOptions,
|
||||
} from "./redact.js";
|
||||
import {
|
||||
registerSecretValueForRedaction,
|
||||
resetSecretRedactionRegistryForTest,
|
||||
} from "./secret-redaction-registry.js";
|
||||
|
||||
const defaults = getDefaultRedactPatterns();
|
||||
let tempDirs: string[] = [];
|
||||
@@ -26,12 +32,75 @@ function writeConfig(source: string): string {
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
resetSecretRedactionRegistryForTest();
|
||||
for (const dir of tempDirs) {
|
||||
fs.rmSync(dir, { force: true, recursive: true });
|
||||
}
|
||||
tempDirs = [];
|
||||
});
|
||||
|
||||
describe("registered exact secret values", () => {
|
||||
it("masks registered values in text and nested structured data", () => {
|
||||
const secret = "registered-exact-secret";
|
||||
registerSecretValueForRedaction(secret);
|
||||
|
||||
expect(redactSensitiveText(`before ${secret} after`, { mode: "off" })).toBe(
|
||||
"before regist…cret after",
|
||||
);
|
||||
expect(redactSecrets({ detail: `before ${secret} after` })).toEqual({
|
||||
detail: "before regist…cret after",
|
||||
});
|
||||
expect(
|
||||
redactToolPayloadTextWithConfig(
|
||||
`full context ${secret}`,
|
||||
withFullContextToolPayloadRedaction(undefined),
|
||||
),
|
||||
).toBe("full context regist…cret");
|
||||
});
|
||||
|
||||
it("ignores values shorter than six characters", () => {
|
||||
registerSecretValueForRedaction("abcde");
|
||||
expect(redactSensitiveText("value abcde", { mode: "off" })).toBe("value abcde");
|
||||
expect(redactSecrets({ detail: "abcde" })).toEqual({ detail: "abcde" });
|
||||
});
|
||||
|
||||
it("masks the percent-encoded form of registered values", () => {
|
||||
const secret = "path/token with+reserved%chars";
|
||||
registerSecretValueForRedaction(secret);
|
||||
|
||||
const encoded = encodeURIComponent(secret);
|
||||
expect(redactSensitiveText(`url path ${encoded}`, { mode: "off" })).not.toContain(encoded);
|
||||
expect(redactSensitiveText(`raw ${secret}`, { mode: "off" })).not.toContain(secret);
|
||||
});
|
||||
|
||||
it("evicts the oldest value after 512 registrations", () => {
|
||||
const first = "exact-registry-value-000";
|
||||
registerSecretValueForRedaction(first);
|
||||
for (let index = 1; index <= 512; index += 1) {
|
||||
registerSecretValueForRedaction(`exact-registry-value-${index.toString().padStart(3, "0")}`);
|
||||
}
|
||||
const last = "exact-registry-value-512";
|
||||
|
||||
expect(redactSensitiveText(first, { mode: "off" })).toBe(first);
|
||||
expect(redactSensitiveText(last, { mode: "off" })).toBe("exact-…-512");
|
||||
});
|
||||
|
||||
it("refreshes duplicate registration recency before eviction", () => {
|
||||
const first = "exact-registry-refresh-000";
|
||||
const second = "exact-registry-refresh-001";
|
||||
for (let index = 0; index < 512; index += 1) {
|
||||
registerSecretValueForRedaction(
|
||||
`exact-registry-refresh-${index.toString().padStart(3, "0")}`,
|
||||
);
|
||||
}
|
||||
registerSecretValueForRedaction(first);
|
||||
registerSecretValueForRedaction("exact-registry-refresh-512");
|
||||
|
||||
expect(redactSensitiveText(first, { mode: "off" })).not.toContain(first);
|
||||
expect(redactSensitiveText(second, { mode: "off" })).toBe(second);
|
||||
});
|
||||
});
|
||||
|
||||
describe("redactSensitiveText", () => {
|
||||
it("masks env assignments while keeping the key", () => {
|
||||
const input = "OPENAI_API_KEY=sk-1234567890abcdef";
|
||||
|
||||
+20
-14
@@ -4,6 +4,7 @@ import { compileConfigRegex } from "../security/config-regex.js";
|
||||
import { readLoggingConfig } from "./config.js";
|
||||
import { replacePatternBounded } from "./redact-bounded.js";
|
||||
import { isFullContextToolPayloadRedaction } from "./redact-internal.js";
|
||||
import { redactRegisteredSecretValues } from "./secret-redaction-registry.js";
|
||||
|
||||
export type RedactSensitiveMode = "off" | "tools";
|
||||
export type RedactPattern = string | RegExp;
|
||||
@@ -987,18 +988,21 @@ export function redactSensitiveText(text: string, options?: RedactOptions): stri
|
||||
if (!text) {
|
||||
return text;
|
||||
}
|
||||
const exactRedacted = redactRegisteredSecretValues(text, maskToken);
|
||||
const resolvedOptions = options ?? resolveConfigRedaction();
|
||||
if (normalizeMode(resolvedOptions.mode) === "off") {
|
||||
return text;
|
||||
return exactRedacted;
|
||||
}
|
||||
if (!resolvedOptions.patterns?.length && !couldMatchDefaultRedactPatterns(text)) {
|
||||
return text;
|
||||
if (!resolvedOptions.patterns?.length && !couldMatchDefaultRedactPatterns(exactRedacted)) {
|
||||
return exactRedacted;
|
||||
}
|
||||
const resolved = resolveRedactOptions(resolvedOptions);
|
||||
if (!resolved.patterns.length) {
|
||||
return text;
|
||||
return exactRedacted;
|
||||
}
|
||||
return redactText(text, resolved.patterns, { redactFormBodies: resolved.redactFormBodies });
|
||||
return redactText(exactRedacted, resolved.patterns, {
|
||||
redactFormBodies: resolved.redactFormBodies,
|
||||
});
|
||||
}
|
||||
|
||||
export function redactToolDetail(detail: string): string {
|
||||
@@ -1030,9 +1034,10 @@ export function redactToolPayloadTextWithConfig(
|
||||
if (!text) {
|
||||
return text;
|
||||
}
|
||||
const exactRedacted = redactRegisteredSecretValues(text, maskToken);
|
||||
if (isFullContextToolPayloadRedaction(loggingConfig)) {
|
||||
const resolved = resolveRedactOptions(resolveToolPayloadRedaction(loggingConfig));
|
||||
return redactText(text, resolved.patterns, {
|
||||
return redactText(exactRedacted, resolved.patterns, {
|
||||
fullContext: true,
|
||||
redactFormBodies: resolved.redactFormBodies,
|
||||
});
|
||||
@@ -1050,11 +1055,12 @@ function redactSensitiveFieldValueWithOptions(
|
||||
options: RedactOptions,
|
||||
path: readonly string[] = [key],
|
||||
): string {
|
||||
const exactRedacted = redactRegisteredSecretValues(value, maskToken);
|
||||
const resolved = resolveRedactOptions(options);
|
||||
if (resolved.mode === "off") {
|
||||
return value;
|
||||
return exactRedacted;
|
||||
}
|
||||
const redacted = redactText(value, resolved.patterns, {
|
||||
const redacted = redactText(exactRedacted, resolved.patterns, {
|
||||
redactFormBodies: resolved.redactFormBodies,
|
||||
});
|
||||
const shouldRedactAppPassword = redacted !== value || STRUCTURED_APP_PASSWORD_FIELD_RE.test(key);
|
||||
@@ -1073,17 +1079,17 @@ function redactSensitiveFieldValueWithOptions(
|
||||
}
|
||||
if (
|
||||
normalizedStructuredKey === "session" &&
|
||||
STRUCTURED_INTERNAL_SOURCE_PATH_VALUE_RE.test(value)
|
||||
STRUCTURED_INTERNAL_SOURCE_PATH_VALUE_RE.test(exactRedacted)
|
||||
) {
|
||||
return value;
|
||||
return exactRedacted;
|
||||
}
|
||||
if (isSensitiveFieldKey(key)) {
|
||||
if (isShellReferenceToKey(key, value)) {
|
||||
return value;
|
||||
if (isShellReferenceToKey(key, exactRedacted)) {
|
||||
return exactRedacted;
|
||||
}
|
||||
return maskToken(value);
|
||||
return maskToken(exactRedacted);
|
||||
}
|
||||
return value;
|
||||
return exactRedacted;
|
||||
}
|
||||
|
||||
export function redactSensitiveFieldValue(
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
const MIN_SECRET_VALUE_LENGTH = 6;
|
||||
const MAX_SECRET_VALUES = 512;
|
||||
|
||||
const registeredValues = new Map<string, true>();
|
||||
let compiledMatcher: RegExp | undefined;
|
||||
let firstChars = new Set<string>();
|
||||
|
||||
function escapeRegExp(value: string): string {
|
||||
return value.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
|
||||
}
|
||||
|
||||
function rebuildProbe(): void {
|
||||
firstChars = new Set([...registeredValues.keys()].map((value) => value[0]));
|
||||
compiledMatcher = undefined;
|
||||
}
|
||||
|
||||
function registerOneSecretValue(value: string): void {
|
||||
if (registeredValues.delete(value)) {
|
||||
registeredValues.set(value, true);
|
||||
return;
|
||||
}
|
||||
registeredValues.set(value, true);
|
||||
if (registeredValues.size > MAX_SECRET_VALUES) {
|
||||
const oldest = registeredValues.keys().next().value;
|
||||
if (oldest !== undefined) {
|
||||
registeredValues.delete(oldest);
|
||||
}
|
||||
}
|
||||
rebuildProbe();
|
||||
}
|
||||
|
||||
/** Registers one resolved secret for exact-value log redaction. */
|
||||
export function registerSecretValueForRedaction(value: string): void {
|
||||
if (value.length < MIN_SECRET_VALUE_LENGTH) {
|
||||
return;
|
||||
}
|
||||
registerOneSecretValue(value);
|
||||
// URL egress percent-encodes injected values; redact that surface form too.
|
||||
const encoded = encodeURIComponent(value);
|
||||
if (encoded !== value) {
|
||||
registerOneSecretValue(encoded);
|
||||
}
|
||||
}
|
||||
|
||||
/** Returns whether a value has SecretRef provenance in the process registry. */
|
||||
export function isSecretValueRegisteredForRedaction(value: string): boolean {
|
||||
return registeredValues.has(value);
|
||||
}
|
||||
|
||||
/** Replaces registered exact values while preserving the caller's mask convention. */
|
||||
export function redactRegisteredSecretValues(
|
||||
text: string,
|
||||
mask: (value: string) => string,
|
||||
): string {
|
||||
if (!text || registeredValues.size === 0) {
|
||||
return text;
|
||||
}
|
||||
let couldMatch = false;
|
||||
for (const firstChar of firstChars) {
|
||||
if (text.includes(firstChar)) {
|
||||
couldMatch = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (!couldMatch) {
|
||||
return text;
|
||||
}
|
||||
compiledMatcher ??= new RegExp(
|
||||
[...registeredValues.keys()]
|
||||
.toSorted((left, right) => right.length - left.length)
|
||||
.map(escapeRegExp)
|
||||
.join("|"),
|
||||
"g",
|
||||
);
|
||||
return text.replace(compiledMatcher, (value) => mask(value));
|
||||
}
|
||||
|
||||
/** Test-only reset for process-global redaction state. */
|
||||
export function resetSecretRedactionRegistryForTest(): void {
|
||||
registeredValues.clear();
|
||||
rebuildProbe();
|
||||
}
|
||||
@@ -3,6 +3,11 @@
|
||||
import path from "node:path";
|
||||
import { MAX_TIMER_TIMEOUT_MS } from "@openclaw/normalization-core/number-coercion";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import {
|
||||
looksLikeSecretSentinel,
|
||||
mintSecretSentinel,
|
||||
resolveSecretSentinel,
|
||||
} from "../secrets/sentinel.js";
|
||||
|
||||
const hoisted = vi.hoisted(() => ({
|
||||
completeMock: vi.fn(),
|
||||
@@ -26,6 +31,7 @@ const hoisted = vi.hoisted(() => ({
|
||||
resolveModelAsyncMock: vi.fn(),
|
||||
resolveModelWithRegistryMock: vi.fn(),
|
||||
resolveCopilotApiTokenMock: vi.fn(),
|
||||
unwrapSecretSentinelsForProviderEgressMock: vi.fn((value: string) => value),
|
||||
}));
|
||||
const {
|
||||
completeMock,
|
||||
@@ -41,6 +47,7 @@ const {
|
||||
resolveModelAsyncMock,
|
||||
resolveModelWithRegistryMock,
|
||||
resolveCopilotApiTokenMock,
|
||||
unwrapSecretSentinelsForProviderEgressMock,
|
||||
} = hoisted;
|
||||
|
||||
type ResolveModelWithRegistryTestParams = {
|
||||
@@ -98,6 +105,7 @@ vi.mock("../agents/models-config.js", async () => ({
|
||||
}));
|
||||
|
||||
vi.mock("../agents/model-auth.js", () => ({
|
||||
applySecretRefHeaderSentinels: (model: unknown) => model,
|
||||
getApiKeyForModel: getApiKeyForModelMock,
|
||||
resolveApiKeyForProvider: resolveApiKeyForProviderMock,
|
||||
requireApiKey: requireApiKeyMock,
|
||||
@@ -107,6 +115,13 @@ vi.mock("../agents/provider-stream.js", () => ({
|
||||
registerProviderStreamForModel: registerProviderStreamForModelMock,
|
||||
}));
|
||||
|
||||
vi.mock("../agents/provider-secret-egress.js", async () => ({
|
||||
...(await vi.importActual<typeof import("../agents/provider-secret-egress.js")>(
|
||||
"../agents/provider-secret-egress.js",
|
||||
)),
|
||||
unwrapSecretSentinelsForProviderEgress: unwrapSecretSentinelsForProviderEgressMock,
|
||||
}));
|
||||
|
||||
vi.mock("../agents/agent-model-discovery.js", () => ({
|
||||
discoverAuthStorage: () => ({
|
||||
setRuntimeApiKey: setRuntimeApiKeyMock,
|
||||
@@ -248,6 +263,36 @@ describe("describeImageWithModel", () => {
|
||||
expect(completeMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("unwraps a sentinel only at the direct MiniMax VLM handoff", async () => {
|
||||
getApiKeyForModelMock.mockResolvedValueOnce({
|
||||
apiKey: "oc-sent-v1-0123456789abcdef01234567",
|
||||
source: "test",
|
||||
mode: "api-key",
|
||||
});
|
||||
unwrapSecretSentinelsForProviderEgressMock.mockReturnValueOnce("resolved-minimax-secret");
|
||||
|
||||
await describeImageWithModel({
|
||||
cfg: {},
|
||||
agentDir: "/tmp/openclaw-agent",
|
||||
provider: "minimax-portal",
|
||||
model: "MiniMax-VL-01",
|
||||
buffer: Buffer.from("png-bytes"),
|
||||
fileName: "image.png",
|
||||
mime: "image/png",
|
||||
timeoutMs: 1000,
|
||||
});
|
||||
|
||||
expect(unwrapSecretSentinelsForProviderEgressMock).toHaveBeenCalledWith(
|
||||
"oc-sent-v1-0123456789abcdef01234567",
|
||||
"MiniMax VLM request",
|
||||
);
|
||||
const [, fetchOptionsValue] = requireFirstMockCall(fetchMock, "fetch");
|
||||
const fetchOptions = requireRecord(fetchOptionsValue, "fetch options");
|
||||
expect(fetchOptions.headers).toMatchObject({
|
||||
Authorization: "Bearer resolved-minimax-secret",
|
||||
});
|
||||
});
|
||||
|
||||
it("uses generic completion for non-canonical minimax-portal image models", async () => {
|
||||
discoverModelsMock.mockReturnValue({
|
||||
find: vi.fn(() => ({
|
||||
@@ -1386,7 +1431,7 @@ describe("describeImageWithModel", () => {
|
||||
timestamp: Date.now(),
|
||||
content: [{ type: "text", text: "A solid red square." }],
|
||||
};
|
||||
const providerStreamFn = vi.fn(() => ({
|
||||
const providerStreamFn = vi.fn((_model: unknown, _context: unknown, _options: unknown) => ({
|
||||
result: vi.fn(async () => providerStreamResult),
|
||||
}));
|
||||
registerProviderStreamForModelMock.mockReturnValueOnce(providerStreamFn);
|
||||
@@ -1439,6 +1484,57 @@ describe("describeImageWithModel", () => {
|
||||
expect(contentTypes).toContain("image");
|
||||
});
|
||||
|
||||
it("keeps an exchanged Copilot image token opaque for sentinel-backed auth", async () => {
|
||||
const sourceSecret = "copilot-image-source-secret";
|
||||
const sourceSentinel = mintSecretSentinel(sourceSecret, {
|
||||
label: "model-auth:github-copilot",
|
||||
});
|
||||
getApiKeyForModelMock.mockResolvedValueOnce({
|
||||
apiKey: sourceSentinel,
|
||||
source: "test",
|
||||
mode: "token",
|
||||
});
|
||||
unwrapSecretSentinelsForProviderEgressMock.mockReturnValueOnce(sourceSecret);
|
||||
const providerStreamFn = vi.fn((_model: unknown, _context: unknown, _options: unknown) => ({
|
||||
result: vi.fn(async () => ({
|
||||
role: "assistant",
|
||||
api: "openai-completions",
|
||||
provider: "github-copilot",
|
||||
model: "gpt-4.1",
|
||||
stopReason: "stop",
|
||||
timestamp: Date.now(),
|
||||
content: [{ type: "text", text: "ok" }],
|
||||
})),
|
||||
}));
|
||||
registerProviderStreamForModelMock.mockReturnValueOnce(providerStreamFn);
|
||||
discoverModelsMock.mockReturnValue({
|
||||
find: vi.fn(() => ({
|
||||
provider: "github-copilot",
|
||||
id: "gpt-4.1",
|
||||
input: ["text", "image"],
|
||||
api: "openai-completions",
|
||||
})),
|
||||
});
|
||||
|
||||
await describeImageWithModel({
|
||||
cfg: {},
|
||||
agentDir: "/tmp/openclaw-agent",
|
||||
provider: "github-copilot",
|
||||
model: "gpt-4.1",
|
||||
buffer: Buffer.from("png-bytes"),
|
||||
fileName: "image.png",
|
||||
mime: "image/png",
|
||||
timeoutMs: 1000,
|
||||
});
|
||||
|
||||
expect(resolveCopilotApiTokenMock).toHaveBeenCalledWith({ githubToken: sourceSecret });
|
||||
const storedToken = setRuntimeApiKeyMock.mock.calls[0]?.[1] as string;
|
||||
expect(looksLikeSecretSentinel(storedToken)).toBe(true);
|
||||
expect(resolveSecretSentinel(storedToken)).toBe("copilot-api-token");
|
||||
const streamOptions = providerStreamFn.mock.calls[0]?.[2] as { apiKey?: string };
|
||||
expect(streamOptions.apiKey).toBe(storedToken);
|
||||
});
|
||||
|
||||
it("fails github-copilot image runtime setup when token exchange fails", async () => {
|
||||
discoverModelsMock.mockReturnValue({
|
||||
find: vi.fn(() => ({
|
||||
|
||||
@@ -5,6 +5,7 @@ import { isRecord } from "@openclaw/normalization-core/record-coerce";
|
||||
import { resolveModelAsync } from "../agents/embedded-agent-runner/model.js";
|
||||
import { isMinimaxVlmModel, minimaxUnderstandImage } from "../agents/minimax-vlm.js";
|
||||
import {
|
||||
applySecretRefHeaderSentinels,
|
||||
getApiKeyForModel,
|
||||
requireApiKey,
|
||||
resolveApiKeyForProvider,
|
||||
@@ -12,6 +13,10 @@ import {
|
||||
import { normalizeModelRef } from "../agents/model-selection.js";
|
||||
import { ensureOpenClawModelsJson } from "../agents/models-config.js";
|
||||
import { resolveProviderRequestCapabilities } from "../agents/provider-attribution.js";
|
||||
import {
|
||||
protectPreparedProviderRuntimeAuth,
|
||||
unwrapSecretSentinelsForProviderEgress,
|
||||
} from "../agents/provider-secret-egress.js";
|
||||
import { registerProviderStreamForModel } from "../agents/provider-stream.js";
|
||||
import {
|
||||
coerceImageAssistantText,
|
||||
@@ -234,6 +239,7 @@ async function prepareResolvedImageRuntime(
|
||||
profileId: params.profile,
|
||||
preferredProfile: params.preferredProfile,
|
||||
store: params.authStore,
|
||||
secretSentinels: true,
|
||||
});
|
||||
// Bedrock's runtime client owns AWS credential-chain resolution. Keep the
|
||||
// empty sentinel out of auth storage and pass it through to the stream.
|
||||
@@ -242,7 +248,7 @@ async function prepareResolvedImageRuntime(
|
||||
apiKeyInfo.mode === "aws-sdk" &&
|
||||
model.api === "bedrock-converse-stream"
|
||||
) {
|
||||
return { apiKey: "", model };
|
||||
return { apiKey: "", model: applySecretRefHeaderSentinels(model, params.cfg) };
|
||||
}
|
||||
let apiKey = requireApiKey(apiKeyInfo, model.provider);
|
||||
// Image tool bypasses prepareRuntimeAuth — exchange OAuth token for
|
||||
@@ -250,16 +256,24 @@ async function prepareResolvedImageRuntime(
|
||||
// matches what runtime chat requests send.
|
||||
if (model.provider === "github-copilot") {
|
||||
const copilotToken = await resolveCopilotApiToken({
|
||||
githubToken: apiKey,
|
||||
githubToken: unwrapSecretSentinelsForProviderEgress(
|
||||
apiKey,
|
||||
"GitHub Copilot image-auth exchange",
|
||||
),
|
||||
});
|
||||
apiKey = copilotToken.token;
|
||||
const runtimeBaseUrl = copilotToken.baseUrl?.trim();
|
||||
const protectedAuth = protectPreparedProviderRuntimeAuth({
|
||||
sourceApiKey: apiKey,
|
||||
provider: model.provider,
|
||||
preparedAuth: { apiKey: copilotToken.token, baseUrl: copilotToken.baseUrl },
|
||||
});
|
||||
apiKey = protectedAuth?.apiKey ?? copilotToken.token;
|
||||
const runtimeBaseUrl = protectedAuth?.baseUrl?.trim();
|
||||
if (runtimeBaseUrl) {
|
||||
model = { ...model, baseUrl: runtimeBaseUrl };
|
||||
}
|
||||
}
|
||||
authStorage.setRuntimeApiKey(model.provider, apiKey);
|
||||
return { apiKey, model };
|
||||
return { apiKey, model: applySecretRefHeaderSentinels(model, params.cfg) };
|
||||
}
|
||||
|
||||
function buildImageContext(
|
||||
@@ -333,13 +347,15 @@ async function describeImagesWithMinimax(params: {
|
||||
images: Array<{ buffer: Buffer; mime?: string }>;
|
||||
}): Promise<ImagesDescriptionResult> {
|
||||
const responses: string[] = [];
|
||||
// MiniMax VLM owns a direct fetch path, so unwrap only at this final handoff.
|
||||
const apiKey = unwrapSecretSentinelsForProviderEgress(params.apiKey, "MiniMax VLM request");
|
||||
for (const [index, image] of params.images.entries()) {
|
||||
const prompt =
|
||||
params.images.length > 1
|
||||
? `${params.prompt}\n\nDescribe image ${index + 1} of ${params.images.length} independently.`
|
||||
: params.prompt;
|
||||
const text = await minimaxUnderstandImage({
|
||||
apiKey: params.apiKey,
|
||||
apiKey,
|
||||
provider: params.provider,
|
||||
prompt,
|
||||
imageDataUrl: `data:${image.mime ?? "image/jpeg"};base64,${image.buffer.toString("base64")}`,
|
||||
@@ -425,6 +441,7 @@ async function resolveMinimaxVlmFallbackRuntime(params: {
|
||||
const auth = await resolveApiKeyForProvider({
|
||||
provider: authProvider,
|
||||
cfg: params.cfg,
|
||||
secretSentinels: true,
|
||||
profileId: params.profile,
|
||||
preferredProfile: params.preferredProfile,
|
||||
agentDir: params.agentDir,
|
||||
|
||||
@@ -1,5 +1,9 @@
|
||||
// Proxy capture runtime tests cover session creation and capture lifecycle.
|
||||
import { beforeEach, describe, expect, it } from "vitest";
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import {
|
||||
registerSecretValueForRedaction,
|
||||
resetSecretRedactionRegistryForTest,
|
||||
} from "../logging/secret-redaction-registry.js";
|
||||
import type { DebugProxySettings } from "./env.js";
|
||||
import {
|
||||
captureHttpExchange,
|
||||
@@ -94,6 +98,7 @@ describe("debug proxy runtime", () => {
|
||||
finalizeDebugProxyCapture(settings, deps);
|
||||
events.length = 0;
|
||||
calls.length = 0;
|
||||
resetSecretRedactionRegistryForTest();
|
||||
fetchTarget.fetch = async () => new Response("{}", { status: 200 });
|
||||
});
|
||||
|
||||
@@ -196,6 +201,77 @@ describe("debug proxy runtime", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("redacts registered exact values in custom headers and URL queries", async () => {
|
||||
const secret = "capture-managed-secret";
|
||||
const pathSecret = "capture/path secret";
|
||||
registerSecretValueForRedaction(secret);
|
||||
registerSecretValueForRedaction(pathSecret);
|
||||
captureHttpExchange(
|
||||
{
|
||||
url: `https://api.example.com/models/${encodeURIComponent(pathSecret)}?key=${encodeURIComponent(secret)}`,
|
||||
method: "GET",
|
||||
requestHeaders: { "X-Managed": `Bearer ${secret}` },
|
||||
response: new Response("{}", { status: 200 }),
|
||||
},
|
||||
settings,
|
||||
deps,
|
||||
);
|
||||
await new Promise((resolve) => {
|
||||
setImmediate(resolve);
|
||||
});
|
||||
|
||||
const request = events.find((event) => event.kind === "request");
|
||||
expect(request?.path).toBe("/models/%5BREDACTED%5D?key=%5BREDACTED%5D");
|
||||
expect(JSON.parse(String(request?.headersJson))).toStrictEqual({
|
||||
"X-Managed": "Bearer [REDACTED]",
|
||||
});
|
||||
});
|
||||
|
||||
it("redacts registered values from failed global-fetch capture events", async () => {
|
||||
const secret = "capture-failure/secret";
|
||||
registerSecretValueForRedaction(secret);
|
||||
fetchTarget.fetch = vi.fn(async () => {
|
||||
throw new Error(`request failed for ${secret}`);
|
||||
}) as typeof fetch;
|
||||
initializeDebugProxyCapture("test", settings, deps);
|
||||
|
||||
await expect(
|
||||
fetchTarget.fetch(`https://api.example.com/models/${encodeURIComponent(secret)}`),
|
||||
).rejects.toThrow("request failed");
|
||||
|
||||
const event = events.find((candidate) => candidate.kind === "error");
|
||||
expect(event?.path).toBe("/models/%5BREDACTED%5D");
|
||||
expect(event?.errorText).toBe("request failed for [REDACTED]");
|
||||
});
|
||||
|
||||
it("keeps capture URLs valid when the full URL is a registered secret", () => {
|
||||
const secretUrl = "https://signed.example/v1/callback";
|
||||
registerSecretValueForRedaction(secretUrl);
|
||||
|
||||
captureHttpExchange(
|
||||
{
|
||||
url: secretUrl,
|
||||
method: "GET",
|
||||
response: new Response("{}", { status: 200 }),
|
||||
},
|
||||
settings,
|
||||
deps,
|
||||
);
|
||||
|
||||
const request = events.find((candidate) => candidate.kind === "request");
|
||||
expect(request?.host).toBe("redacted.invalid");
|
||||
expect(request?.path).toBe("/%5BREDACTED%5D");
|
||||
});
|
||||
|
||||
it("does not fail capture on malformed percent escapes", async () => {
|
||||
registerSecretValueForRedaction("capture-secret");
|
||||
initializeDebugProxyCapture("test", settings, deps);
|
||||
|
||||
await expect(fetchTarget.fetch("https://api.example.com/x#%")).resolves.toBeInstanceOf(
|
||||
Response,
|
||||
);
|
||||
});
|
||||
|
||||
it("skips capturing the body when Content-Length exceeds the cap", async () => {
|
||||
initializeDebugProxyCapture("test", settings, deps);
|
||||
captureHttpExchange(
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { URL } from "node:url";
|
||||
import { normalizeRequestInitHeadersForFetch } from "../infra/fetch-headers.js";
|
||||
import { redactRegisteredSecretValues } from "../logging/secret-redaction-registry.js";
|
||||
import { resolveDebugProxySettings, type DebugProxySettings } from "./env.js";
|
||||
import {
|
||||
closeDebugProxyCaptureStore,
|
||||
@@ -197,11 +198,79 @@ function redactedCaptureHeaders(
|
||||
for (const [name, value] of entries) {
|
||||
// Header names are matched exactly and by sensitive fragments because
|
||||
// providers use many token/key naming variants.
|
||||
redacted[name] = isSensitiveCaptureHeaderName(name) ? REDACTED_CAPTURE_HEADER_VALUE : value;
|
||||
redacted[name] = isSensitiveCaptureHeaderName(name)
|
||||
? REDACTED_CAPTURE_HEADER_VALUE
|
||||
: redactRegisteredSecretValues(value, () => REDACTED_CAPTURE_HEADER_VALUE);
|
||||
}
|
||||
return redacted;
|
||||
}
|
||||
|
||||
function redactCaptureUrl(rawUrl: string): string {
|
||||
let url: URL;
|
||||
try {
|
||||
url = new URL(rawUrl);
|
||||
} catch {
|
||||
return "https://redacted.invalid/%5BREDACTED%5D";
|
||||
}
|
||||
const redactComponent = (value: string) =>
|
||||
redactRegisteredSecretValues(value, () => REDACTED_CAPTURE_HEADER_VALUE);
|
||||
const decodeComponent = (value: string) => {
|
||||
try {
|
||||
return decodeURIComponent(value);
|
||||
} catch {
|
||||
return value;
|
||||
}
|
||||
};
|
||||
if (redactComponent(url.hostname) !== url.hostname) {
|
||||
url.hostname = "redacted.invalid";
|
||||
}
|
||||
for (const key of ["username", "password"] as const) {
|
||||
const decoded = decodeComponent(url[key]);
|
||||
const redacted = redactComponent(decoded);
|
||||
if (redacted !== decoded) {
|
||||
url[key] = redacted;
|
||||
}
|
||||
}
|
||||
url.pathname = url.pathname
|
||||
.split("/")
|
||||
.map((segment) => {
|
||||
try {
|
||||
const decoded = decodeURIComponent(segment);
|
||||
const redacted = redactComponent(decoded);
|
||||
return redacted === decoded ? segment : encodeURIComponent(redacted);
|
||||
} catch {
|
||||
return segment;
|
||||
}
|
||||
})
|
||||
.join("/");
|
||||
const searchParams = new URLSearchParams();
|
||||
let searchChanged = false;
|
||||
for (const [name, value] of url.searchParams.entries()) {
|
||||
const redactedName = redactComponent(name);
|
||||
const redactedValue = redactComponent(value);
|
||||
searchParams.append(redactedName, redactedValue);
|
||||
if (redactedName !== name || redactedValue !== value) {
|
||||
searchChanged = true;
|
||||
}
|
||||
}
|
||||
if (searchChanged) {
|
||||
url.search = searchParams.toString();
|
||||
}
|
||||
const decodedHash = decodeComponent(url.hash.slice(1));
|
||||
const redactedHash = redactComponent(decodedHash);
|
||||
if (redactedHash !== decodedHash) {
|
||||
url.hash = redactedHash;
|
||||
}
|
||||
const serialized = url.toString();
|
||||
return redactComponent(serialized) === serialized
|
||||
? serialized
|
||||
: `${url.protocol}//redacted.invalid/%5BREDACTED%5D`;
|
||||
}
|
||||
|
||||
function redactCaptureText(value: string): string {
|
||||
return redactRegisteredSecretValues(value, () => REDACTED_CAPTURE_HEADER_VALUE);
|
||||
}
|
||||
|
||||
function createHttpCaptureEventBase(params: {
|
||||
settings: DebugProxySettings;
|
||||
rawUrl: string;
|
||||
@@ -285,13 +354,14 @@ function installDebugProxyGlobalFetchPatch(
|
||||
} catch (error) {
|
||||
if (url && /^https?:/i.test(url)) {
|
||||
const store = runtime.getStore();
|
||||
const parsed = new URL(url);
|
||||
const captureUrl = redactCaptureUrl(url);
|
||||
const parsed = new URL(captureUrl);
|
||||
store.recordEvent({
|
||||
sessionId: settings.sessionId,
|
||||
ts: Date.now(),
|
||||
sourceScope: "openclaw",
|
||||
sourceProcess: settings.sourceProcess,
|
||||
protocol: protocolFromUrl(url),
|
||||
protocol: protocolFromUrl(captureUrl),
|
||||
direction: "local",
|
||||
kind: "error",
|
||||
flowId: randomUUID(),
|
||||
@@ -303,7 +373,7 @@ function installDebugProxyGlobalFetchPatch(
|
||||
"GET",
|
||||
host: parsed.host,
|
||||
path: `${parsed.pathname}${parsed.search}`,
|
||||
errorText: error instanceof Error ? error.message : String(error),
|
||||
errorText: redactCaptureText(error instanceof Error ? error.message : String(error)),
|
||||
metaJson: runtime.safeJsonString({ captureOrigin: "global-fetch" }),
|
||||
});
|
||||
}
|
||||
@@ -389,7 +459,8 @@ export function captureHttpExchange(
|
||||
const runtime = resolveRuntimeDeps(deps);
|
||||
const store = runtime.getStore();
|
||||
const flowId = params.flowId ?? randomUUID();
|
||||
const url = new URL(params.url);
|
||||
const captureUrl = redactCaptureUrl(params.url);
|
||||
const url = new URL(captureUrl);
|
||||
const requestBody =
|
||||
typeof params.requestBody === "string" || Buffer.isBuffer(params.requestBody)
|
||||
? params.requestBody
|
||||
@@ -404,7 +475,7 @@ export function captureHttpExchange(
|
||||
store.recordEvent({
|
||||
...createHttpCaptureEventBase({
|
||||
settings,
|
||||
rawUrl: params.url,
|
||||
rawUrl: captureUrl,
|
||||
url,
|
||||
transport: params.transport,
|
||||
direction: "outbound",
|
||||
@@ -427,7 +498,7 @@ export function captureHttpExchange(
|
||||
store.recordEvent({
|
||||
...createHttpCaptureEventBase({
|
||||
settings,
|
||||
rawUrl: params.url,
|
||||
rawUrl: captureUrl,
|
||||
url,
|
||||
transport: params.transport,
|
||||
direction: "inbound",
|
||||
@@ -485,7 +556,7 @@ export function captureHttpExchange(
|
||||
store.recordEvent({
|
||||
...createHttpCaptureEventBase({
|
||||
settings,
|
||||
rawUrl: params.url,
|
||||
rawUrl: captureUrl,
|
||||
url,
|
||||
transport: params.transport,
|
||||
direction: "inbound",
|
||||
@@ -504,7 +575,7 @@ export function captureHttpExchange(
|
||||
store.recordEvent({
|
||||
...createHttpCaptureEventBase({
|
||||
settings,
|
||||
rawUrl: params.url,
|
||||
rawUrl: captureUrl,
|
||||
url,
|
||||
transport: params.transport,
|
||||
direction: "local",
|
||||
@@ -512,7 +583,7 @@ export function captureHttpExchange(
|
||||
flowId,
|
||||
method: params.method,
|
||||
}),
|
||||
errorText: error instanceof Error ? error.message : String(error),
|
||||
errorText: redactCaptureText(error instanceof Error ? error.message : String(error)),
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
/** Tests runtime SecretRef resolution across core config and auth-profile surfaces. */
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { afterEach, describe, expect, it } from "vitest";
|
||||
import { redactSensitiveText } from "../logging/redact.js";
|
||||
import { resetSecretRedactionRegistryForTest } from "../logging/secret-redaction-registry.js";
|
||||
import { asConfig, setupSecretsRuntimeSnapshotTestHooks } from "./runtime.test-support.ts";
|
||||
|
||||
const EMPTY_LOADABLE_PLUGIN_ORIGINS = new Map();
|
||||
@@ -12,6 +14,10 @@ const CODEX_APP_SERVER_TOKEN_REF = {
|
||||
id: "CODEX_APP_SERVER_TOKEN",
|
||||
} as const;
|
||||
|
||||
afterEach(() => {
|
||||
resetSecretRedactionRegistryForTest();
|
||||
});
|
||||
|
||||
function expectWarning(
|
||||
snapshot: Awaited<ReturnType<typeof prepareSecretsRuntimeSnapshot>>,
|
||||
expected: { code: string; path: string },
|
||||
@@ -25,6 +31,22 @@ function expectWarning(
|
||||
}
|
||||
|
||||
describe("secrets runtime snapshot", () => {
|
||||
it("registers every resolved value for exact redaction", async () => {
|
||||
const secret = "runtime-registration-secret";
|
||||
await prepareSecretsRuntimeSnapshot({
|
||||
config: asConfig({
|
||||
talk: {
|
||||
apiKey: { source: "env", provider: "default", id: "TALK_API_KEY" },
|
||||
},
|
||||
}),
|
||||
env: { TALK_API_KEY: secret },
|
||||
includeAuthStoreRefs: false,
|
||||
loadablePluginOrigins: EMPTY_LOADABLE_PLUGIN_ORIGINS,
|
||||
});
|
||||
|
||||
expect(redactSensitiveText(`resolved ${secret}`, { mode: "off" })).toBe("resolved runtim…cret");
|
||||
});
|
||||
|
||||
it("resolves sandbox ssh secret refs for active ssh backends", async () => {
|
||||
const snapshot = await prepareSecretsRuntimeSnapshot({
|
||||
config: asConfig({
|
||||
|
||||
@@ -9,6 +9,7 @@ import {
|
||||
} from "../agents/auth-profiles.js";
|
||||
import type { AuthProfileStore } from "../agents/auth-profiles/types.js";
|
||||
import type { OpenClawConfig } from "../config/types.openclaw.js";
|
||||
import { registerSecretValueForRedaction } from "../logging/secret-redaction-registry.js";
|
||||
import type { PluginManifestRegistry } from "../plugins/manifest-registry.js";
|
||||
import type { PluginMetadataSnapshot } from "../plugins/plugin-metadata-snapshot.js";
|
||||
import type { PluginOrigin } from "../plugins/plugin-origin.types.js";
|
||||
@@ -220,6 +221,11 @@ export async function prepareSecretsRuntimeSnapshot(params: {
|
||||
cache: context.cache,
|
||||
manifestRegistry: context.manifestRegistry,
|
||||
});
|
||||
for (const value of resolved.values()) {
|
||||
if (typeof value === "string") {
|
||||
registerSecretValueForRedaction(value);
|
||||
}
|
||||
}
|
||||
applyResolvedAssignments({
|
||||
assignments: context.assignments,
|
||||
resolved,
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
import { afterEach, describe, expect, it } from "vitest";
|
||||
import { redactSensitiveText } from "../logging/redact.js";
|
||||
import { resetSecretRedactionRegistryForTest } from "../logging/secret-redaction-registry.js";
|
||||
import {
|
||||
looksLikeSecretSentinel,
|
||||
mintSecretSentinel,
|
||||
resolveSecretSentinel,
|
||||
SECRET_SENTINEL_PATTERN,
|
||||
swapSecretSentinelsInText,
|
||||
} from "./sentinel.js";
|
||||
|
||||
describe("secret sentinels", () => {
|
||||
afterEach(() => {
|
||||
delete process.env.OPENCLAW_SECRET_SENTINELS;
|
||||
resetSecretRedactionRegistryForTest();
|
||||
});
|
||||
|
||||
it("mints, recognizes, resolves, and reuses sentinels by value and label", () => {
|
||||
const first = mintSecretSentinel("provider-secret-value", { label: "model-auth:openai" });
|
||||
const repeated = mintSecretSentinel("provider-secret-value", { label: "model-auth:openai" });
|
||||
const otherLabel = mintSecretSentinel("provider-secret-value", { label: "model-auth:other" });
|
||||
|
||||
expect(first).toMatch(/^oc-sent-v1-[0-9a-f]{24}$/);
|
||||
expect(first.match(SECRET_SENTINEL_PATTERN)).toEqual([first]);
|
||||
expect(looksLikeSecretSentinel(first)).toBe(true);
|
||||
expect(resolveSecretSentinel(first)).toBe("provider-secret-value");
|
||||
expect(repeated).toBe(first);
|
||||
expect(otherLabel).not.toBe(first);
|
||||
});
|
||||
|
||||
it("swaps repeated and composed sentinel substrings", () => {
|
||||
const first = mintSecretSentinel("first-secret-value", { label: "model-auth:openai" });
|
||||
const second = mintSecretSentinel("second-secret-value", { label: "model-auth:cloudflare" });
|
||||
|
||||
expect(
|
||||
swapSecretSentinelsInText(`Bearer ${first}; cf-aig-authorization=Bearer ${second}; ${first}`),
|
||||
).toEqual({
|
||||
text: "Bearer first-secret-value; cf-aig-authorization=Bearer second-secret-value; first-secret-value",
|
||||
unknown: [],
|
||||
});
|
||||
});
|
||||
|
||||
it("reports unknown sentinel-shaped values without replacing them", () => {
|
||||
const unknown = "oc-sent-v1-0123456789abcdef01234567";
|
||||
expect(swapSecretSentinelsInText(`Bearer ${unknown}`)).toEqual({
|
||||
text: `Bearer ${unknown}`,
|
||||
unknown: [unknown],
|
||||
});
|
||||
});
|
||||
|
||||
it("treats sentinel-shaped bytes inside resolved values as opaque", () => {
|
||||
const secret = "prefix-oc-sent-v1-0123456789abcdef01234567";
|
||||
const sentinel = mintSecretSentinel(secret, { label: "nested-shape" });
|
||||
|
||||
expect(swapSecretSentinelsInText(`Bearer ${sentinel}`)).toEqual({
|
||||
text: `Bearer ${secret}`,
|
||||
unknown: [],
|
||||
});
|
||||
});
|
||||
|
||||
it.each(["off", " OFF ", "0", "false", "False"])(
|
||||
"returns plaintext when the kill switch is %s",
|
||||
(value) => {
|
||||
process.env.OPENCLAW_SECRET_SENTINELS = value;
|
||||
expect(mintSecretSentinel("kill-switch-secret", { label: "model-auth:test" })).toBe(
|
||||
"kill-switch-secret",
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
it("registers minted values for exact redaction across registry eviction", () => {
|
||||
const first = "sentinel-registry-value-000";
|
||||
const firstSentinel = mintSecretSentinel(first, { label: "model-auth:0" });
|
||||
for (let index = 1; index <= 512; index += 1) {
|
||||
mintSecretSentinel(`sentinel-registry-value-${index.toString().padStart(3, "0")}`, {
|
||||
label: `model-auth:${index}`,
|
||||
});
|
||||
}
|
||||
const last = "sentinel-registry-value-512";
|
||||
|
||||
expect(redactSensitiveText(first, { mode: "tools", patterns: [] })).toBe(first);
|
||||
expect(redactSensitiveText(last, { mode: "tools", patterns: [] })).not.toContain(last);
|
||||
expect(resolveSecretSentinel(firstSentinel)).toBe(first);
|
||||
expect(redactSensitiveText(first, { mode: "tools", patterns: [] })).not.toContain(first);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,67 @@
|
||||
import { randomBytes } from "node:crypto";
|
||||
import { registerSecretValueForRedaction } from "../logging/secret-redaction-registry.js";
|
||||
|
||||
const SECRET_SENTINEL_PREFIX = "oc-sent-v1-";
|
||||
const SECRET_SENTINEL_SOURCE = `${SECRET_SENTINEL_PREFIX}[0-9a-f]{24}`;
|
||||
|
||||
export const SECRET_SENTINEL_PATTERN = new RegExp(SECRET_SENTINEL_SOURCE, "g");
|
||||
|
||||
const valuesBySentinel = new Map<string, string>();
|
||||
const sentinelsByValueAndLabel = new Map<string, Map<string, string>>();
|
||||
|
||||
function secretSentinelsEnabled(env: NodeJS.ProcessEnv = process.env): boolean {
|
||||
const configured = env.OPENCLAW_SECRET_SENTINELS?.trim().toLowerCase();
|
||||
return configured !== "off" && configured !== "0" && configured !== "false";
|
||||
}
|
||||
|
||||
export function looksLikeSecretSentinel(value: string): boolean {
|
||||
return new RegExp(`^${SECRET_SENTINEL_SOURCE}$`).test(value);
|
||||
}
|
||||
|
||||
/** Mints one stable process-local sentinel for a secret value and label. */
|
||||
export function mintSecretSentinel(value: string, meta: { label: string }): string {
|
||||
registerSecretValueForRedaction(value);
|
||||
if (!secretSentinelsEnabled()) {
|
||||
return value;
|
||||
}
|
||||
const byLabel = sentinelsByValueAndLabel.get(value) ?? new Map<string, string>();
|
||||
const existing = byLabel.get(meta.label);
|
||||
if (existing) {
|
||||
return existing;
|
||||
}
|
||||
let sentinel: string;
|
||||
do {
|
||||
sentinel = `${SECRET_SENTINEL_PREFIX}${randomBytes(12).toString("hex")}`;
|
||||
} while (valuesBySentinel.has(sentinel));
|
||||
byLabel.set(meta.label, sentinel);
|
||||
sentinelsByValueAndLabel.set(value, byLabel);
|
||||
valuesBySentinel.set(sentinel, value);
|
||||
return sentinel;
|
||||
}
|
||||
|
||||
/** Resolves a process-local sentinel without exposing the registry itself. */
|
||||
export function resolveSecretSentinel(sentinel: string): string | undefined {
|
||||
const value = valuesBySentinel.get(sentinel);
|
||||
if (value !== undefined) {
|
||||
// Refresh the bounded redaction registry whenever a live sentinel is used.
|
||||
registerSecretValueForRedaction(value);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
/** Swaps every known sentinel substring and reports unknown sentinel-shaped values. */
|
||||
export function swapSecretSentinelsInText(text: string): { text: string; unknown: string[] } {
|
||||
if (!text.includes(SECRET_SENTINEL_PREFIX)) {
|
||||
return { text, unknown: [] };
|
||||
}
|
||||
const unknown = new Set<string>();
|
||||
const swapped = text.replace(new RegExp(SECRET_SENTINEL_SOURCE, "g"), (sentinel) => {
|
||||
const value = resolveSecretSentinel(sentinel);
|
||||
if (value === undefined) {
|
||||
unknown.add(sentinel);
|
||||
return sentinel;
|
||||
}
|
||||
return value;
|
||||
});
|
||||
return { text: swapped, unknown: [...unknown] };
|
||||
}
|
||||
Reference in New Issue
Block a user