mirror of
https://github.com/openclaw/openclaw.git
synced 2026-08-24 19:35:28 -06:00
feat(e2e): record provider media content facts for Mantis proofs (#127830)
* feat(e2e): record provider media content facts * docs(mantis): list staged-media recipe in recipe index * fix(mantis): isolate mock provider evidence * fix(mantis): state provider facts as tamper-evident, not provenance-trusted The mock sidecar makes provider request facts tamper-evident: candidate code cannot alter or remove a recorded request. It cannot make them provenance-authenticated — every process in the SUT container shares the internal network, UID, and gateway config, so nothing distinguishes the gateway flow from any other candidate-originated request. The proof prompt and busy-queue recipe now state exactly that property instead of calling the facts trusted. * fix(mantis): redact unparseable provider request bodies Media redaction walks the parsed JSON body, so a request that fails JSON.parse fell back to logging the raw text — leaking base64 payloads the redactor exists to strip. Unparseable bodies now log a bounded byte-count marker instead; regression posts a malformed body carrying a data URL and asserts the payload never reaches the record. * fix(mantis): expose newest provider records through a seq-stamped tail The lane's requests surface kept the first 100 provider records, so a session longer than the window hid exactly the newest requests a proof asserts on. The mock server now stamps each record with a producer-owned absolute seq ordinal, and the lane reads a bounded 128-record tail — mirroring the sibling botApiRequests window. Regression writes 130 records and asserts the tail keeps seq 3..130; it fails pre-fix.
This commit is contained in:
Ayaan Zaidi
@@ -7,5 +7,6 @@ even when Telegram Desktop pixels match.
|
||||
- `send-failure-injection.md`: outbound Bot API failures and retry behavior
|
||||
- `busy-queue-scripted-provider.md`: ordered slow/fast multi-turn responses
|
||||
- `callback-data-payload-proof.md`: byte-level callback payload differences
|
||||
- `staged-media-provider-proof.md`: staged Telegram media proven through provider content facts
|
||||
|
||||
Return to `mantis-telegram-desktop-proof.md` for limits, cleanup, and publishing.
|
||||
|
||||
@@ -29,5 +29,5 @@ $lane finish --lane baseline
|
||||
|
||||
Repeat for `candidate`. Proof facts: session events and recorded Bot API
|
||||
messages show the slow first and distinct second outcomes without a
|
||||
control-file race. The provider request log (`scriptEntry` 0 then 1, turn
|
||||
order in bodies) is diagnostic context, not the comparison evidence.
|
||||
control-file race. The tamper-evident provider request facts (`scriptEntry` 0
|
||||
then 1, turn order in bodies) independently prove provider arrival order.
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
# Staged media provider proof
|
||||
|
||||
Use when a change alters how an uploaded document or image reaches the provider.
|
||||
|
||||
```bash
|
||||
lane="$OPENCLAW_TELEGRAM_MANTIS_LANE_CMD"
|
||||
media="$MANTIS_OUTPUT_DIR/sample.pdf"
|
||||
sent="$($lane send --lane baseline --media "$media" --text '@{sut} inspect this document')"
|
||||
message_id="$(jq -er '.sent.messageId' <<<"$sent")"
|
||||
$lane observe --lane baseline --seconds 60 --until-provider-requests 1
|
||||
requests="$($lane requests --lane baseline)"
|
||||
jq -e '[.requests[].contentFacts[]? | select(.type == "legacy_media")] | length > 0' \
|
||||
<<<"$requests"
|
||||
$lane finish --lane baseline --focus-message-id "$message_id"
|
||||
```
|
||||
|
||||
For a reply-mention turn, first `send --media "$media"` without text, capture its
|
||||
`.sent.messageId`, then `send --reply-to "$message_id" --text '@{sut} inspect this document'`.
|
||||
A bare unmentioned upload stages the file but produces no provider turn.
|
||||
|
||||
Repeat for `candidate` with its returned message id, selecting `type == "input_file"`.
|
||||
Assert the complete selected facts: `filename`, `mimeType`, and `byteLength` when present.
|
||||
The structured facts are comparison evidence; never scrape `body` strings.
|
||||
|
||||
If baseline needs a tool round trip, take the tool argument from the recorded
|
||||
`legacy_media.filename`. Build a complete response-events JSON array from
|
||||
`toolCallEvents()` in `scripts/e2e/mock-openai-server.mjs`
|
||||
(`response.output_item.added`, `response.function_call_arguments.delta`,
|
||||
`response.output_item.done`, `response.completed`), then install it before the
|
||||
next turn with `mock --lane baseline --response-events-file <public-json>`.
|
||||
@@ -80,7 +80,8 @@ Use `$OPENCLAW_TELEGRAM_MANTIS_LANE_CMD` with `--lane baseline|candidate`:
|
||||
[--until-provider-requests N]` (returns early when all supplied conditions hold;
|
||||
event/text conditions count only events after the cursor, provider count is
|
||||
cumulative for the lane)
|
||||
- `requests` (redacted provider requests; zero is a valid recorded fact)
|
||||
- `requests` (redacted provider requests; media/file items appear as structured
|
||||
`contentFacts`; zero is a valid recorded fact)
|
||||
- `press --message-id ID --button INDEX`
|
||||
- `delete --message-id ID` (only user messages sent in this session)
|
||||
- `desktop --actions-file <public-json> [--timeout-seconds N]` (run an
|
||||
@@ -126,10 +127,13 @@ baseline lane that reproduces the defect is a successful capture. A PR-level
|
||||
pass claim requires an observed, material baseline/candidate difference caused
|
||||
by the changed behavior. That difference may be trusted Bot API payload/status
|
||||
facts even when pixels are identical; screenshots remain comparison context.
|
||||
Provider request logs are diagnostic and pacing signals, not standalone
|
||||
comparison evidence. Identical pixels alone do not force `block` when the
|
||||
trusted recorded facts differ materially. If neither pixels nor trusted facts prove a
|
||||
difference, use `block`. When the expected result is silence, focus the
|
||||
Provider request facts are tamper-evident comparison evidence: the provider
|
||||
sidecar records them outside the candidate runtime, so candidate code cannot
|
||||
alter or remove a recorded request after the fact. Requests still originate
|
||||
inside the SUT, so the facts prove what the candidate runtime sent — the
|
||||
behavior under proof — not who sent it. Identical pixels alone do not force `block`
|
||||
when the recorded facts differ materially. If neither pixels nor recorded facts
|
||||
prove a difference, use `block`. When the expected result is silence, focus the
|
||||
session-owned user message that triggered the silent outcome.
|
||||
Decide before finalizing each lane. If its setup did not exercise the intended
|
||||
behavior, call `block`; do not call `finish` and describe the block only in prose.
|
||||
|
||||
@@ -18,14 +18,149 @@ const port =
|
||||
process.env.MOCK_PORT != null
|
||||
? readTcpPortEnv("MOCK_PORT")
|
||||
: readTcpPortEnv("OPENCLAW_MOCK_OPENAI_PORT");
|
||||
const bindHost = process.env.MOCK_BIND_HOST ?? "127.0.0.1";
|
||||
const successMarker = process.env.SUCCESS_MARKER ?? "OPENCLAW_E2E_OK";
|
||||
const requestLog = process.env.MOCK_REQUEST_LOG;
|
||||
// Absolute record ordinal, stamped at the producer: consumers expose a bounded
|
||||
// tail of the log, so entries must carry their own position. The server starts
|
||||
// once per run, so the counter spans the whole session.
|
||||
let requestLogSeq = 0;
|
||||
const initialResponseChunkDelayMs = process.env.MOCK_RESPONSE_CHUNK_DELAY_MS
|
||||
? readPositiveIntEnv("MOCK_RESPONSE_CHUNK_DELAY_MS", undefined)
|
||||
: 0;
|
||||
const responseControl = process.env.MOCK_RESPONSE_CONTROL;
|
||||
const MAX_CONTENT_FACTS = 128;
|
||||
const MAX_CONTENT_FACT_FILENAME_LENGTH = 1024;
|
||||
const LEGACY_MEDIA_PATTERN =
|
||||
/\[media attached: ([^\]\r\n]+?) \(([a-z][a-z0-9.+-]*\/[a-z0-9.+-]+)\)(?: \| [^\]\r\n]+)?\]/giu;
|
||||
const MEDIA_DATA_URL_PATTERN =
|
||||
/^data:([a-z][a-z0-9.+-]*\/[a-z0-9.+-]+)(?:;[^,]*)*;base64,([\s\S]*)$/iu;
|
||||
let scriptState;
|
||||
|
||||
function parseMediaDataUrl(value) {
|
||||
if (typeof value !== "string") {
|
||||
return undefined;
|
||||
}
|
||||
const match = MEDIA_DATA_URL_PATTERN.exec(value);
|
||||
if (!match) {
|
||||
return undefined;
|
||||
}
|
||||
const encoded = (match[2] ?? "").replace(/\s/gu, "");
|
||||
const valid = /^[A-Za-z0-9+/]*={0,2}$/u.test(encoded) && encoded.length % 4 !== 1;
|
||||
const padding = encoded.endsWith("==") ? 2 : encoded.endsWith("=") ? 1 : 0;
|
||||
return {
|
||||
byteLength: valid ? Math.floor((encoded.length * 3) / 4) - padding : undefined,
|
||||
mimeType: match[1]?.toLowerCase(),
|
||||
};
|
||||
}
|
||||
|
||||
function requestContentFact(item) {
|
||||
const type = typeof item.type === "string" ? item.type.slice(0, 64) : undefined;
|
||||
if (!type) {
|
||||
return undefined;
|
||||
}
|
||||
const fact = { type };
|
||||
if (type !== "input_file" && type !== "input_image") {
|
||||
return fact;
|
||||
}
|
||||
const imageUrl =
|
||||
typeof item.image_url === "string"
|
||||
? item.image_url
|
||||
: item.image_url && typeof item.image_url === "object"
|
||||
? item.image_url.url
|
||||
: undefined;
|
||||
const dataUrl = parseMediaDataUrl(type === "input_file" ? item.file_data : imageUrl);
|
||||
const filename =
|
||||
typeof item.filename === "string"
|
||||
? item.filename.slice(0, MAX_CONTENT_FACT_FILENAME_LENGTH)
|
||||
: undefined;
|
||||
const mimeType =
|
||||
(typeof item.mimeType === "string" ? item.mimeType.slice(0, 128) : undefined) ??
|
||||
(typeof item.mime_type === "string" ? item.mime_type.slice(0, 128) : undefined) ??
|
||||
dataUrl?.mimeType;
|
||||
return {
|
||||
...fact,
|
||||
...(filename ? { filename } : {}),
|
||||
...(mimeType ? { mimeType } : {}),
|
||||
...(dataUrl?.byteLength === undefined ? {} : { byteLength: dataUrl.byteLength }),
|
||||
};
|
||||
}
|
||||
|
||||
function summarizeRequestContent(body) {
|
||||
const facts = [];
|
||||
let firstFact = 0;
|
||||
let truncated = false;
|
||||
const appendFact = (fact) => {
|
||||
if (facts.length < MAX_CONTENT_FACTS) {
|
||||
facts.push(fact);
|
||||
return;
|
||||
}
|
||||
facts[firstFact] = fact;
|
||||
firstFact = (firstFact + 1) % MAX_CONTENT_FACTS;
|
||||
truncated = true;
|
||||
};
|
||||
const appendLegacyMediaFacts = (text) => {
|
||||
for (const match of text.matchAll(LEGACY_MEDIA_PATTERN)) {
|
||||
appendFact({
|
||||
type: "legacy_media",
|
||||
filename: match[1].trim().slice(0, MAX_CONTENT_FACT_FILENAME_LENGTH),
|
||||
mimeType: match[2].toLowerCase(),
|
||||
});
|
||||
}
|
||||
};
|
||||
const visit = (value) => {
|
||||
if (typeof value === "string") {
|
||||
appendFact({ type: "input_text" });
|
||||
appendLegacyMediaFacts(value);
|
||||
return;
|
||||
}
|
||||
if (Array.isArray(value)) {
|
||||
for (const item of value) {
|
||||
visit(item);
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (!value || typeof value !== "object") {
|
||||
return;
|
||||
}
|
||||
if (value.type === "message" || typeof value.type !== "string") {
|
||||
visit(value.content);
|
||||
return;
|
||||
}
|
||||
const fact = requestContentFact(value);
|
||||
if (!fact) {
|
||||
return;
|
||||
}
|
||||
appendFact(fact);
|
||||
if (
|
||||
(fact.type === "input_text" || fact.type === "output_text") &&
|
||||
typeof value.text === "string"
|
||||
) {
|
||||
appendLegacyMediaFacts(value.text);
|
||||
}
|
||||
};
|
||||
|
||||
visit(body?.input ?? body?.messages);
|
||||
const contentFacts = truncated
|
||||
? [...facts.slice(firstFact), ...facts.slice(0, firstFact)]
|
||||
: facts;
|
||||
return { contentFacts, ...(truncated ? { contentFactsTruncated: true } : {}) };
|
||||
}
|
||||
|
||||
function redactRequestLogMedia(body, bodyText) {
|
||||
let redacted = false;
|
||||
const sanitized = JSON.stringify(body, (_key, value) => {
|
||||
const dataUrl = parseMediaDataUrl(value);
|
||||
if (!dataUrl) {
|
||||
return value;
|
||||
}
|
||||
redacted = true;
|
||||
const bytes = dataUrl.byteLength === undefined ? "unknown" : dataUrl.byteLength;
|
||||
return `data:${dataUrl.mimeType};base64,[redacted:${bytes} bytes]`;
|
||||
});
|
||||
return redacted ? sanitized : bodyText;
|
||||
}
|
||||
|
||||
function readResponseEntry(value, label) {
|
||||
if (!value || typeof value !== "object" || Array.isArray(value)) {
|
||||
throw new Error(`${label} is invalid`);
|
||||
@@ -676,19 +811,26 @@ const server = http.createServer((req, res) => {
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
let body;
|
||||
let body = {};
|
||||
let requestLogBody;
|
||||
try {
|
||||
body = bodyText ? JSON.parse(bodyText) : {};
|
||||
requestLogBody = redactRequestLogMedia(body, bodyText);
|
||||
} catch {
|
||||
body = {};
|
||||
// Redaction walks the parsed JSON, so an unparseable body would bypass it
|
||||
// and leak raw base64 media into the provider record. Log a bounded
|
||||
// marker instead of the text.
|
||||
requestLogBody = `[unparseable request body redacted: ${Buffer.byteLength(bodyText)} bytes]`;
|
||||
}
|
||||
if (
|
||||
writeRequestLogEntryOrFail(res, {
|
||||
requestLog,
|
||||
entry: {
|
||||
seq: (requestLogSeq += 1),
|
||||
method: req.method,
|
||||
path: url.pathname,
|
||||
body: boundedRequestLogBody(bodyText, bodyText),
|
||||
body: boundedRequestLogBody(requestLogBody, requestLogBody),
|
||||
...summarizeRequestContent(body),
|
||||
...(selectedResponse?.scriptEntry ? { scriptEntry: selectedResponse.scriptEntry } : {}),
|
||||
},
|
||||
})
|
||||
@@ -821,6 +963,6 @@ const server = http.createServer((req, res) => {
|
||||
});
|
||||
});
|
||||
|
||||
server.listen(port, "127.0.0.1", () => {
|
||||
server.listen(port, bindHost, () => {
|
||||
console.log(`mock-openai listening on ${port}`);
|
||||
});
|
||||
|
||||
@@ -683,20 +683,12 @@ function redact(value: unknown, secret: string): unknown {
|
||||
}
|
||||
|
||||
function providerRequests(state: ActiveSession, secret: string): unknown[] {
|
||||
if (!fs.existsSync(state.sut.requestLog)) {
|
||||
return [];
|
||||
}
|
||||
return fs
|
||||
.readFileSync(state.sut.requestLog, "utf8")
|
||||
.split("\n")
|
||||
.filter(Boolean)
|
||||
.slice(0, 100)
|
||||
.map((line, index) =>
|
||||
Object.assign(
|
||||
{ index: index + 1 },
|
||||
redact(JSON.parse(line), secret) as Record<string, unknown>,
|
||||
),
|
||||
);
|
||||
// Tail window, like botApiRequests: a long session must surface its newest
|
||||
// provider turns. Entries carry a producer-stamped `seq` ordinal, so the
|
||||
// window keeps absolute order without rereading the whole file.
|
||||
return boundedNdjson(state.sut.requestLog, 128).map(
|
||||
(entry) => redact(entry, secret) as Record<string, unknown>,
|
||||
);
|
||||
}
|
||||
|
||||
function boundedNdjson(file: string, limit: number): unknown[] {
|
||||
|
||||
@@ -183,6 +183,7 @@ export function writeSutConfig(params: {
|
||||
gatewayPort: number;
|
||||
groupId: string;
|
||||
mcpAppFixture?: boolean;
|
||||
mockHost: string;
|
||||
mockPort: number;
|
||||
outputDir: string;
|
||||
repoRoot?: string;
|
||||
@@ -278,7 +279,7 @@ export function writeSutConfig(params: {
|
||||
openai: {
|
||||
api: "openai-responses",
|
||||
apiKey: { id: "OPENAI_API_KEY", provider: "default", source: "env" },
|
||||
baseUrl: `http://127.0.0.1:${params.mockPort}/v1`,
|
||||
baseUrl: `http://${params.mockHost}:${params.mockPort}/v1`,
|
||||
models: [
|
||||
{
|
||||
api: "openai-responses",
|
||||
@@ -569,15 +570,15 @@ export async function startMantisSut(params: {
|
||||
onRuntimeDisposed?: () => void;
|
||||
}): Promise<MantisSutRuntime> {
|
||||
const drained = await drainSutUpdates(params.sutToken);
|
||||
const config = writeSutConfig(params);
|
||||
const config = writeSutConfig({ ...params, mockHost: "mock-openai" });
|
||||
// The root wrapper relocates tempRoot into its bounded filesystem, then restores this
|
||||
// exact path as a symlink before Docker starts. Keep controller and claim paths anchored
|
||||
// here so live log reads, mock updates, stop, and destroy all share one runtime identity.
|
||||
const requestLog = path.join(config.tempRoot, "mock-openai-requests.ndjson");
|
||||
const mockLog = path.join(config.tempRoot, "mock-openai.log");
|
||||
const mockResponseControlDir = path.join(config.tempRoot, "mock-control");
|
||||
fs.mkdirSync(mockResponseControlDir, { mode: 0o700 });
|
||||
const mockResponseControl = path.join(mockResponseControlDir, "response.json");
|
||||
const requestLog = path.join(mockResponseControlDir, "mock-openai-requests.ndjson");
|
||||
const mockLog = path.join(mockResponseControlDir, "mock-openai.log");
|
||||
fs.writeFileSync(
|
||||
mockResponseControl,
|
||||
`${JSON.stringify({
|
||||
@@ -586,6 +587,8 @@ export async function startMantisSut(params: {
|
||||
})}\n`,
|
||||
{ mode: 0o600 },
|
||||
);
|
||||
fs.writeFileSync(requestLog, "", { mode: 0o600 });
|
||||
fs.writeFileSync(mockLog, "", { mode: 0o600 });
|
||||
const proxyControlDir = path.join(config.tempRoot, "proxy-control");
|
||||
fs.mkdirSync(proxyControlDir, { mode: 0o700 });
|
||||
const proxyControl = path.join(proxyControlDir, "control.json");
|
||||
|
||||
@@ -951,7 +951,7 @@ export async function startLocalSut(
|
||||
let mock: ReturnType<typeof spawnLogged> | undefined;
|
||||
try {
|
||||
const drained = await drainUpdates(params.sutToken);
|
||||
const config = writeConfig(params);
|
||||
const config = writeConfig({ ...params, mockHost: "127.0.0.1" });
|
||||
const requestLog = path.join(params.outputDir, "mock-openai-requests.ndjson");
|
||||
mock = spawnLoggedCommand(
|
||||
params.nodeBin ?? process.execPath,
|
||||
@@ -1154,7 +1154,7 @@ async function startLocalSutDaemon(params: {
|
||||
};
|
||||
}
|
||||
const drained = await drainSutUpdates(params.sutToken);
|
||||
const config = writeSutConfig(params);
|
||||
const config = writeSutConfig({ ...params, mockHost: "127.0.0.1" });
|
||||
const gatewayPassword = params.mcpAppFixture ? randomUUID() : undefined;
|
||||
const runtimeLogRoot = params.sutContainer ? config.tempRoot : params.outputDir;
|
||||
const requestLog = path.join(runtimeLogRoot, "mock-openai-requests.ndjson");
|
||||
|
||||
@@ -337,6 +337,24 @@ remove_container_or_fail() {
|
||||
fi
|
||||
}
|
||||
|
||||
wait_for_mock_openai() {
|
||||
local container_name="$1"
|
||||
local log_path="$2"
|
||||
local attempt=0
|
||||
until grep -q "mock-openai listening" "$log_path" 2>/dev/null; do
|
||||
if [[ "$("$docker_bin" inspect --format '{{.State.Running}}' "$container_name" 2>/dev/null)" != "true" ]]; then
|
||||
tail -n 20 "$log_path" >&2 || true
|
||||
die "mock OpenAI container exited before readiness"
|
||||
fi
|
||||
attempt=$((attempt + 1))
|
||||
if ((attempt >= 100)); then
|
||||
tail -n 20 "$log_path" >&2 || true
|
||||
die "mock OpenAI container did not become ready within 10 seconds"
|
||||
fi
|
||||
/bin/sleep 0.1
|
||||
done
|
||||
}
|
||||
|
||||
create_bounded_filesystem() {
|
||||
local name="$1"
|
||||
local size="$2"
|
||||
@@ -621,29 +639,7 @@ run_network_probe() {
|
||||
# shellcheck disable=SC2016
|
||||
readonly sut_command='
|
||||
set -eu
|
||||
mock_pid=""
|
||||
gateway_pid=""
|
||||
cleanup() {
|
||||
exit_code=$?
|
||||
trap - EXIT INT TERM
|
||||
if [ -n "$gateway_pid" ]; then kill "$gateway_pid" 2>/dev/null || true; fi
|
||||
if [ -n "$mock_pid" ]; then kill "$mock_pid" 2>/dev/null || true; fi
|
||||
wait 2>/dev/null || true
|
||||
exit "$exit_code"
|
||||
}
|
||||
trap cleanup EXIT INT TERM
|
||||
node /opt/mantis/mock-openai-server.mjs >"$MOCK_LOG" 2>&1 &
|
||||
mock_pid=$!
|
||||
attempt=0
|
||||
until grep -q "mock-openai listening" "$MOCK_LOG" 2>/dev/null; do
|
||||
kill -0 "$mock_pid" 2>/dev/null || exit 1
|
||||
attempt=$((attempt + 1))
|
||||
[ "$attempt" -lt 100 ] || exit 1
|
||||
sleep 0.1
|
||||
done
|
||||
node openclaw.mjs gateway --port "$OPENCLAW_GATEWAY_PORT" >"$GATEWAY_LOG" 2>&1 &
|
||||
gateway_pid=$!
|
||||
wait "$gateway_pid"
|
||||
exec node openclaw.mjs gateway --port "$OPENCLAW_GATEWAY_PORT" >"$GATEWAY_LOG" 2>&1
|
||||
'
|
||||
|
||||
command="${1:-}"
|
||||
@@ -794,13 +790,17 @@ case "$command" in
|
||||
[[ "$(stat -c %a "$response_control_dir")" == "700" ]] \
|
||||
|| die "mock response control directory mode mismatch"
|
||||
response_control="$response_control_dir/response.json"
|
||||
[[ -f "$response_control" && ! -L "$response_control" ]] || die "invalid mock response control"
|
||||
[[ "$(stat -c %u "$response_control")" == "$(id -u mantis-sut)" ]] \
|
||||
|| die "mock response control owner mismatch"
|
||||
[[ "$(stat -c %a "$response_control")" == "600" ]] \
|
||||
|| die "mock response control mode mismatch"
|
||||
[[ "$(stat -c %h "$response_control")" == "1" ]] \
|
||||
|| die "mock response control must not be hard-linked"
|
||||
request_log="$response_control_dir/mock-openai-requests.ndjson"
|
||||
mock_log="$response_control_dir/mock-openai.log"
|
||||
for file in "$response_control" "$request_log" "$mock_log"; do
|
||||
[[ -f "$file" && ! -L "$file" ]] || die "invalid mock control or evidence file"
|
||||
[[ "$(stat -c %u "$file")" == "$(id -u mantis-sut)" ]] \
|
||||
|| die "mock control or evidence file owner mismatch"
|
||||
[[ "$(stat -c %a "$file")" == "600" ]] \
|
||||
|| die "mock control or evidence file mode mismatch"
|
||||
[[ "$(stat -c %h "$file")" == "1" ]] \
|
||||
|| die "mock control or evidence file must not be hard-linked"
|
||||
done
|
||||
proxy_control_dir="$safe_runtime/proxy-control"
|
||||
[[ -d "$proxy_control_dir" && ! -L "$proxy_control_dir" ]] \
|
||||
|| die "invalid Telegram proxy control directory"
|
||||
@@ -819,7 +819,7 @@ case "$command" in
|
||||
[[ "$(stat -c %h "$file")" == "1" ]] \
|
||||
|| die "Telegram proxy control file must not be hard-linked"
|
||||
done
|
||||
for name in gateway.log mock-openai.log mock-openai-requests.ndjson sut-attestation.json; do
|
||||
for name in gateway.log sut-attestation.json; do
|
||||
[[ ! -e "$safe_runtime/$name" && ! -L "$safe_runtime/$name" ]] \
|
||||
|| die "runtime output was pre-created"
|
||||
done
|
||||
@@ -831,7 +831,6 @@ case "$command" in
|
||||
telegram_bot_id="${telegram_bot_token%%:*}"
|
||||
[[ "$telegram_bot_id" =~ ^[1-9][0-9]*$ ]] || die "invalid Telegram bot token"
|
||||
telegram_alias_token="${telegram_bot_id}:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"
|
||||
export SUCCESS_MARKER="$success_marker"
|
||||
export TELEGRAM_BOT_TOKEN="$telegram_alias_token"
|
||||
mock_response_chunk_delay_ms="$(jq -r '.mockResponseChunkDelayMs // ""' "$input_file")"
|
||||
gateway_password="$(jq -r '.gatewayPassword // ""' "$input_file")"
|
||||
@@ -839,19 +838,11 @@ case "$command" in
|
||||
trap - EXIT
|
||||
|
||||
gateway_log="$runtime_source/gateway.log"
|
||||
mock_log="$runtime_source/mock-openai.log"
|
||||
request_log="$runtime_source/mock-openai-requests.ndjson"
|
||||
install -T -o mantis-sut -g mantis-proof -m 0600 /dev/null "$safe_runtime/gateway.log"
|
||||
install -T -o mantis-sut -g mantis-proof -m 0600 /dev/null "$safe_runtime/mock-openai.log"
|
||||
install -T -o mantis-sut -g mantis-proof -m 0600 /dev/null "$safe_runtime/mock-openai-requests.ndjson"
|
||||
export CI=1
|
||||
export GATEWAY_LOG="$gateway_log"
|
||||
export GIT_COMMIT="$attested_sha"
|
||||
export HOME="$runtime_source/container-home"
|
||||
export MOCK_LOG="$mock_log"
|
||||
export MOCK_PORT="$mock_port"
|
||||
export MOCK_REQUEST_LOG="$request_log"
|
||||
export MOCK_RESPONSE_CONTROL="$runtime_source/mock-control/response.json"
|
||||
export NODE_DISABLE_COMPILE_CACHE=1
|
||||
export OPENAI_API_KEY=sk-openclaw-e2e-mock
|
||||
export OPENCLAW_BUILD_PRIVATE_QA=1
|
||||
@@ -861,19 +852,17 @@ case "$command" in
|
||||
export OPENCLAW_STATE_DIR="$runtime_source/state"
|
||||
if [[ -n "$mock_response_chunk_delay_ms" ]]; then
|
||||
require_positive_integer "$mock_response_chunk_delay_ms"
|
||||
export MOCK_RESPONSE_CHUNK_DELAY_MS="$mock_response_chunk_delay_ms"
|
||||
fi
|
||||
if [[ -n "$gateway_password" ]]; then
|
||||
export OPENCLAW_GATEWAY_PASSWORD="$gateway_password"
|
||||
fi
|
||||
|
||||
forwarded_env=(
|
||||
CI GATEWAY_LOG GIT_COMMIT HOME MOCK_LOG MOCK_PORT MOCK_REQUEST_LOG MOCK_RESPONSE_CONTROL NODE_DISABLE_COMPILE_CACHE
|
||||
CI GATEWAY_LOG GIT_COMMIT HOME NODE_DISABLE_COMPILE_CACHE
|
||||
OPENAI_API_KEY OPENCLAW_BUILD_PRIVATE_QA OPENCLAW_CONFIG_PATH
|
||||
OPENCLAW_ENABLE_PRIVATE_QA_CLI OPENCLAW_GATEWAY_PORT OPENCLAW_STATE_DIR
|
||||
SUCCESS_MARKER TELEGRAM_BOT_TOKEN
|
||||
TELEGRAM_BOT_TOKEN
|
||||
)
|
||||
[[ -z "${MOCK_RESPONSE_CHUNK_DELAY_MS:-}" ]] || forwarded_env+=(MOCK_RESPONSE_CHUNK_DELAY_MS)
|
||||
[[ -z "${OPENCLAW_GATEWAY_PASSWORD:-}" ]] || forwarded_env+=(OPENCLAW_GATEWAY_PASSWORD)
|
||||
docker_env=()
|
||||
for name in "${forwarded_env[@]}"; do
|
||||
@@ -882,6 +871,7 @@ case "$command" in
|
||||
|
||||
network_name="${container_name}-net"
|
||||
egress_network_name="${container_name}-egress"
|
||||
mock_container_name="${container_name}-mock-openai"
|
||||
proxy_container_name="${container_name}-telegram-proxy"
|
||||
[[ -f "$telegram_proxy_script" && ! -L "$telegram_proxy_script" ]] \
|
||||
|| die "missing trusted Telegram Bot API proxy"
|
||||
@@ -899,6 +889,7 @@ case "$command" in
|
||||
cleanup_run() {
|
||||
local result=0
|
||||
remove_container_or_fail "$container_name" || result=$?
|
||||
remove_container_or_fail "$mock_container_name" || result=$?
|
||||
remove_container_or_fail "$proxy_container_name" || result=$?
|
||||
cleanup_network "$network_name" || result=$?
|
||||
cleanup_network "$egress_network_name" || result=$?
|
||||
@@ -921,20 +912,44 @@ case "$command" in
|
||||
"$image" node /opt/mantis/telegram-bot-api-proxy.mjs >/dev/null
|
||||
"$docker_bin" network connect --alias telegram-api-proxy "$network_name" "$proxy_container_name"
|
||||
require_runtime_claim_active "$container_name"
|
||||
mock_env=(
|
||||
--env MOCK_BIND_HOST=0.0.0.0
|
||||
--env MOCK_PORT="$mock_port"
|
||||
--env MOCK_REQUEST_LOG=/opt/mantis/mock-control/mock-openai-requests.ndjson
|
||||
--env MOCK_RESPONSE_CONTROL=/opt/mantis/mock-control/response.json
|
||||
--env SUCCESS_MARKER="$success_marker"
|
||||
)
|
||||
if [[ -n "$mock_response_chunk_delay_ms" ]]; then
|
||||
mock_env+=(--env MOCK_RESPONSE_CHUNK_DELAY_MS="$mock_response_chunk_delay_ms")
|
||||
fi
|
||||
"$docker_bin" run --detach --name "$mock_container_name" --network "$network_name" \
|
||||
--network-alias mock-openai \
|
||||
"${container_security_args[@]}" "${proxy_resource_args[@]}" \
|
||||
--mount "type=bind,src=$mock_server_script,dst=/opt/mantis/mock-openai-server.mjs,readonly" \
|
||||
--mount "type=bind,src=$response_control_dir,dst=/opt/mantis/mock-control" \
|
||||
--user "$(id -u mantis-sut):$(id -g mantis-sut)" \
|
||||
"${mock_env[@]}" \
|
||||
"$image" sh -c 'exec node /opt/mantis/mock-openai-server.mjs >/opt/mantis/mock-control/mock-openai.log 2>&1' \
|
||||
>/dev/null
|
||||
wait_for_mock_openai "$mock_container_name" "$mock_log"
|
||||
require_runtime_claim_active "$container_name"
|
||||
# proxy-control holds the proxy's fault rules and recorded Bot API facts.
|
||||
# The SUT runs untrusted candidate code as the same mantis-sut UID, so an
|
||||
# inaccessible tmpfs must shadow the directory inside the runtime mount;
|
||||
# without it the lane under test could rewrite its own trusted evidence.
|
||||
# mock-control holds provider controls and evidence. Shadow it inside the SUT
|
||||
# so candidate code cannot read controls or forge provider evidence.
|
||||
"$docker_bin" run --rm --init --name "$container_name" --network "$network_name" \
|
||||
"${container_security_args[@]}" "${runtime_resource_args[@]}" \
|
||||
--mount "type=bind,src=$repo_root,dst=$repo_root,readonly" \
|
||||
--mount "type=bind,src=$mock_server_script,dst=/opt/mantis/mock-openai-server.mjs,readonly" \
|
||||
--mount "type=bind,src=$safe_runtime,dst=$runtime_source" \
|
||||
--mount "type=tmpfs,dst=$runtime_source/mock-control,tmpfs-size=65536,tmpfs-mode=0000" \
|
||||
--mount "type=tmpfs,dst=$runtime_source/proxy-control,tmpfs-size=65536,tmpfs-mode=0000" \
|
||||
--workdir "$repo_root" \
|
||||
--user "$(id -u mantis-sut):$(id -g mantis-sut)" \
|
||||
"${docker_env[@]}" \
|
||||
"$image" sh -c "$sut_command"
|
||||
remove_container_or_fail "$mock_container_name"
|
||||
remove_container_or_fail "$proxy_container_name"
|
||||
cleanup_network "$network_name"
|
||||
cleanup_network "$egress_network_name"
|
||||
@@ -956,6 +971,7 @@ case "$command" in
|
||||
terminate_runtime_claim
|
||||
stop_result=0
|
||||
remove_container_or_fail "$1" || stop_result=1
|
||||
remove_container_or_fail "${1}-mock-openai" || stop_result=1
|
||||
remove_container_or_fail "${1}-telegram-proxy" || stop_result=1
|
||||
cleanup_network "${1}-net" || stop_result=1
|
||||
cleanup_network "${1}-egress" || stop_result=1
|
||||
@@ -989,6 +1005,12 @@ case "$command" in
|
||||
exists_result=$?
|
||||
((exists_result == 1)) || exit "$exists_result"
|
||||
fi
|
||||
if container_exists "${1}-mock-openai"; then
|
||||
die "refusing to destroy a running mock OpenAI container"
|
||||
else
|
||||
exists_result=$?
|
||||
((exists_result == 1)) || exit "$exists_result"
|
||||
fi
|
||||
for network_name in "${1}-net" "${1}-egress"; do
|
||||
if network_exists "$network_name"; then
|
||||
die "refusing to destroy an active SUT network"
|
||||
|
||||
@@ -17,6 +17,7 @@ const scrubbedEnvKeys = [
|
||||
"CLICKCLACK_FIXTURE_PORT",
|
||||
"CLICKCLACK_FIXTURE_REQUEST_MAX_BYTES",
|
||||
"FIXTURE_PORT",
|
||||
"MOCK_BIND_HOST",
|
||||
"MOCK_PORT",
|
||||
"MOCK_REQUEST_LOG",
|
||||
"MOCK_RESPONSE_CHUNK_DELAY_MS",
|
||||
@@ -370,6 +371,111 @@ describe("mock OpenAI response markers", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it("records bounded media facts without provider payload bytes", async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), "openclaw-mock-content-facts-"));
|
||||
const requestLog = join(root, "requests.ndjson");
|
||||
const pdfBytes = "private-pdf-bytes";
|
||||
const pdfBase64 = Buffer.from(pdfBytes).toString("base64");
|
||||
try {
|
||||
await writeFile(requestLog, "");
|
||||
await withMockServer(mockOpenAiPath, { MOCK_REQUEST_LOG: requestLog }, async (baseUrl) => {
|
||||
const send = async (input: unknown) => {
|
||||
const response = await fetch(`${baseUrl}/v1/responses`, {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({ input, stream: false }),
|
||||
});
|
||||
expect(response.status).toBe(200);
|
||||
};
|
||||
await send([
|
||||
{
|
||||
type: "message",
|
||||
role: "user",
|
||||
content: Array.from({ length: 128 }, (_, index) => ({
|
||||
type: "input_text",
|
||||
text: `historical turn ${index}`,
|
||||
})),
|
||||
},
|
||||
{
|
||||
type: "message",
|
||||
role: "user",
|
||||
content: [
|
||||
{
|
||||
type: "input_file",
|
||||
filename: "proof.pdf",
|
||||
file_data: `data:application/pdf;base64,${pdfBase64}`,
|
||||
},
|
||||
{ type: "input_text", text: "Summarize the staged document." },
|
||||
],
|
||||
},
|
||||
]);
|
||||
await send([
|
||||
{
|
||||
type: "message",
|
||||
role: "user",
|
||||
content: [
|
||||
{
|
||||
type: "input_text",
|
||||
text: "[media attached: /tmp/session/proof.pdf (application/pdf)]\nSummarize it.",
|
||||
},
|
||||
],
|
||||
},
|
||||
]);
|
||||
|
||||
const recorded = await readFile(requestLog, "utf8");
|
||||
const entries = recorded
|
||||
.trim()
|
||||
.split("\n")
|
||||
.map((line) => JSON.parse(line));
|
||||
expect(entries[0]?.contentFacts).toHaveLength(128);
|
||||
expect(entries[0]?.contentFactsTruncated).toBe(true);
|
||||
expect(entries[0]?.contentFacts.slice(-2)).toEqual([
|
||||
{
|
||||
type: "input_file",
|
||||
filename: "proof.pdf",
|
||||
mimeType: "application/pdf",
|
||||
byteLength: Buffer.byteLength(pdfBytes),
|
||||
},
|
||||
{ type: "input_text" },
|
||||
]);
|
||||
expect(entries[1]?.contentFacts).toEqual([
|
||||
{ type: "input_text" },
|
||||
{
|
||||
type: "legacy_media",
|
||||
filename: "/tmp/session/proof.pdf",
|
||||
mimeType: "application/pdf",
|
||||
},
|
||||
]);
|
||||
expect(recorded).not.toContain(pdfBase64);
|
||||
expect(entries[0]?.body).toContain("data:application/pdf;base64,[redacted:17 bytes]");
|
||||
expect(entries.map((entry) => entry.seq)).toEqual([1, 2]);
|
||||
|
||||
// Redaction walks parsed JSON, so an unparseable body must never be
|
||||
// logged as raw text — that path would leak the base64 payload.
|
||||
const malformed = `{"input": "data:application/pdf;base64,${pdfBase64}"`;
|
||||
const response = await fetch(`${baseUrl}/v1/responses`, {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: malformed,
|
||||
});
|
||||
expect(response.status).toBe(200);
|
||||
const withMalformed = await readFile(requestLog, "utf8");
|
||||
expect(withMalformed).not.toContain(pdfBase64);
|
||||
const malformedEntry = withMalformed
|
||||
.trim()
|
||||
.split("\n")
|
||||
.map((line) => JSON.parse(line))
|
||||
.at(-1);
|
||||
expect(malformedEntry?.body).toBe(
|
||||
`[unparseable request body redacted: ${Buffer.byteLength(malformed)} bytes]`,
|
||||
);
|
||||
expect(malformedEntry?.seq).toBe(3);
|
||||
});
|
||||
} finally {
|
||||
await rm(root, { force: true, recursive: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("supports scripted connection drops", async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), "openclaw-mock-response-drop-"));
|
||||
const control = join(root, "response.json");
|
||||
|
||||
@@ -5,6 +5,7 @@ import { parse } from "yaml";
|
||||
|
||||
const PROOF_SCRIPT = "scripts/e2e/telegram-user-crabbox-proof.ts";
|
||||
const MANTIS_SUT_SCRIPT = "scripts/e2e/telegram-mantis-sut.ts";
|
||||
const MOCK_OPENAI_SERVER = "scripts/e2e/mock-openai-server.mjs";
|
||||
const MANTIS_LANE_SCRIPT = "scripts/e2e/telegram-mantis-lane.ts";
|
||||
const DESKTOP_CRABBOX_SCRIPT = "scripts/e2e/telegram-desktop-crabbox.ts";
|
||||
const SUT_CONTAINER_WRAPPER = "scripts/mantis/mantis-sut-container.sh";
|
||||
@@ -773,6 +774,13 @@ describe("Mantis Telegram Desktop proof workflow", () => {
|
||||
expect(prompt).toContain("`requests`");
|
||||
expect(prompt).toContain("`finish [--focus-message-id ID]`");
|
||||
expect(prompt).toContain("Identical pixels alone do not force `block`");
|
||||
// Precise trust claim: the sidecar makes facts tamper-evident (candidate
|
||||
// cannot rewrite records), but requests originate inside the untrusted SUT,
|
||||
// so the prompt must not present them as provenance-authenticated.
|
||||
expect(prompt).toContain("Provider request facts are tamper-evident comparison evidence");
|
||||
expect(prompt).toContain("not who sent it");
|
||||
expect(prompt).not.toContain("trusted, tamper-protected");
|
||||
expect(prompt).not.toContain("Provider request logs are diagnostic and pacing signals");
|
||||
expect(prompt).toContain("mantis-recipes/");
|
||||
expect(prompt).toContain("recipe-suggestion.md");
|
||||
expect(prompt).toContain("do not call `finish` and describe the block only in prose");
|
||||
@@ -1157,6 +1165,7 @@ describe("Mantis Telegram Desktop proof workflow", () => {
|
||||
it("does not pass the full workflow environment into the local Telegram SUT", () => {
|
||||
const sutScript = readFileSync(MANTIS_SUT_SCRIPT, "utf8");
|
||||
const laneScript = readFileSync(MANTIS_LANE_SCRIPT, "utf8");
|
||||
const mockServer = readFileSync(MOCK_OPENAI_SERVER, "utf8");
|
||||
const prompt = readFileSync(PROMPT, "utf8");
|
||||
const workflow = readFileSync(WORKFLOW, "utf8");
|
||||
const wrapper = readFileSync(SUT_CONTAINER_WRAPPER, "utf8");
|
||||
@@ -1228,10 +1237,7 @@ describe("Mantis Telegram Desktop proof workflow", () => {
|
||||
expect(workflow).toContain(
|
||||
'sudo install -m 0444 "$toolchain_build/scripts/e2e/mock-openai-server.mjs"',
|
||||
);
|
||||
expect(wrapper).toContain("node /opt/mantis/mock-openai-server.mjs");
|
||||
expect(wrapper).toContain(
|
||||
'--mount "type=bind,src=$mock_server_script,dst=/opt/mantis/mock-openai-server.mjs,readonly"',
|
||||
);
|
||||
expect(wrapper).not.toContain('node /opt/mantis/mock-openai-server.mjs >"$MOCK_LOG"');
|
||||
expect(wrapper).not.toContain("node scripts/e2e/mock-openai-server.mjs");
|
||||
expect(workflow).toContain('sudo usermod -aG mantis-proof "$recorder_user"');
|
||||
expect(workflow).toContain(
|
||||
@@ -1284,6 +1290,25 @@ describe("Mantis Telegram Desktop proof workflow", () => {
|
||||
expect(wrapper).toContain(
|
||||
"--env TELEGRAM_PROXY_RECORD_FILE=/opt/mantis/proxy-control/requests.ndjson",
|
||||
);
|
||||
const mockContainerSpec = wrapper.slice(
|
||||
wrapper.indexOf('"$docker_bin" run --detach --name "$mock_container_name"'),
|
||||
wrapper.indexOf('wait_for_mock_openai "$mock_container_name"'),
|
||||
);
|
||||
expect(mockContainerSpec).toContain('--network "$network_name"');
|
||||
expect(mockContainerSpec).toContain("--network-alias mock-openai");
|
||||
expect(mockContainerSpec).not.toContain("$egress_network_name");
|
||||
expect(mockContainerSpec).toContain(
|
||||
'--mount "type=bind,src=$mock_server_script,dst=/opt/mantis/mock-openai-server.mjs,readonly"',
|
||||
);
|
||||
expect(mockContainerSpec).toContain(
|
||||
'--mount "type=bind,src=$response_control_dir,dst=/opt/mantis/mock-control"',
|
||||
);
|
||||
expect(wrapper).toContain("--env MOCK_BIND_HOST=0.0.0.0");
|
||||
expect(mockContainerSpec).toContain('--user "$(id -u mantis-sut):$(id -g mantis-sut)"');
|
||||
expect(mockServer).toContain('const bindHost = process.env.MOCK_BIND_HOST ?? "127.0.0.1"');
|
||||
expect(mockServer).toContain("server.listen(port, bindHost");
|
||||
expect(wrapper).toContain('wait_for_mock_openai "$mock_container_name" "$mock_log"');
|
||||
expect(wrapper).toContain("mock OpenAI container exited before readiness");
|
||||
// Candidate code shares the mantis-sut UID with the proxy record sink, so
|
||||
// the SUT container must shadow proxy-control; otherwise the lane under
|
||||
// test could rewrite its own trusted Bot API evidence before publication.
|
||||
@@ -1293,8 +1318,16 @@ describe("Mantis Telegram Desktop proof workflow", () => {
|
||||
expect(wrapper.indexOf(proxyControlShadow)).toBeGreaterThan(
|
||||
wrapper.indexOf('--mount "type=bind,src=$safe_runtime,dst=$runtime_source"'),
|
||||
);
|
||||
const mockControlShadow =
|
||||
'--mount "type=tmpfs,dst=$runtime_source/mock-control,tmpfs-size=65536,tmpfs-mode=0000"';
|
||||
expect(wrapper).toContain(mockControlShadow);
|
||||
expect(wrapper.indexOf(mockControlShadow)).toBeGreaterThan(
|
||||
wrapper.indexOf('--mount "type=bind,src=$safe_runtime,dst=$runtime_source"'),
|
||||
);
|
||||
expect(wrapper).toContain('export TELEGRAM_BOT_TOKEN="$telegram_alias_token"');
|
||||
expect(wrapper).not.toContain('export TELEGRAM_BOT_TOKEN="$telegram_bot_token"');
|
||||
expect(wrapper.match(/remove_container_or_fail "\$mock_container_name"/gu)).toHaveLength(2);
|
||||
expect(wrapper).toContain('remove_container_or_fail "${1}-mock-openai"');
|
||||
expect(wrapper).toContain('remove_container_or_fail "${1}-telegram-proxy"');
|
||||
expect(workflow).toContain(
|
||||
"/usr/local/lib/mantis-toolchain/scripts/e2e/telegram-bot-api-proxy.mjs",
|
||||
@@ -1338,17 +1371,21 @@ describe("Mantis Telegram Desktop proof workflow", () => {
|
||||
'const proxyControlDir = path.join(config.tempRoot, "proxy-control")',
|
||||
);
|
||||
expect(sutScript).toContain(
|
||||
'const requestLog = path.join(config.tempRoot, "mock-openai-requests.ndjson")',
|
||||
'const requestLog = path.join(mockResponseControlDir, "mock-openai-requests.ndjson")',
|
||||
);
|
||||
expect(wrapper).toContain(
|
||||
'export MOCK_RESPONSE_CONTROL="$runtime_source/mock-control/response.json"',
|
||||
expect(sutScript).toContain(
|
||||
'const mockLog = path.join(mockResponseControlDir, "mock-openai.log")',
|
||||
);
|
||||
const forwardedEnv = wrapper.slice(
|
||||
wrapper.indexOf("forwarded_env=("),
|
||||
wrapper.indexOf("docker_env=()"),
|
||||
);
|
||||
expect(forwardedEnv).toContain("MOCK_RESPONSE_CONTROL");
|
||||
expect(forwardedEnv).not.toContain("MOCK_RESPONSE_CONTROL");
|
||||
expect(forwardedEnv).not.toContain("MOCK_REQUEST_LOG");
|
||||
expect(forwardedEnv).not.toContain("MOCK_LOG");
|
||||
expect(forwardedEnv).not.toContain("MOCK_PORT");
|
||||
expect(wrapper).toContain("refusing to destroy a running SUT container");
|
||||
expect(wrapper).toContain("refusing to destroy a running mock OpenAI container");
|
||||
expect(wrapper).toContain('destroy_bounded_filesystem "$runtime_root"');
|
||||
expect(wrapper).toContain('create_runtime_claim "$container_name" "$runtime_source"');
|
||||
expect(wrapper).toContain('cancel_runtime_claim "$1" "$runtime_source"');
|
||||
|
||||
@@ -922,6 +922,83 @@ exit 1
|
||||
}
|
||||
});
|
||||
|
||||
it("exposes provider content facts through requests and terminal lane facts", async () => {
|
||||
const harness = await setupHarness({ userOnlyEvents: true });
|
||||
const contentFacts = [
|
||||
{
|
||||
type: "input_file",
|
||||
filename: "proof.pdf",
|
||||
mimeType: "application/pdf",
|
||||
byteLength: 17,
|
||||
},
|
||||
];
|
||||
fs.writeFileSync(
|
||||
harness.requestLog,
|
||||
`${JSON.stringify({
|
||||
seq: 1,
|
||||
body: "credential=123456:secret-sut-token",
|
||||
contentFacts,
|
||||
path: "/v1/responses",
|
||||
})}\n`,
|
||||
);
|
||||
try {
|
||||
const requests = JSON.parse(
|
||||
(await runLane(harness.env, ["requests", "--lane", "candidate"])).stdout,
|
||||
);
|
||||
expect(requests).toEqual({
|
||||
count: 1,
|
||||
requests: [
|
||||
{
|
||||
seq: 1,
|
||||
body: "credential=[redacted]",
|
||||
contentFacts,
|
||||
path: "/v1/responses",
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
// Tail window: a session with more records than the window must expose
|
||||
// its newest requests — the ones under proof — with their absolute seq.
|
||||
fs.writeFileSync(
|
||||
harness.requestLog,
|
||||
Array.from(
|
||||
{ length: 130 },
|
||||
(_, i) => `${JSON.stringify({ seq: i + 1, body: `turn ${i + 1}` })}\n`,
|
||||
).join(""),
|
||||
);
|
||||
const tail = JSON.parse(
|
||||
(await runLane(harness.env, ["requests", "--lane", "candidate"])).stdout,
|
||||
);
|
||||
expect(tail.count).toBe(128);
|
||||
expect(tail.requests[0]).toEqual({ seq: 3, body: "turn 3" });
|
||||
expect(tail.requests.at(-1)).toEqual({ seq: 130, body: "turn 130" });
|
||||
|
||||
// Restore the single-record log so terminal lane facts mirror the
|
||||
// requests assertion above.
|
||||
fs.writeFileSync(
|
||||
harness.requestLog,
|
||||
`${JSON.stringify({
|
||||
seq: 1,
|
||||
body: "credential=123456:secret-sut-token",
|
||||
contentFacts,
|
||||
path: "/v1/responses",
|
||||
})}\n`,
|
||||
);
|
||||
await runLane(harness.env, ["send", "--lane", "candidate", "--text", "persist facts"]);
|
||||
await runLane(harness.env, ["finish", "--lane", "candidate"]);
|
||||
const facts = JSON.parse(
|
||||
fs.readFileSync(
|
||||
path.join(harness.outputRoot, "candidate", "mantis-lane-facts.json"),
|
||||
"utf8",
|
||||
),
|
||||
);
|
||||
expect(facts.providerRequests).toEqual(requests.requests);
|
||||
expect(JSON.stringify(facts.providerRequests)).not.toContain("secret-sut-token");
|
||||
} finally {
|
||||
await harness.close();
|
||||
}
|
||||
});
|
||||
|
||||
it("finishes an expected-silence proof on the triggering user message", async () => {
|
||||
const harness = await setupHarness({ userOnlyEvents: true });
|
||||
try {
|
||||
|
||||
@@ -201,6 +201,7 @@ describe("Telegram Mantis SUT", () => {
|
||||
},
|
||||
gatewayPort: 19_879,
|
||||
groupId: "-100123456789",
|
||||
mockHost: "mock-openai",
|
||||
mockPort: 19_882,
|
||||
outputDir,
|
||||
testerId: "12345",
|
||||
@@ -212,6 +213,7 @@ describe("Telegram Mantis SUT", () => {
|
||||
expect(config.channels.telegram.streaming).toEqual({ mode: "partial" });
|
||||
expect(config.channels.telegram).not.toHaveProperty("replyToMode");
|
||||
expect(config.commands.ownerAllowFrom).toEqual(["telegram:12345"]);
|
||||
expect(config.models.providers.openai.baseUrl).toBe("http://mock-openai:19882/v1");
|
||||
expect(config.session.sendPolicy).toEqual({ default: "deny" });
|
||||
});
|
||||
});
|
||||
|
||||
@@ -493,6 +493,7 @@ describe("telegram user Crabbox proof log polling", () => {
|
||||
gatewayPort: 19042,
|
||||
groupId: "group",
|
||||
mcpAppFixture: true,
|
||||
mockHost: "127.0.0.1",
|
||||
mockPort: 19043,
|
||||
outputDir: makeTempDir(tempDirs, "openclaw-telegram-proof-"),
|
||||
repoRoot: "/repo",
|
||||
@@ -520,6 +521,7 @@ describe("telegram user Crabbox proof log polling", () => {
|
||||
const configRoot = writeSutConfig({
|
||||
gatewayPort: 19042,
|
||||
groupId: "group",
|
||||
mockHost: "127.0.0.1",
|
||||
mockPort: 19043,
|
||||
outputDir: makeTempDir(tempDirs, "openclaw-telegram-proof-"),
|
||||
testerId: "tester",
|
||||
@@ -532,6 +534,7 @@ describe("telegram user Crabbox proof log polling", () => {
|
||||
executionIdentity: true,
|
||||
messages: "direct",
|
||||
});
|
||||
expect(config.models.providers.openai.baseUrl).toBe("http://127.0.0.1:19043/v1");
|
||||
});
|
||||
|
||||
it("injects the requested Telegram link-preview setting before startup", () => {
|
||||
@@ -539,6 +542,7 @@ describe("telegram user Crabbox proof log polling", () => {
|
||||
configPatch: { channels: { telegram: { linkPreview: false } } },
|
||||
gatewayPort: 19042,
|
||||
groupId: "group",
|
||||
mockHost: "127.0.0.1",
|
||||
mockPort: 19043,
|
||||
outputDir: makeTempDir(tempDirs, "openclaw-telegram-proof-"),
|
||||
testerId: "tester",
|
||||
@@ -546,6 +550,7 @@ describe("telegram user Crabbox proof log polling", () => {
|
||||
const defaultConfigRoot = writeSutConfig({
|
||||
gatewayPort: 19044,
|
||||
groupId: "group",
|
||||
mockHost: "127.0.0.1",
|
||||
mockPort: 19045,
|
||||
outputDir: makeTempDir(tempDirs, "openclaw-telegram-proof-"),
|
||||
testerId: "tester",
|
||||
@@ -570,6 +575,7 @@ describe("telegram user Crabbox proof log polling", () => {
|
||||
},
|
||||
gatewayPort: 19042,
|
||||
groupId: "group",
|
||||
mockHost: "127.0.0.1",
|
||||
mockPort: 19043,
|
||||
outputDir: makeTempDir(tempDirs, "openclaw-telegram-proof-"),
|
||||
testerId: "tester",
|
||||
@@ -577,6 +583,7 @@ describe("telegram user Crabbox proof log polling", () => {
|
||||
const defaultConfigRoot = writeSutConfig({
|
||||
gatewayPort: 19044,
|
||||
groupId: "group",
|
||||
mockHost: "127.0.0.1",
|
||||
mockPort: 19045,
|
||||
outputDir: makeTempDir(tempDirs, "openclaw-telegram-proof-"),
|
||||
testerId: "tester",
|
||||
|
||||
Reference in New Issue
Block a user