mirror of
https://github.com/openclaw/openclaw.git
synced 2026-08-12 21:53:00 -06:00
Fail Codex compaction at the Codex boundary (#85958)
This commit is contained in:
pashpashpash
+2
-2
@@ -247,7 +247,7 @@ Docs: https://docs.openclaw.ai
|
||||
- Gateway/agents: preserve fresh session overrides and metadata when stale cached agent-session entries race with store updates, so subagent model/provider overrides and routing policy survive concurrent writes. (#19328) Thanks @CodeReclaimers.
|
||||
- Control UI/chat: keep chat session search inline with the session selector so the header no longer shows a duplicate standalone search row.
|
||||
- Control UI/chat: collapse focused-mode header chrome and suppress hidden-header scroll updates so focus mode no longer jumps while scrolling. Thanks @amknight.
|
||||
- Codex app-server: restart the native app-server and retry once when server-side compaction times out, so preflight compaction stalls recover instead of failing every dispatch. (#85500)
|
||||
- Codex app-server: leave automatic compaction to native Codex, drop OpenClaw preflight/CLI/context-engine forced compaction for Codex runtime sessions, and still forward explicit `/compact` or plugin compaction requests into Codex while failing native compaction honestly. (#85500)
|
||||
- Restore Control UI gateway token pairing [AI]. (#85459) Thanks @pgondhi987.
|
||||
- OpenAI video: honor configured provider request private-network opt-in for local/custom video endpoints so explicitly trusted mock and self-hosted providers are not blocked. Thanks @shakkernerd.
|
||||
- OpenAI video: send uploaded video edit requests to the documented `/videos/edits` endpoint with a `video` file instead of posting MP4 references to `/videos`. Thanks @shakkernerd.
|
||||
@@ -332,7 +332,7 @@ Docs: https://docs.openclaw.ai
|
||||
- fix: constrain Windows task script names [AI]. (#85064) Thanks @pgondhi987.
|
||||
- Control UI: keep the chat session picker from hiding older or cross-agent configured conversations while preserving the bounded configured-agent refresh. (#85211) Thanks @amknight.
|
||||
- Agents/Anthropic: preserve unsafe integer tool-call input values in streamed Anthropic tool-use JSON, preventing Discord-style IDs from being rounded before dispatch. Fixes #47229. (#83063) Thanks @leno23.
|
||||
- Agents/Codex: estimate tool-heavy prompt pressure at the LLM boundary before provider submission, so persistent sessions compact before overflowing context windows. (#85541) Thanks @fuller-stack-dev and @joshavant.
|
||||
- Agents: estimate tool-heavy prompt pressure at the LLM boundary before provider submission for non-Codex embedded runtimes, so persistent PI-style sessions compact before overflowing context windows. (#85541) Thanks @fuller-stack-dev and @joshavant.
|
||||
- Agents/hooks: wait for local one-shot CLI and Codex `agent_end` plugin hooks before process cleanup so terminal observability flushes reliably. (#85007)
|
||||
- Providers/Google: preserve Gemini 3 cron `thinkingDefault: "low"` when stale catalog metadata says `reasoning:false`, so scheduled runs keep provider-supported thinking instead of downgrading to off. (#85185) Thanks @neeravmakwana.
|
||||
- CLI/agents: allow `openclaw agent --session-key` to target explicit session keys, including agent-scoped legacy keys. (#85121) Thanks @Kaspre.
|
||||
|
||||
@@ -20,8 +20,7 @@ diagnostic surfaces around that boundary.
|
||||
OpenClaw still owns channel routing, session files, visible message delivery,
|
||||
OpenClaw dynamic tools, approvals, media delivery, and a transcript mirror.
|
||||
Codex owns the canonical native thread, native model loop, native tool
|
||||
continuation, and native compaction unless the active OpenClaw context engine
|
||||
declares that it owns compaction.
|
||||
continuation, and native compaction.
|
||||
|
||||
Prompt routing follows the selected runtime, not just the provider string. A
|
||||
native Codex turn receives Codex app-server developer instructions, while an
|
||||
@@ -122,7 +121,7 @@ Supported in Codex runtime v1:
|
||||
| OpenClaw channel routing and delivery | Supported | Telegram, Discord, Slack, WhatsApp, iMessage, and other channels stay outside the model runtime. |
|
||||
| OpenClaw dynamic tools | Supported | Codex asks OpenClaw to execute these tools, so OpenClaw stays in the execution path. |
|
||||
| Prompt and context plugins | Supported | OpenClaw projects OpenClaw-specific prompt/context into the Codex turn while leaving Codex-owned base, model, personality, and configured project-doc prompts in the native Codex lane. Native Codex developer instructions accept only command guidance explicitly scoped to `codex_app_server`; legacy global command hints remain for non-Codex prompt surfaces. |
|
||||
| Context engine lifecycle | Supported | Assemble, ingest, after-turn maintenance, and context-engine compaction coordination run for Codex turns. |
|
||||
| Context engine lifecycle | Supported | Assemble, ingest, and after-turn maintenance run around Codex turns. Context engines do not replace native Codex compaction. |
|
||||
| Dynamic tool hooks | Supported | `before_tool_call`, `after_tool_call`, and tool-result middleware run around OpenClaw-owned dynamic tools. |
|
||||
| Lifecycle hooks | Supported as adapter observations | `llm_input`, `llm_output`, `agent_end`, `before_compaction`, and `after_compaction` fire with honest Codex-mode payloads. |
|
||||
| Final-answer revision gate | Supported through native hook relay | Codex `Stop` is relayed to `before_agent_finalize`; `revise` asks Codex for one more model pass before finalization. |
|
||||
@@ -137,8 +136,8 @@ Not supported in Codex runtime v1:
|
||||
| Native tool argument mutation | Codex native pre-tool hooks can block, but OpenClaw does not rewrite Codex-native tool arguments. | Requires Codex hook/schema support for replacement tool input. |
|
||||
| Editable Codex-native transcript history | Codex owns canonical native thread history. OpenClaw owns a mirror and can project future context, but should not mutate unsupported internals. | Add explicit Codex app-server APIs if native thread surgery is needed. |
|
||||
| `tool_result_persist` for Codex-native tool records | That hook transforms OpenClaw-owned transcript writes, not Codex-native tool records. | Could mirror transformed records, but canonical rewrite needs Codex support. |
|
||||
| Rich native compaction metadata | OpenClaw observes compaction start and completion, but does not receive a stable kept/dropped list, token delta, or summary payload. | Needs richer Codex compaction events. |
|
||||
| Compaction intervention | Current OpenClaw compaction hooks are notification-level in Codex mode. | Add Codex pre/post compaction hooks if plugins need to veto or rewrite native compaction. |
|
||||
| Rich native compaction metadata | OpenClaw can request native compaction, but does not receive a stable kept/dropped list, token delta, completion summary, or summary payload. | Needs richer Codex compaction events. |
|
||||
| Compaction intervention | OpenClaw does not let plugins or context engines veto, rewrite, or replace native Codex compaction. | Add Codex pre/post compaction hooks if plugins need to veto or rewrite native compaction. |
|
||||
| Byte-for-byte model API request capture | OpenClaw can capture app-server requests and notifications, but Codex core builds the final OpenAI API request internally. | Needs a Codex model-request tracing event or debug API. |
|
||||
|
||||
## Native permissions and MCP elicitations
|
||||
@@ -200,13 +199,18 @@ diagnostics bundle.
|
||||
|
||||
## Compaction and transcript mirror
|
||||
|
||||
When the selected model uses the Codex harness, native thread compaction is
|
||||
delegated to Codex app-server unless an active context engine declares
|
||||
`ownsCompaction: true`. Owning context engines compact first and cause OpenClaw
|
||||
to abandon the old Codex backend thread so the next turn can rehydrate a fresh
|
||||
thread from engine-managed context. OpenClaw keeps a transcript mirror for
|
||||
channel history, search, `/new`, `/reset`, and future model or harness
|
||||
switching.
|
||||
When the selected model uses the Codex harness, native thread compaction belongs
|
||||
to Codex app-server. OpenClaw does not run preflight compaction for Codex turns,
|
||||
does not replace Codex compaction with context-engine compaction, and does not
|
||||
fall back to OpenClaw or public OpenAI summarization when native Codex
|
||||
compaction cannot be started. OpenClaw keeps a transcript mirror for channel
|
||||
history, search, `/new`, `/reset`, and future model or harness switching.
|
||||
|
||||
Explicit compaction requests, such as `/compact` or a plugin-requested manual
|
||||
compact operation, start native Codex compaction with `thread/compact/start`.
|
||||
OpenClaw returns after starting that native operation. It does not wait for
|
||||
completion, impose a separate OpenClaw timeout, restart the shared Codex
|
||||
app-server, or record the operation as an OpenClaw-completed compaction.
|
||||
|
||||
When a context engine requests Codex thread-bootstrap projection, OpenClaw
|
||||
projects tool-call names and ids, input shapes, and redacted tool-result content
|
||||
@@ -215,9 +219,9 @@ that projection.
|
||||
|
||||
The mirror includes the user prompt, final assistant text, and lightweight Codex
|
||||
reasoning or plan records when the app-server emits them. Today, OpenClaw only
|
||||
records native compaction start and completion signals. It does not yet expose a
|
||||
human-readable compaction summary or an auditable list of which entries Codex
|
||||
kept after compaction.
|
||||
records explicit native compaction start signals when it requests compaction. It
|
||||
does not expose a human-readable compaction summary or an auditable list of
|
||||
which entries Codex kept after compaction.
|
||||
|
||||
Because Codex owns the canonical native thread, `tool_result_persist` does not
|
||||
currently rewrite Codex-native tool result records. It only applies when
|
||||
|
||||
@@ -125,28 +125,29 @@ Use `openai/gpt-*` model refs for Codex-backed OpenAI agent turns. Prefer
|
||||
`openai-codex:*` auth profiles and `auth.order.openai-codex` remain valid, but
|
||||
do not write new `openai-codex/gpt-*` model refs.
|
||||
|
||||
Do not set `compaction.model` or `compaction.provider` on Codex-backed agents
|
||||
unless a selected context engine owns compaction. Without an owning context
|
||||
engine, Codex compacts through its native app-server thread state, so OpenClaw
|
||||
ignores those local summarizer overrides at runtime and `openclaw doctor --fix`
|
||||
removes them when the agent uses Codex.
|
||||
Do not set `compaction.model` or `compaction.provider` on Codex-backed agents.
|
||||
Codex compacts through its native app-server thread state, so OpenClaw ignores
|
||||
those local summarizer overrides at runtime and `openclaw doctor --fix` removes
|
||||
them when the agent uses Codex.
|
||||
|
||||
Lossless remains supported as a context engine. Configure it through
|
||||
Lossless remains supported as a context engine for assembly, ingestion, and
|
||||
maintenance around Codex turns. Configure it through
|
||||
`plugins.slots.contextEngine: "lossless-claw"` and
|
||||
`plugins.entries.lossless-claw.config.summaryModel`, not through
|
||||
`agents.defaults.compaction.provider`. `openclaw doctor --fix` migrates the old
|
||||
`compaction.provider: "lossless-claw"` shape to the Lossless context-engine slot
|
||||
when Codex is the active runtime.
|
||||
when Codex is the active runtime, but native Codex still owns compaction.
|
||||
|
||||
The native Codex app-server harness supports context engines that require
|
||||
pre-prompt assembly. Generic CLI backends, including `codex-cli`, do not provide
|
||||
that host capability.
|
||||
|
||||
When the active context engine reports `ownsCompaction: true`, `/compact` runs
|
||||
that engine's compaction lifecycle and invalidates the bound Codex app-server
|
||||
thread. The next Codex turn starts a fresh backend thread and rehydrates it from
|
||||
the context engine instead of layering Codex native compaction on top of the
|
||||
engine-owned semantic summary.
|
||||
For Codex-backed agents, `/compact` starts native Codex app-server compaction on
|
||||
the bound thread. OpenClaw does not wait for completion, impose an OpenClaw
|
||||
timeout, restart the shared app-server, or fall back to a context-engine or
|
||||
public OpenAI summarizer. If the native Codex thread binding is missing or
|
||||
stale, the command fails closed so the operator sees the real runtime boundary
|
||||
instead of silently switching compaction backends.
|
||||
|
||||
```json5
|
||||
{
|
||||
@@ -656,10 +657,10 @@ The Codex harness changes the low-level embedded agent executor only.
|
||||
- Codex-native shell, patch, MCP, and native app tools are owned by Codex.
|
||||
OpenClaw can observe or block selected native events through the supported
|
||||
relay, but it does not rewrite native tool arguments.
|
||||
- Codex owns native compaction unless the active OpenClaw context engine
|
||||
declares `ownsCompaction: true`. OpenClaw keeps a transcript mirror for
|
||||
channel history, search, `/new`, `/reset`, and future model or harness
|
||||
switching.
|
||||
- Codex owns native compaction. OpenClaw keeps a transcript mirror for channel
|
||||
history, search, `/new`, `/reset`, and future model or harness switching, but
|
||||
it does not replace Codex compaction with an OpenClaw or context-engine
|
||||
summarizer.
|
||||
- Media generation, media understanding, TTS, approvals, and messaging-tool
|
||||
output continue through the matching OpenClaw provider/model settings.
|
||||
- `tool_result_persist` applies to OpenClaw-owned transcript tool results, not
|
||||
|
||||
@@ -99,50 +99,26 @@ describe("maybeCompactCodexAppServerSession", () => {
|
||||
await fs.rm(tempDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it("waits for native app-server compaction before reporting success", async () => {
|
||||
it("starts native app-server compaction without waiting for completion", async () => {
|
||||
const fake = createFakeCodexClient();
|
||||
setCodexAppServerClientFactoryForTest(async () => fake.client);
|
||||
const sessionFile = await writeTestBinding();
|
||||
|
||||
const pendingResult = startCompaction(sessionFile, { currentTokenCount: 123 });
|
||||
await vi.waitFor(() => {
|
||||
expect(fake.request).toHaveBeenCalledWith("thread/compact/start", { threadId: "thread-1" });
|
||||
});
|
||||
|
||||
let settled = false;
|
||||
void pendingResult.then(() => {
|
||||
settled = true;
|
||||
});
|
||||
await Promise.resolve();
|
||||
expect(settled).toBe(false);
|
||||
|
||||
fake.emit({
|
||||
method: "thread/compacted",
|
||||
params: { threadId: "thread-1", turnId: "turn-1" },
|
||||
});
|
||||
fake.emit({
|
||||
method: "thread/tokenUsage/updated",
|
||||
params: {
|
||||
threadId: "thread-1",
|
||||
tokenUsage: {
|
||||
last_token_usage: {
|
||||
total_tokens: 27_170,
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
const result = requireCompactResult(await pendingResult);
|
||||
const result = requireCompactResult(
|
||||
await startCompaction(sessionFile, { currentTokenCount: 123 }),
|
||||
);
|
||||
|
||||
expect(fake.request).toHaveBeenCalledWith("thread/compact/start", { threadId: "thread-1" });
|
||||
expect(fake.client.addNotificationHandler).not.toHaveBeenCalled();
|
||||
expect(result.ok).toBe(true);
|
||||
expect(result.compacted).toBe(true);
|
||||
expect(result.compacted).toBe(false);
|
||||
expect(result.result?.tokensBefore).toBe(123);
|
||||
expect(result.result?.tokensAfter).toBe(27_170);
|
||||
expect(result.result?.tokensAfter).toBeUndefined();
|
||||
const details = compactDetails(result);
|
||||
expect(details.backend).toBe("codex-app-server");
|
||||
expect(details.threadId).toBe("thread-1");
|
||||
expect(details.signal).toBe("thread/compacted");
|
||||
expect(details.turnId).toBe("turn-1");
|
||||
expect(details.tokenUsageSource).toBe("thread/tokenUsage/updated");
|
||||
expect(details.signal).toBe("thread/compact/start");
|
||||
expect(details.pending).toBe(true);
|
||||
});
|
||||
|
||||
it("blocks native app-server compaction when the current OpenClaw session is sandboxed", async () => {
|
||||
@@ -175,108 +151,14 @@ describe("maybeCompactCodexAppServerSession", () => {
|
||||
expect(fake.request).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("uses native token usage that arrives before compaction completion", async () => {
|
||||
it("does not consume native completion notifications after forwarding the request", async () => {
|
||||
const fake = createFakeCodexClient();
|
||||
setCodexAppServerClientFactoryForTest(async () => fake.client);
|
||||
const sessionFile = await writeTestBinding();
|
||||
|
||||
const pendingResult = startCompaction(sessionFile, { currentTokenCount: 123 });
|
||||
await vi.waitFor(() => {
|
||||
expect(fake.request).toHaveBeenCalledWith("thread/compact/start", { threadId: "thread-1" });
|
||||
});
|
||||
|
||||
fake.emit({
|
||||
method: "thread/tokenUsage/updated",
|
||||
params: {
|
||||
threadId: "thread-1",
|
||||
tokenUsage: {
|
||||
last_token_usage: {
|
||||
total_tokens: 18_004,
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
fake.emit({
|
||||
method: "thread/compacted",
|
||||
params: { threadId: "thread-1", turnId: "turn-1" },
|
||||
});
|
||||
const result = requireCompactResult(await pendingResult);
|
||||
|
||||
expect(result.ok).toBe(true);
|
||||
expect(result.compacted).toBe(true);
|
||||
expect(result.result?.tokensAfter).toBe(18_004);
|
||||
expect(compactDetails(result).tokenUsageSource).toBe("thread/tokenUsage/updated");
|
||||
});
|
||||
|
||||
it("accepts native current token usage with a total alias", async () => {
|
||||
const fake = createFakeCodexClient();
|
||||
setCodexAppServerClientFactoryForTest(async () => fake.client);
|
||||
const sessionFile = await writeTestBinding();
|
||||
|
||||
const pendingResult = startCompaction(sessionFile, { currentTokenCount: 123 });
|
||||
await vi.waitFor(() => {
|
||||
expect(fake.request).toHaveBeenCalledWith("thread/compact/start", { threadId: "thread-1" });
|
||||
});
|
||||
|
||||
fake.emit({
|
||||
method: "thread/tokenUsage/updated",
|
||||
params: {
|
||||
threadId: "thread-1",
|
||||
tokenUsage: {
|
||||
last: {
|
||||
total: 16_384,
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
fake.emit({
|
||||
method: "thread/compacted",
|
||||
params: { threadId: "thread-1", turnId: "turn-1" },
|
||||
});
|
||||
const result = requireCompactResult(await pendingResult);
|
||||
|
||||
expect(result.ok).toBe(true);
|
||||
expect(result.compacted).toBe(true);
|
||||
expect(result.result?.tokensAfter).toBe(16_384);
|
||||
expect(compactDetails(result).tokenUsageSource).toBe("thread/tokenUsage/updated");
|
||||
});
|
||||
|
||||
it("accepts native context-compaction item completion with unknown token count as success", async () => {
|
||||
const fake = createFakeCodexClient();
|
||||
setCodexAppServerClientFactoryForTest(async () => fake.client);
|
||||
const sessionFile = await writeTestBinding();
|
||||
|
||||
const pendingResult = startCompaction(sessionFile);
|
||||
await vi.waitFor(() => {
|
||||
expect(fake.request).toHaveBeenCalledWith("thread/compact/start", { threadId: "thread-1" });
|
||||
});
|
||||
fake.emit({
|
||||
method: "item/completed",
|
||||
params: {
|
||||
threadId: "thread-1",
|
||||
turnId: "turn-1",
|
||||
item: { type: "contextCompaction", id: "compact-1" },
|
||||
},
|
||||
});
|
||||
|
||||
const result = requireCompactResult(await pendingResult);
|
||||
expect(result.ok).toBe(true);
|
||||
expect(result.compacted).toBe(true);
|
||||
expect(result.result?.tokensAfter).toBeUndefined();
|
||||
const details = compactDetails(result);
|
||||
expect(details.signal).toBe("item/completed");
|
||||
expect(details.itemId).toBe("compact-1");
|
||||
});
|
||||
|
||||
it("does not treat zero native token usage as an authoritative post-compaction count", async () => {
|
||||
const fake = createFakeCodexClient();
|
||||
setCodexAppServerClientFactoryForTest(async () => fake.client);
|
||||
const sessionFile = await writeTestBinding();
|
||||
|
||||
const pendingResult = startCompaction(sessionFile, { currentTokenCount: 123 });
|
||||
await vi.waitFor(() => {
|
||||
expect(fake.request).toHaveBeenCalledWith("thread/compact/start", { threadId: "thread-1" });
|
||||
});
|
||||
const result = requireCompactResult(
|
||||
await startCompaction(sessionFile, { currentTokenCount: 123 }),
|
||||
);
|
||||
fake.emit({
|
||||
method: "thread/compacted",
|
||||
params: { threadId: "thread-1", turnId: "turn-1" },
|
||||
@@ -293,11 +175,11 @@ describe("maybeCompactCodexAppServerSession", () => {
|
||||
},
|
||||
});
|
||||
|
||||
const result = requireCompactResult(await pendingResult);
|
||||
expect(result.ok).toBe(true);
|
||||
expect(result.compacted).toBe(true);
|
||||
expect(result.compacted).toBe(false);
|
||||
expect(result.result?.tokensAfter).toBeUndefined();
|
||||
expect(compactDetails(result).tokenUsageSource).toBeUndefined();
|
||||
expect(compactDetails(result).signal).toBe("thread/compact/start");
|
||||
});
|
||||
|
||||
it("reuses the bound auth profile for native compaction", async () => {
|
||||
@@ -309,17 +191,10 @@ describe("maybeCompactCodexAppServerSession", () => {
|
||||
});
|
||||
const sessionFile = await writeTestBinding({ authProfileId: "openai-codex:work" });
|
||||
|
||||
const pendingResult = startCompaction(sessionFile);
|
||||
await vi.waitFor(() => {
|
||||
expect(fake.request).toHaveBeenCalledWith("thread/compact/start", { threadId: "thread-1" });
|
||||
});
|
||||
fake.emit({
|
||||
method: "thread/compacted",
|
||||
params: { threadId: "thread-1", turnId: "turn-1" },
|
||||
});
|
||||
await pendingResult;
|
||||
const result = requireCompactResult(await startCompaction(sessionFile));
|
||||
|
||||
expect(seenAuthProfileId).toBe("openai-codex:work");
|
||||
expect(result.ok).toBe(true);
|
||||
});
|
||||
|
||||
it("reports missing thread bindings as failed native compaction", async () => {
|
||||
@@ -355,87 +230,27 @@ describe("maybeCompactCodexAppServerSession", () => {
|
||||
expect(result.result).toBeUndefined();
|
||||
});
|
||||
|
||||
it("restarts the Codex app-server and retries when native compaction times out", async () => {
|
||||
const previousTimeout = process.env.OPENCLAW_CODEX_COMPACTION_WAIT_TIMEOUT_MS;
|
||||
process.env.OPENCLAW_CODEX_COMPACTION_WAIT_TIMEOUT_MS = "100";
|
||||
const warn = vi.spyOn(embeddedAgentLog, "warn").mockImplementation(() => undefined);
|
||||
try {
|
||||
const first = createFakeCodexClient();
|
||||
const second = createFakeCodexClient();
|
||||
let factoryCalls = 0;
|
||||
const factory = vi.fn(async () => {
|
||||
factoryCalls += 1;
|
||||
if (factoryCalls === 1) {
|
||||
return first.client;
|
||||
}
|
||||
return second.client;
|
||||
});
|
||||
setCodexAppServerClientFactoryForTest(factory);
|
||||
const sessionFile = await writeTestBinding();
|
||||
it("does not impose an OpenClaw timeout after Codex accepts native compaction", async () => {
|
||||
const fake = createFakeCodexClient();
|
||||
const factory = vi.fn(async () => fake.client);
|
||||
setCodexAppServerClientFactoryForTest(factory);
|
||||
const sessionFile = await writeTestBinding();
|
||||
|
||||
const pendingResult = startCompaction(sessionFile, { currentTokenCount: 456 });
|
||||
await vi.waitFor(() => {
|
||||
expect(first.request).toHaveBeenCalledWith("thread/compact/start", {
|
||||
threadId: "thread-1",
|
||||
});
|
||||
});
|
||||
const result = requireCompactResult(
|
||||
await startCompaction(sessionFile, { currentTokenCount: 456 }),
|
||||
);
|
||||
|
||||
await vi.waitFor(() => {
|
||||
expect(first.close).toHaveBeenCalledTimes(1);
|
||||
expect(second.request).toHaveBeenCalledWith("thread/compact/start", {
|
||||
threadId: "thread-1",
|
||||
});
|
||||
});
|
||||
second.emit({
|
||||
method: "thread/tokenUsage/updated",
|
||||
params: {
|
||||
threadId: "thread-1",
|
||||
tokenUsage: {
|
||||
last_token_usage: {
|
||||
total_tokens: 12_345,
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
second.emit({
|
||||
method: "item/completed",
|
||||
params: {
|
||||
threadId: "thread-1",
|
||||
turnId: "turn-2",
|
||||
item: { type: "contextCompaction", id: "compact-2" },
|
||||
},
|
||||
});
|
||||
|
||||
const result = requireCompactResult(await pendingResult);
|
||||
expect(result.ok).toBe(true);
|
||||
expect(result.compacted).toBe(true);
|
||||
expect(result.result?.tokensAfter).toBe(12_345);
|
||||
expect(factory).toHaveBeenCalledTimes(2);
|
||||
expect(second.close).not.toHaveBeenCalled();
|
||||
expect(await readCodexAppServerBinding(sessionFile)).toBeDefined();
|
||||
const details = compactDetails(result);
|
||||
expect(details.signal).toBe("item/completed");
|
||||
expect(details.itemId).toBe("compact-2");
|
||||
expect(details.compactionAttempts).toBe(2);
|
||||
expect(details.recoveredAfterAppServerRestart).toBe(true);
|
||||
expect(warn).toHaveBeenCalledWith(
|
||||
"codex app-server compaction timed out; restarting app-server",
|
||||
expect.objectContaining({
|
||||
sessionId: "session-1",
|
||||
sessionKey: "agent:main:session-1",
|
||||
threadId: "thread-1",
|
||||
attempt: 1,
|
||||
maxAttempts: 2,
|
||||
}),
|
||||
);
|
||||
} finally {
|
||||
if (previousTimeout === undefined) {
|
||||
delete process.env.OPENCLAW_CODEX_COMPACTION_WAIT_TIMEOUT_MS;
|
||||
} else {
|
||||
process.env.OPENCLAW_CODEX_COMPACTION_WAIT_TIMEOUT_MS = previousTimeout;
|
||||
}
|
||||
warn.mockRestore();
|
||||
}
|
||||
expect(result.ok).toBe(true);
|
||||
expect(result.compacted).toBe(false);
|
||||
expect(compactDetails(result)).toMatchObject({
|
||||
backend: "codex-app-server",
|
||||
threadId: "thread-1",
|
||||
signal: "thread/compact/start",
|
||||
pending: true,
|
||||
});
|
||||
expect(factory).toHaveBeenCalledTimes(1);
|
||||
expect(fake.close).not.toHaveBeenCalled();
|
||||
expect(await readCodexAppServerBinding(sessionFile)).toBeDefined();
|
||||
});
|
||||
|
||||
it("warns when stale OpenClaw compaction overrides are ignored", async () => {
|
||||
@@ -444,7 +259,7 @@ describe("maybeCompactCodexAppServerSession", () => {
|
||||
setCodexAppServerClientFactoryForTest(async () => fake.client);
|
||||
const sessionFile = await writeTestBinding();
|
||||
|
||||
const pendingResult = maybeCompactCodexAppServerSession({
|
||||
await maybeCompactCodexAppServerSession({
|
||||
sessionId: "session-1",
|
||||
sessionKey: "agent:main:session-1",
|
||||
sessionFile,
|
||||
@@ -460,15 +275,8 @@ describe("maybeCompactCodexAppServerSession", () => {
|
||||
},
|
||||
},
|
||||
});
|
||||
await vi.waitFor(() => {
|
||||
expect(fake.request).toHaveBeenCalledWith("thread/compact/start", { threadId: "thread-1" });
|
||||
});
|
||||
fake.emit({
|
||||
method: "thread/compacted",
|
||||
params: { threadId: "thread-1", turnId: "turn-1" },
|
||||
});
|
||||
await pendingResult;
|
||||
|
||||
expect(fake.request).toHaveBeenCalledWith("thread/compact/start", { threadId: "thread-1" });
|
||||
expect(warn).toHaveBeenCalledWith(
|
||||
"ignoring OpenClaw compaction overrides for Codex app-server compaction; Codex uses native server-side compaction",
|
||||
{
|
||||
@@ -486,16 +294,16 @@ describe("maybeCompactCodexAppServerSession", () => {
|
||||
setCodexAppServerClientFactoryForTest(async () => fake.client);
|
||||
const sessionFile = await writeTestBinding();
|
||||
|
||||
const pendingResult = maybeCompactCodexAppServerSession({
|
||||
await maybeCompactCodexAppServerSession({
|
||||
sessionId: "session-1",
|
||||
sessionKey: "agent:nik:session-1",
|
||||
sessionKey: "agent:sara:session-1",
|
||||
sessionFile,
|
||||
workspaceDir: tempDir,
|
||||
config: {
|
||||
agents: {
|
||||
list: [
|
||||
{
|
||||
id: "nik",
|
||||
id: "sara",
|
||||
compaction: {
|
||||
model: "openai/gpt-5.4-mini",
|
||||
provider: "openai",
|
||||
@@ -505,21 +313,17 @@ describe("maybeCompactCodexAppServerSession", () => {
|
||||
},
|
||||
},
|
||||
});
|
||||
await vi.waitFor(() => {
|
||||
expect(fake.request).toHaveBeenCalledWith("thread/compact/start", { threadId: "thread-1" });
|
||||
});
|
||||
fake.emit({
|
||||
method: "thread/compacted",
|
||||
params: { threadId: "thread-1", turnId: "turn-1" },
|
||||
});
|
||||
await pendingResult;
|
||||
|
||||
expect(fake.request).toHaveBeenCalledWith("thread/compact/start", { threadId: "thread-1" });
|
||||
expect(warn).toHaveBeenCalledWith(
|
||||
"ignoring OpenClaw compaction overrides for Codex app-server compaction; Codex uses native server-side compaction",
|
||||
{
|
||||
sessionId: "session-1",
|
||||
sessionKey: "agent:nik:session-1",
|
||||
ignoredConfig: ["agents.list.nik.compaction.model", "agents.list.nik.compaction.provider"],
|
||||
sessionKey: "agent:sara:session-1",
|
||||
ignoredConfig: [
|
||||
"agents.list.sara.compaction.model",
|
||||
"agents.list.sara.compaction.provider",
|
||||
],
|
||||
},
|
||||
);
|
||||
warn.mockRestore();
|
||||
@@ -531,7 +335,7 @@ describe("maybeCompactCodexAppServerSession", () => {
|
||||
setCodexAppServerClientFactoryForTest(async () => fake.client);
|
||||
const sessionFile = await writeTestBinding();
|
||||
|
||||
const pendingResult = maybeCompactCodexAppServerSession({
|
||||
await maybeCompactCodexAppServerSession({
|
||||
sessionId: "session-1",
|
||||
sessionKey: "agent:nik:session-1",
|
||||
sessionFile,
|
||||
@@ -554,15 +358,8 @@ describe("maybeCompactCodexAppServerSession", () => {
|
||||
},
|
||||
},
|
||||
});
|
||||
await vi.waitFor(() => {
|
||||
expect(fake.request).toHaveBeenCalledWith("thread/compact/start", { threadId: "thread-1" });
|
||||
});
|
||||
fake.emit({
|
||||
method: "thread/compacted",
|
||||
params: { threadId: "thread-1", turnId: "turn-1" },
|
||||
});
|
||||
await pendingResult;
|
||||
|
||||
expect(fake.request).toHaveBeenCalledWith("thread/compact/start", { threadId: "thread-1" });
|
||||
expect(warn).toHaveBeenCalledWith(
|
||||
"ignoring OpenClaw compaction overrides for Codex app-server compaction; Codex uses native server-side compaction",
|
||||
{
|
||||
@@ -574,8 +371,10 @@ describe("maybeCompactCodexAppServerSession", () => {
|
||||
warn.mockRestore();
|
||||
});
|
||||
|
||||
it("does not warn for legacy Lossless config when the Lossless context engine slot is active", async () => {
|
||||
it("warns for legacy Lossless config even when the Lossless context engine slot is active", async () => {
|
||||
const warn = vi.spyOn(embeddedAgentLog, "warn").mockImplementation(() => undefined);
|
||||
const fake = createFakeCodexClient();
|
||||
setCodexAppServerClientFactoryForTest(async () => fake.client);
|
||||
const sessionFile = await writeTestBinding();
|
||||
const contextEngine: ContextEngine = {
|
||||
info: { id: "lcm", name: "Lossless Context Manager", ownsCompaction: true },
|
||||
@@ -586,7 +385,7 @@ describe("maybeCompactCodexAppServerSession", () => {
|
||||
|
||||
await maybeCompactCodexAppServerSession({
|
||||
sessionId: "session-1",
|
||||
sessionKey: "agent:main:session-1",
|
||||
sessionKey: "agent:lossless:session-1",
|
||||
sessionFile,
|
||||
workspaceDir: tempDir,
|
||||
contextEngine,
|
||||
@@ -597,25 +396,38 @@ describe("maybeCompactCodexAppServerSession", () => {
|
||||
},
|
||||
},
|
||||
agents: {
|
||||
defaults: {
|
||||
compaction: {
|
||||
model: "openai/gpt-5.4",
|
||||
provider: "lossless-claw",
|
||||
list: [
|
||||
{
|
||||
id: "lossless",
|
||||
compaction: {
|
||||
model: "openai/gpt-5.4",
|
||||
provider: "lossless-claw",
|
||||
},
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
expect(warn).not.toHaveBeenCalledWith(
|
||||
expect(fake.request).toHaveBeenCalledWith("thread/compact/start", { threadId: "thread-1" });
|
||||
expect(warn).toHaveBeenCalledWith(
|
||||
"ignoring OpenClaw compaction overrides for Codex app-server compaction; Codex uses native server-side compaction",
|
||||
expect.anything(),
|
||||
{
|
||||
sessionId: "session-1",
|
||||
sessionKey: "agent:lossless:session-1",
|
||||
ignoredConfig: [
|
||||
"agents.list.lossless.compaction.model",
|
||||
"agents.list.lossless.compaction.provider",
|
||||
],
|
||||
},
|
||||
);
|
||||
warn.mockRestore();
|
||||
});
|
||||
|
||||
it("does not warn for inherited legacy Lossless provider when the Lossless slot is active", async () => {
|
||||
it("warns for inherited legacy Lossless provider when the Lossless slot is active", async () => {
|
||||
const warn = vi.spyOn(embeddedAgentLog, "warn").mockImplementation(() => undefined);
|
||||
const fake = createFakeCodexClient();
|
||||
setCodexAppServerClientFactoryForTest(async () => fake.client);
|
||||
const sessionFile = await writeTestBinding();
|
||||
const contextEngine: ContextEngine = {
|
||||
info: { id: "lcm", name: "Lossless Context Manager", ownsCompaction: true },
|
||||
@@ -626,7 +438,7 @@ describe("maybeCompactCodexAppServerSession", () => {
|
||||
|
||||
await maybeCompactCodexAppServerSession({
|
||||
sessionId: "session-1",
|
||||
sessionKey: "agent:nik:session-1",
|
||||
sessionKey: "agent:lossless-child:session-1",
|
||||
sessionFile,
|
||||
workspaceDir: tempDir,
|
||||
contextEngine,
|
||||
@@ -644,7 +456,7 @@ describe("maybeCompactCodexAppServerSession", () => {
|
||||
},
|
||||
list: [
|
||||
{
|
||||
id: "nik",
|
||||
id: "lossless-child",
|
||||
compaction: {
|
||||
model: "openai/gpt-5.4-mini",
|
||||
},
|
||||
@@ -654,9 +466,17 @@ describe("maybeCompactCodexAppServerSession", () => {
|
||||
},
|
||||
});
|
||||
|
||||
expect(warn).not.toHaveBeenCalledWith(
|
||||
expect(fake.request).toHaveBeenCalledWith("thread/compact/start", { threadId: "thread-1" });
|
||||
expect(warn).toHaveBeenCalledWith(
|
||||
"ignoring OpenClaw compaction overrides for Codex app-server compaction; Codex uses native server-side compaction",
|
||||
expect.anything(),
|
||||
{
|
||||
sessionId: "session-1",
|
||||
sessionKey: "agent:lossless-child:session-1",
|
||||
ignoredConfig: [
|
||||
"agents.defaults.compaction.provider",
|
||||
"agents.list.lossless-child.compaction.model",
|
||||
],
|
||||
},
|
||||
);
|
||||
warn.mockRestore();
|
||||
});
|
||||
@@ -688,8 +508,9 @@ describe("maybeCompactCodexAppServerSession", () => {
|
||||
expect(factory).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("runs owning context-engine compaction and invalidates the Codex thread binding", async () => {
|
||||
const info = vi.spyOn(embeddedAgentLog, "info").mockImplementation(() => undefined);
|
||||
it("forwards compaction to native Codex even when a context engine owns compaction", async () => {
|
||||
const fake = createFakeCodexClient();
|
||||
setCodexAppServerClientFactoryForTest(async () => fake.client);
|
||||
const sessionFile = await writeTestBinding();
|
||||
const compact = vi.fn(async () => ({
|
||||
ok: true,
|
||||
@@ -697,8 +518,7 @@ describe("maybeCompactCodexAppServerSession", () => {
|
||||
result: {
|
||||
summary: "engine summary",
|
||||
firstKeptEntryId: "entry-1",
|
||||
tokensBefore: 55,
|
||||
details: { engine: "lossless-claw" },
|
||||
tokensBefore: 123,
|
||||
},
|
||||
}));
|
||||
const maintain = vi.fn(
|
||||
@@ -716,399 +536,67 @@ describe("maybeCompactCodexAppServerSession", () => {
|
||||
maintain,
|
||||
};
|
||||
|
||||
const pendingResult = maybeCompactCodexAppServerSession({
|
||||
const result = requireCompactResult(
|
||||
await maybeCompactCodexAppServerSession({
|
||||
sessionId: "session-1",
|
||||
sessionKey: "agent:main:session-1",
|
||||
sessionFile,
|
||||
workspaceDir: tempDir,
|
||||
contextEngine,
|
||||
contextEngineRuntimeContext: { workspaceDir: tempDir, provider: "codex" },
|
||||
currentTokenCount: 123,
|
||||
trigger: "manual",
|
||||
}),
|
||||
);
|
||||
|
||||
expect(fake.request).toHaveBeenCalledWith("thread/compact/start", { threadId: "thread-1" });
|
||||
expect(result.ok).toBe(true);
|
||||
expect(result.compacted).toBe(false);
|
||||
expect(compactDetails(result)).toMatchObject({
|
||||
backend: "codex-app-server",
|
||||
threadId: "thread-1",
|
||||
signal: "thread/compact/start",
|
||||
pending: true,
|
||||
});
|
||||
expect(compact).not.toHaveBeenCalled();
|
||||
expect(maintain).not.toHaveBeenCalled();
|
||||
expect(await readCodexAppServerBinding(sessionFile)).toMatchObject({
|
||||
threadId: "thread-1",
|
||||
});
|
||||
});
|
||||
|
||||
it("requires a Codex binding instead of delegating to an owning context engine", async () => {
|
||||
const compact = vi.fn(async () => ({
|
||||
ok: true,
|
||||
compacted: true,
|
||||
result: {
|
||||
summary: "engine summary",
|
||||
firstKeptEntryId: "entry-1",
|
||||
tokensBefore: 123,
|
||||
},
|
||||
}));
|
||||
const contextEngine: ContextEngine = {
|
||||
info: { id: "lossless-claw", name: "Lossless Claw", ownsCompaction: true },
|
||||
assemble: vi.fn() as never,
|
||||
ingest: vi.fn() as never,
|
||||
compact,
|
||||
};
|
||||
|
||||
const result = await maybeCompactCodexAppServerSession({
|
||||
sessionId: "session-1",
|
||||
sessionKey: "agent:main:session-1",
|
||||
sessionFile,
|
||||
sessionFile: path.join(tempDir, "missing-binding.jsonl"),
|
||||
workspaceDir: tempDir,
|
||||
contextEngine,
|
||||
contextTokenBudget: 777,
|
||||
contextEngineRuntimeContext: { workspaceDir: tempDir, provider: "codex" },
|
||||
currentTokenCount: 123,
|
||||
trigger: "manual",
|
||||
});
|
||||
|
||||
const result = requireCompactResult(await pendingResult);
|
||||
expect(result.ok).toBe(true);
|
||||
expect(result.compacted).toBe(true);
|
||||
expect(result.result?.summary).toBe("engine summary");
|
||||
expect(result.result?.firstKeptEntryId).toBe("entry-1");
|
||||
expect(result.result?.tokensBefore).toBe(55);
|
||||
const details = compactDetails(result);
|
||||
expect(details.engine).toBe("lossless-claw");
|
||||
expect(details.codexThreadBindingInvalidated).toBe(true);
|
||||
expect(await readCodexAppServerBinding(sessionFile)).toBeUndefined();
|
||||
expect(compact).toHaveBeenCalledTimes(1);
|
||||
expect(compact).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
sessionId: "session-1",
|
||||
sessionKey: "agent:main:session-1",
|
||||
sessionFile,
|
||||
tokenBudget: 777,
|
||||
currentTokenCount: 123,
|
||||
compactionTarget: "threshold",
|
||||
customInstructions: undefined,
|
||||
force: true,
|
||||
runtimeContext: { workspaceDir: tempDir, provider: "codex" },
|
||||
abortSignal: expect.any(AbortSignal),
|
||||
}),
|
||||
);
|
||||
expect(maintain).toHaveBeenCalledTimes(1);
|
||||
const [maintainCall] = maintain.mock.calls[0] ?? [];
|
||||
const maintainParams = maintainCall as
|
||||
| {
|
||||
sessionId?: string;
|
||||
sessionKey?: string;
|
||||
sessionFile?: string;
|
||||
runtimeContext?: { workspaceDir?: string; provider?: string };
|
||||
}
|
||||
| undefined;
|
||||
expect(maintainParams?.sessionId).toBe("session-1");
|
||||
expect(maintainParams?.sessionKey).toBe("agent:main:session-1");
|
||||
expect(maintainParams?.sessionFile).toBe(sessionFile);
|
||||
expect(maintainParams?.runtimeContext?.workspaceDir).toBe(tempDir);
|
||||
expect(maintainParams?.runtimeContext?.provider).toBe("codex");
|
||||
expect(info).toHaveBeenCalledWith(
|
||||
"starting context-engine-owned Codex app-server compaction",
|
||||
expect.objectContaining({
|
||||
sessionId: "session-1",
|
||||
sessionKey: "agent:main:session-1",
|
||||
engineId: "lossless-claw",
|
||||
tokenBudget: 777,
|
||||
currentTokenCount: 123,
|
||||
trigger: "manual",
|
||||
compactionTarget: "threshold",
|
||||
force: true,
|
||||
}),
|
||||
);
|
||||
expect(info).toHaveBeenCalledWith(
|
||||
"completed context-engine-owned Codex app-server compaction",
|
||||
expect.objectContaining({
|
||||
sessionId: "session-1",
|
||||
sessionKey: "agent:main:session-1",
|
||||
engineId: "lossless-claw",
|
||||
ok: true,
|
||||
compacted: true,
|
||||
codexThreadBindingInvalidated: true,
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it("honors explicit force for budget-triggered owning context-engine compaction", async () => {
|
||||
const info = vi.spyOn(embeddedAgentLog, "info").mockImplementation(() => undefined);
|
||||
const sessionFile = await writeTestBinding();
|
||||
const compact = vi.fn(async () => ({
|
||||
ok: true,
|
||||
compacted: true,
|
||||
result: {
|
||||
summary: "engine summary",
|
||||
firstKeptEntryId: "entry-1",
|
||||
tokensBefore: 900,
|
||||
tokensAfter: 100,
|
||||
},
|
||||
}));
|
||||
const contextEngine: ContextEngine = {
|
||||
info: { id: "lossless-claw", name: "Lossless Claw", ownsCompaction: true },
|
||||
assemble: vi.fn() as never,
|
||||
ingest: vi.fn() as never,
|
||||
compact,
|
||||
};
|
||||
|
||||
const result = requireCompactResult(
|
||||
await maybeCompactCodexAppServerSession({
|
||||
sessionId: "session-1",
|
||||
sessionKey: "agent:main:session-1",
|
||||
sessionFile,
|
||||
workspaceDir: tempDir,
|
||||
contextEngine,
|
||||
contextTokenBudget: 777,
|
||||
currentTokenCount: 900,
|
||||
trigger: "budget",
|
||||
force: true,
|
||||
}),
|
||||
);
|
||||
|
||||
expect(result.ok).toBe(true);
|
||||
expect(result.compacted).toBe(true);
|
||||
expect(compact).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
compactionTarget: "budget",
|
||||
force: true,
|
||||
}),
|
||||
);
|
||||
expect(info).toHaveBeenCalledWith(
|
||||
"starting context-engine-owned Codex app-server compaction",
|
||||
expect.objectContaining({
|
||||
trigger: "budget",
|
||||
compactionTarget: "budget",
|
||||
force: true,
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it("adopts successor transcript handles after owning context-engine compaction", async () => {
|
||||
const sessionFile = await writeTestBinding();
|
||||
const successorFile = path.join(tempDir, "session.compacted.jsonl");
|
||||
await writeCodexAppServerBinding(successorFile, {
|
||||
threadId: "thread-successor",
|
||||
cwd: tempDir,
|
||||
});
|
||||
const compact = vi.fn(async () => ({
|
||||
ok: true,
|
||||
compacted: true,
|
||||
result: {
|
||||
summary: "engine summary",
|
||||
firstKeptEntryId: "entry-1",
|
||||
tokensBefore: 55,
|
||||
sessionId: "session-1-compacted",
|
||||
sessionFile: successorFile,
|
||||
},
|
||||
}));
|
||||
const maintain = vi.fn(
|
||||
async (_params: Parameters<NonNullable<ContextEngine["maintain"]>>[0]) => ({
|
||||
changed: false,
|
||||
bytesFreed: 0,
|
||||
rewrittenEntries: 0,
|
||||
}),
|
||||
);
|
||||
const contextEngine: ContextEngine = {
|
||||
info: { id: "lossless-claw", name: "Lossless Claw", ownsCompaction: true },
|
||||
assemble: vi.fn() as never,
|
||||
ingest: vi.fn() as never,
|
||||
compact,
|
||||
maintain,
|
||||
};
|
||||
|
||||
const result = requireCompactResult(
|
||||
await maybeCompactCodexAppServerSession({
|
||||
sessionId: "session-1",
|
||||
sessionKey: "agent:main:session-1",
|
||||
sessionFile,
|
||||
workspaceDir: tempDir,
|
||||
contextEngine,
|
||||
}),
|
||||
);
|
||||
|
||||
expect(result.ok).toBe(true);
|
||||
expect(result.compacted).toBe(true);
|
||||
expect(result.result?.sessionId).toBe("session-1-compacted");
|
||||
expect(result.result?.sessionFile).toBe(successorFile);
|
||||
expect(await readCodexAppServerBinding(sessionFile)).toBeUndefined();
|
||||
expect(await readCodexAppServerBinding(successorFile)).toBeUndefined();
|
||||
expect(maintain).toHaveBeenCalledTimes(1);
|
||||
const [maintainCall] = maintain.mock.calls[0] ?? [];
|
||||
const maintainParams = maintainCall as
|
||||
| {
|
||||
sessionId?: string;
|
||||
sessionFile?: string;
|
||||
}
|
||||
| undefined;
|
||||
expect(maintainParams?.sessionId).toBe("session-1-compacted");
|
||||
expect(maintainParams?.sessionFile).toBe(successorFile);
|
||||
});
|
||||
|
||||
it("returns context-engine compaction success when maintenance fails", async () => {
|
||||
const sessionFile = await writeTestBinding();
|
||||
const compact = vi.fn(async () => ({
|
||||
ok: true,
|
||||
compacted: true,
|
||||
result: {
|
||||
summary: "engine summary",
|
||||
firstKeptEntryId: "entry-1",
|
||||
tokensBefore: 55,
|
||||
},
|
||||
}));
|
||||
const contextEngine: ContextEngine = {
|
||||
info: { id: "lossless-claw", name: "Lossless Claw", ownsCompaction: true },
|
||||
assemble: vi.fn() as never,
|
||||
ingest: vi.fn() as never,
|
||||
compact,
|
||||
maintain: vi.fn(async () => {
|
||||
throw new Error("maintenance boom");
|
||||
}),
|
||||
};
|
||||
|
||||
const pendingResult = maybeCompactCodexAppServerSession({
|
||||
sessionId: "session-1",
|
||||
sessionKey: "agent:main:session-1",
|
||||
sessionFile,
|
||||
workspaceDir: tempDir,
|
||||
contextEngine,
|
||||
});
|
||||
|
||||
const result = requireCompactResult(await pendingResult);
|
||||
expect(result.ok).toBe(true);
|
||||
expect(result.compacted).toBe(true);
|
||||
expect(result.result?.summary).toBe("engine summary");
|
||||
const details = compactDetails(result);
|
||||
expect(details.codexThreadBindingInvalidated).toBe(true);
|
||||
expect(compact).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("does not require a Codex binding when the owning context engine compacts", async () => {
|
||||
const compact = vi.fn(async () => ({
|
||||
ok: true,
|
||||
compacted: true,
|
||||
result: {
|
||||
summary: "engine summary",
|
||||
firstKeptEntryId: "entry-1",
|
||||
tokensBefore: 8,
|
||||
},
|
||||
}));
|
||||
const maintain = vi.fn(async () => ({
|
||||
changed: false,
|
||||
bytesFreed: 0,
|
||||
rewrittenEntries: 0,
|
||||
}));
|
||||
const contextEngine: ContextEngine = {
|
||||
info: { id: "lossless-claw", name: "Lossless Claw", ownsCompaction: true },
|
||||
assemble: vi.fn() as never,
|
||||
ingest: vi.fn() as never,
|
||||
compact,
|
||||
maintain,
|
||||
};
|
||||
|
||||
const result = await maybeCompactCodexAppServerSession({
|
||||
sessionId: "session-1",
|
||||
sessionKey: "agent:main:session-1",
|
||||
sessionFile: path.join(tempDir, "missing-binding.jsonl"),
|
||||
workspaceDir: tempDir,
|
||||
contextEngine,
|
||||
});
|
||||
|
||||
const compactResult = requireCompactResult(result);
|
||||
expect(compactResult.ok).toBe(true);
|
||||
expect(compactResult.compacted).toBe(true);
|
||||
expect(compactResult.result?.summary).toBe("engine summary");
|
||||
expect(compact).toHaveBeenCalledTimes(1);
|
||||
expect(maintain).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("does not run context-engine maintenance when owning compaction does not compact", async () => {
|
||||
const maintain = vi.fn(async () => ({
|
||||
changed: false,
|
||||
bytesFreed: 0,
|
||||
rewrittenEntries: 0,
|
||||
}));
|
||||
const contextEngine: ContextEngine = {
|
||||
info: { id: "lossless-claw", name: "Lossless Claw", ownsCompaction: true },
|
||||
assemble: vi.fn() as never,
|
||||
ingest: vi.fn() as never,
|
||||
compact: vi.fn(async () => ({
|
||||
ok: true,
|
||||
compacted: false,
|
||||
reason: "below threshold",
|
||||
})),
|
||||
maintain,
|
||||
};
|
||||
|
||||
const result = await maybeCompactCodexAppServerSession({
|
||||
sessionId: "session-1",
|
||||
sessionKey: "agent:main:session-1",
|
||||
sessionFile: path.join(tempDir, "missing-binding.jsonl"),
|
||||
workspaceDir: tempDir,
|
||||
contextEngine,
|
||||
});
|
||||
|
||||
const compactResult = requireCompactResult(result);
|
||||
expect(compactResult.ok).toBe(true);
|
||||
expect(compactResult.compacted).toBe(false);
|
||||
expect(compactResult.reason).toBe("below threshold");
|
||||
expect(maintain).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
describe("owning context-engine compaction safety timeout", () => {
|
||||
afterEach(() => {
|
||||
vi.useRealTimers();
|
||||
});
|
||||
|
||||
it("bounds a hung owning context-engine compact() and reports a clean ok:false", async () => {
|
||||
const sessionFile = await writeTestBinding();
|
||||
const compact = vi.fn<ContextEngine["compact"]>(() => new Promise(() => {}));
|
||||
const contextEngine: ContextEngine = {
|
||||
info: { id: "lossless-claw", name: "Lossless Claw", ownsCompaction: true },
|
||||
assemble: vi.fn() as never,
|
||||
ingest: vi.fn() as never,
|
||||
compact,
|
||||
};
|
||||
|
||||
vi.useFakeTimers();
|
||||
const pendingResult = maybeCompactCodexAppServerSession({
|
||||
sessionId: "session-1",
|
||||
sessionKey: "agent:main:session-1",
|
||||
sessionFile,
|
||||
workspaceDir: tempDir,
|
||||
contextEngine,
|
||||
// 1 s host-resolved compaction timeout.
|
||||
config: { agents: { defaults: { compaction: { timeoutSeconds: 1 } } } },
|
||||
});
|
||||
|
||||
await vi.advanceTimersByTimeAsync(1_000);
|
||||
const result = requireCompactResult(await pendingResult);
|
||||
|
||||
expect(result.ok).toBe(false);
|
||||
expect(result.compacted).toBe(false);
|
||||
expect(result.reason).toContain("timed out");
|
||||
expect(compact).toHaveBeenCalledTimes(1);
|
||||
expect(vi.getTimerCount()).toBe(0);
|
||||
});
|
||||
|
||||
it("threads a composed caller abort signal into the owning context-engine compact()", async () => {
|
||||
const sessionFile = await writeTestBinding();
|
||||
const controller = new AbortController();
|
||||
const compact = vi.fn<ContextEngine["compact"]>(async () => ({
|
||||
ok: true,
|
||||
compacted: false,
|
||||
reason: "below threshold",
|
||||
}));
|
||||
const contextEngine: ContextEngine = {
|
||||
info: { id: "lossless-claw", name: "Lossless Claw", ownsCompaction: true },
|
||||
assemble: vi.fn() as never,
|
||||
ingest: vi.fn() as never,
|
||||
compact,
|
||||
};
|
||||
|
||||
await maybeCompactCodexAppServerSession({
|
||||
sessionId: "session-1",
|
||||
sessionKey: "agent:main:session-1",
|
||||
sessionFile,
|
||||
workspaceDir: tempDir,
|
||||
contextEngine,
|
||||
abortSignal: controller.signal,
|
||||
});
|
||||
|
||||
expect(compact).toHaveBeenCalledTimes(1);
|
||||
expect(compact.mock.calls[0]?.[0]?.abortSignal).toBeInstanceOf(AbortSignal);
|
||||
});
|
||||
|
||||
it("aborts a hung owning context-engine compact() when the caller signal fires", async () => {
|
||||
const sessionFile = await writeTestBinding();
|
||||
const controller = new AbortController();
|
||||
const compact = vi.fn<ContextEngine["compact"]>(() => new Promise(() => {}));
|
||||
const contextEngine: ContextEngine = {
|
||||
info: { id: "lossless-claw", name: "Lossless Claw", ownsCompaction: true },
|
||||
assemble: vi.fn() as never,
|
||||
ingest: vi.fn() as never,
|
||||
compact,
|
||||
};
|
||||
|
||||
const pendingResult = maybeCompactCodexAppServerSession({
|
||||
sessionId: "session-1",
|
||||
sessionKey: "agent:main:session-1",
|
||||
sessionFile,
|
||||
workspaceDir: tempDir,
|
||||
contextEngine,
|
||||
abortSignal: controller.signal,
|
||||
});
|
||||
|
||||
controller.abort(new Error("run aborted"));
|
||||
const result = requireCompactResult(await pendingResult);
|
||||
|
||||
expect(result.ok).toBe(false);
|
||||
expect(result.compacted).toBe(false);
|
||||
expect(result.reason).toContain("run aborted");
|
||||
expect(compact).toHaveBeenCalledTimes(1);
|
||||
expect(result).toMatchObject({
|
||||
ok: false,
|
||||
compacted: false,
|
||||
failure: { reason: "missing_thread_binding" },
|
||||
});
|
||||
expect(compact).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1121,14 +609,17 @@ function createFakeCodexClient(): {
|
||||
const handlers = new Set<(notification: CodexServerNotification) => void>();
|
||||
const request = vi.fn(async () => ({}));
|
||||
const close = vi.fn();
|
||||
const addNotificationHandler = vi.fn(
|
||||
(handler: (notification: CodexServerNotification) => void) => {
|
||||
handlers.add(handler);
|
||||
return () => handlers.delete(handler);
|
||||
},
|
||||
);
|
||||
return {
|
||||
client: {
|
||||
request,
|
||||
close,
|
||||
addNotificationHandler(handler: (notification: CodexServerNotification) => void) {
|
||||
handlers.add(handler);
|
||||
return () => handlers.delete(handler);
|
||||
},
|
||||
addNotificationHandler,
|
||||
} as unknown as CodexAppServerClient,
|
||||
request,
|
||||
close,
|
||||
|
||||
@@ -1,11 +1,5 @@
|
||||
import {
|
||||
compactContextEngineWithSafetyTimeout,
|
||||
embeddedAgentLog,
|
||||
formatErrorMessage,
|
||||
isActiveHarnessContextEngine,
|
||||
resolveCompactionTimeoutMs,
|
||||
resolveContextEngineOwnerPluginId,
|
||||
runHarnessContextEngineMaintenance,
|
||||
type CompactEmbeddedPiSessionParams,
|
||||
type EmbeddedPiCompactResult,
|
||||
} from "openclaw/plugin-sdk/agent-harness-runtime";
|
||||
@@ -13,198 +7,27 @@ import {
|
||||
defaultCodexAppServerClientFactory,
|
||||
type CodexAppServerClientFactory,
|
||||
} from "./client-factory.js";
|
||||
import type { CodexAppServerClient, CodexServerNotificationHandler } from "./client.js";
|
||||
import { resolveCodexAppServerRuntimeOptions } from "./config.js";
|
||||
import { isJsonObject, type CodexServerNotification, type JsonObject } from "./protocol.js";
|
||||
import type { JsonObject } from "./protocol.js";
|
||||
import { resolveCodexNativeExecutionBlock } from "./sandbox-guard.js";
|
||||
import { clearCodexAppServerBinding, readCodexAppServerBinding } from "./session-binding.js";
|
||||
type CodexNativeCompactionCompletion = {
|
||||
signal: "thread/compacted" | "item/completed";
|
||||
turnId?: string;
|
||||
itemId?: string;
|
||||
tokensAfter?: number;
|
||||
};
|
||||
type CodexNativeCompactionWaiter = {
|
||||
promise: Promise<CodexNativeCompactionCompletion>;
|
||||
startTimeout: () => void;
|
||||
cancel: () => void;
|
||||
};
|
||||
|
||||
const DEFAULT_CODEX_COMPACTION_WAIT_TIMEOUT_MS = 5 * 60 * 1000;
|
||||
const CODEX_COMPACTION_TOKEN_USAGE_GRACE_MS = 250;
|
||||
const MAX_CODEX_NATIVE_COMPACTION_ATTEMPTS = 2;
|
||||
const warnedIgnoredCompactionOverrides = new Set<string>();
|
||||
|
||||
class CodexNativeCompactionTimeoutError extends Error {
|
||||
constructor(readonly threadId: string) {
|
||||
super(`timed out waiting for codex app-server compaction for ${threadId}`);
|
||||
}
|
||||
}
|
||||
|
||||
export async function maybeCompactCodexAppServerSession(
|
||||
params: CompactEmbeddedPiSessionParams,
|
||||
options: { pluginConfig?: unknown; clientFactory?: CodexAppServerClientFactory } = {},
|
||||
): Promise<EmbeddedPiCompactResult | undefined> {
|
||||
const activeContextEngine = isActiveHarnessContextEngine(params.contextEngine)
|
||||
? params.contextEngine
|
||||
: undefined;
|
||||
if (activeContextEngine?.info.ownsCompaction) {
|
||||
return await compactOwningContextEngine(params, activeContextEngine);
|
||||
}
|
||||
warnIfIgnoringOpenClawCompactionOverrides(params);
|
||||
const nativeResult = await compactCodexNativeThread(params, options);
|
||||
if (activeContextEngine && nativeResult?.ok && nativeResult.compacted) {
|
||||
try {
|
||||
await runHarnessContextEngineMaintenance({
|
||||
contextEngine: activeContextEngine,
|
||||
sessionId: params.sessionId,
|
||||
sessionKey: params.sessionKey,
|
||||
sessionFile: params.sessionFile,
|
||||
reason: "compaction",
|
||||
runtimeContext: params.contextEngineRuntimeContext,
|
||||
config: params.config,
|
||||
});
|
||||
} catch (error) {
|
||||
embeddedAgentLog.warn("context engine compaction maintenance failed after Codex compaction", {
|
||||
sessionId: params.sessionId,
|
||||
engineId: activeContextEngine.info.id,
|
||||
error: formatErrorMessage(error),
|
||||
});
|
||||
}
|
||||
}
|
||||
return nativeResult;
|
||||
}
|
||||
|
||||
async function compactOwningContextEngine(
|
||||
params: CompactEmbeddedPiSessionParams,
|
||||
contextEngine: NonNullable<CompactEmbeddedPiSessionParams["contextEngine"]>,
|
||||
): Promise<EmbeddedPiCompactResult> {
|
||||
const compactionTarget = params.trigger === "manual" ? "threshold" : "budget";
|
||||
const force = params.force === true || params.trigger === "manual";
|
||||
embeddedAgentLog.info("starting context-engine-owned Codex app-server compaction", {
|
||||
sessionId: params.sessionId,
|
||||
sessionKey: params.sessionKey,
|
||||
engineId: contextEngine.info.id,
|
||||
tokenBudget: params.contextTokenBudget,
|
||||
currentTokenCount: params.currentTokenCount,
|
||||
trigger: params.trigger,
|
||||
compactionTarget,
|
||||
force,
|
||||
});
|
||||
let result: Awaited<ReturnType<typeof contextEngine.compact>>;
|
||||
try {
|
||||
// Bound the plugin-owned compaction with the same finite safety timeout
|
||||
// that protects native runtime compaction, and thread the caller's abort
|
||||
// signal through, so a slow/hung plugin compact() cannot hang the Codex
|
||||
// compaction lane indefinitely. A timeout/abort (or any thrown error) is
|
||||
// converted to a clean { ok: false } result by the catch below.
|
||||
result = await compactContextEngineWithSafetyTimeout(
|
||||
contextEngine,
|
||||
{
|
||||
sessionId: params.sessionId,
|
||||
sessionKey: params.sessionKey,
|
||||
sessionFile: params.sessionFile,
|
||||
tokenBudget: params.contextTokenBudget,
|
||||
currentTokenCount: params.currentTokenCount,
|
||||
compactionTarget,
|
||||
customInstructions: params.customInstructions,
|
||||
force,
|
||||
runtimeContext: params.contextEngineRuntimeContext,
|
||||
},
|
||||
resolveCompactionTimeoutMs(params.config),
|
||||
params.abortSignal,
|
||||
);
|
||||
} catch (error) {
|
||||
embeddedAgentLog.warn("context-engine-owned Codex app-server compaction failed", {
|
||||
sessionId: params.sessionId,
|
||||
sessionKey: params.sessionKey,
|
||||
engineId: contextEngine.info.id,
|
||||
error: formatErrorMessage(error),
|
||||
});
|
||||
return {
|
||||
ok: false,
|
||||
compacted: false,
|
||||
reason: `context engine compaction failed: ${formatErrorMessage(error)}`,
|
||||
};
|
||||
}
|
||||
|
||||
if (result.ok && result.compacted) {
|
||||
const compactedSessionId = result.result?.sessionId ?? params.sessionId;
|
||||
const compactedSessionFile = result.result?.sessionFile ?? params.sessionFile;
|
||||
try {
|
||||
await runHarnessContextEngineMaintenance({
|
||||
contextEngine,
|
||||
sessionId: compactedSessionId,
|
||||
sessionKey: params.sessionKey,
|
||||
sessionFile: compactedSessionFile,
|
||||
reason: "compaction",
|
||||
runtimeContext: params.contextEngineRuntimeContext,
|
||||
config: params.config,
|
||||
});
|
||||
} catch (error) {
|
||||
embeddedAgentLog.warn("context engine compaction maintenance failed", {
|
||||
sessionId: compactedSessionId,
|
||||
engineId: contextEngine.info.id,
|
||||
error: formatErrorMessage(error),
|
||||
});
|
||||
}
|
||||
await clearCodexAppServerBinding(params.sessionFile, { config: params.config });
|
||||
if (compactedSessionFile !== params.sessionFile) {
|
||||
await clearCodexAppServerBinding(compactedSessionFile, { config: params.config });
|
||||
}
|
||||
}
|
||||
|
||||
embeddedAgentLog.info("completed context-engine-owned Codex app-server compaction", {
|
||||
sessionId: params.sessionId,
|
||||
sessionKey: params.sessionKey,
|
||||
engineId: contextEngine.info.id,
|
||||
ok: result.ok,
|
||||
compacted: result.compacted,
|
||||
reason: result.reason,
|
||||
codexThreadBindingInvalidated: result.ok && result.compacted,
|
||||
});
|
||||
return {
|
||||
ok: result.ok,
|
||||
compacted: result.compacted,
|
||||
reason: result.reason,
|
||||
result: result.result
|
||||
? {
|
||||
...result.result,
|
||||
summary: result.result.summary ?? "",
|
||||
firstKeptEntryId: result.result.firstKeptEntryId ?? "",
|
||||
details: mergeContextEngineCompactionDetails(result.result.details, {
|
||||
codexThreadBindingInvalidated: result.ok && result.compacted,
|
||||
}),
|
||||
}
|
||||
: result.ok && result.compacted
|
||||
? {
|
||||
summary: "",
|
||||
firstKeptEntryId: "",
|
||||
tokensBefore: params.currentTokenCount ?? 0,
|
||||
details: { codexThreadBindingInvalidated: true },
|
||||
}
|
||||
: undefined,
|
||||
};
|
||||
}
|
||||
|
||||
function mergeContextEngineCompactionDetails(
|
||||
details: unknown,
|
||||
extra: Record<string, unknown>,
|
||||
): unknown {
|
||||
if (details && typeof details === "object" && !Array.isArray(details)) {
|
||||
return {
|
||||
...(details as Record<string, unknown>),
|
||||
...extra,
|
||||
};
|
||||
}
|
||||
return extra;
|
||||
// Codex owns automatic context-pressure compaction for Codex runtime sessions.
|
||||
// This entry point is only for explicit/manual compaction requests. OpenClaw
|
||||
// starts native Codex compaction for the bound thread and returns immediately;
|
||||
// Codex reports and applies the compaction inside its own app-server session.
|
||||
return compactCodexNativeThread(params, options);
|
||||
}
|
||||
|
||||
function warnIfIgnoringOpenClawCompactionOverrides(params: CompactEmbeddedPiSessionParams): void {
|
||||
const activeContextEngine = isActiveHarnessContextEngine(params.contextEngine)
|
||||
? params.contextEngine
|
||||
: undefined;
|
||||
const ignoredConfig = readIgnoredCompactionOverridePaths(params, activeContextEngine);
|
||||
const ignoredConfig = readIgnoredCompactionOverridePaths(params);
|
||||
if (ignoredConfig.length === 0) {
|
||||
return;
|
||||
}
|
||||
@@ -223,21 +46,8 @@ function warnIfIgnoringOpenClawCompactionOverrides(params: CompactEmbeddedPiSess
|
||||
);
|
||||
}
|
||||
|
||||
function readIgnoredCompactionOverridePaths(
|
||||
params: CompactEmbeddedPiSessionParams,
|
||||
activeContextEngine?: CompactEmbeddedPiSessionParams["contextEngine"],
|
||||
): string[] {
|
||||
function readIgnoredCompactionOverridePaths(params: CompactEmbeddedPiSessionParams): string[] {
|
||||
const ignored = new Set<string>();
|
||||
const configuredContextEngine = readStringPath(params.config, [
|
||||
"plugins",
|
||||
"slots",
|
||||
"contextEngine",
|
||||
]);
|
||||
const runtimeContextEnginePlugin =
|
||||
typeof params.contextEngineRuntimeContext?.contextEnginePluginId === "string"
|
||||
? params.contextEngineRuntimeContext.contextEnginePluginId.trim()
|
||||
: "";
|
||||
const activeContextEnginePlugin = resolveContextEngineOwnerPluginId(activeContextEngine);
|
||||
for (const entry of readCompactionOverrideEntries(params)) {
|
||||
const localProvider =
|
||||
typeof entry.record.provider === "string" ? entry.record.provider.trim() : "";
|
||||
@@ -245,20 +55,11 @@ function readIgnoredCompactionOverridePaths(
|
||||
!localProvider && typeof entry.inheritedRecord?.provider === "string"
|
||||
? entry.inheritedRecord.provider.trim()
|
||||
: "";
|
||||
const provider = localProvider || inheritedProvider;
|
||||
const providerPath = localProvider
|
||||
? `${entry.path}.compaction.provider`
|
||||
: inheritedProvider && entry.inheritedPath
|
||||
? `${entry.inheritedPath}.compaction.provider`
|
||||
: undefined;
|
||||
const activeLosslessContextEngine =
|
||||
provider.toLowerCase() === "lossless-claw" &&
|
||||
(activeContextEnginePlugin === "lossless-claw" ||
|
||||
runtimeContextEnginePlugin.toLowerCase() === "lossless-claw" ||
|
||||
configuredContextEngine?.toLowerCase() === "lossless-claw");
|
||||
if (activeLosslessContextEngine) {
|
||||
continue;
|
||||
}
|
||||
if (typeof entry.record.model === "string" && entry.record.model.trim()) {
|
||||
ignored.add(`${entry.path}.compaction.model`);
|
||||
}
|
||||
@@ -322,14 +123,6 @@ function readRecord(value: unknown): Record<string, unknown> | undefined {
|
||||
: undefined;
|
||||
}
|
||||
|
||||
function readStringPath(value: unknown, path: readonly string[]): string | undefined {
|
||||
let current = value;
|
||||
for (const segment of path) {
|
||||
current = readRecord(current)?.[segment];
|
||||
}
|
||||
return typeof current === "string" && current.trim() ? current.trim() : undefined;
|
||||
}
|
||||
|
||||
async function compactCodexNativeThread(
|
||||
params: CompactEmbeddedPiSessionParams,
|
||||
options: { pluginConfig?: unknown; clientFactory?: CodexAppServerClientFactory } = {},
|
||||
@@ -361,106 +154,54 @@ async function compactCodexNativeThread(
|
||||
}
|
||||
|
||||
const clientFactory = options.clientFactory ?? defaultCodexAppServerClientFactory;
|
||||
let completion: CodexNativeCompactionCompletion | undefined;
|
||||
let attempt = 0;
|
||||
for (attempt = 1; attempt <= MAX_CODEX_NATIVE_COMPACTION_ATTEMPTS; attempt += 1) {
|
||||
const client = await clientFactory(
|
||||
appServer.start,
|
||||
requestedAuthProfileId ?? binding.authProfileId,
|
||||
params.agentDir,
|
||||
params.config,
|
||||
);
|
||||
const waiter = createCodexNativeCompactionWaiter(client, binding.threadId);
|
||||
try {
|
||||
await client.request("thread/compact/start", {
|
||||
const client = await clientFactory(
|
||||
appServer.start,
|
||||
requestedAuthProfileId ?? binding.authProfileId,
|
||||
params.agentDir,
|
||||
params.config,
|
||||
);
|
||||
try {
|
||||
await client.request("thread/compact/start", {
|
||||
threadId: binding.threadId,
|
||||
});
|
||||
embeddedAgentLog.info("started codex app-server compaction", {
|
||||
sessionId: params.sessionId,
|
||||
threadId: binding.threadId,
|
||||
});
|
||||
} catch (error) {
|
||||
if (isCodexThreadNotFoundError(error)) {
|
||||
await clearCodexAppServerBinding(params.sessionFile, { config: params.config });
|
||||
return failedCodexThreadBindingCompactionResult(params, {
|
||||
threadId: binding.threadId,
|
||||
});
|
||||
embeddedAgentLog.info("started codex app-server compaction", {
|
||||
sessionId: params.sessionId,
|
||||
threadId: binding.threadId,
|
||||
attempt,
|
||||
});
|
||||
waiter.startTimeout();
|
||||
completion = await waiter.promise;
|
||||
break;
|
||||
} catch (error) {
|
||||
waiter.cancel();
|
||||
if (isCodexThreadNotFoundError(error)) {
|
||||
await clearCodexAppServerBinding(params.sessionFile, { config: params.config });
|
||||
return failedCodexThreadBindingCompactionResult(params, {
|
||||
threadId: binding.threadId,
|
||||
reason: formatCompactionError(error),
|
||||
recovery: "stale_thread_binding",
|
||||
});
|
||||
}
|
||||
if (
|
||||
isCodexNativeCompactionTimeoutError(error, binding.threadId) &&
|
||||
attempt < MAX_CODEX_NATIVE_COMPACTION_ATTEMPTS
|
||||
) {
|
||||
restartCodexAppServerAfterNativeCompactionTimeout(
|
||||
client,
|
||||
params,
|
||||
binding.threadId,
|
||||
attempt,
|
||||
);
|
||||
continue;
|
||||
}
|
||||
if (isCodexNativeCompactionTimeoutError(error, binding.threadId)) {
|
||||
restartCodexAppServerAfterNativeCompactionTimeout(
|
||||
client,
|
||||
params,
|
||||
binding.threadId,
|
||||
attempt,
|
||||
);
|
||||
}
|
||||
return {
|
||||
ok: false,
|
||||
compacted: false,
|
||||
reason: formatCompactionError(error),
|
||||
};
|
||||
recovery: "stale_thread_binding",
|
||||
});
|
||||
}
|
||||
}
|
||||
if (!completion) {
|
||||
embeddedAgentLog.warn("codex app-server compaction failed", {
|
||||
sessionId: params.sessionId,
|
||||
sessionKey: params.sessionKey,
|
||||
threadId: binding.threadId,
|
||||
reason: formatCompactionError(error),
|
||||
});
|
||||
return {
|
||||
ok: false,
|
||||
compacted: false,
|
||||
reason: `codex app-server compaction did not complete for ${binding.threadId}`,
|
||||
reason: formatCompactionError(error),
|
||||
};
|
||||
}
|
||||
embeddedAgentLog.info("completed codex app-server compaction", {
|
||||
sessionId: params.sessionId,
|
||||
threadId: binding.threadId,
|
||||
signal: completion.signal,
|
||||
turnId: completion.turnId,
|
||||
itemId: completion.itemId,
|
||||
tokensAfter: completion.tokensAfter,
|
||||
});
|
||||
const resultDetails: JsonObject = {
|
||||
backend: "codex-app-server",
|
||||
threadId: binding.threadId,
|
||||
signal: completion.signal,
|
||||
signal: "thread/compact/start",
|
||||
pending: true,
|
||||
};
|
||||
if (completion.turnId) {
|
||||
resultDetails.turnId = completion.turnId;
|
||||
}
|
||||
if (completion.itemId) {
|
||||
resultDetails.itemId = completion.itemId;
|
||||
}
|
||||
if (completion.tokensAfter !== undefined) {
|
||||
resultDetails.tokenUsageSource = "thread/tokenUsage/updated";
|
||||
}
|
||||
if (attempt > 1) {
|
||||
resultDetails.compactionAttempts = attempt;
|
||||
resultDetails.recoveredAfterAppServerRestart = true;
|
||||
}
|
||||
return {
|
||||
ok: true,
|
||||
compacted: true,
|
||||
compacted: false,
|
||||
result: {
|
||||
summary: "",
|
||||
firstKeptEntryId: "",
|
||||
tokensBefore: params.currentTokenCount ?? 0,
|
||||
...(completion.tokensAfter !== undefined ? { tokensAfter: completion.tokensAfter } : {}),
|
||||
details: resultDetails,
|
||||
},
|
||||
};
|
||||
@@ -496,227 +237,6 @@ function isCodexThreadNotFoundError(error: unknown): boolean {
|
||||
return formatCompactionError(error).toLowerCase().includes("thread not found");
|
||||
}
|
||||
|
||||
function isCodexNativeCompactionTimeoutError(error: unknown, threadId: string): boolean {
|
||||
return error instanceof CodexNativeCompactionTimeoutError && error.threadId === threadId;
|
||||
}
|
||||
|
||||
function restartCodexAppServerAfterNativeCompactionTimeout(
|
||||
client: CodexAppServerClient,
|
||||
params: CompactEmbeddedPiSessionParams,
|
||||
threadId: string,
|
||||
attempt: number,
|
||||
): void {
|
||||
embeddedAgentLog.warn("codex app-server compaction timed out; restarting app-server", {
|
||||
sessionId: params.sessionId,
|
||||
sessionKey: params.sessionKey,
|
||||
threadId,
|
||||
attempt,
|
||||
maxAttempts: MAX_CODEX_NATIVE_COMPACTION_ATTEMPTS,
|
||||
});
|
||||
client.close();
|
||||
}
|
||||
|
||||
function createCodexNativeCompactionWaiter(
|
||||
client: CodexAppServerClient,
|
||||
threadId: string,
|
||||
): CodexNativeCompactionWaiter {
|
||||
let settled = false;
|
||||
let removeHandler: () => void = () => {};
|
||||
let timeout: ReturnType<typeof setTimeout> | undefined;
|
||||
let tokenUsageGraceTimeout: ReturnType<typeof setTimeout> | undefined;
|
||||
let failWaiter: (error: Error) => void = () => {};
|
||||
|
||||
const promise = new Promise<CodexNativeCompactionCompletion>((resolve, reject) => {
|
||||
const cleanup = (): void => {
|
||||
removeHandler();
|
||||
if (timeout) {
|
||||
clearTimeout(timeout);
|
||||
}
|
||||
if (tokenUsageGraceTimeout) {
|
||||
clearTimeout(tokenUsageGraceTimeout);
|
||||
}
|
||||
};
|
||||
const complete = (completion: CodexNativeCompactionCompletion): void => {
|
||||
if (settled) {
|
||||
return;
|
||||
}
|
||||
settled = true;
|
||||
cleanup();
|
||||
resolve(completion);
|
||||
};
|
||||
const fail = (error: Error): void => {
|
||||
if (settled) {
|
||||
return;
|
||||
}
|
||||
settled = true;
|
||||
cleanup();
|
||||
reject(error);
|
||||
};
|
||||
let latestTokensAfter: number | undefined;
|
||||
const completionWithLatestTokenUsage = (
|
||||
completion: CodexNativeCompactionCompletion,
|
||||
): CodexNativeCompactionCompletion =>
|
||||
latestTokensAfter === undefined
|
||||
? completion
|
||||
: { ...completion, tokensAfter: latestTokensAfter };
|
||||
const completeAfterTokenUsageGrace = (completion: CodexNativeCompactionCompletion): void => {
|
||||
if (settled || tokenUsageGraceTimeout) {
|
||||
return;
|
||||
}
|
||||
if (timeout) {
|
||||
clearTimeout(timeout);
|
||||
timeout = undefined;
|
||||
}
|
||||
tokenUsageGraceTimeout = setTimeout(
|
||||
() => complete(completionWithLatestTokenUsage(observedCompletion ?? completion)),
|
||||
CODEX_COMPACTION_TOKEN_USAGE_GRACE_MS,
|
||||
);
|
||||
tokenUsageGraceTimeout.unref?.();
|
||||
};
|
||||
failWaiter = fail;
|
||||
let observedCompletion: CodexNativeCompactionCompletion | undefined;
|
||||
const handler: CodexServerNotificationHandler = (notification) => {
|
||||
const tokensAfter = readNativeCompactionTokenUsage(notification, threadId);
|
||||
if (tokensAfter !== undefined) {
|
||||
latestTokensAfter = tokensAfter;
|
||||
if (observedCompletion) {
|
||||
complete(completionWithLatestTokenUsage(observedCompletion));
|
||||
return;
|
||||
}
|
||||
}
|
||||
const completion = readNativeCompactionCompletion(notification, threadId);
|
||||
if (completion) {
|
||||
observedCompletion = completionWithLatestTokenUsage({
|
||||
...observedCompletion,
|
||||
...completion,
|
||||
});
|
||||
if (latestTokensAfter !== undefined) {
|
||||
complete(observedCompletion);
|
||||
return;
|
||||
}
|
||||
completeAfterTokenUsageGrace(observedCompletion);
|
||||
}
|
||||
};
|
||||
removeHandler = client.addNotificationHandler(handler);
|
||||
});
|
||||
|
||||
return {
|
||||
promise,
|
||||
startTimeout(): void {
|
||||
if (settled || timeout) {
|
||||
return;
|
||||
}
|
||||
timeout = setTimeout(() => {
|
||||
failWaiter(new CodexNativeCompactionTimeoutError(threadId));
|
||||
}, resolveCompactionWaitTimeoutMs());
|
||||
timeout.unref?.();
|
||||
},
|
||||
cancel(): void {
|
||||
if (settled) {
|
||||
return;
|
||||
}
|
||||
settled = true;
|
||||
removeHandler();
|
||||
if (timeout) {
|
||||
clearTimeout(timeout);
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function readNativeCompactionTokenUsage(
|
||||
notification: CodexServerNotification,
|
||||
threadId: string,
|
||||
): number | undefined {
|
||||
const params = notification.params;
|
||||
if (!isJsonObject(params) || readString(params, "threadId", "thread_id") !== threadId) {
|
||||
return undefined;
|
||||
}
|
||||
if (notification.method !== "thread/tokenUsage/updated") {
|
||||
return undefined;
|
||||
}
|
||||
const tokenUsage = isJsonObject(params.tokenUsage) ? params.tokenUsage : undefined;
|
||||
const currentUsage = readCodexCurrentTokenUsage(tokenUsage) ?? readCodexCurrentTokenUsage(params);
|
||||
return readCodexTotalTokens(currentUsage);
|
||||
}
|
||||
|
||||
function readCodexCurrentTokenUsage(value: JsonObject | undefined): JsonObject | undefined {
|
||||
if (!value) {
|
||||
return undefined;
|
||||
}
|
||||
for (const key of [
|
||||
"last",
|
||||
"current",
|
||||
"lastCall",
|
||||
"lastCallUsage",
|
||||
"lastTokenUsage",
|
||||
"last_token_usage",
|
||||
]) {
|
||||
const usage = value[key];
|
||||
if (isJsonObject(usage)) {
|
||||
return usage;
|
||||
}
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
function readCodexTotalTokens(value: JsonObject | undefined): number | undefined {
|
||||
const totalTokens = value?.total_tokens ?? value?.totalTokens ?? value?.total;
|
||||
return typeof totalTokens === "number" && Number.isFinite(totalTokens) && totalTokens > 0
|
||||
? Math.floor(totalTokens)
|
||||
: undefined;
|
||||
}
|
||||
|
||||
function readNativeCompactionCompletion(
|
||||
notification: CodexServerNotification,
|
||||
threadId: string,
|
||||
): CodexNativeCompactionCompletion | undefined {
|
||||
const params = notification.params;
|
||||
if (!isJsonObject(params) || readString(params, "threadId", "thread_id") !== threadId) {
|
||||
return undefined;
|
||||
}
|
||||
if (notification.method === "thread/compacted") {
|
||||
return {
|
||||
signal: "thread/compacted",
|
||||
turnId: readString(params, "turnId", "turn_id"),
|
||||
};
|
||||
}
|
||||
if (notification.method !== "item/completed") {
|
||||
return undefined;
|
||||
}
|
||||
const item = isJsonObject(params.item) ? params.item : undefined;
|
||||
if (readString(item, "type") !== "contextCompaction") {
|
||||
return undefined;
|
||||
}
|
||||
return {
|
||||
signal: "item/completed",
|
||||
turnId: readString(params, "turnId", "turn_id"),
|
||||
itemId: readString(item, "id") ?? readString(params, "itemId", "item_id", "id"),
|
||||
};
|
||||
}
|
||||
|
||||
function resolveCompactionWaitTimeoutMs(): number {
|
||||
const raw = process.env.OPENCLAW_CODEX_COMPACTION_WAIT_TIMEOUT_MS?.trim();
|
||||
const parsed = raw ? Number.parseInt(raw, 10) : Number.NaN;
|
||||
if (Number.isFinite(parsed) && parsed > 0) {
|
||||
return parsed;
|
||||
}
|
||||
return DEFAULT_CODEX_COMPACTION_WAIT_TIMEOUT_MS;
|
||||
}
|
||||
|
||||
function readString(params: JsonObject | undefined, ...keys: string[]): string | undefined {
|
||||
if (!params) {
|
||||
return undefined;
|
||||
}
|
||||
for (const key of keys) {
|
||||
const value = params[key];
|
||||
if (typeof value === "string") {
|
||||
return value;
|
||||
}
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
function formatCompactionError(error: unknown): string {
|
||||
if (error instanceof Error) {
|
||||
return error.message;
|
||||
|
||||
@@ -877,9 +877,8 @@ describe("runCodexAppServerAttempt context-engine lifecycle", () => {
|
||||
expect(savedBinding?.contextEngine?.projection).toBeUndefined();
|
||||
});
|
||||
|
||||
it("retries a resumed context-engine thread on a fresh Codex thread after early context overflow", async () => {
|
||||
it("retries a resumed context-engine thread on a fresh Codex thread without plugin compaction", async () => {
|
||||
const sessionFile = path.join(tempDir, "session.jsonl");
|
||||
const successorFile = path.join(tempDir, "session.compacted.jsonl");
|
||||
const workspaceDir = path.join(tempDir, "workspace");
|
||||
SessionManager.open(sessionFile).appendMessage(
|
||||
assistantMessage("pre-compaction context", Date.now()) as never,
|
||||
@@ -900,12 +899,7 @@ describe("runCodexAppServerAttempt context-engine lifecycle", () => {
|
||||
},
|
||||
},
|
||||
});
|
||||
let epoch = "epoch-before";
|
||||
const compact = vi.fn(async () => {
|
||||
epoch = "epoch-after";
|
||||
SessionManager.open(successorFile).appendMessage(
|
||||
assistantMessage("successor compacted context", Date.now()) as never,
|
||||
);
|
||||
return {
|
||||
ok: true,
|
||||
compacted: true,
|
||||
@@ -914,7 +908,6 @@ describe("runCodexAppServerAttempt context-engine lifecycle", () => {
|
||||
firstKeptEntryId: "entry-1",
|
||||
tokensBefore: 10,
|
||||
sessionId: "session-1-compacted",
|
||||
sessionFile: successorFile,
|
||||
},
|
||||
};
|
||||
});
|
||||
@@ -922,12 +915,12 @@ describe("runCodexAppServerAttempt context-engine lifecycle", () => {
|
||||
async ({ messages, prompt }: Parameters<ContextEngine["assemble"]>[0]) => ({
|
||||
messages: [
|
||||
...messages,
|
||||
assistantMessage(`context ${epoch}`, 10),
|
||||
assistantMessage("context epoch-before", 10),
|
||||
userMessage(prompt ?? "", 11),
|
||||
],
|
||||
estimatedTokens: 42,
|
||||
systemPromptAddition: "context-engine system",
|
||||
contextProjection: { mode: "thread_bootstrap" as const, epoch },
|
||||
contextProjection: { mode: "thread_bootstrap" as const, epoch: "epoch-before" },
|
||||
}),
|
||||
);
|
||||
const contextEngine = createContextEngine({ assemble, compact });
|
||||
@@ -975,40 +968,18 @@ describe("runCodexAppServerAttempt context-engine lifecycle", () => {
|
||||
const result = await run;
|
||||
|
||||
expect(result.assistantTexts).toContain("fresh answer");
|
||||
expect(compact).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
sessionId: "session-1",
|
||||
sessionKey: "agent:main:session-1",
|
||||
sessionFile,
|
||||
tokenBudget: 400_000,
|
||||
currentTokenCount: 400_000,
|
||||
compactionTarget: "threshold",
|
||||
force: true,
|
||||
}),
|
||||
);
|
||||
expect(assemble).toHaveBeenCalledTimes(2);
|
||||
const retryAssembleParams = assemble.mock.calls[1]?.[0];
|
||||
expect(retryAssembleParams?.messages.map((message) => message.role)).toEqual(["assistant"]);
|
||||
const retryAssembleMessageTexts = retryAssembleParams?.messages.map((message) => {
|
||||
if (!("content" in message) || !Array.isArray(message.content)) {
|
||||
return "";
|
||||
}
|
||||
const firstContent = message.content[0];
|
||||
return typeof firstContent === "object" && firstContent !== null && "text" in firstContent
|
||||
? firstContent.text
|
||||
: "";
|
||||
});
|
||||
expect(retryAssembleMessageTexts).toEqual(["successor compacted context"]);
|
||||
expect(compact).not.toHaveBeenCalled();
|
||||
expect(assemble).toHaveBeenCalledTimes(1);
|
||||
const retryInputText = getRequestInputTextAt(harness, -1);
|
||||
expect(retryInputText).toContain("successor compacted context");
|
||||
expect(retryInputText).not.toContain("pre-compaction context");
|
||||
const savedBinding = await readCodexAppServerBinding(successorFile);
|
||||
expect(retryInputText).toBe("hello");
|
||||
expect(retryInputText).not.toContain("successor compacted context");
|
||||
const savedBinding = await readCodexAppServerBinding(sessionFile);
|
||||
expect(savedBinding?.threadId).toBe("thread-fresh");
|
||||
expect(savedBinding?.contextEngine?.engineId).toBe("lossless-claw");
|
||||
expect(savedBinding?.contextEngine?.projection?.epoch).toBe("epoch-after");
|
||||
expect(savedBinding?.contextEngine?.projection?.epoch).toBe("epoch-before");
|
||||
});
|
||||
|
||||
it("compacts over-budget rendered context-engine prompts before Codex turn/start", async () => {
|
||||
it("does not pre-compact over-budget rendered context-engine prompts before Codex turn/start", async () => {
|
||||
const sessionFile = path.join(tempDir, "session.jsonl");
|
||||
const workspaceDir = path.join(tempDir, "workspace");
|
||||
SessionManager.open(sessionFile).appendMessage(
|
||||
@@ -1025,20 +996,11 @@ describe("runCodexAppServerAttempt context-engine lifecycle", () => {
|
||||
compacted: true,
|
||||
result: { summary: "summary", firstKeptEntryId: "entry-1", tokensBefore: 100_000 },
|
||||
}));
|
||||
const assemble = vi
|
||||
.fn<ContextEngine["assemble"]>()
|
||||
.mockResolvedValueOnce({
|
||||
messages: Array.from({ length: 8 }, (_, index) =>
|
||||
toolResultMessage(hugePayload, index + 1),
|
||||
),
|
||||
estimatedTokens: 100_000,
|
||||
contextProjection: { mode: "thread_bootstrap", epoch: "epoch-before" },
|
||||
})
|
||||
.mockResolvedValueOnce({
|
||||
messages: [assistantMessage("successor compacted context", 2) as never],
|
||||
estimatedTokens: 100,
|
||||
contextProjection: { mode: "thread_bootstrap", epoch: "epoch-after" },
|
||||
});
|
||||
const assemble = vi.fn<ContextEngine["assemble"]>().mockResolvedValue({
|
||||
messages: Array.from({ length: 8 }, (_, index) => toolResultMessage(hugePayload, index + 1)),
|
||||
estimatedTokens: 100_000,
|
||||
contextProjection: { mode: "thread_bootstrap", epoch: "epoch-before" },
|
||||
});
|
||||
const contextEngine = createContextEngine({ assemble, compact });
|
||||
const harness = createStartedThreadHarness();
|
||||
const params = createParams(sessionFile, workspaceDir);
|
||||
@@ -1048,22 +1010,58 @@ describe("runCodexAppServerAttempt context-engine lifecycle", () => {
|
||||
const run = runCodexAppServerAttempt(params);
|
||||
await harness.waitForMethod("turn/start");
|
||||
|
||||
expect(compact).toHaveBeenCalledTimes(1);
|
||||
expect(assemble).toHaveBeenCalledTimes(2);
|
||||
expect(compact).not.toHaveBeenCalled();
|
||||
expect(assemble).toHaveBeenCalledTimes(1);
|
||||
expect(harness.requests.map((request) => request.method)).toEqual([
|
||||
"thread/start",
|
||||
"turn/start",
|
||||
]);
|
||||
const inputText = getRequestInputText(harness);
|
||||
expect(inputText).toContain("successor compacted context");
|
||||
expect(inputText).not.toContain("0123456789abcdef");
|
||||
expect(inputText).toContain("0123456789abcdef");
|
||||
|
||||
await harness.completeTurn();
|
||||
const result = await run;
|
||||
expect(result.assistantTexts).toContain("final answer");
|
||||
});
|
||||
|
||||
it("bounds a hung owning context-engine compaction during Codex overflow recovery", async () => {
|
||||
it("fails first-turn Codex context overflow instead of falling back to OpenClaw compaction", async () => {
|
||||
const sessionFile = path.join(tempDir, "session.jsonl");
|
||||
const workspaceDir = path.join(tempDir, "workspace");
|
||||
const compact = vi.fn<ContextEngine["compact"]>(async () => ({
|
||||
ok: true,
|
||||
compacted: true,
|
||||
result: { summary: "summary", firstKeptEntryId: "entry-1", tokensBefore: 100_000 },
|
||||
}));
|
||||
const assemble = vi.fn<ContextEngine["assemble"]>().mockResolvedValue({
|
||||
messages: [assistantMessage("large projected context", 10)],
|
||||
estimatedTokens: 100_000,
|
||||
contextProjection: { mode: "thread_bootstrap", epoch: "epoch-before" },
|
||||
});
|
||||
const contextEngine = createContextEngine({ assemble, compact });
|
||||
const harness = createStartedThreadHarness(async (method) => {
|
||||
if (method === "turn/start") {
|
||||
throw new Error("Codex ran out of room in the model's context window");
|
||||
}
|
||||
return undefined;
|
||||
});
|
||||
const params = createParams(sessionFile, workspaceDir);
|
||||
params.contextEngine = contextEngine;
|
||||
params.contextTokenBudget = 16_000;
|
||||
|
||||
await expect(runCodexAppServerAttempt(params)).rejects.toThrow(
|
||||
"Codex ran out of room in the model's context window",
|
||||
);
|
||||
|
||||
expect(compact).not.toHaveBeenCalled();
|
||||
expect(assemble).toHaveBeenCalledTimes(1);
|
||||
expect(harness.requests.map((request) => request.method)).toEqual([
|
||||
"thread/start",
|
||||
"turn/start",
|
||||
"thread/unsubscribe",
|
||||
]);
|
||||
});
|
||||
|
||||
it("does not call hung owning context-engine compaction during Codex overflow recovery", async () => {
|
||||
const sessionFile = path.join(tempDir, "session.jsonl");
|
||||
const workspaceDir = path.join(tempDir, "workspace");
|
||||
SessionManager.open(sessionFile).appendMessage(
|
||||
@@ -1085,10 +1083,6 @@ describe("runCodexAppServerAttempt context-engine lifecycle", () => {
|
||||
},
|
||||
},
|
||||
});
|
||||
// Owning-engine compaction that never settles. Without the safety timeout
|
||||
// the awaited compact() would hang the whole Codex overflow-recovery turn;
|
||||
// with it the call is bounded and forced compaction reports failure so the
|
||||
// run still proceeds on a fresh thread.
|
||||
const compact = vi.fn<ContextEngine["compact"]>(() => new Promise(() => {}));
|
||||
const assemble = vi.fn(
|
||||
async ({ messages, prompt }: Parameters<ContextEngine["assemble"]>[0]) => ({
|
||||
@@ -1118,11 +1112,6 @@ describe("runCodexAppServerAttempt context-engine lifecycle", () => {
|
||||
const params = createParams(sessionFile, workspaceDir);
|
||||
params.contextEngine = contextEngine;
|
||||
params.contextTokenBudget = 400_000;
|
||||
// 1 s host-resolved compaction timeout so the hung compact() is bounded
|
||||
// well within the 5 s run timeout used by this harness.
|
||||
params.config = {
|
||||
agents: { defaults: { compaction: { timeoutSeconds: 1 } } },
|
||||
} as EmbeddedRunAttemptParams["config"];
|
||||
|
||||
const run = runCodexAppServerAttempt(params);
|
||||
await vi.waitFor(
|
||||
@@ -1150,10 +1139,7 @@ describe("runCodexAppServerAttempt context-engine lifecycle", () => {
|
||||
const result = await run;
|
||||
|
||||
expect(result.assistantTexts).toContain("fresh answer");
|
||||
expect(compact).toHaveBeenCalledTimes(1);
|
||||
// The run-level abort signal is threaded into the owning-engine compact()
|
||||
// so a cooperating engine can cancel its own in-flight work.
|
||||
expect(compact.mock.calls[0]?.[0]?.abortSignal).toBeInstanceOf(AbortSignal);
|
||||
expect(compact).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("keeps current inbound context at the front of the Codex context-engine prompt", async () => {
|
||||
|
||||
@@ -12,12 +12,9 @@ import {
|
||||
buildEmbeddedAttemptToolRunContext,
|
||||
CODEX_APP_SERVER_CONTEXT_ENGINE_HOST,
|
||||
clearActiveEmbeddedRun,
|
||||
compactContextEngineWithSafetyTimeout,
|
||||
embeddedAgentLog,
|
||||
emitAgentEvent as emitGlobalAgentEvent,
|
||||
estimateRenderedLlmBoundaryTokenPressure,
|
||||
finalizeHarnessContextEngineTurn,
|
||||
formatPrePromptPrecheckLog,
|
||||
formatErrorMessage,
|
||||
getBeforeToolCallPolicyDiagnosticState,
|
||||
isActiveHarnessContextEngine,
|
||||
@@ -26,7 +23,6 @@ import {
|
||||
normalizeAgentRuntimeTools,
|
||||
resolveAttemptSpawnWorkspaceDir,
|
||||
resolveAgentHarnessBeforePromptBuildResult,
|
||||
resolveCompactionTimeoutMs,
|
||||
resolveModelAuthMode,
|
||||
resolveContextEngineOwnerPluginId,
|
||||
resolveSandboxContext,
|
||||
@@ -37,8 +33,6 @@ import {
|
||||
runAgentHarnessLlmInputHook,
|
||||
runAgentHarnessLlmOutputHook,
|
||||
runHarnessContextEngineMaintenance,
|
||||
shouldPreemptivelyCompactBeforePrompt,
|
||||
PREEMPTIVE_OVERFLOW_ERROR_TEXT,
|
||||
registerNativeHookRelay,
|
||||
resolveBootstrapContextForRun,
|
||||
setActiveEmbeddedRun,
|
||||
@@ -94,7 +88,6 @@ import {
|
||||
type CodexPluginConfig,
|
||||
} from "./config.js";
|
||||
import {
|
||||
DEFAULT_CODEX_PROJECTION_RESERVE_TOKENS,
|
||||
projectContextEngineAssemblyForCodex,
|
||||
resolveCodexContextEngineProjectionMaxChars,
|
||||
resolveCodexContextEngineProjectionReserveTokens,
|
||||
@@ -1042,16 +1035,6 @@ export async function runCodexAppServerAttempt(
|
||||
sessionId: activeSessionId,
|
||||
sessionFile: activeSessionFile,
|
||||
});
|
||||
const adoptContextEngineCompactionTranscript = (compactResult: {
|
||||
result?: { sessionId?: string; sessionFile?: string };
|
||||
}): void => {
|
||||
if (compactResult.result?.sessionId) {
|
||||
activeSessionId = compactResult.result.sessionId;
|
||||
}
|
||||
if (compactResult.result?.sessionFile) {
|
||||
activeSessionFile = compactResult.result.sessionFile;
|
||||
}
|
||||
};
|
||||
const startupAuthAccountCacheKey = await resolveCodexAppServerAuthAccountCacheKey({
|
||||
authProfileId: startupAuthProfileId,
|
||||
authProfileStore: params.authProfileStore,
|
||||
@@ -1179,86 +1162,6 @@ export async function runCodexAppServerAttempt(
|
||||
contextEnginePluginId: activeContextEnginePluginId,
|
||||
tokenBudget: params.contextTokenBudget,
|
||||
});
|
||||
const forceContextEngineCompactionForCodexOverflow = async (
|
||||
error: unknown,
|
||||
options: { threadId?: string } = {},
|
||||
): Promise<boolean> => {
|
||||
if (!activeContextEngine?.info.ownsCompaction) {
|
||||
return false;
|
||||
}
|
||||
embeddedAgentLog.warn(
|
||||
"codex app-server context-engine prompt overflowed; forcing context-engine compaction",
|
||||
{
|
||||
sessionId: activeSessionId,
|
||||
sessionKey: contextSessionKey,
|
||||
...(options.threadId ? { threadId: options.threadId } : {}),
|
||||
engineId: activeContextEngine.info.id,
|
||||
tokenBudget: params.contextTokenBudget,
|
||||
error: formatErrorMessage(error),
|
||||
},
|
||||
);
|
||||
try {
|
||||
const runtimeContext = buildActiveContextEngineRuntimeContext();
|
||||
const overflowTokenCount = params.contextTokenBudget ?? params.contextWindowInfo?.tokens;
|
||||
// Bound the plugin-owned compaction with the same finite safety timeout
|
||||
// that protects native runtime compaction, and thread the run-level
|
||||
// abort signal through, so a slow/hung plugin compact() cannot stall
|
||||
// Codex overflow recovery indefinitely.
|
||||
const compactResult = await compactContextEngineWithSafetyTimeout(
|
||||
activeContextEngine,
|
||||
{
|
||||
sessionId: activeSessionId,
|
||||
sessionKey: contextSessionKey,
|
||||
sessionFile: activeSessionFile,
|
||||
tokenBudget: params.contextTokenBudget,
|
||||
force: true,
|
||||
...(overflowTokenCount ? { currentTokenCount: overflowTokenCount } : {}),
|
||||
compactionTarget: "threshold",
|
||||
runtimeContext: overflowTokenCount
|
||||
? {
|
||||
...runtimeContext,
|
||||
currentTokenCount: overflowTokenCount,
|
||||
}
|
||||
: runtimeContext,
|
||||
},
|
||||
resolveCompactionTimeoutMs(params.config),
|
||||
runAbortController.signal,
|
||||
);
|
||||
embeddedAgentLog.info("codex app-server context-engine forced compaction result", {
|
||||
sessionId: activeSessionId,
|
||||
sessionKey: contextSessionKey,
|
||||
engineId: activeContextEngine.info.id,
|
||||
ok: compactResult.ok,
|
||||
compacted: compactResult.compacted,
|
||||
reason: compactResult.reason,
|
||||
tokensBefore: compactResult.result?.tokensBefore,
|
||||
tokensAfter: compactResult.result?.tokensAfter,
|
||||
});
|
||||
if (!compactResult.ok || !compactResult.compacted) {
|
||||
return false;
|
||||
}
|
||||
adoptContextEngineCompactionTranscript(compactResult);
|
||||
const maintenanceRuntimeContext = buildActiveContextEngineRuntimeContext();
|
||||
await runHarnessContextEngineMaintenance({
|
||||
contextEngine: activeContextEngine,
|
||||
sessionId: activeSessionId,
|
||||
sessionKey: contextSessionKey,
|
||||
sessionFile: activeSessionFile,
|
||||
reason: "compaction",
|
||||
runtimeContext: maintenanceRuntimeContext,
|
||||
config: params.config,
|
||||
});
|
||||
return true;
|
||||
} catch (compactErr) {
|
||||
embeddedAgentLog.warn("codex app-server context-engine forced compaction failed", {
|
||||
sessionId: params.sessionId,
|
||||
sessionKey: contextSessionKey,
|
||||
engineId: activeContextEngine.info.id,
|
||||
error: formatErrorMessage(compactErr),
|
||||
});
|
||||
return false;
|
||||
}
|
||||
};
|
||||
if (activeContextEngine) {
|
||||
await bootstrapHarnessContextEngine({
|
||||
hadSessionFile,
|
||||
@@ -1296,12 +1199,6 @@ export async function runCodexAppServerAttempt(
|
||||
let developerInstructions = baseDeveloperInstructions;
|
||||
let prePromptMessageCount = historyMessages.length;
|
||||
let contextEngineProjection: CodexContextEngineThreadBootstrapProjection | undefined;
|
||||
const resetCodexPromptInputs = () => {
|
||||
promptText = params.prompt;
|
||||
developerInstructions = baseDeveloperInstructions;
|
||||
prePromptMessageCount = historyMessages.length;
|
||||
contextEngineProjection = undefined;
|
||||
};
|
||||
const applyActiveContextEngineProjection = async (
|
||||
decisionStartupBinding: CodexAppServerThreadBinding | undefined,
|
||||
) => {
|
||||
@@ -1412,9 +1309,6 @@ export async function runCodexAppServerAttempt(
|
||||
const decorateCodexTurnPromptText = (prompt: string) =>
|
||||
prependCodexOpenClawPromptContext(prompt, openClawPromptContext);
|
||||
let codexTurnPromptText = decorateCodexTurnPromptText(promptBuild.prompt);
|
||||
const refreshCodexTurnPromptText = () => {
|
||||
codexTurnPromptText = decorateCodexTurnPromptText(promptBuild.prompt);
|
||||
};
|
||||
const buildCodexTurnCollaborationDeveloperInstructions = () =>
|
||||
buildTurnCollaborationMode(params, {
|
||||
turnScopedDeveloperInstructions: workspaceBootstrapContext.turnScopedDeveloperInstructions,
|
||||
@@ -1426,109 +1320,6 @@ export async function runCodexAppServerAttempt(
|
||||
promptBuild.developerInstructions,
|
||||
buildCodexTurnCollaborationDeveloperInstructions(),
|
||||
);
|
||||
const rebuildPromptAfterContextEngineCompaction = async () => {
|
||||
historyMessages =
|
||||
(await readMirroredSessionHistoryMessages(activeSessionFile)) ?? historyMessages;
|
||||
resetCodexPromptInputs();
|
||||
try {
|
||||
await applyActiveContextEngineProjection(undefined);
|
||||
} catch (assembleErr) {
|
||||
embeddedAgentLog.warn(
|
||||
"context engine assemble failed after forced compaction; using Codex baseline prompt",
|
||||
{
|
||||
error: formatErrorMessage(assembleErr),
|
||||
},
|
||||
);
|
||||
}
|
||||
promptBuild = await buildPromptFromCurrentInputs();
|
||||
refreshCodexTurnPromptText();
|
||||
};
|
||||
const buildCodexProviderBoundaryPrecheck = () => {
|
||||
const contextTokenBudget =
|
||||
typeof params.contextTokenBudget === "number" && Number.isFinite(params.contextTokenBudget)
|
||||
? Math.floor(params.contextTokenBudget)
|
||||
: typeof params.contextWindowInfo?.tokens === "number" &&
|
||||
Number.isFinite(params.contextWindowInfo.tokens)
|
||||
? Math.floor(params.contextWindowInfo.tokens)
|
||||
: undefined;
|
||||
if (!contextTokenBudget || contextTokenBudget <= 0) {
|
||||
return undefined;
|
||||
}
|
||||
const reserveTokens =
|
||||
resolveCodexContextEngineProjectionReserveTokens({ config: params.config }) ??
|
||||
DEFAULT_CODEX_PROJECTION_RESERVE_TOKENS;
|
||||
const renderedDeveloperInstructions = buildRenderedCodexDeveloperInstructions();
|
||||
const renderedChars = codexTurnPromptText.length + renderedDeveloperInstructions.length;
|
||||
return shouldPreemptivelyCompactBeforePrompt({
|
||||
messages: historyMessages,
|
||||
systemPrompt: renderedDeveloperInstructions,
|
||||
prompt: codexTurnPromptText,
|
||||
contextTokenBudget,
|
||||
reserveTokens,
|
||||
llmBoundaryTokenPressure: {
|
||||
estimatedPromptTokens: estimateRenderedLlmBoundaryTokenPressure({
|
||||
systemPrompt: renderedDeveloperInstructions,
|
||||
prompt: codexTurnPromptText,
|
||||
}),
|
||||
source: "codex_app_server_rendered_prompt",
|
||||
renderedChars,
|
||||
},
|
||||
});
|
||||
};
|
||||
const maybeCompactContextEngineForProviderBoundaryPrecheck = async () => {
|
||||
if (!activeContextEngine?.info.ownsCompaction || !contextEngineProjection) {
|
||||
return;
|
||||
}
|
||||
const precheck = buildCodexProviderBoundaryPrecheck();
|
||||
if (!precheck) {
|
||||
return;
|
||||
}
|
||||
embeddedAgentLog.debug(
|
||||
formatPrePromptPrecheckLog({
|
||||
result: precheck,
|
||||
provider: params.provider,
|
||||
modelId: params.modelId,
|
||||
messageCount: historyMessages.length,
|
||||
contextTokenBudget:
|
||||
typeof params.contextTokenBudget === "number"
|
||||
? params.contextTokenBudget
|
||||
: (params.contextWindowInfo?.tokens ?? 0),
|
||||
reserveTokens:
|
||||
resolveCodexContextEngineProjectionReserveTokens({ config: params.config }) ??
|
||||
DEFAULT_CODEX_PROJECTION_RESERVE_TOKENS,
|
||||
...(contextSessionKey ? { sessionKey: contextSessionKey } : {}),
|
||||
...(activeSessionId ? { sessionId: activeSessionId } : {}),
|
||||
...(activeSessionFile ? { sessionFile: activeSessionFile } : {}),
|
||||
}),
|
||||
);
|
||||
if (precheck.route === "fits") {
|
||||
return;
|
||||
}
|
||||
const compacted = await forceContextEngineCompactionForCodexOverflow(
|
||||
PREEMPTIVE_OVERFLOW_ERROR_TEXT,
|
||||
);
|
||||
if (!compacted) {
|
||||
throw new Error(PREEMPTIVE_OVERFLOW_ERROR_TEXT);
|
||||
}
|
||||
await rebuildPromptAfterContextEngineCompaction();
|
||||
const afterCompactionPrecheck = buildCodexProviderBoundaryPrecheck();
|
||||
if (!afterCompactionPrecheck || afterCompactionPrecheck.route === "fits") {
|
||||
return;
|
||||
}
|
||||
embeddedAgentLog.warn(
|
||||
"codex app-server provider-boundary precheck still overflowed after compaction",
|
||||
{
|
||||
sessionId: activeSessionId,
|
||||
sessionKey: contextSessionKey,
|
||||
route: afterCompactionPrecheck.route,
|
||||
estimatedPromptTokens: afterCompactionPrecheck.estimatedPromptTokens,
|
||||
promptBudgetBeforeReserve: afterCompactionPrecheck.promptBudgetBeforeReserve,
|
||||
overflowTokens: afterCompactionPrecheck.overflowTokens,
|
||||
},
|
||||
);
|
||||
throw new Error(PREEMPTIVE_OVERFLOW_ERROR_TEXT);
|
||||
};
|
||||
await maybeCompactContextEngineForProviderBoundaryPrecheck();
|
||||
const systemPromptReport = buildCodexSystemPromptReport({
|
||||
attempt: params,
|
||||
sessionKey: contextSessionKey,
|
||||
@@ -2929,6 +2720,9 @@ export async function runCodexAppServerAttempt(
|
||||
}) &&
|
||||
restartContextEngineCodexThread
|
||||
) {
|
||||
// Do not try to pre-compact or summarize through OpenClaw here. Codex owns
|
||||
// automatic compaction; OpenClaw may only discard a stale projection thread
|
||||
// and let Codex start cleanly.
|
||||
embeddedAgentLog.warn(
|
||||
"codex app-server context-engine turn overflowed on resume; retrying with fresh thread",
|
||||
{
|
||||
@@ -2938,19 +2732,10 @@ export async function runCodexAppServerAttempt(
|
||||
);
|
||||
try {
|
||||
const preRetrySessionFile = activeSessionFile;
|
||||
const compactedForRetry = await forceContextEngineCompactionForCodexOverflow(
|
||||
turnStartError,
|
||||
{
|
||||
threadId: thread.threadId,
|
||||
},
|
||||
);
|
||||
await clearCodexAppServerBinding(preRetrySessionFile);
|
||||
if (activeSessionFile !== preRetrySessionFile) {
|
||||
await clearCodexAppServerBinding(activeSessionFile);
|
||||
}
|
||||
if (compactedForRetry) {
|
||||
await rebuildPromptAfterContextEngineCompaction();
|
||||
}
|
||||
thread = await restartContextEngineCodexThread();
|
||||
emitCodexAppServerEvent(params, {
|
||||
stream: "codex_app_server.lifecycle",
|
||||
|
||||
@@ -177,7 +177,7 @@ describe("runCliTurnCompactionLifecycle", () => {
|
||||
expect(updatedEntry?.claudeCliSessionId).toBeUndefined();
|
||||
});
|
||||
|
||||
it("routes OpenAI Codex harness CLI compaction through native harness compaction", async () => {
|
||||
it("skips OpenClaw automatic CLI compaction for OpenAI Codex runtime sessions", async () => {
|
||||
const sessionKey = "agent:main:codex";
|
||||
const sessionId = "session-codex";
|
||||
const sessionFile = path.join(tmpDir, "session-codex.jsonl");
|
||||
@@ -197,8 +197,7 @@ describe("runCliTurnCompactionLifecycle", () => {
|
||||
await fs.writeFile(storePath, JSON.stringify(sessionStore, null, 2), "utf-8");
|
||||
|
||||
const compactCalls: Array<Parameters<ContextEngine["compact"]>[0]> = [];
|
||||
const contextEngine = buildContextEngine({ compactCalls });
|
||||
const resolveContextEngine = vi.fn(async () => contextEngine);
|
||||
const resolveContextEngine = vi.fn(async () => buildContextEngine({ compactCalls }));
|
||||
const ensureSelectedAgentHarnessPlugin = vi.fn(async () => undefined);
|
||||
const compactAgentHarnessSession = vi.fn(async () => ({
|
||||
ok: true,
|
||||
@@ -250,49 +249,65 @@ describe("runCliTurnCompactionLifecycle", () => {
|
||||
model: "gpt-5.5",
|
||||
});
|
||||
|
||||
expect(resolveContextEngine).toHaveBeenCalledTimes(1);
|
||||
expect(applyPiAutoCompactionGuard).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
contextEngineInfo: contextEngine.info,
|
||||
}),
|
||||
);
|
||||
expect(ensureSelectedAgentHarnessPlugin).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
provider: "openai",
|
||||
modelId: "gpt-5.5",
|
||||
sessionKey,
|
||||
agentHarnessRuntimeOverride: "codex",
|
||||
}),
|
||||
);
|
||||
expect(applyPiAutoCompactionGuard.mock.invocationCallOrder[0] ?? 0).toBeLessThan(
|
||||
compactAgentHarnessSession.mock.invocationCallOrder[0] ?? 0,
|
||||
);
|
||||
expect(compactAgentHarnessSession).toHaveBeenCalledTimes(1);
|
||||
const compactAgentHarnessSessionCalls = compactAgentHarnessSession.mock
|
||||
.calls as unknown as Array<[Record<string, unknown>]>;
|
||||
expect(compactAgentHarnessSessionCalls[0]?.[0]).toMatchObject({
|
||||
expect(resolveContextEngine).not.toHaveBeenCalled();
|
||||
expect(applyPiAutoCompactionGuard).not.toHaveBeenCalled();
|
||||
expect(ensureSelectedAgentHarnessPlugin).not.toHaveBeenCalled();
|
||||
expect(compactAgentHarnessSession).not.toHaveBeenCalled();
|
||||
expect(compactCalls).toHaveLength(0);
|
||||
expect(recordCliCompactionInStore).not.toHaveBeenCalled();
|
||||
expect(updatedEntry).toBe(sessionEntry);
|
||||
});
|
||||
|
||||
it("skips OpenClaw automatic CLI compaction when OpenAI resolves to Codex by policy", async () => {
|
||||
const sessionKey = "agent:main:codex-policy";
|
||||
const sessionId = "session-codex-policy";
|
||||
const sessionFile = path.join(tmpDir, "session-codex-policy.jsonl");
|
||||
const storePath = path.join(tmpDir, "sessions-codex-policy.json");
|
||||
await writeSessionFile({ sessionFile, sessionId });
|
||||
|
||||
const sessionEntry: SessionEntry = {
|
||||
sessionId,
|
||||
updatedAt: Date.now(),
|
||||
sessionFile,
|
||||
contextTokens: 1_000,
|
||||
totalTokens: 950,
|
||||
totalTokensFresh: true,
|
||||
};
|
||||
const sessionStore: Record<string, SessionEntry> = { [sessionKey]: sessionEntry };
|
||||
await fs.writeFile(storePath, JSON.stringify(sessionStore, null, 2), "utf-8");
|
||||
|
||||
const openSessionManager = vi.fn(() => {
|
||||
throw new Error("OpenClaw must not inspect Codex transcripts for automatic compaction");
|
||||
});
|
||||
const resolveContextEngine = vi.fn();
|
||||
const ensureSelectedAgentHarnessPlugin = vi.fn();
|
||||
const compactAgentHarnessSession = vi.fn();
|
||||
setCliCompactionTestDeps({
|
||||
openSessionManager: openSessionManager as never,
|
||||
resolveContextEngine: resolveContextEngine as never,
|
||||
ensureSelectedAgentHarnessPlugin: ensureSelectedAgentHarnessPlugin as never,
|
||||
maybeCompactAgentHarnessSession: compactAgentHarnessSession as never,
|
||||
});
|
||||
|
||||
const updatedEntry = await runCliTurnCompactionLifecycle({
|
||||
cfg: {} as OpenClawConfig,
|
||||
sessionId,
|
||||
sessionKey,
|
||||
sessionFile,
|
||||
sessionEntry,
|
||||
sessionStore,
|
||||
storePath,
|
||||
sessionAgentId: "main",
|
||||
workspaceDir: tmpDir,
|
||||
agentDir: tmpDir,
|
||||
provider: "openai",
|
||||
model: "gpt-5.5",
|
||||
contextTokenBudget: 1_000,
|
||||
currentTokenCount: 950,
|
||||
contextEngine,
|
||||
agentHarnessId: "codex",
|
||||
trigger: "budget",
|
||||
force: true,
|
||||
});
|
||||
expect(compactCalls).toHaveLength(0);
|
||||
expect(recordCliCompactionInStore).toHaveBeenCalledTimes(1);
|
||||
expect(recordCliCompactionInStore).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
provider: "openai",
|
||||
sessionKey,
|
||||
tokensAfter: 100,
|
||||
}),
|
||||
);
|
||||
expect(updatedEntry?.compactionCount).toBe(1);
|
||||
|
||||
expect(openSessionManager).not.toHaveBeenCalled();
|
||||
expect(resolveContextEngine).not.toHaveBeenCalled();
|
||||
expect(ensureSelectedAgentHarnessPlugin).not.toHaveBeenCalled();
|
||||
expect(compactAgentHarnessSession).not.toHaveBeenCalled();
|
||||
expect(updatedEntry).toBe(sessionEntry);
|
||||
});
|
||||
|
||||
it("ignores stale native harness ids when the active provider no longer matches", async () => {
|
||||
@@ -359,240 +374,6 @@ describe("runCliTurnCompactionLifecycle", () => {
|
||||
expect(compactCalls).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("surfaces nonrecoverable native harness CLI compaction failures", async () => {
|
||||
const sessionKey = "agent:main:codex-native-failure";
|
||||
const sessionId = "session-codex-native-failure";
|
||||
const sessionFile = path.join(tmpDir, "session-codex-native-failure.jsonl");
|
||||
const storePath = path.join(tmpDir, "sessions-codex-native-failure.json");
|
||||
await writeSessionFile({ sessionFile, sessionId });
|
||||
|
||||
const sessionEntry: SessionEntry = {
|
||||
sessionId,
|
||||
updatedAt: Date.now(),
|
||||
sessionFile,
|
||||
contextTokens: 1_000,
|
||||
totalTokens: 950,
|
||||
totalTokensFresh: true,
|
||||
agentHarnessId: "codex",
|
||||
};
|
||||
const sessionStore: Record<string, SessionEntry> = { [sessionKey]: sessionEntry };
|
||||
await fs.writeFile(storePath, JSON.stringify(sessionStore, null, 2), "utf-8");
|
||||
|
||||
const compactCalls: Array<Parameters<ContextEngine["compact"]>[0]> = [];
|
||||
const ensureSelectedAgentHarnessPlugin = vi.fn(async () => undefined);
|
||||
const compactAgentHarnessSession = vi.fn(async () => ({
|
||||
ok: false,
|
||||
compacted: false,
|
||||
reason: "timed out waiting for codex app-server compaction",
|
||||
}));
|
||||
const recordCliCompactionInStore = vi.fn();
|
||||
setCliCompactionTestDeps({
|
||||
resolveContextEngine: async () => buildContextEngine({ compactCalls }),
|
||||
ensureSelectedAgentHarnessPlugin,
|
||||
maybeCompactAgentHarnessSession: compactAgentHarnessSession as never,
|
||||
createPreparedEmbeddedPiSettingsManager: async () => ({
|
||||
getCompactionReserveTokens: () => 200,
|
||||
getCompactionKeepRecentTokens: () => 0,
|
||||
applyOverrides: () => {},
|
||||
}),
|
||||
shouldPreemptivelyCompactBeforePrompt: () => ({
|
||||
route: "fits",
|
||||
shouldCompact: false,
|
||||
estimatedPromptTokens: 600,
|
||||
promptBudgetBeforeReserve: 800,
|
||||
overflowTokens: 0,
|
||||
toolResultReducibleChars: 0,
|
||||
effectiveReserveTokens: 200,
|
||||
}),
|
||||
resolveLiveToolResultMaxChars: () => 20_000,
|
||||
recordCliCompactionInStore,
|
||||
});
|
||||
|
||||
await expect(
|
||||
runCliTurnCompactionLifecycle({
|
||||
cfg: {} as OpenClawConfig,
|
||||
sessionId,
|
||||
sessionKey,
|
||||
sessionEntry,
|
||||
sessionStore,
|
||||
storePath,
|
||||
sessionAgentId: "main",
|
||||
workspaceDir: tmpDir,
|
||||
agentDir: tmpDir,
|
||||
provider: "codex",
|
||||
model: "gpt-5.5",
|
||||
}),
|
||||
).rejects.toThrow(
|
||||
"CLI native harness compaction failed for codex/gpt-5.5: timed out waiting for codex app-server compaction",
|
||||
);
|
||||
|
||||
expect(compactAgentHarnessSession).toHaveBeenCalledTimes(1);
|
||||
expect(compactCalls).toHaveLength(0);
|
||||
expect(recordCliCompactionInStore).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("does not fall back when native harness compaction returns no result", async () => {
|
||||
const sessionKey = "agent:main:codex-native-empty";
|
||||
const sessionId = "session-codex-native-empty";
|
||||
const sessionFile = path.join(tmpDir, "session-codex-native-empty.jsonl");
|
||||
const storePath = path.join(tmpDir, "sessions-codex-native-empty.json");
|
||||
await writeSessionFile({ sessionFile, sessionId });
|
||||
|
||||
const sessionEntry: SessionEntry = {
|
||||
sessionId,
|
||||
updatedAt: Date.now(),
|
||||
sessionFile,
|
||||
contextTokens: 1_000,
|
||||
totalTokens: 950,
|
||||
totalTokensFresh: true,
|
||||
agentHarnessId: "codex",
|
||||
};
|
||||
const sessionStore: Record<string, SessionEntry> = { [sessionKey]: sessionEntry };
|
||||
await fs.writeFile(storePath, JSON.stringify(sessionStore, null, 2), "utf-8");
|
||||
|
||||
const compactCalls: Array<Parameters<ContextEngine["compact"]>[0]> = [];
|
||||
setCliCompactionTestDeps({
|
||||
resolveContextEngine: async () => buildContextEngine({ compactCalls }),
|
||||
ensureSelectedAgentHarnessPlugin: vi.fn(async () => undefined),
|
||||
maybeCompactAgentHarnessSession: vi.fn(async () => undefined) as never,
|
||||
createPreparedEmbeddedPiSettingsManager: async () => ({
|
||||
getCompactionReserveTokens: () => 200,
|
||||
getCompactionKeepRecentTokens: () => 0,
|
||||
applyOverrides: () => {},
|
||||
}),
|
||||
shouldPreemptivelyCompactBeforePrompt: () => ({
|
||||
route: "fits",
|
||||
shouldCompact: false,
|
||||
estimatedPromptTokens: 600,
|
||||
promptBudgetBeforeReserve: 800,
|
||||
overflowTokens: 0,
|
||||
toolResultReducibleChars: 0,
|
||||
effectiveReserveTokens: 200,
|
||||
}),
|
||||
resolveLiveToolResultMaxChars: () => 20_000,
|
||||
});
|
||||
|
||||
await expect(
|
||||
runCliTurnCompactionLifecycle({
|
||||
cfg: {} as OpenClawConfig,
|
||||
sessionId,
|
||||
sessionKey,
|
||||
sessionEntry,
|
||||
sessionStore,
|
||||
storePath,
|
||||
sessionAgentId: "main",
|
||||
workspaceDir: tmpDir,
|
||||
agentDir: tmpDir,
|
||||
provider: "codex",
|
||||
model: "gpt-5.5",
|
||||
}),
|
||||
).rejects.toThrow(
|
||||
"CLI native harness compaction failed for codex/gpt-5.5: native harness compaction did not reduce context",
|
||||
);
|
||||
expect(compactCalls).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("passes owning context engines into native harness CLI compaction", async () => {
|
||||
const sessionKey = "agent:main:codex-owned-engine";
|
||||
const sessionId = "session-codex-owned-engine";
|
||||
const sessionFile = path.join(tmpDir, "session-codex-owned-engine.jsonl");
|
||||
const storePath = path.join(tmpDir, "sessions-codex-owned-engine.json");
|
||||
await writeSessionFile({ sessionFile, sessionId });
|
||||
|
||||
const sessionEntry: SessionEntry = {
|
||||
sessionId,
|
||||
updatedAt: Date.now(),
|
||||
sessionFile,
|
||||
contextTokens: 1_000,
|
||||
totalTokens: 950,
|
||||
totalTokensFresh: true,
|
||||
agentHarnessId: "codex",
|
||||
};
|
||||
const sessionStore: Record<string, SessionEntry> = { [sessionKey]: sessionEntry };
|
||||
await fs.writeFile(storePath, JSON.stringify(sessionStore, null, 2), "utf-8");
|
||||
|
||||
const compactCalls: Array<Parameters<ContextEngine["compact"]>[0]> = [];
|
||||
const contextEngine = {
|
||||
...buildContextEngine({ compactCalls }),
|
||||
info: {
|
||||
id: "lossless-claw",
|
||||
name: "Lossless Claw",
|
||||
ownsCompaction: true,
|
||||
},
|
||||
} satisfies ContextEngine;
|
||||
const ensureSelectedAgentHarnessPlugin = vi.fn(async () => undefined);
|
||||
const compactAgentHarnessSession = vi.fn(async (compactParams) => {
|
||||
expect(compactParams.contextEngine).toBe(contextEngine);
|
||||
expect(compactParams.contextEngineRuntimeContext).toMatchObject({
|
||||
currentTokenCount: 950,
|
||||
tokenBudget: 1_000,
|
||||
trigger: "cli_native_budget",
|
||||
});
|
||||
return {
|
||||
ok: true,
|
||||
compacted: true,
|
||||
result: {
|
||||
summary: "engine-owned",
|
||||
firstKeptEntryId: "entry-1",
|
||||
tokensBefore: 950,
|
||||
tokensAfter: 42,
|
||||
sessionId: "session-codex-owned-engine-rotated",
|
||||
sessionFile: path.join(tmpDir, "session-codex-owned-engine-rotated.jsonl"),
|
||||
},
|
||||
};
|
||||
});
|
||||
const recordCliCompactionInStore = vi.fn(async () => ({
|
||||
...sessionEntry,
|
||||
compactionCount: 1,
|
||||
}));
|
||||
setCliCompactionTestDeps({
|
||||
resolveContextEngine: async () => contextEngine,
|
||||
ensureSelectedAgentHarnessPlugin,
|
||||
maybeCompactAgentHarnessSession: compactAgentHarnessSession as never,
|
||||
createPreparedEmbeddedPiSettingsManager: async () => ({
|
||||
getCompactionReserveTokens: () => 200,
|
||||
getCompactionKeepRecentTokens: () => 0,
|
||||
applyOverrides: () => {},
|
||||
}),
|
||||
shouldPreemptivelyCompactBeforePrompt: () => ({
|
||||
route: "fits",
|
||||
shouldCompact: false,
|
||||
estimatedPromptTokens: 600,
|
||||
promptBudgetBeforeReserve: 800,
|
||||
overflowTokens: 0,
|
||||
toolResultReducibleChars: 0,
|
||||
effectiveReserveTokens: 200,
|
||||
}),
|
||||
resolveLiveToolResultMaxChars: () => 20_000,
|
||||
recordCliCompactionInStore,
|
||||
});
|
||||
|
||||
await runCliTurnCompactionLifecycle({
|
||||
cfg: {} as OpenClawConfig,
|
||||
sessionId,
|
||||
sessionKey,
|
||||
sessionEntry,
|
||||
sessionStore,
|
||||
storePath,
|
||||
sessionAgentId: "main",
|
||||
workspaceDir: tmpDir,
|
||||
agentDir: tmpDir,
|
||||
provider: "codex",
|
||||
model: "gpt-5.5",
|
||||
});
|
||||
|
||||
expect(compactAgentHarnessSession).toHaveBeenCalledTimes(1);
|
||||
expect(recordCliCompactionInStore).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
provider: "codex",
|
||||
sessionKey,
|
||||
tokensAfter: 42,
|
||||
newSessionId: "session-codex-owned-engine-rotated",
|
||||
newSessionFile: path.join(tmpDir, "session-codex-owned-engine-rotated.jsonl"),
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it("falls back to context-engine compaction when a pinned harness has no native compactor", async () => {
|
||||
const sessionKey = "agent:main:external-harness";
|
||||
const sessionId = "session-external-harness";
|
||||
@@ -675,95 +456,11 @@ describe("runCliTurnCompactionLifecycle", () => {
|
||||
expect(updatedEntry?.compactionCount).toBe(1);
|
||||
});
|
||||
|
||||
it("falls back to context-engine compaction when Codex native binding is stale", async () => {
|
||||
const sessionKey = "agent:main:codex-stale-binding";
|
||||
const sessionId = "session-codex-stale-binding";
|
||||
const sessionFile = path.join(tmpDir, "session-codex-stale-binding.jsonl");
|
||||
const storePath = path.join(tmpDir, "sessions-codex-stale-binding.json");
|
||||
await writeSessionFile({ sessionFile, sessionId });
|
||||
|
||||
const sessionEntry: SessionEntry = {
|
||||
sessionId,
|
||||
updatedAt: Date.now(),
|
||||
sessionFile,
|
||||
contextTokens: 1_000,
|
||||
totalTokens: 950,
|
||||
totalTokensFresh: true,
|
||||
agentHarnessId: "codex",
|
||||
};
|
||||
const sessionStore: Record<string, SessionEntry> = { [sessionKey]: sessionEntry };
|
||||
await fs.writeFile(storePath, JSON.stringify(sessionStore, null, 2), "utf-8");
|
||||
|
||||
const compactCalls: Array<Parameters<ContextEngine["compact"]>[0]> = [];
|
||||
const ensureSelectedAgentHarnessPlugin = vi.fn(async () => undefined);
|
||||
const compactAgentHarnessSession = vi.fn(async () => ({
|
||||
ok: false,
|
||||
compacted: false,
|
||||
reason: "thread not found: thread-1",
|
||||
failure: {
|
||||
reason: "stale_thread_binding",
|
||||
},
|
||||
}));
|
||||
const maintenance = vi.fn(async () => ({ changed: false, bytesFreed: 0, rewrittenEntries: 0 }));
|
||||
const recordCliCompactionInStore = vi.fn(async () => ({
|
||||
...sessionEntry,
|
||||
compactionCount: 1,
|
||||
}));
|
||||
setCliCompactionTestDeps({
|
||||
resolveContextEngine: async () => buildContextEngine({ compactCalls }),
|
||||
ensureSelectedAgentHarnessPlugin,
|
||||
maybeCompactAgentHarnessSession: compactAgentHarnessSession as never,
|
||||
createPreparedEmbeddedPiSettingsManager: async () => ({
|
||||
getCompactionReserveTokens: () => 200,
|
||||
getCompactionKeepRecentTokens: () => 0,
|
||||
applyOverrides: () => {},
|
||||
}),
|
||||
shouldPreemptivelyCompactBeforePrompt: () => ({
|
||||
route: "fits",
|
||||
shouldCompact: false,
|
||||
estimatedPromptTokens: 600,
|
||||
promptBudgetBeforeReserve: 800,
|
||||
overflowTokens: 0,
|
||||
toolResultReducibleChars: 0,
|
||||
effectiveReserveTokens: 200,
|
||||
}),
|
||||
resolveLiveToolResultMaxChars: () => 20_000,
|
||||
runContextEngineMaintenance: maintenance,
|
||||
recordCliCompactionInStore,
|
||||
});
|
||||
|
||||
const updatedEntry = await runCliTurnCompactionLifecycle({
|
||||
cfg: {} as OpenClawConfig,
|
||||
sessionId,
|
||||
sessionKey,
|
||||
sessionEntry,
|
||||
sessionStore,
|
||||
storePath,
|
||||
sessionAgentId: "main",
|
||||
workspaceDir: tmpDir,
|
||||
agentDir: tmpDir,
|
||||
provider: "codex",
|
||||
model: "gpt-5.5",
|
||||
});
|
||||
|
||||
expect(compactAgentHarnessSession).toHaveBeenCalledTimes(1);
|
||||
expect(compactCalls).toHaveLength(1);
|
||||
expect(maintenance).toHaveBeenCalledTimes(1);
|
||||
expect(recordCliCompactionInStore).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
provider: "codex",
|
||||
sessionKey,
|
||||
tokensAfter: undefined,
|
||||
}),
|
||||
);
|
||||
expect(updatedEntry?.compactionCount).toBe(1);
|
||||
});
|
||||
|
||||
it("keeps successful context-engine fallback when post-compaction maintenance fails", async () => {
|
||||
const sessionKey = "agent:main:codex-stale-maintenance";
|
||||
const sessionId = "session-codex-stale-maintenance";
|
||||
const sessionFile = path.join(tmpDir, "session-codex-stale-maintenance.jsonl");
|
||||
const storePath = path.join(tmpDir, "sessions-codex-stale-maintenance.json");
|
||||
const sessionKey = "agent:main:external-harness-stale-maintenance";
|
||||
const sessionId = "session-external-harness-stale-maintenance";
|
||||
const sessionFile = path.join(tmpDir, "session-external-harness-stale-maintenance.jsonl");
|
||||
const storePath = path.join(tmpDir, "sessions-external-harness-stale-maintenance.json");
|
||||
await writeSessionFile({ sessionFile, sessionId });
|
||||
|
||||
const sessionEntry: SessionEntry = {
|
||||
@@ -773,7 +470,7 @@ describe("runCliTurnCompactionLifecycle", () => {
|
||||
contextTokens: 1_000,
|
||||
totalTokens: 950,
|
||||
totalTokensFresh: true,
|
||||
agentHarnessId: "codex",
|
||||
agentHarnessId: "external-harness",
|
||||
};
|
||||
const sessionStore: Record<string, SessionEntry> = { [sessionKey]: sessionEntry };
|
||||
await fs.writeFile(storePath, JSON.stringify(sessionStore, null, 2), "utf-8");
|
||||
@@ -824,14 +521,14 @@ describe("runCliTurnCompactionLifecycle", () => {
|
||||
sessionAgentId: "main",
|
||||
workspaceDir: tmpDir,
|
||||
agentDir: tmpDir,
|
||||
provider: "codex",
|
||||
model: "gpt-5.5",
|
||||
provider: "external-harness",
|
||||
model: "model",
|
||||
});
|
||||
|
||||
expect(compactCalls).toHaveLength(1);
|
||||
expect(maintenance).toHaveBeenCalledTimes(1);
|
||||
expect(recordCliCompactionInStore).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ provider: "codex", sessionKey }),
|
||||
expect.objectContaining({ provider: "external-harness", sessionKey }),
|
||||
);
|
||||
expect(updatedEntry?.compactionCount).toBe(1);
|
||||
});
|
||||
|
||||
@@ -7,6 +7,7 @@ import { ensureContextEnginesInitialized as ensureContextEnginesInitializedImpl
|
||||
import { resolveContextEngine as resolveContextEngineImpl } from "../../context-engine/registry.js";
|
||||
import type { ContextEngine } from "../../context-engine/types.js";
|
||||
import { createSubsystemLogger } from "../../logging/subsystem.js";
|
||||
import { resolveAgentHarnessPolicy } from "../harness/policy.js";
|
||||
import { ensureSelectedAgentHarnessPlugin as ensureSelectedAgentHarnessPluginImpl } from "../harness/runtime-plugin.js";
|
||||
import { maybeCompactAgentHarnessSession as maybeCompactAgentHarnessSessionImpl } from "../harness/selection.js";
|
||||
import { buildEmbeddedCompactionRuntimeContext } from "../pi-embedded-runner/compaction-runtime-context.js";
|
||||
@@ -181,6 +182,45 @@ function isRecoverableNativeHarnessCompactionFailure(
|
||||
);
|
||||
}
|
||||
|
||||
function isCodexNativeHarnessCompactionSession(
|
||||
sessionEntry: SessionEntry,
|
||||
provider: string,
|
||||
): boolean {
|
||||
const harnessId = sessionEntry.agentHarnessId?.trim().toLowerCase();
|
||||
const providerId = provider.trim().toLowerCase();
|
||||
return (
|
||||
harnessId === "codex" &&
|
||||
(providerId === "codex" || providerId === "openai" || providerId === "openai-codex")
|
||||
);
|
||||
}
|
||||
|
||||
function shouldSkipAutomaticCompactionForCodexRuntime(params: {
|
||||
cfg: OpenClawConfig;
|
||||
sessionEntry: SessionEntry;
|
||||
sessionAgentId: string;
|
||||
sessionKey: string;
|
||||
provider: string;
|
||||
model: string;
|
||||
}): boolean {
|
||||
const runtimeOverride = params.sessionEntry.agentRuntimeOverride?.trim().toLowerCase();
|
||||
if (runtimeOverride && runtimeOverride !== "auto" && runtimeOverride !== "default") {
|
||||
return runtimeOverride === "codex";
|
||||
}
|
||||
const harnessId = params.sessionEntry.agentHarnessId?.trim().toLowerCase();
|
||||
if (harnessId) {
|
||||
return isCodexNativeHarnessCompactionSession(params.sessionEntry, params.provider);
|
||||
}
|
||||
return (
|
||||
resolveAgentHarnessPolicy({
|
||||
provider: params.provider,
|
||||
modelId: params.model,
|
||||
config: params.cfg,
|
||||
agentId: params.sessionAgentId,
|
||||
sessionKey: params.sessionKey,
|
||||
}).runtime === "codex"
|
||||
);
|
||||
}
|
||||
|
||||
function readAgentIdFromSessionKey(sessionKey: string): string | undefined {
|
||||
const parts = sessionKey.trim().split(":");
|
||||
return parts[0] === "agent" && parts[1]?.trim() ? parts[1].trim() : undefined;
|
||||
@@ -401,8 +441,9 @@ async function compactNativeHarnessCliTranscript(params: {
|
||||
|
||||
if (!result?.compacted) {
|
||||
const fallbackToContextEngine =
|
||||
isUnsupportedNativeHarnessCompaction(result) ||
|
||||
isRecoverableNativeHarnessCompactionFailure(result);
|
||||
!isCodexNativeHarnessCompactionSession(params.sessionEntry, params.provider) &&
|
||||
(isUnsupportedNativeHarnessCompaction(result) ||
|
||||
isRecoverableNativeHarnessCompactionFailure(result));
|
||||
log.warn(
|
||||
`CLI native harness compaction did not reduce context for ${params.provider}/${params.model}: ${result?.reason ?? "nothing to compact"}`,
|
||||
);
|
||||
@@ -435,10 +476,32 @@ export async function runCliTurnCompactionLifecycle(params: {
|
||||
thinkLevel?: Parameters<typeof buildEmbeddedCompactionRuntimeContext>[0]["thinkLevel"];
|
||||
extraSystemPrompt?: string;
|
||||
}): Promise<SessionEntry | undefined> {
|
||||
const sessionFile = params.sessionEntry?.sessionFile;
|
||||
const contextTokenBudget = resolvePositiveInteger(params.sessionEntry?.contextTokens);
|
||||
const sessionEntry = params.sessionEntry;
|
||||
const sessionFile = sessionEntry?.sessionFile;
|
||||
const contextTokenBudget = resolvePositiveInteger(sessionEntry?.contextTokens);
|
||||
if (!sessionFile || !contextTokenBudget) {
|
||||
return params.sessionEntry;
|
||||
return sessionEntry;
|
||||
}
|
||||
if (
|
||||
shouldSkipAutomaticCompactionForCodexRuntime({
|
||||
cfg: params.cfg,
|
||||
sessionEntry,
|
||||
sessionAgentId: params.sessionAgentId,
|
||||
sessionKey: params.sessionKey,
|
||||
provider: params.provider,
|
||||
model: params.model,
|
||||
})
|
||||
) {
|
||||
// Codex CLI/app-server runtimes own their automatic transcript compaction.
|
||||
// Avoid resurrecting OpenClaw's paternalistic budget fallback here; explicit
|
||||
// /compact or plugin compaction still forwards through the harness path.
|
||||
log.debug("skipping OpenClaw CLI compaction for Codex runtime session", {
|
||||
sessionId: params.sessionId,
|
||||
sessionKey: params.sessionKey,
|
||||
provider: params.provider,
|
||||
model: params.model,
|
||||
});
|
||||
return sessionEntry;
|
||||
}
|
||||
|
||||
const sessionManager = cliCompactionDeps.openSessionManager(sessionFile);
|
||||
|
||||
@@ -1372,7 +1372,7 @@ describe("compactEmbeddedPiSession hooks (ownsCompaction engine)", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("falls back to context-engine compaction when native harness binding is recoverable", async () => {
|
||||
it("does not fall back to context-engine compaction for Codex native binding failures", async () => {
|
||||
maybeCompactAgentHarnessSessionMock.mockResolvedValueOnce({
|
||||
ok: false,
|
||||
compacted: false,
|
||||
@@ -1389,10 +1389,11 @@ describe("compactEmbeddedPiSession hooks (ownsCompaction engine)", () => {
|
||||
}),
|
||||
);
|
||||
|
||||
expect(result.ok).toBe(true);
|
||||
expect(result.compacted).toBe(true);
|
||||
expect(result.ok).toBe(false);
|
||||
expect(result.compacted).toBe(false);
|
||||
expect(result.reason).toBe("no codex app-server thread binding");
|
||||
expect(maybeCompactAgentHarnessSessionMock).toHaveBeenCalledTimes(1);
|
||||
expect(contextEngineCompactMock).toHaveBeenCalledTimes(1);
|
||||
expect(contextEngineCompactMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("does not fire after_compaction when compaction fails", async () => {
|
||||
|
||||
@@ -50,7 +50,12 @@ import type { EmbeddedPiCompactResult } from "./types.js";
|
||||
|
||||
function shouldFallbackAfterHarnessCompaction(
|
||||
result: EmbeddedPiCompactResult | undefined,
|
||||
harnessPolicyRuntime: string | undefined,
|
||||
explicitHarnessId: string | undefined,
|
||||
): boolean {
|
||||
if (harnessPolicyRuntime === "codex" || explicitHarnessId === "codex") {
|
||||
return false;
|
||||
}
|
||||
return (
|
||||
result?.ok === false &&
|
||||
(result.failure?.reason === "missing_thread_binding" ||
|
||||
@@ -126,6 +131,13 @@ export async function compactEmbeddedPiSession(
|
||||
contextTokenBudget,
|
||||
contextEnginePluginId: resolveContextEngineOwnerPluginId(contextEngine),
|
||||
});
|
||||
const harnessPolicy = resolveAgentHarnessPolicy({
|
||||
provider: params.provider,
|
||||
modelId: params.model,
|
||||
config: params.config,
|
||||
agentId: agentIds.sessionAgentId,
|
||||
sessionKey: params.sessionKey,
|
||||
});
|
||||
const harnessResult = await maybeCompactAgentHarnessSession({
|
||||
...params,
|
||||
contextEngine,
|
||||
@@ -133,7 +145,13 @@ export async function compactEmbeddedPiSession(
|
||||
contextEngineRuntimeContext,
|
||||
});
|
||||
if (harnessResult) {
|
||||
if (!shouldFallbackAfterHarnessCompaction(harnessResult)) {
|
||||
if (
|
||||
!shouldFallbackAfterHarnessCompaction(
|
||||
harnessResult,
|
||||
harnessPolicy.runtime,
|
||||
params.agentHarnessId,
|
||||
)
|
||||
) {
|
||||
await contextEngine.dispose?.();
|
||||
return harnessResult;
|
||||
}
|
||||
|
||||
@@ -818,7 +818,7 @@ describe("runMemoryFlushIfNeeded", () => {
|
||||
expect(compactCall.currentTokenCount).toBeGreaterThanOrEqual(100_000);
|
||||
});
|
||||
|
||||
it("uses the persisted Codex runtime context window for OpenAI preflight compaction", async () => {
|
||||
it("skips OpenClaw preflight compaction for persisted Codex runtime sessions", async () => {
|
||||
registerMemoryFlushPlanResolverForTest(() => ({
|
||||
softThresholdTokens: 4_000,
|
||||
forceFlushTranscriptBytes: 1_000_000_000,
|
||||
@@ -835,7 +835,7 @@ describe("runMemoryFlushIfNeeded", () => {
|
||||
agentHarnessId: "codex",
|
||||
};
|
||||
|
||||
await runPreflightCompactionIfNeeded({
|
||||
const entry = await runPreflightCompactionIfNeeded({
|
||||
cfg: {
|
||||
models: {
|
||||
providers: {
|
||||
@@ -860,12 +860,11 @@ describe("runMemoryFlushIfNeeded", () => {
|
||||
replyOperation: createReplyOperation(),
|
||||
});
|
||||
|
||||
expect(compactEmbeddedPiSessionMock).toHaveBeenCalledTimes(1);
|
||||
const compactCall = requireCompactEmbeddedPiSessionCall();
|
||||
expect(compactCall.currentTokenCount).toBe(347_000);
|
||||
expect(entry).toBe(sessionEntry);
|
||||
expect(compactEmbeddedPiSessionMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("still compacts when a fresh persisted token total is over the threshold", async () => {
|
||||
it("leaves fresh over-threshold Codex token snapshots to native Codex auto-compaction", async () => {
|
||||
registerMemoryFlushPlanResolverForTest(() => ({
|
||||
softThresholdTokens: 4_000,
|
||||
forceFlushTranscriptBytes: 1_000_000_000,
|
||||
@@ -882,7 +881,7 @@ describe("runMemoryFlushIfNeeded", () => {
|
||||
agentHarnessId: "codex",
|
||||
};
|
||||
|
||||
await runPreflightCompactionIfNeeded({
|
||||
const entry = await runPreflightCompactionIfNeeded({
|
||||
cfg: {
|
||||
models: {
|
||||
providers: {
|
||||
@@ -907,9 +906,55 @@ describe("runMemoryFlushIfNeeded", () => {
|
||||
replyOperation: createReplyOperation(),
|
||||
});
|
||||
|
||||
expect(compactEmbeddedPiSessionMock).toHaveBeenCalledTimes(1);
|
||||
const compactCall = requireCompactEmbeddedPiSessionCall();
|
||||
expect(compactCall.currentTokenCount).toBe(347_000);
|
||||
expect(entry).toBe(sessionEntry);
|
||||
expect(compactEmbeddedPiSessionMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("leaves policy-resolved OpenAI Codex sessions to native Codex auto-compaction", async () => {
|
||||
registerMemoryFlushPlanResolverForTest(() => ({
|
||||
softThresholdTokens: 4_000,
|
||||
forceFlushTranscriptBytes: 1_000_000_000,
|
||||
reserveTokensFloor: 0,
|
||||
prompt: "Pre-compaction memory flush.\nNO_REPLY",
|
||||
systemPrompt: "Write memory to memory/YYYY-MM-DD.md.",
|
||||
relativePath: "memory/2023-11-14.md",
|
||||
}));
|
||||
const sessionEntry: SessionEntry = {
|
||||
sessionId: "session",
|
||||
updatedAt: Date.now(),
|
||||
totalTokens: 347_000,
|
||||
totalTokensFresh: false,
|
||||
};
|
||||
|
||||
const entry = await runPreflightCompactionIfNeeded({
|
||||
cfg: {
|
||||
models: {
|
||||
providers: {
|
||||
openai: { models: [{ id: "gpt-5.5", contextWindow: 1_000_000 }] },
|
||||
"openai-codex": { models: [{ id: "gpt-5.5", contextWindow: 350_000 }] },
|
||||
},
|
||||
},
|
||||
agents: { defaults: { compaction: { memoryFlush: {} } } },
|
||||
} as never,
|
||||
followupRun: createTestFollowupRun({
|
||||
provider: "openai",
|
||||
model: "gpt-5.5",
|
||||
sessionId: "session",
|
||||
sessionKey: "agent:main:telegram:default:direct:12345",
|
||||
runtimePolicySessionKey: "agent:main:telegram:default:direct:12345",
|
||||
}),
|
||||
defaultModel: "gpt-5.5",
|
||||
sessionEntry,
|
||||
sessionStore: { "agent:main:telegram:default:direct:12345": sessionEntry },
|
||||
sessionKey: "agent:main:telegram:default:direct:12345",
|
||||
runtimePolicySessionKey: "agent:main:telegram:default:direct:12345",
|
||||
storePath: path.join(rootDir, "sessions.json"),
|
||||
isHeartbeat: false,
|
||||
replyOperation: createReplyOperation(),
|
||||
});
|
||||
|
||||
expect(entry).toBe(sessionEntry);
|
||||
expect(compactEmbeddedPiSessionMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("keeps the OpenAI API context window for persisted PI runtime overrides", async () => {
|
||||
|
||||
@@ -229,6 +229,19 @@ function resolveFollowupContextConfigProvider(params: {
|
||||
runtimePolicySessionKey?: string;
|
||||
}): string {
|
||||
const provider = params.followupRun.run.provider;
|
||||
return resolveContextConfigProviderForRuntime({
|
||||
provider,
|
||||
runtimeId: resolveFollowupAgentRuntimeId(params),
|
||||
});
|
||||
}
|
||||
|
||||
function resolveFollowupAgentRuntimeId(params: {
|
||||
cfg: OpenClawConfig;
|
||||
followupRun: FollowupRun;
|
||||
sessionEntry?: SessionEntry;
|
||||
sessionKey?: string;
|
||||
runtimePolicySessionKey?: string;
|
||||
}): string {
|
||||
const matchingSessionEntry =
|
||||
params.sessionEntry?.sessionId === params.followupRun.run.sessionId
|
||||
? params.sessionEntry
|
||||
@@ -243,13 +256,10 @@ function resolveFollowupContextConfigProvider(params: {
|
||||
? persistedRuntimeOverride
|
||||
: matchingSessionEntry?.agentHarnessId;
|
||||
if (persistedRuntimeId) {
|
||||
return resolveContextConfigProviderForRuntime({
|
||||
provider,
|
||||
runtimeId: persistedRuntimeId,
|
||||
});
|
||||
return persistedRuntimeId;
|
||||
}
|
||||
const harnessPolicy = resolveAgentHarnessPolicy({
|
||||
provider,
|
||||
provider: params.followupRun.run.provider,
|
||||
modelId: params.followupRun.run.model,
|
||||
config: params.cfg,
|
||||
agentId: params.followupRun.run.agentId,
|
||||
@@ -259,10 +269,17 @@ function resolveFollowupContextConfigProvider(params: {
|
||||
params.followupRun.run.runtimePolicySessionKey ??
|
||||
params.followupRun.run.sessionKey,
|
||||
});
|
||||
return resolveContextConfigProviderForRuntime({
|
||||
provider,
|
||||
runtimeId: harnessPolicy.runtime,
|
||||
});
|
||||
return harnessPolicy.runtime;
|
||||
}
|
||||
|
||||
function followupUsesCodexRuntime(params: {
|
||||
cfg: OpenClawConfig;
|
||||
followupRun: FollowupRun;
|
||||
sessionEntry?: SessionEntry;
|
||||
sessionKey?: string;
|
||||
runtimePolicySessionKey?: string;
|
||||
}): boolean {
|
||||
return normalizeLowercaseStringOrEmpty(resolveFollowupAgentRuntimeId(params)) === "codex";
|
||||
}
|
||||
|
||||
function resolveVisibleMemoryFlushErrorPayloads(payloads?: ReplyPayload[]): ReplyPayload[] {
|
||||
@@ -613,6 +630,23 @@ export async function runPreflightCompactionIfNeeded(params: {
|
||||
if (params.isHeartbeat || isCli) {
|
||||
return entry ?? params.sessionEntry;
|
||||
}
|
||||
if (
|
||||
followupUsesCodexRuntime({
|
||||
cfg: params.cfg,
|
||||
followupRun: params.followupRun,
|
||||
sessionEntry: entry,
|
||||
sessionKey: params.sessionKey,
|
||||
runtimePolicySessionKey: params.runtimePolicySessionKey,
|
||||
})
|
||||
) {
|
||||
// Codex runtime sessions should reach Codex with their real thread state.
|
||||
// Its harness owns automatic compaction; OpenClaw preflight compaction is
|
||||
// only for non-Codex embedded runtimes.
|
||||
logVerbose(
|
||||
`preflightCompaction skipped: sessionKey=${params.sessionKey} runtime=codex reason=codex_native_auto_compaction`,
|
||||
);
|
||||
return entry ?? params.sessionEntry;
|
||||
}
|
||||
|
||||
const contextWindowTokens = resolveMemoryFlushContextWindowTokens({
|
||||
cfg: params.cfg,
|
||||
|
||||
@@ -347,6 +347,41 @@ describe("handleCompactCommand", () => {
|
||||
expect(call.tokensAfter).toBe(321);
|
||||
});
|
||||
|
||||
it("reports started Codex native compaction without incrementing completed compaction state", async () => {
|
||||
vi.mocked(compactEmbeddedPiSession).mockResolvedValueOnce({
|
||||
ok: true,
|
||||
compacted: false,
|
||||
result: {
|
||||
summary: "",
|
||||
firstKeptEntryId: "",
|
||||
tokensBefore: 199_000,
|
||||
details: {
|
||||
backend: "codex-app-server",
|
||||
threadId: "thread-1",
|
||||
signal: "thread/compact/start",
|
||||
pending: true,
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
const result = await handleCompactCommand(
|
||||
{
|
||||
...buildCompactParams("/compact", {
|
||||
commands: { text: true },
|
||||
channels: { whatsapp: { allowFrom: ["*"] } },
|
||||
} as OpenClawConfig),
|
||||
sessionEntry: {
|
||||
sessionId: "live-session",
|
||||
updatedAt: Date.now(),
|
||||
},
|
||||
} as HandleCommandsParams,
|
||||
true,
|
||||
);
|
||||
|
||||
expect(result?.reply?.text).toContain("Codex compaction started");
|
||||
expect(vi.mocked(incrementCompactionCount)).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("resolves /compact context budget from the active Codex runtime config instead of stale session metadata", async () => {
|
||||
vi.mocked(compactEmbeddedPiSession).mockResolvedValueOnce({
|
||||
ok: true,
|
||||
|
||||
@@ -83,6 +83,19 @@ function formatCompactionReason(reason?: string): string | undefined {
|
||||
return text;
|
||||
}
|
||||
|
||||
function isCodexNativeCompactionStartedResult(result: { result?: { details?: unknown } }): boolean {
|
||||
const details = result.result?.details;
|
||||
if (!details || typeof details !== "object" || Array.isArray(details)) {
|
||||
return false;
|
||||
}
|
||||
const record = details as Record<string, unknown>;
|
||||
return (
|
||||
record.backend === "codex-app-server" &&
|
||||
record.signal === "thread/compact/start" &&
|
||||
record.pending === true
|
||||
);
|
||||
}
|
||||
|
||||
function resolveManualCompactContextTokenBudget(params: {
|
||||
cfg: OpenClawConfig;
|
||||
provider?: string;
|
||||
@@ -267,17 +280,20 @@ export const handleCompactCommand: CommandHandler = async (params) => {
|
||||
ownerNumbers: params.command.ownerList.length > 0 ? params.command.ownerList : undefined,
|
||||
});
|
||||
|
||||
const codexNativeCompactionStarted = isCodexNativeCompactionStartedResult(result);
|
||||
const compactLabel =
|
||||
result.ok || isCompactionSkipReason(result.reason)
|
||||
? result.compacted
|
||||
? result.result?.tokensBefore != null && result.result?.tokensAfter != null
|
||||
? `Compacted (${runtime.formatTokenCount(result.result.tokensBefore)} → ${runtime.formatTokenCount(result.result.tokensAfter)})`
|
||||
: result.result?.tokensBefore
|
||||
? `Compacted (${runtime.formatTokenCount(result.result.tokensBefore)} before)`
|
||||
: "Compacted"
|
||||
: "Compaction skipped"
|
||||
? codexNativeCompactionStarted
|
||||
? "Codex compaction started"
|
||||
: result.compacted
|
||||
? result.result?.tokensBefore != null && result.result?.tokensAfter != null
|
||||
? `Compacted (${runtime.formatTokenCount(result.result.tokensBefore)} → ${runtime.formatTokenCount(result.result.tokensAfter)})`
|
||||
: result.result?.tokensBefore
|
||||
? `Compacted (${runtime.formatTokenCount(result.result.tokensBefore)} before)`
|
||||
: "Compacted"
|
||||
: "Compaction skipped"
|
||||
: "Compaction failed";
|
||||
if (result.ok && result.compacted) {
|
||||
if (result.ok && result.compacted && !codexNativeCompactionStarted) {
|
||||
await runtime.incrementCompactionCount({
|
||||
cfg: params.cfg,
|
||||
sessionEntry: targetSessionEntry,
|
||||
|
||||
@@ -1343,6 +1343,29 @@ describe("agentCliCommand", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("does not run a fresh embedded session when a /compact control command times out", async () => {
|
||||
await withTempStore(async () => {
|
||||
callGateway.mockRejectedValue(createGatewayTimeoutError());
|
||||
|
||||
await expect(
|
||||
agentCliCommand(
|
||||
{
|
||||
message: "/compact",
|
||||
sessionId: "locked-session",
|
||||
runId: "locked-run",
|
||||
},
|
||||
runtime,
|
||||
),
|
||||
).rejects.toThrow("gateway timeout");
|
||||
|
||||
expect(callGateway).toHaveBeenCalledTimes(1);
|
||||
expect(agentCommand).not.toHaveBeenCalled();
|
||||
expect(
|
||||
mockMessages(runtime.error).some((message) => message.includes("EMBEDDED FALLBACK")),
|
||||
).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
it("uses the explicit session key agent for timeout fallback sessions", async () => {
|
||||
await withTempStore(async () => {
|
||||
callGateway.mockRejectedValue(createGatewayTimeoutError());
|
||||
|
||||
@@ -144,6 +144,11 @@ function isGatewayAgentTimeoutError(err: unknown): boolean {
|
||||
return err instanceof Error && err.message.includes("gateway request timeout for agent");
|
||||
}
|
||||
|
||||
function isControlCommandThatMustNotFallback(opts: Pick<AgentCliOpts, "message">): boolean {
|
||||
const normalized = opts.message.trim().toLowerCase();
|
||||
return normalized === "/compact" || normalized.startsWith("/compact ");
|
||||
}
|
||||
|
||||
function isGatewayAgentEmbeddedFallbackError(err: unknown): boolean {
|
||||
return isGatewayTransportError(err);
|
||||
}
|
||||
@@ -730,6 +735,9 @@ export async function agentCliCommand(
|
||||
throw err;
|
||||
}
|
||||
if (isGatewayAgentTimeoutError(err)) {
|
||||
if (isControlCommandThatMustNotFallback(dispatchOpts)) {
|
||||
throw err;
|
||||
}
|
||||
const fallbackAgentId = resolveAgentIdForGatewayTimeoutFallback(dispatchOpts);
|
||||
const fallbackSession = createGatewayTimeoutFallbackSession(fallbackAgentId);
|
||||
runtime.error?.(
|
||||
|
||||
@@ -391,6 +391,91 @@ test("sessions.compact without maxLines runs embedded manual compaction for chec
|
||||
ws.close();
|
||||
});
|
||||
|
||||
test("sessions.compact treats Codex native compaction start as pending, not completed", async () => {
|
||||
const { dir, storePath } = await createSessionStoreDir();
|
||||
await fs.writeFile(
|
||||
path.join(dir, "sess-codex.jsonl"),
|
||||
`${JSON.stringify({ role: "user", content: "hello codex" })}\n`,
|
||||
"utf-8",
|
||||
);
|
||||
await writeSessionStore({
|
||||
entries: {
|
||||
main: sessionStoreEntry("sess-codex", {
|
||||
agentHarnessId: "codex",
|
||||
compactionCount: 2,
|
||||
totalTokens: 54_321,
|
||||
totalTokensFresh: true,
|
||||
}),
|
||||
},
|
||||
});
|
||||
embeddedRunMock.compactEmbeddedPiSession.mockResolvedValueOnce({
|
||||
ok: true,
|
||||
compacted: false,
|
||||
result: {
|
||||
summary: "",
|
||||
firstKeptEntryId: "",
|
||||
tokensBefore: 54_321,
|
||||
details: {
|
||||
backend: "codex-app-server",
|
||||
threadId: "thread-1",
|
||||
signal: "thread/compact/start",
|
||||
pending: true,
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
const { ws } = await openClient();
|
||||
await rpcReq(ws, "sessions.subscribe", {});
|
||||
const endEventPromise = onceMessage(
|
||||
ws,
|
||||
(message) =>
|
||||
message.type === "event" &&
|
||||
message.event === "session.operation" &&
|
||||
(message.payload as { operation?: unknown; phase?: unknown })?.operation === "compact" &&
|
||||
(message.payload as { operation?: unknown; phase?: unknown })?.phase === "end",
|
||||
);
|
||||
|
||||
const compacted = await rpcReq<{
|
||||
ok: true;
|
||||
key: string;
|
||||
compacted: boolean;
|
||||
result?: { details?: unknown };
|
||||
}>(ws, "sessions.compact", {
|
||||
key: "main",
|
||||
});
|
||||
|
||||
expect(compacted.ok).toBe(true);
|
||||
expect(compacted.payload?.key).toBe("agent:main:main");
|
||||
expect(compacted.payload?.compacted).toBe(false);
|
||||
expect(compacted.payload?.result?.details).toMatchObject({
|
||||
backend: "codex-app-server",
|
||||
threadId: "thread-1",
|
||||
signal: "thread/compact/start",
|
||||
pending: true,
|
||||
});
|
||||
const endEvent = await endEventPromise;
|
||||
expect(endEvent.payload).toMatchObject({
|
||||
operation: "compact",
|
||||
phase: "end",
|
||||
sessionKey: "agent:main:main",
|
||||
completed: false,
|
||||
});
|
||||
|
||||
const store = JSON.parse(await fs.readFile(storePath, "utf-8")) as Record<
|
||||
string,
|
||||
{
|
||||
compactionCount?: number;
|
||||
totalTokens?: number;
|
||||
totalTokensFresh?: boolean;
|
||||
}
|
||||
>;
|
||||
expect(store["agent:main:main"]?.compactionCount).toBe(2);
|
||||
expect(store["agent:main:main"]?.totalTokens).toBe(54_321);
|
||||
expect(store["agent:main:main"]?.totalTokensFresh).toBe(true);
|
||||
|
||||
ws.close();
|
||||
});
|
||||
|
||||
test("sessions.patch preserves nested model ids under provider overrides", async () => {
|
||||
const dir = await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-gw-sessions-nested-"));
|
||||
const storePath = path.join(dir, "sessions.json");
|
||||
|
||||
Reference in New Issue
Block a user