fix: isolate compaction structure audit

This commit is contained in:
joshavant
2026-08-12 19:55:57 -05:00
committed by Josh Avant
parent 102b1c1f3c
commit 8eb9739724
7 changed files with 66 additions and 69 deletions
+7 -5
View File
@@ -22,11 +22,13 @@ The full conversation history stays on disk. Compaction only changes what the mo
New configs default `agents.defaults.compaction.mode` to `"safeguard"` (stricter guardrails, summary quality audits). Set `mode: "default"` explicitly to opt out.
</Note>
With the built-in safeguard quality guard enabled, OpenClaw assembles suffixes,
applies the final summary budget, and then audits the exact text that would be
stored. Invalid output gets only the configured number of corrective attempts.
If no finalized summary passes, compaction stops before writing a transcript
entry, keeps the original history, and surfaces the existing recovery outcome.
With the built-in safeguard quality guard enabled, OpenClaw applies the final
summary budget before validation. Required headings must remain in the retained
generated body, while pending asks and exact identifiers must remain in the
exact text that would be stored. Invalid output gets only the configured number
of corrective attempts. If no finalized summary passes, compaction stops before
writing a transcript entry, keeps the original history, and surfaces the
existing recovery outcome.
## Auto-compaction
+1 -1
View File
@@ -618,7 +618,7 @@ Delegated consults with a requesting agent keep that requester as their owner. W
- `keepRecentTokens`: agent cut-point budget for keeping the most recent transcript tail verbatim. Default: `20000`.
- `recentTurnsPreserve`: number of most recent user/assistant turns kept verbatim outside safeguard summarization. Default: `3`.
- `identifierPolicy`: `strict` (default) or `off`. `strict` prepends built-in opaque identifier retention guidance during compaction summarization.
- `qualityGuard`: bounded validation for built-in safeguard summaries. Enabled by default in safeguard mode. OpenClaw audits the exact finalized text after suffix assembly and budgeting; when no attempt passes, it preserves the original history and returns a compaction failure instead of storing known-invalid context. Set `enabled: false` to skip the audit. Configured compaction-provider output keeps its existing provider-owned validation behavior.
- `qualityGuard`: bounded validation for built-in safeguard summaries. Enabled by default in safeguard mode. After final budgeting, required headings must remain in the retained generated body, while pending asks and exact identifiers must remain in the exact artifact to be stored. When no attempt passes, OpenClaw preserves the original history and returns a compaction failure instead of storing known-invalid context. Set `enabled: false` to skip the audit. Configured compaction-provider output keeps its existing provider-owned validation behavior.
- `midTurnPrecheck`: optional tool-loop pressure check. When `enabled: true`, OpenClaw checks context pressure after tool results are appended and before the next model call. If the context no longer fits, it aborts the current attempt before submitting the prompt and reuses the existing precheck recovery path to truncate tool results or compact and retry. Works with both `default` and `safeguard` compaction modes. Default: disabled.
- `postIndexSync`: post-compaction session-memory reindex mode. Default: `"async"`. Use `"await"` for strongest freshness, `"async"` for lower compaction latency, or `"off"` only when session-memory sync is handled elsewhere.
- `postCompactionSections`: optional AGENTS.md H2/H3 section names to re-inject after compaction. Leave unset or use `[]` to disable.
@@ -248,7 +248,7 @@ Plugins register a compaction provider via `registerCompactionProvider()` on the
- `provider`: id of a registered compaction provider plugin. Leave unset for default LLM summarization. Setting a `provider` forces `mode: "safeguard"`.
- Providers receive the same compaction instructions and identifier-preservation policy as the built-in path, and the safeguard still preserves recent-turn and split-turn suffix context after provider output.
- Built-in safeguard summarization re-distills prior summaries with new messages instead of preserving the full previous summary verbatim.
- Safeguard mode enables built-in summary quality audits by default. The audit runs after suffix assembly and final budgeting against the exact text that would be persisted. Corrective attempts stay within `qualityGuard.maxRetries`; exhaustion or a corrective generation failure cancels before append and leaves the original transcript authoritative. Set `qualityGuard.enabled: false` to skip this behavior. Configured compaction-provider output remains outside the built-in audit loop.
- Safeguard mode enables built-in summary quality audits by default. After final budgeting, the retained generated body must contain the required headings, and the exact artifact to be persisted must retain pending asks and exact identifiers. Corrective attempts stay within `qualityGuard.maxRetries`; exhaustion or a corrective generation failure cancels before append and leaves the original transcript authoritative. Set `qualityGuard.enabled: false` to skip this behavior. Configured compaction-provider output remains outside the built-in audit loop.
- If the provider fails or returns an empty result, OpenClaw falls back to built-in LLM summarization automatically. Abort/timeout signals the caller explicitly triggered are re-thrown, not swallowed, so cancellation is always respected.
Source: `src/plugins/compaction-provider.ts`, `src/agents/agent-hooks/compaction-safeguard.ts`.
@@ -198,12 +198,13 @@ function hasAskOverlap(summary: string, latestAsk: string | null): boolean {
/** Audits a candidate summary for required sections, pending asks, and identifier preservation. */
export function auditSummaryQuality(params: {
summary: string;
structuralSummary: string;
identifiers: string[];
latestAsk: string | null;
identifierPolicy?: CompactionSummarizationInstructions["identifierPolicy"];
}): { ok: boolean; reasons: string[] } {
const reasons: string[] = [];
const lines = new Set(normalizedSummaryLines(params.summary));
const lines = new Set(normalizedSummaryLines(params.structuralSummary));
for (const section of REQUIRED_SUMMARY_SECTIONS) {
if (!lines.has(section)) {
reasons.push(`missing_section:${section}`);
@@ -89,7 +89,7 @@ const {
resolveRecentTurnsPreserve,
resolveQualityGuardMaxRetries,
extractOpaqueIdentifiers,
auditSummaryQuality,
auditSummaryQuality: auditSummaryQualityOwner,
capCompactionSummary,
capCompactionSummaryPreservingSuffix,
formatFileOperations,
@@ -104,6 +104,15 @@ const {
SUMMARY_TRUNCATED_MARKER,
} = testing;
function auditSummaryQuality(
params: Omit<
Parameters<typeof compactionQualityModule.auditSummaryQuality>[0],
"structuralSummary"
>,
) {
return auditSummaryQualityOwner({ ...params, structuralSummary: params.summary });
}
beforeEach(() => {
testing.setSummarizeInStagesForTest(mockSummarizeInStages);
mockAuditSummaryQuality.mockImplementation(actualCompactionQualityModule.auditSummaryQuality);
@@ -2121,27 +2130,9 @@ describe("compaction-safeguard recent-turn preservation", () => {
const auditInput = requireRecord(mockCallArg(mockAuditSummaryQuality));
expect(auditInput.latestAsk).toBe(sourceText);
expect(auditInput.identifiers).toEqual([identifier]);
expect(auditInput.summary).toBe(
[
"## Decisions",
"No prior history.",
"",
"## Open TODOs",
"None.",
"",
"## Constraints/Rules",
"None.",
"",
"## Pending user asks",
"None.",
"",
"## Exact identifiers",
"None captured.",
"",
"## Recent turns preserved verbatim",
`- User: ${"x".repeat(600)}...`,
].join("\n"),
);
expect(auditInput.summary).toContain("## Recent turns preserved verbatim");
expect(auditInput.summary).not.toContain(identifier);
expect(auditInput.summary).not.toContain(latestAsk);
expect(mockAuditSummaryQuality.mock.results[0]?.value).toEqual({
ok: false,
reasons: [`missing_identifiers:${identifier}`, "latest_user_ask_not_reflected"],
@@ -2160,8 +2151,21 @@ describe("compaction-safeguard recent-turn preservation", () => {
it("retries when generated summary misses headings even if preserved turns contain them", async () => {
mockSummarizeInStages.mockReset();
const preservedUserText = [
"latest ask status",
"## Decisions",
"from preserved turns",
"## Open TODOs",
"from preserved turns",
"## Constraints/Rules",
"from preserved turns",
"## Pending user asks",
"latest ask status",
"## Exact identifiers",
"/tmp/preserved-turn-bypass.log",
].join("\n");
mockSummarizeInStages
.mockResolvedValueOnce(summaryResult("latest ask status"))
.mockResolvedValueOnce(summaryResult("invalid generated body"))
.mockResolvedValueOnce(
summaryResult(
[
@@ -2174,7 +2178,7 @@ describe("compaction-safeguard recent-turn preservation", () => {
"## Pending user asks",
"latest ask status",
"## Exact identifiers",
"None.",
"/tmp/preserved-turn-bypass.log",
].join("\n"),
),
);
@@ -2206,28 +2210,7 @@ describe("compaction-safeguard recent-turn preservation", () => {
timestamp: 1.5,
} as unknown as AgentMessage,
{ role: "assistant", content: "older reply", timestamp: 2 } as unknown as AgentMessage,
{ role: "user", content: "latest ask status", timestamp: 3 },
{
role: "assistant",
content: [
{
type: "text",
text: [
"## Decisions",
"from preserved turns",
"## Open TODOs",
"from preserved turns",
"## Constraints/Rules",
"from preserved turns",
"## Pending user asks",
"from preserved turns",
"## Exact identifiers",
"from preserved turns",
].join("\n"),
},
],
timestamp: 4,
} as unknown as AgentMessage,
{ role: "user", content: preservedUserText, timestamp: 3 },
],
turnPrefixMessages: [],
firstKeptEntryId: "entry-1",
@@ -2252,11 +2235,15 @@ describe("compaction-safeguard recent-turn preservation", () => {
expect(result.cancel).not.toBe(true);
expect(mockSummarizeInStages).toHaveBeenCalledTimes(2);
const firstAudit = requireRecord(mockCallArg(mockAuditSummaryQuality));
expect(firstAudit.structuralSummary).toBe("invalid generated body");
expect(firstAudit.summary).toContain(preservedUserText);
const secondCall = mockCallArg(mockSummarizeInStages, 1) as {
customInstructions?: string;
};
expect(secondCall.customInstructions).toContain("Quality check feedback");
expect(secondCall.customInstructions).toContain("missing_section:## Decisions");
expect(result.compaction?.summary).toContain("## Decisions");
});
it("audits preserved latest asks in the exact finalized artifact", async () => {
+15 -12
View File
@@ -944,9 +944,14 @@ export default function compactionSafeguardExtension(api: ExtensionAPI): void {
fileOpsSummary,
workspaceContext: await workspaceContextPromise,
});
const bodyBudget = Math.max(0, MAX_COMPACTION_SUMMARY_CHARS - suffix.length);
return {
summary: capCompactionSummaryPreservingSuffix(body, suffix),
bodyBudget: Math.max(0, MAX_COMPACTION_SUMMARY_CHARS - suffix.length),
structuralSummary:
suffix.length >= MAX_COMPACTION_SUMMARY_CHARS
? ""
: capCompactionSummary(body, bodyBudget),
bodyBudget,
};
};
const compactionResult = (summary: string) => ({
@@ -957,11 +962,6 @@ export default function compactionSafeguardExtension(api: ExtensionAPI): void {
details: { readFiles, modifiedFiles },
},
});
const finalizeSummary = async (
body: string,
sections: { splitTurnSection?: string; preservedTurnsSection?: string },
) => compactionResult((await finalizeSummaryText(body, sections)).summary);
if (providerId) {
const compactionProvider: CompactionProvider | undefined = getCompactionProvider(providerId);
if (compactionProvider) {
@@ -979,12 +979,13 @@ export default function compactionSafeguardExtension(api: ExtensionAPI): void {
messages: baseMessagesToSummarize,
recentTurnsPreserve,
});
return await finalizeSummary(providerResult, {
const finalized = await finalizeSummaryText(providerResult, {
splitTurnSection: preparation.isSplitTurn
? formatSplitTurnContextSection(turnPrefixMessages)
: "",
preservedTurnsSection: formatPreservedTurnsSection(preservedMessages),
});
return compactionResult(finalized.summary);
}
log.warn(
`Compaction provider "${compactionProvider.id}" returned empty result, falling back to LLM.`,
@@ -1148,7 +1149,6 @@ export default function compactionSafeguardExtension(api: ExtensionAPI): void {
let currentInstructions = structuredInstructions;
const totalAttempts = qualityGuardEnabled ? qualityGuardMaxRetries + 1 : 1;
let lastAuditReasons: string[] = [];
for (let attempt = 0; attempt < totalAttempts; attempt += 1) {
let splitTurnSectionLocal = "";
@@ -1179,7 +1179,7 @@ export default function compactionSafeguardExtension(api: ExtensionAPI): void {
if (signal?.aborted) {
signal.throwIfAborted();
}
if (lastAuditReasons.length > 0) {
if (attempt > 0) {
log.warn(
"Compaction safeguard: corrective generation failed; " +
`reasonCode=corrective_generation_failed attempt=${attempt + 1}`,
@@ -1192,8 +1192,11 @@ export default function compactionSafeguardExtension(api: ExtensionAPI): void {
}
throw attemptError;
}
const finalized = await finalizeSummaryText(historySummary, {
splitTurnSection: splitTurnSectionLocal,
const structuralSummary = appendSummarySection(
historySummary,
splitTurnSectionLocal ? `\n\n${splitTurnSectionLocal}` : "",
);
const finalized = await finalizeSummaryText(structuralSummary, {
preservedTurnsSection: preservedTurnsSectionLocal,
});
@@ -1205,6 +1208,7 @@ export default function compactionSafeguardExtension(api: ExtensionAPI): void {
}
const quality = auditSummaryQuality({
summary: finalized.summary,
structuralSummary: finalized.structuralSummary,
identifiers,
latestAsk: latestUserAsk,
identifierPolicy,
@@ -1212,7 +1216,6 @@ export default function compactionSafeguardExtension(api: ExtensionAPI): void {
if (quality.ok) {
return compactionResult(finalized.summary);
}
lastAuditReasons = quality.reasons;
if (!canRegenerate || attempt >= totalAttempts - 1) {
const reasonCodes = [
...new Set(quality.reasons.map((reason) => reason.split(":", 1)[0])),
@@ -418,9 +418,13 @@ describe("AgentSession loop correctness", () => {
const databasePath = resolveSqliteTargetFromSessionStorePath(target.storePath).path;
expect(closeOpenClawAgentDatabaseByPath(databasePath)).toBe(true);
const reopened = SessionManager.open(target, dir);
expect(reopened.getBranch()).toEqual(persistedBefore.slice(1));
expect(reopened.getBranch().some((entry) => entry.type === "compaction")).toBe(false);
expect(reopened.buildSessionContext()).toEqual(contextBefore);
try {
expect(reopened.getBranch()).toEqual(persistedBefore.slice(1));
expect(reopened.getBranch().some((entry) => entry.type === "compaction")).toBe(false);
expect(reopened.buildSessionContext()).toEqual(contextBefore);
} finally {
closeOpenClawAgentDatabaseByPath(databasePath);
}
});
it("keeps a successful high-usage response and performs threshold maintenance without retry", async () => {