fix(auto-reply): recover stranded message-tool finals by default (#99536)

In messages.visibleReplies "message_tool" sessions, a successful agent turn that produced a substantive private final without calling message(action=send) previously left the user with silence and only an operator log. The gateway now enqueues one protected front-of-queue retry prompting the model to deliver the reply, and falls back to a sanitized visible diagnostic when the retry cannot be enqueued or also strands. Queue overflow protection is unified with the in-flight-aware drop policy (skip in-flight or protected items, reject when nothing is droppable), rejected overflow no longer refreshes the drain debounce, heartbeat turns are excluded from recovery, and recovery retries no longer share the client turn's queued-turn lifecycle.

Fixes #85714

Thanks to Eva (@100yenadmin) for the contribution.
This commit is contained in:
Eva
2026-07-10 15:31:01 +07:00
committed by GitHub
parent 26d200c6a3
commit ae63a48e94
22 changed files with 1762 additions and 100 deletions
+4 -4
View File
@@ -824,9 +824,9 @@ See the full channel index: [Channels](/channels).
Group messages default to **require mention** (metadata mention or safe regex patterns). Applies to WhatsApp, Telegram, Discord, Google Chat, and iMessage group chats.
Visible replies are controlled separately. Normal group, channel, and internal WebChat direct requests default to automatic final delivery: final assistant text posts through the legacy visible reply path. Opt into `messages.visibleReplies: "message_tool"` or `messages.groupChat.visibleReplies: "message_tool"` when visible output should only post after the agent calls `message(action=send)`. If the model returns final text without calling the message tool in an opted-in tool-only mode, that final text stays private and the gateway verbose log records suppressed payload metadata.
Visible replies are controlled separately. Normal group, channel, and internal WebChat direct requests default to automatic final delivery: final assistant text posts through the legacy visible reply path. Opt into `messages.visibleReplies: "message_tool"` or `messages.groupChat.visibleReplies: "message_tool"` when visible output should only post after the agent calls `message(action=send)`. If the model returns a substantive final answer without calling the message tool in an opted-in tool-only mode, that final text stays private, the gateway verbose log records suppressed payload metadata, and OpenClaw enqueues one recovery retry asking the model to deliver the same reply via `message(action=send)`.
Tool-only visible replies require a model/runtime that reliably calls tools, and are recommended for shared ambient rooms on latest-generation models such as GPT 5.5. Some weaker models can answer final text but fail to understand that source-visible output must be sent with `message(action=send)`. For those models, use `"automatic"` so the final assistant turn is the visible reply path. If the session log shows assistant text with `didSendViaMessagingTool: false`, the model produced private final text instead of calling the message tool. Switch to a stronger tool-calling model for that channel, inspect the gateway verbose log for the suppressed payload summary, or set `messages.groupChat.visibleReplies: "automatic"` to use visible final replies for every group/channel request.
Tool-only visible replies require a model/runtime that reliably calls tools, and are recommended for shared ambient rooms on latest-generation models such as GPT 5.5. Some weaker models can answer final text but fail to understand that source-visible output must be sent with `message(action=send)`. OpenClaw recovers the common stranded-final case by default only when the final is substantive, the source turn was not a room event, send policy did not deny delivery, and no source reply was already sent. Recovery is bounded to one retry; it suppresses persistence for the synthetic retry prompt and keeps that retry out of collect batching so it cannot merge with unrelated queued prompts. If the retry also strands or cannot be enqueued, OpenClaw delivers only a sanitized diagnostic such as "I generated a reply but could not deliver it to this chat. Please try again." The original private final text is never marked for automatic source delivery. For models that repeatedly strand replies, use `"automatic"` so the final assistant turn is the visible reply path, switch to a stronger tool-calling model, inspect the gateway verbose log for the suppressed payload summary, or set `messages.groupChat.visibleReplies: "automatic"` to use visible final replies for every group/channel request.
If the message tool is unavailable under the active tool policy, OpenClaw falls back to automatic visible replies instead of silently suppressing the response. `openclaw doctor` warns about this mismatch.
@@ -836,9 +836,9 @@ This rule applies to normal agent final text. Plugin-owned conversation bindings
Symptom: a group/channel @mention shows the typing indicator and the gateway log reports `dispatch complete (queuedFinal=false, replies=0)`, but no message lands in the room. DMs to the same agent reply normally.
Cause: the group/channel visible-reply mode resolves to `"message_tool"`, so OpenClaw runs the turn but suppresses the final assistant text unless the agent calls `message(action=send)`. There is no `NO_REPLY` contract in this mode; no message-tool call means no source reply. There is no error because suppression is the configured behavior. Normal group and channel turns default to `"automatic"`, so this symptom only appears when `messages.groupChat.visibleReplies` (or global `messages.visibleReplies`) is explicitly set to `"message_tool"`. Harness `defaultVisibleReplies` does not apply here — the group/channel resolver ignores it; it only affects direct/source chats (the Codex harness suppresses direct-chat finals that way).
Cause: the group/channel visible-reply mode resolves to `"message_tool"`, so OpenClaw runs the turn but suppresses final assistant text unless the agent calls `message(action=send)`. There is no `NO_REPLY` contract in this mode; no message-tool call means the original final text is private. For substantive source turns OpenClaw now attempts one guarded recovery retry; short notes, explicit silence, room events, send-policy-denied turns, and already delivered turns are not retried. Normal group and channel turns default to `"automatic"`, so this symptom only appears when `messages.groupChat.visibleReplies` (or global `messages.visibleReplies`) is explicitly set to `"message_tool"`. Harness `defaultVisibleReplies` does not apply here — the group/channel resolver ignores it; it only affects direct/source chats (the Codex harness suppresses direct-chat finals that way).
Fix: either pick a stronger tool-calling model, remove the explicit `"message_tool"` override to fall back to the `"automatic"` default, or set `messages.groupChat.visibleReplies: "automatic"` to force visible replies for every group/channel request. The gateway hot-reloads `messages` config after the file is saved; only restart the gateway when file watching or config reload is disabled in the deployment.
Fix: either pick a stronger tool-calling model, remove the explicit `"message_tool"` override to fall back to the `"automatic"` default, or set `messages.groupChat.visibleReplies: "automatic"` to force visible replies for every group/channel request. A substantive stranded final should no longer end as silent success; it should either recover through one `message(action=send)` retry or show the sanitized delivery-failure diagnostic. The gateway hot-reloads `messages` config after the file is saved; only restart the gateway when file watching or config reload is disabled in the deployment.
**Mention types:**
@@ -368,6 +368,74 @@ describe("qa mock openai server", () => {
expect(text.match(/[.!?]+(?:\s|$)/g)).toHaveLength(2);
});
it("recovers the stranded-final fixture by calling the message tool on the retry prompt", async () => {
const server = await startMockServer();
const initialBody = await expectResponsesJson<{
output?: Array<{ content?: Array<{ text?: string }> }>;
}>(server, {
stream: false,
model: "gpt-5.5",
tools: [MESSAGE_TOOL],
input: [
makeUserInput(
"qa stranded final recovery check. Include `QA-STRANDED-85714` in a thorough multi-sentence answer, but do not call any tool yet.",
),
],
});
const initialText = initialBody.output?.[0]?.content?.[0]?.text ?? "";
expect(initialText).toContain("QA-STRANDED-85714");
expect(initialText.length).toBeGreaterThanOrEqual(120);
expect(outputItems(initialBody).some((item) => item.type === "function_call")).toBe(false);
const retryBody = await expectResponsesJson(server, {
stream: false,
model: "gpt-5.5",
tools: [MESSAGE_TOOL],
input: [
makeUserInput(
[
"qa stranded final recovery check.",
"Your previous reply was not delivered to the conversation because you did not call message(action=send).",
initialText,
].join(" "),
),
],
});
const toolCall = outputToolCall(retryBody, "message");
expect(outputToolArgsFromItem(toolCall)).toEqual({
action: "send",
message: "QA-STRANDED-85714",
});
});
it("keeps the retry-failure stranded-final fixture as text without a message tool call", async () => {
const server = await startMockServer();
const body = await expectResponsesJson<{
output?: Array<{ content?: Array<{ text?: string }> }>;
}>(server, {
stream: false,
model: "gpt-5.5",
tools: [MESSAGE_TOOL],
input: [
makeUserInput(
[
"Your previous reply was not delivered to the conversation because you did not call message(action=send).",
"Include `QA-STRANDED-RETRY-FAIL-RAW` in a thorough multi-sentence answer, but do not call any tool.",
].join(" "),
),
],
});
const text = body.output?.[0]?.content?.[0]?.text ?? "";
expect(text).toContain("QA-STRANDED-RETRY-FAIL-RAW");
expect(text.length).toBeGreaterThanOrEqual(120);
expect(outputItems(body).some((item) => item.type === "function_call")).toBe(false);
});
it("keeps final-only marker preview deltas separate from the final answer", async () => {
const server = await startMockServer({ finalOnlyMarkerPauseMs: 1 });
const response = await fetch(`${server.baseUrl}/v1/responses`, {
@@ -171,6 +171,11 @@ const QA_TOOL_PROGRESS_PROMPT_RE = /tool progress qa check/i;
const QA_GROUP_VISIBLE_REPLY_TOOL_PROMPT_RE = /qa group visible reply tool check/i;
const QA_GROUP_MESSAGE_UNAVAILABLE_FALLBACK_PROMPT_RE =
/qa group message unavailable fallback check/i;
const QA_STRANDED_FINAL_RECOVERY_PROMPT_RE = /qa stranded final recovery check/i;
const QA_STRANDED_FINAL_RETRY_FAILURE_PROMPT_RE = /qa stranded final retry failure check/i;
const QA_STRANDED_FINAL_RETRY_PROMPT_RE = /you did not call message\(action=send\)/i;
const QA_STRANDED_FINAL_RETRY_FAILURE_MARKER =
"QA-STRANDED-RETRY-FAIL-RAW";
const QA_TELEGRAM_CURRENT_SESSION_STATUS_PROMPT_RE = /telegram current session_status qa check/i;
const QA_TELEGRAM_STREAM_SINGLE_MARKER = "QA-TELEGRAM-STREAM-SINGLE-OK";
const QA_TELEGRAM_LONG_FINAL_THREE_CHUNK_PROMPT_RE = /telegram long final three chunk qa check/i;
@@ -195,6 +200,28 @@ const QA_WHATSAPP_REPLY_TO_BOT_TRIGGER_MARKER_RE =
const QA_WHATSAPP_BATCHED_FINAL_MARKER_RE = /\bWHATSAPP_QA_BATCHED_FINAL_([A-Z0-9]+)\b/u;
const QA_SUBAGENT_DIRECT_FALLBACK_PROMPT_RE = /subagent direct fallback qa check/i;
const QA_SUBAGENT_DIRECT_FALLBACK_WORKER_RE = /subagent direct fallback worker/i;
function buildStrandedFinalRecoveryText(): string {
return [
"QA-STRANDED-85714 confirms this is a substantive private final reply that initially skipped the message tool.",
"The reply is intentionally long enough to exercise message_tool_only stranded-final recovery before the retry delivers it visibly.",
].join(" ");
}
function buildStrandedFinalRetryFailureText(): string {
return [
"QA-STRANDED-RETRY-FAIL-RAW confirms this retry also produced a substantive private final reply instead of calling the message tool.",
"This text must remain private so the gateway can deliver only its sanitized failure diagnostic to the source chat.",
].join(" ");
}
function isStrandedFinalRetryFailureRequest(allInputText: string): boolean {
return (
QA_STRANDED_FINAL_RETRY_FAILURE_PROMPT_RE.test(allInputText) ||
(QA_STRANDED_FINAL_RETRY_PROMPT_RE.test(allInputText) &&
allInputText.includes(QA_STRANDED_FINAL_RETRY_FAILURE_MARKER))
);
}
const QA_SUBAGENT_DIRECT_FALLBACK_MARKER = "QA-SUBAGENT-DIRECT-FALLBACK-OK";
const QA_IMAGE_GENERATION_PROMPT_RE =
/image generation check|capability flip image check|\/tool\s+image_generate/i;
@@ -1496,6 +1523,14 @@ function buildAssistantText(
"The response is long enough to exercise message_tool_only private-final detection while remaining private to the agent transcript.",
].join(" ");
}
if (isStrandedFinalRetryFailureRequest(allInputText)) {
return buildStrandedFinalRetryFailureText();
}
if (QA_STRANDED_FINAL_RECOVERY_PROMPT_RE.test(allInputText)) {
return QA_STRANDED_FINAL_RETRY_PROMPT_RE.test(allInputText)
? "QA-STRANDED-85714"
: buildStrandedFinalRecoveryText();
}
if (/tool continuity check/i.test(prompt) && toolOutput) {
return `Protocol note: model switch handoff confirmed on ${model || "the requested model"}. QA mission from QA_KICKOFF_TASK.md still applies: understand this OpenClaw repo from source + docs before acting.`;
}
@@ -2519,6 +2554,21 @@ async function buildResponsesPayload(
},
]);
}
if (isStrandedFinalRetryFailureRequest(allInputText)) {
return buildAssistantEvents(buildStrandedFinalRetryFailureText());
}
if (QA_STRANDED_FINAL_RECOVERY_PROMPT_RE.test(allInputText)) {
if (QA_STRANDED_FINAL_RETRY_PROMPT_RE.test(allInputText)) {
if (!toolOutput && hasDeclaredTool(body, "message")) {
return buildToolCallEventsWithArgs("message", {
action: "send",
message: "QA-STRANDED-85714",
});
}
return buildAssistantEvents("");
}
return buildAssistantEvents(buildStrandedFinalRecoveryText());
}
if (QA_GROUP_VISIBLE_REPLY_TOOL_PROMPT_RE.test(allInputText)) {
const marker = exactMarkerDirective ?? exactReplyDirective ?? "QA-GROUP-TOOL-OK";
if (!toolOutput && hasDeclaredTool(body, "message")) {
+11 -7
View File
@@ -647,18 +647,22 @@ describe("qa scenario catalog", () => {
const strandedConfig = readQaScenarioExecutionConfig("message-tool-stranded-final-reply") as
| { requiredChannelDriver?: string; requiredProviderMode?: string }
| undefined;
const retryFailureConfig = readQaScenarioExecutionConfig(
"message-tool-stranded-final-retry-failure",
) as { requiredProviderMode?: string } | undefined;
const stranded = readQaScenarioById("message-tool-stranded-final-reply");
const strandedFlow = JSON.stringify(stranded.execution.flow);
const retryFailure = readQaScenarioById("message-tool-stranded-final-retry-failure");
const heartbeat = readQaScenarioById("commitments-heartbeat-target-none");
const heartbeatFlow = JSON.stringify(heartbeat.execution.flow);
expect(strandedConfig?.requiredProviderMode).toBe("mock-openai");
expect(strandedConfig?.requiredChannelDriver).toBe("qa-channel");
expect(strandedFlow).toContain("this seeded scenario is mock-openai only");
expect(strandedFlow).toContain("state.getSnapshot().events.slice(eventStartIndex)");
expect(strandedFlow).toContain("message.deleted !== true");
expect(strandedFlow).toContain("config.expectedMarker");
expect(strandedFlow).not.toContain("waitForNoOutbound");
expect(retryFailureConfig?.requiredProviderMode).toBe("mock-openai");
expect(JSON.stringify(stranded.execution.flow)).toContain(
"this seeded scenario is mock-openai only",
);
expect(JSON.stringify(retryFailure.execution.flow)).toContain(
"this seeded scenario is mock-openai only",
);
expect(heartbeatFlow).toContain("sessionKey");
expect(heartbeatFlow).toContain("commitmentOutbound.length === 0");
expect(heartbeatFlow).not.toContain("waitForNoOutbound");
@@ -1,4 +1,4 @@
title: Message-tool-only private final reply warning
title: Message-tool-only stranded final reply recovery
scenario:
id: message-tool-stranded-final-reply
@@ -9,14 +9,15 @@ scenario:
secondary:
- channels.qa-channel
- tools.message
objective: Reproduce #85714 under messages.visibleReplies=message_tool a long private final reply that never calls the message tool is never delivered, and the gateway emits the private-final WARN.
objective: Reproduce #85714 end to end under messages.visibleReplies=message_tool. A long private final reply that never calls the message tool must warn, enqueue one recovery retry by default, and deliver the original reply via message(action=send).
gatewayConfigPatch:
messages:
visibleReplies: message_tool
successCriteria:
- The mock provider returns a long normal final answer and does not plan the message tool.
- Under message_tool_only delivery the private final is absent from the complete outbound event history, and any progress lifecycle output is deleted.
- The gateway logs the private-final WARN from source-reply/private-final.
- The mock provider first returns a long normal final answer and does not plan the message tool.
- The gateway logs the private-final WARN from source-reply/private-final and enqueues one retry.
- The retry turn calls message(action=send), and the direct conversation receives the marker exactly once.
- Recovery does not loop after the single retry delivery.
docsRefs:
- docs/channels/qa-channel.md
codeRefs:
@@ -25,19 +26,19 @@ scenario:
- src/auto-reply/reply/dispatch-from-config.ts
execution:
kind: flow
summary: Send a direct message_tool_only turn whose model reply omits the message tool, and verify the private final never reaches outbound event history.
summary: Send a direct message_tool_only turn whose model reply omits the message tool, then verify the default one-shot retry delivers via message(action=send).
config:
requiredProviderMode: mock-openai
requiredChannelDriver: qa-channel
conversationId: qa-stranded-dm
promptSnippet: qa private final reply warning check
prompt: "qa private final reply warning check. Reply to me directly in two complete sentences with `QA-STRANDED-85714` in the first sentence and a short explanation in the second sentence. Do NOT call any tool. Do NOT use the message tool."
promptSnippet: qa stranded final recovery check
prompt: "qa stranded final recovery check. Include `QA-STRANDED-85714` in a thorough multi-sentence answer, but do not call any tool yet."
expectedMarker: QA-STRANDED-85714
privateFinalLogNeedle: "source-reply/private-final"
retryPromptNeedle: "you did not call message(action=send)"
flow:
steps:
- name: warns for substantive private final text when the model omits the message tool
- name: recovers a substantive private final through one message tool retry
actions:
- assert:
expr: "env.providerMode === config.requiredProviderMode"
@@ -51,12 +52,6 @@ flow:
- ref: env
- 60000
- call: reset
- set: eventStartIndex
value:
expr: state.getSnapshot().events.length
- set: outboundStartIndex
value:
expr: "state.getSnapshot().messages.filter((message) => message.direction === 'outbound').length"
- set: logCursor
value:
expr: markGatewayLogCursor()
@@ -68,46 +63,60 @@ flow:
id:
expr: config.conversationId
kind: direct
senderId: alice
senderId:
expr: config.conversationId
senderName: Alice
text:
expr: config.prompt
- call: waitForCondition
- call: waitForOutboundMessage
saveAs: outbound
args:
- ref: state
- lambda:
expr: "String(readGatewayLogs() ?? '').slice(logCursor).includes(config.privateFinalLogNeedle) ? true : undefined"
- expr: liveTurnTimeoutMs(env, 30000)
- 100
- call: sleep
args:
- expr: liveTurnTimeoutMs(env, 30000)
- set: scenarioOutbound
value:
expr: "state.getSnapshot().messages.filter((message) => message.direction === 'outbound').slice(outboundStartIndex)"
- set: scenarioOutboundEvents
value:
expr: "state.getSnapshot().events.slice(eventStartIndex).filter((event) => ['outbound-message', 'message-edited', 'message-deleted'].includes(event.kind) && event.message?.direction === 'outbound')"
- set: survivingOutbound
value:
expr: "scenarioOutbound.filter((message) => message.deleted !== true)"
- assert:
expr: "scenarioOutboundEvents.every((event) => !String(event.message?.text ?? '').includes(config.expectedMarker))"
message:
expr: "`private final marker escaped into outbound event history: ${JSON.stringify(scenarioOutboundEvents.map((event) => ({ kind: event.kind, text: event.message?.text ?? '' })))} `"
- assert:
expr: survivingOutbound.length === 0
message:
expr: "`expected no surviving outbound messages, saw ${JSON.stringify(survivingOutbound.map((message) => message.text))}`"
params: [candidate]
expr: "candidate.conversation.id === config.conversationId && candidate.conversation.kind === 'direct' && String(candidate.text ?? '').includes(config.expectedMarker)"
- expr: liveTurnTimeoutMs(env, 180000)
- set: scenarioRequests
value:
expr: "env.mock ? (await fetchJson(`${env.mock.baseUrl}/debug/requests`)).slice(requestCountBefore).filter((request) => String(request.allInputText ?? '').includes(config.promptSnippet)) : []"
- set: strandedRequests
value:
expr: "scenarioRequests.filter((request) => !String(request.allInputText ?? '').includes(config.retryPromptNeedle))"
- set: retryDeliveryRequests
value:
expr: "scenarioRequests.filter((request) => String(request.allInputText ?? '').includes(config.retryPromptNeedle) && request.plannedToolName === 'message')"
- assert:
expr: "!env.mock || scenarioRequests.length > 0"
message: expected mock request evidence that the turn actually ran
expr: "!env.mock || strandedRequests.length > 0"
message: expected mock request evidence that the stranding turn actually ran
- assert:
expr: "!env.mock || scenarioRequests.every((request) => request.plannedToolName !== 'message')"
expr: "!env.mock || strandedRequests.every((request) => request.plannedToolName !== 'message')"
message:
expr: "`model should not have planned the message tool, saw ${JSON.stringify(scenarioRequests.map((request) => request.plannedToolName ?? null))}`"
expr: "`turn 1 should not have planned the message tool, saw ${JSON.stringify(strandedRequests.map((request) => request.plannedToolName ?? null))}`"
- assert:
expr: "!env.mock || retryDeliveryRequests.length === 1"
message:
expr: "`expected exactly one stranded-reply retry that delivers via the message tool, saw ${retryDeliveryRequests.length}`"
- assert:
expr: "!env.mock || retryDeliveryRequests.every((request) => request.plannedToolArgs?.action === 'send' && request.plannedToolArgs?.message === config.expectedMarker)"
message:
expr: "`expected message(action=send) with the marker, saw ${JSON.stringify(retryDeliveryRequests.map((request) => ({ plannedToolName: request.plannedToolName ?? null, plannedToolArgs: request.plannedToolArgs ?? null })))} `"
- set: matchingOutbound
value:
expr: "state.getSnapshot().messages.filter((message) => message.direction === 'outbound' && message.conversation.id === config.conversationId && String(message.text ?? '').includes(config.expectedMarker))"
- assert:
expr: matchingOutbound.length === 1
message:
expr: "`expected exactly one recovered visible reply, saw ${matchingOutbound.length}`"
- call: sleep
args:
- expr: liveTurnTimeoutMs(env, 8000)
- set: settledOutbound
value:
expr: "state.getSnapshot().messages.filter((message) => message.direction === 'outbound' && message.conversation.id === config.conversationId && String(message.text ?? '').includes(config.expectedMarker))"
- assert:
expr: settledOutbound.length === 1
message:
expr: "`recovery must not loop: expected one marker reply after settling, saw ${settledOutbound.length}`"
- set: privateFinalLog
value:
expr: "String(readGatewayLogs() ?? '').slice(logCursor)"
@@ -117,5 +126,5 @@ flow:
- assert:
expr: "privateFinalLog.includes(config.privateFinalLogNeedle)"
message:
expr: "`expected the gateway to log ${config.privateFinalLogNeedle} after a substantive private message_tool_only reply, but it was absent`"
detailsExpr: "`private final absent from ${scenarioOutboundEvents.length} outbound lifecycle events; transient deleted=${scenarioOutbound.filter((message) => message.deleted === true).length}; WARN logged=${privateFinalLog.includes(config.privateFinalLogNeedle)}; mock requests=${scenarioRequests.length}; gateway log: ${privateFinalLine}`"
expr: "`expected the gateway to log ${config.privateFinalLogNeedle} after the stranded substantive reply, but it was absent`"
detailsExpr: "`recovered=${matchingOutbound.length}; stranded turns=${strandedRequests.length}; retry delivery turns=${retryDeliveryRequests.length}; WARN logged=${privateFinalLog.includes(config.privateFinalLogNeedle)}; gateway log: ${privateFinalLine}`"
@@ -0,0 +1,121 @@
title: Message-tool-only stranded final retry failure diagnostic
scenario:
id: message-tool-stranded-final-retry-failure
surface: channel
coverage:
primary:
- channels.direct-visible-replies
secondary:
- channels.qa-channel
- tools.message
objective: Prove #85714 retry exhaustion is fail-closed and visible. If the one stranded-final recovery retry also omits message(action=send), OpenClaw must deliver only a sanitized diagnostic and must not leak the original private final text.
gatewayConfigPatch:
messages:
visibleReplies: message_tool
successCriteria:
- The mock provider returns a long normal final answer without planning the message tool on both the original turn and the retry turn.
- The direct conversation receives the sanitized delivery-failure diagnostic exactly once.
- The raw private final marker is never delivered to the direct conversation.
- No second stranded-reply retry is enqueued.
docsRefs:
- docs/channels/qa-channel.md
codeRefs:
- src/auto-reply/reply/agent-runner.ts
- src/auto-reply/reply/private-message-tool-final.ts
- src/auto-reply/reply/dispatch-from-config.ts
execution:
kind: flow
summary: Send a direct message_tool_only turn whose original run and retry both omit message(action=send), then verify only the sanitized diagnostic is visible.
config:
requiredProviderMode: mock-openai
conversationId: qa-stranded-retry-failure-dm
promptSnippet: qa stranded final retry failure check
prompt: "qa stranded final retry failure check. Include `QA-STRANDED-RETRY-FAIL-RAW` in a thorough multi-sentence answer, but do not call any tool."
rawMarker: QA-STRANDED-RETRY-FAIL-RAW
expectedDiagnostic: "I generated a reply but could not deliver it to this chat. Please try again."
retryPromptNeedle: "you did not call message(action=send)"
flow:
steps:
- name: emits a sanitized diagnostic when the one retry strands again
actions:
- assert:
expr: "env.providerMode === config.requiredProviderMode"
message: this seeded scenario is mock-openai only
- call: waitForGatewayHealthy
args:
- ref: env
- 60000
- call: waitForQaChannelReady
args:
- ref: env
- 60000
- call: reset
- set: requestCountBefore
value:
expr: "env.mock ? (await fetchJson(`${env.mock.baseUrl}/debug/requests`)).length : 0"
- sendInbound:
conversation:
id:
expr: config.conversationId
kind: direct
senderId:
expr: config.conversationId
senderName: Alice
text:
expr: config.prompt
- call: waitForOutboundMessage
saveAs: outbound
args:
- ref: state
- lambda:
params: [candidate]
expr: "candidate.conversation.id === config.conversationId && candidate.conversation.kind === 'direct' && String(candidate.text ?? '') === config.expectedDiagnostic"
- expr: liveTurnTimeoutMs(env, 180000)
- set: scenarioRequests
value:
expr: "env.mock ? (await fetchJson(`${env.mock.baseUrl}/debug/requests`)).slice(requestCountBefore).filter((request) => String(request.allInputText ?? '').includes(config.rawMarker)) : []"
- set: retryRequests
value:
expr: "scenarioRequests.filter((request) => String(request.allInputText ?? '').includes(config.retryPromptNeedle))"
- assert:
expr: "!env.mock || scenarioRequests.length >= 2"
message:
expr: "`expected original and retry mock requests, saw ${scenarioRequests.length}`"
- assert:
expr: "!env.mock || scenarioRequests.every((request) => request.plannedToolName !== 'message')"
message:
expr: "`retry failure fixture must not plan message tool, saw ${JSON.stringify(scenarioRequests.map((request) => request.plannedToolName ?? null))}`"
- assert:
expr: "!env.mock || retryRequests.length === 1"
message:
expr: "`expected exactly one stranded-reply retry request, saw ${retryRequests.length}`"
- set: diagnosticOutbound
value:
expr: "state.getSnapshot().messages.filter((message) => message.direction === 'outbound' && message.conversation.id === config.conversationId && String(message.text ?? '') === config.expectedDiagnostic)"
- set: rawOutbound
value:
expr: "state.getSnapshot().messages.filter((message) => message.direction === 'outbound' && message.conversation.id === config.conversationId && String(message.text ?? '').includes(config.rawMarker))"
- assert:
expr: diagnosticOutbound.length === 1
message:
expr: "`expected exactly one sanitized diagnostic, saw ${diagnosticOutbound.length}`"
- assert:
expr: rawOutbound.length === 0
message:
expr: "`raw stranded final text must not be delivered, saw ${rawOutbound.length} raw outbound messages`"
- call: sleep
args:
- expr: liveTurnTimeoutMs(env, 8000)
- set: settledRequests
value:
expr: "env.mock ? (await fetchJson(`${env.mock.baseUrl}/debug/requests`)).slice(requestCountBefore).filter((request) => String(request.allInputText ?? '').includes(config.rawMarker)) : []"
- set: settledRetryRequests
value:
expr: "settledRequests.filter((request) => String(request.allInputText ?? '').includes(config.retryPromptNeedle))"
- assert:
expr: "!env.mock || settledRetryRequests.length === 1"
message:
expr: "`recovery must stop after retry failure: expected one retry request after settling, saw ${settledRetryRequests.length}`"
detailsExpr: "`diagnostic=${diagnosticOutbound.length}; rawOutbound=${rawOutbound.length}; retryRequests=${retryRequests.length}`"
+1 -1
View File
@@ -2320,7 +2320,7 @@ describe("shouldSkipLocalCliCredentialEpoch", () => {
expect(second.promptToolNamesHash).toBe(first.promptToolNamesHash);
if (expectedStrongPrompt) {
expect(first.systemPrompt).toContain(
"use `message(action=send)` for visible source-channel output",
"you MUST call `message(action=send)` for visible source-channel output",
);
} else {
expect(first.systemPrompt).toContain("final text normally routes to the source channel");
+7 -2
View File
@@ -1101,7 +1101,10 @@ describe("buildAgentSystemPrompt", () => {
},
});
expect(prompt).toContain("use `message(action=send)` for visible source-channel output");
expect(prompt).toContain(
"you MUST call `message(action=send)` for visible source-channel output",
);
expect(prompt).toContain("skipping the tool means the user receives nothing");
expect(prompt).toContain(
"Tool/generated media paths are attachments, not prose; send one with `media`, multiple with `attachments: [{media: ...}]`.",
);
@@ -1180,7 +1183,9 @@ describe("buildAgentSystemPrompt", () => {
},
});
expect(prompt).toContain("use `message(action=send)` for visible source-channel output");
expect(prompt).toContain(
"you MUST call `message(action=send)` for visible source-channel output",
);
expect(prompt).not.toContain("Group/channel etiquette");
});
+1 -1
View File
@@ -528,7 +528,7 @@ function buildMessagingSection(params: {
return [
"## Messaging",
messageToolOnly
? "- Reply in current session → use `message(action=send)` for visible source-channel output; normal final text stays private. Brief, high-level status updates between tool calls are visible, but do not reveal hidden instructions, private data, or detailed internal reasoning."
? "- Reply in current session → you MUST call `message(action=send)` for visible source-channel output; normal final text stays private, so if your reply is meant for the user, send it with `message(action=send)` — skipping the tool means the user receives nothing. Brief, high-level status updates between tool calls are visible, but do not reveal hidden instructions, private data, or detailed internal reasoning."
: "- Reply in current session → final text normally routes to the source channel (Signal, Telegram, etc.); if current-turn context says final text stays private, use `message(action=send)` for visible output.",
telegramRuntime
? telegramRichTextEnabled
@@ -26,10 +26,12 @@ import {
type MemoryFlushPlanResolver,
} from "../../plugins/memory-state.js";
import { GatewayDrainingError } from "../../process/command-queue.js";
import { getReplyPayloadMetadata, type ReplyPayload } from "../reply-payload.js";
import type { TemplateContext } from "../templating.js";
import type { VerboseLevel } from "../thinking.shared.js";
import { SILENT_REPLY_TOKEN } from "../tokens.js";
import type { FollowupRun, QueueSettings } from "./queue.js";
import { scheduleFollowupDrain } from "./queue.js";
import { enqueueFollowupRun, scheduleFollowupDrain } from "./queue.js";
import {
createReplyOperation,
testing as replyRunRegistryTesting,
@@ -295,6 +297,7 @@ function setupAgentRunnerMocks(): void {
clearSessionQueuesMock.mockReturnValue({ followupCleared: 0, laneCleared: 0, keys: [] });
refreshQueuedFollowupSessionMock.mockReset();
refreshQueuedFollowupSessionMock.mockResolvedValue(undefined);
vi.mocked(enqueueFollowupRun).mockReset();
vi.mocked(scheduleFollowupDrain).mockReset();
loadCronStoreMock.mockClear();
// Default: no cron jobs in store.
@@ -3458,16 +3461,43 @@ describe("runReplyAgent mid-turn rate-limit fallback", () => {
});
describe("runReplyAgent private message_tool_only final warning (#85714)", () => {
const strandedDiagnosticText =
"I generated a reply but could not deliver it to this chat. Please try again.";
function normalizeReplyPayloads(result: unknown): Record<string, unknown>[] {
const payloads = Array.isArray(result) ? result : [result];
return payloads.map((payload, index) => requireRecord(payload, `reply payload ${index}`));
}
async function runPrivateFinalCase(params: {
messagingToolSentTargets?: unknown[];
messagingToolSourceReplyPayloads?: Array<{ text?: string }>;
didDeliverSourceReplyViaMessageTool?: boolean;
finalAssistantText?: string;
finalAssistantRawText?: string;
payloads?: ReplyPayload[];
payloadText?: string;
successfulCronAdds?: number;
resolvedVerboseLevel?: VerboseLevel;
isNewSession?: boolean;
inboundEventKind?: string;
transcriptPrompt?: string;
summaryLine?: string;
strandedReplyRetry?: boolean;
sendPolicyDenied?: boolean;
isHeartbeat?: boolean;
replyOperation?: ReturnType<typeof createReplyOperation>;
queuedLifecycle?: FollowupRun["queuedLifecycle"];
}) {
const tmp = await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-stranded-"));
const storePath = path.join(tmp, "sessions.json");
const sessionKey = "stranded";
const sessionEntry = { sessionId: "session", updatedAt: Date.now(), totalTokens: 1_000 };
const sessionEntry = {
sessionId: "session",
updatedAt: Date.now(),
totalTokens: 1_000,
...(params.sendPolicyDenied ? { sendPolicy: "deny" as const } : {}),
};
await fs.writeFile(storePath, JSON.stringify({ [sessionKey]: sessionEntry }, null, 2), "utf-8");
const finalAssistantText =
@@ -3477,11 +3507,23 @@ describe("runReplyAgent private message_tool_only final warning (#85714)", () =>
// payloadText can differ from the assistant text to simulate metadata-only
// payloads (verbose notices, usage line) that must NOT trigger the warn —
// detection keys off the assistant final text, not the payload bundle.
payloads: [{ text: params.payloadText ?? finalAssistantText }],
meta: { agentMeta: {}, finalAssistantVisibleText: finalAssistantText },
payloads: params.payloads ?? [{ text: params.payloadText ?? finalAssistantText }],
meta: {
agentMeta: {},
finalAssistantVisibleText: finalAssistantText,
...(params.finalAssistantRawText
? { finalAssistantRawText: params.finalAssistantRawText }
: {}),
},
...(params.messagingToolSentTargets
? { messagingToolSentTargets: params.messagingToolSentTargets }
: {}),
...(params.messagingToolSourceReplyPayloads
? { messagingToolSourceReplyPayloads: params.messagingToolSourceReplyPayloads }
: {}),
...(params.didDeliverSourceReplyViaMessageTool
? { didDeliverSourceReplyViaMessageTool: true }
: {}),
...(params.successfulCronAdds === undefined
? {}
: { successfulCronAdds: params.successfulCronAdds }),
@@ -3489,15 +3531,20 @@ describe("runReplyAgent private message_tool_only final warning (#85714)", () =>
const sessionCtx = {
Provider: "whatsapp",
OriginatingChannel: "whatsapp",
OriginatingTo: "+15550001111",
AccountId: "primary",
MessageSid: "msg",
ChatType: "direct",
...(params.inboundEventKind ? { InboundEventKind: params.inboundEventKind } : {}),
} as unknown as TemplateContext;
const followupRun = {
prompt: "hello",
summaryLine: "hello",
summaryLine: params.summaryLine ?? "hello",
...(params.strandedReplyRetry ? { strandedReplyRetry: true } : {}),
enqueuedAt: Date.now(),
...(params.transcriptPrompt ? { transcriptPrompt: params.transcriptPrompt } : {}),
...(params.queuedLifecycle ? { queuedLifecycle: params.queuedLifecycle } : {}),
run: {
agentId: "main",
agentDir: "/tmp/agent",
@@ -3522,7 +3569,7 @@ describe("runReplyAgent private message_tool_only final warning (#85714)", () =>
},
} as unknown as FollowupRun;
await runReplyAgent({
const result = await runReplyAgent({
commandBody: "hello",
followupRun,
queueKey: sessionKey,
@@ -3539,13 +3586,16 @@ describe("runReplyAgent private message_tool_only final warning (#85714)", () =>
storePath,
defaultModel: "anthropic/claude-opus-4-6",
agentCfgContextTokens: 200_000,
resolvedVerboseLevel: "off",
isNewSession: false,
resolvedVerboseLevel: params.resolvedVerboseLevel ?? "off",
isNewSession: params.isNewSession ?? false,
blockStreamingEnabled: false,
resolvedBlockStreamingBreak: "message_end",
shouldInjectGroupIntro: false,
typingMode: "instant",
...(params.isHeartbeat ? { opts: { isHeartbeat: true } } : {}),
...(params.replyOperation ? { replyOperation: params.replyOperation } : {}),
});
return { storePath, tmp, sessionKey, result, finalAssistantText };
}
it("warns when a substantive private final reply never used the message tool", async () => {
@@ -3554,24 +3604,123 @@ describe("runReplyAgent private message_tool_only final warning (#85714)", () =>
expect(warnPrivateFinalSpy.mock.calls[0]?.[0]).toMatchObject({ sessionKey: "stranded" });
});
it("does not warn for a short private final reply", async () => {
it("enqueues a one-shot recovery retry by default for substantive stranded finals", async () => {
const parentOnComplete = vi.fn();
const parentLifecycle = { onComplete: parentOnComplete };
const { finalAssistantText } = await runPrivateFinalCase({
queuedLifecycle: parentLifecycle,
});
expect(warnPrivateFinalSpy).toHaveBeenCalledTimes(1);
expect(vi.mocked(enqueueFollowupRun)).toHaveBeenCalledTimes(1);
const retryRun = vi.mocked(enqueueFollowupRun).mock.calls[0]?.[1];
const messagesConfig = retryRun?.run?.config?.messages as Record<string, unknown> | undefined;
expect(messagesConfig).toEqual({ visibleReplies: "message_tool" });
expect(retryRun?.summaryLine).toBe("stranded-reply-retry");
expect(retryRun?.strandedReplyRetry).toBe(true);
expect(retryRun?.prompt).toContain("message(action=send)");
expect(retryRun?.prompt).toContain(finalAssistantText);
// System retry must not inherit the client turn's one-shot lifecycle identity.
expect(retryRun?.queuedLifecycle).toBeUndefined();
expect(parentLifecycle.onComplete).toBe(parentOnComplete);
expect(parentOnComplete).not.toHaveBeenCalled();
});
it("uses visible final text, not raw assistant text, in the recovery retry prompt", async () => {
const visibleFinal =
"Visible answer that has already been normalized for the user-facing final response and is long enough to trigger recovery. It includes a second complete sentence so the substantive-final detector treats it as a real reply.";
await runPrivateFinalCase({
finalAssistantText: visibleFinal,
finalAssistantRawText: `<final>${visibleFinal}</final>`,
});
expect(vi.mocked(enqueueFollowupRun)).toHaveBeenCalledTimes(1);
const retryRun = vi.mocked(enqueueFollowupRun).mock.calls[0]?.[1];
expect(retryRun?.prompt).toContain(visibleFinal);
expect(retryRun?.prompt).not.toContain("<final>");
});
it("uses normalized delivery text, not reply directive tags, in the recovery retry prompt", async () => {
const normalizedFinal =
"Visible answer that should be threaded to the current message and is long enough to trigger recovery. It includes another complete sentence so the substantive-final detector treats it as a real reply.";
await runPrivateFinalCase({
finalAssistantText: `[[reply_to_current]] ${normalizedFinal}`,
payloadText: `[[reply_to_current]] ${normalizedFinal}`,
});
expect(vi.mocked(enqueueFollowupRun)).toHaveBeenCalledTimes(1);
const retryRun = vi.mocked(enqueueFollowupRun).mock.calls[0]?.[1];
expect(retryRun?.prompt).toContain(normalizedFinal);
expect(retryRun?.prompt).not.toContain("[[reply_to_current]]");
});
it("excludes raw trace and status payloads from the recovery retry prompt", async () => {
const visibleFinal =
"Visible answer that should be delivered to the source chat. It includes another complete sentence so the substantive-final detector treats it as a real reply.";
const rawTraceText =
"🔎 Model Input (User Role):\n```text\nsecret user trace that must not reach chat\n```";
const statusText = "🧩 Active Memory: status=ok query=private-context";
await runPrivateFinalCase({
finalAssistantText: visibleFinal,
payloads: [
{ text: visibleFinal },
{ text: rawTraceText },
{ text: statusText, isStatusNotice: true },
],
});
expect(vi.mocked(enqueueFollowupRun)).toHaveBeenCalledTimes(1);
const retryRun = vi.mocked(enqueueFollowupRun).mock.calls[0]?.[1];
expect(retryRun?.prompt).toContain(visibleFinal);
expect(retryRun?.prompt).not.toContain("secret user trace");
expect(retryRun?.prompt).not.toContain("Active Memory");
});
it("suppresses retry prompt persistence and keeps the retry out of collect batches", async () => {
await runPrivateFinalCase({ transcriptPrompt: "original user question" });
expect(vi.mocked(enqueueFollowupRun)).toHaveBeenCalledTimes(1);
const retryRun = vi.mocked(enqueueFollowupRun).mock.calls[0]?.[1];
expect(retryRun?.transcriptPrompt).toBeUndefined();
expect(retryRun?.userTurnTranscriptRecorder).toBeUndefined();
expect(retryRun?.currentInboundContext).toBeUndefined();
expect(retryRun?.run?.suppressNextUserMessagePersistence).toBe(true);
expect(retryRun?.run?.sourceReplyDeliveryMode).toBe("message_tool_only");
expect(retryRun?.disableCollectBatching).toBe(true);
expect(vi.mocked(enqueueFollowupRun).mock.calls[0]?.[3]).toBe("none");
expect(vi.mocked(enqueueFollowupRun).mock.calls[0]?.[5]).toBe(false);
expect(vi.mocked(enqueueFollowupRun).mock.calls[0]?.[6]).toEqual({ position: "front" });
});
it("does not warn or enqueue retry for a short private final reply", async () => {
await runPrivateFinalCase({ finalAssistantText: "Nothing to send here." });
expect(warnPrivateFinalSpy).not.toHaveBeenCalled();
expect(vi.mocked(enqueueFollowupRun)).not.toHaveBeenCalled();
});
it("does not warn when the message tool delivered this turn", async () => {
it("does not warn or enqueue retry when the message tool delivered this turn", async () => {
await runPrivateFinalCase({
messagingToolSentTargets: [{ tool: "message", provider: "whatsapp", to: "+15550001111" }],
didDeliverSourceReplyViaMessageTool: true,
});
expect(warnPrivateFinalSpy).not.toHaveBeenCalled();
expect(vi.mocked(enqueueFollowupRun)).not.toHaveBeenCalled();
});
it("still warns when only an unrelated cron side effect succeeded", async () => {
it("still retries when the message tool sent only to a non-source target", async () => {
await runPrivateFinalCase({
messagingToolSentTargets: [{ tool: "message", provider: "whatsapp", to: "+15559998888" }],
});
expect(warnPrivateFinalSpy).toHaveBeenCalledTimes(1);
expect(vi.mocked(enqueueFollowupRun)).toHaveBeenCalledTimes(1);
});
it("still retries when only an unrelated cron side effect succeeded", async () => {
await runPrivateFinalCase({ successfulCronAdds: 1 });
expect(warnPrivateFinalSpy).toHaveBeenCalledTimes(1);
expect(vi.mocked(enqueueFollowupRun)).toHaveBeenCalledTimes(1);
});
it("does not warn on an intentional NO_REPLY turn even when metadata payloads remain", async () => {
it("does not warn or enqueue retry on an intentional NO_REPLY turn even when metadata payloads remain", async () => {
// Assistant went silent (NO_REPLY), but a verbose/usage metadata payload
// survives in finalPayloads. The warn must key off the assistant text, not
// the payload bundle, so no private-final warning should fire.
@@ -3580,5 +3729,142 @@ describe("runReplyAgent private message_tool_only final warning (#85714)", () =>
payloadText: "Auto-compaction complete (count 1).",
});
expect(warnPrivateFinalSpy).not.toHaveBeenCalled();
expect(vi.mocked(enqueueFollowupRun)).not.toHaveBeenCalled();
});
it("does not warn or enqueue retry for room_event turns", async () => {
await runPrivateFinalCase({ inboundEventKind: "room_event" });
expect(warnPrivateFinalSpy).not.toHaveBeenCalled();
expect(vi.mocked(enqueueFollowupRun)).not.toHaveBeenCalled();
});
it("does not warn, enqueue retry, or emit diagnostic for heartbeat runs", async () => {
const { result } = await runPrivateFinalCase({ isHeartbeat: true });
expect(warnPrivateFinalSpy).not.toHaveBeenCalled();
expect(vi.mocked(enqueueFollowupRun)).not.toHaveBeenCalled();
const payloads = result === undefined ? [] : normalizeReplyPayloads(result);
expect(payloads.some((payload) => payload.text === strandedDiagnosticText)).toBe(false);
});
it("does not warn or enqueue retry when send policy denied source delivery", async () => {
await runPrivateFinalCase({ sendPolicyDenied: true });
expect(warnPrivateFinalSpy).not.toHaveBeenCalled();
expect(vi.mocked(enqueueFollowupRun)).not.toHaveBeenCalled();
});
it("does not enqueue a second retry when a stranded-reply retry strands again", async () => {
const { result, finalAssistantText } = await runPrivateFinalCase({
summaryLine: "stranded-reply-retry",
strandedReplyRetry: true,
});
expect(warnPrivateFinalSpy).toHaveBeenCalledTimes(1);
expect(vi.mocked(enqueueFollowupRun)).not.toHaveBeenCalled();
const payloads = normalizeReplyPayloads(result);
const original = payloads.find((payload) => payload.text === finalAssistantText);
const diagnostic = payloads.find((payload) => payload.text === strandedDiagnosticText);
expect(original).toBeDefined();
expect(getReplyPayloadMetadata(original ?? {})?.deliverDespiteSourceReplySuppression).not.toBe(
true,
);
expect(diagnostic).toBeDefined();
expect(diagnostic?.isError).toBe(true);
expect(diagnostic?.isStatusNotice).toBe(true);
expect(getReplyPayloadMetadata(diagnostic ?? {})?.deliverDespiteSourceReplySuppression).toBe(
true,
);
});
it("does not treat user-controlled summary text as the internal retry marker", async () => {
await runPrivateFinalCase({
summaryLine: "stranded-reply-retry",
});
expect(warnPrivateFinalSpy).toHaveBeenCalledTimes(1);
expect(vi.mocked(enqueueFollowupRun)).toHaveBeenCalledTimes(1);
});
it("does not emit retry-failure diagnostic after internal source reply delivery", async () => {
const { result } = await runPrivateFinalCase({
summaryLine: "stranded-reply-retry",
strandedReplyRetry: true,
messagingToolSourceReplyPayloads: [{ text: "visible recovered reply" }],
finalAssistantText: "",
payloadText: "",
});
const payloads = result === undefined ? [] : normalizeReplyPayloads(result);
expect(payloads.some((payload) => payload.text === strandedDiagnosticText)).toBe(false);
});
it("emits the sanitized diagnostic when a stranded-reply retry produces no source delivery", async () => {
const { result } = await runPrivateFinalCase({
summaryLine: "stranded-reply-retry",
strandedReplyRetry: true,
finalAssistantText: "",
payloadText: "",
});
expect(warnPrivateFinalSpy).not.toHaveBeenCalled();
expect(vi.mocked(enqueueFollowupRun)).not.toHaveBeenCalled();
const payloads = normalizeReplyPayloads(result);
const diagnostic = payloads.find((payload) => payload.text === strandedDiagnosticText);
expect(diagnostic).toBeDefined();
expect(diagnostic?.isError).toBe(true);
expect(diagnostic?.isStatusNotice).toBe(true);
expect(getReplyPayloadMetadata(diagnostic ?? {})?.deliverDespiteSourceReplySuppression).toBe(
true,
);
});
it("emits the same sanitized diagnostic when the retry cannot be enqueued", async () => {
vi.mocked(enqueueFollowupRun).mockReturnValueOnce(false);
const { result, finalAssistantText } = await runPrivateFinalCase({});
expect(warnPrivateFinalSpy).toHaveBeenCalledTimes(1);
expect(vi.mocked(enqueueFollowupRun)).toHaveBeenCalledTimes(1);
const payloads = normalizeReplyPayloads(result);
const original = payloads.find((payload) => payload.text === finalAssistantText);
const diagnostic = payloads.find((payload) => payload.text === strandedDiagnosticText);
expect(original).toBeDefined();
expect(getReplyPayloadMetadata(original ?? {})?.deliverDespiteSourceReplySuppression).not.toBe(
true,
);
expect(diagnostic).toBeDefined();
expect(diagnostic?.isError).toBe(true);
expect(diagnostic?.isStatusNotice).toBe(true);
expect(getReplyPayloadMetadata(diagnostic ?? {})?.deliverDespiteSourceReplySuppression).toBe(
true,
);
});
it("schedules the stranded-reply retry drain only after the active reply operation clears", async () => {
const sessionKey = "stranded";
const replyOperation = createReplyOperation({
sessionKey,
sessionId: "session",
resetTriggered: false,
});
vi.mocked(enqueueFollowupRun).mockReturnValueOnce(true);
const drainOrder: string[] = [];
vi.mocked(scheduleFollowupDrain).mockImplementation((key) => {
expect(key).toBe(sessionKey);
expect(replyRunRegistry.get(sessionKey)).toBeUndefined();
drainOrder.push("drain");
});
await runPrivateFinalCase({ replyOperation });
expect(vi.mocked(enqueueFollowupRun)).toHaveBeenCalledTimes(1);
expect(replyRunRegistry.get(sessionKey)).toBe(replyOperation);
expect(scheduleFollowupDrain).not.toHaveBeenCalled();
drainOrder.push("clear");
replyOperation.complete();
expect(drainOrder[0]).toBe("clear");
expect(scheduleFollowupDrain).toHaveBeenCalledTimes(1);
});
});
+94 -10
View File
@@ -120,6 +120,7 @@ import {
type FollowupRun,
type QueueSettings,
} from "./queue.js";
import { normalizeReplyPayloadDirectives } from "./reply-delivery.js";
import { createReplyMediaContext } from "./reply-media-paths.js";
import { resolveReplyOperationRunState } from "./reply-operation-run-state.js";
import {
@@ -133,6 +134,10 @@ import { buildReplyUsageState, recordReplyUsageState } from "./reply-usage-state
import { resolveRoutedDeliveryThreadId } from "./routed-delivery-thread.js";
import { incrementRunCompactionCount, persistRunSessionUsage } from "./session-run-accounting.js";
import { resolveSourceReplyVisibilityPolicy } from "./source-reply-delivery-mode.js";
import {
buildStrandedReplyDeliveryFailurePayload,
buildStrandedReplyRetryFollowupRun,
} from "./stranded-reply-recovery.js";
import { createTypingSignaler } from "./typing-mode.js";
import type { TypingController } from "./typing.js";
@@ -1044,6 +1049,15 @@ function buildPendingFinalDeliveryText(payloads: ReplyPayload[]): string {
return sanitizePendingFinalDeliveryText(text);
}
function normalizeAssistantFinalDeliveryText(text: string): string {
const parsed = normalizeReplyPayloadDirectives({
payload: { text },
trimLeadingWhitespace: true,
parseMode: "auto",
});
return sanitizePendingFinalDeliveryText(parsed.payload.text ?? "");
}
function enqueueCommitmentExtractionForTurn(params: {
cfg: OpenClawConfig;
commandBody: string;
@@ -1969,6 +1983,10 @@ export async function runReplyAgent(params: {
});
const committedMessagingToolSourceReplyDelivery =
hasCommittedSourceReplyDeliveryEvidence(runResult);
// #85714: the stranded-retry diagnostic gates on committed source-reply
// evidence. `committedMessagingToolSourceReplyDelivery` is that exact signal
// after the delivery-evidence refactor extracted it into a shared helper.
const committedSourceReplyDelivery = committedMessagingToolSourceReplyDelivery;
const successfulSideEffectDelivery =
successfulSourceReplyDelivery ||
committedMessagingToolSourceReplyDelivery ||
@@ -2010,10 +2028,30 @@ export async function runReplyAgent(params: {
sessionCtx,
cfg,
});
if (
opts?.sourceReplyDeliveryMode === "message_tool_only" &&
committedMessagingToolSourceReplyDelivery
) {
const buildStrandedRetryMissingDeliveryDiagnostic = (): ReplyPayload | undefined => {
if (!sessionKey || !storePath || followupRun.strandedReplyRetry !== true) {
return undefined;
}
if (sessionCtx.InboundEventKind === "room_event" || committedSourceReplyDelivery) {
return undefined;
}
const sourceReplyPolicy = resolveSourceReplyPolicy({
cfg,
sessionCtx,
sessionEntry: activeSessionEntry,
sessionKey,
runtimePolicySessionKey,
opts,
});
if (
sourceReplyPolicy.sourceReplyDeliveryMode !== "message_tool_only" ||
sourceReplyPolicy.sendPolicyDenied
) {
return undefined;
}
return buildStrandedReplyDeliveryFailurePayload();
};
if (opts?.sourceReplyDeliveryMode === "message_tool_only" && committedSourceReplyDelivery) {
await opts.onObservedReplyDelivery?.();
}
const currentMessageId = sessionCtx.MessageSidFull ?? sessionCtx.MessageSid;
@@ -2178,6 +2216,10 @@ export async function runReplyAgent(params: {
if (silentFallbackFailurePayload) {
return silentFallbackFailurePayload;
}
const strandedRetryDiagnostic = buildStrandedRetryMissingDeliveryDiagnostic();
if (strandedRetryDiagnostic) {
return returnWithQueuedFollowupDrain(strandedRetryDiagnostic);
}
return returnWithQueuedFollowupDrain(undefined);
}
@@ -2242,6 +2284,10 @@ export async function runReplyAgent(params: {
if (silentFallbackFailurePayload) {
return silentFallbackFailurePayload;
}
const strandedRetryDiagnostic = buildStrandedRetryMissingDeliveryDiagnostic();
if (strandedRetryDiagnostic) {
return returnWithQueuedFollowupDrain(strandedRetryDiagnostic);
}
return returnWithQueuedFollowupDrain(undefined);
}
@@ -2565,7 +2611,8 @@ export async function runReplyAgent(params: {
// Capture only policy-visible final payloads in session store to support
// durable delivery retries. Hidden reasoning, message-tool-only replies,
// and sendPolicy-denied replies must not become heartbeat-replayable text.
if (sessionKey && storePath && finalPayloads.length > 0) {
const isStrandedReplyRetryRun = followupRun.strandedReplyRetry === true;
if (sessionKey && storePath && (finalPayloads.length > 0 || isStrandedReplyRetryRun)) {
const sourceReplyPolicy = resolveSourceReplyPolicy({
cfg,
sessionCtx,
@@ -2578,15 +2625,31 @@ export async function runReplyAgent(params: {
// #85714: warn only for unusually substantive private final text. In
// message_tool_only, no tool call can be intentional silence, and
// finalDeliveryText also includes verbose/status/usage metadata.
const assistantFinalText = rawAssistantText ?? "";
if (
const assistantFinalText = normalizeAssistantFinalDeliveryText(
typeof runResult.meta?.finalAssistantVisibleText === "string"
? runResult.meta.finalAssistantVisibleText
: (rawAssistantText ?? ""),
);
const isRoomEvent = sessionCtx.InboundEventKind === "room_event";
// Heartbeats already deliver fallback finals via sendDurableMessageBatch;
// recovering here would duplicate that message.
const isStrandedReply =
!isHeartbeat &&
!isRoomEvent &&
shouldWarnAboutPrivateMessageToolFinal({
sourceReplyDeliveryMode: sourceReplyPolicy.sourceReplyDeliveryMode,
sendPolicyDenied: sourceReplyPolicy.sendPolicyDenied,
successfulSourceReplyDelivery,
successfulSourceReplyDelivery: committedSourceReplyDelivery,
finalText: assistantFinalText,
})
) {
});
const retryMissingSourceDelivery =
isStrandedReplyRetryRun &&
!isHeartbeat &&
!isRoomEvent &&
sourceReplyPolicy.sourceReplyDeliveryMode === "message_tool_only" &&
!sourceReplyPolicy.sendPolicyDenied &&
!committedSourceReplyDelivery;
if (isStrandedReply) {
warnPrivateMessageToolFinal({
sessionKey,
channel:
@@ -2597,6 +2660,27 @@ export async function runReplyAgent(params: {
finalTextLength: assistantFinalText.trim().length,
});
}
if (isStrandedReply || retryMissingSourceDelivery) {
if (isStrandedReplyRetryRun) {
finalPayloads = [...finalPayloads, buildStrandedReplyDeliveryFailurePayload()];
} else {
const retryEnqueued = enqueueFollowupRun(
queueKey,
buildStrandedReplyRetryFollowupRun(followupRun, {
finalText: assistantFinalText,
sourceReplyDeliveryMode: sourceReplyPolicy.sourceReplyDeliveryMode,
}),
resolvedQueue,
"none",
runFollowupTurn,
false,
{ position: "front" },
);
if (!retryEnqueued) {
finalPayloads = [...finalPayloads, buildStrandedReplyDeliveryFailurePayload()];
}
}
}
const pendingText = sourceReplyPolicy.suppressDelivery ? "" : finalDeliveryText;
const agentId = followupRun.run.agentId;
const heartbeatAgentCfg = agentId ? resolveAgentConfig(cfg, agentId)?.heartbeat : undefined;
+338 -3
View File
@@ -233,9 +233,24 @@ function clearFollowupQueueForFollowupTest(key: string): number {
return cleared;
}
function enqueueFollowupRunForFollowupTest(key: string, run: FollowupRun): boolean {
function enqueueFollowupRunForFollowupTest(
key: string,
run: FollowupRun,
_settings?: QueueSettings,
_dedupeMode?: unknown,
_runFollowup?: unknown,
_restartIfIdle?: unknown,
options?: { position?: "tail" | "front" },
): boolean {
if (options?.position === "front") {
run.protectFromQueueOverflow = true;
}
const queue = getFollowupTestQueue(key);
queue.items.push(run);
if (options?.position === "front") {
queue.items.unshift(run);
} else {
queue.items.push(run);
}
queue.lastRun = run.run;
return true;
}
@@ -394,6 +409,7 @@ async function loadFreshFollowupRunnerModuleForTest() {
isFollowupRunAborted: (run: Pick<FollowupRun, "abortSignal" | "queueAbortSignal">) =>
run.abortSignal?.aborted === true || run.queueAbortSignal?.aborted === true,
refreshQueuedFollowupSession: refreshQueuedFollowupSessionForFollowupTest,
resolveQueueSettings: (): QueueSettings => ({ mode: "followup" }),
}));
vi.doMock("./session-run-accounting.js", () => ({
persistRunSessionUsage: persistRunSessionUsageForFollowupTest,
@@ -4487,13 +4503,20 @@ describe("createFollowupRunner messaging delivery and dedupe", () => {
sessionKey: string;
storePath: string;
opts: GetReplyOptions;
onObservedReplyDelivery: () => Promise<void>;
}> = {},
) {
if (overrides.storePath && overrides.sessionStore) {
registerFollowupTestSessionStore(overrides.storePath, overrides.sessionStore);
}
return createFollowupRunner({
opts: { ...overrides.opts, onBlockReply },
opts: {
...overrides.opts,
onBlockReply,
...(overrides.onObservedReplyDelivery
? { onObservedReplyDelivery: overrides.onObservedReplyDelivery }
: {}),
},
typing: createMockTypingController(),
typingMode: "instant",
defaultModel: "anthropic/claude-opus-4-6",
@@ -4513,6 +4536,7 @@ describe("createFollowupRunner messaging delivery and dedupe", () => {
sessionKey: string;
storePath: string;
opts: GetReplyOptions;
onObservedReplyDelivery: () => Promise<void>;
}>;
agentEvent?: { stream: string; data: Record<string, unknown> };
}) {
@@ -5520,6 +5544,317 @@ describe("createFollowupRunner messaging delivery and dedupe", () => {
expect(onBlockReply).not.toHaveBeenCalled();
});
it("enqueues a one-shot recovery retry for substantive message-tool-only queued followup finals", async () => {
const finalText =
"Here is the answer the queued user asked for. It includes enough detail to be a visible response, and it has another sentence so the substantive-final detector treats it as a real reply.";
const parentOnComplete = vi.fn();
const parentLifecycle = { onComplete: parentOnComplete };
const queued = baseQueuedRun("discord");
const { onBlockReply } = await runMessagingCase({
agentResult: {
payloads: [{ text: finalText }],
meta: { finalAssistantVisibleText: finalText },
},
queued: {
...queued,
originatingChannel: "discord",
originatingTo: "channel:C1",
queuedLifecycle: parentLifecycle,
run: {
...queued.run,
sourceReplyDeliveryMode: "message_tool_only",
},
} as FollowupRun,
});
expect(onBlockReply).not.toHaveBeenCalled();
expect(routeReplyMock).not.toHaveBeenCalled();
const retry = FOLLOWUP_TEST_QUEUES.get("main")?.items[0];
expect(retry?.summaryLine).toBe("stranded-reply-retry");
expect(retry?.strandedReplyRetry).toBe(true);
expect(retry?.disableCollectBatching).toBe(true);
expect(retry?.protectFromQueueOverflow).toBe(true);
expect(retry?.transcriptPrompt).toBeUndefined();
expect(retry?.userTurnTranscriptRecorder).toBeUndefined();
expect(retry?.currentInboundContext).toBeUndefined();
expect(retry?.run.suppressNextUserMessagePersistence).toBe(true);
expect(retry?.run.sourceReplyDeliveryMode).toBe("message_tool_only");
expect(retry?.prompt).toContain("message(action=send)");
expect(retry?.prompt).toContain(finalText);
// System retry detaches from the client turn lifecycle; parent completion owns onComplete once.
expect(retry?.queuedLifecycle).toBeUndefined();
expect(parentOnComplete).toHaveBeenCalledTimes(1);
});
it("excludes raw trace and status payloads from queued stranded recovery prompts", async () => {
const finalText =
"Here is the answer the queued user asked for. It includes enough detail to be a visible response, and it has another sentence so the substantive-final detector treats it as a real reply.";
const queued = baseQueuedRun("discord");
await runMessagingCase({
agentResult: {
payloads: [
{ text: finalText },
{
text: "🔎 Model Input (User Role):\n```text\nsecret queued trace that must not reach chat\n```",
},
{ text: "🧩 Active Memory: status=ok query=private-context", isStatusNotice: true },
],
meta: { finalAssistantVisibleText: finalText },
},
queued: {
...queued,
originatingChannel: "discord",
originatingTo: "channel:C1",
run: {
...queued.run,
sourceReplyDeliveryMode: "message_tool_only",
},
} as FollowupRun,
});
const retry = FOLLOWUP_TEST_QUEUES.get("main")?.items[0];
expect(retry?.prompt).toContain(finalText);
expect(retry?.prompt).not.toContain("secret queued trace");
expect(retry?.prompt).not.toContain("Active Memory");
});
it("does not enqueue stranded recovery for message-tool-only queued room events", async () => {
const finalText =
"Here is a long ambient room-event note that must stay private. It has enough text and another sentence to otherwise look substantive.";
const queued = baseQueuedRun("discord");
await runMessagingCase({
agentResult: {
payloads: [{ text: finalText }],
meta: { finalAssistantVisibleText: finalText },
},
queued: {
...queued,
currentInboundEventKind: "room_event",
originatingChannel: "discord",
originatingTo: "channel:C1",
run: {
...queued.run,
sourceReplyDeliveryMode: "message_tool_only",
},
} as FollowupRun,
});
expect(FOLLOWUP_TEST_QUEUES.get("main")?.items).toBeUndefined();
expect(routeReplyMock).not.toHaveBeenCalled();
});
it("does not enqueue stranded recovery when queued followup send policy denies delivery", async () => {
const finalText =
"Here is a long reply for a denied session. It includes enough detail to be substantive, but send-policy denial must remain an intentional delivery block.";
const queued = baseQueuedRun("discord");
const sessionEntry: SessionEntry = {
sessionId: "session",
updatedAt: Date.now(),
sendPolicy: "deny",
};
await runMessagingCase({
agentResult: {
payloads: [{ text: finalText }],
meta: { finalAssistantVisibleText: finalText },
},
queued: {
...queued,
originatingChannel: "discord",
originatingTo: "channel:C1",
run: {
...queued.run,
sourceReplyDeliveryMode: "message_tool_only",
},
} as FollowupRun,
runnerOverrides: { sessionEntry, sessionKey: "main" },
});
expect(FOLLOWUP_TEST_QUEUES.get("main")?.items).toBeUndefined();
expect(routeReplyMock).not.toHaveBeenCalled();
});
it("routes sanitized diagnostics when message-tool-only stranded retry strands again", async () => {
const queued = baseQueuedRun("discord");
const { onBlockReply } = await runMessagingCase({
agentResult: {
payloads: [{ text: "raw private final" }],
},
queued: {
...queued,
summaryLine: "stranded-reply-retry",
strandedReplyRetry: true,
originatingChannel: "discord",
originatingTo: "channel:C1",
run: {
...queued.run,
sourceReplyDeliveryMode: "message_tool_only",
},
} as FollowupRun,
});
expect(onBlockReply).not.toHaveBeenCalled();
expect(routeReplyMock).toHaveBeenCalledTimes(1);
expect(routeReplyMock.mock.calls[0]?.[0]?.payload?.text).toBe(
"I generated a reply but could not deliver it to this chat. Please try again.",
);
expect(String(routeReplyMock.mock.calls[0]?.[0]?.payload?.text)).not.toContain(
"raw private final",
);
});
it("routes sanitized diagnostics when message-tool-only stranded retry returns no payloads", async () => {
const queued = baseQueuedRun("discord");
const { onBlockReply } = await runMessagingCase({
agentResult: { payloads: [] },
queued: {
...queued,
summaryLine: "stranded-reply-retry",
strandedReplyRetry: true,
originatingChannel: "discord",
originatingTo: "channel:C1",
run: {
...queued.run,
sourceReplyDeliveryMode: "message_tool_only",
},
} as FollowupRun,
});
expect(onBlockReply).not.toHaveBeenCalled();
expect(routeReplyMock).toHaveBeenCalledTimes(1);
expect(routeReplyMock.mock.calls[0]?.[0]?.payload?.text).toBe(
"I generated a reply but could not deliver it to this chat. Please try again.",
);
});
it("does not route retry diagnostics when send policy denies delivery", async () => {
const queued = baseQueuedRun("discord");
const sessionEntry: SessionEntry = {
sessionId: "session",
updatedAt: Date.now(),
sendPolicy: "deny",
};
const { onBlockReply } = await runMessagingCase({
agentResult: { payloads: [] },
queued: {
...queued,
summaryLine: "stranded-reply-retry",
strandedReplyRetry: true,
originatingChannel: "discord",
originatingTo: "channel:C1",
run: {
...queued.run,
sourceReplyDeliveryMode: "message_tool_only",
},
} as FollowupRun,
runnerOverrides: { sessionEntry, sessionKey: "main" },
});
expect(onBlockReply).not.toHaveBeenCalled();
expect(routeReplyMock).not.toHaveBeenCalled();
});
it("does not treat the summary marker alone as a stranded retry", async () => {
const queued = baseQueuedRun("discord");
const { onBlockReply } = await runMessagingCase({
agentResult: { payloads: [] },
queued: {
...queued,
summaryLine: "stranded-reply-retry",
originatingChannel: "discord",
originatingTo: "channel:C1",
run: {
...queued.run,
sourceReplyDeliveryMode: "message_tool_only",
},
} as FollowupRun,
});
expect(onBlockReply).not.toHaveBeenCalled();
expect(routeReplyMock).not.toHaveBeenCalled();
});
it("does not route retry diagnostics after message-tool delivery evidence", async () => {
const queued = baseQueuedRun("discord");
const onObservedReplyDelivery = vi.fn(async () => {});
const { onBlockReply } = await runMessagingCase({
agentResult: {
payloads: [],
didDeliverSourceReplyViaMessageTool: true,
messagingToolSentTexts: ["visible recovered reply"],
messagingToolSentTargets: [{ tool: "message", provider: "discord", to: "channel:C1" }],
},
queued: {
...queued,
summaryLine: "stranded-reply-retry",
strandedReplyRetry: true,
originatingChannel: "discord",
originatingTo: "channel:C1",
run: {
...queued.run,
sourceReplyDeliveryMode: "message_tool_only",
},
} as FollowupRun,
runnerOverrides: { onObservedReplyDelivery },
});
expect(onBlockReply).not.toHaveBeenCalled();
expect(routeReplyMock).not.toHaveBeenCalled();
expect(onObservedReplyDelivery).toHaveBeenCalledTimes(1);
});
it("routes retry diagnostics when message-tool sends to a non-source target", async () => {
const queued = baseQueuedRun("discord");
const { onBlockReply } = await runMessagingCase({
agentResult: {
payloads: [],
didSendViaMessagingTool: true,
messagingToolSentTexts: ["sent somewhere else"],
messagingToolSentTargets: [{ tool: "message", provider: "discord", to: "channel:OTHER" }],
},
queued: {
...queued,
summaryLine: "stranded-reply-retry",
strandedReplyRetry: true,
originatingChannel: "discord",
originatingTo: "channel:C1",
run: {
...queued.run,
sourceReplyDeliveryMode: "message_tool_only",
},
} as FollowupRun,
});
expect(onBlockReply).not.toHaveBeenCalled();
expect(routeReplyMock).toHaveBeenCalledTimes(1);
expect(routeReplyMock.mock.calls[0]?.[0]?.payload?.text).toBe(
"I generated a reply but could not deliver it to this chat. Please try again.",
);
});
it("does not route retry diagnostics after internal source-reply payloads", async () => {
const queued = baseQueuedRun("webchat");
const { onBlockReply } = await runMessagingCase({
agentResult: {
payloads: [],
messagingToolSourceReplyPayloads: [{ text: "visible recovered reply" }],
},
queued: {
...queued,
summaryLine: "stranded-reply-retry",
strandedReplyRetry: true,
originatingChannel: "webchat",
originatingTo: undefined,
run: {
...queued.run,
sourceReplyDeliveryMode: "message_tool_only",
},
} as FollowupRun,
});
expect(onBlockReply).not.toHaveBeenCalled();
expect(routeReplyMock).not.toHaveBeenCalled();
});
it("lets provider followup route hooks force dispatcher delivery", async () => {
resolveProviderFollowupFallbackRouteMock.mockReturnValue({
route: "dispatcher",
+203 -2
View File
@@ -13,6 +13,7 @@ import { resolveContextTokensForModel } from "../../agents/context.js";
import { DEFAULT_CONTEXT_TOKENS } from "../../agents/defaults.js";
import {
hasCommittedSourceReplyDeliveryEvidence,
hasVisibleAgentPayload,
hasVisibleOutboundDeliveryEvidence,
} from "../../agents/embedded-agent-runner/delivery-evidence.js";
import {
@@ -98,19 +99,32 @@ import {
import { resolveFollowupDeliveryPayloads } from "./followup-delivery.js";
import { refreshActiveGoalContext } from "./inbound-meta.js";
import { resolveOriginMessageProvider } from "./origin-routing.js";
import { sanitizePendingFinalDeliveryText } from "./pending-final-delivery.js";
import {
shouldWarnAboutPrivateMessageToolFinal,
warnPrivateMessageToolFinal,
} from "./private-message-tool-final.js";
import {
completeFollowupRunLifecycle,
enqueueFollowupRun,
FollowupRunDeferredError,
isFollowupRunAborted,
refreshQueuedFollowupSession,
type FollowupRun,
resolveQueueSettings,
} from "./queue.js";
import { normalizeReplyPayloadDirectives } from "./reply-delivery.js";
import type { ReplyDispatchKind } from "./reply-dispatcher.types.js";
import type { ReplyOperation } from "./reply-run-registry.js";
import { admitReplyTurn } from "./reply-turn-admission.js";
import { buildReplyUsageState } from "./reply-usage-state.js";
import { isRoutableChannel, routeReply } from "./route-reply.js";
import { incrementRunCompactionCount, persistRunSessionUsage } from "./session-run-accounting.js";
import { resolveSourceReplyVisibilityPolicy } from "./source-reply-delivery-mode.js";
import {
buildStrandedReplyDeliveryFailurePayload,
buildStrandedReplyRetryFollowupRun,
} from "./stranded-reply-recovery.js";
import { createTypingSignaler } from "./typing-mode.js";
import type { TypingController } from "./typing.js";
@@ -154,6 +168,33 @@ function resolveFollowupAbortSignal(
type FollowupAgentEvent = { stream: string; data: Record<string, unknown> };
function isStrandedReplyRetryFollowup(queued: FollowupRun): boolean {
return (
queued.strandedReplyRetry === true &&
queued.currentInboundEventKind !== "room_event" &&
queued.run.sourceReplyDeliveryMode === "message_tool_only"
);
}
function hasSuccessfulFollowupSourceReplyDelivery(params: {
didDeliverSourceReplyViaMessageTool?: boolean;
messagingToolSourceReplyPayloads?: EmbeddedAgentRunResult["messagingToolSourceReplyPayloads"];
}): boolean {
return (
params.didDeliverSourceReplyViaMessageTool === true ||
hasVisibleAgentPayload({ payloads: params.messagingToolSourceReplyPayloads })
);
}
function normalizeAssistantFinalDeliveryText(text: string): string {
const parsed = normalizeReplyPayloadDirectives({
payload: { text },
trimLeadingWhitespace: true,
parseMode: "auto",
});
return sanitizePendingFinalDeliveryText(parsed.payload.text ?? "");
}
function readApprovalScopeValue(value: unknown): "turn" | "session" | undefined {
return value === "turn" || value === "session" ? value : undefined;
}
@@ -415,7 +456,9 @@ export function createFollowupRunner(params: {
const sendablePayloads = payloads.filter(
(payload): payload is ReplyPayload =>
hasOutboundReplyContent(payload) && !deliveryPlan.isSilentPayload(payload),
hasOutboundReplyContent(payload) &&
(!deliveryPlan.isSilentPayload(payload) ||
getReplyPayloadMetadata(payload)?.deliverDespiteSourceReplySuppression === true),
);
if (sendablePayloads.length === 0) {
@@ -546,7 +589,7 @@ export function createFollowupRunner(params: {
return deliveredAnyPayload;
};
return async (queued: FollowupRun) => {
const runFollowupTurn = async (queued: FollowupRun) => {
if (isFollowupRunAborted(queued)) {
completeFollowupRunLifecycle(queued);
typing.markRunComplete();
@@ -1514,6 +1557,135 @@ export function createFollowupRunner(params: {
fallbackContextTokens: activeSessionEntry?.contextTokens ?? DEFAULT_CONTEXT_TOKENS,
allowAsyncLoad: false,
}) ?? DEFAULT_CONTEXT_TOKENS;
const deliverStrandedReplyRetryFailureDiagnostic = async () => {
if (!isStrandedReplyRetryFollowup(effectiveQueued)) {
return false;
}
const sourceReplyPolicy = resolveSourceReplyVisibilityPolicy({
cfg: runtimeConfig,
ctx: {
ChatType: queued.originatingChatType ?? run.chatType,
InboundEventKind: queued.currentInboundEventKind,
Provider: queued.originatingChannel ?? run.messageProvider,
Surface: queued.originatingChannel ?? run.messageProvider,
},
requested: run.sourceReplyDeliveryMode ?? opts?.sourceReplyDeliveryMode,
sendPolicy: resolveSendPolicy({
cfg: runtimeConfig,
entry: activeSessionEntry,
sessionKey: run.runtimePolicySessionKey ?? replySessionKey,
channel:
queued.originatingChannel ?? run.messageProvider ?? activeSessionEntry?.channel,
chatType: activeSessionEntry?.chatType,
}),
});
if (sourceReplyPolicy.sendPolicyDenied) {
return false;
}
if (
hasSuccessfulFollowupSourceReplyDelivery({
didDeliverSourceReplyViaMessageTool: runResult.didDeliverSourceReplyViaMessageTool,
messagingToolSourceReplyPayloads: runResult.messagingToolSourceReplyPayloads,
})
) {
await opts?.onObservedReplyDelivery?.();
return false;
}
await sendFollowupPayloads(
[buildStrandedReplyDeliveryFailurePayload()],
effectiveQueued,
{
provider: providerUsed,
modelId: modelUsed,
},
{ runId },
);
return true;
};
const enqueueStrandedReplyRecoveryRetry = async () => {
if (isStrandedReplyRetryFollowup(effectiveQueued)) {
return false;
}
// Heartbeat turns can reach this path: runReplyAgent builds the
// followup runner with opts.isHeartbeat and may enqueue-followup while
// another run is active. Heartbeats already deliver fallback finals
// via sendDurableMessageBatch, so recovery would duplicate delivery.
if (opts?.isHeartbeat === true) {
return false;
}
const sourceReplyPolicy = resolveSourceReplyVisibilityPolicy({
cfg: runtimeConfig,
ctx: {
ChatType: queued.originatingChatType ?? run.chatType,
InboundEventKind: queued.currentInboundEventKind,
Provider: queued.originatingChannel ?? run.messageProvider,
Surface: queued.originatingChannel ?? run.messageProvider,
},
requested: run.sourceReplyDeliveryMode ?? opts?.sourceReplyDeliveryMode,
sendPolicy: resolveSendPolicy({
cfg: runtimeConfig,
entry: activeSessionEntry,
sessionKey: run.runtimePolicySessionKey ?? replySessionKey,
channel:
queued.originatingChannel ?? run.messageProvider ?? activeSessionEntry?.channel,
chatType: activeSessionEntry?.chatType,
}),
});
const assistantFinalText =
typeof runResult.meta?.finalAssistantVisibleText === "string"
? normalizeAssistantFinalDeliveryText(runResult.meta.finalAssistantVisibleText)
: "";
const isStrandedReply =
queued.currentInboundEventKind !== "room_event" &&
shouldWarnAboutPrivateMessageToolFinal({
sourceReplyDeliveryMode: sourceReplyPolicy.sourceReplyDeliveryMode,
sendPolicyDenied: sourceReplyPolicy.sendPolicyDenied,
successfulSourceReplyDelivery: hasSuccessfulFollowupSourceReplyDelivery({
didDeliverSourceReplyViaMessageTool: runResult.didDeliverSourceReplyViaMessageTool,
messagingToolSourceReplyPayloads: runResult.messagingToolSourceReplyPayloads,
}),
finalText: assistantFinalText,
});
if (!isStrandedReply) {
return false;
}
warnPrivateMessageToolFinal({
sessionKey: replySessionKey,
channel: queued.originatingChannel ?? run.messageProvider ?? activeSessionEntry?.channel,
finalTextLength: assistantFinalText.trim().length,
});
const retryEnqueued =
typeof replySessionKey === "string" &&
replySessionKey.length > 0 &&
enqueueFollowupRun(
replySessionKey,
buildStrandedReplyRetryFollowupRun(effectiveQueued, {
finalText: assistantFinalText,
sourceReplyDeliveryMode: sourceReplyPolicy.sourceReplyDeliveryMode,
}),
resolveQueueSettings({
cfg: runtimeConfig,
channel: queued.originatingChannel ?? run.messageProvider,
sessionEntry: activeSessionEntry,
}),
"none",
runFollowupTurn,
false,
{ position: "front" },
);
if (!retryEnqueued) {
await sendFollowupPayloads(
[buildStrandedReplyDeliveryFailurePayload()],
effectiveQueued,
{
provider: providerUsed,
modelId: modelUsed,
},
{ runId },
);
}
return true;
};
if (storePath && replySessionKey) {
await persistRunSessionUsage({
@@ -1619,6 +1791,12 @@ export function createFollowupRunner(params: {
}
if (finalPayloads.length === 0) {
if (await enqueueStrandedReplyRecoveryRetry()) {
return;
}
if (await deliverStrandedReplyRetryFailureDiagnostic()) {
return;
}
return;
}
if (
@@ -1729,6 +1907,28 @@ export function createFollowupRunner(params: {
}
if (run.sourceReplyDeliveryMode === "message_tool_only") {
const suppressionDeliverablePayloads = deliveryPayloads.filter(
(payload) =>
getReplyPayloadMetadata(payload)?.deliverDespiteSourceReplySuppression === true,
);
if (suppressionDeliverablePayloads.length > 0) {
await sendFollowupPayloads(
suppressionDeliverablePayloads,
effectiveQueued,
{
provider: providerUsed,
modelId: modelUsed,
},
{ runId },
);
return;
}
if (await enqueueStrandedReplyRecoveryRetry()) {
return;
}
if (await deliverStrandedReplyRetryFailureDiagnostic()) {
return;
}
logVerbose(
"followup queue: automatic source delivery suppressed by sourceReplyDeliveryMode: message_tool_only",
);
@@ -1774,4 +1974,5 @@ export function createFollowupRunner(params: {
typing.markDispatchIdle();
}
};
return runFollowupTurn;
}
+250
View File
@@ -1853,6 +1853,256 @@ describe("followup queue collect routing", () => {
expect(calls[1]?.prompt).toBe("second");
});
it("drains a disableCollectBatching retry individually instead of collecting it", async () => {
const strandedReplyRetryMarker = "stranded-reply-retry";
const key = `test-collect-disable-batching-${Date.now()}`;
const calls: FollowupRun[] = [];
const done = createDeferred<void>();
const expectedCalls = 3;
const runFollowup = async (run: FollowupRun) => {
calls.push(run);
if (calls.length >= expectedCalls) {
done.resolve();
}
};
const settings: QueueSettings = {
mode: "collect",
debounceMs: 0,
cap: 50,
dropPolicy: "summarize",
};
const route = { originatingChannel: "slack" as const, originatingTo: "channel:A" };
const retryPrompt = "[System] Please deliver this reply now by calling message(action=send).";
enqueueFollowupRun(key, createRun({ prompt: "normal one", ...route }), settings);
enqueueFollowupRun(
key,
{
...createRun({ prompt: retryPrompt, ...route }),
summaryLine: strandedReplyRetryMarker,
disableCollectBatching: true,
},
settings,
);
enqueueFollowupRun(key, createRun({ prompt: "normal two", ...route }), settings);
scheduleFollowupDrain(key, runFollowup);
await done.promise;
expect(calls).toHaveLength(3);
const retryCall = calls.find((call) => call.prompt === retryPrompt);
expect(retryCall).toBeDefined();
expect(retryCall?.prompt).not.toContain("[Queued messages while agent was busy]");
expect(retryCall?.prompt).not.toContain("Queued #");
expect(retryCall?.summaryLine).toBe(strandedReplyRetryMarker);
for (const call of calls) {
if (call.prompt.includes(retryPrompt)) {
expect(call.prompt).not.toContain("normal one");
expect(call.prompt).not.toContain("normal two");
}
}
});
it("can prepend priority followups before already queued items", () => {
const key = `test-priority-followup-front-${Date.now()}`;
const settings: QueueSettings = {
mode: "followup",
debounceMs: 0,
cap: 50,
dropPolicy: "summarize",
};
enqueueFollowupRun(key, createRun({ prompt: "queued later one" }), settings);
enqueueFollowupRun(key, createRun({ prompt: "queued later two" }), settings);
enqueueFollowupRun(
key,
createRun({ prompt: "priority retry" }),
settings,
"none",
undefined,
false,
{ position: "front" },
);
expect(getExistingFollowupQueue(key)?.items.map((item) => item.prompt)).toEqual([
"priority retry",
"queued later one",
"queued later two",
]);
expect(getExistingFollowupQueue(key)?.items[0]?.protectFromQueueOverflow).toBe(true);
});
it("preserves prepended priority followups during old-item overflow eviction", () => {
const key = `test-priority-followup-overflow-${Date.now()}`;
const settings: QueueSettings = {
mode: "followup",
debounceMs: 0,
cap: 2,
dropPolicy: "old",
};
enqueueFollowupRun(key, createRun({ prompt: "queued later one" }), settings);
enqueueFollowupRun(key, createRun({ prompt: "queued later two" }), settings);
enqueueFollowupRun(
key,
createRun({ prompt: "priority retry" }),
settings,
"none",
undefined,
false,
{ position: "front" },
);
enqueueFollowupRun(key, createRun({ prompt: "queued later three" }), settings);
expect(getExistingFollowupQueue(key)?.items.map((item) => item.prompt)).toEqual([
"priority retry",
"queued later three",
]);
});
it("keeps a cap-one protected priority followup instead of evicting it", () => {
const key = `test-priority-followup-cap-one-${Date.now()}`;
const settings: QueueSettings = {
mode: "followup",
debounceMs: 0,
cap: 1,
dropPolicy: "summarize",
};
const priorityAccepted = enqueueFollowupRun(
key,
createRun({ prompt: "priority retry" }),
settings,
"none",
undefined,
false,
{ position: "front" },
);
const normalAccepted = enqueueFollowupRun(
key,
createRun({ prompt: "normal after priority" }),
settings,
);
expect(priorityAccepted).toBe(true);
expect(normalAccepted).toBe(false);
expect(getExistingFollowupQueue(key)?.items.map((item) => item.prompt)).toEqual([
"priority retry",
]);
expect(getExistingFollowupQueue(key)?.summarySources).toHaveLength(0);
});
it("does not advance debounce stamp when overflow rejects an incoming message", () => {
const key = `test-priority-followup-debounce-reject-${Date.now()}`;
const settings: QueueSettings = {
mode: "followup",
debounceMs: 5_000,
cap: 1,
dropPolicy: "old",
};
const priorityAccepted = enqueueFollowupRun(
key,
createRun({ prompt: "priority retry" }),
settings,
"none",
undefined,
false,
{ position: "front" },
);
const queue = getExistingFollowupQueue(key);
expect(priorityAccepted).toBe(true);
expect(queue).toBeDefined();
const stampedAt = queue!.lastEnqueuedAt;
expect(stampedAt).toBeGreaterThan(0);
const rejected = enqueueFollowupRun(key, createRun({ prompt: "busy chat noise" }), settings);
expect(rejected).toBe(false);
expect(getExistingFollowupQueue(key)?.lastEnqueuedAt).toBe(stampedAt);
expect(getExistingFollowupQueue(key)?.items.map((item) => item.prompt)).toEqual([
"priority retry",
]);
});
it("leaves the queue untouched when protected overflow cannot drop enough items", () => {
const key = `test-priority-followup-atomic-overflow-${Date.now()}`;
const initialSettings: QueueSettings = {
mode: "followup",
debounceMs: 0,
cap: 3,
dropPolicy: "summarize",
};
const shrunkSettings: QueueSettings = {
...initialSettings,
cap: 1,
};
enqueueFollowupRun(
key,
createRun({ prompt: "priority retry" }),
initialSettings,
"none",
undefined,
false,
{ position: "front" },
);
enqueueFollowupRun(key, createRun({ prompt: "normal one" }), initialSettings);
enqueueFollowupRun(key, createRun({ prompt: "normal two" }), initialSettings);
const accepted = enqueueFollowupRun(
key,
createRun({ prompt: "normal after shrink" }),
shrunkSettings,
);
expect(accepted).toBe(false);
expect(getExistingFollowupQueue(key)?.items.map((item) => item.prompt)).toEqual([
"priority retry",
"normal one",
"normal two",
]);
expect(getExistingFollowupQueue(key)?.summarySources).toHaveLength(0);
expect(getExistingFollowupQueue(key)?.summaryLines).toHaveLength(0);
});
it("drains protected priority followups before overflow summaries", async () => {
const key = `test-priority-followup-before-summary-${Date.now()}`;
const calls: FollowupRun[] = [];
const done = createDeferred<void>();
const runFollowup = async (run: FollowupRun) => {
calls.push(run);
if (calls.length >= 2) {
done.resolve();
}
};
const settings: QueueSettings = {
mode: "followup",
debounceMs: 0,
cap: 1,
dropPolicy: "summarize",
};
enqueueFollowupRun(key, createRun({ prompt: "overflowed normal" }), settings);
enqueueFollowupRun(
key,
createRun({ prompt: "priority retry" }),
settings,
"none",
undefined,
false,
{ position: "front" },
);
scheduleFollowupDrain(key, runFollowup);
await done.promise;
expect(calls).toHaveLength(2);
expect(calls[0]?.prompt).toBe("priority retry");
expect(calls[1]?.prompt).toContain("[Queue overflow] Dropped 1 message due to cap.");
expect(calls[1]?.prompt).toContain("- overflowed normal");
});
it("carries image payloads across collected batches", async () => {
const key = `test-collect-images-${Date.now()}`;
const calls: FollowupRun[] = [];
+2
View File
@@ -11,8 +11,10 @@ export {
export { resolveQueueSettings } from "./queue/settings-runtime.js";
export { clearFollowupQueue, refreshQueuedFollowupSession } from "./queue/state.js";
export type {
EnqueueFollowupRunOptions,
FollowupRun,
QueueDedupeMode,
QueueInsertPosition,
QueueDropPolicy,
QueueMode,
QueueSettings,
+21 -1
View File
@@ -387,6 +387,10 @@ function resolveAggregateOwner(items: readonly FollowupRun[]): FollowupRun | und
);
}
function requiresIndividualCollectDrain(item: FollowupRun): boolean {
return item.disableCollectBatching === true || hasRuntimeOnlyFollowupMetadata(item);
}
type AggregateCancellation = {
signal?: AbortSignal;
admit: () => void;
@@ -813,6 +817,19 @@ function resolveOverflowSummarySourceGroup(queue: {
return sources;
}
async function drainProtectedPriorityFollowup(
items: FollowupRun[],
runFollowup: (run: FollowupRun) => Promise<void>,
): Promise<boolean> {
const priority = items.find((item) => item.protectFromQueueOverflow === true);
if (!priority) {
return false;
}
await runFollowup(priority);
removeQueuedItemsByRef(items, [priority]);
return true;
}
export function createOverflowSummaryRetrySource(source: FollowupRun): FollowupRun {
return {
prompt: source.prompt,
@@ -1078,6 +1095,9 @@ export function scheduleFollowupDrain(
if (queue.items.length === 0 && queue.droppedCount === 0) {
break;
}
if (await drainProtectedPriorityFollowup(queue.items, effectiveRunFollowup)) {
continue;
}
if (
queue.droppedCount > 0 &&
(await drainOverflowSummaryGroup({
@@ -1096,7 +1116,7 @@ export function scheduleFollowupDrain(
// If so, process individually to preserve per-message routing.
const isCrossChannel =
hasCrossChannelItems(queue.items, resolveCrossChannelKey) ||
queue.items.some(hasRuntimeOnlyFollowupMetadata);
queue.items.some(requiresIndividualCollectDrain);
if (collectState.forceIndividualCollect && !isCrossChannel && queue.items.length > 1) {
collectState.forceIndividualCollect = false;
}
+15 -3
View File
@@ -20,6 +20,7 @@ import {
completeFollowupRunLifecycle,
isFollowupRunAborted,
markFollowupRunEnqueued,
type EnqueueFollowupRunOptions,
type FollowupRun,
type QueueDedupeMode,
type QueueSettings,
@@ -102,10 +103,14 @@ export function enqueueFollowupRun(
dedupeMode: QueueDedupeMode = "message-id",
runFollowup?: (run: FollowupRun) => Promise<void>,
restartIfIdle = true,
options: EnqueueFollowupRunOptions = {},
): boolean {
if (isFollowupRunAborted(run)) {
return false;
}
if (options.position === "front") {
run.protectFromQueueOverflow = true;
}
const queue = getFollowupQueue(key, settings);
const recentMessageIdKey = dedupeMode !== "none" ? buildRecentMessageIdKey(run, key) : undefined;
if (recentMessageIdKey && RECENT_QUEUE_MESSAGE_IDS.peek(recentMessageIdKey)) {
@@ -132,8 +137,6 @@ export function enqueueFollowupRun(
if (!markFollowupRunEnqueued(run)) {
return false;
}
queue.lastEnqueuedAt = Date.now();
queue.lastRun = run.run;
const shouldEnqueue = applyQueueDropPolicy({
queue,
@@ -148,6 +151,7 @@ export function enqueueFollowupRun(
completeFollowupRunLifecycle(item);
}
},
isProtected: (item) => item.protectFromQueueOverflow === true,
});
if (queue.dropPolicy === "summarize") {
const overflow = queue.summarySources.length - queue.summaryLines.length;
@@ -184,9 +188,17 @@ export function enqueueFollowupRun(
completeFollowupRunLifecycle(run);
return false;
}
// Only admitted items refresh debounce; rejected overflow must not starve
// protected stranded-reply retries waiting for the quiet window.
queue.lastEnqueuedAt = Date.now();
queue.lastRun = run.run;
run.queueAbortSignal = queue.abortController.signal;
queue.items.push(run);
if (options.position === "front") {
queue.items.unshift(run);
} else {
queue.items.push(run);
}
if (recentMessageIdKey) {
RECENT_QUEUE_MESSAGE_IDS.check(recentMessageIdKey);
}
+12
View File
@@ -37,6 +37,12 @@ export type QueueSettings = {
export type QueueDedupeMode = "message-id" | "prompt" | "none";
export type QueueInsertPosition = "tail" | "front";
export type EnqueueFollowupRunOptions = {
position?: QueueInsertPosition;
};
export class FollowupRunDeferredError extends Error {
constructor(message = "Follow-up run deferred") {
super(message);
@@ -72,6 +78,12 @@ export type FollowupRun = {
/** Provider message ID, when available (for deduplication). */
messageId?: string;
summaryLine?: string;
/** Force individual drain; never merge this run into a collect batch. */
disableCollectBatching?: boolean;
/** Internal marker for the one-shot stranded final recovery retry. */
strandedReplyRetry?: boolean;
/** Preserve priority runs when old-item queue overflow eviction runs before drain. */
protectFromQueueOverflow?: boolean;
enqueuedAt: number;
images?: Array<{ type: "image"; data: string; mimeType: string }>;
imageOrder?: PromptImageOrderEntry[];
@@ -0,0 +1,48 @@
import { describe, expect, it, vi } from "vitest";
import { completeFollowupRunLifecycle, markFollowupRunEnqueued } from "./queue/types.js";
import {
buildStrandedReplyRetryFollowupRun,
STRANDED_REPLY_RETRY_MARKER,
} from "./stranded-reply-recovery.js";
import { createMockFollowupRun } from "./test-helpers.js";
describe("buildStrandedReplyRetryFollowupRun lifecycle ownership", () => {
it("does not share the client turn's queuedLifecycle with the system retry", () => {
const onComplete = vi.fn();
const onEnqueued = vi.fn(() => true);
const parent = createMockFollowupRun({
prompt: "user question",
transcriptPrompt: "user question",
queuedLifecycle: { onComplete, onEnqueued },
admissionSessionId: "sess-rotated",
onFollowupAdmissionWaitChange: vi.fn(),
});
const retry = buildStrandedReplyRetryFollowupRun(parent, {
finalText: "A substantive stranded final that must be re-delivered via message(action=send).",
sourceReplyDeliveryMode: "message_tool_only",
});
expect(retry.queuedLifecycle).toBeUndefined();
expect(retry.strandedReplyRetry).toBe(true);
expect(retry.summaryLine).toBe(STRANDED_REPLY_RETRY_MARKER);
// Session routing stays; only the client-turn lifecycle identity is detached.
expect(retry.admissionSessionId).toBe("sess-rotated");
expect(retry.onFollowupAdmissionWaitChange).toBe(parent.onFollowupAdmissionWaitChange);
expect(retry.run.sessionKey).toBe(parent.run.sessionKey);
// mark/complete no-op when lifecycle is absent (drop-policy onDrop path too).
expect(markFollowupRunEnqueued(retry)).toBe(true);
expect(onEnqueued).not.toHaveBeenCalled();
completeFollowupRunLifecycle(retry);
expect(onComplete).not.toHaveBeenCalled();
// Parent still owns the one-shot lifecycle; retry completion must not steal it.
expect(markFollowupRunEnqueued(parent)).toBe(true);
expect(onEnqueued).toHaveBeenCalledTimes(1);
completeFollowupRunLifecycle(parent);
expect(onComplete).toHaveBeenCalledTimes(1);
completeFollowupRunLifecycle(parent);
expect(onComplete).toHaveBeenCalledTimes(1);
});
});
@@ -0,0 +1,55 @@
import type { SourceReplyDeliveryMode } from "../get-reply-options.types.js";
import { markReplyPayloadForSourceSuppressionDelivery } from "../reply-payload.js";
import type { ReplyPayload } from "../types.js";
import type { FollowupRun } from "./queue/types.js";
export const STRANDED_REPLY_RETRY_MARKER = "stranded-reply-retry";
export const STRANDED_REPLY_DELIVERY_FAILURE_TEXT =
"I generated a reply but could not deliver it to this chat. Please try again.";
export function buildStrandedReplyDeliveryFailurePayload(): ReplyPayload {
return markReplyPayloadForSourceSuppressionDelivery({
text: STRANDED_REPLY_DELIVERY_FAILURE_TEXT,
isError: true,
isStatusNotice: true,
});
}
export function buildStrandedReplyRetryPrompt(finalText: string): string {
return (
`[System] Your previous reply was not delivered to the conversation because ` +
`you did not call message(action=send). Your reply text was:\n\n` +
`"${finalText}"\n\n` +
`Please deliver this reply now by calling message(action=send). ` +
`Do not add any extra commentary; just deliver the original reply.`
);
}
/** Build the one-shot recovery followup that re-prompts message(action=send). */
export function buildStrandedReplyRetryFollowupRun(
base: FollowupRun,
params: {
finalText: string;
sourceReplyDeliveryMode: SourceReplyDeliveryMode | undefined;
},
): FollowupRun {
return {
...base,
prompt: buildStrandedReplyRetryPrompt(params.finalText),
summaryLine: STRANDED_REPLY_RETRY_MARKER,
strandedReplyRetry: true,
disableCollectBatching: true,
transcriptPrompt: undefined,
userTurnTranscriptRecorder: undefined,
currentInboundContext: undefined,
// Internally generated system turn: the client turn's lifecycle (gateway cancel
// identity) completes with the parent run. queuedLifecycle is one-shot WeakSet-tracked,
// so a shared object would be double-owned and free cancel while the retry still runs.
queuedLifecycle: undefined,
run: {
...base.run,
sourceReplyDeliveryMode: params.sourceReplyDeliveryMode,
suppressNextUserMessagePersistence: true,
},
};
}
+86
View File
@@ -261,6 +261,92 @@ describe("drainNextQueueItem", () => {
expect(dropped).toEqual(["m2", "m3"]);
expect(queue.items).toEqual([m1, m4]);
});
it("skips protected items when selecting drop victims", () => {
type Item = { id: string; protected?: boolean };
const protectedItem: Item = { id: "priority", protected: true };
const normalA: Item = { id: "a" };
const normalB: Item = { id: "b" };
const normalC: Item = { id: "c" };
const queue = {
items: [protectedItem, normalA, normalB, normalC],
cap: 3,
dropPolicy: "old" as const,
droppedCount: 0,
summaryLines: [] as string[],
};
const dropped: string[] = [];
// pending=4, cap=3 → drop 2 oldest unprotected; protected stays.
const shouldEnqueue = applyQueueDropPolicy({
queue,
summarize: (item) => item.id,
isProtected: (item) => item.protected === true,
onDrop: (items) => {
dropped.push(...items.map((item) => item.id));
},
});
expect(shouldEnqueue).toBe(true);
expect(dropped).toEqual(["a", "b"]);
expect(queue.items).toEqual([protectedItem, normalC]);
});
it("rejects admission without mutating when only protected items can be dropped", () => {
type Item = { id: string; protected?: boolean };
const priority: Item = { id: "priority", protected: true };
const alsoProtected: Item = { id: "also", protected: true };
const queue = {
items: [priority, alsoProtected],
cap: 1,
dropPolicy: "old" as const,
droppedCount: 0,
summaryLines: [] as string[],
};
const dropped: string[] = [];
const shouldEnqueue = applyQueueDropPolicy({
queue,
summarize: (item) => item.id,
isProtected: (item) => item.protected === true,
onDrop: (items) => {
dropped.push(...items.map((item) => item.id));
},
});
expect(shouldEnqueue).toBe(false);
expect(dropped).toEqual([]);
expect(queue.items).toEqual([priority, alsoProtected]);
});
it("rejects when pending work is only in-flight or protected", () => {
type Item = { id: string; protected?: boolean };
const active: Item = { id: "active" };
const priority: Item = { id: "priority", protected: true };
const queue = {
items: [active, priority],
cap: 1,
dropPolicy: "old" as const,
droppedCount: 0,
summaryLines: [] as string[],
};
const inFlight = new Set<Item>([active]);
const dropped: string[] = [];
const shouldEnqueue = applyQueueDropPolicy({
queue,
inFlight,
summarize: (item) => item.id,
isProtected: (item) => item.protected === true,
onDrop: (items) => {
dropped.push(...items.map((item) => item.id));
},
});
expect(shouldEnqueue).toBe(false);
expect(dropped).toEqual([]);
expect(queue.items).toEqual([active, priority]);
});
});
describe("hasCrossChannelItems", () => {
+20 -6
View File
@@ -113,6 +113,7 @@ export function applyQueueDropPolicy<T>(params: {
summaryLimit?: number;
onDrop?: (items: T[]) => void;
inFlight?: ReadonlySet<T>;
isProtected?: (item: T) => boolean;
}): boolean {
const cap = params.queue.cap;
const pendingCount = countPendingQueueItems(params.queue.items, params.inFlight);
@@ -123,15 +124,28 @@ export function applyQueueDropPolicy<T>(params: {
return false;
}
const dropCount = pendingCount - cap + 1;
const dropped: T[] = [];
// Active identities remain in the shared array until delivery succeeds; evict only pending work.
for (let index = 0; dropped.length < dropCount; ) {
// Collect victim indices first. In-flight identities stay until delivery
// succeeds; protected priority runs (e.g. stranded-reply retries) also stay.
// Only mutate the queue when enough victims exist so a partial drop cannot
// admit overflow when the queue is full of in-flight/protected work.
const victimIndices: number[] = [];
for (
let index = 0;
index < params.queue.items.length && victimIndices.length < dropCount;
index += 1
) {
const item = params.queue.items[index];
if (params.inFlight?.has(item)) {
index += 1;
if (params.inFlight?.has(item) || params.isProtected?.(item) === true) {
continue;
}
dropped.push(...params.queue.items.splice(index, 1));
victimIndices.push(index);
}
if (victimIndices.length < dropCount) {
return false;
}
const dropped: T[] = [];
for (let i = victimIndices.length - 1; i >= 0; i -= 1) {
dropped.unshift(...params.queue.items.splice(victimIndices[i], 1));
}
params.onDrop?.(dropped);
if (params.queue.dropPolicy === "summarize") {