Repository navigation
Stop Codex harness retries on active-goal router failures #39156
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 3 commits
46dcdb9
9a18c3d
b6c554f
8e32314
8f77cb1
3057d3a
a2cf029
8fdb912
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -25,6 +25,7 @@ const { | |
| validateCodexOpenAIBaseURLFromReflect, | ||
| hasNoopInSafeOutputs, | ||
| } = require("./codex_harness.cjs"); | ||
| const { detectNonRetryableHarnessGuard } = require("./harness_retry_guard.cjs"); | ||
|
|
||
| const agentTempDir = "/tmp/gh-aw/agent"; | ||
|
|
||
|
|
@@ -412,6 +413,7 @@ env_key = "OPENAI_API_KEY" | |
| if (attempt === 0 && isAuthenticationFailedError(result.output)) return false; | ||
| if (isMissingApiKeyError(result.output)) return false; | ||
| if (hasNumerousPermissionDeniedIssues(result.output)) return false; | ||
| if (detectNonRetryableHarnessGuard(result.output).goalAlreadyActive) return false; | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [/tdd] The local 💡 Suggestion — use the full guard object for consistency// Replace the single goalAlreadyActive check with the full guard:
const nonRetryableGuard = detectNonRetryableHarnessGuard(result.output);
if (nonRetryableGuard.aiCreditsExceeded || nonRetryableGuard.awfAPIProxyBlockingRequests || nonRetryableGuard.goalAlreadyActive) return false;This mirrors the exact production branch in
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed. Updated
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Test exercises a local reimplementation of The 💡 Why this mattersThe local Consider adding a lightweight integration test (similar to the noop pre-flight tests below this block) that runs the actual harness stub with an output containing the active-goal error and asserts it does not retry.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed. Updated |
||
| const isTransient = RATE_LIMIT_ERROR_PATTERN.test(result.output) || SERVER_ERROR_PATTERN.test(result.output); | ||
| return attempt < MAX_RETRIES && (result.hasOutput || isTransient); | ||
| } | ||
|
|
@@ -461,6 +463,15 @@ env_key = "OPENAI_API_KEY" | |
| const result = { exitCode: 1, hasOutput: true, output: "permission denied\npermission denied\npermission denied" }; | ||
| expect(shouldRetry(result, 0)).toBe(false); | ||
| }); | ||
|
|
||
| it("does not retry when codex reports an existing active goal", () => { | ||
| const result = { | ||
| exitCode: 1, | ||
| hasOutput: true, | ||
| output: "cannot create a new goal because this thread already has a goal; use update_goal only when the existing goal is complete", | ||
| }; | ||
| expect(shouldRetry(result, 0)).toBe(false); | ||
| }); | ||
| }); | ||
|
|
||
| describe("noop pre-flight and retry guard", () => { | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -5,17 +5,19 @@ | |
| const AI_CREDITS_EXCEEDED_PATTERNS = [/\bmax[\s_-]*ai[\s_-]*credits[\s_-]*exceeded\b/i, /\bai[\s_-]*credits[\s_-]*rate[\s_-]*limit[\s_-]*error\b/i, /ai[\s_-]*credits?.*(?:rate[\s-]*limit|limit exceeded|budget exceeded|exceeded)/i]; | ||
|
|
||
| const AWF_API_PROXY_BLOCKING_REQUESTS_PATTERNS = [/\bawf\b.*\bapi[\s_-]*proxy\b.*\bblocking requests\b/i, /\bapi[\s_-]*proxy\b.*\bblocking requests\b/i, /\bapi[\s_-]*proxy\b.*\bblocked requests?\b/i, /\bDIFC_FILTERED\b/]; | ||
| const GOAL_ALREADY_ACTIVE_PATTERNS = [/\bthis thread already has a goal\b.*\buse update_goal only when the existing goal is complete\b/i]; | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed. Replaced
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [/diagnose] 💡 SuggestionsOption A — anchor on the stable first phrase only: const GOAL_ALREADY_ACTIVE_PATTERNS = [
/\bthis thread already has a goal\b/i,
];The first phrase is already specific enough to avoid false positives and needs no secondary anchor. Option B — make const GOAL_ALREADY_ACTIVE_PATTERNS = [
/\bthis thread already has a goal\b[\s\S]*\buse update_goal\b/i, // full message, any whitespace
/\bthis thread already has a goal\b/i, // fallback for partial output
];For comparison,
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed. Used Option B (
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Regex is a single compound match with no fallback — any message change silently disables the guard. Unlike 💡 Suggested fixAdd at least one shorter fallback so minor message variations are still caught: const GOAL_ALREADY_ACTIVE_PATTERNS = [
// prefer the full compound message for precision
/\bthis thread already has a goal\b.*\buse update_goal only when the existing goal is complete\b/i,
// fallback: catches rewording or multi-line output
/\bthis thread already has a goal\b/i,
];Alternatively, add the
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed. Replaced the single compound |
||
|
|
||
| /** | ||
| * Detect retry guard conditions that should stop harness retries immediately. | ||
| * @param {unknown} output | ||
| * @returns {{ aiCreditsExceeded: boolean, awfAPIProxyBlockingRequests: boolean }} | ||
| * @returns {{ aiCreditsExceeded: boolean, awfAPIProxyBlockingRequests: boolean, goalAlreadyActive: boolean }} | ||
| */ | ||
| function detectNonRetryableHarnessGuard(output) { | ||
| const safeOutput = typeof output === "string" ? output : ""; | ||
| return { | ||
| aiCreditsExceeded: AI_CREDITS_EXCEEDED_PATTERNS.some(pattern => pattern.test(safeOutput)), | ||
| awfAPIProxyBlockingRequests: AWF_API_PROXY_BLOCKING_REQUESTS_PATTERNS.some(pattern => pattern.test(safeOutput)), | ||
| goalAlreadyActive: GOAL_ALREADY_ACTIVE_PATTERNS.some(pattern => pattern.test(safeOutput)), | ||
| }; | ||
| } | ||
|
|
||
|
|
@@ -24,5 +26,6 @@ if (typeof module !== "undefined" && module.exports) { | |
| detectNonRetryableHarnessGuard, | ||
| AI_CREDITS_EXCEEDED_PATTERNS, | ||
| AWF_API_PROXY_BLOCKING_REQUESTS_PATTERNS, | ||
| GOAL_ALREADY_ACTIVE_PATTERNS, | ||
| }; | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -65,5 +65,13 @@ describe("harness_retry_guard.cjs", () => { | |
| const result = detectNonRetryableHarnessGuard("transient network timeout"); | ||
| expect(result.aiCreditsExceeded).toBe(false); | ||
| expect(result.awfAPIProxyBlockingRequests).toBe(false); | ||
| expect(result.goalAlreadyActive).toBe(false); | ||
| }); | ||
|
|
||
| it("detects goal already active markers", () => { | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [/tdd] The new test only validates the exact canonical message. Consider adding a negative case that confirms a partial first-phrase match alone does NOT trigger detection — this would make the two-phrase requirement an explicit, documented contract rather than an implicit one. 💡 Suggested additionsit("does not detect goal active from first phrase alone", () => {
const result = detectNonRetryableHarnessGuard("this thread already has a goal");
expect(result.goalAlreadyActive).toBe(false); // adjust to true if single-phrase is desired
});
it("detects goal already active when embedded in longer output", () => {
const result = detectNonRetryableHarnessGuard(
"[codex] cannot create a new goal because this thread already has a goal; use update_goal only when the existing goal is complete\nExit code: 1"
);
expect(result.goalAlreadyActive).toBe(true);
});Also: the existing
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Addressed. Added: (1) negative case asserting the first phrase alone does not trigger detection, (2) embedded-output multiline test, and (3) |
||
| const result = detectNonRetryableHarnessGuard("cannot create a new goal because this thread already has a goal; use update_goal only when the existing goal is complete"); | ||
| expect(result.aiCreditsExceeded).toBe(false); | ||
| expect(result.awfAPIProxyBlockingRequests).toBe(false); | ||
| expect(result.goalAlreadyActive).toBe(true); | ||
| }); | ||
| }); | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed. The reason string now reads:
"goal is already active for this thread (use update_goal when the current goal is complete)".