Skip to content
3 changes: 2 additions & 1 deletion actions/setup/js/codex_harness.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -439,10 +439,11 @@ async function main() {
}

const nonRetryableGuard = detectNonRetryableHarnessGuard(result.output);
if (nonRetryableGuard.aiCreditsExceeded || nonRetryableGuard.awfAPIProxyBlockingRequests) {
if (nonRetryableGuard.aiCreditsExceeded || nonRetryableGuard.awfAPIProxyBlockingRequests || nonRetryableGuard.goalAlreadyActive) {
const reasons = [];
if (nonRetryableGuard.aiCreditsExceeded) reasons.push("AI credits budget exceeded");
if (nonRetryableGuard.awfAPIProxyBlockingRequests) reasons.push("AWF API proxy is blocking requests");
if (nonRetryableGuard.goalAlreadyActive) reasons.push("goal is already active for this thread");

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. The reason string now reads: "goal is already active for this thread (use update_goal when the current goal is complete)".

log(`attempt ${attempt + 1}: ${reasons.join(" and ")} — not retrying (non-retryable guard condition)`);
break;
}
Expand Down
11 changes: 11 additions & 0 deletions actions/setup/js/codex_harness.test.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ const {
validateCodexOpenAIBaseURLFromReflect,
hasNoopInSafeOutputs,
} = require("./codex_harness.cjs");
const { detectNonRetryableHarnessGuard } = require("./harness_retry_guard.cjs");

const agentTempDir = "/tmp/gh-aw/agent";

Expand Down Expand Up @@ -412,6 +413,7 @@ env_key = "OPENAI_API_KEY"
if (attempt === 0 && isAuthenticationFailedError(result.output)) return false;
if (isMissingApiKeyError(result.output)) return false;
if (hasNumerousPermissionDeniedIssues(result.output)) return false;
if (detectNonRetryableHarnessGuard(result.output).goalAlreadyActive) return false;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] The local shouldRetry mock now checks goalAlreadyActive alone, while the production code checks all three guard flags together via a single detectNonRetryableHarnessGuard call. This creates drift: aiCreditsExceeded and awfAPIProxyBlockingRequests are absent from this mock, so a future reader testing "does aiCreditsExceeded prevent a retry?" via this describe block would find no coverage.

💡 Suggestion — use the full guard object for consistency
// Replace the single goalAlreadyActive check with the full guard:
const nonRetryableGuard = detectNonRetryableHarnessGuard(result.output);
if (nonRetryableGuard.aiCreditsExceeded || nonRetryableGuard.awfAPIProxyBlockingRequests || nonRetryableGuard.goalAlreadyActive) return false;

This mirrors the exact production branch in codex_harness.cjs:442 and ensures the mock keeps pace with future guard extensions automatically.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. Updated shouldRetry to mirror the production branch exactly: const nonRetryableGuard = detectNonRetryableHarnessGuard(result.output); if (nonRetryableGuard.aiCreditsExceeded || nonRetryableGuard.awfAPIProxyBlockingRequests || nonRetryableGuard.goalAlreadyActive) return false;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test exercises a local reimplementation of shouldRetry, not the production retry path — regression in codex_harness.cjs would not be caught here.

The shouldRetry at line 409 is an inline closure used only in this test block. The new check at line 416 correctly adds goalAlreadyActive, but the test verifies the local copy of the logic, not the actual detectNonRetryableHarnessGuard+break path inside main() in codex_harness.cjs. If the production guard on line 442 of that file were accidentally removed, all tests in this describe block would still pass.

💡 Why this matters

The local shouldRetry also omits the aiCreditsExceeded and awfAPIProxyBlockingRequests checks that are in production, confirming it has always been a simplified proxy. The harness_retry_guard.test.cjs tests do cover detection in isolation, but no test directly exercises the codex_harness.cjs retry guard end-to-end for the goalAlreadyActive signal.

Consider adding a lightweight integration test (similar to the noop pre-flight tests below this block) that runs the actual harness stub with an output containing the active-goal error and asserts it does not retry.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. Updated shouldRetry to mirror the full production guard (aiCreditsExceeded || awfAPIProxyBlockingRequests || goalAlreadyActive), so removing any of those guards from codex_harness.cjs would break the test.

const isTransient = RATE_LIMIT_ERROR_PATTERN.test(result.output) || SERVER_ERROR_PATTERN.test(result.output);
return attempt < MAX_RETRIES && (result.hasOutput || isTransient);
}
Expand Down Expand Up @@ -461,6 +463,15 @@ env_key = "OPENAI_API_KEY"
const result = { exitCode: 1, hasOutput: true, output: "permission denied\npermission denied\npermission denied" };
expect(shouldRetry(result, 0)).toBe(false);
});

it("does not retry when codex reports an existing active goal", () => {
const result = {
exitCode: 1,
hasOutput: true,
output: "cannot create a new goal because this thread already has a goal; use update_goal only when the existing goal is complete",
};
expect(shouldRetry(result, 0)).toBe(false);
});
});

describe("noop pre-flight and retry guard", () => {
Expand Down
5 changes: 4 additions & 1 deletion actions/setup/js/harness_retry_guard.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -5,17 +5,19 @@
const AI_CREDITS_EXCEEDED_PATTERNS = [/\bmax[\s_-]*ai[\s_-]*credits[\s_-]*exceeded\b/i, /\bai[\s_-]*credits[\s_-]*rate[\s_-]*limit[\s_-]*error\b/i, /ai[\s_-]*credits?.*(?:rate[\s-]*limit|limit exceeded|budget exceeded|exceeded)/i];

const AWF_API_PROXY_BLOCKING_REQUESTS_PATTERNS = [/\bawf\b.*\bapi[\s_-]*proxy\b.*\bblocking requests\b/i, /\bapi[\s_-]*proxy\b.*\bblocking requests\b/i, /\bapi[\s_-]*proxy\b.*\bblocked requests?\b/i, /\bDIFC_FILTERED\b/];
const GOAL_ALREADY_ACTIVE_PATTERNS = [/\bthis thread already has a goal\b.*\buse update_goal only when the existing goal is complete\b/i];

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. Replaced .* with [\s\S]*? so the pattern matches across newlines, and simplified the second anchor to \buse update_goal\b for flexibility. A dedicated multiline test confirms the behavior.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/diagnose] GOAL_ALREADY_ACTIVE_PATTERNS is more brittle than its sibling arrays. The two-phrase .* requirement means the guard silently fails if: (a) the error spans multiple lines (.* does not match \n), or (b) Codex rephrases either clause. Silent failure here restores the exact token-burn regression this PR is fixing.

💡 Suggestions

Option A — anchor on the stable first phrase only:

const GOAL_ALREADY_ACTIVE_PATTERNS = [
  /\bthis thread already has a goal\b/i,
];

The first phrase is already specific enough to avoid false positives and needs no secondary anchor.

Option B — make .* newline-safe (dot-all) and add a shorter fallback:

const GOAL_ALREADY_ACTIVE_PATTERNS = [
  /\bthis thread already has a goal\b[\s\S]*\buse update_goal\b/i, // full message, any whitespace
  /\bthis thread already has a goal\b/i,                          // fallback for partial output
];

For comparison, AI_CREDITS_EXCEEDED_PATTERNS uses 3 patterns to handle variations; GOAL_ALREADY_ACTIVE_PATTERNS is the only array with a single two-phrase requirement.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. Used Option B ([\s\S]*?) — the pattern now matches across newlines and uses \buse update_goal\b as the second anchor, bringing it in line with the multi-alternative approach of sibling patterns.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regex is a single compound match with no fallback — any message change silently disables the guard.

Unlike AI_CREDITS_EXCEEDED_PATTERNS (3 alternatives) and AWF_API_PROXY_BLOCKING_REQUESTS_PATTERNS (4 alternatives), GOAL_ALREADY_ACTIVE_PATTERNS has one pattern that requires both phrases to appear in order on the same line. The .* quantifier does not cross newlines, so a multi-line output or a slight rewording of the codex error defeats it silently. The failure mode is invisible: the guard misses, retries continue against a permanently-broken state, and credits are wasted.

💡 Suggested fix

Add at least one shorter fallback so minor message variations are still caught:

const GOAL_ALREADY_ACTIVE_PATTERNS = [
  // prefer the full compound message for precision
  /\bthis thread already has a goal\b.*\buse update_goal only when the existing goal is complete\b/i,
  // fallback: catches rewording or multi-line output
  /\bthis thread already has a goal\b/i,
];

Alternatively, add the s (dotAll) flag to the compound regex so it can match across newlines: /.../is.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. Replaced the single compound .* regex with [\s\S]*? so it matches across newlines. The two-phrase requirement is preserved for precision, with the negative test documenting that the first phrase alone is insufficient.


/**
* Detect retry guard conditions that should stop harness retries immediately.
* @param {unknown} output
* @returns {{ aiCreditsExceeded: boolean, awfAPIProxyBlockingRequests: boolean }}
* @returns {{ aiCreditsExceeded: boolean, awfAPIProxyBlockingRequests: boolean, goalAlreadyActive: boolean }}
*/
function detectNonRetryableHarnessGuard(output) {
const safeOutput = typeof output === "string" ? output : "";
return {
aiCreditsExceeded: AI_CREDITS_EXCEEDED_PATTERNS.some(pattern => pattern.test(safeOutput)),
awfAPIProxyBlockingRequests: AWF_API_PROXY_BLOCKING_REQUESTS_PATTERNS.some(pattern => pattern.test(safeOutput)),
goalAlreadyActive: GOAL_ALREADY_ACTIVE_PATTERNS.some(pattern => pattern.test(safeOutput)),
};
}

Expand All @@ -24,5 +26,6 @@ if (typeof module !== "undefined" && module.exports) {
detectNonRetryableHarnessGuard,
AI_CREDITS_EXCEEDED_PATTERNS,
AWF_API_PROXY_BLOCKING_REQUESTS_PATTERNS,
GOAL_ALREADY_ACTIVE_PATTERNS,
};
}
8 changes: 8 additions & 0 deletions actions/setup/js/harness_retry_guard.test.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -65,5 +65,13 @@ describe("harness_retry_guard.cjs", () => {
const result = detectNonRetryableHarnessGuard("transient network timeout");
expect(result.aiCreditsExceeded).toBe(false);
expect(result.awfAPIProxyBlockingRequests).toBe(false);
expect(result.goalAlreadyActive).toBe(false);
});

it("detects goal already active markers", () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] The new test only validates the exact canonical message. Consider adding a negative case that confirms a partial first-phrase match alone does NOT trigger detection — this would make the two-phrase requirement an explicit, documented contract rather than an implicit one.

💡 Suggested additions
it("does not detect goal active from first phrase alone", () => {
  const result = detectNonRetryableHarnessGuard("this thread already has a goal");
  expect(result.goalAlreadyActive).toBe(false); // adjust to true if single-phrase is desired
});

it("detects goal already active when embedded in longer output", () => {
  const result = detectNonRetryableHarnessGuard(
    "[codex] cannot create a new goal because this thread already has a goal; use update_goal only when the existing goal is complete\nExit code: 1"
  );
  expect(result.goalAlreadyActive).toBe(true);
});

Also: the existing null input test (line 52) and "both flags" test (line 58) were not updated to assert goalAlreadyActive: false — worth adding for completeness.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed. Added: (1) negative case asserting the first phrase alone does not trigger detection, (2) embedded-output multiline test, and (3) goalAlreadyActive: false assertions in the null input and both flags tests.

const result = detectNonRetryableHarnessGuard("cannot create a new goal because this thread already has a goal; use update_goal only when the existing goal is complete");
expect(result.aiCreditsExceeded).toBe(false);
expect(result.awfAPIProxyBlockingRequests).toBe(false);
expect(result.goalAlreadyActive).toBe(true);
});
});
Loading