Improve diagnostics for missing terminal outputs by Copilot · Pull Request #66557 · github/gh-aw

Copilot · 2026-10-07T12:15:31Z

Missing terminal safe outputs and engine-driver failures currently provide too little context for maintainers to distinguish outages, rejected requests, and prompt exhaustion. Failure issues should include useful diagnostics without exposing potentially sensitive driver messages, and repeated failures should consolidate by workflow and cause.

Safe diagnostics: Include the driver exit code, generic engine error type, retry count/statuses, and classified cause. Raw driver messages are omitted.
Cause-based deduplication: Reuse an open failure issue for the same workflow and classified cause, regardless of other changing failure categories.

Example:

Failure classification: engine_outage
Driver exit code: 1
Last engine error type: capi_server_error
Retry attempts observed: 2 (HTTP 502, HTTP 503)

Fixes [AW Top 10] 01 Improve diagnostics for missing terminal outputs #66485

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

pelikhan · 2026-10-07T13:56:36Z

@copilot resolve the merge conflicts in this pull request

…-diagnostics # Conflicts: # actions/setup/js/handle_agent_failure.cjs Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot · 2026-10-07T14:08:43Z

@copilot resolve the merge conflicts in this pull request

Merged main and resolved the conflict in handle_agent_failure.cjs, preserving both changes. Merge commit: 98fcaef28c.

Copilot · 2026-10-07T16:23:02Z

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Install Go language server (gopls)

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

Copilot

🟡 Changes recommended

Structured retries and engine errors can be omitted, diagnostics may be truncated, and deduplication remains limited to 24 hours.

4 open findings

Derive engine error type from canonical execution metadata · New
Detect Claude retries from structured claude.system events · New
Prioritize required diagnostics before limiting output · New
Deduplicate failures across all open matching issues · New

What changed in this PR

Improves missing-terminal-output failure issues with safe diagnostics and cause-based grouping.

Changes:

Classifies failures and records exit code, engine error type, and retries.
Propagates validated collector metadata into failure handling.
Deduplicates issues by workflow and classified cause.

File	Description
`actions/setup/js/load_agent_output.cjs`	Validates collector diagnostics metadata.
`actions/setup/js/load_agent_output.test.cjs`	Tests metadata preservation.
`actions/setup/js/handle_agent_failure.cjs`	Adds cause-based titles, matching, and log suppression.
`actions/setup/js/handle_agent_failure.test.cjs`	Tests titles, reuse, and suppression.
`actions/setup/js/empty_output_outcome.cjs`	Classifies failures and builds safe diagnostics.
`actions/setup/js/empty_output_outcome.test.cjs`	Tests classification and diagnostic extraction.
`actions/setup/js/collect_ndjson_output.cjs`	Persists collector diagnostics.
`actions/setup/js/collect_ndjson_output.test.cjs`	Updates collected-output expectations.

🧠 Review effort: Balanced

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

  for (const command of extractDeniedCommands(attributedDiagnostics)) diagnostics.add(`Permission denied: ${command}`);
  const engineSummary = agentErrorSummaryText(safeStdio);
  if (engineSummary) diagnostics.add(engineSummary);
+  const engineErrorType = engineSummary.match(/\(([a-z][a-z0-9_]*)\)/)?.[1] || "";


+  const isRequestRejection = ["invalid_safe_outputs", "safeoutputs_cli_error"].includes(reason) || executionCategories.some(category => REQUEST_REJECTION_ERROR_CATEGORIES.has(category)) || errorCodes.some(code => code >= 400 && code < 500);
+  const isEngineOutage = reason === "engine_driver_failure" || executionCategories.some(category => ["agentic_engine_timeout", "capi_server_error", "sandbox_runtime_crash"].includes(category)) || errorCodes.some(code => code >= 500);
+  const failureCause = isPromptExhaustion ? "prompt_exhaustion" : isRequestRejection ? "request_rejection" : isEngineOutage ? "engine_outage" : "prompt_exhaustion";
+  const retryEvents = events.filter(event => event.type === "claude.api_retry" || (event.type === "system" && event.data.subtype === "api_retry"));


+  diagnostics.add(`Failure classification: ${failureCause}`);
+  if (engineErrorType) {
+    diagnostics.add(`Last engine error type: ${engineErrorType}`);
+  } else if (reason === "engine_driver_failure") {
+    diagnostics.add("Last engine error type: unknown");
+  }
+  diagnostics.add(`Retry attempts observed: ${retryCount}${retryStatusCodes.length ? ` (HTTP ${retryStatusCodes.join(", HTTP ")})` : ""}`);
  const details = [...diagnostics].slice(0, 20).join("\n");


+  const failureCauseQuery = typeof failureCause === "string" ? ` "failure_cause: ${failureCause}"` : "";
+  const searchQuery = `repo:${owner}/${repo} is:issue is:open label:agentic-workflows created:>=${since} ` + `"gh-aw-agentic-workflow:" "workflow_id: ${escapedWorkflowId}"${failureCauseQuery} in:body`;


github-actions · 2026-10-07T20:31:55Z

🎉 This pull request is included in a new release.

Release: v0.91.5

Initial plan

d229c55

Copilot AI assigned Copilot and pelikhan Oct 7, 2026

Copilot started work on behalf of pelikhan October 7, 2026 12:35 View session

Copilot AI linked an issue Oct 7, 2026 that may be closed by this pull request

[AW Top 10] 01 Improve diagnostics for missing terminal outputs #66485

Closed

2 tasks

Copilot AI and others added 3 commits October 7, 2026 12:56

Add terminal output failure diagnostics

09f0d74

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Refine terminal failure classification

324f3f6

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Type terminal failure diagnostics

b0636ed

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Improve diagnostics for missing terminal outputs~~ Improve diagnostics for missing terminal outputs Oct 7, 2026

Copilot finished work on behalf of pelikhan October 7, 2026 13:01

Copilot AI requested a review from pelikhan October 7, 2026 13:01

Copilot started work on behalf of pelikhan October 7, 2026 13:57 View session

Merge remote-tracking branch 'origin/main' into copilot/aw-10-improve…

98fcaef

…-diagnostics # Conflicts: # actions/setup/js/handle_agent_failure.cjs Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot finished work on behalf of pelikhan October 7, 2026 14:09

pelikhan marked this pull request as ready for review October 7, 2026 16:12

Copilot AI balanced review requested due to automatic review settings October 7, 2026 16:12

Copilot started reviewing on behalf of pelikhan October 7, 2026 16:13 View session

Copilot AI reviewed Oct 7, 2026

View reviewed changes

pelikhan merged commit 60924f7 into main Oct 7, 2026
12 of 13 checks passed

pelikhan deleted the copilot/aw-10-improve-diagnostics branch October 7, 2026 16:43

Navigation Menu

Repository navigation

Improve diagnostics for missing terminal outputs - #66557

Improve diagnostics for missing terminal outputs#66557
pelikhan merged 5 commits into
mainfrom
copilot/aw-10-improve-diagnostics

Copilot AI commented Oct 7, 2026 •

edited

Loading

Uh oh!

pelikhan commented Oct 7, 2026

Uh oh!

Copilot AI commented Oct 7, 2026

Uh oh!

Copilot AI commented Oct 7, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

github-actions Bot commented Oct 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		const failureCauseQuery = typeof failureCause === "string" ? ` "failure_cause: ${failureCause}"` : "";
		const searchQuery = `repo:${owner}/${repo} is:issue is:open label:agentic-workflows created:>=${since} ` + `"gh-aw-agentic-workflow:" "workflow_id: ${escapedWorkflowId}"${failureCauseQuery} in:body`;

Conversation

Copilot AI commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pelikhan commented Oct 7, 2026

Uh oh!

Copilot AI commented Oct 7, 2026

Uh oh!

Copilot AI commented Oct 7, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

github-actions Bot commented Oct 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Oct 7, 2026 •

edited

Loading