Skip to content

[deep-report] Fix fleet-wide 'Redact secrets in logs' step failure (ERR_VALIDATION, 4 workflows, 3 engines) #66880

Description

@github-actions

Description

The "Redact secrets in logs" post-execution step is failing fleet-wide with ERR_VALIDATION: Failed to remove artifact source after secret redaction failed. Confirmed across 3 different agentic engines (codex, copilot, claude) and 4 workflows on 2026-10-07: Daily AWF Spec Compiler Surfacing Review, Daily Ambient Context Optimizer, Daily Caveman Optimizer, Deep Report. It is new as of 2026-10-07 (absent from the 2026-10-06 Agent Job Health report) and was the single largest failure cluster that day (33% of all agent-job failures, 4/12).

In each failing run, redact_secrets.cjs logs warnings for replacing several non-writable files in place (e.g. /tmp/gh-aw/sandbox/firewall/audit/awf-resolved-config.json, docker-compose.redacted.yml, image-manifest.json, policy-manifest.json, sensitive-paths.json, or /tmp/gh-aw/mcp-logs/safeoutputs/server.log), then a subsequent step tries to remove the original artifact source and fails. The agent's own execution completes successfully in all 4 cases — only this post-processing cleanup step fails, but it still fails the whole job.

Expected Impact

Stops a fleet-wide, engine-agnostic post-processing failure that is currently turning otherwise-successful agent runs into failed jobs. Representative run: https://github.com/github/gh-aw/actions/runs/37695677725 (also https://github.com/github/gh-aw/actions/runs/37676391386).

Suggested Fix Area

Likely in the secret-redaction/artifact-cleanup pipeline (redact_secrets.cjs or the artifact-source removal step) — make artifact-source removal tolerant of files it just rewrote in place (permission/ownership mismatch after in-place replacement, or a race with firewall-sandbox teardown).

Suggested Agent

PR Sous Chef / general Go or Node fix agent.

Estimated Effort

Medium (root cause needs isolating; fix itself is likely small).

Data Source

DeepReport Intelligence Briefing analysis, 2026-10-08. Source: discussion "[agent-job-health] Agent Job Health: 2026-10-07 daily report" (#66705). Live-verified: no matching open issue found via search_issues for "redact secrets", "Failed to remove artifact source after secret redaction failed", or issues created 2026-10-07/08 mentioning "redact".

Generated by 🔬 Deep Report · claude · agent · 342.4 AIC · ⌖ 10.9 AIC · ⊞ 7.1K · ◷

  • expires on Oct 10, 2026, 4:53 AM UTC-08:00

Activity

  1. github-actions commented on Oct 9, 2026

    @github-actions
    ContributorAuthor

    🍪 Issue Monster selected this for Copilot

    Ive identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

    If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

    Om nom nom! 🍪

    🍪 Om nom nom by Issue Monster · pi · gpt54 · 9.79 AIC · ⌖ 7.2 AIC · ⊞ 14K · ◷

  2. github-actions commented on Oct 9, 2026

    @github-actions
    ContributorAuthor

    🍪 Issue Monster selected this for Copilot

    I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

    If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

    Om nom nom! 🍪

    🍪 Om nom nom by Issue Monster · pi · gpt54 · 18.9 AIC · ⌖ 7.06 AIC · ⊞ 14K · ◷

  3. github-actions commented on Oct 9, 2026

    @github-actions
    ContributorAuthor

    🍪 Issue Monster selected this for Copilot

    I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

    If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

    Om nom nom! 🍪

    🍪 Om nom nom by Issue Monster · pi · gpt54 · 11.2 AIC · ⌖ 10.8 AIC · ⊞ 14K · ◷

  4. github-actions commented on Oct 9, 2026

    @github-actions
    ContributorAuthor

    🍪 Issue Monster selected this for Copilot

    I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

    If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

    Om nom nom! 🍪

    🍪 Om nom nom by Issue Monster · pi · gpt54 · 8.97 AIC · ⌖ 8.98 AIC · ⊞ 14K · ◷

  5. github-actions commented on Oct 9, 2026

    @github-actions
    ContributorAuthor

    🍪 Issue Monster selected this for Copilot

    I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

    If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

    Om nom nom! 🍪

    🍪 Om nom nom by Issue Monster · pi · gpt54 · 10.6 AIC · ⌖ 9.81 AIC · ⊞ 14K · ◷

  6. github-actions commented on Oct 9, 2026

    @github-actions
    ContributorAuthor

    🍪 Issue Monster selected this for Copilot

    I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

    If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

    Om nom nom! 🍪

    🍪 Om nom nom by Issue Monster · pi · gpt54 · 12.9 AIC · ⌖ 13.8 AIC · ⊞ 14.1K · ◷

  7. github-actions commented on Oct 9, 2026

    @github-actions
    ContributorAuthor

    🍪 Issue Monster selected this for Copilot

    I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

    If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

    Om nom nom! 🍪

    🍪 Om nom nom by Issue Monster · pi · gpt54 · 10.7 AIC · ⌖ 12.2 AIC · ⊞ 14.1K · ◷

  8. github-actions commented on Oct 9, 2026

    @github-actions
    ContributorAuthor

    🍪 Issue Monster selected this for Copilot

    I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

    If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

    Om nom nom! 🍪

    🍪 Om nom nom by Issue Monster · pi · gpt54 · 10.3 AIC · ⌖ 8.42 AIC · ⊞ 14.4K · ◷

  9. github-actions commented on Oct 9, 2026

    @github-actions
    ContributorAuthor

    🍪 Issue Monster selected this for Copilot

    I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

    If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

    Om nom nom! 🍪

    🍪 Om nom nom by Issue Monster · pi · gpt54 · 22.3 AIC · ⌖ 7.16 AIC · ⊞ 14.3K · ◷

  10. github-actions commented on Oct 9, 2026

    @github-actions
    ContributorAuthor

    🍪 Issue Monster selected this for Copilot

    I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

    If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

    Om nom nom! 🍪

    🍪 Om nom nom by Issue Monster · pi · gpt54 · 19.9 AIC · ⌖ 8.99 AIC · ⊞ 14.3K · ◷

  11. github-actions commented on Oct 9, 2026

    @github-actions
    ContributorAuthor

    🍪 Issue Monster selected this for Copilot

    I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

    If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

    Om nom nom! 🍪

    🍪 Om nom nom by Issue Monster · pi · gpt54 · 11.6 AIC · ⌖ 8.57 AIC · ⊞ 14.3K · ◷

  12. github-actions commented on Oct 9, 2026

    @github-actions
    ContributorAuthor

    🍪 Issue Monster selected this for Copilot

    I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

    If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

    Om nom nom! 🍪

    🍪 Om nom nom by Issue Monster · pi · gpt54 · 11.9 AIC · ⌖ 10.3 AIC · ⊞ 14.1K · ◷

  13. github-actions commented on Oct 9, 2026

    @github-actions
    ContributorAuthor

    🍪 Issue Monster selected this for Copilot

    I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

    If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

    Om nom nom! 🍪

    🍪 Om nom nom by Issue Monster · pi · gpt54 · 15.9 AIC · ⌖ 9.89 AIC · ⊞ 14.1K · ◷

  14. github-actions commented on Oct 9, 2026

    @github-actions
    ContributorAuthor

    🍪 Issue Monster selected this for Copilot

    I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

    If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

    Om nom nom! 🍪

    🍪 Om nom nom by Issue Monster · pi · gpt54 · 11.3 AIC · ⌖ 8.75 AIC · ⊞ 14.1K · ◷

  15. github-actions commented on Oct 10, 2026

    @github-actions
    ContributorAuthor

    🍪 Issue Monster selected this for Copilot

    I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

    If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

    Om nom nom! 🍪

    🍪 Om nom nom by Issue Monster · pi · gpt54 · 28.9 AIC · ⌖ 9.79 AIC · ⊞ 14.2K · ◷

  16. github-actions commented on Oct 10, 2026

    @github-actions
    ContributorAuthor

    🍪 Issue Monster selected this for Copilot

    I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

    If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

    Om nom nom! 🍪

    🍪 Om nom nom by Issue Monster · pi · gpt54 · 21.4 AIC · ⌖ 13.2 AIC · ⊞ 14.1K · ◷

  17. github-actions commented on Oct 10, 2026

    @github-actions
    ContributorAuthor

    🍪 Issue Monster selected this for Copilot

    I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

    If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

    Om nom nom! 🍪

    🍪 Om nom nom by Issue Monster · pi · gpt54 · 10 AIC · ⌖ 8.9 AIC · ⊞ 14K · ◷

  18. pelikhan commented on Oct 10, 2026

    @pelikhan
    Collaborator

    Duplicate/instance of #66818, now fixed. Root cause: PR #66160 made writable-file preparation in redact_secrets.cjs unconditional, causing the redaction step to fail on unchanged/secret-free files in read-only directories (e.g. firewall sandbox sandbox/firewall/audit/*), even though the agent step itself succeeded. Fixed by PR #66995 (merged 2026-10-08, building on #66160/#66698), which only rewrites files when redactions occurred or custom masking is configured. Confirmed present on main with passing regression tests. Closing in favor of #66818.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions