Count generated jobs in the daily AIC guardrail by Copilot · Pull Request #66298 · github/gh-aw

Copilot · 2026-10-06T22:15:15Z

The daily AIC guardrail treated completed Agent and Detection jobs as having no billable usage because it matched only lowercase job names. It also reused the resulting zero-credit scan entries, allowing runs to start despite exceeding the daily limit.

Accounting: Map generated job display names to canonical component keys before checking coverage and reading usage.
Cache: Bump the scan coverage version so affected recorded-zero entries are re-inspected.
Regression coverage: Check generated job names, missing accounting, and reinspection of old zero-credit entries.

Fixes Daily AIC guardrail counts completed v0.91.2 runs as zero #66293

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

pelikhan · 2026-10-06T23:00:49Z

@copilot consider using the job id instead of name if available.

Copilot

Copilot review overview

🟡 Changes recommended

The generated Evaluations job remains unmapped, allowing its usage to be omitted and cached.

Review effort: Balanced
Findings: 1

Open (1)

Map Evaluations job name to evals component key · New

What changed in this PR

Fixes daily AIC accounting for generated job display names and invalidates stale scan entries.

Changes:

Normalizes Agent and Detection job names.
Bumps cache coverage version and adds regression tests.

File	Description
`actions/setup/js/daily_aic_component_coverage.cjs`	Normalizes billable job names.
`actions/setup/js/daily_aic_component_coverage.test.cjs`	Tests generated display names.
`actions/setup/js/daily_aic_cache_helpers.cjs`	Invalidates old coverage entries.
`actions/setup/js/daily_aic_scan.test.cjs`	Tests stale-zero reinspection.

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Copilot · 2026-10-06T23:52:57Z

+      const name = typeof job.name === "string" ? job.name.toLowerCase() : "";
+      if (!Object.hasOwn(COMPONENT_FILES, name)) continue;


Already addressed in 8d273c1: the generated Evaluations name maps to evals, with regression coverage for the generated name. The job-ID preference was added in the same commit, and the changes are preserved after merging main in cbe472d.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot · 2026-10-06T23:11:04Z

@copilot consider using the job id instead of name if available.

Done in 8d273c1. Component lookup now prefers job.job_id when the job object has one. It falls back to the display name, and Evaluations now maps to evals. The GitHub jobs API usually doesn't return the workflow job key, so the name fallback is still what runs in most cases.

gh-aw-bot · 2026-10-06T23:39:49Z

@copilot address the following outstanding work in one pass:

Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.
Review (actions/setup/js/daily_aic_component_coverage.cjs:32): The generated eval job is named Evaluations (pkg/workflow/compiler_job_annotations.go:17), so lowercasing yields evaluations, which is not the evals key in COMPONENT_FILES. Runs with evaluations will still omit that component's AIC, and the incomplete total can now be cached under coverage version 2. Map Evaluations to evals and add it to the generated-name regression case. - Count generated jobs in the daily AIC guardrail #66298 (comment)

Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI.

Sous-chef head: 8d273c1
Sous-chef work: 5661ea79c99caf633917906455c9b341e56372c5f328c4721fd1b650ebb0e8ab
Sous-chef state: 71c17a0f993d699e657980f42dd782e3e627106b56574ce2398749f00606b74b

Generated by 👨‍🍳 PR Sous Chef · pi · haiku45 · 8.96 AIC · ⌖ 12.3 AIC · ⊞ 1K · ◷
_{Comment /souschef to run again}

…-guardrail-counts Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

gh-aw-bot · 2026-10-07T00:56:43Z

@copilot address the following outstanding work in one pass:

Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.
Fix failing check JS Tests (shard 3/4) (FAILURE): https://github.com/github/gh-aw/actions/runs/37548963477/job/112572868708.

Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI.

Sous-chef head: cbe472d
Sous-chef work: 99fc195661b90dc631a5960c0642aa21aa4812000fe92377f25629fbdc6e618e
Sous-chef state: c2fa7acccf9a2a31f15a511dbb5a041b5b65ca11007ad484d7408961920d0168

Generated by 👨‍🍳 PR Sous Chef · pi · haiku45 · 10.6 AIC · ⌖ 6.06 AIC · ⊞ 1K · ◷
_{Comment /souschef to run again}

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot · 2026-10-07T01:10:29Z

@copilot address the following outstanding work in one pass:

Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes....

make merge-main succeeded; the branch already included the fetched latest main. The Evaluations → evals mapping and generated-name regression test were already present in 8d273c1 and remain intact. The shard failure is fixed in 202d80b by making the weighted-distribution test deterministic.

@copilot address the following outstanding work in one pass:

Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes....

make merge-main succeeded; the branch already included the fetched latest main. Fixed the flaky weighted-distribution test in 202d80b with deterministic samples; the focused test and impacted JavaScript suite pass.

github-actions · 2026-10-07T20:32:33Z

🎉 This pull request is included in a new release.

Release: v0.91.5

Initial plan

049f6e7

Copilot AI assigned Copilot and pelikhan Oct 6, 2026

Copilot started work on behalf of pelikhan October 6, 2026 22:15 View session

Copilot AI linked an issue Oct 6, 2026 that may be closed by this pull request

Daily AIC guardrail counts completed v0.91.2 runs as zero #66293

Closed

Count displayed daily AIC component jobs and invalidate stale scans

a4551e0

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Fix daily AI Credits guardrail counts after upgrade to v0.91.2~~ Count generated jobs in the daily AIC guardrail Oct 6, 2026

Copilot finished work on behalf of pelikhan October 6, 2026 22:25

Copilot AI requested a review from pelikhan October 6, 2026 22:25

pelikhan marked this pull request as ready for review October 6, 2026 23:00

Copilot AI balanced review requested due to automatic review settings October 6, 2026 23:00

Copilot started reviewing on behalf of pelikhan October 6, 2026 23:01 View session

Copilot AI reviewed Oct 6, 2026

View reviewed changes

Copilot started work on behalf of pelikhan October 6, 2026 23:06 View session

Prefer job_id and map Evaluations in daily AIC component lookup

8d273c1

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot finished work on behalf of pelikhan October 6, 2026 23:11

github-actions Bot mentioned this pull request Oct 6, 2026

[pr-sous-chef] PR Sous Chef — #65979 nudged #65986

Closed

Copilot started work on behalf of gh-aw-bot October 6, 2026 23:40 View session

Merge remote-tracking branch 'origin/main' into copilot/fix-daily-aic…

cbe472d

…-guardrail-counts Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot finished work on behalf of gh-aw-bot October 6, 2026 23:54

Copilot AI requested a review from gh-aw-bot October 6, 2026 23:54

github-actions Bot mentioned this pull request Oct 7, 2026

[pr-sous-chef] 🍳 PR Sous Chef Summary: 4 PRs nudged toward completion #66387

Closed

Copilot started work on behalf of gh-aw-bot October 7, 2026 00:57 View session

Make weighted variant test deterministic

202d80b

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot finished work on behalf of gh-aw-bot October 7, 2026 01:11

github-actions Bot mentioned this pull request Oct 7, 2026

[PR Triage Report] 2026-10-07 Copilot PRs #66477

Closed

pelikhan merged commit bc4077a into main Oct 7, 2026
2 checks passed

pelikhan deleted the copilot/fix-daily-aic-guardrail-counts branch October 7, 2026 10:36

Navigation Menu

Repository navigation

Count generated jobs in the daily AIC guardrail - #66298

Count generated jobs in the daily AIC guardrail#66298
pelikhan merged 5 commits into
mainfrom
copilot/fix-daily-aic-guardrail-counts

Copilot AI commented Oct 6, 2026 •

edited

Loading

Uh oh!

pelikhan commented Oct 6, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Oct 6, 2026

Uh oh!

Copilot AI commented Oct 6, 2026

Uh oh!

gh-aw-bot commented Oct 6, 2026

Uh oh!

gh-aw-bot commented Oct 7, 2026

Uh oh!

Copilot AI commented Oct 7, 2026 •

edited

Loading

Uh oh!

Uh oh!

github-actions Bot commented Oct 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		const name = typeof job.name === "string" ? job.name.toLowerCase() : "";
		if (!Object.hasOwn(COMPONENT_FILES, name)) continue;

Conversation

Copilot AI commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pelikhan commented Oct 6, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Copilot AI Oct 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI commented Oct 6, 2026

Uh oh!

gh-aw-bot commented Oct 6, 2026

Uh oh!

gh-aw-bot commented Oct 7, 2026

Uh oh!

Copilot AI commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Oct 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Copilot AI commented Oct 6, 2026 •

edited

Loading

Copilot AI commented Oct 7, 2026 •

edited

Loading