Fix AOAI smoke issue targeting and clarify PR safe-output limits by Copilot · Pull Request #65409 · github/gh-aw

Copilot · 2026-10-03T19:08:43Z

The AOAI smoke workflow supplies a newly created issue’s temporary ID to set_issue_type, but its implicit triggering target rejects that ID on scheduled runs. Separately, the PR safe-output reference leaves the run-level quota and same-run follow-up push behavior unclear.

Issue targeting: Set set-issue-type.target: "*" so the smoke workflow can target the issue it created. Add a regression assertion for the source and compiled configuration.
PR limits: Clarify that create-pull-request.max is shared across all agents in a workflow run, and that a PR created in the current agent run cannot receive a push_to_pull_request_branch call before it exists remotely.

Fixes [AW Top 10] 08 Fix safe-output quotas and implicit targets #65379

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot

Copilot review overview

🟡 Changes recommended

The wildcard target needs tighter limits, and the regression assertion can miss handler-config drift.

Review effort: Balanced
Findings: 2

Open (2)

Constrain issue type handler to one Bug update · New
Scope assertion to handler configuration variable · New

What changed in this PR

Updates AOAI smoke-test issue targeting and clarifies pull-request safe-output limits.

Changes:

Enables temporary-ID issue-type targeting and updates the compiled workflow.
Adds regression coverage for source and compiled configuration.
Documents PR quotas and same-run push limitations.

File	Description
`.github/workflows/smoke-copilot-aoai-entra.md`	Enables explicit issue targeting.
`.github/workflows/smoke-copilot-aoai-entra.lock.yml`	Regenerates the compiled workflow.
`pkg/cli/smoke_copilot_aoai_workflow_contract_test.go`	Adds configuration assertions.
`docs/src/content/docs/reference/safe-outputs-pull-requests.md`	Clarifies quotas and follow-up pushes.
`.github/skills/agentic-workflows/SKILL.md`	Reorganizes work-queue routing guidance.

    remove-labels:
      allowed: [smoke]
    set-issue-type:
+      target: "*"


+	assert.Contains(t, string(compiled), `\"set_issue_type\":{\"target\":\"*\"}`,
+		"Compiled safe-output handler must use the explicit issue target")


github-actions · 2026-10-03T19:46:39Z

✅ Ponytail Reviewer completed successfully!

Lean already. Ship.

Generated by Ponytail Reviewer for #65409

github-actions · 2026-10-03T19:46:40Z

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel skipped because pre-fetch PR data was unavailable: unable to fetch test file diff

🧪 Test quality analysis by Test Quality Sentinel

github-actions · 2026-10-03T19:46:46Z

✅ PR Code Quality Reviewer completed the code quality review.

PR review submitted via safeoutputs; no additional GitHub write beyond the review body/comments is needed if submission fails.

🔎 Code quality review by PR Code Quality Reviewer

github-actions · 2026-10-03T19:46:50Z

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

github-actions · 2026-10-03T19:47:15Z

✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft.

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

🏗️ ADR gate enforced by Design Decision Gate 🏗️

github-actions · 2026-10-03T19:50:58Z

Comment Memory

reviewed_at: 2026-10-03T19:48:01Z
review_event: REQUEST_CHANGES
top_themes:
  - wildcard set-issue-type target broadens issue write scope
  - lockfile assertion is not scoped to handler config
files_reviewed:
  - .github/skills/agentic-workflows/SKILL.md
  - .github/workflows/smoke-copilot-aoai-entra.md
  - .github/workflows/smoke-copilot-aoai-entra.lock.yml
  - docs/src/content/docs/reference/safe-outputs-pull-requests.md
  - pkg/cli/smoke_copilot_aoai_workflow_contract_test.go
comment_count: 0

Note

This comment is managed by comment memory.

It stores persistent context for this thread in the code block at the top of this comment.
Edit only the text inside the backtick fences; workflow metadata and the footer are regenerated automatically.

Learn more about comment memory

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt54 · 34.8 AIC · ⌖ 7.1 AIC · ⊞ 20.2K · ◷
_{Comment /review to run again}

github-actions

Request changes

Two blocking problems remain in the changed lines.

Blocking themes

set-issue-type.target: "*" fixes temporary-ID resolution, but it also widens the workflow from the triggering item to arbitrary repository issues. If wildcard targeting is unavoidable here, it needs compensating guardrails (max: 1 at minimum, ideally narrower validation) before this stops being a permissions regression.
TestSmokeCopilotAOAIEntraSetIssueTypeTarget still substring-matches the entire lock file for "set_issue_type":{"target":"*"}. That fragment can appear in GH_AW_SAFE_OUTPUTS_CONFIG, so the test still passes when the apply-time handler config is wrong. Scope the assertion to GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG or parse the emitted JSON instead of searching the whole file.

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt54 · 34.8 AIC · ⌖ 7.1 AIC · ⊞ 20.2K
_{Comment /review to run again}

github-actions

Skills-Based Review 🧠

Applied /diagnosing-bugs (issue targeting fix) and /grill-with-docs (safe-output limit clarifications). Small, well-scoped PR — the fix, the regression test, and the doc updates line up correctly against HEAD.

📋 Verification performed

Confirmed smoke-copilot-aoai-entra.md/.lock.yml at HEAD actually contain target: "*" / {"target":"*"}, so the new TestSmokeCopilotAOAIEntraSetIssueTypeTarget test passes cleanly (verified via git stash + go test ./pkg/cli/ -run TestSmokeCopilotAOAIEntraSetIssueTypeTarget).
Confirmed target: "*" on set-issue-type is an established pattern already used across dozens of other workflows in this repo (ai-moderator.md, bot-detection.md, etc.), so this isn't introducing a new class of risk — it's consistent with existing convention for temporary-ID-driven safe outputs.
The SKILL.md diff is a no-op reordering (alphabetizing the file list and routing bullet) — no functional change.
Doc wording for create-pull-request.max and the push-to-pull-request-branch same-run caveat reads correctly and matches the staged/post-agent processing model described elsewhere in the same file.

Positive Highlights

✅ Root cause fix (missing target: "*") is paired with a regression test, not just the markdown change — good /diagnosing-bugs discipline.
✅ Doc clarifications are precise and address a real source of confusion (shared-quota semantics, same-run push ordering) rather than vague wording.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · sonnet50 · 74.1 AIC · ⌖ 14.3 AIC · ⊞ 10K
_{Comment /matt to run again}

github-actions

Review summary

Focused review using the refactor_cleanup/mixed_unclear Impeccable lenses (doc + config tidy-up PR touching workflow frontmatter, docs, SKILL.md, and a Go test).

Verified as correct:

set-issue-type.target: "*" is present in both the source .md and the compiled .lock.yml (GH_AW_SAFE_OUTPUTS_CONFIG and GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG), and gh aw compile reproduces the committed lock file byte-for-byte — no drift.
The new regression test TestSmokeCopilotAOAIEntraSetIssueTypeTarget passes against the PR head.
The push_to_pull_request_branch doc claim (temporary_id doesn't help target a same-run PR) matches the implementation — push_to_pull_request_branch.cjs has no temporary-ID/issue-target resolution path, unlike set_issue_type.cjs.

New finding (posted inline): the .github/skills/agentic-workflows/SKILL.md diff isn't just an alphabetical reorder — it deletes the "Orchestrate durable work with a work queue..." routing bullet that tells agents when to load work-queue.md. That's unrelated to this PR's stated scope and looks like an accidental drop.

Two pre-existing Copilot review comments (wildcard target: "*" broadening issue-type write scope to any issue in the repo, and the new test assertion matching both GH_AW_SAFE_OUTPUTS_CONFIG/GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG without distinguishing them) are still open and still valid — not duplicated here.

🧵 Reviewed using Impeccable skills by Impeccable Skills Reviewer · copilot · sonnet50 · 164.4 AIC · ⌖ 13.2 AIC · ⊞ 8.1K

github-actions · 2026-10-03T19:59:31Z

@@ -106,7 +106,6 @@ After loading the matching workflow prompt or skill, follow it directly:
 - Render compact markdown charts: `.github/aw/asciicharts.md`


This reorder (moving work-queue.md into alphabetical position in the "Load these files" list) silently drops the corresponding routing bullet two lines below: - Orchestrate durable work with a work queue (dispatcher/worker roles or queue inspection): \.github/aw/work-queue.md`. That line is deleted entirely in this diff and not reintroduced anywhere else, so agents loading this skill lose the only pointer that tells them when to read work-queue.md`. This looks unintentional and unrelated to the PR's stated purpose (AOAI smoke fix / PR safe-output docs). Please restore the deleted routing bullet (or confirm the removal is intentional and explain why).

@copilot please address this.

github-actions · 2026-10-05T23:09:42Z

🎉 This pull request is included in a new release.

Release: v0.91.0

Initial plan

a999efd

Copilot AI assigned Copilot and pelikhan Oct 3, 2026

Copilot AI linked an issue Oct 3, 2026 that may be closed by this pull request

[AW Top 10] 08 Fix safe-output quotas and implicit targets #65379

Closed

1 task

Copilot started work on behalf of pelikhan October 3, 2026 19:08

View session

Copilot AI and others added 2 commits October 3, 2026 19:20

Fix AOAI smoke issue target and document PR safe-output limits

01b50bc

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Apply remaining changes

c34d5f9

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Fix safe output quotas and implicit targets~~ Fix AOAI smoke issue targeting and clarify PR safe-output limits Oct 3, 2026

Copilot finished work on behalf of pelikhan October 3, 2026 19:25

Copilot AI requested a review from pelikhan October 3, 2026 19:25

pelikhan marked this pull request as ready for review October 3, 2026 19:32

Copilot AI balanced review requested due to automatic review settings October 3, 2026 19:32

Copilot started reviewing on behalf of pelikhan October 3, 2026 19:32

View session

Copilot AI reviewed Oct 3, 2026

View reviewed changes

github-actions Bot requested changes Oct 3, 2026

View reviewed changes

github-actions Bot mentioned this pull request Oct 3, 2026

[aw] Design Decision Gate 🏗️ produced no safe outputs #65363

Closed

github-actions Bot added the needs-review label Oct 3, 2026

github-actions Bot approved these changes Oct 3, 2026

View reviewed changes

github-actions Bot reviewed Oct 3, 2026

View reviewed changes

pelikhan merged commit 3bf82a8 into main Oct 3, 2026
106 of 108 checks passed

pelikhan deleted the copilot/aw-top-10-08-fix-safe-output-quotas branch October 3, 2026 20:42

github-actions Bot mentioned this pull request Oct 4, 2026

Suggested test: copilot/create-pull-request-then-push-same-run-rejected githubnext/gh-aw-test#19914

Closed

Navigation Menu

Repository navigation

Fix AOAI smoke issue targeting and clarify PR safe-output limits - #65409

Fix AOAI smoke issue targeting and clarify PR safe-output limits#65409
pelikhan merged 3 commits into
mainfrom
copilot/aw-top-10-08-fix-safe-output-quotas

Copilot AI commented Oct 3, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

github-actions Bot commented Oct 3, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 3, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 3, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 3, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 3, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 3, 2026

Uh oh!

github-actions Bot left a comment

Uh oh!

github-actions Bot left a comment

Uh oh!

github-actions Bot left a comment

Uh oh!

github-actions Bot Oct 3, 2026

Uh oh!

Uh oh!

github-actions Bot commented Oct 5, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		assert.Contains(t, string(compiled), `\"set_issue_type\":{\"target\":\"*\"}`,
		"Compiled safe-output handler must use the explicit issue target")

		@@ -106,7 +106,6 @@ After loading the matching workflow prompt or skill, follow it directly:
		- Render compact markdown charts: `.github/aw/asciicharts.md`

Conversation

Copilot AI commented Oct 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

github-actions Bot commented Oct 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 3, 2026

Comment Memory

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Request changes

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Skills-Based Review 🧠

Positive Highlights

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Review summary

Uh oh!

github-actions Bot Oct 3, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions Bot commented Oct 5, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Oct 3, 2026 •

edited

Loading

github-actions Bot commented Oct 3, 2026 •

edited

Loading

github-actions Bot commented Oct 3, 2026 •

edited

Loading

github-actions Bot commented Oct 3, 2026 •

edited

Loading

github-actions Bot commented Oct 3, 2026 •

edited

Loading

github-actions Bot commented Oct 3, 2026 •

edited

Loading