Seed ESLint factory work and allow experimental dry-run notices by pelikhan · Pull Request #67023 · github/gh-aw

pelikhan · 2026-10-08T23:26:37Z

The ESLint factory dispatcher had no recurring producer and could report an uninitialized queue as an empty backlog. Make it admit a bounded daily cohort before scheduling, while preserving native queue authority and administrator-controlled deployment.

Changes

Prepare three independent, UTC-date-keyed tasks for miner, refiner, and monster. Same-day admissions are idempotent; scheduling remains bounded by installed Policy, producer entitlements, immutable worker profiles, and repository-scoped output contracts.
Provide a validated Policy template generator without automatically installing Policy. Distinguish missing deployment from an empty queue, and cancel write-capable Claims for benign no-effect outcomes instead of stranding delivery. Explicit no-write assignments retain completed/noop behavior.
Fix remote workflow validation to use paginated, host-aware discovery rather than the default 50-workflow listing limit.
Add explicit compile --allow-experimental acknowledgment for built-in experimental-feature notices. Preserve warning counts and visible reporting; other warnings, security validation, model diagnostics, and scanner failures remain fatal.
Use actual Git-read evidence for hosted queue visibility rather than installation-token collaborator flags. Preserve repository identity validation, API denials, and the proof required before classifying a queue ref as absent.
Update Linter Factory docs, queue/debugging instructions, formal-model boundaries, and release changesets.
Integrate current main and preserve concurrent Copilot upgrades to Go 1.26.9 and golang.org/x/net v0.60.0. Regenerate the stale frontmatter reference and assert both work-queue: true and work-queue: null instead of masking the missing boolean example.

Architecture decision

ADR: ADR-67023: Acknowledge Experimental-Feature Notices via an Explicit Compile Flag.

The completed decision record covers operator-requested acknowledgment, fixed-table classification, synchronized counters, independent fatal diagnostics, alternatives, and regression evidence. Its status remains Proposed pending maintainer acceptance.

Validation

Formatting and standard make lint passed on the combined branch.
Rebuilt local ./gh-aw and recompiled all 333 normal workflows after both integrations.
Reproduced the failing CI documentation assertion on the merged tree, regenerated the reference with the existing generator, and passed its original assertion without weakening it. The schema-documentation generator tests also passed.
Re-ran the exact failing JavaScript shard: 138 files and 3,651 tests passed with Actions workspace context and canonical temporary paths. Focused factory/queue suites, compiler/CLI regressions, schema freshness, and impacted Go/JavaScript tests passed.
GOTOOLCHAIN=go1.26.9 make security-govulncheck passed: zero reachable vulnerabilities and zero vulnerable imported packages. One advisory in a required module is informational because no affected code path was found.
Earlier four-workflow dry-run with local ./gh-aw compile ... --dry-run --allow-experimental --json passed with four valid workflow results, three acknowledged notices, and shellcheck coverage. Committed locks are normal production compilations, not diagnostic locks.
Source/lock reviews found no confirmed regression. Optional Docker scanners were not run. Broader inherited NVX host-preflight fixture checks did not pass with this macOS host's system tools; that check is not claimed as passing.

Known baseline failure: final make agent-report-progress still exits nonzero on unchanged custom Go lint findings involving diagnostic-print error handling, function lengths, and existing slice indexing. Standard lint and impacted tests passed. No lint rules, assertions, or checks were suppressed.

CI hand-off: CI on the final pushed HEAD remains unverified. A maintainer must re-trigger CI before merge; the earlier JavaScript artifact and Go-advisory failures have been corrected and verified locally.

Deployment boundary

Two explicitly authorized live dispatcher runs were performed using local ./gh-aw. Run 37860259692 exposed the installation-token visibility misclassification. After the fix, run 37861825738 passed that boundary and failed closed with work_queue_policy_missing during activation.

No Policy was installed, worker identity invented, queue protection changed, native Work admitted, Claim created, or worker launched. End-to-end admission, launch, Result, and Release remain unverified. Deployment still requires administrator-installed Policy, verified principals, immutable worker revisions, and the documented writer protections. Both one-run live-test authorizations are consumed; this finishing pass did not dispatch workflows.

Seed an idempotent daily worker cohort, document Policy setup and queue diagnostics, paginate remote workflow discovery, and cancel write-capable no-effect Claims. Add explicit compile --allow-experimental while preserving all other dry-run gates. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-10-08T23:33:15Z

✅ PR Code Quality Reviewer completed the code quality review.

No GitHub write emitted yet while validating safeoutputs access.
Could not submit PR Seed ESLint factory work and allow experimental dry-run notices #67023 review output: both create_pull_request_review_comment and submit_pull_request_review were denied by the environment with approval-unavailable errors in this non-interactive run. No GitHub review/comment was posted.

🔎 Code quality review by PR Code Quality Reviewer

github-actions · 2026-10-08T23:33:16Z

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

github-actions · 2026-10-08T23:33:18Z

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel skipped because pre-fetch PR data was unavailable: unable to fetch test file diff

🧪 Test quality analysis by Test Quality Sentinel

github-actions · 2026-10-08T23:33:22Z

✅ Ponytail Reviewer completed successfully!

Lean already. Ship.

Generated by Ponytail Reviewer for #67023

github-actions · 2026-10-08T23:33:23Z

✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft.

🏗️ ADR gate enforced by Design Decision Gate 🏗️

github-actions · 2026-10-08T23:39:11Z

🏗️ Design Decision Gate — ADR Required

This PR makes significant changes to core business logic (237 new lines across pkg/, cmd/, and api/-class directories) but does not have a linked Architecture Decision Record (ADR).

📄 Draft ADR committed: docs/adr/67023-explicit-acknowledgment-for-experimental-compile-notices.md — review and complete it before merging.

🔒 This PR cannot merge until an ADR is linked in the PR body.

📋 What to do next

Review the draft ADR committed to your branch — it was generated from the PR diff (pkg/cli/compile_development.go, pkg/cli/compile_development_report.go, pkg/cli/compile_config.go, cmd/gh-aw/main.go)
Complete the missing sections — add context the AI couldn't infer, refine the decision rationale, and list real alternatives you considered
Commit the finalized ADR to docs/adr/ on your branch
Reference the ADR in this PR body by adding a line such as:

ADR: ADR-67023: Acknowledge Experimental-Feature Notices via an Explicit Compile Flag

Once an ADR is linked in the PR body, this gate will re-run and verify the implementation matches the decision.

🔍 Decision inferred from this PR

Decision: add an opt-in compile --allow-experimental flag that acknowledges only built-in experimental-feature notices during dry-run validation, keeping the notices visible and all other warnings/scanner failures fatal.
Driver: dry-run fails on any warning, so workflows that use still-experimental built-ins (ESLint factory dispatcher/miner/refiner/monster) can never produce a clean validation result.
Alternatives: keep experimental notices fatal; a generic warning-suppression/allowlist flag; per-workflow frontmatter opt-in.
Consequences: auditable, quantified acceptance (accepted_experimental_warnings in JSON) vs. a new CLI/report compatibility surface and a count-based (not identity-based) validity predicate.

Note: this PR also bundles unrelated changes (ESLint factory daily cohort admission, paginated remote workflow discovery). Those were treated as secondary to the primary architectural decision above.

📋 Michael Nygard ADR Format Reference

An ADR must contain these four sections to be considered complete:

Context — What is the problem? What forces are at play?
Decision — What did you decide? Why?
Alternatives Considered — What else could have been done?
Consequences — What are the trade-offs (positive and negative)?

All ADRs are stored in docs/adr/ as Markdown files numbered by PR number.

🏗️ ADR gate enforced by Design Decision Gate 🏗️ · pi · opus50 · 47.7 AIC · ⌖ 50.4 AIC · ⊞ 1.8K · ◷
_{Comment /review to run again}

Copilot

🔵 Needs a closer look

It introduces autonomous write-capable queue admission and dispatch behavior without complete optional scanner coverage.

0 open findings

What changed in this PR

Adds bounded daily ESLint factory admissions, improves remote workflow discovery, and permits explicit dry-run acknowledgment of experimental notices.

Changes:

Adds date-keyed factory plans, policy generation, and corrected Claim cancellation behavior.
Adds paginated, host-aware remote workflow discovery.
Adds compile --allow-experimental with warning accounting and reporting.

Security review found no confirmed regression, but optional Docker scanners were not run.

File	Description
`specs/eslint-factory/README.md`	Updates formal-model boundaries.
`pkg/workflow/compiler_validators.go`	Classifies experimental notices.
`pkg/workflow/compiler_types.go`	Stores experimental warning count.
`pkg/workflow/compiler_mutators.go`	Manages the new counter.
`pkg/workflow/compiler_experimental_warning_test.go`	Tests notice accounting.
`pkg/cli/workflows.go`	Adds paginated, host-aware discovery.
`pkg/cli/run_workflow_validation.go`	Reuses centralized discovery.
`pkg/cli/run_workflow_validation_test.go`	Tests pagination and hosts.
`pkg/cli/compile_development.go`	Allows acknowledged notices.
`pkg/cli/compile_development_report.go`	Reports accepted notice counts.
`pkg/cli/compile_development_report_test.go`	Tests dry-run gating.
`pkg/cli/compile_config.go`	Adds configuration option.
`docs/src/content/docs/reference/compilation-process.md`	Documents the new flag.
`docs/src/content/docs/patterns/linter-factory.md`	Documents deployment and diagnosis.
`cmd/gh-aw/main.go`	Registers and propagates the flag.
`cmd/gh-aw/compile_flags_test.go`	Tests flag propagation.
`actions/setup/js/work_queue_documentation.test.cjs`	Tests queue documentation.
`actions/setup/js/eslint_factory_portfolio.test.cjs`	Tests factory plans and policy.
`actions/setup/js/eslint_factory_portfolio.cjs`	Generates plans and policy.
`.github/workflows/eslint-monster.md`	Corrects no-effect cancellation.
`.github/workflows/eslint-monster.lock.yml`	Regenerates monster workflow.
`.github/workflows/eslint-miner.md`	Corrects no-rule cancellation.
`.github/workflows/eslint-miner.lock.yml`	Regenerates miner workflow.
`.github/workflows/eslint-factory-dispatcher.md`	Adds daily admission flow.
`.github/workflows/eslint-factory-dispatcher.lock.yml`	Regenerates dispatcher workflow.
`.github/aw/work-queue.md`	Clarifies queue semantics.
`.github/aw/debug-agentic-workflow.md`	Documents experimental opt-in.
`.changeset/patch-eslint-factory-producer.md`	Records factory fixes.
`.changeset/minor-compile-allow-experimental.md`	Records the new CLI option.

🧠 Review effort: Balanced

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

github-actions

Skills-Based Review 🧠

Applied /tdd and /codebase-design (pr-triage: new_feature, high-impact files: eslint_factory_portfolio.cjs, eslint-factory-dispatcher.md, compile_development.go, compiler_experimental_warning_test.go, run_workflow_validation.go).

I read the full diff, the new JS module (eslint_factory_portfolio.cjs + its test), the Go experimental-warning accounting (compile_development.go, compile_development_report.go, compiler_mutators.go/compiler_validators.go), and the fetchGitHubWorkflows pagination fix. I also verified claims against the repo: go build ./..., go vet, and the targeted Go test suite (pkg/workflow, pkg/cli, cmd/gh-aw) all pass locally. JS tests could not be executed in this sandbox (npm registry blocked by a self-signed-cert proxy), so I statically traced eslint_factory_portfolio.test.cjs and work_queue_documentation.test.cjs against the implementation instead.

📋 Key Themes & Highlights

Key Themes

No actionable bugs found. The accepted-experimental-warning threshold is computed/checked in three places (compile_development.go's two comparisons and compile_development_report.go's <=); all three agree and are exercised by TestDryRunExperimentalOptIn's table (including a forged "Using experimental feature: ..." string injected into a structured per-result warning, which correctly still fails the gate — good defense against spoofing the opt-in). This logic is spread thin across two files; a future /codebase-design pass could centralize it into one helper to reduce the chance of the three checks drifting apart, but it's not a blocking issue today given the test coverage.
The new eslint_factory_portfolio.cjs plan/policy builders have solid input validation (repo slug regex, UTC date round-trip check, positive-decimal principal IDs) and the test suite exercises idempotent same-day admission, producer-entitlement rejection, and the dispatcher's actual github-script preparation step via a sandboxed AsyncFunction replay — a nice example of testing embedded workflow scripts without executing CI.
fetchGitHubWorkflows's pagination/host fix (run_workflow_validation.go, workflows.go) replaces an un-paginated 50-workflow gh workflow list with --paginate --slurp plus repoutil.NormalizeRepoForAPI for GHES hosts, matched by a table-driven test using a fake gh script — correctly covers later-page, disabled-workflow, enterprise-host, suffix-collision (not-eslint-factory-dispatcher.lock.yml vs eslint-factory-dispatcher.lock.yml), and malformed-JSON cases.
Compiled lock file (eslint-factory-dispatcher.lock.yml) correctly reflects the new bash: [cat ...] tool allowlist and the Prepare the UTC ESLint factory cohort step — verified by direct inspection, not just trusting the diff.

Positive Highlights

✅ Good regression-test design: tests actively try to defeat the new opt-in (forged experimental-notice text, ordinary/aggregate/safe-update/schedule/inventory warnings, scanner failures) and assert the gate still fails for all of them.
✅ Docs (linter-factory.md, work-queue.md, debug-agentic-workflow.md) stay consistent with real CLI subcommands (stats, state, explain, policy, --no-baseline) — spot-checked against work_command.go/audit_command.go.
✅ Clear separation between "uninitialized queue" (deployment failure → missing_data) and "empty queue" (noop), reinforced consistently across the dispatcher prompt, docs, and worker "cancel vs. noop" wording for write-capable Claims.

No blocking issues identified; approving.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · sonnet50 · 203.8 AIC · ⌖ 14.5 AIC · ⊞ 10.1K
_{Comment /matt to run again}

The live dispatcher failed on installation-token collaborator pull metadata before reading queue refs. Establish contents visibility through actual Git reads, preserve real API denials, and require a readable default ref before reporting an absent queue in a nonempty repository. Add regressions and document the activation/Policy boundary. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

gh-aw-bot · 2026-10-09T00:03:35Z

@copilot address the following outstanding work in one pass:

Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.
Fix failing check activation (FAILURE): https://github.com/github/gh-aw/actions/runs/37861825738/job/113599141515.
Fix failing check JS Tests (shard 2/4) (FAILURE): https://github.com/github/gh-aw/actions/runs/37861464057/job/113598080176.
Fix failing check lint-go-govulncheck (FAILURE): https://github.com/github/gh-aw/actions/runs/37861464002/job/113598077529.

Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI.

Sous-chef head: ae0ccdf
Sous-chef work: 043185828d65a534dd8b2b6b4d00e0d47bf21a2940d3f277c0ed15da7f668648 4ce4dd407e604d374ecd66edfd2afc18c54e2d0b3733dbac37ac8e9c8834fdba a0e3721d61627208515dab43433d4bfd19a9675eb2d47d1232914f6f7362193f
Sous-chef state: 0d2b9b926ae7ef9cbbe1e00540509e84a0b6551d64010e5b9d77c84957598fe9

Generated by 👨‍🍳 PR Sous Chef · pi · haiku45 · 4.73 AIC · ⌖ 7.53 AIC · ⊞ 1K · ◷
_{Comment /souschef to run again}

…eue-debugging Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Regenerate the frontmatter reference to restore the work-queue boolean example tested by CI. Complete the experimental-notice ADR with explicit classification, independent validation gates, alternatives, and regression evidence. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot · 2026-10-09T00:21:50Z

@copilot address the following outstanding work in one pass:

Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes....

Merged latest main in b979f8c. Commit 4ba8726 fixes the stale JS frontmatter assertion and updates Go to 1.26.9 plus golang.org/x/net v0.60.0, the versions indicated by the govulncheck findings. The activation run still fails closed because no authoritative queue branch or Policy is installed; bypassing that would start a write-capable dispatcher without configured authority. The local govulncheck rerun was blocked by sandbox DNS reaching vuln.go.dev. There are no review threads for this conversation comment to resolve.

…-es-linter-queue-debugging

Keep the concurrent null-format assertion and verify the boolean example restored by regeneration, rather than masking a stale documentation artifact. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

gh-aw-bot · 2026-10-09T01:07:27Z

@copilot address the following outstanding work in one pass:

Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.
Fix failing check Unit tests (S-Z and other) (FAILURE): https://github.com/github/gh-aw/actions/runs/37865793371/job/113612601555.

Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI.

Sous-chef head: de01388
Sous-chef work: 48a183c3e95bbf17204364ee4c81061de61a873e8585852342c81727b022959c
Sous-chef state: d2f58340e1173556ac81dfb2345ea90ed0cd351acd07dde513c752b2d1eefeb3

Generated by 👨‍🍳 PR Sous Chef · pi · haiku45 · 4.51 AIC · ⌖ 7.85 AIC · ⊞ 1K · ◷
_{Comment /souschef to run again}

…eue-debugging # Conflicts: # go.sum Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot · 2026-10-09T01:15:54Z

@copilot address the following outstanding work in one pass:

Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes....

Merged latest main in 6a42b82379 and fixed the unit-test failure by embedding work_queue_logging.cjs; the focused test and full actions/setup tests pass. make lint and impacted Go/JS tests passed. The final progress gate still reports existing custom Go lint findings in untouched files. No review threads were open to resolve. CI on this pushed HEAD remains unverified and needs a maintainer-triggered rerun.

github-actions · 2026-10-09T15:49:32Z

🎉 This pull request is included in a new release.

Release: v0.91.7

pelikhan marked this pull request as ready for review October 8, 2026 23:32

Copilot AI balanced review requested due to automatic review settings October 8, 2026 23:32

Copilot started reviewing on behalf of pelikhan October 8, 2026 23:32 View session

Add draft ADR-67023 for explicit experimental-notice acknowledgment

8eb0f0f

Copilot AI reviewed Oct 8, 2026

View reviewed changes

github-actions Bot approved these changes Oct 8, 2026

View reviewed changes

github-actions Bot mentioned this pull request Oct 9, 2026

[pr-sous-chef] PR Sous Chef 🍳 – Nudges Posted #67031

Closed

Copilot started work on behalf of gh-aw-bot October 9, 2026 00:03 View session

Copilot AI and others added 3 commits October 9, 2026 00:09

Merge remote-tracking branch 'origin/main' into pelikhan-es-linter-qu…

b979f8c

…eue-debugging Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Fix merge follow-up CI failures

4ba8726

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot finished work on behalf of gh-aw-bot October 9, 2026 00:22

Copilot AI requested a review from gh-aw-bot October 9, 2026 00:22

pelikhan and others added 2 commits October 8, 2026 17:22

Merge commit '4ba87263f37e2b21c597576ba121cde10eb0b261' into pelikhan…

9aac18c

…-es-linter-queue-debugging

Cover both work-queue activation formats in generated docs

de01388

Keep the concurrent null-format assertion and verify the boolean example restored by regeneration, rather than masking a stale documentation artifact. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot started work on behalf of gh-aw-bot October 9, 2026 01:07 View session

Merge remote-tracking branch 'origin/main' into pelikhan-es-linter-qu…

6a42b82

…eue-debugging # Conflicts: # go.sum Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot finished work on behalf of gh-aw-bot October 9, 2026 01:19

pelikhan merged commit 18b9238 into main Oct 9, 2026
48 checks passed

pelikhan deleted the pelikhan-es-linter-queue-debugging branch October 9, 2026 01:50

Conversation

pelikhan commented Oct 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Architecture decision

Validation

Deployment boundary

Uh oh!

github-actions Bot commented Oct 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 8, 2026

🏗️ Design Decision Gate — ADR Required

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Skills-Based Review 🧠

Key Themes

Positive Highlights

Uh oh!

gh-aw-bot commented Oct 9, 2026

Uh oh!

Copilot AI commented Oct 9, 2026

Uh oh!

gh-aw-bot commented Oct 9, 2026

Uh oh!

Copilot AI commented Oct 9, 2026

Uh oh!

Uh oh!

github-actions Bot commented Oct 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

pelikhan commented Oct 8, 2026 •

edited

Loading

github-actions Bot commented Oct 8, 2026 •

edited

Loading

github-actions Bot commented Oct 8, 2026 •

edited

Loading

github-actions Bot commented Oct 8, 2026 •

edited

Loading

github-actions Bot commented Oct 8, 2026 •

edited

Loading

github-actions Bot commented Oct 8, 2026 •

edited

Loading