Fix GPT 6.1 Sol pricing propagation to the firewall by pelikhan · Pull Request #67413 · github/gh-aw

pelikhan · 2026-10-10T11:56:05Z

Why

The Schema Consistency Checker failed because AWF rejected gpt-6.1-sol requests with unknown_model_ai_credits. gh-aw already contains the model's pricing, but compilation did not pass those rates to the firewall's independent pricing table.

Fixes #67357

Approach

Register an offline pricing resolver that matches the exact provider and model in gh-aw's embedded catalog. Existing compiler plumbing includes the resolved rates in both GH_AW_INFO_MODEL_COSTS and apiProxy.providers.

Explicit frontmatter pricing remains authoritative. Unknown models and dynamic aliases do not receive guessed or prefix-based rates, and AI-credit limits remain unchanged. The fix applies to concrete catalog models generally, not just GPT 6.1 Sol; the 141 affected workflow lock files were regenerated. Cost-management documentation explains the behavior.

Validation

Focused pricing/compiler tests pass, including OpenAI and Copilot GPT 6.1 Sol pricing, override precedence, and rejection of unrelated model/provider matches.
The compiled issue workflow includes the expected input, output, cache-read, and cache-write rates while retaining its 1,500-credit cap.
All 334 workflows compile, and the workflow-drift check passes.
Incremental Go lint reports zero new issues.

The standard final gate is not green: it reports existing custom-linter findings in unchanged portions of touched files and encountered a concurrent golangci-lint lock. The broader CLI suite also fails on macOS path/architecture assumptions and unrelated tests. No workflow was dispatched.

Include exact provider-specific catalog pricing in compiled workflows so AWF can enforce AI-credit limits for GPT 6.1 Sol even when its own pricing table lags behind gh-aw. Preserve frontmatter overrides and leave unknown models unchanged. Fixes #67357 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-10-10T12:08:37Z

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel skipped because pre-fetch PR data was unavailable: unable to fetch test file diff
No test files were added or modified in this PR. Test Quality Sentinel skipped.

🧪 Test quality analysis by Test Quality Sentinel

github-actions · 2026-10-10T12:08:39Z

✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft.

No ADR enforcement needed: PR #67413 does not have the 'implementation' label and has 0 new lines of code in default business logic directories (threshold: 100).

🏗️ ADR gate enforced by Design Decision Gate 🏗️

github-actions · 2026-10-10T12:08:40Z

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

github-actions · 2026-10-10T12:08:43Z

✅ Ponytail Reviewer completed successfully!

Review not performed: the required pre-fetched PR diff artifact is empty.

Generated by Ponytail Reviewer for #67413

github-actions · 2026-10-10T12:08:46Z

✅ PR Code Quality Reviewer completed the code quality review.

🔎 Code quality review by PR Code Quality Reviewer

Copilot

🟡 Changes recommended

Parameterized models retain query suffixes during pricing resolution, so supported forms such as gpt-6.1-sol?effort=high still miss firewall pricing.

1 open finding

Normalize parameterized model IDs before pricing lookup and merge · New

What changed in this PR

Propagates embedded model pricing into firewall configuration, addressing unknown_model_ai_credits for GPT 6.1 Sol.

Changes:

Registers exact provider/model pricing resolution during compilation.
Adds regression tests and cost-management documentation.
Regenerates affected workflow locks with firewall pricing overlays.

Static review found no permission or network-access regressions; scanners were not independently rerun.

File	Description
`pkg/workflow/compiler_types.go`	Updates resolver documentation.
`pkg/workflow/compiler_mutators.go`	Exposes resolver configuration.
`pkg/workflow/compiler_model_pricing.go`	Documents catalog pricing propagation.
`pkg/cli/model_costs.go`	Adds exact catalog lookup.
`pkg/cli/model_costs_test.go`	Tests exact matching.
`pkg/cli/compile_compiler_setup.go`	Registers the resolver.
`pkg/cli/compile_compiler_setup_test.go`	Tests firewall propagation and precedence.
`docs/src/content/docs/reference/cost-management.md`	Documents compiled pricing behavior.
`.github/workflows/smoke-copilot.lock.yml`	Adds Copilot pricing overlays.
`.github/workflows/smoke-copilot-arm.lock.yml`	Adds Copilot pricing overlays.
`.github/workflows/smoke-codex.lock.yml`	Adds OpenAI pricing overlays.
`.github/workflows/smoke-ci.lock.yml`	Adds GPT pricing metadata.
`.github/workflows/notion-issue-summary.lock.yml`	Adds OpenAI pricing configuration.
`.github/workflows/hippo-embed.lock.yml`	Adds Copilot pricing configuration.
`.github/workflows/example-permissions-warning.lock.yml`	Adds model pricing configuration.
`.github/workflows/engine-conformance-pydantic-ai.lock.yml`	Adds conformance pricing configuration.
`.github/workflows/engine-conformance-pi.lock.yml`	Adds conformance pricing configuration.
`.github/workflows/engine-conformance-opencode.lock.yml`	Adds conformance pricing configuration.
`.github/workflows/engine-conformance-goose.lock.yml`	Adds conformance pricing configuration.
`.github/workflows/engine-conformance-deepseek-harness.lock.yml`	Adds conformance pricing configuration.
`.github/workflows/engine-conformance-crush.lock.yml`	Adds conformance pricing configuration.
`.github/workflows/engine-conformance-aider.lock.yml`	Adds conformance pricing configuration.
`.github/workflows/codex-github-remote-mcp-test.lock.yml`	Adds OpenAI pricing configuration.

🧠 Review effort: Balanced

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

github-actions

Request changes

The new pricing propagation still misses supported model spellings and alias-expanded targets, so unknown_model_ai_credits can still fire in valid configurations.

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt54 · 97.7 AIC · ⌖ 5.43 AIC · ⊞ 19.9K
_{Comment /review to run again}

Comments that could not be inline-anchored

pkg/workflow/compiler_model_pricing.go:94

This resolver now passes the full model identifier through to the catalog lookup, so supported forms like copilot/gpt-6.1-sol?effort=high still miss pricing injection and can keep failing with unknown_model_ai_credits.

<details><summary>💡 Strip model parameters before resolving pricing</summary>

Other model-validation paths normalize identifiers by cutting off the ?... suffix before comparing them. Doing the same here before the provider split would make parameterized identifiers reuse…

pkg/cli/model_costs.go:149

The new exact lookup only lower-cases and trims, so catalog-equivalent spellings that the rest of gh-aw already normalizes (GPT_6_1_SOL, mixed ./_ separators) will skip the injected pricing and regress back to firewall rejections.

<details><summary>💡 Reuse comparable-model normalization here</summary>

NormalizeComparableModelID is already the repository-wide comparison rule for model identifiers. Applying it in this exact-match path would keep the new compiler overlay consistent with…

github-actions

Skills-Based Review 🧠

Applied /diagnosing-bugs and /tdd to the 7 real source files behind this fix (pkg/cli/{model_costs,compile_compiler_setup}{,_test}.go, pkg/workflow/{compiler_model_pricing,compiler_mutators,compiler_types}.go); the other 141 changed files are regenerated .lock.yml artifacts.

📋 Key Themes & Highlights

Key Themes

One normalization gap (flagged inline): findExactModelPricing compares a lowercased/trimmed model string directly against catalog entries, skipping the NormalizeComparableModelID pass that the fuzzy findModelPricing applies. Dot/hyphen/underscore variants of the same model ID will silently miss the new exact-match guardrail. Low risk today since catalog IDs are canonical, but worth a regression test given this is exactly the kind of "looks matched but doesn't" bug the PR is designed to prevent.

Positive Highlights

✅ Root cause correctly addressed: wiring SetModelPricingResolver with findExactModelPricing as the new default resolver, rather than only patching symptoms in individual workflows.
✅ TestCreateAndConfigureCompiler_GPT61SolFirewallPricing is a strong regression test — it exercises engine/provider/model permutations end-to-end through BuildAWFConfigJSON and confirms frontmatter overrides still win, directly covering the originating bug (unknown_model_ai_credits).
✅ TestFindExactModelPricing covers the guardrail boundary well: cross-provider rejection, unknown models, and the auto dynamic alias are all exercised.
✅ Doc update in cost-management.md accurately describes the new behavior without overclaiming scope.

No blocking issues found — approving with one actionable suggestion left as an inline comment.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · sonnet50 · 154.5 AIC · ⌖ 14.5 AIC · ⊞ 10.1K
_{Comment /matt to run again}

Strip model parameters before pricing lookup, frontmatter override detection, and firewall overlay emission. Reuse provider-scoped comparable model matching after literal matches and cover parameterized IDs, separator-equivalent spellings, override precedence, and unknown models. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

gh-aw-bot · 2026-10-10T12:57:22Z

Caution

agentic threat detected
Threat detection flagged this output in warn mode. Manual review is REQUIRED before any follow-up automation.

Details

Potential security threats were detected in the agent output.

Review the workflow run logs for details.

@copilot address the following outstanding work in one pass:

Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.
Review (pkg/cli/model_costs.go:146): [/tdd] Good guardrail test coverage for exact-match semantics, but there's an untested normalization gap: findExactModelPricing only lowercases/trims the model string, while the fuzzy findModelPricing also runs it through modelsdev.NormalizeComparableModelID (dots/underscores → hyphens). A lookup like gpt-6-1-sol (hyphen instead of dot) silently misses here but matches in the fuzzy path — an easy source of confusing "works with compile defaults but fails via this guardrail" bugs. - Fix GPT 6.1 Sol pricing propagation to the firewall #67413 (comment)

Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI.

Sous-chef head: 5c08462
Sous-chef work: 2a376f8fb207dcf8d58436c7289f50e53801fb86d777d731e03a8a458b58d69b
Sous-chef state: a221bce263aef98967a5489ebf2493a0b9829ebf1dba6b103b612dbc8d51f86a

Generated by 👨‍🍳 PR Sous Chef · pi · haiku45 · 3.37 AIC · ⌖ 11.7 AIC · ⊞ 1K · ◷
_{Comment /souschef to run again}

Cover OpenAI and Copilot hyphenated GPT 6.1 Sol identifiers while retaining wrong-provider and unknown-suffix rejection. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

…pricing # Conflicts: # .github/workflows/avenger.lock.yml Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Regenerate the Avenger workflow lock from merged sources, preserving upstream engine changes and catalog pricing propagation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

…to pelikhan-gpt-6-1-sol-pricing # Conflicts: # pkg/cli/model_costs_test.go Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Avoid npx selecting an unpinned alpha formatter when worktree dependencies are missing. Use the installed setup package formatter and fail with npm ci instructions when unavailable. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot · 2026-10-10T13:18:01Z

[!CAUTION]
agentic threat detected
Threat detection flagged this output in warn mode. Manual review is REQUIRED before any follow-up automation....

Merged latest main via make merge-main (93b6523), regenerated the Avenger lock conflict, and integrated the concurrent branch update (19b529e). Hyphen normalization coverage and firewall-propagation tests pass (9cd7ff7); replied to the listed review thread. Formatting, impacted tests, schema freshness, and incremental Go lint pass; pre-existing custom-linter findings still block the local gate. Automated review was unavailable; fallback code review found no issues. CodeQL Actions found no alerts, but Go scanning was incomplete. No CI was triggered. Thread resolution could not be completed: the available API tools expose no resolve operation, gh has no token, and browser access failed; it still needs manual resolution.

pelikhan marked this pull request as ready for review October 10, 2026 12:07

Copilot AI balanced review requested due to automatic review settings October 10, 2026 12:07

Copilot started reviewing on behalf of pelikhan October 10, 2026 12:08 View session

pelikhan mentioned this pull request Oct 10, 2026

Eliminate independently maintained model-pricing stores in gh-aw and AWF github/gh-aw-firewall#9787

Closed

github-actions Bot mentioned this pull request Oct 10, 2026

[aw] No-Op Runs #65943

Open

Copilot AI reviewed Oct 10, 2026

View reviewed changes

Comment thread pkg/cli/model_costs.go

github-actions Bot requested changes Oct 10, 2026

View reviewed changes

github-actions Bot approved these changes Oct 10, 2026

View reviewed changes

Comment thread pkg/cli/model_costs.go

github-actions Bot mentioned this pull request Oct 10, 2026

[pr-sous-chef] PR Sous Chef Status Report - 2026-10-10 10:25 #67397

Open

Copilot started work on behalf of gh-aw-bot October 10, 2026 12:57 View session

pelikhan and others added 7 commits October 10, 2026 06:04

Test hyphenated model pricing normalization

6e6d479

Cover OpenAI and Copilot hyphenated GPT 6.1 Sol identifiers while retaining wrong-provider and unknown-suffix rejection. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Merge remote-tracking branch 'origin/main' into pelikhan-gpt-6-1-sol-…

93b6523

…pricing # Conflicts: # .github/workflows/avenger.lock.yml Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Merge origin/main into pelikhan-gpt-6-1-sol-pricing

d85824a

Regenerate the Avenger workflow lock from merged sources, preserving upstream engine changes and catalog pricing propagation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Test hyphen-equivalent exact model pricing and firewall propagation

9cd7ff7

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Merge remote-tracking branch 'origin/pelikhan-gpt-6-1-sol-pricing' in…

19b529e

…to pelikhan-gpt-6-1-sol-pricing # Conflicts: # pkg/cli/model_costs_test.go Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Merge concurrent pricing regression updates

446b4a0

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

pelikhan merged commit a180a82 into main Oct 10, 2026
27 checks passed

pelikhan deleted the pelikhan-gpt-6-1-sol-pricing branch October 10, 2026 13:14

Copilot finished work on behalf of gh-aw-bot October 10, 2026 13:18

Copilot AI requested a review from gh-aw-bot October 10, 2026 13:18

Copilot AI mentioned this pull request Oct 10, 2026

Add versioned shared model-pricing catalog support github/gh-aw-firewall#9791

Merged

Navigation Menu

Repository navigation

Fix GPT 6.1 Sol pricing propagation to the firewall - #67413

Fix GPT 6.1 Sol pricing propagation to the firewall#67413
pelikhan merged 9 commits into
mainfrom
pelikhan-gpt-6-1-sol-pricing

pelikhan commented Oct 10, 2026

Uh oh!

github-actions Bot commented Oct 10, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 10, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 10, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 10, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 10, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

github-actions Bot left a comment

Uh oh!

github-actions Bot left a comment

Uh oh!

Uh oh!

gh-aw-bot commented Oct 10, 2026

Uh oh!

Uh oh!

Copilot AI commented Oct 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

pelikhan commented Oct 10, 2026

Why

Approach

Validation

Uh oh!

github-actions Bot commented Oct 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Request changes

Comments that could not be inline-anchored

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Skills-Based Review 🧠

Key Themes

Positive Highlights

Uh oh!

Uh oh!

gh-aw-bot commented Oct 10, 2026

Uh oh!

Uh oh!

Copilot AI commented Oct 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

github-actions Bot commented Oct 10, 2026 •

edited

Loading

github-actions Bot commented Oct 10, 2026 •

edited

Loading

github-actions Bot commented Oct 10, 2026 •

edited

Loading

github-actions Bot commented Oct 10, 2026 •

edited

Loading

github-actions Bot commented Oct 10, 2026 •

edited

Loading