Docs: add experimental Copilot model-routing guide by Copilot · Pull Request #66338 · github/gh-aw

Copilot · 2026-10-07T00:18:58Z

engine.model-routing works end to end, but the generated reference does not explain its runtime behavior, configuration semantics, or constraints. This change proposes an author-facing reference covering those gaps.

Routing guide: Explains per-run model and effort selection, quality profiles, advisory selection versus request policy, version requirements, and image overrides.
Operational caveats: Documents sub-agent API-family compatibility, nondeterministic selection, and the threat-detection failure that permits safe outputs without a verdict.
Observability: Covers routing and credit logs, current CLI presentation limits, exit code 78, degraded classifications, deviations, and upstream errors. Links upstream contracts rather than duplicating them.
Reference integration: Adds engine/frontmatter links, expands schema descriptions, regenerates routing-related entries, and adds documentation regression checks.

engine:
  id: copilot
  model-routing:
    goal: cost
    mode: auto
    allowed-models:
      - gpt-5.4-mini
      - gpt-5.4
      - gpt-5.6-luna

Fixes Docs: add a model routing guide for engine.model-routing #66337

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

Copilot

Copilot review overview

🟢 Approval recommended

The documentation accurately reflects current compiler behavior and upstream routing contracts, with appropriate regression coverage.

Review effort: Balanced
Findings: None

What changed in this PR

Adds an experimental Copilot model-routing reference covering configuration, runtime behavior, limitations, observability, and troubleshooting.

Changes:

Adds and links a comprehensive model-routing guide.
Expands schema descriptions and regenerated reference output.
Adds regression checks for generated routing documentation.

File	Description
`scripts/generate-schema-docs.test.js`	Verifies generated routing guidance.
`pkg/parser/schemas/main_workflow_schema.json`	Expands routing field descriptions.
`docs/src/content/docs/reference/model-routing.md`	Adds the model-routing reference.
`docs/src/content/docs/reference/frontmatter.md`	Links routing from frontmatter documentation.
`docs/src/content/docs/reference/frontmatter-full.md`	Regenerates schema-derived documentation.
`docs/src/content/docs/reference/engines.md`	Links routing from the Copilot engine section.

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

github-actions · 2026-10-07T04:31:13Z

✅ Ponytail Reviewer completed successfully!

Generated by Ponytail Reviewer for #66338

github-actions · 2026-10-07T04:31:15Z

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

github-actions · 2026-10-07T04:31:25Z

🔎 PR Code Quality Reviewer is reviewing code quality for this pull request...

github-actions · 2026-10-07T04:31:26Z

✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft.

No ADR enforcement needed for PR #66338: the PR does not carry the 'implementation' label (has_implementation_label=false) and adds only 4 lines in business logic directories (default_business_additions=4, threshold=100, has_custom_config=false, file_count=6).

🏗️ ADR gate enforced by Design Decision Gate 🏗️

github-actions · 2026-10-07T04:31:54Z

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel skipped because pre-fetch PR data was unavailable: unable to fetch test file diff

🧪 Test quality analysis by Test Quality Sentinel

github-actions

L159-166: delete: six exact prose assertions for generated schema docs. Keep the generator's existing structural coverage; wording belongs in the source schema and guide.

net: -8 lines possible.

Generated by ✂️ Ponytail Reviewer for #66338 · codex · gpt56 · 8.77 AIC · ⌖ 6.41 AIC · ⊞ 13.4K
_{Comment /ponytail to run again}

github-actions · 2026-10-07T04:34:43Z

 allPassed &= assertContains(output, "report-failed-jobs:", "Safe outputs example should include report-failed-jobs");

+// Test 12: Model routing descriptions explain values and link to the guide
+const unwrappedOutput = output.replace(/\n\s*#\s*/g, " ");


L159-166: delete: six exact prose assertions for generated schema docs. Keep the generator's existing structural coverage; wording belongs in the source schema and guide.

github-actions

Skills-Based Review 🧠

Applied /grill-with-docs — this is a well-researched, accurate documentation PR with only a single minor, non-blocking nit.

📋 Key Themes & Highlights

Verification performed

Cross-checked every claim in model-routing.md against pkg/workflow/model_routing.go, pkg/constants/version_constants.go, and pkg/parser/schemas/main_workflow_schema.json (goal/mode enums, AWF min-versions v0.28.29/v0.28.33, default AWF/router versions, expression-vs-policy rejection logic, sub-agent API-family constraint).
Confirmed cited cross-repo references (gh-aw#66234, gh-aw#66291, gh-aw-firewall#9509) exist and match their described content.
Ran node scripts/generate-schema-docs.test.js — all 26 assertions pass, including the 6 new ones for the model-routing schema descriptions.
Confirmed frontmatter-full.md is machine-generated from the schema, so the schema and reference doc stay in sync by construction.
Checked sidebar order: 605 for model-routing.md doesn't collide with existing reference pages.

Minor note

One inline comment: the new page's internal links (../model-routing/, ../sandbox/) use relative paths, while all other cross-references under reference/ use the absolute /gh-aw/reference/<slug>/ form. Not a functional bug, just a style inconsistency worth aligning for future-author clarity.

Positive highlights

✅ Clearly separates the advisory selection model from the hard request-policy (allowed-models vs. admitted sub-agent models) — exactly the kind of precise domain-language distinction /grill-with-docs looks for.
✅ Surfaces the threat-detection-fails-silently limitation prominently with a warning callout, rather than burying a real operational risk.
✅ Links to upstream AWF/router contracts instead of duplicating schema details that could drift.
✅ Adds regression tests tying generated docs to the new schema wording.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · sonnet50 · 112.9 AIC · ⌖ 14.6 AIC · ⊞ 10.1K
_{Comment /matt to run again}

github-actions · 2026-10-07T04:40:56Z


+### Model Routing (`engine.model-routing:`)
+
+Enables experimental per-run model and reasoning-effort selection for Copilot with the AWF firewall. Requires `goal` (`cost` or `cost-speed`), `mode` (`economy`, `balanced`, `robust`, or `auto`), and a non-empty `allowed-models` list. Overrides the fixed `engine.model` and effort settings. See [Model Routing](../model-routing/) for value meanings, requirements, sub-agent constraints, known limitations, and logs.


[/grill-with-docs] Minor link-convention inconsistency: this and the ../sandbox/ link below use relative paths, while every other cross-reference in reference/*.md (including the rest of this same file) uses the absolute /gh-aw/reference/<slug>/ form.

💡 Suggested fix

Change to match the established convention used throughout reference/:

See [Model Routing](/gh-aw/reference/model-routing/) for value meanings, requirements, sub-agent constraints, known limitations, and logs.

Relative links (../x/) still resolve under Starlight's routing, so this won't break the build, but it diverges from the project's established domain-language/navigation convention and could confuse future doc authors about which style to use. Not blocking.

@copilot please address this.

gh-aw-bot · 2026-10-07T05:06:02Z

@copilot address the following outstanding work in one pass:

Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.
Review (scripts/generate-schema-docs.test.js:160): L159-166: delete: six exact prose assertions for generated schema docs. Keep the generator's existing structural coverage; wording belongs in the source schema and guide. - Docs: add experimental Copilot model-routing guide #66338 (comment)
Review (docs/src/content/docs/reference/frontmatter.md:214): [/grill-with-docs] Minor link-convention inconsistency: this and the ../sandbox/ link below use relative paths, while every other cross-reference in reference/*.md (including the rest of this same file) uses the absolute /gh-aw/reference/<slug>/ form. - Docs: add experimental Copilot model-routing guide #66338 (comment)
Fix failing check build (FAILURE): https://github.com/github/gh-aw/actions/runs/37552087989/job/112631808323.
Fix failing check Conclusion (FAILURE): https://github.com/github/gh-aw/actions/runs/37566036774/job/112633871578.

Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI.

Sous-chef head: ee404e1
Sous-chef work: 06fbcf03cf3ab21f1f8a7c7014e45cd66006dec0f55f7be66790ef1ea2115f73 3c28f08508fdd2ad4be24ddf07bf9a178c05b716f2b4bd0b8f5d33985c48c903 6be5b3f36c0996c81834ee81fefd7beeb7281fc012ea00d20062c9950be69cf3 f7883e8c8105930e955354caa741d783bcca067e66097fe9612fb72b90823998
Sous-chef state: 8cd0253510cb14ab993a54141d48e1b886c1394d903da5205149aa01e06a4ca7

Generated by 👨‍🍳 PR Sous Chef · pi · haiku45 · 3.27 AIC · ⌖ 8.05 AIC · ⊞ 1K · ◷
_{Comment /souschef to run again}

github-actions · 2026-10-07T20:32:21Z

🎉 This pull request is included in a new release.

Release: v0.91.5

Initial plan

65cfa71

Copilot AI assigned Copilot and lpcox Oct 7, 2026

Copilot started work on behalf of lpcox October 7, 2026 00:19 View session

Copilot AI linked an issue Oct 7, 2026 that may be closed by this pull request

Docs: add a model routing guide for engine.model-routing #66337

Closed

Document experimental Copilot model routing

ee404e1

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Add a model routing guide for engine.model-routing~~ Docs: add experimental Copilot model-routing guide Oct 7, 2026

Copilot finished work on behalf of lpcox October 7, 2026 00:29

Copilot AI requested a review from lpcox October 7, 2026 00:29

lpcox marked this pull request as ready for review October 7, 2026 03:17

Copilot AI balanced review requested due to automatic review settings October 7, 2026 03:17

Copilot started reviewing on behalf of lpcox October 7, 2026 03:17 View session

Copilot AI reviewed Oct 7, 2026

View reviewed changes

github-actions Bot reviewed Oct 7, 2026

View reviewed changes

github-actions Bot mentioned this pull request Oct 7, 2026

[aw] No-Op Runs #65943

Open

github-actions Bot reviewed Oct 7, 2026

View reviewed changes

github-actions Bot mentioned this pull request Oct 7, 2026

[pr-sous-chef] 🍳 PR Sous Chef Summary: 4 PRs nudged toward completion #66387

Closed

Copilot started work on behalf of gh-aw-bot October 7, 2026 05:06 View session

lpcox merged commit 8c52586 into main Oct 7, 2026
109 of 134 checks passed

lpcox deleted the copilot/add-model-routing-guide branch October 7, 2026 05:07

Copilot stopped work on behalf of gh-aw-bot due to an error October 7, 2026 05:07
The session was cancelled by the user.

Copilot AI requested a review from gh-aw-bot October 7, 2026 05:07

This was referenced Oct 7, 2026

gh aw audit, view, outcomes and logs build OWNER\REPO on Windows, so artifact downloads fail #66637

Closed

Model routing: show AWF routing decisions in gh aw logs and gh aw audit #66638

Closed

This was referenced Oct 7, 2026

Fix Windows repository slugs in artifact downloads and metadata caching #66644

Merged

Surface model-routing decisions in logs, audits, and unified sessions #66645

Merged

Navigation Menu

Repository navigation

Docs: add experimental Copilot model-routing guide - #66338

Docs: add experimental Copilot model-routing guide#66338
lpcox merged 2 commits into
mainfrom
copilot/add-model-routing-guide

Copilot AI commented Oct 7, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

github-actions Bot commented Oct 7, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 7, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 7, 2026

Uh oh!

github-actions Bot commented Oct 7, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 7, 2026 •

edited

Loading

Uh oh!

github-actions Bot left a comment

Uh oh!

github-actions Bot Oct 7, 2026

Uh oh!

github-actions Bot left a comment

Uh oh!

github-actions Bot Oct 7, 2026

Uh oh!

gh-aw-bot commented Oct 7, 2026

Uh oh!

Uh oh!

github-actions Bot commented Oct 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants


		### Model Routing (`engine.model-routing:`)

		Enables experimental per-run model and reasoning-effort selection for Copilot with the AWF firewall. Requires `goal` (`cost` or `cost-speed`), `mode` (`economy`, `balanced`, `robust`, or `auto`), and a non-empty `allowed-models` list. Overrides the fixed `engine.model` and effort settings. See [Model Routing](../model-routing/) for value meanings, requirements, sub-agent constraints, known limitations, and logs.

Conversation

Copilot AI commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟢 Approval recommended

Uh oh!

github-actions Bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 7, 2026

Uh oh!

github-actions Bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Oct 7, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Skills-Based Review 🧠

Verification performed

Minor note

Positive highlights

Uh oh!

github-actions Bot Oct 7, 2026

Choose a reason for hiding this comment

Uh oh!

gh-aw-bot commented Oct 7, 2026

Uh oh!

Uh oh!

github-actions Bot commented Oct 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Copilot AI commented Oct 7, 2026 •

edited

Loading

github-actions Bot commented Oct 7, 2026 •

edited

Loading

github-actions Bot commented Oct 7, 2026 •

edited

Loading

github-actions Bot commented Oct 7, 2026 •

edited

Loading

github-actions Bot commented Oct 7, 2026 •

edited

Loading