Skip to content

Report AWF-routed model and effort across workflow outputs - #66963

Merged
pelikhan merged 7 commits into
mainfrom
copilot/fix-model-routing-reporting
Oct 8, 2026
Merged

pelikhan merged 7 commits into
mainfrom
copilot/fix-model-routing-reporting

Conversation

Copilot AI commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

With model routing enabled, gh-aw reported compile-time placeholders instead of the model and effort selected at runtime. This made workflow metadata, generated footers, audit/logs, and OTEL attribution misleading.

  • Capture and propagate routing: Persist validated AWF selection, effort, status, and requested model in aw_info.json; expose runtime model, effort, and status to safe-output and conclusion jobs. Harness outcomes remain advisory and cannot override the proxy-selected model.
  • Attribute outputs consistently: Use the effective model in footers and markers, audit/logs, and OTEL spans. Legacy routed runs recover model and effort from routing logs; failed or rejected routing does not attribute a placeholder or classifier model as the agent model.
  • Preserve compatibility: Non-routed footer attribution remains unchanged; regenerate the audit and logs JSON schemas.

Example:

copilot · routed: gpt56 xhigh · …
<!-- gh-aw-agentic-workflow: …, model: gpt-5.6-luna, effort: xhigh, routed: true, … -->

@SivaKesava1
SivaKesava1 marked this pull request as ready for review October 8, 2026 18:32
Copilot AI balanced review requested due to automatic review settings October 8, 2026 18:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review any files in this pull request. Check if the Files changed in this pull request are included in default exclusions.

@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

✅ Test Quality Sentinel completed test quality analysis.

No test files were added or modified in this PR. Test Quality Sentinel skipped.

🧪 Test quality analysis by Test Quality Sentinel

Copilot AI changed the title [WIP] Fix model routing reporting for generated artifacts Report AWF-routed model and effort across workflow outputs Oct 8, 2026
Copilot AI requested a review from SivaKesava1 October 8, 2026 18:55
Comment thread actions/setup/js/model_attribution.cjs
@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot address the following outstanding work in one pass:

  1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.
  2. Review (actions/setup/js/model_attribution.cjs:28): resolveAwInfoPath makes <tmp>/agent/aw_info.json take precedence over the runner-written <tmp>/aw_info.json for both getFallbackModel and getModelRouting. /tmp/gh-aw/agent/ is the agent's own output/artifact directory, so a prompt-injected agent can drop an aw_info.json there containing an arbitrary fallback_model or model_routing: {status: "selected", wire_model: ...}; the post-run step then reads that file in resolveEffectiveModel, and the forged value... - Report AWF-routed model and effort across workflow outputs #66963 (comment)
  3. Fix failing check impacted-js-tests (FAILURE): https://github.com/github/gh-aw/actions/runs/37828018918/job/113499266571.
  4. Fix failing check js-typecheck (FAILURE): https://github.com/github/gh-aw/actions/runs/37828018918/job/113499266382.
  5. Fix failing check lint-go-golangci (1) (FAILURE): https://github.com/github/gh-aw/actions/runs/37828019044/job/113499272190.
  6. Fix failing check lint-go-golangci (2) (FAILURE): https://github.com/github/gh-aw/actions/runs/37828019044/job/113499272277.

Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI.

Sous-chef head: 1c32a32
Sous-chef work: 12e2d92b8c5b535f1d3147e14962e78f9a50d376e8a16f13e4326968dc5aa3f5 20c5fb9e2e37672daf5b7271dae9c95ff59f03cf41d0cde5ac7e2acc9456d2d1 7d72f19bdd83c76851f4c80d370e3b4b82a3e3dd13e124d6c9371f25bcb6d2cd 7fdcb480e7a98cc7e0f466a49ad1c7f7e8c5cfc22f8fbcc711724ed335220761 9eb0f4a255dfff57cc1ba1801cd9c7f6c0cb12fbc2ef9f6a3416ee19e0b7164e
Sous-chef state: 9ea883e0ec9f823ded9685cb45bf8cd6860cc8b06b06e26a23c8ee88ac0dfbb2

Generated by 👨‍🍳 PR Sous Chef · pi · haiku45 · 7.98 AIC · ⌖ 15 AIC · ⊞ 1K · ◷
Comment /souschef to run again

@SivaKesava1

Copy link
Copy Markdown
Collaborator

@copilot /pr-finisher

Verified at 1c32a329 in our private sandbox (routing, AWF v0.28.44):

  • Copilot: aw_info.json has model: gpt-5.6-luna plus a model_routing block (status selected, source awf-routing, effort high, endpoint /responses, mode economy, router 0.1.3). The created issue's footer reads · copilot · routed: gpt56 high · (it used to read · copilot · auto ·).
  • pi: the same aw_info.json block, and the footer reads · pi · routed: gpt56 high · (it used to read · pi · agent ·).

The Claude engine wasn't tested here (its routed runs also need #66956 and #66957).

Copilot AI and others added 2 commits October 8, 2026 19:50
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot AI commented Oct 8, 2026

Copy link
Copy Markdown
Contributor Author

@copilot address the following outstanding work in one pass:

  1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes....

Merged the latest main and pushed fixes in 5274af0. The runner-only attribution source is enforced, JS typecheck and impacted tests pass, and the reported Go lint findings are fixed. Local validation passed; the existing CI results are stale after this push and need a human re-trigger.

Comment thread actions/setup/js/parse_token_usage.cjs Outdated
Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com>

Copilot AI commented Oct 8, 2026

Copy link
Copy Markdown
Contributor Author

@copilot /pr-finisher

Verified at 1c32a329 in our private sandbox (routing, AWF v0.28.44):...

The finisher pass found and fixed a related trust-boundary issue in 5274af0: sandbox-writable reflect data can no longer establish the attributed model without a corroborating proxy routing record. Targeted tests, typecheck, lint, and the final local gate pass. Claude remains unverified here; CI on the latest commit needs a maintainer re-trigger.

@SivaKesava1
SivaKesava1 requested a review from pelikhan October 8, 2026 20:46
@SivaKesava1

Copy link
Copy Markdown
Collaborator

Ready for review. Re-verified at 07791cbb, after the trust-boundary fixes, in our private sandbox (routing, AWF v0.28.44). Copilot and pi routed t01 both still show · routed: gpt56 high · in the created issue's footer, and runner aw_info.json has the model_routing block (selected, awf-routing, gpt-5.6-luna high /responses, economy, router 0.1.3). So attribution still works with the runner-only source and the proxy-log corroboration. Both code-scanning threads are resolved; Copilot fixed both. The Claude engine wasn't covered. CI on 07791cbb needs a re-trigger.

@pelikhan
pelikhan merged commit 5a8304e into main Oct 8, 2026
45 checks passed
@pelikhan
pelikhan deleted the copilot/fix-model-routing-reporting branch October 8, 2026 20:51
@github-actions

github-actions Bot commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

🎉 This pull request is included in a new release.

Release: v0.91.7

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Model routing: aw_info.json, footers, audit/logs and OTEL report the placeholder model (auto/agent) instead of the AWF-routed model and effort

6 participants