Skip to content

[model-resolution] Model resolution audit findings #66468

Description

@github-actions

🔬 Sub-Agent Model Resolution Audit — 2026-10-07

Period: last 2 days · Runs analyzed: 18 · With sub-agents: 9


⚠️ Mismatches Found (24 total)

Workflow Sub-agent Declared Actual Model Status
[aw] Failure Investigator (6h) failure-classifier small claude-sonnet-5 ⚠️ size mismatch
[aw] Failure Investigator (6h) issue-matcher small claude-sonnet-5 ⚠️ size mismatch
[aw] Failure Investigator (6h) cluster-evidence-extractor small claude-sonnet-5 ⚠️ size mismatch
Mismatch details

All 8 sampled runs of [aw] Failure Investigator (6h) resolved every declared small sub-agent to claude-sonnet-5 (large):

  • run 37574472659
  • run 37572644919
  • run 37569726943
  • run 37565285885
  • run 37563931251
  • run 37561000989
  • run 37559283703
  • run 37556039445

Workflow source confirms each sub-agent is declared model: small in .github/workflows/aw-failure-investigator.md (lines 469-506), so this is a runtime resolution mismatch, not a declaration mismatch.


✅ Correct Resolutions (1 sub-agent verified)

Workflow Sub-agent Declared Actual Model
PR Code Quality Reviewer grumpy-coder small gpt-5.4-mini-2026-03-17

📋 Analysis Notes

  • I used agenticworkflows audit for authoritative model usage because agent-stdio.log heuristics can misclassify shell/tool names as model requests.
  • The sampled set also included several workflows with only top-level model: declarations; those were not counted as sub-agent resolution checks.
  • No other sampled sub-agent workflows showed a size-class mismatch in the audited runs.

Generated by 🔬 Daily Sub-Agent Model Resolution Audit · copilot · gpt54mini · 66.5 AIC · ⌖ 2.48 AIC · ⊞ 14.7K · ◷

  • expires on Oct 9, 2026, 9:54 PM UTC-08:00

Activity

  1. github-actions commented on Oct 8, 2026

    @github-actions
    ContributorAuthor

    This issue is being closed as outdated. A newer issue has been created: #66801

    View newer issue


    This action was performed automatically by the Daily Sub-Agent Model Resolution Audit workflow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions