Skip to content

Daily Sub-Agent Model Resolution Audit #111

Daily Sub-Agent Model Resolution Audit

Daily Sub-Agent Model Resolution Audit #111

Triggered via schedule October 10, 2026 05:33
Status Success
Total duration 26m 20s
Artifacts 7
Fit to window
Zoom out
Zoom in

Annotations

16 errors, 1 warning, and 2 notices
agent
" source=\"$root/api-proxy-logs/token-usag... (+ 82 more)
agent
shared audit types and state used across the audit_*.go files (+ 82 more)
agent
rendering of the final audit report/output and (+ 177 more)
agent
cobra command definition, flag parsing, and dispatch (+ 238 more)
agent
%[1]s logs --parse ...
agent
` + string(constants.CL...
agent
shared audit types and state used across the audit_*.go files\n // (command wiring, run pipeline, analysis fan-out, summary building, and\n // report rendering).\n \n import (\n \t\"github.com/github/gh-aw/pkg/logger\"\n )\n \n var auditLog = logger.New(\"cli:audit\")\n \n // AuditOptions contains shared options for audit and audit-diff execution.\n type AuditOptions struct {\n \tOwner string\n \tRepo string\n \tHostname string\n \tOutputDir string\n \tVerbose bool\n \tParse bool\n \tJSONOutput bool\n \tJobID int64\n \tStepNumber int\n \tFormat string\n \tArtifactSets []string\n \tExperimentFilter string\n \tVariantFilter string\n \tRuntimeFilter string\n \tEvalsOnly bool\n \tGroup bool\n \tNoBaseline bool\n }\n \n type auditRunConfig struct {\n \trunID int64\n \towner string\n \trepo string\n \thostname string\n \toutputDir string\n \tverbose bool\n \tparse bool\n \tjsonOutput bool\n \tjobID int64\n \tstepNumber int\n \tartifactFilter []string\n \tincludeWorkQueue bool\n \texperimentFilter string\n \tvariantFilter string\n \truntimeFilter string\n \tevalsOnly bool\n \tgroup bool\n \tnoBaseline bool\n \t// evalsArtifactRequested is true when evals were requested via --evals or\n \t// explicit --artifacts evals, and is used to trigger legacy dedicated-evals\n \t// fallback behavior for older runs.\n \tevalsArtifactRequested bool\n }\n \n // auditAnalysisResults is populated concurrently during audit collection.\n // Each field is written by exactly one goroutine (or one launch* helper that\n // exclusively owns a disjoint field set) before collectAuditAnalysisResults\n // waits on the shared errgroup and reads the aggregate result.\n //\n // Keep launch* helper field ownership disjoint: sharing a field between\n // goroutines without adding synchronization would introduce a data race.\n type auditAnalysisResults struct {\n \tmetrics LogMetrics\n \tfailedJobCount int\n \tjobDetails []JobInfoWithDuration\n \tmissingTools []MissingToolReport\n \tmissingData []MissingDataReport\n \tnoops []NoopReport\n \tmcpFailures []MCPFailureReport\n \tskillActivations []SkillActivation\n \tgatewaySteeringEvents []GatewaySteeringEvent\n \taccessAnalysis *DomainAnalysis\n \tfirewallAnalysis *FirewallAnalysis\n \tpolicyAnalysis *PolicyAnalysis\n \tmcpToolUsage *MCPToolUsageData\n \ttokenUsageSummary *TokenUsageSummary\n \tmodelRouting *ModelRoutingSummary\n \tworkingSet *WorkingSetMetrics\n \tledger *LedgerActivity\n \tworkQueue *WorkQueueReport\n \tfriction *FrictionCostSummary\n \tredactedDomainsAnalysis *RedactedDomainsAnalysis\n \trateLimitUsage *GitHubRateLimitUsage\n \tartifacts []string\n \tsafeItemsCount int\n }\n \n"}}}
agent
rendering of the final audit report/output and\n // post-processing of agent and firewall logs.\n \n import (\n \t\"context\"\n \t\"fmt\"\n \t\"os\"\n \t\"path/filepath\"\n \n \t\"github.com/github/gh-aw/pkg/console\"\n \t\"github.com/github/gh-aw/pkg/fileutil\"\n )\n \n // renderAuditReport builds and renders the audit report from a fully-populated processedRun.\n // It is called both when serving from a cached run summary and after a fresh processing pass,\n // ensuring that the two paths produce identical output.\n func renderAuditReport(ctx context.Context, processedRun ProcessedRun, metrics LogMetrics, mcpToolUsage *MCPToolUsageData, opts AuditOptions) error {\n \trunID := processedRun.Run.DatabaseID\n \trunOutputDir := opts.OutputDir\n \tauditLog.Printf(\"Rendering audit report for run %d (noBaseline=%t, group=%t)\", runID, opts.NoBaseline, opts.Group)\n \tprocessedRun.Run.SafeItemsCount = len(resolveCreatedItems(runOutputDir, processedRun.SafeOutputs))\n \tauditData, ok := loadCachedAuditData(runOutputDir, processedRun.Run, auditCacheSourceFull)\n \tif ok {\n \t\t// A cached entry always carries the default (baseline) comparison, so drop it\n \t\t// when the caller opted out of baseline comparison.\n \t\tif opts.NoBaseline {\n \t\t\tauditData.Comparison = noBaselineComparison()\n \t\t}\n \t} else {\n \t\tauditData = buildRenderedAuditDataFromCache(ctx, processedRun, metrics, mcpToolUsage, runOutputDir, opts)\n \t\tauditData.CacheSource = auditCacheSourceFull\n \t\tauditData.SchemaVersion = auditSchemaVersion\n \t\t// Never persist an opt-out result: it would be reused as the full cache entry\n \t\t// by a later default audit and suppress its baseline comparison.\n \t\tif !opts.NoBaseline {\n \t\t\tif err := writeAuditData(runOutputDir, auditData); err != nil {\n \t\t\t\treturn err\n \t\t\t}\n \t\t}\n \t}\n \tif auditData.Ledger == nil {\n \t\tauditData.Ledger = processedRun.Ledger\n \t}\n \t// NoBaseline runs avoid writing audit-cache changes, including queue backfills.\n \tif mergeWorkQueueReport(&auditData.WorkQueue, processedRun.WorkQueue) && !opts.NoBaseline {\n \t\tif err := writeAuditData(runOutputDir, auditData); err != nil {\n \t\t\treturn err\n \t\t}\n \t}\n \tif opts.Group {\n \t\tparseAuditLogsIfRequested(runID, runOutputDir, opts)\n \t\treturn nil\n \t}\n \tif err := renderAuditOutput(auditData, runOutputDir, opts.JSONOutput, opts.Verbose); err != nil {\n \t\treturn err\n \t}\n \trenderAuditGatewayMetrics(runOutputDir, opts.Verbose)\n \trenderAuditUnifiedTimeline(runOutputDir, opts.Verbose)\n \tparseAuditLogsIfRequested(runID, runOutputDir, opts)\n \trenderAuditCompletion(runOutputDir, opts.JSONOutput)\n \treturn nil\n }\n \n func buildRenderedAuditDataFromCache(ctx context.Context, processedRun ProcessedRun, metrics LogMetrics, mcpToolUsage *MCPToolUsageData, runOutputDir string, opts AuditOptions) AuditData {\n \tauditData, ok := loadCachedAuditData(runOutputDir, processedRun.Run, auditCacheSourceLogs)\n \tif !ok {\n \t\treturn buildRenderedAuditData(ctx, processedRun, metrics, mcpToolUsage, runOutputDir, opts)\n \t}\n \tif auditData.Ledger == nil {\n \t\tauditData.Ledger = processedRun.Ledger\n \t}\n \tmergeWorkQueueReport(&auditData.WorkQueue, processedRun.WorkQueue)\n \tcreatedItems := resolveCreatedItems(runOutputDir, processedRun.SafeOutputs)\n \taddAuditOutcomeSummary(ctx, &auditData, createdItems)\n \tauditData.Comparison = buildRenderedAuditComparison(ctx, processedRun, createdItems, runOutputDir, opts)\n \treturn auditData\n }\n \n func buildRenderedAuditData(ctx context.Context, processedRun ProcessedRun, metrics LogMetrics, mcpToolUsage *MCPToolUsageData, runOutputDir string, opts AuditOptions) AuditData {\n \tcurrentCreatedItems := resolveCreatedItems(runOutputDir, processedRun.SafeOutputs)\n \tauditData := buildAuditData(ctx, processedRun, metrics, mcpToolUsage)\n \tauditData.Comparison = buildRenderedAuditComparison(ctx, processedRun, currentCreatedItems, runOutputDir, opts)\n \treturn auditData\n }\n \n func buildRenderedAuditComparison(ctx context.Context, processedRun ProcessedRun, c
agent
\tif opts.jsonOutput || opts.format == \"json\" {"}}}
agent
%[1]s logs --parse # Parse logs and generate Markdown reports"}}}
agent
cobra command definition, flag parsing, and dispatch\n // between single-run audit and multi-run diff mode.\n \n import (\n \t\"context\"\n \t\"errors\"\n \t\"fmt\"\n \t\"os\"\n \t\"strings\"\n \n \t\"github.com/github/gh-aw/pkg/console\"\n \t\"github.com/github/gh-aw/pkg/constants\"\n \t\"github.com/github/gh-aw/pkg/logger\"\n \t\"github.com/github/gh-aw/pkg/parser\"\n \t\"github.com/spf13/cobra\"\n )\n \n var auditCommandLog = logger.New(\"cli:audit_command\")\n \n var auditCommandLong = `Audit one or more workflow runs by downloading artifacts and logs, detecting errors,\n analyzing MCP tool usage, and generating a concise report suitable for AI agents.\n \n When a single run is provided, generates a detailed Markdown report for that run.\n When two or more runs are provided, the first is used as the base (reference) and the\n remaining runs are compared against it, producing a diff report.\n \n Each argument accepts:\n - A numeric run ID (e.g., 1234567890)\n - A GitHub Actions run URL (e.g., https://github.com/owner/repo/actions/runs/1234567890)\n - A GitHub Actions job URL (e.g., https://github.com/owner/repo/actions/runs/1234567890/job/9876543210)\n - A GitHub Actions job URL with step (e.g., https://github.com/owner/repo/actions/runs/1234567890/job/9876543210#step:7:1)\n - A GitHub workflow run URL (e.g., https://github.com/owner/repo/runs/1234567890)\n - GitHub Enterprise URLs (e.g., https://github.example.com/owner/repo/actions/runs/1234567890)\n \n When a job URL is provided (single-run mode only):\n - If a step number is included (#step:7:1), extracts that specific step's output\n - If no step number, finds and extracts the first failing step's output\n - Saves job logs to the output directory`\n \n var auditCommandExample = ` ` + string(constants.CLIExtensionPrefix) + ` audit 1234567890 --repo owner/repo # Audit with bare run ID (--repo required)\n ` + string(constants.CLIExtensionPrefix) + ` audit https://github.com/owner/repo/actions/runs/1234567890 # Audit from run URL\n ` + string(constants.CLIExtensionPrefix) + ` audit https://github.com/owner/repo/actions/runs/1234567890/job/9876543210 # Audit job and extract first failing step\n ` + string(constants.CLIExtensionPrefix) + ` audit https://github.com/owner/repo/actions/runs/1234567890/job/9876543210#step:7:1 # Extract step 7 output\n ` + string(constants.CLIExtensionPrefix) + ` audit https://github.com/owner/repo/runs/1234567890 # Audit from workflow run URL\n ` + string(constants.CLIExtensionPrefix) + ` audit https://github.example.com/owner/repo/actions/runs/1234567890 # Audit from GitHub Enterprise\n ` + string(constants.CLIExtensionPrefix) + ` audit 1234567890 -o ./audit-reports # Custom output directory\n ` + string(constants.CLIExtensionPrefix) + ` audit 1234567890 -v # Verbose output\n ` + string(constants.CLIExtensionPrefix) + ` audit 1234567890 --parse # Parse agent logs and firewall logs, generating log.md and firewall.md\n ` + string(constants.CLIExtensionPrefix) + ` audit 1234567890 --repo owner/repo # Audit run from a specific repository\n ` + string(constants.CLIExtensionPrefix) + ` audit 1234567890 1234567891 # Diff two runs (base vs comparison)\n ` + string(constants.CLIExtensionPrefix) + ` audit 1234567890 1234567891 1234567892 # Diff base against multiple runs\n ` + string(constants.CLIExtensionPrefix) + ` audit 1234567890 1234567891 --format markdown # Markdown diff output for PR comments\n ` + string(constants.CLIExtensionPrefix) + ` audit 1234567890 1234567891 --group # Group findings by run and audit code\n ` + string(constants.CLIExtensionPrefix) + ` audit 1234567890 --runtime cloud-hypervisor # Skip run unless sandbox agent runtime matches`\n \n type auditCommandOptions struct {\n \toutputDir string\n \tverbose bool\n \tjsonOutput bool\n \tparse bool\n \trepoFlag string\n \tformat string\n \tartifacts []string\n \tstdin bool\n \texperimentFilter string\n \tvariantFilter string\n \truntim
agent
gonum.org/v1/gonum@v0.17.0: Get \"https://proxy.golang.org/gonum.org/v1/gonum/@v/v0.17.0.zip\": tls: failed to verify certificate: x509: certificate is not valid for any names, but wanted to match proxy.golang.org\n"}]}}}
agent
\tRunID string `json:\"run_id\"`\n\nmcp_server_operations_test.go (2 match(es)):\n \t\t\t\"run_id_or_url\": \"1\",\n \t\t\t\t\t\t\"run_id_or_url\": \"1\",\n\nlogs_parsing_test.go (3 match(es)):\n \t\t\"run_id\": 123456789,\n \t\t\"run_id\": 987654321,\n \t\t\"run_id\": 123456789\n\n[omitted 230 match(es) in 89 file(s); see original output for full results]"}}}
agent
` + string(constants.CLIExtensionPrefix) + ` audit https://github.com/owner/repo/actions/runs/1234567890/job/9876543210#step:7: 1 # Extract step 7 output"}}}
agent
gonum.org/v1/gonum@v0.17.0: Get \"https://proxy.golang.org/gonum.org/v1/gonum/@v/v0.17.0.zip\": tls: failed to verify certificate: x509: certificate is not valid for any names, but wanted to match proxy.golang.org\n"}]}}}
agent
\t\tTools: []string{\"compile\", \"audit\", \"logs\", \"inspect\", \"status\", \"audit-diff\", \"work-queue\"},"}}}
agent
Skipping inaccessible directory during secret redaction: /tmp/gh-aw/aw-mcp
agent
"The ubuntu-latest label will migrate to Ubuntu 26 beginning October 19, 2026. For more information, see https://github.com/actions/runner-images/issues/14748"
detection
"The ubuntu-latest label will migrate to Ubuntu 26 beginning October 19, 2026. For more information, see https://github.com/actions/runner-images/issues/14748"

Artifacts

Produced during runtime
Name Size Digest
agent
33.9 MB
sha256:3e53cedbe7b1f70e5c337cee28f6466b151f2060db6222b3996e220b2b200fdc
agent-output-fallback
19.3 KB
sha256:9c7446b511691c66b7367c9ce5963f647bed87396a80b8d91a9b1a98ec4e8bae
detection
20.6 KB
sha256:957ee6ae83f55092411fb06884bad4113bee52d2d484c7808298840d390a7d02
github~gh-aw~XQM15Q.dockerbuild
21.7 KB
sha256:d98c57a2eafb1799a9de687f6f1cdb02f5dc74b483b2bb2e87fc8e6e925a0753
info
704 Bytes
sha256:3ca8537516d92de67bc75ff490b8f31be18d05cafc9b2264e64b1f418f045206
safe-outputs-items
597 Bytes
sha256:2b5b8bd1b7ee4d47865509854c1570e2a06cb89dd0501cfe1de02393b37b8c7c
usage
772 KB
sha256:2c44cfcad96e0a7899e1848b63395d602cda58decca77703502f523b263f9e4c