Skip to content

Commit 4f7ee4b

Browse files
authored
feat: report dispatch coordinator operations in logs and audit (#65163)
1 parent 46e10e2 commit 4f7ee4b

38 files changed

Lines changed: 1829 additions & 204 deletions
Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
# ADR-65163: Report Dispatch Coordinator Operations in Audit and Logs
2+
3+
**Date**: 2026-10-03
4+
**Status**: Draft
5+
**Deciders**: gh-aw maintainers (review pending)
6+
7+
---
8+
9+
### Context
10+
11+
The dispatcher work coordinator already records evidence across activation snapshots, agent finish-intent files, and workflow job logs, but that evidence is split across artifacts and is difficult to inspect from a single `gh aw` report. This pull request extends both `gh aw logs` and `gh aw audit` so operators can investigate queue publication and worker reconciliation without manually correlating multiple downloaded files. The change must preserve the existing distinction between activation-time queue facts and run-time coordinator operations, keep compact `logs` defaults unchanged, and work with cached summaries so older downloads can be refreshed when coordinator evidence is incomplete.
12+
13+
### Decision
14+
15+
We will introduce a first-class `work_queue` report section backed by a `DispatchCoordinatorReport` extracted from activation artifacts, agent finish-intent artifacts, and workflow logs. We will add a dedicated `work-queue` artifact set for `gh aw logs`, include the same evidence in default `gh aw audit`, and persist the resulting coordinator data through audit and cached-logs schema updates so cached reports can be regenerated or backfilled when workflow logs are missing. We will treat activation snapshot transactions as historical queue facts, `finish_intent` as an unverified agent request, and workflow-log operations as separate publication and reconciliation diagnostics with source-file provenance.
16+
17+
Artifact selection is preserved explicitly rather than inferred from expanded artifact names: `github-api` and `work-queue` download the same artifact names but have different reporting intent. Usage-only logs do not expose previously cached work-queue evidence. Coordinator operations are read only from compiler-owned step log paths, excluding agent execution and whole-job copies. Workflow logs are staged and published with a completion marker after successful extraction; incomplete directories are not accepted as evidence. Required snapshot fields are checked for presence even when nullable, and text rendering escapes control characters without changing JSON values.
18+
19+
### Alternatives Considered
20+
21+
#### Alternative 1: Keep coordinator evidence only in raw artifacts
22+
23+
This would avoid new report fields, artifact-set plumbing, and cache-schema changes. It was not chosen because the PR's stated goal is to make queue publication and worker reconciliation investigable from a single logs or audit report, which raw artifact spelunking does not provide.
24+
25+
#### Alternative 2: Add coordinator reporting only to `audit`
26+
27+
Limiting the feature to `gh aw audit` would reduce the surface area of the change and avoid adding a new `logs` artifact set. It was not chosen because the PR explicitly adds `gh aw logs <run-id> --artifacts work-queue --json` support and updates cached logs output, indicating that operators need the same evidence in logs workflows as well as in audit workflows.
28+
29+
### Consequences
30+
31+
#### Positive
32+
- Operators can inspect queue snapshots, assigned worker identifiers, finish intents, and timestamped coordinator operations from one structured `work_queue` section.
33+
- `gh aw logs` gains an explicit `work-queue` download mode while `gh aw audit` includes the evidence by default, improving discoverability without changing compact logs defaults.
34+
- Cached audit and logs summaries retain coordinator evidence and can backfill partial older reports when artifacts or workflow logs were previously missing.
35+
36+
#### Negative
37+
- The implementation increases cache schema versions and adds more coupling between artifact download logic, workflow-log availability, and report rendering.
38+
- Coordinator reporting now depends on parsing and validating multiple evidence sources, including workflow log text patterns that may require maintenance if runtime messages change.
39+
- Downloading coordinator evidence for logs requires workflow logs in addition to activation and agent artifacts, which can increase report generation cost compared with usage-only logs.
40+
41+
#### Neutral
42+
- The decision extends the existing dispatch-work-coordinator architecture rather than changing queue authority or protocol semantics.
43+
- The report intentionally separates historical queue facts (`snapshot.transactions`) from run-specific operational diagnostics (`operations`), preserving the existing mental model for coordinator evidence.
44+
45+
---
46+
47+
*ADR created by [adr-writer agent]. Review and finalize before changing status from Draft to Accepted.*

‎docs/src/content/docs/reference/audit.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,17 @@ The Friction section attributes execution-friction costs by driver and source. I
8484

8585
The Gateway Steering Events section reports `token_steering` and `timeout_steering` warnings emitted when a run approaches its AI Credits or time limit. JSON output includes each event's type, message, and timestamp when available.
8686

87+
The Work Queue section reports the activation-time queue snapshot, the assigned worker's work and claim IDs, the agent's finish intent, and timestamped coordinator operations from workflow job logs. In JSON, `work_queue.snapshot.transactions` contains the queue facts captured at activation, while `work_queue.operations` contains this run's publication and reconciliation diagnostics with their source files. A `finish_intent` is a request, not evidence that completion was verified. The field is omitted when no coordinator evidence is available.
88+
89+
Operations are accepted only from the compiler-owned snapshot, finish-intent-copy, and claim-reconciliation step logs, not from agent stdout or whole-job log copies. Incomplete workflow-log downloads are retried before their operations are reported. Text reports escape control characters in stored identifiers.
90+
91+
`audit` downloads this evidence by default. For `logs`, which defaults to compact usage downloads, select `work-queue` (activation and agent artifacts plus workflow logs) or `all`. Both commands include `work_queue` in their JSON reports, per run for `logs`, and preserve it in cached summaries. Usage-only logs omit the section even when a previous request cached work-queue evidence:
92+
93+
```bash
94+
gh aw logs 1234567890 --artifacts work-queue --json
95+
gh aw audit 1234567890 --artifacts work-queue --json
96+
```
97+
8798
The Observability Insights section includes `skill_activations` when skill-invocation evidence is found. Each entry reports the skill name, `status` (`invoked`), the detection `source` (`agent_output` or `log_parse`), and provenance fields in JSON output. This makes it possible to distinguish skills that were merely restored or installed from skills that were actually invoked during the run.
8899

89100
The Graders section is present when the run recorded deterministic grader results (`graders` declared in the workflow frontmatter). The `graders` object in JSON output lists each grader (`id`, `name`, `status`, `value`, `unit`, `passed`, and, when declared in the grader manifest, `direction` and `threshold`) plus aggregate counts: `total`, `passed`, `failed`, `error_count`, and `unavailable_count`. Grader results are read from the compact `usage` artifact (mirrored there by the conclusion job), the unified `agent` artifact, or the `agent-output-fallback` artifact, so they are available even when `--artifacts usage` narrows the download. The same `graders` object is included per run in `gh aw logs --json` output.

‎docs/src/content/docs/setup/cli.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -598,7 +598,7 @@ gh aw audit 1234567890 --runtime cloud-hypervisor # Skip run unless sandb
598598

599599
The `--repo` flag accepts `owner/repo` format and is required when passing a bare numeric run ID without a full URL, allowing the command to locate the correct repository.
600600

601-
The `--artifacts` flag selects which artifact sets to download (default: `all`). Valid sets include `activation`, `agent`, `all`, `detection`, `evals`, `experiment`, `firewall`, `github-api`, `graders`, `info`, `mcp`, and `usage`. Use `all` to download the full artifact set. Unlike `gh aw logs`, which defaults to `usage`, `audit` defaults to `all` for comprehensive analysis. The `--experiment` flag filters to runs that include the named experiment; `--variant` further restricts to a specific variant value and requires `--experiment` to be set. The `--output/-o` flag overrides the output directory.
601+
The `--artifacts` flag selects which artifact sets to download (default: `all`). Valid sets include `activation`, `agent`, `all`, `detection`, `evals`, `experiment`, `firewall`, `github-api`, `graders`, `info`, `mcp`, `usage`, and `work-queue`. The `work-queue` set downloads queue snapshots, worker finish intents, and workflow logs for coordinator operation reporting. Use `all` to download the full artifact set. Unlike `gh aw logs`, which defaults to `usage`, `audit` defaults to `all` for comprehensive analysis. The `--experiment` flag filters to runs that include the named experiment; `--variant` further restricts to a specific variant value and requires `--experiment` to be set. The `--output/-o` flag overrides the output directory.
602602

603603
Logs are saved to `.github/aw/logs/run-{id}/` with filenames indicating the extraction level. Pre-agent failures (integrity filtering, missing secrets, binary install) surface the actual error in `failure_analysis.error_summary`. Invalid run IDs return a human-readable error.
604604

‎pkg/cli/audit.go‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@ type auditRunConfig struct {
4343
jobID int64
4444
stepNumber int
4545
artifactFilter []string
46+
includeWorkQueue bool
4647
experimentFilter string
4748
variantFilter string
4849
runtimeFilter string
@@ -79,6 +80,7 @@ type auditAnalysisResults struct {
7980
tokenUsageSummary *TokenUsageSummary
8081
workingSet *WorkingSetMetrics
8182
ledger *LedgerActivity
83+
dispatchCoordinator *DispatchCoordinatorReport
8284
friction *FrictionCostSummary
8385
redactedDomainsAnalysis *RedactedDomainsAnalysis
8486
rateLimitUsage *GitHubRateLimitUsage

‎pkg/cli/audit_analysis_fanout.go‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -207,6 +207,11 @@ func launchFirewallAnalysis(g *errgroup.Group, gctx context.Context, results *au
207207

208208
// launchSupplementalAuditAnalyses exclusively writes redactedDomainsAnalysis, rateLimitUsage, artifacts, and safeItemsCount.
209209
func launchSupplementalAuditAnalyses(g *errgroup.Group, gctx context.Context, results *auditAnalysisResults, runOutputDir string, verbose bool) {
210+
runAuditAnalysis(g, gctx, true, "extractDispatchCoordinatorReport", "Failed to extract dispatch coordinator operations", func(v *DispatchCoordinatorReport) {
211+
results.dispatchCoordinator = v
212+
}, func() (*DispatchCoordinatorReport, error) {
213+
return extractDispatchCoordinatorReport(runOutputDir)
214+
})
210215
runAuditAnalysis(g, gctx, verbose, "analyzeRedactedDomains", "Failed to analyze redacted domains", func(v *RedactedDomainsAnalysis) {
211216
results.redactedDomainsAnalysis = v
212217
}, func() (*RedactedDomainsAnalysis, error) {

‎pkg/cli/audit_cache.go‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ import (
1313

1414
type auditCacheSource string
1515

16-
const auditSchemaVersion = 2
16+
const auditSchemaVersion = 5
1717

1818
const (
1919
auditCacheSourceFull auditCacheSource = "full"
@@ -99,6 +99,9 @@ func writeLogsAuditFile(processedRun ProcessedRun, processedRuns []ProcessedRun,
9999
auditData, _ = buildLocalAuditData(processedRun, metrics, processedRun.MCPToolUsage)
100100
auditData.CacheSource = auditCacheSourceLogs
101101
}
102+
if processedRun.DispatchCoordinator != nil {
103+
auditData.DispatchCoordinator = processedRun.DispatchCoordinator
104+
}
102105
hydratedProcessedRuns := hydrateProcessedRunsWithCachedAudit(processedRuns)
103106
auditData.Comparison = buildAuditComparisonForProcessedRuns(hydrateProcessedRunWithCachedAudit(processedRun), hydratedProcessedRuns)
104107
if err := writeAuditData(runOutputDir, auditData); err != nil {
@@ -141,5 +144,8 @@ func hydrateProcessedRunWithCachedAudit(processedRun ProcessedRun) ProcessedRun
141144
if len(processedRun.GatewaySteeringEvents) == 0 {
142145
processedRun.GatewaySteeringEvents = audit.GatewaySteeringEvents
143146
}
147+
if processedRun.DispatchCoordinator == nil {
148+
processedRun.DispatchCoordinator = audit.DispatchCoordinator
149+
}
144150
return processedRun
145151
}

‎pkg/cli/audit_group_test.go‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -260,6 +260,7 @@ func writeCachedAuditRunFixture(t *testing.T, outputDir string, runID int64, fin
260260
},
261261
}
262262
require.NoError(t, saveRunSummary(runOutputDir, summary, false))
263+
markWorkflowLogsComplete(t, runOutputDir)
263264

264265
auditData := AuditData{
265266
SchemaVersion: auditSchemaVersion,

‎pkg/cli/audit_render_output.go‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,12 @@ func renderAuditReport(ctx context.Context, processedRun ProcessedRun, metrics L
4242
if auditData.Ledger == nil {
4343
auditData.Ledger = processedRun.Ledger
4444
}
45+
// NoBaseline runs avoid writing audit-cache changes, including coordinator backfills.
46+
if mergeDispatchCoordinatorReport(&auditData.DispatchCoordinator, processedRun.DispatchCoordinator) && !opts.NoBaseline {
47+
if err := writeAuditData(runOutputDir, auditData); err != nil {
48+
return err
49+
}
50+
}
4551
if opts.Group {
4652
parseAuditLogsIfRequested(runID, runOutputDir, opts)
4753
return nil
@@ -64,6 +70,7 @@ func buildRenderedAuditDataFromCache(ctx context.Context, processedRun Processed
6470
if auditData.Ledger == nil {
6571
auditData.Ledger = processedRun.Ledger
6672
}
73+
mergeDispatchCoordinatorReport(&auditData.DispatchCoordinator, processedRun.DispatchCoordinator)
6774
createdItems := resolveCreatedItems(runOutputDir, processedRun.SafeOutputs)
6875
addAuditOutcomeSummary(ctx, &auditData, createdItems)
6976
auditData.Comparison = buildRenderedAuditComparison(ctx, processedRun, createdItems, runOutputDir, opts)

0 commit comments

Comments
 (0)