Executive Summary
In the last 180 days, two high-value bug fixes in the sample-replay pipeline and one improvement-tier linter addition delivered 205 total outcome value from 3 accepted PR outcomes. The bug fixes — repairing push_to_pull_request_branch branch derivation and multi-repo checkout manifest lookup — each scored 85 (objective: bug). PR Description Updater carries 100% of attributable delivered value (IE = 0.234 value/AIC). Documentation workflows consumed 35.4M + 1.3M model tokens with zero accepted outcomes — all 4 documentation PRs were closed without merge, representing a significant efficiency gap.
Summary
| Metric |
PRs |
Safe-output Issues |
Combined |
| Analyzed with linked issue (merged/closed/excl. open) |
7 (3 merged / 4 closed) |
0 |
7 |
| Excluded (no linked closing issue) |
993 of 1,000-PR sample |
— |
993 |
| Outcomes mapped to objectives |
7 |
0 |
7 |
| Unmapped outcomes |
0 |
0 |
0 |
| Accepted outcome count |
3 |
0 |
3 |
| Total outcome value |
205 |
0 |
205 |
| AI Credits |
876.3 AIC (3 of 7 runs; 4 missing) |
n/a |
876.3 AIC |
| Impact Efficiency |
0.234 value/AIC (partial) |
n/a |
0.234 |
What We Worked On
bug — Value: 85 · 2 outcomes · Total: 170 · AIC: 486.4 · IE: 0.350
- PR #37863 — Fix
push_to_pull_request_branch always derives from PR head ref → root #37835 (bug, workflows, compiler)
- PR #37565 — Fix multi-repo checkout manifest-first lookup → root #37545 (
bug, workflows, compiler)
⚠️ Highest AIC category: bug — 486.4 AIC, IE = 0.350. Highest return per credit.
improvement — Value: 35 · 1 outcome · Total: 35 · AIC: 389.9* · IE: ~0.090*
- PR #38185 —
[linter-miner] Add execcommandwithoutcontext Go analyzer → root #38143 (improvement, quick-win, automation)
*AIC shown is from the PR Description Updater description-update run, not the linter-miner creating run (AIC unknown). True IE uncertain.
Which Workflows Drove That Impact
1. PR Description Updater (pr-description-caveman) — 205 value, 100% share, IE 0.234
⚠️ Attribution caveat: PR #38185 was created by github-actions[bot] / linter-miner workflow; PRs #37863 and #37565 were created by a human contributor. PR Description Updater only updated their descriptions.
2. Daily Documentation Healer — 0 value, ~35.4M tokens (non-std AIC), IE 0
3 rejected PRs: #36767 (root #36275), #36371 (root #34686), #35330 (root #34686). PRs #36371 and #35330 both targeted the same root issue — duplicated effort, both rejected.
3. Daily Documentation Updater — 0 value, ~1.3M tokens (non-std AIC), IE 0
1 rejected PR: #35906 (root #35506).
Key callouts: Most value + largest share + most standard AIC: PR Description Updater. Most non-standard AIC with zero return: Daily Documentation Healer (35.4M tokens, 3 runs, 0 accepted).
Top Outcomes by Outcome Value
| Outcome |
Workflow |
Type |
Root / Objective |
Obj. Value |
Outcome Value |
| PR #37863 Fix push_to_pull_request_branch |
PR Description Updater |
PR merged |
#37835 bug |
85 |
85 |
| PR #37565 Fix multi-repo checkout lookup |
PR Description Updater |
PR merged |
#37545 bug |
85 |
85 |
| PR #38185 Add execcommandwithoutcontext linter |
PR Description Updater* |
PR merged |
#38143 improvement |
35 |
35 |
| PR #36767 Document undocumented frontmatter fields |
Daily Documentation Healer |
PR closed |
#36275 improvement |
35 |
0 |
| PR #36371 Fix broken doc-link anchors (attempt 1) |
Daily Documentation Healer |
PR closed |
#34686 improvement |
35 |
0 |
| PR #35330 Fix broken doc-link anchors (attempt 2) |
Daily Documentation Healer |
PR closed |
#34686 improvement |
35 |
0 |
| PR #35906 Document user-invokable frontmatter field |
Daily Documentation Updater |
PR closed |
#35506 improvement |
35 |
0 |
Unmapped Outcomes
None. All 7 analyzed outcomes had root issues with labels matching .github/objective-mapping.json. Labels cookie and deep-report present on some issues are not in the mapping and were ignored.
Interpretation
Accepted outcome count (3/7, 43%) is a poor cost signal — it doesn't capture the cost of 4 failed documentation PRs. Impact Efficiency (0.234 value/AIC, partial) better reflects value-per-credit:
- 🏅 Best IE:
bug category at 0.350 — the two sample-replay reliability fixes returned the most value per AIC spent.
- ⚠️ Zero-return AIC: Documentation workflows consumed 35.4M + 1.3M model tokens (non-standard units) with all PRs rejected. The Daily Documentation Healer opened two PRs for the same root issue (#34686), both closed without merge — suggesting a systematic mismatch between what the workflow produces and what maintainers accept.
- 📌 Attribution gap: PR Description Updater appears as the sole workflow contributor, but it only rewrites PR descriptions. The actual value-producing workflows (linter-miner, human contributor dsyme) are not separately tracked in available AIC data.
Data Quality
| Area |
Status |
| PR root tracing |
3/500 merged (0.6%), 4/500 closed (0.8%) had linked issues. Dataset capped at 500 each. |
| Safe-output label coverage |
N/A — 0 safe-output outcomes in window. |
| Workflow attribution |
PR #38185 body shows PR Description Updater but was created by linter-miner (title [linter-miner], creator github-actions[bot]). PRs #37863/#37565 created by human dsyme — technically outside "PRs created by workflow runs" scope. |
| AIC availability |
workflow-logs.json empty (0 bytes). AIC extracted from PR body footers (live fallback). 3 runs: 876.3 AIC (standard). 4 runs: non-standard model-token units (M format, not convertible to AIC without a rate). 4 of 7 runs (57%) have missing standard AIC. |
| AIC source |
Live fallback (PR body footer extraction) — not deterministic precomputed data. |
Generated by 📊 Impact Efficiency Report · 436.5 AIC · ⌖ 14.4 AIC · ⊞ 23.1K · ◷
Executive Summary
In the last 180 days, two high-value bug fixes in the sample-replay pipeline and one improvement-tier linter addition delivered 205 total outcome value from 3 accepted PR outcomes. The bug fixes — repairing
push_to_pull_request_branchbranch derivation and multi-repo checkout manifest lookup — each scored 85 (objective:bug). PR Description Updater carries 100% of attributable delivered value (IE = 0.234 value/AIC). Documentation workflows consumed 35.4M + 1.3M model tokens with zero accepted outcomes — all 4 documentation PRs were closed without merge, representing a significant efficiency gap.Summary
What We Worked On
bug— Value: 85 · 2 outcomes · Total: 170 · AIC: 486.4 · IE: 0.350push_to_pull_request_branchalways derives from PR head ref → root #37835 (bug,workflows,compiler)bug,workflows,compiler)improvement— Value: 35 · 1 outcome · Total: 35 · AIC: 389.9* · IE: ~0.090*[linter-miner]AddexeccommandwithoutcontextGo analyzer → root #38143 (improvement,quick-win,automation)Which Workflows Drove That Impact
1. PR Description Updater (
pr-description-caveman) — 205 value, 100% share, IE 0.234bug(170),improvement(35)github-actions[bot]/ linter-miner workflow; PRs #37863 and #37565 were created by a human contributor. PR Description Updater only updated their descriptions.2. Daily Documentation Healer — 0 value, ~35.4M tokens (non-std AIC), IE 0
3 rejected PRs: #36767 (root #36275), #36371 (root #34686), #35330 (root #34686). PRs #36371 and #35330 both targeted the same root issue — duplicated effort, both rejected.
3. Daily Documentation Updater — 0 value, ~1.3M tokens (non-std AIC), IE 0
1 rejected PR: #35906 (root #35506).
Key callouts: Most value + largest share + most standard AIC: PR Description Updater. Most non-standard AIC with zero return: Daily Documentation Healer (35.4M tokens, 3 runs, 0 accepted).
Top Outcomes by Outcome Value
bugbugimprovementimprovementimprovementimprovementimprovementUnmapped Outcomes
None. All 7 analyzed outcomes had root issues with labels matching
.github/objective-mapping.json. Labelscookieanddeep-reportpresent on some issues are not in the mapping and were ignored.Interpretation
Accepted outcome count (3/7, 43%) is a poor cost signal — it doesn't capture the cost of 4 failed documentation PRs. Impact Efficiency (0.234 value/AIC, partial) better reflects value-per-credit:
bugcategory at 0.350 — the two sample-replay reliability fixes returned the most value per AIC spent.Data Quality
[linter-miner], creatorgithub-actions[bot]). PRs #37863/#37565 created by humandsyme— technically outside "PRs created by workflow runs" scope.workflow-logs.jsonempty (0 bytes). AIC extracted from PR body footers (live fallback). 3 runs: 876.3 AIC (standard). 4 runs: non-standard model-token units (M format, not convertible to AIC without a rate). 4 of 7 runs (57%) have missing standard AIC.