Repository navigation
feat(import): attribute ccusage --by-agent rows to the agent that reported them - #1190
Merged
junhoyeo merged 1 commit intoAug 24, 2026
Merged
Conversation
…orted them A --by-agent export carries an `agents` array per row, one entry per agent with its own `modelBreakdowns`. The row-level breakdowns are a merge of those keyed by model name (ccusage's `aggregate_model_breakdowns`), so a model two agents both used cannot be split back apart from them, and `resolve_client` has to guess by model family. Read the per-agent entries instead: they name the client outright, which also covers the thirteen ccusage agent ids whose models match none of the three families `model_family` knows. Each entry's own `totalTokens` becomes a per-agent cross-check, the counterpart of the row-level one, since a row total that still adds up can hide one agent's buckets being counted twice.
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #1128, which you invited in the merge comment: "
resolve_clientsplits by model family, not by the client that made the call... ccusage's--by-agentoutput does, since each agent gets its ownmodelBreakdowns. That's the natural follow-up if you're interested; I didn't want to build it against a shape I couldn't capture a real sample of."I captured one. This adds
agentssupport to the ccusage importer, so a--by-agentexport is attributed to the client that actually reported each breakdown instead of to whichever listed agent the model family matched.The real sample
npx ccusage@20.0.20 daily --by-agent --json, run against this machine's own Claude Code (~7.5 GB) and Codex (~8.9 GB) local session logs: 34 days, 2026-07-22 to 2026-08-24, 39 agent-day cells, 30,152,041,119 tokens / $29,733.75, five of them multi-agent days.The shape it actually emits, per row:
{ "agent": "all", "period": "2026-08-20", "metadata": { "agents": ["claude", "codex"] }, "modelBreakdowns": [ ... merged across agents ... ], "agents": [ { "agent": "claude", "inputTokens": 210877, "outputTokens": 6845166, "cacheCreationTokens": 42883113, "cacheReadTokens": 1243450521, "totalTokens": 1293389677, "totalCost": 1162.413754899997, "modelsUsed": [...], "modelBreakdowns": [ ... ] }, { "agent": "codex", "...": "..." } ] }What the file proves, checked on all 34 rows:
input + output + cacheCreation + cacheRead == totalTokensholds per agent, not just per row (39/39). So the anchor you added at row level has a per-agent counterpart, and this PR wires it in as a cross-check.modelBreakdownssum exactly to per-agent totals. No tolerance needed on real data.metadatastill carries only the agent name list. Reasoning is not in there (see limitations).Why the row-level breakdown cannot recover the split
This is not inference from the file; it is ccusage's own code.
AllAccumulator::into_rowbuilds the row-levelmodelBreakdownswithaggregate_model_breakdowns, which merges entries keyed by model name across agents (rust/crates/ccusage-adapter-all/src/types.rs). Once two agents have used the same model on the same day, the row-level entry is their sum and nothing in it says so.agentsis the only place the split survives.There is a second, quieter loss.
model_familyknows three families; ccusage'sBUILT_IN_AGENT_NAMESlists sixteen agent ids (claude, codex, opencode, amp, droid, codebuff, hermes, pi, goose, openclaw, kilo, copilot, gemini, kimi, qwen, grok). On a multi-agent day the other thirteen match no family, resolve tounknown, and the import warns that the leaderboard would reject it. All sixteen are already registered tokscaleClientIds, so a--by-agentrow needs no inference at all — just the same spelling normalization the importer already applies.Before / after
Real day pair recomposed as a same-family collision (the token values are real — two real Claude-only days, 2026-08-17 and 2026-08-18 — with the second agent relabelled
opencode, since I have no real OpenCode usage on this machine):Both after-values are what the file declares for each agent.
On the real 34-day export, where
claudeandcodexhave disjoint families and the guess was already right, output is byte-identical to before, and every one of the 39 agent-day cells matches the declaredtotalTokensexactly and the declaredtotalCostto within $0.01. The import now also says which of the two paths produced the numbers:What changed
CcusageDailygainsagents: Vec<CcusageAgentBreakdown>(absent without--by-agent, so old exports are untouched).attribute_agent, which only normalizes the spelling and flags an id tokscale does not register.resolve_clientkeeps its doc comment but is now explicitly the fallback for exports that predate--by-agent.totalTokensbecomes a per-agent cross-check, appended tobreakdown_reconciliation_warningson divergence — a row total that still adds up can hide one agent's buckets being counted twice.is_empty_daynow also looks atagents, so a row carrying only per-agent data is not dropped.add_model_breakdownand is shared by both paths, so they cannot drift.ImportOutcome.agent_attributed_rowscounts rows attributed this way, and the CLI prints the line above.Six tests, all in the existing
mod tests: parity with the family guess when families are disjoint, the per-agent declared-total cross-check, two agents sharing one model, agents the family guess cannot name (amp/droid), an entry with no breakdowns, and the divergence warning.I checked they can actually fail. Reverting the per-agent loop to the family guess turns two of them red:
and disabling the cross-check turns the sixth red on its own.
cargo test --workspace --all-features: 3158 passed / 0 failed.cargo clippy --locked --workspace --all-features -- -D warningsandcargo fmt --all -- --check: clean.Limitations, stated up front
claudeandcodex, and their families are disjoint — so the real file cannot itself exhibit the collision. That case rests on ccusage's merge code plus the tests and the recomposed fixture above. I have no real OpenCode usage and did not invent one.--by-agentimport, unchanged by this PR. ccusage drops it when merging intoagent: "all":row_jsonwritesmetadataas{ "agents": [...] }wheneverrow.metadataisNone, which is always true for unified rows. Confirmed on the real file — all 34 rows carrymetadatawith onlyagents, including the five days Codex ran with real reasoning tokens. The agent entries carry no metadata either, so there is nothing to read.CcusageAgentBreakdownaccepts a flatreasoningOutputTokensif ccusage ever adds one.ccusage codex daily --jsondoes carry it, and the importer already handles that shape (thecostUSD+ flat-reasoning path from feat: add ccusage import format #1128) — I confirmed that on a real capture too.--by-agentalso addsagentsto weekly and monthly rows. The importer only consumesdaily; unchanged and out of scope.ImportOutcomegains a public field. Crate-internal in practice, but it is a struct-literal-breaking change for any out-of-tree construction.AI assistance
This patch was authored with Claude Code by INTFRAME, reviewed before submission, and I will respond to review here myself. Same disclosure as #1128.
Summary by cubic
Attributes
ccusage--by-agentexports to the agent that reported each breakdown, removing the model-family guess when the export provides per-agent data. Old behavior: attribute per-model usage by inferred family across listed agents. New behavior: attribute by the row’sagentsentries; fall back to family inference only whenagentsis absent.daily[].agents[]and maps each entry via exact agent id normalization; unknown ids are still flagged.breakdown_reconciliation_warnings.--by-agent; per-model mapping is shared to prevent drift.Migration
agent_attributed_rows.Written for commit 2d74df5. Summary will update on new commits.