Skip to content

feat(import): attribute ccusage --by-agent rows to the agent that reported them - #1190

Merged
junhoyeo merged 1 commit into
junhoyeo:mainfrom
aron-intframe:feat/ccusage-by-agent-attribution
Aug 24, 2026
Merged

junhoyeo merged 1 commit into
junhoyeo:mainfrom
aron-intframe:feat/ccusage-by-agent-attribution

Conversation

@aron-intframe

@aron-intframe aron-intframe commented Aug 24, 2026 •

Copy link
Copy Markdown
Contributor

Follow-up to #1128, which you invited in the merge comment: "resolve_client splits by model family, not by the client that made the call... ccusage's --by-agent output does, since each agent gets its own modelBreakdowns. That's the natural follow-up if you're interested; I didn't want to build it against a shape I couldn't capture a real sample of."

I captured one. This adds agents support to the ccusage importer, so a --by-agent export is attributed to the client that actually reported each breakdown instead of to whichever listed agent the model family matched.

The real sample

npx ccusage@20.0.20 daily --by-agent --json, run against this machine's own Claude Code (~7.5 GB) and Codex (~8.9 GB) local session logs: 34 days, 2026-07-22 to 2026-08-24, 39 agent-day cells, 30,152,041,119 tokens / $29,733.75, five of them multi-agent days.

The shape it actually emits, per row:

{ "agent": "all", "period": "2026-08-20",
  "metadata": { "agents": ["claude", "codex"] },
  "modelBreakdowns": [ ... merged across agents ... ],
  "agents": [
    { "agent": "claude", "inputTokens": 210877, "outputTokens": 6845166,
      "cacheCreationTokens": 42883113, "cacheReadTokens": 1243450521,
      "totalTokens": 1293389677, "totalCost": 1162.413754899997,
      "modelsUsed": [...], "modelBreakdowns": [ ... ] },
    { "agent": "codex", "...": "..." }
  ] }

What the file proves, checked on all 34 rows:

  • input + output + cacheCreation + cacheRead == totalTokens holds per agent, not just per row (39/39). So the anchor you added at row level has a per-agent counterpart, and this PR wires it in as a cross-check.
  • Per-agent buckets sum exactly to the row buckets, and per-agent modelBreakdowns sum exactly to per-agent totals. No tolerance needed on real data.
  • metadata still carries only the agent name list. Reasoning is not in there (see limitations).

Why the row-level breakdown cannot recover the split

This is not inference from the file; it is ccusage's own code. AllAccumulator::into_row builds the row-level modelBreakdowns with aggregate_model_breakdowns, which merges entries keyed by model name across agents (rust/crates/ccusage-adapter-all/src/types.rs). Once two agents have used the same model on the same day, the row-level entry is their sum and nothing in it says so. agents is the only place the split survives.

There is a second, quieter loss. model_family knows three families; ccusage's BUILT_IN_AGENT_NAMES lists sixteen agent ids (claude, codex, opencode, amp, droid, codebuff, hermes, pi, goose, openclaw, kilo, copilot, gemini, kimi, qwen, grok). On a multi-agent day the other thirteen match no family, resolve to unknown, and the import warns that the leaderboard would reject it. All sixteen are already registered tokscale ClientIds, so a --by-agent row needs no inference at all — just the same spelling normalization the importer already applies.

Before / after

Real day pair recomposed as a same-family collision (the token values are real — two real Claude-only days, 2026-08-17 and 2026-08-18 — with the second agent relabelled opencode, since I have no real OpenCode usage on this machine):

before:  Clients: claude              claude 3,316,732,440   opencode absent
after:   Clients: claude, opencode    claude 1,599,032,021   opencode 1,717,700,419

Both after-values are what the file declares for each agent.

On the real 34-day export, where claude and codex have disjoint families and the guess was already right, output is byte-identical to before, and every one of the 39 agent-day cells matches the declared totalTokens exactly and the declared totalCost to within $0.01. The import now also says which of the two paths produced the numbers:

    Clients: claude, codex
    Models: 7
    Client attribution: exact, from the export's per-agent breakdowns (34 row(s))

What changed

  • CcusageDaily gains agents: Vec<CcusageAgentBreakdown> (absent without --by-agent, so old exports are untouched).
  • When present, each entry's breakdowns are attributed via attribute_agent, which only normalizes the spelling and flags an id tokscale does not register. resolve_client keeps its doc comment but is now explicitly the fallback for exports that predate --by-agent.
  • Each entry's own totalTokens becomes a per-agent cross-check, appended to breakdown_reconciliation_warnings on divergence — a row total that still adds up can hide one agent's buckets being counted twice.
  • is_empty_day now also looks at agents, so a row carrying only per-agent data is not dropped.
  • The per-model mapping was extracted into add_model_breakdown and is shared by both paths, so they cannot drift.
  • ImportOutcome.agent_attributed_rows counts rows attributed this way, and the CLI prints the line above.

Six tests, all in the existing mod tests: parity with the family guess when families are disjoint, the per-agent declared-total cross-check, two agents sharing one model, agents the family guess cannot name (amp/droid), an entry with no breakdowns, and the divergence warning.

I checked they can actually fail. Reverting the per-agent loop to the family guess turns two of them red:

left:  [("claude", "claude-opus-5")]
right: [("claude", "claude-opus-5"), ("opencode", "claude-opus-5")]
left:  [("unknown", "amp-fast"), ("unknown", "droid-core")]
right: [("amp", "amp-fast"), ("droid", "droid-core")]

and disabling the cross-check turns the sixth red on its own.

cargo test --workspace --all-features: 3158 passed / 0 failed. cargo clippy --locked --workspace --all-features -- -D warnings and cargo fmt --all -- --check: clean.

Limitations, stated up front

  • My real capture only has claude and codex, and their families are disjoint — so the real file cannot itself exhibit the collision. That case rests on ccusage's merge code plus the tests and the recomposed fixture above. I have no real OpenCode usage and did not invent one.
  • Reasoning is still zero for Codex on a --by-agent import, unchanged by this PR. ccusage drops it when merging into agent: "all": row_json writes metadata as { "agents": [...] } whenever row.metadata is None, which is always true for unified rows. Confirmed on the real file — all 34 rows carry metadata with only agents, including the five days Codex ran with real reasoning tokens. The agent entries carry no metadata either, so there is nothing to read. CcusageAgentBreakdown accepts a flat reasoningOutputTokens if ccusage ever adds one. ccusage codex daily --json does carry it, and the importer already handles that shape (the costUSD + flat-reasoning path from feat: add ccusage import format #1128) — I confirmed that on a real capture too.
  • --by-agent also adds agents to weekly and monthly rows. The importer only consumes daily; unchanged and out of scope.
  • ImportOutcome gains a public field. Crate-internal in practice, but it is a struct-literal-breaking change for any out-of-tree construction.
  • Frontend checks not run — bun is not installed here. The change is Rust-only and touches no frontend code.

AI assistance

This patch was authored with Claude Code by INTFRAME, reviewed before submission, and I will respond to review here myself. Same disclosure as #1128.


Summary by cubic

Attributes ccusage --by-agent exports to the agent that reported each breakdown, removing the model-family guess when the export provides per-agent data. Old behavior: attribute per-model usage by inferred family across listed agents. New behavior: attribute by the row’s agents entries; fall back to family inference only when agents is absent.

  • Parses daily[].agents[] and maps each entry via exact agent id normalization; unknown ids are still flagged.
  • Adds a per-agent totalTokens cross-check and surfaces any divergence in breakdown_reconciliation_warnings.
  • Keeps the prior path for exports without --by-agent; per-model mapping is shared to prevent drift.
  • Treats rows with only per-agent data as non-empty; aggregates and suspect-cost checks unchanged.
  • Adds ImportOutcome.agent_attributed_rows and prints “Client attribution: exact…” in the CLI.

Migration

  • If constructing ImportOutcome via struct literals out of tree, add the new field agent_attributed_rows.

Written for commit 2d74df5. Summary will update on new commits.

Review in cubic

…orted them

A --by-agent export carries an `agents` array per row, one entry per agent
with its own `modelBreakdowns`. The row-level breakdowns are a merge of those
keyed by model name (ccusage's `aggregate_model_breakdowns`), so a model two
agents both used cannot be split back apart from them, and `resolve_client`
has to guess by model family. Read the per-agent entries instead: they name
the client outright, which also covers the thirteen ccusage agent ids whose
models match none of the three families `model_family` knows.

Each entry's own `totalTokens` becomes a per-agent cross-check, the
counterpart of the row-level one, since a row total that still adds up can
hide one agent's buckets being counted twice.
@vercel

vercel Bot commented Aug 24, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
tokscale Ignored Ignored Preview Aug 24, 2026 1:39pm

Request Review

@junhoyeo
junhoyeo merged commit 9886511 into junhoyeo:main Aug 24, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants