Normalize unified agent session metadata and AIC accounting by Copilot · Pull Request #65344 · github/gh-aw

Copilot · 2026-10-03T16:23:39Z

Unified session records mixed field naming, repeated engine metadata, and lacked a clear way to resolve detection and eval AIC. They also omitted the trigger type and some available version metadata.

Consistent records: Use camelCase for known unified-session fields, including nested token usage. Remove the duplicate engine field while leaving unknown native extensions opaque.
Run context: Record the trigger type, requested model, and available gh-aw, AWF, MCPG, and agent versions.
Phase accounting: Resolve detection and eval AIC from explicit evidence or known model pricing. Preserve explicit zero and checkpoint totals rather than estimating over them.
Reader compatibility: Reconcile camelCase token-overflow markers without retaining stale counts.

✨ PR Review Safe Output Test - Run 37138576096

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

clients2.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "clients2.google.com"

See Network Configuration for more information.

💥 [THE END] — Illustrated by Smoke Claude · claude · sonnet46 · 50.8 AIC · ⌖ 3.91 AIC · ⊞ 853 · ◷
_{Comment /smoke-claude to run again}

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

pelikhan · 2026-10-03T16:40:45Z

@copilot update go parser

Copilot

Copilot review overview

🟡 Changes recommended

Canonical-field precedence, checkpoint preservation, and unified usage rendering remain incorrect in several paths.

Review effort: Balanced
Findings: 4

Open (4)

Prefer canonical overflowed_tokens field · New
Prefer canonical snake_case fields by presence · New
Normalize camelCase aliases in unified statistics output · New
Preserve aiCreditsTotal checkpoint alias · New

What changed in this PR

Normalizes unified-session metadata and usage fields while adding phase-specific AIC resolution and reader compatibility.

Changes:

Standardizes known payloads on camelCase and expands workflow metadata.
Estimates detection/eval AIC only when explicit accounting is absent.
Updates usage rendering, reconciliation, tests, documentation, and embedded assets.

File	Description
`docs/src/content/docs/specs/unified-agent-session-specification.md`	Documents normalized metadata and accounting.
`actions/setup/session_parsers.go`	Embeds model-pricing resources.
`actions/setup/js/unified_session.test.cjs`	Tests metadata and phase AIC behavior.
`actions/setup/js/unified_session.cjs`	Passes source phase into normalization.
`actions/setup/js/unified_session_render.cjs`	Renders normalized field names.
`actions/setup/js/unified_session_payload.test.cjs`	Tests payload normalization and overflow handling.
`actions/setup/js/unified_session_payload.cjs`	Implements normalization and AIC resolution.
`actions/setup/js/log_parser_shared.cjs`	Normalizes usage before detailed rendering.
`actions/setup/js/agent_session.cjs`	Adds camelCase reader compatibility.
`.github/workflows/smoke-codex-auto.lock.yml`	Regenerates setup context propagation.

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Copilot · 2026-10-03T17:45:54Z

+  const totalTokens = usage.totalTokens ?? usage.total_tokens;
+  const inputTokens = usage.inputTokens ?? usage.input_tokens;
+  const outputTokens = usage.outputTokens ?? usage.output_tokens;
+  if (isTokenCount(totalTokens)) return totalTokens;
+  if (!isTokenCount(inputTokens) && !isTokenCount(outputTokens)) return undefined;
+  const cache =
+    (usage.inputTokensIncludeCache ?? usage.input_tokens_include_cache) === false ? (usage.cacheCreationInputTokens ?? usage.cache_creation_input_tokens ?? 0) + (usage.cacheReadInputTokens ?? usage.cache_read_input_tokens ?? 0) : 0;
+  const total = (inputTokens ?? 0) + (outputTokens ?? 0) + cache;
  return isTokenCount(total) ? total : undefined;


Fixed in f5d8d90: usage normalization and total calculation now prefer snake_case fields by own-property presence, including zero and invalid canonical values.

Copilot · 2026-10-03T17:45:54Z


  if (lastEntry.usage) {
-    const usage = lastEntry.usage;
+    const usage = normalizeSessionUsage(lastEntry.usage) ?? lastEntry.usage;


Fixed in f5d8d90: statistics rendering now uses the shared normalizeSessionUsage projection, so camelCase totals, token components, and cache semantics render consistently.

Copilot · 2026-10-03T17:45:54Z

-    aic: ["aic", "ai_credits_this_response"],
-    totalAic: ["totalAic", "ai_credits_total", "ai_credits"],
+    aic: ["aic", "ai_credits_this_response", "aiCreditsThisResponse"],
+    totalAic: ["totalAic", "ai_credits_total", "ai_credits", "aiCredits"],


Fixed in f5d8d90: aiCreditsTotal is now accepted as an authoritative totalAic alias and suppresses per-request estimation.

github-actions · 2026-10-03T16:52:04Z

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel skipped because pre-fetch PR data was unavailable: unable to fetch test file diff

🧪 Test quality analysis by Test Quality Sentinel

github-actions · 2026-10-03T16:52:07Z

✅ Ponytail Reviewer completed successfully!

Lean already. Ship.

Generated by Ponytail Reviewer for #65344

github-actions · 2026-10-03T16:52:07Z

✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft.

No ADR enforcement needed: PR does not have the "implementation" label and has ≤100 new lines of code in business logic directories.

🏗️ ADR gate enforced by Design Decision Gate 🏗️

github-actions · 2026-10-03T16:52:08Z

✅ PR Code Quality Reviewer completed the code quality review.

🔎 Code quality review by PR Code Quality Reviewer

github-actions · 2026-10-03T16:52:17Z

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

github-actions · 2026-10-03T16:58:53Z

Comment Memory

reviewed_at: 2026-10-03T16:53:34Z
review_event: REQUEST_CHANGES
top_themes:
  - canonical-field precedence regressions
  - incomplete camelCase usage reader compatibility
  - missing aiCreditsTotal alias handling
files_reviewed:
  - actions/setup/js/agent_session.cjs
  - actions/setup/js/log_parser_shared.cjs
  - actions/setup/js/unified_session.cjs
  - actions/setup/js/unified_session.test.cjs
  - actions/setup/js/unified_session_payload.cjs
  - actions/setup/js/unified_session_payload.test.cjs
  - actions/setup/js/unified_session_render.cjs
  - actions/setup/session_parsers.go
  - docs/src/content/docs/specs/unified-agent-session-specification.md
comment_count: 1

Note

This comment is managed by comment memory.

It stores persistent context for this thread in the code block at the top of this comment.
Edit only the text inside the backtick fences; workflow metadata and the footer are regenerated automatically.

Learn more about comment memory

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt54 · 73.9 AIC · ⌖ 9.17 AIC · ⊞ 20.2K · ◷
_{Comment /review to run again}

github-actions

Request changes

This normalization pass still regresses schema-compatibility rules in a few places, so I don't think it's safe to merge yet.

Blocking themes

Mixed camelCase/snake_case records still have precedence bugs: alias values can override explicitly present canonical fields.
The reader-compatibility work is incomplete, so some summaries still render camelCase accounting as partial or otherwise incorrect totals.
usage.report still misses at least one established AIC alias (aiCreditsTotal), which drops authoritative checkpoint totals and can trigger the wrong fallback behavior.

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt54 · 73.9 AIC · ⌖ 9.17 AIC · ⊞ 20.2K
_{Comment /review to run again}

Comments that could not be inline-anchored

actions/setup/js/log_parser_shared.cjs:320

This formatter still reports some camelCase usage objects as partial accounting, so the compatibility fix is incomplete and can silently misstate rendered token totals.

<details><summary>💡 Why this is still wrong</summary>

normalizeSessionUsage() only backfills input_tokens, output_tokens, and the cache token counts. It leaves totalTokens, inputTokensIncludeCache, and reasoningOutputTokens in camelCase only. The code below then decides whether the total is authoritative from `usa…

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot · 2026-10-03T17:00:30Z

@copilot update go parser

Updated the Go JSONL parser to support camelCase usage fields and overflow markers, preserving snake_case precedence. Added regression tests; committed as e1fd0a4.

github-actions

Verified the four existing automated review comments against the current diff/code; all are accurate and blocking:

agent_session.cjs:39 (normalizeSessionUsage) — canonical-field precedence is reversed for overflowed_tokens/overflowedTokens.
agent_session.cjs:348 (sessionTokenTotal) — camelCase aliases now win over explicit canonical snake_case values (e.g. {total_tokens: 7, totalTokens: 99} → 99).
log_parser_shared.cjs:316 — only the "detailed information" path normalizes usage; log_parser_format.cjs's appendStatistics (used for step-summary/plain-text rendering) still reads raw usage.input_tokens/cache_read_input_tokens and will silently show nothing for camelCase-only usage records.
unified_session_payload.cjs:126 (totalAic aliases) — misses aiCreditsTotal, which ai_credits_context.cjs:15 (AI_CREDITS_TOTAL_FIELDS) treats as an equally canonical checkpoint field elsewhere in the codebase.

No additional high-signal issues found beyond these. Requesting changes to resolve the alias-precedence and incomplete-normalization bugs above before merge.

🧵 Reviewed using Impeccable skills by Impeccable Skills Reviewer · copilot · sonnet50 · 67.2 AIC · ⌖ 13.1 AIC · ⊞ 8.1K

github-actions

Skills-Based Review 🧠

Applied /codebase-design and /tdd to the camelCase normalization and new AIC-resolution logic.

📋 Key Themes & Highlights

Key Themes

Existing bot review comments are still valid and unresolved: the precedence bug in normalizeSessionUsage/sessionTokenTotal (agent_session.cjs:39,348) where ?? makes the newly-accepted camelCase alias win over an explicitly present canonical snake_case value, contradicting the spec's "canonical value wins by presence" rule; the aiCreditsTotal alias gap in unified_session_payload.cjs:126; and the partial normalization in log_parser_shared.cjs that doesn't cover appendStatistics/CLI-style summary paths. These should be addressed before merge — they go directly against the behavior guaranteed in the spec's own wording.
AIC phase-gating is implicit: resolveUsageAic is restricted to phase === "detection" || "evals" via a literal check buried in normalizeUnifiedSessionEvent, rather than a named, co-located constant — makes the coupling between phase and pricing resolution easy to miss when adding a new phase.
New test coverage is good but has gaps: the detection/evals AIC test (unified_session.test.cjs) covers the "happy path" combinations well but doesn't exercise the guard-clause edge cases in resolveUsageAic (negative/non-finite tokens, cache-only usage with zero input/output tokens).

Positive Highlights

✅ Clear, well-documented spec updates (precedence rules, trigger type provenance, version metadata) make the intended behavior easy to verify against.
✅ The overflowed_tokens/overflowedTokens alias reconciliation correctly round-trips through both canonical and camelCase forms without losing entries.
✅ Good test discipline overall — most new behavior (trigger type, requested model, version fields, workflow.info consolidation) is backed by targeted assertions.

@copilot please address the review comments above.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · sonnet50 · 96 AIC · ⌖ 14.6 AIC · ⊞ 10K
_{Comment /matt to run again}

github-actions · 2026-10-03T17:02:38Z

+    if (event.type === "usage.report" && (phase === "detection" || phase === "evals")) resolveUsageAic(source, data, usage);
  }
  if (event.type === "tool.execution_complete" && (source.is_error === true || source.result?.isError === true || source.result?.is_error === true)) data.isError = true;
  if (event.type === "experiment.assignment") {


[/codebase-design] resolveUsageAic is gated by a hardcoded phase allowlist (phase === "detection" || phase === "evals") buried inside normalizeUnifiedSessionEvent, far from the EVENT_FIELDS/USAGE_FIELDS tables that define the rest of the accounting shape.

💡 Suggestion

If a future phase (e.g. a new eval/detection source, or conclusion) needs pricing-derived AIC, this silently skips it. Consider hoisting the phase list to a named constant (e.g. AIC_RESOLVABLE_PHASES) next to USAGE_FIELDS so the coupling between phase and pricing resolution is visible and easy to extend/test.

@copilot please address this.

Fixed in f5d8d90: the detection/evals pricing phases are now named in AIC_RESOLVABLE_PHASES beside USAGE_FIELDS.

github-actions · 2026-10-03T17:02:38Z

    });
    expect(events.find(event => event.type === "workflow.info").data).not.toHaveProperty("secret");
  });



[/tdd] New test covers the happy paths for resolveUsageAic (zero tokens, unknown model, explicit ai_credits/ai_credits_this_response) but not the guard-clause edge cases inside the function itself: negative/non-finite token values, and a usage record where only cache tokens are present (cacheReadInputTokens/cacheCreationInputTokens > 0 but inputTokens/outputTokens are 0 or absent).

💡 Suggested additional cases

// cache-only usage should still compute AIC write("evals/evals_token_usage.jsonl", [ { provider: "openai", model: "gpt-4o-mini", cache_read_input_tokens: 500 }, ]); // negative/non-finite tokens should be rejected, not silently coerced write("threat-detection/detection_usage.jsonl", [ { provider: "openai", model: "gpt-4o-mini", input_tokens: -5 }, ]);

Without these, a regression in the tokens.some(...)/tokens.every(...) guard in resolveUsageAic (unified_session_payload.cjs:183-184) could silently start producing negative or NaN AIC values without any test failing.

@copilot please address this.

Added in f5d8d90: tests cover cache-only pricing, negative and non-finite token rejection, and leaving other phases unestimated.

github-actions

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

clients2.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "clients2.google.com"

See Network Configuration for more information.

💥 [THE END] — Illustrated by Smoke Claude · claude · sonnet46 · 50.8 AIC · ⌖ 3.91 AIC · ⊞ 853
_{Comment /smoke-claude to run again}

github-actions · 2026-10-03T17:04:35Z

@@ -2,17 +2,18 @@

 /** @typedef {import("./types/agent_session").SessionEvent} SessionEvent */
 /** @typedef {Record<string, string[]>} Fields */


🤖 Smoke test review comment (Run 37138576096): Good use of named imports from model_costs.cjs — this keeps the dependency explicit and tree-shakeable.

No change was needed in f5d8d90; the import from model_costs.cjs remains a named import.

github-actions · 2026-10-03T17:04:35Z

@@ -30,13 +30,15 @@ function isMetric(value) {
 function normalizeSessionUsage(usage) {
  if (!usage || typeof usage !== "object" || Array.isArray(usage)) return undefined;


🤖 Smoke test review comment (Run 37138576096): The normalizeSessionUsage function correctly handles the object spread — consider adding a comment explaining why arrays are explicitly excluded.

Added a concise explanation in f5d8d90: usage is a named-field object, so arrays cannot represent its token fields.

pelikhan · 2026-10-03T17:21:57Z

@copilot run pr-finisher skill

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot · 2026-10-03T17:46:03Z

@copilot run pr-finisher skill

Ran the PR-finisher pass and pushed the review fixes in f5d8d90. Local impacted validation passed; the broad make test remains blocked by the pre-existing missing embedded Gemini parser module. Review replies are posted; this integration has no thread-resolution action, and CI needs a maintainer re-trigger after the push.

Dovr

github-actions · 2026-10-05T23:10:19Z

🎉 This pull request is included in a new release.

Release: v0.91.0

Copilot AI and others added 4 commits October 3, 2026 16:10

Outline unified session consistency changes

23e9e7d

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Normalize unified session metadata and phase usage

ebbd670

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Handle nested phase AIC evidence and validate session types

9607c61

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Honor camelCase overflow markers when reconciling sessions

e1d78fb

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI assigned Copilot and pelikhan Oct 3, 2026

Copilot AI changed the title ~~Normalize unified agent session fields and resolve phase AIC~~ Normalize unified agent session metadata and AIC accounting Oct 3, 2026

Copilot created this pull request from a session on behalf of pelikhan October 3, 2026 16:24 View session

Copilot finished work on behalf of pelikhan October 3, 2026 16:24

Copilot AI requested a review from pelikhan October 3, 2026 16:24

pelikhan marked this pull request as ready for review October 3, 2026 16:40

Copilot AI balanced review requested due to automatic review settings October 3, 2026 16:40

Copilot started reviewing on behalf of pelikhan October 3, 2026 16:40 View session

Copilot AI reviewed Oct 3, 2026

View reviewed changes

Copilot started work on behalf of pelikhan October 3, 2026 16:48 View session

github-actions Bot requested changes Oct 3, 2026

View reviewed changes

Support camelCase usage in Go parser

e1fd0a4

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot finished work on behalf of pelikhan October 3, 2026 17:01

github-actions Bot previously requested changes Oct 3, 2026

View reviewed changes

github-actions Bot reviewed Oct 3, 2026

View reviewed changes

github-actions Bot mentioned this pull request Oct 3, 2026

Smoke Test: Claude - 37138576096 #65349

Closed

github-actions Bot reviewed Oct 3, 2026

View reviewed changes

Copilot started work on behalf of pelikhan October 3, 2026 17:22 View session

Fix session usage precedence and AIC handling

f5d8d90

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot finished work on behalf of pelikhan October 3, 2026 17:46

pelikhan merged commit 2198d3c into main Oct 3, 2026
1 check passed

pelikhan deleted the copilot/improve-agent-unified-session branch October 3, 2026 18:08

		@@ -2,17 +2,18 @@

		/** @typedef {import("./types/agent_session").SessionEvent} SessionEvent */
		/** @typedef {Record<string, string[]>} Fields */

		@@ -30,13 +30,15 @@ function isMetric(value) {
		function normalizeSessionUsage(usage) {
		if (!usage \|\| typeof usage !== "object" \|\| Array.isArray(usage)) return undefined;

Conversation

Copilot AI commented Oct 3, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pelikhan commented Oct 3, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Copilot AI Oct 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 3, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Oct 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 3, 2026

Comment Memory

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Request changes

Comments that could not be inline-anchored

Uh oh!

Copilot AI commented Oct 3, 2026

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Skills-Based Review 🧠

Key Themes

Positive Highlights

Uh oh!

github-actions Bot Oct 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 3, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Oct 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 3, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Oct 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 3, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Oct 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 3, 2026

Copilot AI commented Oct 3, 2026 •

edited by github-actions Bot

Loading

github-actions Bot commented Oct 3, 2026 •

edited

Loading

github-actions Bot commented Oct 3, 2026 •

edited

Loading

github-actions Bot commented Oct 3, 2026 •

edited

Loading

github-actions Bot commented Oct 3, 2026 •

edited

Loading

github-actions Bot commented Oct 3, 2026 •

edited

Loading