Skip to content

Attribute friction costs to audit sources - #64338

Merged
pelikhan merged 3 commits into
mainfrom
copilot/ensure-friction-cost-identification
Sep 29, 2026
Merged

pelikhan merged 3 commits into
mainfrom
copilot/ensure-friction-cost-identification

Conversation

Copilot AI commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Audit friction totals don’t show which source incurred the cost. This change makes source- and driver-level costs visible in console and JSON output.

  • Console: Label each driver and event with its source; show available AIC, token, turn, tool-call, and latency costs. Omit unavailable AIC rather than implying zero.
  • JSON and docs: Verify source attribution survives serialization and document friction.drivers[] and friction.events[].
source=firewall driver=firewall_block occurrences=2 counted=2 state=statistical aic=0.25 tokens=64
source=firewall driver=firewall_block: blocked.example x2 counted=2 state=statistical aic=0.25 tokens=64

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI requested a review from pelikhan September 29, 2026 20:15
@pelikhan
pelikhan marked this pull request as ready for review September 29, 2026 20:22
Copilot AI balanced review requested due to automatic review settings September 29, 2026 20:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The focused implementation matches the stated behavior and includes appropriate coverage.

Review effort: Balanced
Findings: None

What changed in this PR

Adds source-attributed friction costs to audit console output and documents their JSON representation.

Changes:

  • Labels friction drivers and events with their source and available costs.
  • Tests console rendering and JSON source attribution.
  • Documents friction driver and event fields.
File Description
pkg/​cli/​audit_report_render.go Renders source-attributed friction details.
pkg/​cli/​friction_cost_test.go Verifies console and JSON output.
docs/​src/​content/​docs/​reference/​audit.md Documents friction reporting.

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

⚠️ Security scanning failed for Design Decision Gate 🏗️. Review the logs for details.

No ADR enforcement needed: PR does not have the implementation label and has ≤100 new lines of code in business logic directories.

🏗️ ADR gate enforced by Design Decision Gate 🏗️

@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

✅ PR Code Quality Reviewer completed the code quality review.

🔎 Code quality review by PR Code Quality Reviewer

@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel skipped because pre-fetch PR data was unavailable: unable to fetch test file diff

🧪 Test quality analysis by Test Quality Sentinel

@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

✅ Ponytail Reviewer completed successfully!

Lean already. Ship.

Generated by Ponytail Reviewer for #64338

@github-actions

Copy link
Copy Markdown
Contributor

Comment Memory

reviewed_at: 2026-09-29T20:37:50.510+00:00
review_event: COMMENT
top_themes:
  - no blocking issues found in friction source attribution changes
  - renderer/docs/tests stayed aligned
files_reviewed:
  - docs/src/content/docs/reference/audit.md
  - pkg/cli/audit_report_render.go
  - pkg/cli/friction_cost_test.go
comment_count: 0

Note

This comment is managed by comment memory.

It stores persistent context for this thread in the code block at the top of this comment.
Edit only the text inside the backtick fences; workflow metadata and the footer are regenerated automatically.

Learn more about comment memory

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt54 · 26.9 AIC · ⌖ 7.23 AIC · ⊞ 19.3K · ◷
Comment /review to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict

No blocking correctness or maintainability issue stood out in the changed lines.

What I checked

The renderer/docs/tests move together: source attribution is visible on both driver and event rows, unavailable AIC stays omitted instead of being printed as a misleading zero, and the JSON round-trip coverage still exercises the source-attributed fields.

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt54 · 26.9 AIC · ⌖ 7.23 AIC · ⊞ 19.3K
Comment /review to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /codebase-design and /tdd — 2 non-blocking suggestions, no correctness bugs found.

📋 Key Themes & Highlights

Key Themes

  • Duplicated formatting logic: frictionCostDetails (new, audit_report_render.go) mirrors the dimension-formatting conditionals already in frictionSummaryLine (friction_cost.go). Extracting a shared helper would prevent the two from drifting as new cost dimensions are added.
  • Driver vs. event AIC-gating asymmetry: the event renderer correctly consults event.DimensionStates["aic"] to override event.State when deciding whether to show aic=, but the driver renderer only looks at driver.State, which is currently hardcoded to unavailable in deriveFrictionFromLogs's aggregation loop and never revisited. Not a bug today (aggregated derived-driver costs are ToolCalls-only), but there's no test pinning this behavior, so a future change to driver aggregation could silently regress AIC visibility.

Positive Highlights

  • ✅ New behavior is well covered: TestRenderConsoleFrictionAttributesCostsToSource and the TestAuditDataSerializesFriction source/cost assertions directly validate the PR's stated goal (source attribution surviving both console and JSON output).
  • ✅ Docs updated in lockstep with the JSON/console schema changes (audit.md), and existing tests (TestRenderConsoleFrictionShowsAggregateAndEvents) were updated rather than left stale.
  • ✅ frictionSourceLabel cleanly handles the empty-source case ("unknown") rather than printing a blank source=.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · sonnet50 · 59.7 AIC · ⌖ 14.5 AIC · ⊞ 10K
Comment /matt to run again

return source
}

func frictionCostDetails(cost FrictionCost, aicState string) string {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/codebase-design] frictionCostDetails re-implements the same dimension formatting already present in frictionSummaryLine (friction_cost.go:161-190) — same tokens/turns/tool-calls/latency conditionals, duplicated verbatim across two files.

💡 Suggested fix

Extract a single shared helper (e.g. formatFrictionCostParts(cost FrictionCost, aicState string) []string) that both frictionSummaryLine and frictionCostDetails call, so future dimension additions (e.g. a new cost axis) only need one edit site instead of two that can drift apart.

@copilot please address this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shared the token, turn, tool-call, and latency formatting between aggregate and driver/event output in pkg/cli/friction_cost.go (d15f9cd). The existing AIC, ratio, and uncertainty ordering is preserved.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed with a standard correctness/maintainability pass (this is Go CLI console-output formatting, not a UI/frontend change, so Impeccable audit/critique/harden modes don't directly apply).

Findings: none blocking.

  • frictionSourceLabel defaulting to "unknown" matches existing codebase conventions (see audit_diff.go, gateway_logs_types.go, etc.).
  • Driver-level frictionCostDetails gates AIC display off driver.State while event-level gates off event.DimensionStates["aic"] when present — this is intentional since FrictionDriverSummary has no per-dimension state map, and behavior matches new/updated tests.
  • New tests (TestRenderConsoleFrictionAttributesCostsToSource, JSON source-attribution assertions in TestAuditDataSerializesFriction) directly cover the new behavior and pass.
  • go build ./pkg/cli/..., go vet ./pkg/cli/..., and go test ./pkg/cli/... -run Friction all pass; gofmt -l reports no issues.
  • Docs update in audit.md accurately describes the new friction.drivers[]/friction.events[] fields and console format.

No actionable issues found.

🧵 Reviewed using Impeccable skills by Impeccable Skills Reviewer · copilot · sonnet50 · 53.4 AIC · ⌖ 13.1 AIC · ⊞ 8.1K

@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot run pr-finisher skill

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot please review the latest comments, refresh the branch if needed, and run the pr-finisher skill before handing this back for maintainer review.

Generated by 👨‍🍳 PR Sous Chef · pi · gpt54 · 12.8 AIC · ⌖ 8.67 AIC · ⊞ 9.5K · ◷
Comment /souschef to run again

Copilot AI and others added 2 commits September 29, 2026 21:33
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
…on-cost-identification

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Sep 29, 2026

Copy link
Copy Markdown
Contributor Author

@copilot run pr-finisher skill

Ran the PR finisher pass: addressed the review suggestion in d15f9cd, merged current main in eadce86, and validated focused/impacted tests, lint, and workflow compilation. Remaining hand-off: resolve the replied review thread, address the pre-existing custom-linter findings, and re-trigger CI for the new HEAD.

@pelikhan
pelikhan merged commit 4d8a260 into main Sep 29, 2026
5 checks passed
@pelikhan
pelikhan deleted the copilot/ensure-friction-cost-identification branch September 29, 2026 21:43

Copilot AI commented Sep 29, 2026

Copy link
Copy Markdown
Contributor Author

@copilot please review the latest comments, refresh the branch if needed, and run the pr-finisher skill before handing this back for maintainer review.

Generated by [👨‍🍳 PR Sous Chef](https...

PR #64338 was merged by a maintainer at 21:43 UTC before this pass. The prior finisher changes and branch refresh are included in the merged head (eadce86); no further branch changes are appropriate.

Copilot AI restored the copilot/ensure-friction-cost-identification branch September 29, 2026 21:48
Copilot AI requested a review from gh-aw-bot September 29, 2026 21:48
@github-actions

Copy link
Copy Markdown
Contributor

🎉 This pull request is included in a new release.

Release: v0.90.1

@github-actions
github-actions Bot deleted the copilot/ensure-friction-cost-identification branch October 7, 2026 02:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants