Executive Summary
AIC (AI Credits) consumption could not be quantified for the last 24 hours. Neither the Sentry spans dataset for the github/gh-aw project nor Grafana Tempo exposed any recognized AIC attribute (gh-aw.aic, gh_aw.aic, agent_usage.aic, aic) on gen_ai.chat / gen_ai.* spans. Workflow attribution (gh-aw.workflow.name, gh-aw.run.id) is present and reliable in Sentry, so event/trace volume per workflow is reported below as a proxy for activity, but no AIC totals can be derived — this is an instrumentation/observability gap, not zero usage.
- Sentry AIC: not queryable (field absent on all 5,389
gen_ai.chat spans sampled).
- Grafana AIC: not evaluated — the Tempo-specific query tools (
tempo_get-attribute-names, tempo_traceql-search, tempo_get-trace) were not available in this session; only list_datasources/get_datasource could be called.
Key Metrics
| Metric |
Value |
| Events analyzed |
5,389 (gen_ai.chat spans, 24h, Sentry) |
| Events with AIC data |
0 |
| Events with AIC data (Sentry) |
0 |
| Events with AIC data (Grafana) |
unknown (tooling unavailable) |
| Total AIC |
unknown (not zero — field absent) |
| Unique workflows |
49 (named) + 4,500 unattributed (gh-aw.workflow.name = null) + 4 [Filtered] |
| Avg AIC/event |
n/a |
| P95 AIC/event |
n/a |
Top 10 Workflows by Event Volume (proxy — AIC unavailable)
| Workflow |
Events (gen_ai.chat, 24h) |
Total AIC |
Avg AIC/Event |
(unattributed, gh-aw.workflow.name=null) |
4,500 |
unknown |
unknown |
| Issue Monster |
78 |
unknown |
unknown |
| Test Quality Sentinel |
68 |
unknown |
unknown |
| Design Decision Gate 🏗️ |
64 |
unknown |
unknown |
| Matt Pocock Skills Reviewer |
62 |
unknown |
unknown |
| PR Code Quality Reviewer |
62 |
unknown |
unknown |
| Ponytail Reviewer |
60 |
unknown |
unknown |
| Impeccable Skills Reviewer |
58 |
unknown |
unknown |
| [aw] Failure Investigator (6h) |
42 |
unknown |
unknown |
| PR Sous Chef |
42 |
unknown |
unknown |
Note: the large "unattributed" bucket (4,500 of 5,389 events, ~83%) is itself a data-quality gap — most chat spans in this window lack gh-aw.workflow.name, so workflow attribution of raw volume is incomplete, independent of the AIC gap.
Grafana AIC Findings
Grafana AIC was not evaluated/queryable in this run.
list_datasources confirmed a Tempo datasource exists: grafanacloud-ghaw-traces (uid grafanacloud-traces), proxied via https://tempo-prod-10-prod-eu-west-2.grafana.net/tempo.- However, the Tempo-specific MCP tools required by the task (
tempo_get-attribute-names, tempo_traceql-search, tempo_get-trace) were not exposed in this session — only generic list_datasources/get_datasource were callable.
- No TraceQL query could be executed, so it is unknown whether
gh-aw.aic (or equivalents) exists as a numeric, indexed span attribute in Tempo.
- Evidence line:
get_datasource(uid=grafanacloud-traces) succeeded and returned datasource config, but no trace-level query tool was available to confirm attribute presence.
Data Quality and Gaps
- AIC fields absent in Sentry: searched
gen_ai.chat/gen_ai.*/ai* spans with has:gh-aw.aic → 0 results; spot-checked individual span detail fields → no gh-aw.aic, gh_aw.aic, agent_usage.aic, or aic attributes present.
- Events missing workflow identifiers: ~4,500 of 5,389
gen_ai.chat spans (83%) have gh-aw.workflow.name = null; 4 further spans are bucketed as [Filtered] by Sentry (likely PII/length scrubbing).
- Assumptions/fallbacks used: per spec, Sentry AIC was meant to be normalized to
0 when absent, but doing so uniformly would misleadingly report "0 AIC total" as if usage were confirmed-zero; this report instead states AIC as unknown/unavailable to avoid fabricating a false zero-consumption signal.
- Sentry-specific caveat:
count() aggregate queries require the sort field to also be a selected column/groupby (HTTP 400 otherwise) — worked around by including count() in both fields and sort.
- Grafana-specific caveat: Tempo query tools (search/attribute-names/get-trace) were unavailable in this execution; only datasource metadata could be retrieved. Cannot confirm attribute typing (string vs numeric) or indexing status for any AIC-like attribute.
- Errors/logs companion checks: not run in this session — time/tool budget was spent confirming AIC field absence across spans datasets; recommend including
errors and logs dataset checks in the next run for completeness.
Recommendations
- Instrument AIC emission: confirm whether the agent/runtime actually emits an AIC (AI Credits) numeric span attribute anywhere in the pipeline. If not yet implemented, add a numeric attribute (e.g.
gh-aw.aic) to gen_ai.chat / gen_ai.invoke_agent spans in the OTLP emitter (see actions/setup/js/send_otlp_span.cjs) so cost can be tracked per workflow/run.
- Fix workflow attribution gap: 83% of
gen_ai.chat spans lack gh-aw.workflow.name/gh-aw.run.id. Investigate why these resource attributes are dropped for a majority of spans (possibly non-workflow-triggered chats, e.g. gateway/dependabot activity) and ensure consistent tagging.
- Enable Tempo TraceQL tooling: grant/enable the Tempo-specific MCP tools (
tempo_get-attribute-names, tempo_traceql-search, tempo_get-trace) for this workflow so Grafana-side AIC/cost data can be cross-validated against Sentry in future runs.
- Re-run with full query loop once AIC instrumentation exists: validate with
errors and logs dataset companion checks, and sample at least one trace end-to-end to confirm AIC propagation.
References
Generated by 📊 Daily AIC Consumption Report (Sentry + Grafana OTel) · goose · sonnet50 · 38.2 AIC · ⌖ 0.687 AIC · ⊞ 11.2K · ◷
Executive Summary
AIC (AI Credits) consumption could not be quantified for the last 24 hours. Neither the Sentry
spansdataset for thegithub/gh-awproject nor Grafana Tempo exposed any recognized AIC attribute (gh-aw.aic,gh_aw.aic,agent_usage.aic,aic) ongen_ai.chat/gen_ai.*spans. Workflow attribution (gh-aw.workflow.name,gh-aw.run.id) is present and reliable in Sentry, so event/trace volume per workflow is reported below as a proxy for activity, but no AIC totals can be derived — this is an instrumentation/observability gap, not zero usage.gen_ai.chatspans sampled).tempo_get-attribute-names,tempo_traceql-search,tempo_get-trace) were not available in this session; onlylist_datasources/get_datasourcecould be called.Key Metrics
gen_ai.chatspans, 24h, Sentry)gh-aw.workflow.name= null) + 4[Filtered]Top 10 Workflows by Event Volume (proxy — AIC unavailable)
gh-aw.workflow.name=null)Note: the large "unattributed" bucket (4,500 of 5,389 events, ~83%) is itself a data-quality gap — most chat spans in this window lack
gh-aw.workflow.name, so workflow attribution of raw volume is incomplete, independent of the AIC gap.Grafana AIC Findings
Grafana AIC was not evaluated/queryable in this run.
list_datasourcesconfirmed a Tempo datasource exists:grafanacloud-ghaw-traces(uidgrafanacloud-traces), proxied viahttps://tempo-prod-10-prod-eu-west-2.grafana.net/tempo.tempo_get-attribute-names,tempo_traceql-search,tempo_get-trace) were not exposed in this session — only genericlist_datasources/get_datasourcewere callable.gh-aw.aic(or equivalents) exists as a numeric, indexed span attribute in Tempo.get_datasource(uid=grafanacloud-traces)succeeded and returned datasource config, but no trace-level query tool was available to confirm attribute presence.Data Quality and Gaps
gen_ai.chat/gen_ai.*/ai*spans withhas:gh-aw.aic→ 0 results; spot-checked individual span detail fields → nogh-aw.aic,gh_aw.aic,agent_usage.aic, oraicattributes present.gen_ai.chatspans (83%) havegh-aw.workflow.name= null; 4 further spans are bucketed as[Filtered]by Sentry (likely PII/length scrubbing).0when absent, but doing so uniformly would misleadingly report "0 AIC total" as if usage were confirmed-zero; this report instead states AIC as unknown/unavailable to avoid fabricating a false zero-consumption signal.count()aggregate queries require the sort field to also be a selected column/groupby (HTTP 400 otherwise) — worked around by includingcount()in both fields and sort.errorsandlogsdataset checks in the next run for completeness.Recommendations
gh-aw.aic) togen_ai.chat/gen_ai.invoke_agentspans in the OTLP emitter (seeactions/setup/js/send_otlp_span.cjs) so cost can be tracked per workflow/run.gen_ai.chatspans lackgh-aw.workflow.name/gh-aw.run.id. Investigate why these resource attributes are dropped for a majority of spans (possibly non-workflow-triggered chats, e.g. gateway/dependabot activity) and ensure consistent tagging.tempo_get-attribute-names,tempo_traceql-search,tempo_get-trace) for this workflow so Grafana-side AIC/cost data can be cross-validated against Sentry in future runs.errorsandlogsdataset companion checks, and sample at least one trace end-to-end to confirm AIC propagation.References
grafanacloud-ghaw-traces(uid:grafanacloud-traces)