Executive Summary
- Total AIC: Not queryable — no recognized AIC attributes (
gh-aw.aic, gh_aw.aic, agent_usage.aic, aic) were found on any Sentry span/event in the last 24h.
- Workflow count: Not derivable from AIC data; span/transaction metadata shows multiple gh-aw workflow runs (
gh-aw.activation.setup, several gen_ai.chat spans across models: copilot/claude-sonnet-5, copilot/gpt-5.4, copilot/gpt-6-astra, claude-haiku-4.5, openai/gpt-5.6-sol, generic agent).
- Sentry AIC queryable? No — confirmed absent on all sampled spans (gen_ai.chat, http.server, default).
- Grafana AIC queryable? Unknown/unconfirmed — the Tempo TraceQL query tools (
tempo_get-attribute-names, tempo_traceql-search, tempo_get-trace) described in the task instructions were not available as callable tools in this session; only list_datasources/get_datasource were available, which confirmed a Tempo datasource (grafanacloud-ghaw-traces, uid grafanacloud-traces) exists but AIC presence could not be directly verified.
Key Metrics
| Metric |
Value |
| Events analyzed |
70 (50 generic spans sample + 20 gen_ai.chat spans sample, Sentry spans dataset, 24h) |
| Events with AIC data |
0 |
| Events with AIC data (Sentry) |
0 |
| Events with AIC data (Grafana) |
Unknown (tooling unavailable) |
| Total AIC |
Unknown — no numeric AIC values found |
| Unique workflows |
Not derivable (no github.workflow/AIC fields populated in sampled spans) |
| Avg AIC/event |
N/A |
| P95 AIC/event |
N/A |
Top 10 Workflows by AIC Consumption
Not available — no events carried AIC or workflow-attribution fields in the sampled data. Observed gen_ai.chat span counts by model (not AIC, just event counts) for visibility only:
| Model (span.description) |
Events (sampled) |
Total AIC |
Avg AIC/Event |
| chat claude-haiku-4.5 |
9 |
Unknown |
Unknown |
| chat agent |
4 |
Unknown |
Unknown |
| chat copilot/gpt-6-astra |
2 |
Unknown |
Unknown |
| chat copilot/claude-sonnet-5 |
2 |
Unknown |
Unknown |
| chat copilot/gpt-5.4 |
2 |
Unknown |
Unknown |
| chat openai/gpt-5.6-sol |
1 |
Unknown |
Unknown |
Grafana AIC Findings
Grafana AIC is not confirmed queryable in this run.
list_datasources(type=tempo) succeeded and returned one Tempo datasource: grafanacloud-ghaw-traces (uid grafanacloud-traces), confirmed via get_datasource.- The TraceQL/Tempo query tools required by the task (
tempo_get-attribute-names, tempo_traceql-search, tempo_get-trace) were not present among the tools available to this agent session — only list_datasources and get_datasource were callable. This is a tooling/MCP-server-exposure gap in this run, not a confirmed data gap in Tempo itself.
- No trace-level evidence of AIC attributes could be gathered from Grafana as a result.
- Grafana base URL may be redacted in safe-outputs rendering; datasource UID
grafanacloud-traces is provided above for reference.
Data Quality and Gaps
- Events missing workflow identifiers: All sampled spans lacked
github.workflow/gh_aw.workflow fields in the returned payload; transaction names (gateway.request, gh-aw.activation.setup, default) were used as weak proxies but are not true workflow names.
- Events missing AIC attributes: 100% of sampled Sentry spans (70/70) had no recognized AIC field (
gh-aw.aic, gh_aw.aic, agent_usage.aic, aic).
- Assumptions/fallbacks used: Per spec, Sentry spans without AIC fields were treated as
0; this makes the absence highly visible — effectively total_aic = 0 from Sentry, which should be read as "not instrumented" rather than "no cost incurred."
- Sentry-specific caveats:
search_events (AI-powered search) was not available in this MCP build, so list_events with direct query syntax was used per fallback instructions. Query span.op:ai* returned 0 results; span.op:gen_ai.chat returned 20 chat spans but none exposed AIC fields in the field list requested.
- Errors dataset: queried, 0 results in 24h — explicitly confirmed, not skipped.
- Logs dataset: queried, 20 results returned, but all are internal engine/runtime logs (
llm runtime selected, stream, process, loop, evaluated permission checks) with no AIC/cost fields present.
- Grafana-specific caveats: Tempo TraceQL tools were unavailable in this session despite being referenced in the task instructions; only datasource discovery tools worked. This should be treated as an environment/tool-exposure gap, not evidence that Tempo itself lacks AIC attributes.
Recommendations
- Instrument AIC emission on spans: Confirm the OTLP span emitter (
actions/setup/js/send_otlp_span.cjs) actually sets a numeric gh-aw.aic (or equivalent) attribute on gen_ai.chat/gen_ai.execute_tool spans. Current data suggests this attribute is either not emitted or not indexed/searchable in Sentry EAP.
- Add workflow attribution attributes: Ensure spans carry
github.workflow (or gh_aw.workflow) consistently so future reports can aggregate AIC by workflow instead of falling back to generic transaction names like default/gateway.request.
- Restore/verify Tempo MCP tool availability: The Grafana extension in this environment only exposed
list_datasources/get_datasource; tempo_traceql-search and related tools should be enabled so Grafana-side AIC cross-checks are possible in future runs.
- Re-run with
search_events enabled: If an embedded LLM provider becomes configured for the Sentry MCP server, re-run this report using search_events for richer natural-language AIC aggregation and to double check the list_events fallback findings.
References
Generated by 📊 Daily AIC Consumption Report (Sentry + Grafana OTel) · goose · sonnet50 · 28.9 AIC · ⌖ 8.24 AIC · ⊞ 11.2K · ◷
Executive Summary
gh-aw.aic,gh_aw.aic,agent_usage.aic,aic) were found on any Sentry span/event in the last 24h.gh-aw.activation.setup, severalgen_ai.chatspans across models:copilot/claude-sonnet-5,copilot/gpt-5.4,copilot/gpt-6-astra,claude-haiku-4.5,openai/gpt-5.6-sol, genericagent).tempo_get-attribute-names,tempo_traceql-search,tempo_get-trace) described in the task instructions were not available as callable tools in this session; onlylist_datasources/get_datasourcewere available, which confirmed a Tempo datasource (grafanacloud-ghaw-traces, uidgrafanacloud-traces) exists but AIC presence could not be directly verified.Key Metrics
gen_ai.chatspans sample, Sentryspansdataset, 24h)github.workflow/AIC fields populated in sampled spans)Top 10 Workflows by AIC Consumption
Not available — no events carried AIC or workflow-attribution fields in the sampled data. Observed
gen_ai.chatspan counts by model (not AIC, just event counts) for visibility only:Grafana AIC Findings
Grafana AIC is not confirmed queryable in this run.
list_datasources(type=tempo)succeeded and returned one Tempo datasource:grafanacloud-ghaw-traces(uidgrafanacloud-traces), confirmed viaget_datasource.tempo_get-attribute-names,tempo_traceql-search,tempo_get-trace) were not present among the tools available to this agent session — onlylist_datasourcesandget_datasourcewere callable. This is a tooling/MCP-server-exposure gap in this run, not a confirmed data gap in Tempo itself.grafanacloud-tracesis provided above for reference.Data Quality and Gaps
github.workflow/gh_aw.workflowfields in the returned payload; transaction names (gateway.request,gh-aw.activation.setup,default) were used as weak proxies but are not true workflow names.gh-aw.aic,gh_aw.aic,agent_usage.aic,aic).0; this makes the absence highly visible — effectivelytotal_aic = 0from Sentry, which should be read as "not instrumented" rather than "no cost incurred."search_events(AI-powered search) was not available in this MCP build, solist_eventswith direct query syntax was used per fallback instructions. Queryspan.op:ai*returned 0 results;span.op:gen_ai.chatreturned 20 chat spans but none exposed AIC fields in the field list requested.llm runtime selected,stream,process,loop,evaluatedpermission checks) with no AIC/cost fields present.Recommendations
actions/setup/js/send_otlp_span.cjs) actually sets a numericgh-aw.aic(or equivalent) attribute ongen_ai.chat/gen_ai.execute_toolspans. Current data suggests this attribute is either not emitted or not indexed/searchable in Sentry EAP.github.workflow(orgh_aw.workflow) consistently so future reports can aggregate AIC by workflow instead of falling back to generic transaction names likedefault/gateway.request.list_datasources/get_datasource;tempo_traceql-searchand related tools should be enabled so Grafana-side AIC cross-checks are possible in future runs.search_eventsenabled: If an embedded LLM provider becomes configured for the Sentry MCP server, re-run this report usingsearch_eventsfor richer natural-language AIC aggregation and to double check thelist_eventsfallback findings.References
span.op:ai*, 0 results): https://github.sentry.io/explore/traces/?query=span.op:ai*&project=4511347087179777&field=id&field=span.op&field=span.description&field=span.duration&field=transaction&field=timestamp&field=project&field=trace&sort=-timestamp&statsPeriod=24h&table=spangrafanacloud-traces, namegrafanacloud-ghaw-traces