Repository navigation
Cut AI Moderator token usage by pre-fetching capped moderation context - #52891
Merged
Merged
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Contributor
|
Thanks for diving into the AI Moderator token usage investigation 🔬! This draft PR is well-scoped and properly linked to #52739. The task checklist is clear:
Looks like the investigation phase is wrapping up nicely. Good luck with the optimization pass!
|
Copilot
AI
changed the title
[WIP] Investigate high token usage in AI Moderator workflow
Cut AI Moderator token usage by pre-fetching capped moderation context
Aug 15, 2026
pelikhan
marked this pull request as ready for review
August 15, 2026 13:20
Contributor
There was a problem hiding this comment.
Pull request overview
Reduces AI Moderator token usage by moving GitHub context retrieval into a bounded deterministic prefetch step.
Changes:
- Prefetches capped issue, comment, and PR diff context.
- Rewrites moderation instructions to use prefetched files with fewer turns.
- Regenerates the compiled workflow.
Show a summary per file
| File | Description |
|---|---|
.github/workflows/ai-moderator.md |
Adds context prefetching and revises the agent prompt. |
.github/workflows/ai-moderator.lock.yml |
Incorporates the generated prefetch step. |
Review details
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
Suppressed comments (2)
.github/workflows/ai-moderator.md:92
- Swallowing a comment-fetch failure leaves
commentnull while theissue_commentrun continues, so the agent may assess the parent issue instead of the triggering comment and emit an incorrect action. Fail the prefetch when the required comment cannot be retrieved.
gh api "repos/$EXPR_GITHUB_REPOSITORY/issues/comments/$COMMENT_ID" > "$RAW_COMMENT" || echo '{}' > "$RAW_COMMENT"
.github/workflows/ai-moderator.md:96
|| truealso hides genuine authentication, network, and API failures, producing an empty patch that the agent can treat as a clean diff and labelai-inspected. Capture the command output before truncating so a real fetch failure stops the run without theheadpipeline's SIGPIPE concern.
if [ -n "${PR_NUMBER:-}" ]; then
{ gh pr diff "$PR_NUMBER" --repo "$EXPR_GITHUB_REPOSITORY" || true; } \
| head -n "$DIFF_MAX_LINES" > /tmp/gh-aw/agent/pr-diff.patch
- Files reviewed: 2/2 changed files
- Comments generated: 3
- Review effort level: Balanced
| agent: | ||
| runtime: gvisor | ||
| sudo: false | ||
| pre-agent-steps: |
| echo '{}' > "$RAW_COMMENT" | ||
| ITEM_NUMBER="${ISSUE_NUMBER:-${PR_NUMBER:-}}" | ||
| if [ -n "$ITEM_NUMBER" ]; then | ||
| gh api "repos/$EXPR_GITHUB_REPOSITORY/issues/$ITEM_NUMBER" > "$RAW_ISSUE" || echo '{}' > "$RAW_ISSUE" |
| PR_NUMBER: ${{ github.event.pull_request.number }} | ||
| COMMENT_ID: ${{ github.event.comment.id }} | ||
| BODY_MAX_CHARS: "6000" | ||
| DIFF_MAX_LINES: "200" |
Contributor
|
🎉 This pull request is included in a new release. Release: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
AI Moderatorworkflow was flagged for abnormally high token usage. A 30-run / 14-day sample confirms the lead: 3.0M tokens across 30 runs (~100k/run) versus a ~26k fleet average, with 47–109 GitHub API calls per run.Root cause
Audit of run 31867160774: 11 model requests × ~12–15k input tokens each. The workflow runs in
gh-proxymode (no MCP tool schemas), so cost isturns × ambient context. The prompt explicitly told the agent to fetch all issue/comment/PR content itself — including an uncappedpull_request_read get_diff— so every fetch/analysis step added another full-context turn.Changes to
.github/workflows/ai-moderator.mdpre-agent-stepsDataOps prefetch — deterministicgh apicalls write a compact/tmp/gh-aw/agent/moderation-context.json(event, actor, item, comment; bodies clipped to 6000 chars) and/tmp/gh-aw/agent/pr-diff.patchcapped at 200 lines. Same pattern asshared/pr-diff-data-fetch.mdused by PR Code Quality Reviewer.get_diff; the spam log is read in the same turn as the context files.Bodies remain unsanitized original content, preserving detection fidelity; only length is bounded.
Notes for reviewers
ai-moderator.mdcarriesredirect: githubnext/agentics/workflows/ai-moderator.md@main, so a futuregh aw updatewould overwrite this. The same change should be landed upstream to make it stick.experiments:variant if the reviewer prefers an A/B before promoting.gh aw auditsuggests a downgrade is plausible, but that trades detection quality and should be measured separately.