Add unreviewed-merge metric to the reviewer table - #2
Merged
Merged
Conversation
Surfaces how often a PR was merged while a reviewer's requested review was still outstanding — the reviews that were asked for but overtaken by the merge, which the existing turnaround percentiles cannot show because they only measure reviews that were actually submitted. The query now pulls each PR's REVIEW_REQUESTED_EVENT and REVIEW_REQUEST_REMOVED_EVENT timeline items. Those events are replayed chronologically per reviewer to decide whether the request was still open at merge time, so: - a request withdrawn before the merge is not a missed review, and - a request added after the merge is not counted either. Team and mannequin requests are skipped since they name no individual to attribute the miss to; bot requests follow the existing --include-bots flag. Reviewers who were requested but never reviewed now appear in the table, which previously was keyed only on submitted reviews. Their percentile columns render as "-". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds an Unreviewed column (with a Requested denominator) to the Reviewers table: how often a PR was merged while that reviewer's requested review was still outstanding.
Why
The existing turnaround percentiles only measure reviews that were actually submitted, so a review that was asked for and then overtaken by the merge is invisible — it silently drops out of the stats rather than showing up as a slow one. This makes that case countable.
How
The GraphQL query now pulls each PR's
REVIEW_REQUESTED_EVENTandREVIEW_REQUEST_REMOVED_EVENTtimeline items.review_requestsreplays those events chronologically per reviewer to determine whether the request was still open at merge time, andreviewer_statscounts it as unreviewed when the PR merged with the request outstanding and no review from them submitted bymergedAt.Three judgment calls worth a reviewer's attention:
--include-botsflag.Side effect: reviewers who were requested but never reviewed now appear in the table, which previously was keyed only on submitted reviews. Their percentile columns render as
-.Verification
cargo buildandcargo clippyare clean.Ran against
cli/cliover a two-month window and cross-checked every reviewer against an independent Python implementation over the same raw API data — all six matched exactly (babakks 80/16, BagToad 73/23, williammartin 37/16, tidy-dev 21/7, sergiou87 2/0, niik 1/1). That data exercised bot requests (76, correctly excluded by default and included under--include-bots), team requests (231, skipped), and removal events (4).Hand-verified cli/cli#14019: niik was requested at 11:23:58 and it merged at 11:31:05 with no review — correctly flagged.
No live PR exercised the "request added after merge" branch, so that path is reasoned-through rather than observed.
Notes
cargo fmt --checkstill reports one diff atsrc/main.rs:594. That is pre-existing onmaster(confirmed viagit show master:src/main.rs) and was left alone rather than adding unrelated churn.Size (p75)/Size (p99)columns the code emits today.🤖 Generated with Claude Code