AI Agent Code Review: What Pull Request Metrics Cannot See
New research shows AI-agent review and rationale can move outside pull requests. Learn what PR metrics still measure, what disappears, and how to preserve accountability.
By Ahmad Hajjar
On October 26, 2026, researchers Christoph Treude and Sebastian Baltes submitted a preprint asking what could replace pull requests in AI-agent software development. Their review covered 50 documents from 31 organizations and found that most proposed responses still keep pull requests in some form.
The diagram separates events recorded on a GitHub pull request from work that may remain in agent sessions, internal communication, or discarded proposals.
The important measurement finding is narrower. Pull request analytics can only analyze activity that reaches the pull request record. A review completed inside an agent conversation may influence the change without producing a submitted GitHub review, comment, or approval.
What the October 26 research found
The preprint separates five roles that pull requests often combine: integration, review, rationale, accountability, and external contribution.
The authors found that "most responses retained pull requests in some form." Several proposals moved review or rationale into agent conversations and session records. Only one reviewed organization, Zed, reported using a replacement in practice.
That makes this an early map of a changing workflow, not evidence that pull requests have already disappeared. It also exposes a data problem. The authors warn that "reviews conducted in agent conversations leave no pull-request comments."
A pull request can therefore remain the integration gate while no longer containing the complete review history.
What pull request metrics can still measure
Pull request metadata remains useful when the workflow records its decisions there.
GitQuick calculates time to first review from the pull request creation timestamp and the first submitted review timestamp. It counts review activity from submitted review states such as approved, changes requested, and commented. It can also measure merge timing, reviewer load, pull request size, and recorded review coverage.
These metrics answer concrete questions:
- How long did a pull request wait for its first recorded review?
- Which accounts submitted reviews or approvals?
- How many analyzed pull requests merged without a recorded review?
- Where did the visible pull request process slow down?
They do not answer whether an agent inspected the change in a separate session, whether a developer reviewed a patch in an editor, or why an unsubmitted proposal was discarded.
What the current public snapshots show
GitQuick's public Showcase listed 21 organizations when this article was drafted on October 29, 2026. The table below describes only activity recorded in the analyzed GitHub pull requests.
| Organization | Snapshot window | PRs analyzed | Recorded review events | Median time to first recorded review |
|---|---|---|---|---|
| GitHub | Sep 24 to Oct 24, 2026 | 5,337 | 10,967 | 0.27h, n=3,632 |
| OpenAI | Sep 29 to Oct 29, 2026 | 4,596 | 9,999 | 0.28h, n=2,677 |
| Anthropic | Sep 28 to Oct 28, 2026 | 3,308 | 1,186 | 27.87h, n=965 |
Each median is paired with its eligible sample. A pull request contributes to time to first review only when both the creation timestamp and first recorded review timestamp exist and form a non-negative interval.
The rows are not performance rankings. The organizations have different repositories, contribution models, automation, and review policies. The review-event count is also not the number of reviewed pull requests. One pull request can receive several submitted reviews.
Most importantly, the table cannot show an agent review that never produced a GitHub review event. It cannot show internal chat, session rationale, local inspection, or rejected work that never became a pull request. The public metadata cannot establish code quality or causality.
The missing-review ambiguity
When a merged pull request has no recorded review, two interpretations are possible.
The first is process bypass. The change may have merged without independent review.
The second is telemetry loss. Someone or something may have reviewed the change in an agent session, editor, internal tool, or synchronous discussion without writing the decision back to GitHub.
GitHub metadata alone cannot distinguish those cases. Calling every missing review a bypass overstates the evidence. Assuming every missing review happened elsewhere creates the opposite problem.
Treat the metric as an audit queue. Sample the affected pull requests, inspect repository policy, and check whether another system holds the review evidence.
How engineering teams should adapt measurement
Start by defining the review system of record. If GitHub approvals protect the branch, GitHub remains the accountability record even when analysis happens elsewhere. Require the final decision and reviewer identity to return to the pull request.
Then separate two layers of telemetry:
- Agent-session activity. Record the session identifier, model or tool, human interventions, validation steps, and the change it produced.
- Pull request activity. Record submitted reviews, requested changes, approvals, merge events, and the accounts responsible for them.
Link the layers with a durable change or session identifier. Do not assume a shared timestamp is enough.
When a team moves review outside pull requests, mark a measurement break. A drop in review comments may reflect a workflow change rather than less review. Historical comparisons need an annotation or a new baseline.
Finally, measure changes that never become pull requests. Agent proposals can be rejected, abandoned, or applied directly. A pull-request-only denominator excludes all of them. Useful complementary counts include sessions started, changes proposed, changes accepted, changes sent to a pull request, and changes merged.
Do not combine those counts into a funnel unless the cohorts and identifiers match.
Where GitQuick fits
GitQuick remains useful for the part of the delivery process represented in GitHub pull requests. Its sample sizes, stage timings, review states, reviewer loads, and Signals make that visible record easier to audit across repositories.
It does not currently ingest agent-session logs or internal review conversations. A team moving decisions outside GitHub should treat GitQuick as the pull-request layer of a broader measurement system, not as proof that it observed every review.
Takeaway
AI agents may change where review and rationale happen before they replace pull requests as integration gates.
Keep pull request metrics, but state their boundary. Record final decisions in the system that controls merging. Add session-level telemetry when agent conversations carry review work. Reset baselines when the workflow changes.
For related guidance, read Your Code Review Metrics Are Missing Part of the Story and AI Code Review Quality: What Workflow Metrics Can and Cannot Prove.
Explore the GitHub public Showcase, compare all public organizations, or try GitQuick on your organization.
Disclaimer: The cited research is a preprint submitted October 26, 2026. GitQuick uses public GitHub metadata for the Showcase. The snapshots cover the organization-wide windows shown above and were fetched on October 29, 2026. Pull request metadata cannot prove code quality, review completeness, developer performance, intent, or causality. GitQuick is not affiliated with or endorsed by GitHub or any organization mentioned.