Code Review Was Never Mainly About Bugs

September 9, 2026 · 18 min read

Software Engineer

A conveyor belt of identical pale grey slabs, each a stack of code diffs, running away into a dim indigo hall and vanishing in haze. One small wooden desk with a lit amber lamp sits beside the belt; only the two or three slabs inside its pool of light show any teal annotation marks. Every other slab passes untouched into the dark.

Every strategy for surviving the flood of machine-written pull requests optimizes the same thing: catching bugs. The research on code review says catching bugs was never the job it was best at.

note

Tooling and team practice in this area are moving fast; the vendor and adoption details here reflect September 2026. The research findings are older and have held up better than the tooling has.

Review is four jobs wearing one name

Code review looks like one activity because it happens in one place, at one moment, through one button. It is actually four jobs that happen to share a user interface.

The first is finding defects — the bug, the unhandled case, the off-by-one. The second is code comprehension: at the end of a review, at least one person who did not write the change knows it exists and roughly how it works. The third is design pressure — the quiet discipline of knowing someone will look, which kills a certain class of shortcut before it is ever typed. The fourth is shared ownership: after review, the change belongs to the team rather than to its author, which is what makes it safe for anyone to touch it later.

Bundling these four together was never a design decision. It was an accident of the tooling: the diff was the only artifact that existed, so every job got attached to it. That accident held up for twenty years because the volume was human-sized.

It stopped holding up when artificial intelligence (AI) agents started writing most of the code. And the important thing about the unbundling is that the four jobs are not equally replaceable.

What the research says review actually delivers

The best evidence on what review produces is over a decade old and comes from Microsoft. Alberto Bacchelli and Christian Bird observed, interviewed and surveyed developers and managers, then hand-classified hundreds of real review comments across diverse teams^[1]. They asked two separate questions: what do people think review is for, and what do reviews actually contain?

The two answers do not match.

Finding defects was the top stated motivation — for managers it was the first reason given, and for developers and testers it ranked first too. But when the comments themselves were classified, "defect" came out as only the fourth most frequent category out of nine, with 78 comments, or 14% of the total. The largest category was code improvements — better practices, dead code removal, naming — at 165 comments, or 29%. And the defect comments that did appear were mostly small: 65 of the 78 were logical issues at what the authors call the "micro" level, not the conceptual or design-level problems practitioners said they expected to catch.

The paper's own summary of what review is really doing is blunt:

Moreover, we find that code and change understanding is the key aspect of code reviewing and that developers employ a wide range of mechanisms to meet their understanding needs, most of which are not met by current tools.

Alongside understanding, the study names three benefits that show up whether or not anyone intended them: knowledge transfer, increased team awareness, and the creation of alternative solutions to problems.

So the honest description of code review, before AI ever entered the picture, is that it was a comprehension ritual with a defect-finding cover story. Teams bought it for the bugs and were paid mostly in understanding. That is a fine trade, right up until someone notices the cover story is the only part being measured — and replaces the ritual with a machine that does the cover story better.

The human read was already a fiction

There is a second, less comfortable finding, and it is the one that decides whether "just review everything by hand" is a real option.

Reviewers have a hard ceiling, and it is low. The largest study of lightweight review ever run — SmartBear's ten-month case study of 2,500 reviews covering 3.2 million lines of code (LOC) written by 50 developers at Cisco — found that defect detection collapses with both size and speed^[2]. Reviewers "wear out" after about 60 minutes and stop finding additional defects, which the authors translate into a practical limit of 300–400 LOC before performance drops. On rate, reviewers slower than 400 lines per hour were above average at uncovering defects; past 450 lines per hour, defect density came out below average in 87% of cases. Their single best piece of advice was to review 100 to 300 lines at a time and spend 30 to 60 minutes on it.

Google's numbers show a system built to respect exactly that ceiling. Across roughly 9 million reviewed changes, over 35% modify a single file, more than 10% modify a single line, and the median change is 24 lines^[3]. Fewer than 25% of changes have more than one reviewer; the median reviewer count is 1. As the authors put it, "the majority of changes are small, have one reviewer and no comments other than the authorization to commit."

Twenty-four lines. That is what a functioning code review process looks like at the largest scale anyone has measured it — not because Google reviewers are lazy, but because a 24-line change is inside the window where a human read actually finds things.

Now put the current volume next to that. GitHub's Octoverse reports 43.2 million pull requests (PRs) merged per month, up 23% year over year, and nearly a billion commits pushed in a single year, up 25.1%. The Copilot coding agent alone authored more than a million PRs between May and September 2025^[4]. And the agent-written PR is not a 24-line PR; the whole appeal of delegating to an agent is that it returns more than you would have typed.

This reframes the panic. The industry keeps saying AI broke code review. AI did not break code review — it removed the precondition code review always silently depended on. The human read stopped being effective somewhere north of 400 lines, and we have been merrily approving 1,200-line diffs for years while telling ourselves the practice still worked. Agents did not introduce the fiction. They industrialized it until it became impossible to keep believing.

What each coping strategy actually buys

Gergely Orosz's survey of how teams are actually responding^[5] catalogues the field well: humans review the AI review, triage by "blast radius", review the plan or tests or database schema instead of the implementation, produce less code, review everything by hand, or drop human review altogether.

Read that list against the four jobs and a pattern falls out immediately. Almost every strategy is scored on the defect column, and almost none of them are scored on the other three.

Approach	Defect finding	Code comprehension	Design pressure	Cost per change
Humans review the AI review	Strong	Weak	Medium	Low
Triage by blast radius	Good	Medium	Medium	Low
Review the plan, tests or schema	Medium	Good	Strong	Medium
Produce less code (small PRs)	Strong	Strong	Good	High
Review everything by hand	Medium	Strong	Strong	High
No human review	Good	None	Weak	Lowest

strong good medium weak

Two cells in that table are worth arguing about.

The first is "review everything by hand", scored only medium on defect finding. That looks wrong until you remember the Cisco ceiling: a human reading a 900-line agent diff is not doing the thing the research measured. Reading everything and finding things are different activities, and past a few hundred lines only the first one is still happening.

The second is that the machine genuinely wins the defect column. Uber's uReview analyzes over 90% of its roughly 65,000 weekly diffs; engineers mark 75% of its comments useful, against 51% of human-written comments judged to be real bugs and addressed in the same changeset^[6]. That is not a tie. On the job review was sold on, a well-tuned pipeline outperforms the people it replaced.

Which is exactly why the argument cannot end there. If the defect column is settled, the only question left is what you are doing about the other three — and the table says most teams have answered "nothing", without ever putting it that way.

Blast radius is the right instinct on the wrong axis

Risk-based triage is the most sensible thing on that list, and it is the one I would push hardest on, because it is a good idea sorted along a single axis.

Blast radius asks: if this change is wrong, how much breaks? Authentication, non-additive schema migrations, public application programming interface (API) surface, payment paths — those get a human. Everything else merges on green.

But "how much breaks if this is wrong" and "how expensive is this to understand later" are different questions with different answers. A change can be entirely safe to ship and still be the reason nobody can debug the system in eighteen months.

The top-left quadrant is where the bill accumulates. A new caching layer, a hand-rolled retry policy, a genuinely clever optimization: none of them will take production down this week, so every risk-based rule waves them through. All of them are load-bearing, non-obvious, and now understood by exactly nobody — because the agent that wrote them does not remember, and no human ever read them.

Blast-radius triage is not wrong. It is incomplete, and it is incomplete in a direction that produces no immediate symptoms, which is the worst possible direction for a heuristic to fail in.

The bill that arrives late

Unbought comprehension does not bounce. It accrues, and it comes due in three places.

Incidents. Debugging is comprehension under time pressure. The 03:00 question is never "is this code correct" — it is "what was this supposed to do". A team that has never read its own caching layer is not going to read it well for the first time during an outage, and pointing the agent at it only works if someone can tell whether the answer is plausible.

Onboarding. New engineers used to learn a codebase by having their changes reviewed and by reviewing others'. Bacchelli and Bird found that knowledge transfer runs in both directions through review. Delete the ritual and you have deleted the mechanism, usually without noticing, because nobody logs "did not learn the system this quarter".

Drift. Design pressure is what stops fourteen slightly different retry helpers from existing. Nothing else in the pipeline does that job: linters check style, tests check behavior, and neither has an opinion about whether this is the fifth implementation of the same idea.

The DevOps Research and Assessment (DORA) team's 2025 report is the sharpest framing of why this bill is easy to miss. Across nearly 5,000 respondents, 90% use AI at work and more than 80% believe it has increased their productivity — while 30% report little or no trust in the code it generates^[7]. The report's central finding is that AI is an amplifier: it magnifies an organization's existing strengths and weaknesses rather than supplying new ones. A team that already understood its system will use agents to understand it faster. A team that did not will now not-understand it at ten times the rate.

And there is a reason the belief goes unchallenged. When METR ran a randomized controlled trial with 16 experienced open-source developers across 246 real tasks in their own repositories, the developers took 19% longer with AI tools available — while estimating afterwards that they had been sped up by 20%^[8]. METR is careful about the limits of that result, and it says nothing about agent workflows a year later. But the shape of the error is the point: a large gap between felt and actual, invisible from the inside. Reading the output is one of the few loops that closes that gap. Remove it and the only signal left is the feeling.

Buying comprehension deliberately

The conclusion is not "keep reviewing everything". At current volumes that is arithmetic nobody wins, and the Cisco ceiling says the read stopped working long before the calendar did.

The conclusion is narrower: if you remove the human read, name the thing that now buys comprehension. The failure mode is not deleting review. It is deleting review and buying nothing, because the line item was never on the budget in the first place — it arrived free, bundled with a bug hunt.

Mechanisms that actually buy it, roughly in order of cost:

A plan reviewed before the code exists. The cheapest comprehension available, because it is the one artifact written in prose by something trying to be understood. Reviewing a spec is not a compromise; it is review moved to where a human still has an edge.
Schema and contract review, always by hand. Data outlives code. Business logic can be regenerated in an afternoon; a non-additive migration cannot be un-run.
Architecture decision records. One page per non-obvious decision, capturing why — the exact thing a diff never shows and an agent never remembers.
Enforced small diffs. The only strategy in the whole table that scores well on every column, and it is unpopular for one honest reason: it costs the author time to split work an agent produced in one shot.
Deliberate reading with no PR attached. Pick a subsystem, read it, write a paragraph. Unglamorous, and the only mechanism on this list that scales down to a team of one.

Note what is missing from that flowchart: bugs. Not because they stopped mattering, but because they are the one part of this that is genuinely handled. Run the model review on everything, all the time — it is cheap, it is tireless, and on Uber's numbers it is better at the job than the humans it replaced. Then spend the human attention you just freed on the three things the model cannot do at all.

Where the human read still earns its cost

The version of this I would defend is short. Give the machine the defect pass everywhere, without exception and without a human double-checking most of it — reviewing the review is mostly a comfort ritual, and the table above shows it buying almost nothing beyond what the review already found. Keep the human read for changes that alter the shape of the system rather than its behavior: schemas, contracts, boundaries, anything a future engineer will have to reason about rather than merely run. And treat comprehension as a named, funded line item, not as a by-product you hope falls out of some other process.

The uncomfortable part is that this was already true in 2013. Review was delivering understanding and being credited with bug-finding, and nobody had to resolve the discrepancy because both arrived in the same envelope. Agents split the envelope. They took the job we were measuring and left behind the job we were actually buying — and the only real mistake available now is to look at the measured half, see it well covered, and conclude the whole thing is handled.

If you want the mechanics of wiring a model review into the pipeline, I wrote about the task-and-review loop separately. This post is about the part that loop cannot give you, no matter how well you build it.

Review is four jobs wearing one name​

What the research says review actually delivers​

The human read was already a fiction​

What each coping strategy actually buys​

Blast radius is the right instinct on the wrong axis​

The bill that arrives late​

Buying comprehension deliberately​

Where the human read still earns its cost​

References​