Skip to main content

Skimming AI-Written Code Is a Bet: Review the Structure, Not the Diff

· 17 min read
Pere Pages
Software Engineer
A small figure inspects a single glowing brick through a large magnifying glass while the whole brick tower above leans badly out of true; an architectural blueprint lies unrolled and ignored under their feet

When a coding agent writes most of the code, reading every diff stops being realistic, and "reading diagonally" — skimming — becomes the default. It is a real bet, not a safe one: architecture doesn't break in any single diff, it erodes across all of them, and skimming checks exactly the part that was never the problem.

The mental model: the diff is the wrong unit

A diff shows whether one change is reasonable. Architecture is a property of the whole codebase: which modules exist, which way dependencies point, where state lives, how many ways there are to do the same thing. None of that is visible in a single change, and all of it is what decides whether the project is still pleasant to work in six months from now.

An artificial intelligence (AI) coding agent is very good at the first and structurally blind to the second. Each task gets a competent, locally sensible answer. The trouble is the sum.

Skimming a diff answers "is this change fine?", and the answer is almost always yes — the question that matters is "is the codebase still the shape we decided on?", and no diff can answer it. So the fix isn't to read more code. It's to move attention from the lines to the structure, and to make the intended structure something the agent can read.

Why an agent drifts

An agent drifts for a mechanical reason, not a competence one: it starts every session without the memory a long-serving engineer carries, and it treats whatever is already in the repository as the convention to follow.

A human who has been on a project for a year holds the whole shape in their head. They know there is already a date helper, that the data layer is the only place allowed to call the network, that the odd-looking wrapper exists because of an incident last spring. An agent knows what is in its context window — the working memory of a large language model (LLM), holding the conversation, the files it has read and the output of the commands it has run. Anthropic's own guidance for Claude Code names this as the constraint everything else follows from: the window fills fast, and performance degrades as it fills.[1] The helper written three weeks ago in another folder is simply not there unless something puts it there.

The second half is subtler. An agent is a diligent imitator: it looks at the surrounding code and matches it. That is the right behaviour, and it means a mistake made once becomes a pattern by the third time it is copied. There is nobody in the loop who remembers that the first instance was a shortcut.

Four ways it goes wrong

The drift shows up in four recognisable forms, and what they share is that each one passes a skim.

Local optimization

Each task is solved well in isolation, and the solutions don't know about each other. After a few weeks of feature work the same concern has several implementations:

// features/orders/api.ts — week 1
export const getOrders = () => fetch('/api/orders').then((r) => r.json());

// features/invoices/useInvoices.ts — week 2
const { data } = useQuery({ queryKey: ['invoices'], queryFn: () => http.get('/invoices') });

// features/customers/service.ts — week 4
export class CustomerService {
async list() { return (await axios.get(`${BASE}/customers`)).data; }
}

Three diffs, each fine. Together: three ways of fetching data, three error-handling behaviours, and a new dependency nobody decided on.

Drift, not collapse

Architecture rarely explodes in one commit. It erodes through expedient exceptions: a value passed down through four components "temporarily", a utils.ts that grows into a module everything imports, a user-interface component that reaches into the database layer once because the proper route needed two more files. Research on architecture erosion describes the same thing in human teams, and the symptoms it catalogues — layering violations, cyclic dependencies — are exactly these.[2] An agent only changes the speed.

Over-engineering by default

Left without pushback, an agent tends to build for a future nobody asked for: an abstraction for something that happens once, a configuration layer with a single setting, generic types that hide what the function actually takes.

// Asked for: format a price in euros.
export function createFormatter<T extends FormatterConfig = DefaultFormatterConfig>(
strategy: FormattingStrategy<T>,
options?: Partial<FormatterOptions<T>>,
): Formatter<T> { /* 60 lines */ }

On a skim this reads as thoroughness. It is cost: every later change has to understand and preserve machinery with one caller.

Missing the "why"

The agent follows the conventions it finds, including bad ones it introduced itself. A code comment or a rule says what to do; almost nothing in a repository says why, and without the why there is no way to tell a deliberate constraint from an accident. So accidents get the same respect as decisions, and mistakes compound instead of getting corrected.

Failure modeWhat the diff looks likeWhat it costs later
Local optimizationA clean, self-contained solutionDuplicate helpers, inconsistent behaviour, fixes applied in one copy only
DriftA small, pragmatic exceptionBoundaries that no longer mean anything; changes ripple everywhere
Over-engineeringCareful, flexible, well-typed codeIndirection to maintain for cases that never arrive
Missing the "why"Faithful imitation of existing codeBad patterns harden into the house style

The evidence, such as it is

The data available so far points the same way as the anecdotes: AI-assisted codebases get more code and more duplication, and the teams that fare well are the ones with structure around the tool.

GitClear's analysis of 211 million changed lines from 2020 to 2024 found that, between 2021 and 2024, copy-pasted lines rose from 8.3% to 12.3% of changes while refactoring-related lines fell from 25% to under 10%; 2024 was the first year in its data in which copy-pasted lines outnumbered moved ones.[3] Moving code is what consolidation looks like in version control, so that is local optimization measured at scale. The 2025 report from DevOps Research and Assessment (DORA) frames AI as an amplifier that magnifies "an organization's existing strengths and weaknesses", with the returns coming from the surrounding system rather than the tool.[4]

Neither source measures architectural drift under an autonomous agent directly, and nothing reliable does yet. So the following is a practitioner's estimate, not a statistic: with zero steering, the odds of meaningful architectural drift over a few weeks of agent-heavy work are well above even. With a written specification and light structural review they drop a long way, and most of the speed survives.

That matters because of where the bill lands. Martin Fowler's design stamina hypothesis — explicitly a hypothesis — is that neglecting design buys speed only briefly, after which every feature costs more than it would have.[5] An agent shortens the brief part: it reaches the point where the mess slows things down faster than a human team would, because it produces changes faster. The rest of this post is five rails that keep the speed without the erosion, in the order they pay off.

Rail 1: write the architecture down

The single highest-leverage move is a short written description of the intended structure, in a file the agent reads at the start of every session. It replaces the memory the agent doesn't have.

Claude Code reads CLAUDE.md automatically; other agents have an equivalent, and a plain ARCHITECTURE.md referenced from it works everywhere. What goes in is what a skim can't recover from the code: the folder structure and what each part is for, the direction data flows, where state lives, and — most useful of all — what is forbidden, with the reason.

# Architecture

## Layout
- `src/features/<name>/` — one folder per feature; features never import each other.
- `src/shared/` — code used by 2+ features. Nothing here imports from `features/`.
- `src/data/` — the ONLY place that talks to the network.

## Data flow
- Server state: TanStack Query, hooks live in `src/data/`. Components never call `fetch`.
- Client state: local `useState` first; a store only when 2+ routes need the value.

## Forbidden (and why)
- No new dependencies without asking — every one is a permanent maintenance cost.
- No `utils.ts` catch-alls — name the module after what it does.
- No abstraction with a single caller — we inline until the second use exists.

Two properties matter more than completeness. Keep it short: Anthropic's guidance is blunt that a bloated instruction file causes the agent to ignore the instructions in it, and suggests asking of every line whether removing it would cause mistakes.[1] And write the reasons: the "and why" half of each line is what lets the agent — and the next human — tell a constraint from an accident. Decisions with real history behind them deserve an architecture decision record the file can point to.

Prose is advisory, though. For the two or three rules whose violation does the most damage, move them from prose to a check that fails. Import boundaries are the classic case, and tools such as dependency-cruiser or ESLint's no-restricted-imports enforce them in continuous integration (CI):

// .dependency-cruiser.cjs
module.exports = {
forbidden: [
{
name: 'features-are-isolated',
severity: 'error',
from: { path: '^src/features/([^/]+)/' },
to: { path: '^src/features/([^/]+)/', pathNot: '^src/features/$1/' },
},
{
name: 'only-data-imports-http-clients',
severity: 'error',
from: { pathNot: '^src/data/' },
to: { path: 'node_modules/(axios|ky)' },
},
],
};

A dependency checker only sees imports, so the global fetch needs its own guard — ESLint's no-restricted-globals, switched off for src/data/. An agent that runs the linter gets the boundary as an error message it can act on, which is far more reliable than a sentence it may not weigh. The full ladder from written rule to hard guarantee is its own subject, covered in From Prose to Guarantees.

Rail 2: review the structure, not the code

With the structure written down, review becomes a comparison against it, and only a small share of changes need the comparison. As a rule of thumb, maybe one diff in ten alters the shape of the system; that tenth is where the damage is done, and it is identifiable without reading a line of logic.

The signals are all visible from the file list and the dependency manifest:

  • New folders or top-level files — a new concept has entered the system. Is it in the right place, and is it really new?
  • New dependencies — a permanent cost and often a second way of doing something the project already does.
  • New cross-module imports — the dependency graph just changed direction somewhere.
  • Anything touching shared state or shared code — the blast radius is every consumer.
  • New files named utils, helpers, common, manager — tomorrow's god module, the one everything ends up depending on.

Three commands surface most of it before opening a single file:

git diff --stat --diff-filter=A main...HEAD # files that didn't exist before
git diff main...HEAD -- package.json # dependencies added or changed
git diff main...HEAD -- src/shared src/data # edits to code everyone depends on

That turns review into triage:

The last branch matters. Sometimes the agent's deviation is right and the document is stale; changing the document in the same pull request keeps the written architecture true, which is the only thing that makes it worth reading. This is the same reallocation argued at length in Code Review Was Never Mainly About Bugs: machines are good at the line-level checks, so the scarce human attention belongs on the decisions a machine can't see are decisions.

Rail 3: make it audit itself

An agent is poor at noticing its own drift while working and good at finding it when asked directly, because the audit puts the whole concern into context at once — the very thing a feature task never does.

The prompts that work are inventory questions with a request to flag disagreement:

List every place this codebase fetches data. Group them by approach, name the approach the architecture document prescribes, and list the files that deviate.

List every module imported by more than five others. For each, say in one sentence what it is for. Flag any whose purpose you can't state in one sentence.

Find abstractions — generic functions, factories, configuration objects — with exactly one caller, and propose inlining them.

Run the audit in a fresh session rather than the one that wrote the code; Anthropic's guidance makes the same point about review in general, noting that a fresh context isn't biased toward code it has just written.[1] Every couple of weeks is enough, or after any burst of feature work. Treat the output as a list of candidates rather than a to-do list: an auditor asked to find inconsistencies will always find some, and fixing every one is its own form of over-engineering. The ones that matter are those that contradict the written architecture — and each either becomes a cleanup task or a line added to the document so it doesn't recur.

Rail 4: read the tests

If only one kind of file gets a careful human read, make it the tests: a test file states what a module believes its contract is, in a fraction of the space the implementation takes.

Twenty lines of it('…') descriptions reveal what the implementation hides in two hundred: which inputs the module thinks it accepts, which errors it thinks are its job, what it assumes about its collaborators. Wrong assumptions are visible there — a test that mocks four other modules is describing a boundary problem; a test suite with no failure cases is describing a module that doesn't know it can fail. And tests double as the mechanism that makes the rest of the workflow safe to skim: Anthropic's guidance calls a runnable check the difference between a session to watch and one to walk away from.[1]

The habit to avoid is approving tests by their count or their green tick. An agent asked to make tests pass will make them pass; whether they assert the right contract is the human part.

Rail 5: small tasks, and a plan before anything cross-cutting

The cheapest place to catch an architectural mistake is before it is written, so anything that touches more than one module should start as a plan rather than an implementation.

The economics are lopsided. Rejecting a bad plan costs the seconds it takes to read a paragraph and say "no, put that in the data layer". Unwinding a bad implementation — after other changes have been built on top of it — costs hours, and is exactly the work that tends not to get done. Claude Code has a plan mode for this, in which the agent explores and proposes without editing files; its documentation recommends planning when a change spans multiple files or the approach is uncertain, and skipping it when the diff could be described in one sentence.[1]

A plan is also the one artefact where structure is the whole content. It lists the files to create, the modules to touch and the dependencies to add — precisely the signals from the structural review, presented before they cost anything. Small tasks help for the same reason: a task scoped to one module can't quietly rearrange three.

Which rail catches what

No single rail covers all four failure modes; together they overlap enough that each mode is caught at least twice.

RailLocal optimizationDriftOver-engineeringMissing the "why"
1. Written architecturePreventsHelpsHelpsPrevents
2. Structural reviewHelpsCatchesHelps
3. Self-auditCatchesHelpsCatches
4. Reading the testsHelpsHelpsHelps
5. Plan first, small tasksHelpsPreventsPrevents

The total cost is modest: an hour to write the first version of the document, a minute of triage on most changes and a proper look at the structural tenth, an audit every couple of weeks, and the discipline to ask for a plan. Against that, the agent still writes nearly all the code.

The short version

  • Skimming is a bet, and the odds are poor. Each diff looks reasonable; the erosion is in the sum, which no diff shows.
  • The cause is mechanical. No memory across sessions, and faithful imitation of whatever is in the repository — including earlier mistakes.
  • Four failure modes: local optimization, drift through small exceptions, over-engineering, and conventions followed without their reasons.
  • Write the architecture down — short, with the forbidden list and the reasons — and turn the most damaging rules into failing checks.
  • Review structure, not code: new folders, new dependencies, new cross-module imports, shared state. About a tenth of changes.
  • Ask for audits in a fresh session, read the tests as contracts, and get a plan before anything cross-cutting.
  • Keep the document true. When the agent is right and the document is wrong, change the document in the same pull request.

References

  1. Best practices for Claude Code — Claude Code Docs, Anthropic
  2. Ruiyin Li, Peng Liang, Mohamed Soliman, Paris Avgeriou, Understanding Software Architecture Erosion: A Systematic Mapping Study — Journal of Software: Evolution and Process (arXiv:2112.10934)
  3. AI Copilot Code Quality: 2025 Data Suggests 4x Growth in Code Clones — GitClear
  4. State of AI-assisted Software Development 2025 — DORA
  5. Martin Fowler, Design Stamina Hypothesis — martinfowler.com