Review the diff, not the vibe
AI writes code that reads well. It runs. If there are tests, they pass. So you glance at it, it looks right, and you ship it. That is the trap, because plausible and correct are not the same thing - and a plausible bug ships just as easily as plausible working code.
I have caught myself doing it. The output looks so clean and so confident that you extend it credit it has not earned. The fix is boring and it works: read the diff. Every line. Like a pull request.
Plausible is a specific kind of dangerous
Bad code that looks bad is easy - you feel the smell and you slow down. AI does not produce that. It produces code that looks like a competent person wrote it, which is exactly what disarms you. The variable names are good. The structure is clean. There is even a comment. And buried in the middle is an assumption that is reasonable in general and wrong for your system.
That is the failure mode to hold in your head: not obvious garbage, but confident, well-formatted wrongness.
Where it quietly fails
After enough of these reviews you start to see the same categories come up:
- Unhandled edge cases. The happy path is perfect. The empty list, the null, the duplicate, the timezone - untouched. The code is not wrong so much as unfinished, and it does not look unfinished.
- Reasonable assumptions that are wrong for you. It assumes the ID is a number because it usually is. In your system it is a string. Nothing about the code looks incorrect until it meets your data.
- Subtle logic errors in clean code. An off-by-one, an inverted condition, a
>=where you meant>. Small, easy to read straight past because everything around it is tidy. - Security gaps. User input concatenated into a query, a check that is missing, a secret logged in passing. It rarely announces itself.
- Over-engineering. You asked for a constant and got a configurable system with three strategy classes. It works. It is also five times the surface area you needed, and now you own all of it.
- Hallucinated or misused APIs. A method that does not exist, or one that does but takes different arguments than the code assumes. This one at least tends to fail loudly.
- Silent scope creep. The part that scares me most. You ask for one thing and it also renames a variable three functions away, or reformats a file, or quietly drops a branch it decided was dead. Nothing in the reply mentions it.
None of these are exotic. They are the ordinary output of a system optimizing for text that looks right, not behavior that is right.
What it actually is
Here is the framing that makes the discipline click: you are reviewing a pull request from a very fast, very confident junior developer. That is not an insult to the tool. It is an accurate description of what you are getting - enormous speed, real competence, no memory of your system's scars, and total confidence whether it is right or wrong.
You would never merge a junior's PR unread just because the description sounded sure of itself. Same standard here. The confidence in the output is not evidence. The diff is the evidence.
What I always check
I read the whole diff, but four things get deliberate attention:
The logic and the edge cases. Not "does this look reasonable" - actually trace it. What happens on empty input, on the boundary value, on the malformed row. Read the condition and ask what it does when the obvious case is not the case.
Whether it matches the conventions. Does it use the helper we already have, or reinvent one? Does it follow how the rest of the codebase does this exact thing? Consistency is not fussiness. It is what keeps the codebase legible six months out.
Whether it touched anything outside the scope. This is why you read the whole diff, not just the file you were thinking about. If I asked for a change to one function and the diff spans four files, every line I did not ask for is guilty until I understand why it is there.
Whether it is doing too much. Is the solution proportional to the problem? If it built a framework where a function would do, I send it back. Deleting the over-engineered version costs an afternoon now and saves the maintenance forever.
Make it prove the work
Reading catches a lot. It does not catch everything, because you can misread the same way the model mis-wrote. So the last step is: make it demonstrate.
Run it. Add the test that would fail if the logic were wrong. Take the screenshot of the actual screen. "Done" has to mean shown, not claimed. When I make the agent produce evidence - a passing test I can read, a real call with real output, the browser showing the thing working - the false confidence has nowhere to hide. The claim gets checked against reality instead of against how good the code looks.
This is also the honest test of whether your review was any good. If you cannot make it prove the work, you did not finish reviewing it.
Where the work moved
The thing to internalize is that your job changed. It used to be writing the code. Now the code arrives in seconds and your job is judging it - is this correct, does it fit, is it too much, did it stay in bounds. That judgment does not happen in the prompt. It happens in the diff. The diff is where the work lives now, and treating it as a formality is treating the actual job as a formality.
The speed is real. I would not build the way I do without it. But it is only a gain if the review is real too - otherwise you have not sped up shipping software, you have sped up shipping bugs.