Claude already reviews pull requests well. A 14-line file made it review them my way.
I planted six problems in a pull request and asked Claude Code to review it, first with no rules, then with a short REVIEW.md and a hook. Same bugs found, a very different review. Here's the setup and the honest numbers.
Same 6 bugs found, a far more useful review
Level: Intermediate. About 25 minutes. Needs git, Node 20+ and a Claude Code login.
I expected a trick here. Something that turns a mediocre AI code review into a good one.
That's not what happened. Claude Code's review, with no instructions beyond "review this," was already strong. What changed with a short rules file was something else: the review got shorter, more consistent, and it started catching things specific to my project that no general review would know about.
Let me show you, because the difference is easy to miss if you only look at the bugs found.
The pull request I booby-trapped
I made a tiny order service, then wrote a pull request the way a hurried teammate might. It adds an order search feature and, in the same change, quietly does these things:
- Builds a SQL query by gluing user input into a string (a classic SQL injection).
- Changes paging from
(page - 1) * pageSizetopage * pageSize, so page 1 skips the first ten orders. - Rewrites the two tests so they pass against the bug: an exact check of the order IDs becomes "the list isn't empty."
- Deletes the
npm teststep from the CI workflow. - Cuts payment retries from 3 to 1. The README still says 3.
- Adds a comment to a vendored third-party file, plus a pile of style sins:
var, a missing semicolon, everything crammed on one line.
The tests pass. That's the nasty part. On this branch, npm test is green.
Round one: just ask
claude -p "Review the pull request on this branch. Compare it with main (git diff main...HEAD) and write your review to review.md."
It took 60 seconds. And it was good. It found the injection, the paging bug, the weakened tests, the missing CI step and the billing change. It even spotted a rounding bug in a file the PR never touched (toFixed(1) instead of toFixed(2), so $19.99 prints as $20.0), because the PR adds the first real caller.
If you've been assuming you need a clever setup to get a decent review, you probably don't. Hold on to that.
But read it as a teammate would. It runs 587 words. It labels things "Blocking," "Should fix" and "Nits" with no count up front. Two of its three nits are style complaints about a semicolon and var. It saw the retry change but never connected it to the README, which still promises three retries. And there's no way to tell at a glance which findings need a human to decide rather than a developer to fix.
Round two: fourteen lines of rules
Anthropic's Claude Code team describes a file called REVIEW.md, written by the people who own the code, that says what matters in their part of it. They don't publish theirs, so I wrote my own for this repo:
A few of those lines are worth a closer look.
"Check that README and comments are still true." This is the one a generic review will never do on its own, because it has no idea your README is a promise to users.
"If a test was changed, ask whether it was changed to be right or just to pass." That one targets the exact trick in the pull request.
"Anything under src/billing/ or .github/workflows/ needs a human." Some changes shouldn't be settled by a bot. This tells Claude to hand them over with a question for the author.
The nit cap. At most three, and never formatting. This is what stops the review from reading like a lecture.
One detail that matters more than it looks: Claude reads this file from main, not from the pull request branch.
git show main:REVIEW.md
If it read the file from the PR itself, the PR could quietly rewrite the rules it's being judged by. The Anthropic team does it the same way, and the reason is just as plain.
The hook that checks the shape of the answer
Rules in a text file are requests. A hook is a lock. I added a Stop hook that checks the review before Claude is allowed to finish:
// .claude/hooks/check-review.js
const fs = require('fs');
const fail = (m) => { console.error('review.md rejected: ' + m); process.exit(2); };
if (!fs.existsSync('review.md')) fail('review.md was not written');
const lines = fs.readFileSync('review.md', 'utf8').split('\n');
if (!/^\d+ Important, \d+ Nit, \d+ Pre-existing$/.test(lines[0].trim()))
fail('first line must be exactly "N Important, N Nit, N Pre-existing"');
const findings = lines.filter((l) => /^\s*[-*]\s*\**(Important|Nit|Pre-existing)/.test(l));
const bare = findings.filter((l) => !/[\w./-]+:\d+/.test(l));
if (bare.length) fail('every finding needs path:line, missing on: ' + bare[0].trim().slice(0, 80));
if (findings.filter((l) => /Nit/.test(l)).length > 3) fail('more than 3 nits');
{ "hooks": { "Stop": [ { "hooks": [ { "type": "command", "command": "node .claude/hooks/check-review.js" } ] } ] } }
Exit code 2 sends the message back to Claude and tells it to keep working. To see what the hook is for, I ran it on round one's review. It rejects it straight away:
Round two result
Same pull request, same model, now with REVIEW.md and the hook. It took 93 seconds, a bit longer than before. The review came back at 357 words, and it opens with a line you can scan in a second: 3 Important, 1 Nit, 1 Pre-existing.
What changed:
- It connected the code to the README. The paging finding cites
README.md:4("pages are 1-based"). The billing finding citesREADME.md:5("retried 3 times"). Round one missed both. - The semicolon and
varcomplaints are gone. The only nit left is about a search that matches part of one field but the whole of another, which is a fair question about what the author meant. - The old rounding bug moved to its own "Pre-existing" line, with no blame attached.
- The billing change and the missing CI step went under "Needs a human," each with a question to ask the author. The review hands the decision to a person instead of making it.
- It skipped the vendored file and just noted it was edited.
What I'm not telling you it does
Two honest caveats.
First, my own rule moved the CI problem. In round one, Claude called the deleted test step "blocking." In round two it sits under "Needs a human," because I told it workflow files go there. That's what I asked for, but if you'd rather it shout, change the rule.
Second, I can't tell you whether the hook blocked Claude's first attempt in round two. It used 9 turns against 6 in round one, which hints that it might have, but I didn't log it. What I can say is that round one's output fails the check, so the hook would have pushed it to reformat.
Cost, as Claude Code reported it: about $0.61 for round one and $0.43 for round two. I'm on a Pro plan, so I wasn't billed per run. Treat those as the rough size of the job.
Make it yours
- Write down the three things that bit your team this year. Each one is a rule.
- List the folders where a mistake is expensive: billing, auth, deploy scripts, database migrations. Put them under "needs a human."
- Commit
REVIEW.mdto your main branch and read it withgit show main:REVIEW.md. - Add a hook that checks the output's shape: a count line,
path:lineon every finding, a cap on nits. - Run it on your next three real pull requests and trim the rules that never fire.
Anthropic's team says a good REVIEW.md is short, and that nitpicks cost more when an agent wrote the code, because every nit starts another round. That matches what I saw on a small scale.
One thing I haven't tried here: running this on GitHub so the findings show up as pull request comments. That uses the Claude Code GitHub Action, and I'll cover it in a separate guide once I've tested it end to end.
Sources
- https://claude.dev/blog/solving-code-review-at-anthropic/
Editorial note: We ran the steps in this guide ourselves and the screenshots come from those runs. Your results may differ with your plan, model or settings, so check anything important. Cover illustrations are AI-generated. Discovia is independent and not affiliated with Anthropic.
New guides and tools
Hands-on Claude guides, straight to your inbox
Step-by-step walkthroughs of what's new in Claude and Anthropic's tools, with real screenshots and honest results, including what didn't work. No hype, no jargon, no noise.
By subscribing, you agree to receive newsletter and marketing emails, and accept our Terms of Use and Privacy Policy. You can unsubscribe anytime.
Claude Code Lab
Tests of Claude Code features: hooks, skills, rules files and workflows, with the numbers.
More Claude Code guides →

