Getting AI to review your code and actually catch things

Why generic AI review fails

Paste a file into a chat window and ask "review this code" and you get a wall of output: add type hints, consider extracting a function, you might want error handling here, this variable could be named better. None of it is wrong. All of it is noise. After three of those you stop reading, which means the one time it spots a real off-by-one error, you scroll past it.

Three things fix this, and they're the actual content of this project:

Review the diff, not the file. You aren't asking about code you wrote six months ago. Sending the whole file invites commentary on everything and buries what changed.

Say what you don't want. Explicitly forbidding style opinions, naming suggestions and "consider adding tests" removes about 80% of the output and almost none of the value.

Give it permission to say nothing. If a model is asked to review something, it will find something — that's what it was asked to do. Telling it that "no issues" is a valid and expected answer is what makes silence meaningful, and silence is what makes you read the output when it isn't silent.

That's the transferable lesson here: constrain the ask. It applies to every agent you'll ever build.

Step 1 — The script

Save as review.py anywhere on your path. Works on any language — it only reads diffs.

#!/usr/bin/env python3
"""Review staged git changes. Usage: review [--all] [--focus "something"]"""
import subprocess, sys, os, argparse
import anthropic

MAX_DIFF_CHARS = 60_000

SYSTEM = """You are a careful senior engineer reviewing a colleague's diff before
it is committed. You are looking for problems that would cause real damage:

  - Logic errors: off-by-one, inverted conditions, wrong variable, bad boundary
  - Unhandled cases: None/null, empty collections, failed network calls, division
  - Resource problems: unclosed files or connections, leaks, unbounded growth
  - Concurrency: race conditions, shared mutable state, missing locks
  - Security: injection, secrets in code, missing authorisation, unsafe input
  - Data loss: destructive operations without guards, missing transactions
  - Contract breaks: changed signature or return shape that callers still assume

Do NOT comment on any of the following, ever:
  - Formatting, style, naming, or import order
  - Missing type hints or docstrings
  - Suggestions to extract functions or "consider refactoring"
  - Missing tests
  - General praise, summaries of what the code does, or restating the diff

IMPORTANT: If you find no problems in the categories above, reply with exactly:
NO ISSUES FOUND

That is a normal, expected, and good answer. Do not invent minor concerns to
appear useful. A false alarm costs more than a missed nitpick.

If you do find problems, list each as:

[severity] file:line — the problem
  why it matters, in one or two sentences

where severity is CRITICAL (will break or lose data), HIGH (will misbehave in a
realistic case), or MEDIUM (a real bug but narrow in scope). Nothing below
MEDIUM is worth reporting. Order them worst first. Be specific about the actual
values or conditions that trigger the problem."""


def get_diff(everything):
    args = ["git", "diff", "-U8"]
    if not everything:
        args.append("--staged")
    result = subprocess.run(args, capture_output=True, text=True)
    if result.returncode != 0:
        sys.exit("Not a git repository, or git failed.")
    return result.stdout


def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("--all", action="store_true",
                        help="review unstaged changes too")
    parser.add_argument("--focus", default="",
                        help="extra instruction, e.g. 'watch the auth logic'")
    args = parser.parse_args()

    diff = get_diff(args.all)
    if not diff.strip():
        print("Nothing to review. Stage some changes first (git add), "
              "or use --all.")
        return

    if len(diff) > MAX_DIFF_CHARS:
        print(f"Diff is {len(diff):,} chars — reviewing the first "
              f"{MAX_DIFF_CHARS:,}. Consider committing in smaller pieces.")
        diff = diff[:MAX_DIFF_CHARS]

    question = f"Review this diff.\n\n{args.focus}\n\n```diff\n{diff}\n```"

    client = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])
    reply = client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=2000,
        system=SYSTEM,
        messages=[{"role": "user", "content": question}],
    )

    output = reply.content[0].text.strip()
    if output == "NO ISSUES FOUND":
        print("\033[92m✓ No issues found.\033[0m")
    else:
        print("\033[93m" + output + "\033[0m")


if __name__ == "__main__":
    main()

Step 2 — Make it a command

pip install anthropic
chmod +x review.py
sudo ln -s "$(pwd)/review.py" /usr/local/bin/review

Add your key to your shell profile so it's always there:

echo 'export ANTHROPIC_API_KEY="sk-ant-..."' >> ~/.zshrc

Now, from any repository:

git add -p          # stage what you're about to commit
review              # check it before you do

Step 3 — Prove it works

A review tool you haven't tested is a review tool you'll wrongly trust. Stage a file with a deliberate, realistic bug — an off-by-one in a loop bound, a resource opened and never closed, a condition that's inverted — and confirm it's caught. Then stage a clean, boring change and confirm you get ✓ No issues found.

That second test is the important one. If it can't stay quiet, the prompt needs tightening before you rely on it.

Using it well

Review small diffs. Fifty lines gets a focused, accurate answer. Two thousand lines gets a shallow skim. This is also just good practice — the tool rewards the habit.

Use --focus when you're worried. "I refactored how sessions expire, check that logic carefully" produces a far better review than the default, because you've told it where the risk is.

Treat it as a second pair of eyes, not a gate. It will occasionally flag something that's fine because it can't see the rest of the codebase. Read, judge, move on. Never let it block a commit automatically.

Don't put it in CI. A non-deterministic reviewer failing builds intermittently will be disabled within a fortnight. Its place is before the commit, in your hands.

When it goes wrong

It still nitpicks. Add the specific thing to the forbidden list. Naming the exact behaviour you don't want works much better than a general "be concise".

False alarms about missing context. The model can't see functions your diff calls but doesn't change. Either accept that, or add the relevant file as context for a specific review.

It never finds anything, on anything. Test with a known bug. If it misses an obvious off-by-one, you're probably on too small a model — this task genuinely needs the mid-tier or better.

Empty output. You forgot to git add. Staged is the default deliberately, because the staged set is what you're actually about to commit.

What this costs

A typical diff of a few hundred lines is a cent or two per review with a mid-tier model. Reviewing before every commit through a working week lands in the low single-digit dollars a month — trivially worth it the first time it catches something that would have reached production.

Make it yours

Encode your team's actual rules. This is where the real value is. Add the things your team keeps getting wrong: "all database writes must go through the repository layer", "never log the full request body", "every API handler must validate the tenant ID". A model enforcing your five house rules beats any generic linter.

Per-project rules. Read a .reviewrules file from the repository root and append it to the system prompt, so each codebase carries its own standards.

Pre-push instead of pre-commit. A git hook on pre-push reviews the whole branch — later, but catches things that emerged across several commits.

Ask it to explain, not just flag. A --explain flag that asks for the reasoning behind each finding turns the tool into a way to learn, which is worth more than the bug-catching in the long run.