Acceptance testing: how to check AI-written code before going live
The answer first: AI code review is not about grinding through code line by line, but walking through a 4-step checklist — look at the diff, run checks, go through the security red lines, and explain it clearly. Only after these four steps does "looks fine" become "checked and verified".
Why must this step exist? Look back at the practice described in Karpathy's widely shared tweet: "I 'Accept All' always, I don't read the diffs anymore" — I always accept everything and no longer look at the diffs. He also admitted: "The code grows beyond my usual comprehension" — the code has grown beyond his usual comprehension. Engineer Simon Willison's definition of vibe coding is just as blunt: "building software with an LLM without reviewing the code it writes" — building software with an LLM but not reviewing the code it writes.
So "review or not" is the dividing line between vibe coding and serious software development: for a weekend toy built for fun, you can skip the diff; for anything going live, review is mandatory. This lesson is the fourth discipline set by the workflow guide: "acceptance testing before launch" — run pass/fail checks and be able to explain what the code does.
Works with all four tools (Claude Code, Codex, OpenCode, WorkBuddy), completable in 10 minutes from zero.
What you'll be able to do after this lesson
- Use a 4-step acceptance checklist to review AI-delivered code: look at the diff, run checks and demand evidence, go through the security red lines, and explain it clearly — turning "looks fine" into "checked and verified"
- Be able to say what AI code review focuses on: the logic errors, configuration errors, and security vulnerability hotspots identified by quantitative research, checked off one by one
- Use the built-in review features of the tools to have "an AI that didn't write this code" vet it for you:
/code-reviewand/security-reviewin Claude Code, dedicated review in Codex, and the review-diff command in OpenCode - Know when the original "skip the diff" approach is fine and when acceptance testing is mandatory — for anything meant for others, find an experienced person to vet it before release
Your current predicament
- The AI says "done, all features running normally," and when you open it the page does run — but it quietly changed three files you never asked it to touch, and you have no idea
- You've heard that AI-written code has 2.74 times more security vulnerabilities than human-written code, and you want to check but don't know where to start: you can't read the code, so what exactly does AI code review check, and in what order?
- Your project is about to be used by others (friends, clients, app stores) and you feel uneasy: if there's a hidden hole, will the first to find it be a user or an attacker?
When to use this technique
When you need to:
- Upgrade "the AI says it's done" to "checked and verified, ready to launch"
- Keep risks in your own hands before releasing something meant for others
- An AI code-checking checklist executable even without understanding code
And you don't want to:
- Grind through every line of AI-generated code line by line (this lesson doesn't require you to fully understand it)
- Rely on user error reports after launch to discover vulnerabilities
- Carry the weekend-toy "Accept All" approach straight over to a serious product
🎒 Preparation before starting
- Read Error loop: feed the error message back verbatim — problems found during acceptance ultimately feed back into that lesson's fix loop
- Have installed and used at least one tool: Claude Code, Codex, OpenCode, or WorkBuddy
- Have a small "working" project just delivered by AI on hand — practicing on a real deliverable sticks better than reading examples
- 10 minutes and a glass of water ☕
Core idea
The essence of acceptance testing: turning "looks done" into "checks prove it's done." Claude Code official best practices put it plainly: without runnable checks, the AI's only basis for finishing is "looks done" — in effect, you become its test suite, and every problem only surfaces the moment you run it yourself.
So why can't you rely on just "run it and see"? Because quantitative research has put AI code quality issues on the table:
| Research | What it found | What it means for acceptance |
|---|---|---|
| CodeRabbit (2025-12, analyzing 470 open-source PRs on GitHub) | AI-involved code has about 1.7 times more "major" issues than human-written code; 75% more configuration errors; security vulnerabilities are 2.74 times higher | You can't just check whether features run; spot-check by hotspot |
| Veracode (2025-10, three-year tracking) | Models have significantly improved at generating runnable code, but the security of generated code has not improved overall | "The model was updated, so it should be safer" doesn't hold; every batch of code must be checked |
| The Lovable incident (2025-05) | Of 1,645 web apps created by the AI application platform Lovable, 170 had the security flaw that "anyone can access user profiles" | Launch without acceptance testing, and holes appear in batches |
The full story behind these numbers is discussed in detail in Vibe Coding: Is it reliable: risks, controversies, and real cases. This lesson takes just one thing from it: AI code quality issues are not random; they have hotspots — logic errors, configuration errors, and security vulnerabilities. The acceptance checklist is designed around these hotspots.
A preview of the 4-step checklist, each step blocking one kind of pitfall:
| Step | Action | What it blocks |
|---|---|---|
| 1. Review the diff | Go over every AI change one by one | It changed things you never asked it to change |
| 2. Run the checks | Pass/fail checks + commands and output as evidence | The fake completion of "looks done" |
| 3. Clear the red lines | Check secrets, permissions, configuration, and logic item by item | High-frequency zones for security holes and configuration errors |
| 4. Be able to explain it | You can explain to others what the code does, and get someone to vet it when needed | Code you don't even understand yourself goes live |
One sentence to remember this lesson: Review the diff, run the checks, clear the red lines, and only ship once you can explain it.
Follow Along
(Use the project your AI just delivered as your material and work through all 4 steps: from seeing clearly what it changed, to having the confidence to ship. Paste any problems you find back into the error loop to fix.)
Step 1: Review the diff — AI code review starts with "what did it change"
Why: What the AI produces not matching what you had in mind is the number-one common pitfall listed in the official best practices — it cannot read requirements from your brain, it can only guess. And "Accept All without reviewing the diff" throws away every chance to catch a wrong guess. So the first step of AI code review is not reading the code, it's looking at the list of changes: first find out which files it touched, then talk about checking.
Where to view changes in each tool:
| Tool | How to view the diff |
|---|---|
| Claude Code | Run /diff in the session to view workspace changes, including all edits the AI has made so far |
| Codex | Inspect commands and diffs as changes appear (official positioning: Steer the active turn, inspect commands and diffs as they appear) |
| OpenCode | Run git diff --stat in the terminal to see the list of changed files, and git diff to see the actual content |
| WorkBuddy | Deliverables come back as "inspectable results"; code development scenarios include review by design — go through what it did item by item in the task conversation |
If the project is in a Git repository, git status and git diff work across all four tools — this is the most fundamental fallback entry point.
While reviewing, ask only three questions:
- Are all the changed files ones I asked it to change?
- Are there any extra files, any extra dependencies?
- Among the deleted lines, is there anything that doesn't look like "refactoring"?
For changes of unknown origin, ask it directly (the first magic template on this page):
Which files did you change in this delivery? Explain each change one by one — what it does and
which of my requirements it corresponds to. Did you change anything beyond what I explicitly asked for?💡 Have the AI flag risks itself
There's a ready-made practice in the official workflow: before submitting, have the AI highlight potential risks and areas to watch (Review Claude's generated PR before submitting and ask Claude to highlight potential risks or considerations). Its self-assessment can't replace your review, but it can point you in the right direction — the places it flags are the places you focus on.
What you should see: a list of changes + the AI's explanation for each change. If a file you never asked it to touch shows up, its explanation must convince you; for changes it can't explain, either have it revert them or mark them for investigation.
- Small check: I've reviewed the list of changed files and there are no changes of unknown origin; if there are, the AI's explanation convinces me
Step 2: Run pass/fail checks — demand evidence, not "it's done"
Why: "The page opens" does not mean "it's done." The error loop lesson applied pass/fail checks to bug fixing; at acceptance time they're the final gate: the deliverable must have actually passed the checks, and you must see the evidence with your own eyes. In the official words: "'All tests pass' on its own doesn't tell you anything" — what's needed is the commands it ran and the full output.
A check can be any one of these (official list — use whatever your project has):
| Check | Example |
|---|---|
| Tests | The project's test command, all passing |
| Build | The build/compile command's exit code is 0 |
| Lint | The code linting tool reports no errors |
| Comparison script | Output matches the expected file line by line |
| Page screenshot | How it actually looks running in the browser, compared against what you wanted |
Copy the acceptance prompt directly (the second magic template on this page):
Delivery acceptance: please paste the check commands you ran and their full output,
including: test commands and build commands.
If any item didn't pass, explain the reason and your fix plan — don't just say "completed."If you're using Claude Code, there's one extra killer move: after it says it's done, run /verify — this official skill has it build and run your application and look at the running result itself, catching problems that tests and type checks miss (things like a blank page or an unclickable button — the "all tests green but the thing doesn't work" trap).
What you should see: the AI pastes back the commands it ran and the full output; you pick one of them and re-run it yourself, with matching results. If it doesn't match, it either didn't run or ran something else — send it back to the error loop.
- Small check: "it's done" is followed by commands and output, and I've re-run at least one of them myself
Step 3: Go through the security red lines — check AI code quality hot spots item by item
Why: CodeRabbit's research gives the problem distribution for AI-assisted code — logic errors (wrong dependencies, control-flow defects) occur at a higher rate, configuration errors are 75% more common, and security vulnerabilities are 2.74x more frequent. AI code review can't spread its effort evenly; it must investigate the hot spots item by item. Four red lines, each with a clear way to check:
| Red line | What to check | Why it's a hot spot |
|---|---|---|
| 1. Secrets | Whether API keys, passwords, or tokens were hardcoded into the code or committed to the repository | Willison's security checklist, item 2: whenever code involves secrets, you must understand it, otherwise leaks happen easily |
| 2. Permissions | Whether the "who can access what" gates are still in place: login, authorization, data visibility | The Lovable incident: 170/1,645 apps let anyone browse user profiles — permission holes appear in batches |
| 3. Configuration | Whether configuration values are correct, and whether production/test environments got mixed up | CodeRabbit: configuration errors are 75% more common in AI-assisted code |
| 4. Logic | Whether the control flow of core processes is correct: conditions, boundaries, exception handling | CodeRabbit: logic errors (wrong dependencies, control-flow defects) occur at a higher rate |
You can check the first two red lines even without knowing code: for the secrets red line, do a project-wide search for words like key, secret, token, password; if you get a hit, have the AI explain what it is and whether it will be exposed along with the code. For the permissions red line, access the app once using "someone else's account" or while logged out; if you see something you shouldn't, there's a hole. The last two lines are covered by tool-based review:
| Tool | Review command |
|---|---|
| Claude Code | /code-review checks the current diff for correctness bugs (alias /review; add --fix to apply fixes directly); /security-review specializes in security vulnerabilities—injection, authentication issues, data exposure |
| Codex | Dedicated review: run a review on uncommitted changes, a specific commit, or against a base branch; reports issues ranked by priority and doesn't touch your working tree |
| OpenCode | Official example command review-diff: automatically feeds the output of git diff --stat && git diff into the review prompt (see the advanced tips below) |
⚠️ Tool review reports are not gospel
The officials have specifically warned: a reviewer asked to find problems will always find something. If a review report lists a dozen or more items, don't fix them all wholesale—the cost of over-fixing is a pile of useless abstraction layers and tests for scenarios that will never happen. Prioritize by "affects correctness and security": red-line items must be handled, style items can be let go.
You should see: each of the four red lines has an answer item by item; the tool review has run, with no hard flaws such as leaked secrets, missing permissions, or high-risk vulnerabilities. Paste the issues found back into the error loop to fix them—exact error text, check commands, demand evidence; just reuse the whole playbook from that lesson.
- Small check: I've gone through all four red lines—secrets, permissions, configuration, logic—and pasted the issues found back into the error loop
Step 4: You can't call it done unless you can explain it—get someone to vet anything others will use
Why: Willison's golden rule: "My golden rule for production-quality AI-assisted programming is that I won't commit any code to my repository if I couldn't explain exactly what it does to somebody else"—if you can't accurately explain to someone else what a piece of code does, don't commit it to the repository. He also gave a dividing criterion: if the code was written by an LLM, but you have reviewed it, thoroughly tested it, and can explain to someone else how it works—then it's not vibe coding, it's proper software development.
The operation has two layers; choose based on where the project is headed:
Layer one: explain it to yourself (all projects). Feed the diff or key files back to the AI and have it explain them to you (the third magic template on this page):
Explain the core logic of this change to me in plain language: what does this project do
from start to finish? Where does the data come from, and where does it go? Which changes are the most critical, and why?
Just assume I know nothing about code.Wherever you don't understand is a risk point—either keep asking until you do, or flag it for focused checking. This step only counts as passed when you can explain the project clearly in your own words.
Layer two: independent vetting (projects for others to use). Item 6 of Willison's security checklist, verbatim: "If you're going to vibe code anything that might be used by other people, I recommend checking in with someone more experienced for a vibe check"—if you're going to vibe code anything that might be used by other people, get someone more experienced to take a look. If you can't find a person, at least use "an AI that didn't help write this code" as an independent reviewer:
- Claude Code's
/code-reviewworks on exactly this principle—it opens a brand-new subagent to review the diff, one that has never seen your conversation and won't be biased by the reasoning from when the code was written - For tools without this command, open a second session to do the same thing: one session writes code, the other reviews the files, and findings get pasted back into the first session to fix
You should see: you can explain clearly "what this project does and how" in two or three plain sentences; for projects others will use, an experienced person or an independent reviewer has gone through one round.
- Small check: I can explain to a friend what this code does; for projects others will use, I've arranged a person or an independent review to vet it
Checkpoint ✅
- I've looked at the AI's change list (diff); there are no changes of unknown origin
- "It's done" has evidence: check command + full output, and I've re-run at least one myself
- All four security red lines—secrets, permissions, configuration, logic—have been checked one by one
- I can explain in my own words what the code does, and I've dealt with the parts I couldn't explain
- For things others will use, I've had an experienced person look at them, or passed an independent review (
/code-review, second session)
Pitfall warnings
❌ "The page opens, acceptance passed"
A page opening only proves the main path isn't broken; it proves nothing else. Verbatim from the official best practices: when there are no runnable checks, the AI's basis for calling it done is "it looks finished"—you become its test suite, and every problem lies dormant until you personally run it and it blows up, while attackers won't stick to the main path. Correct approach: run the pass/fail checks from Step 2, demanding commands and output as evidence; use Claude Code to add another gate with /verify, letting it run the app itself and see the results.
❌ Treating "the AI says all tests passed" as evidence
"'All tests pass' on its own says nothing"—that's a verbatim quote from the officials. Check results relayed by the AI may have been run, may be guesses, or may have been run against an old version of the code. Correct approach: demand it paste the commands it ran and the full output, then you pick one and re-run it to compare results. If the evidence doesn't match, send it back into the error loop.
❌ The review report listed 15 items, and I stayed up all night fixing them all
A reviewer asked to find problems will always find something—the officials specifically warned about this one. The cost of fixing everything is a pile of useless abstraction layers and tests for scenarios that will never happen, making the code harder to explain. Correct approach: prioritize by "affects correctness and security"—anything involving secrets, permissions, configuration, or logic must be fixed; let style items go; when fixing, fix only the hard flaws one at a time, and re-run the checks after each fix.
❌ Taking the weekend-toy approach straight into a product others will use
Karpathy himself has drawn the line: this "Accept All without looking at the diff" approach works fine for a throwaway weekend project. Applying it wholesale to something others will use means handing users a safety-vulnerability factor that is 2.74 times higher — that's exactly how 170 "anyone can access user data" apps came to exist on the Lovable platform. The right approach: set acceptance rigor based on "who it's for": for personal toys, get it running plus a quick scan for secrets; for things others will use, go through all four steps and then have an experienced person review it.
Advanced tips (optional)
Write acceptance criteria into the project manual. The official best practices say: if the project already has checks, the most useful line in CLAUDE.md is the command to run them. Write "what checks must be run before delivery, and how to submit evidence" into AGENTS.md / CLAUDE.md, and every new session automatically carries the acceptance criteria—see AGENTS.md and CLAUDE.md: leave the AI a project manual for how to write it.
OpenCode users' review-diff command. The official docs give a ready-made custom command example: create .opencode/commands/review-diff.md in the project with two lines of content—Review this diff: plus !`git diff --stat && git diff`. After that, every time you run /review-diff, it automatically stuffs the changes into the review prompt.
Make checks intercept automatically. Claude Code supports Stop hooks: when Claude wants to wrap up, it first runs your check script (e.g., a type check), and if it fails, it's sent back to keep fixing—acceptance goes from "a human watching" to "a mechanism intercepting." An advanced technique; if you're interested, see the Enforce the check section of the official best-practices document.
Readers going the mini-program or plugin store route: platform review is the final external acceptance gate—WeChat mini-programs have an official self-check checklist before submission (name compliance, core functionality, content safety); see Submission and release: a guide to avoiding pitfalls in WeChat's review rules; for Chrome Web Store review and staged rollout, see Store review and release strategy.
Lesson summary
✅ You've completed the full chain of AI code review: read the diff (go through the change list item by item) → run checks (pass/fail + command output as evidence) → check red lines (secrets, permissions, configuration, logic—the four high-frequency problem areas) → be able to explain it (you can explain it to someone else, and for things others will use, get someone to vet it).
✅ You now know why acceptance review can't be skipped: CodeRabbit's quantitative research says code with AI involvement has 2.74 times more security vulnerabilities than human-written code and 75% more configuration errors; Veracode's three-year tracking says model capability keeps rising while security doesn't. Problems cluster in high-frequency areas, and the checklist is designed around those areas.
✅ You've drawn the usage boundary between the two approaches: Karpathy's original version, "don't look at the diff," is for weekend toys; for anything going live, review is the dividing line between vibe coding and software development—Willison puts it bluntly: if it's been reviewed, tested, and you can explain it, it's not vibe coding, it's software development.
Key concepts recap:
| Concept | In one sentence | Where to go deeper |
|---|---|---|
| Read the diff | The first step of acceptance review: first know what the AI changed | Step 1 on this page |
| Demand evidence | Commands + output, not just the words "it's done" | Step 2 on this page |
| Security red lines | Secrets, permissions, configuration, logic—the high-frequency problem areas in AI code | Step 3 on this page |
| Be able to explain it | If you can't explain it, don't commit—the golden rule | Step 4 on this page |
| Vetting | For things others will use, have an experienced person or an independent review take a look | Step 4 on this page |
Next lesson preview
With the acceptance checklist done, the code is safe to ship—but wait, this lesson's red lines only checked "whether secrets were written into the code," and haven't yet addressed "how to safeguard the secrets themselves." Next lesson: Cost and Safety: Three Lines of Defense—Secrets, Bills, and Sandboxes.
You will learn:
- How to safeguard API keys: this lesson covers leaks, the next gives you a complete custody plan
- How to set billing limits: how to prevent the horror stories of APIs without billing caps running up thousands of dollars in bills through vibe coding
- How to use sandboxes: draw an activity boundary for the AI—safety for beginners starts with sandboxes
- How to vet data privacy: confirm whether your data ever leaves your machine