Reviewing AI-written code

AI-written code can look correct and still be wrong. Four short snippets show where to look.

An AI chatbot returns a short function that shows a student's courses, with tidy lines and sensible names. You paste it into your project and the page works on the first run. One line in it is faulty, and nothing on the screen points to it.

The code the AI wroteLooks correct
One faulty lineHidden defect
YouReviewer
Think first. The code ran and looks right. What would you check before relying on it, and what would you need to know about the system to check it?
Students usually say they would run it again or read it more slowly. Point out that the checks need facts the code does not show: how many rows the tables hold and what else runs at the same moment.

Each snippet is a short Python function from a web application, and it looks correct. The heading on the code says what the function was asked to do. Click the one line you think causes the problem.

More detail

Your score

Found
0 of 4
First guess
—

Jump to a snippet

The full explanation

Where these problems come from

The first three appear in The N+1 query problem, SQL basics and Transactions. The fourth is a method that does not exist. All four have been shipped in real systems.

Generated code almost always looks like correct code. Check what the model could not have known: what is in your tables, what runs twice, and what happens when two people click at the same moment. The invented method fails at once and is easy to catch. The other three run quietly and give wrong results.

More detail

Why reading is not enough

Reading the code does not reveal an N+1 problem, because it only shows up with a number of rows the model never saw. A race condition is only correct when nobody else is using the code. You cannot see either one by reading.

Scale of the task

With four short snippets, you knew each held exactly one defect. An agent will hand you three hundred lines and tell you the tests pass.

The two terms are often used as if they meant the same thing. Compare how each one works.

Vibe coding

You describe what you want and run what comes back. You judge it by whether it seems to work.

Agentic coding

An agent runs commands and edits your files in a loop. It acts as well as suggests.

You can now tell vibe coding from agentic coding and say which needs more review.

More detail

Where vibe coding comes from

The term comes from Andrej Karpathy, who described it as a fine way to build throwaway projects, where being wrong costs you a deleted folder.

What an agent has

An agent has a shell (a window for typing commands), your repository (the folder holding your project's code) and your test suite (a set of automated checks that report pass or fail). It works in the loop described in AI agents. It can run things, read failures and revise. It can do more, and it can do more damage without being asked.

When a prototype ships

Vibe coding is a reasonable way to find out whether an idea is worth building. It stops being reasonable when somebody depends on the result, and that moment usually arrives without an announcement. The prototype worked, the demo went well, and nobody stopped to read it.

What to write down in the next session

Watch forWrite down
One confident errorWhat it claimed, and how you found out it was wrong
A moment it changed course after an Observation (a result sent back by a tool)What contradicted it, and how many steps it had already built on the wrong answer
Anything it did that nobody asked forExtra files, extra layers of code, a library that was not needed

Bring those notes to the next session. They come from your own work.