AI agents: the think, act and observe loop

An agent is a language model connected to tools. It thinks, acts and checks the result, then repeats.

An AI coding agent that you asked to fix a bug opens a file, edits a line, runs the tests and reads the result on screen. Two tests fail, so it edits another line and runs the tests again. A chat model only produces text. An agent is a model that can also act and see what happened.

Decides what to doThought
Edits a file or runs a commandAction
Reads the test resultObservation
Think first. Of thinking, acting and looking at the result, which could a model that only produces text never do by itself, and why?
Students usually say the model cannot act. Push for the second half: it also cannot know what happened without a result from outside itself.

The agent is asked to fix a bug: some students are enlisted in the same course twice. Press Step to add one line at a time. Thought and Action lines come from the model. Observation lines come from a tool, such as a test runner. The task is done at step 15, so watch what happens next.

More detail

What the loop is costing

Iteration
—
Context in use
—
Spent so far
—

Context in use is how much of the model's context window the conversation has filled. The context grows with every step, and the cost per step grows faster than the step count.

Tools an agent typically has

A shell is a window where you type commands. A file system is the folders and files on a computer. A test runner checks whether code still works. Observation lines can also come from an API, which is a way for one program to request something from another.

The name of the pattern

This pattern is called ReAct, short for reason and act. Observation lines are the only lines that can contradict the model.

See The context window for how the window fills.

Four things go wrong in long loops. The model may not stop when the task is done. The context window fills up and pushes out your first instruction. A wrong observation is treated as fact in every later step. Each step costs more than the one before.

You can now name four ways an agent goes wrong and what a person should watch.

More detail

It does not know when to stop

The model stops when it predicts a special end-of-output token. That token is one more prediction in the sequence, so it can come too early or not at all.

The context window fills

Every thought, action and observation is added to it. After twenty steps your original instruction may no longer be in what the model can see.

Errors compound

A wrong observation at step three is treated as fact in every step after it. The model's confidence does not fade on its own.

Cost grows with each step

You pay per step, the context grows each time, and attention cost rises with the square of the context length. The twentieth step costs more than the first for the same work.

The task in the trace ends at step 15. At which step would you have stepped in?