What this lesson is about
In this lesson we will look at what happens when a model “thinks” before it answers. Many AI assistants today show a label such as “Thinking…” before they reply. Some also show the thinking itself: the model works through the problem, tries different approaches and corrects itself. The settings often let you choose how thoroughly the model should think. Models that can think this way are called reasoning models.
As we saw in Lesson 1, the model can predict the next token. So it can “think” only in the same way it does everything else. Understanding this clearly helps us decide when to turn reasoning on and what to expect from it. It also tells us how far to trust what the model writes about its own thinking.
First, we will look at what a model’s reasoning is and where this ability comes from. Then we will see why reasoning helps with some tasks but adds almost nothing to others. We will also see what it costs. After that, we will find out why smooth, convincing reasoning is not proof that the answer is right. At the end of the lesson, we will talk about how to use reasoning in your own work.
What reasoning is
Let’s start with the main point: the model’s reasoning is text. Before the model writes its answer, it writes a draft. It works through the task, writes down intermediate results and checks them. Sometimes it drops an approach that is not working and tries another. It writes the draft the way it writes everything else, token by token, and only then moves on to the answer.
Chat apps usually collapse the draft or show a short summary of it. For the model, though, it is still part of the text it continues.
To see why the draft helps, recall Lesson 1. The model chooses each next token based on all the text before it. If the model writes the answer right away, it gets only one step of computation for each token of that answer. That step always involves the same amount of computation, however hard the task is. Each token of the draft adds one more such step.
To write the first digit of the result of a hard task, the model must, in a sense, already know the result. It has to do all the intermediate calculations “in its head”, inside that single step. When a draft comes before the answer, the intermediate conclusions are written down in the text. Each next step can then build on them. It is like multiplying large numbers in your head versus with long multiplication on paper. Paper does not make a person smarter, but it saves them from holding everything in their head.
Models are trained specifically to write useful drafts. They are given many tasks whose answers can be checked, such as math or programming problems. Drafts that lead to the correct result are rewarded. Over time, the model learns to reason in useful ways: to break a task into parts, check itself and notice mistakes.
Did you know? where “reason step by step” came from
Before reasoning models appeared, people got a similar effect by hand. They added “think step by step” to the request, and accuracy on calculation and logic tasks really did improve. This technique is called “chain of thought”. Reasoning models have, in effect, learned to do it on their own, without being reminded.
Keep in mind that reasoning is made of the same tokens as the answer. We tested this on two current models. We asked each to multiply 83749261 by 57381924 and write only the number. Both answered correctly, and the answer itself took only a few tokens. Before it, though, the models wrote drafts of 153 to 698 tokens.
One model kept the text of its draft hidden. The only sign of it was the token count in the usage report. When you work through the API, the model’s developer sends this report along with the answer. The second model returned the text of its reasoning as well. In a regular chat app, users usually do not see such a report.
As we said in Lesson 2, when you work through the API, you pay for reasoning tokens just as you pay for answer tokens. Oddly enough, these models do not let you turn reasoning off at all. When we tried, the program we used to access them silently set the lowest level instead. The level changed only the length of the draft. For one of the models, the draft was noticeably shorter at the low level.
When reasoning helps
The way reasoning works shows where it helps most: in tasks that naturally break down into steps. Examples are calculations and logic problems, planning under several constraints, and writing and debugging code. Others are finding contradictions in a document and comparing several options against a set of criteria. In such tasks, the answer is built from intermediate conclusions. If the model writes them down in the draft, each step becomes simpler. A mistake in one of them is also easier to notice and fix.
That said, what counts as a “difficult” task keeps changing. Until recently, language models found it hard to multiply two eight-digit numbers without a single mistake. In our tests, both models managed it even at the lowest reasoning level. The difference between short and long reasoning shows up in tasks that truly have many steps, constraints or options to work through.
We also checked whether reasoning helps when the model lacks the knowledge it needs. In Lesson 4, one of the models confidently invented the second sentence of Frank R. Stockton’s little-known story “The Transferred Ghost.” We repeated the test, asking the model the same question three times with short reasoning and three times with long reasoning.
With short reasoning, the model quoted both sentences correctly twice and invented the second sentence once. With long reasoning, it was right once. Once it invented the second sentence, with the words “somewhat whimsical, hospitality” and more invented text. Once it got one word wrong, writing “host” instead of “hospitality.” It never declined to answer. Three attempts are too few for firm conclusions, but in this test, long reasoning did not make the model more careful.
Let’s look at what actually happened here. Reasoning gave the model no new knowledge. It may help the model recall more accurately what it learned in training, but it cannot create information the model does not have. The solution is the same as in Lesson 4: give the model the text it needs in the request.
What reasoning costs
Reasoning has a price, and it comes in two parts. The first is time. While the model writes its draft, the user waits. On a hard task, the wait can stretch from a few seconds to several minutes.
| Part | Tokens |
|---|---|
| The answer: two sentences | 51–77 |
| The draft before it | 415–837 |
Kimi K3.1 Pro, three runs. The numbers come from the usage report the API sends back.
The second part is tokens. In the quote test, the model spent 415 to 837 tokens on long reasoning to write just two sentences. One of those answers was still invented. When you work through the API, you pay for every one of these tokens. With a chat subscription, long reasoning can count against your usage limits.
So there is no point in turning on maximum reasoning for every task. To rephrase a paragraph, translate a letter or answer a simple question, you usually do not need a long draft. The model will handle these tasks without one, and you will only wait longer and pay more. Many modern models decide for themselves how much to reason, depending on how hard the task is. In the settings, you can also choose how thoroughly the model thinks. It makes sense to raise this level for tasks that truly take many steps and to keep it low for simple ones.
Why convincing reasoning is not proof
One question remains: how far we can trust the reasoning itself. The model may show a neat line of thought, with each step following from the one before. Then it is tempting to think the answer has been checked. But there are three reasons to be careful.
The first reason follows from what we already know. Reasoning is text that the model writes in the same way as its answer, so it can make the same kinds of mistakes in it. A wrong intermediate conclusion, stated confidently, leads to a wrong answer, and smooth wording does not give it away. As with an ordinary answer, the model does not erase what it has already written in the draft. So a mistake early in the draft can shape everything that follows. The model may notice the mistake later and correct it, but it may also miss it.
The second reason is practical. In a chat, the user often sees only a shortened summary of the draft. Many developers do not show the full draft even through the API. A summary helps you see which direction the model took, but it need not include every step.
The third reason is that even the full draft does not always honestly reflect why the model reached its answer. Researchers test this in a simple way. They slip a hint with the answer into a test question. For example, they mention in passing that a well-known professor thinks a certain option is correct. Or they hide the answer in a technical note attached to the question. Then they check whether the model admits that it used the hint.
A 2026 study tested twelve reasoning models that their developers had released openly. The researchers picked out the cases where the hint changed the model’s answer. In these cases, the models mentioned the hint in their draft almost 88% of the time. In the answer itself, they mentioned it only about a third of the time. In almost 12% of cases, they did not mention the hint anywhere. The reasoning looked like the model’s own work, but the answer had in fact come from the hint.
Did you know? how such tests began
Researchers have run similar tests since 2023. In 2025, Anthropic ran one as well. In that test, Claude 3.7 Sonnet acknowledged the hint in its reasoning in about a quarter of cases. DeepSeek R1 did so in about two fifths. The 2026 study described above repeated this test on twelve open models.
So the model’s reasoning is useful as a pointer to where a mistake might be, but not as proof that the answer is right. Check the answer itself: recalculate the key figures, compare the facts with the source, and run the code. We will look at how to do this in detail in the next lesson.
How to use reasoning at work
Let’s turn what we have covered into a few practical rules. Some tasks have many steps, such as data analysis, a project plan, a search for contradictions in a contract, or complex code. For these, it is worth choosing a model with reasoning or a more thorough reasoning level in the settings. For short, simple tasks, let the model decide how much to reason, or choose a low level. That way you save time and do not use up your limits.
Keep in mind, too, that the answer in a chat does not always come from the model the user chose. ChatGPT, for example, may continue the conversation on another model when the usage limit for reasoning runs out. If safety filters flag a request, Claude answers it with a less capable model. It warns the user about this and labels the answer with that model’s name. The model switcher then stays on that model until the end of the conversation.
So if the answers suddenly get noticeably weaker, check which model gave them. If needed, choose the right model again.
Another good habit is to ask the model to show its key steps and assumptions in the answer. That means what data it used, what it calculated, and what its conclusion rests on. Such an answer is easier to check than a bare result, and a mistake in it is easier to find. If the task needs information the model may not have, include it in the request. As we saw, reasoning cannot fill that gap.
What follows from this
Let’s sum up. The model’s reasoning is a draft that it writes token by token before it answers. Its intermediate conclusions are written down in the text, and each new step builds on them. This is why a model with reasoning makes fewer mistakes in tasks with many steps. Three practical points follow about when and how to use reasoning.
First, reasoning helps with tasks that take many steps and adds almost nothing to simple ones. It is worth choosing the reasoning level to fit the task, because the draft costs us time and tokens.
Second, reasoning does not add knowledge. It may help the model recall what it learned, but it cannot create information the model lacks. If the information is missing, we need to give it in the request.
Third, convincing reasoning does not prove that the answer is right. Reasoning can contain mistakes, and the chat app often shows only a summary of it. Even the full draft does not always reflect the real reasons behind the answer. What we need to check is the result.
In the next lesson, we will look at how to check the model’s answers: which mistakes happen most often and how to catch them.