Yes, chain-of-thought prompting still works, but mostly on standard AI models and mostly for multi-step math and logic. On reasoning models, which reason internally before they reply, OpenAI says a "think step by step" line may not help, and a Wharton study found the gains were "marginal at best" while answers took 20-80% longer.
So the useful question is no longer whether to type the magic phrase, but whether your model is already thinking. If thinking is on, change the setting, not the prompt. If thinking is off, add a step-by-step line only when the task has steps you would work out on paper, like sums, formulas or logic puzzles.
What is chain-of-thought prompting?
Chain-of-thought prompting means getting an AI model to write out "a series of intermediate reasoning steps" before it gives its answer, a bit like a teacher asking you to show your working. Two versions are worth knowing.
- Few-shot chain-of-thought. The original method, from Wei et al., put a few worked examples into the prompt, reasoning included. They prompted a 540B-parameter model (parameters are the internal values a model learns, so 540B means 540 billion of them) with just eight examples.
- Zero-shot chain-of-thought. Kojima et al., in a paper submitted on 24 May 2022, showed that simply adding "Let's think step by step" before each answer turned large models into decent reasoners. Zero-shot just means no examples.
Picture a new assistant totalling three supplier quotes. Asking them to list each line before giving the total catches slips, but it also takes longer. That trade-off between accuracy and waiting time runs through everything below.
Does chain-of-thought prompting still work, according to the research?
As of November 2026, the key studies show chain-of-thought prompting gave big gains on older models, helps mainly on math and logic, adds small average gains on newer standard models, and brings marginal gains at best on reasoning models. A standard model answers straight away; a reasoning model, such as OpenAI's o3 or o4-mini, reasons internally before it replies.
The Wharton Generative AI Labs report compared five standard models (Claude 3.5 Sonnet, GPT-4o, GPT-4o-mini, Gemini 2.0 Flash and Gemini Pro 1.5) with three reasoning models (o3-mini, o4-mini and Gemini 2.5 Flash). The table puts it next to the other key studies. Names like MultiArith, GSM8K, MMLU and GPQA Diamond are standard test sets, so results can be compared.
| Study | What was tested | What the step-by-step prompt did | What it means for you |
|---|---|---|---|
| Kojima et al. | InstructGPT (text-davinci-002) and the 540B-parameter PaLM | With text-davinci-002, MultiArith accuracy rose from 17.7% to 78.7% and GSM8K from 10.4% to 40.7%. PaLM showed similar-sized gains. | One line was a big deal on older models. |
| Wei et al. | Eight worked examples, 540B-parameter model | Reasoning emerged in large enough models. | Worked examples can teach a method. |
| Sprague et al. | Over 100 papers, plus 20 datasets across 14 models | Strong gains mainly on math or logic. On MMLU, answering directly scored almost the same. | Save it for real calculation. |
| Wharton | 198 PhD-level GPQA Diamond questions, 25 trials each | Standard models: small gain, more varied answers, 35-600% (5-15 seconds) slower. Reasoning models: marginal gains at best, 20-80% (10-20 seconds) slower. | On reasoning models, it mostly buys waiting time. |
| Liu et al. | Spotting hidden statistical patterns, recognising faces, learning labels full of exceptions | Up to 36.3% absolute accuracy lost for OpenAI o1-preview versus GPT-4o. Drops across eight other large language models and all six vision-language models (models that also read images). | Skip it for gut-feel pattern tasks. |
For reasoning models, Wharton calls the extra 20-80% of time "a substantial cost for what are often negligible gains in accuracy." Three more details matter for everyday use.
Step-by-step answers can wobble. Wharton found chain-of-thought can add variability, and in three of the five standard models it made errors on easy questions more likely. Think of the colleague who talks themselves out of the right answer to a simple question.
The opposite extreme hurts too. Wharton warns that demanding a direct answer with no explanation can harm a standard model by blocking its natural reasoning. So answer in one word only is not a safe default either.
For exact numbers, a tool beats a monologue. Sprague et al. found chain-of-thought rarely outperforms tool-augmented approaches, such as a symbolic solver (software that does exact algebra). If your chatbot can run code or a spreadsheet formula, let it do the invoice totals.
One honest caveat: none of these studies tested the newest models, and Wharton used a single benchmark. Treat the numbers as a strong direction, not a law of nature.
Should you tell a reasoning model to think step by step?
Usually not. As of November 2026, OpenAI's reasoning guide says prompting its reasoning models to "think step by step" or "explain your reasoning" may not improve results, because they already reason internally. Anthropic advises general instructions over prescriptive steps when Claude is thinking, and Google's Gemini docs offer thinking levels that set how much Gemini thinks.
| Vendor | Thinking on | Thinking off | Where you set it | For hard tasks |
|---|---|---|---|---|
| OpenAI | Simple, direct prompts. No step-by-step lines. Try no examples first. | The step-by-step warning is for reasoning models. GPT models are faster and tend to cost less. | Thinking slider in ChatGPT, or Instant | Pick an o-series model when accuracy matters most |
| Anthropic | General goals over step lists. Worked examples still help. | Manual chain-of-thought is the fallback. | Effort level plus Thinking toggle | More effort, thinking on, or both |
| Pick low, medium (default) or high on Gemini 4 Argon. | Not possible on Gemini 4 Argon. | Thinking level or budget in the Gemini API | A high thinking budget for complex math and coding |
What OpenAI's reasoning guide says
OpenAI's reasoning best practices guide splits its models into reasoning models (o3 and o4-mini, for example) and GPT models (like GPT-4.1). GPT models are faster and tend to cost less, it says, while o-series models are reliable decision makers when accuracy and reliability matter most.
Its core advice for reasoning models is to keep prompts brief and clear, mark the parts of your input with markdown, XML tags or section titles, and try no examples before adding any. In OpenAI's words, "These models perform best with straightforward prompts."
The guide names o3, o4-mini and GPT-4.1 and shows no updated date, so read it as a principle, not a current model list.
In ChatGPT, OpenAI's help center says to use the Thinking slider to choose your effort, and describes Instant as fast responses for everyday questions. A separate setting decides whether ChatGPT adds thinking automatically while Instant is selected, and Free and Go users can use Think for harder questions.
What Anthropic says about Claude
Anthropic's prompting best practices say that when Claude is thinking, you should prefer general instructions over prescriptive steps. A prompt such as "think thoroughly" often produces better reasoning than a hand-written step-by-step plan. Anthropic also claims Claude's reasoning frequently exceeds what a human would prescribe; that is a vendor claim, but the lesson holds: describe the goal, not the route.
Examples still earn their place. Anthropic says multishot examples (its word for few-shot) work with thinking, and that worked examples shape how Claude approaches similar problems.
With thinking off, manual chain-of-thought is the fallback: ask Claude to think through the problem first and put the final answer in <answer> tags. On some of Anthropic's newest models, asking the model to write out its reasoning, say in <thinking> tags, might be declined, so use built-in thinking there.
One wording trap: with extended thinking disabled, Claude Opus 4.5 is particularly sensitive to the word think and its variants. Anthropic suggests consider, evaluate or reason through instead.
In the Claude apps, effort sets how much thinking Claude applies, and effort and thinking are separate settings you can combine any way you like. Anthropic's help center sums it up: "For complex tasks, raise the effort level, turn on thinking, or both." For everyday tasks the defaults work well, and Low or Medium effort stretches your usage further.
What Google says about Gemini
Google's Gemini thinking docs don't tell you to stop writing step-by-step prompts. Instead, when it explains how much Gemini should think, Google says many common requests benefit from some step-by-step processing, and Gemini 4 Argon (gemini-4-argon) offers low, medium (the default) and high thinking levels. You cannot turn thinking off on Gemini 4 Argon.
Google adds that simple requests such as fact retrieval or classification may not need complex reasoning. Hard tasks like complex math and coding call for a high thinking budget. If an answer misses, Google suggests reading Gemini's thought summaries carefully to see where it went off track.
Zero-shot, few-shot or chain-of-thought: what's the difference?
Zero-shot prompting gives the model only the task, few-shot prompting adds a few examples of the answer you want, and chain-of-thought prompting asks for, or shows, the reasoning first. They combine: Kojima's one-liner is zero-shot chain-of-thought, and Wei's worked examples are few-shot chain-of-thought.
| Style | What you add | Everyday example | Best for |
|---|---|---|---|
| Zero-shot | Only the task | Sort these emails into complaint, question or praise. | Easy tasks, and a first try on a reasoning model |
| Few-shot | Two or three examples with answers | Three labelled past emails, then the new one | Fixed formats, tone and labels |
| Zero-shot chain-of-thought | One line asking for reasoning first | Work through this one step at a time, then give the total. | Standard models doing sums or logic |
| Few-shot chain-of-thought | Worked examples that show the reasoning | A solved refund case that shows the policy in action | Repeat tasks that need the same method |
On examples, the vendors lean differently. OpenAI says reasoning models often don't need few-shot examples, so start without them; Anthropic says worked examples still shape how Claude reasons. If your team has a house method, such as how you apply a refund policy, one worked example is worth adding.
For other techniques ranked by how well research backs them, see our guide to 10 AI prompts that improve your answers, ranked by evidence.
When should you add a chain-of-thought instruction?
Add a step-by-step instruction only when the model's thinking is off and the task has steps you would work out on paper, such as sums, formulas or logic puzzles. In every other case, change the thinking setting, ask directly, or hand the math to a tool.
| Your situation | Add a step-by-step line? | Do this instead, or as well | Why |
|---|---|---|---|
| Thinking on, hard task | No | Raise the effort or thinking level. Add a self-check. | OpenAI says it may not help. Wharton saw marginal gains at best for 20-80% more time. |
| Thinking on, easy task | No | Lower the effort or use a faster mode. | Claude answers easy queries directly. Low and Medium effort stretch usage. |
| Thinking off, multi-step math or logic | Yes | Ask for the reasoning first and the final answer on its own line. | Gains concentrate in math and logic. It is Anthropic's fallback. |
| Thinking off, facts, labels or rewrites | No, but don't forbid explanation | Ask plainly. | Much smaller gains. 35-600% slower. More easy-question errors in three of five models. |
| Gut-feel pattern task, such as recognising faces | No | Ask for the answer directly. | Liu et al. saw accuracy drop across many models. |
| Exact numbers, tool available | Not as a substitute | Let it run code or a calculator. | It rarely beats tool-augmented approaches. |
Here is a handy tie-breaker based on a finding from Sprague et al. On MMLU, skipping chain-of-thought gave almost identical accuracy unless the question or the model's answer contained an equals sign, a sign of real calculation.
So ask yourself: would I write an equals sign while solving this? If yes and thinking is off, add the line.
Not sure whether thinking is on? Here is where to look.
- ChatGPT: the Thinking slider sets the effort, and Instant can add thinking automatically if that setting is on.
- Claude apps: the Thinking toggle doesn't appear for models where thinking is always on, so pick a lower effort to make them think less.
- Gemini 4 Argon: it always thinks, so your only choice is the thinking level.
Wharton's own advice is to weigh potential gains against extra token use and response time. Tokens are the chunks of text a model reads and writes. Chain-of-thought generally needs far more of them, which adds cost and time.
What chain-of-thought prompt examples can you copy?
The chain-of-thought prompts below each fit one situation: on a standard model you add one work through it first line and ask for the final answer separately, and on a reasoning model you swap that line for a clear goal and a self-check. Replace the square-bracket parts with your own details.
Standard model, thinking off: comparing quotes
I'm comparing three supplier quotes for the same order.
For each quote, add the delivery fee and subtract any discount.
Work through one quote at a time before you answer.
On the last line, write only: Cheapest: [supplier] at [total].
[paste the three quotes]
Claude with thinking off: Anthropic's fallback pattern
[describe the problem]
Reason through the problem before you answer.
Put only your final answer inside <answer></answer> tags.
This follows Anthropic's fallback advice and uses reason through rather than think, which matters on Claude Opus 4.5 with extended thinking disabled.
Reasoning model or thinking on: goal plus self-check
## Goal
[what you want, in one sentence]
## Context
[the facts, numbers or documents it needs]
## Constraints
[length, tone, anything to avoid]
Before you finish, verify your answer against [test criteria].
The headings mark each part of your input, which is what OpenAI's advice on delimiters asks for. The last line is Anthropic's suggested self-check, which it says catches errors reliably, especially for coding and math. Notice what's missing: no think step by step.
Repeat task: a worked example that teaches your method
Here is how we decide refund requests.
Example
Request: [customer message]
Reasoning: The order arrived [X] days ago. Our policy allows returns within [Y] days. [X] is less than [Y], so the refund is allowed.
Decision: Approve
Now decide this one the same way.
Request: [new customer message]
This is few-shot chain-of-thought. It suits standard models, and Anthropic says worked examples like this also shape how Claude approaches similar problems when it is thinking.
Which prompt should you use on your next task?
Start by checking whether the model is already thinking, because that one fact decides the rest. Then follow this three-step plan.
- Check the mode. Look at the model picker or Thinking control before you write anything. Some models think all the time.
- If thinking is on, drop the ritual. Delete think step by step lines, write the goal and constraints clearly, and add a self-check. Raise the effort or thinking level only for hard math, code or analysis.
- If thinking is off, test once. Run a real math or logic task you repeat with and without a step-by-step line, and time both. Keep the line only if the answers are clearly better, since Wharton measured 35-600% longer response times on standard models.
Still torn? Use this quick pick.
- Pick a step-by-step line if you are on a fast, non-thinking mode and doing sums or logic.
- Pick a higher thinking setting if the model already reasons on its own and the task is hard.
- Pick a calculator or code tool if the exact number matters more than the explanation.
Model names and setting labels were checked against OpenAI, Anthropic and Google's own pages in November 2026, and app menus move often, so look for the nearest match if yours looks different.
