Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Showed that adding worked examples with step-by-step reasoning to a prompt makes large models far better at maths and logic problems.
Instead of showing a model plain question and answer pairs, show it the question, the reasoning steps, then the answer. With eight such examples, PaLM 540B more than tripled its score on grade-school maths word problems.
Making models bigger had not been enough to crack multi-step arithmetic, commonsense and symbolic reasoning.
Prompts with eight hand-written exemplars, each an input, a chain of intermediate reasoning steps in plain language, and the answer. No fine-tuning.
On GSM8K, PaLM 540B went from 17.9% with standard prompting to 56.9%, beating the previous best of 55% from a fine-tuned model.
Why it matters
It showed that how you ask changes what a model can do, and that the gains only appear at around 100B parameters. Today’s reasoning models build the same step-by-step idea into training.
This is a lite card. The full version adds the core idea in technical detail, limitations and open questions, what it builds on and led to, and key terms.