What is Least-to-Most Prompting?
Least-to-Most Prompting comes from the paper Least-to-Most Prompting Enables Complex Reasoning in Large Language Models by Denny Zhou and colleagues at Google, published at ICLR 2023. It was designed to fix a specific weakness of Chain-of-Thought: models do well on problems similar to the examples in the prompt, but struggle when the real problem is harder than anything they were shown.
The technique borrows a strategy from education research: teach by moving from the least difficult part to the most difficult. It has two stages:
- Decomposition: the model lists the sub-problems that need to be solved, ordered so that each builds on the previous ones.
- Sequential solving: the model solves each sub-problem in turn, with the question and all previous answers in context, until the final sub-problem answers the original question.
On SCAN, a benchmark that tests whether models can combine known pieces into new instructions, Least-to-Most with 14 examples reached at least 99% accuracy, while Chain-of-Thought reached about 16%.
When to Use Least-to-Most Prompting
Multi-Step Math
Word problems where intermediate quantities must be computed before the final one can be.
Coding Tasks
Features that need a data model, then an API, then UI changes, each depending on the previous step.
Project Planning
Turn a large goal into ordered milestones, then detail each milestone with the previous ones as context.
Research Questions
Answer background questions first, then the comparative or causal question that depends on them.
Compositional Instructions
Commands built from smaller commands, such as data transformations or query building.
Data Analysis
Clean, aggregate, then interpret: each stage needs the output of the one before.
How to Use Least-to-Most Prompting
- 1
Ask for a decomposition
"To answer this, what simpler questions do we need to answer first?" Ask for a numbered list ordered from simplest to the final question.
- 2
Review the sub-problems
Check that answering all of them really answers the original question. For important work, fix the list before solving.
- 3
Solve one sub-problem at a time
Give the model the original question, all previous sub-problems with their answers, and the next sub-problem.
- 4
Answer the original question last
The final sub-problem should be the original question itself, now answered with everything solved so far in context.
Prompt Examples
Problem: Elsa has 5 apples. Anna has 2 more apples than Elsa. Kristoff has twice as many as Anna. How many apples do they have together? Stage 1: List the sub-questions we must answer first, from simplest to the original question. Stage 2: Answer each sub-question in order. Before each one, restate the answers you already have. Final answer on the last line.
Call 1:
We want to add "export invoices as PDF" to our billing web app (Node.js API, React frontend, Postgres).
Decompose this into the smallest set of sub-problems we must solve, ordered so that each depends only on earlier ones. Return a numbered list only.
Call 2 (repeat per sub-problem):
Goal: add "export invoices as PDF".
Already solved:
{{ previous_sub_problems_and_answers }}
Now solve sub-problem {{ n }}: {{ sub_problem }}
Be concrete: files, functions, data structures.Pros and Cons
| 🟢 Pros | 🔴 Cons |
|---|---|
| Generalises to problems harder than the examples shown | More calls or longer output than single-pass reasoning |
| Decomposition is visible and can be reviewed before solving | A wrong decomposition derails every later step |
| Each step has a narrow focus, reducing errors | Overkill for simple, single-step tasks |
| Maps directly onto multi-call pipelines and agents | Context grows with each solved sub-problem |
Least-to-Most Prompting Prompt Template
Copy this template and replace each {{ variable }}. It uses Prompt Edit's variable syntax, so you can paste it into the app and fill the fields from a form every time you reuse it.
Problem: {{ problem }}
Stage 1: Break the problem into the simpler sub-problems we must solve first. Number them from simplest to the original problem.
Stage 2: Solve the sub-problems in order. Before each one, restate the answers you already have.
Final answer: {{ output_format }}Frequently Asked Questions
What is Least-to-Most Prompting?
Least-to-Most Prompting is a two-stage technique introduced by Denny Zhou and colleagues at Google (2022, ICLR 2023). First the model decomposes a complex problem into a list of simpler sub-problems. Then it solves them in order, from easiest to hardest, and each answer is added to the context for the next sub-problem.
How is Least-to-Most different from Chain-of-Thought?
Chain-of-Thought reasons through the whole problem in one pass. Least-to-Most separates planning from solving: it first makes the decomposition explicit, then solves each sub-problem with the previous answers available. That makes it better at problems harder than the examples shown in the prompt, which is where Chain-of-Thought tends to break down.
What results did Least-to-Most achieve?
On the SCAN compositional generalisation benchmark, GPT-3 (code-davinci-002) with Least-to-Most reached at least 99% accuracy using just 14 examples, compared with about 16% using Chain-of-Thought. Specialised neural-symbolic models had needed over 15,000 training examples for similar results.
Is Least-to-Most the same as Prompt Chaining?
They are related. Prompt Chaining is the general pattern of feeding one prompt's output into the next, with steps you design by hand. Least-to-Most is a specific reasoning method where the model itself produces the decomposition and the sub-problems are ordered from simplest to hardest.
Can I run Least-to-Most in a single prompt?
Yes. Ask for a numbered list of sub-problems first, then instruct the model to solve them one at a time, restating the previous answers before each step. For long or high-stakes tasks, separate calls give you more control and let you check the decomposition before solving.
When should I avoid Least-to-Most?
For simple tasks the decomposition step is overhead. It also fails if the decomposition is wrong, so review the sub-problem list for complex work, or ask the model to check that solving all sub-problems actually answers the original question.
Origin & Sources
- Zhou et al. (2022). Least-to-Most Prompting Enables Complex Reasoning in Large Language Models. arXiv / ICLR 2023.