Blog
If AI only predicts the next word, how can it solve maths?
Predicting the next word sounds simple. In practice, it can mean finding the next valid step in a long chain of reasoning.
Imagine someone tells you, “AI is only guessing which word comes next.” A minute later, you read that an AI system solved a difficult maths problem. Both statements can be true. Most of the confusion sits inside one word: only.
A language model builds its answer one small piece at a time. These pieces are called tokens: words, parts of words, or punctuation marks. But it does not choose the next piece like a roll of the dice. It considers your question, the text already written, and relationships learned during training.
How AI predicts the next word — and the next step
Think about a chess player. On every turn, they “only” choose the next move. Yet a good choice requires seeing the whole board, knowing the rules, and anticipating what may happen next. Calling chess “choosing the next move” is accurate but incomplete.
Language models work in a similar way. The word “bank” means one thing in “I deposited money at the bank” and another beside a river. The surrounding words change its meaning. A mechanism that helps the model track these relationships is at the heart of an architecture called a Transformer.
Mathematics is an even more tightly controlled language. The next line of a proof must follow from the problem, definitions, and previous steps. If a model has learned enough patterns of mathematical reasoning, it can propose one plausible step, then another, eventually forming a new solution.
Why the maths is not magic
This does not mean every confident-looking answer is correct. A language model can produce a very convincing mistake. Successful mathematics systems often add two important ingredients: they explore several possible paths and check the result.
Google DeepMind’s AlphaProof, for example, combined a language model with a search for possible proofs and Lean, software that can formally verify each step. In 2024, it helped solve new International Mathematical Olympiad problems. That is impressive, but it is not the same as automatically solving any famous problem that mathematicians have studied for decades.
What this means when you use AI
“It predicts the next token” describes how the answer is produced. It does not tell you that the answer is simple or reliable.
In a business, treat AI as a fast draft-maker, not the final judge. Let it propose an email, calculation, or procedure. Wherever a mistake matters, check it with a calculator, company rules, trusted data, or a person. AI’s strength is not perfection. It is quickly suggesting a useful next step.
Sources
- GPT-4 Technical Report: training through next-token prediction
- Attention Is All You Need: the original Transformer paper
- Google DeepMind: AlphaProof and the 2024 Mathematics Olympiad
Do you have a process where AI needs a reliable safety check? Get in touch — we will design a solution that does not rely on a confident-sounding answer alone.