What Sort of Maths Are LLMs Good At?

I'm Matt Wood, and this is For Your Information. A live list of riffs and links for you and your agent, drawn from what I'm reading, noticing, questioning, concluding, and revising.

Explores the mathematical capabilities and limitations of large language models following their recent breakthroughs in major open problems, analyzing whether they excel particularly at finding counterexamples versus proofs.

gowers.wordpress.com →

Connections

Supports

Autoform Bot: Multi-agent system for translating LaTeX mathematics into verified Lean 4 proofs Autoform Bot translating LaTeX mathematics into Lean 4 proofs is a specific instantiation of LLM mathematical capability that the new item's analysis would apply to
MaxProof: Scaling Mathematical Proof with Generative-Verifier RL and Population-Level Test-Time Scaling MaxProof's work on scaling mathematical proof generation with RL is a concrete example of the kind of mathematical capability the new item is analyzing and categorizing
Leiden Declaration on Artificial Intelligence and Mathematics The Leiden Declaration on AI and Mathematics provides a formal framework for evaluating AI's role in mathematics, which directly contextualizes the capability analysis the new item performs

Related to

LLMs can't jump 'LLMs can't jump' likely examines limitations of LLMs in structured reasoning tasks, directly complementing the new item's investigation of mathematical strengths and weaknesses

Challenges

Schema: Frontier Models with the Right Harness Achieve ~99% on ARC-AGI-3 Public Frontier models achieving ~99% on ARC-AGI-3 with the right harness suggests broader reasoning capability that the new item's analysis of mathematical limitations may need to account for or qualify

Supported by

Comparing 11 Different AI Models Examining what LLMs are good at mathematically is a specific instance of the comparative prompt-based evaluation methodology used in the 11-model comparison, providing complementary evidence about model capability differences.

Type:	link
First published:	Wednesday, 12 August 2026 at 15:48 UTC
Location:	https://gowers.wordpress.com/2026/08/12/what-sort-of-maths-are-llms-good-at/
URL:	https://mattwood.fyi/i/cd6a8d69