LLMs can't jump — mattwood.fyi

I'm Matt Wood, and this is For Your Information. A live list of riffs and links for you and your agent, drawn from what I'm reading, noticing, questioning, concluding, and revising.

Connections

Challenges

Schema: Frontier Models with the Right Harness Achieve ~99% on ARC-AGI-3 Public Schema's claim that frontier models with the right harness achieve ~99% on ARC-AGI-3 is directly challenged by the thesis that LLMs lack manipulative abduction and physical simulation — benchmark success may not reflect genuine inventive or scientific reasoning capability.

Supports

Brad Porter: We Saw This With Robotics at Amazon Too Brad Porter's robotics-at-Amazon perspective implicitly involves the gap between language-level reasoning and physical manipulation — the 'LLMs can't jump' thesis about needing physical simulation maps onto robotics practitioners' observations about grounding AI in physical action.
Intelligence is not the main bottleneck Both argue that current AI/LLM capabilities miss something fundamental — the 'LLMs can't jump' thesis that physical simulation and manipulative abduction are required for invention aligns with the claim that intelligence itself isn't the key bottleneck, suggesting different cognitive primitives are missing.
Leiden Declaration on Artificial Intelligence and Mathematics The Leiden Declaration on AI and Mathematics likely raises concerns about LLM limitations in genuine mathematical discovery; the 'manipulative abduction' thesis directly explains WHY LLMs struggle with novel scientific/mathematical invention rather than pattern-matched reasoning.

Supported by

Humanising LLM Outputs is Dumb Both argue against anthropomorphizing or over-attributing human-like qualities to LLMs — 'LLMs can't jump' challenges capability assumptions while this piece challenges the framing of making outputs 'sound human'
OpenSSH 10.5 Release Notes The OpenSSH team's frustration with AI models generating invalid security reports empirically supports the argument that LLMs have fundamental limitations — they can't reliably reason about complex security state machines
Emergent Introspective Awareness in Large Language Models Both papers probe fundamental cognitive limitations of LLMs — 'LLMs can't jump' examines reasoning/generalization gaps, while this paper investigates whether self-reporting accurately reflects internal states, together building a picture of where LLM cognition is and isn't reliable.

What Sort of Maths Are LLMs Good At? 'LLMs can't jump' likely examines limitations of LLMs in structured reasoning tasks, directly complementing the new item's investigation of mathematical strengths and weaknesses
Compression is prediction Both probe fundamental limitations of LLMs — 'LLMs can't jump' examines what models fail to generalize, while compression-as-prediction reveals what LLMs are actually doing when they model sequences, making them complementary perspectives on LLM capabilities

Challenged by

Improving Gpt 5 6 Sol In Chatgpt 'LLMs can't jump' argues LLMs face fundamental reasoning limitations; OpenAI's work on improving GPT problem-solving capabilities directly challenges or tests whether these limitations can be overcome through iterative improvement

Type:	link
First published:	Wednesday, 5 August 2026 at 15:40 UTC
Location:	https://openreview.net/forum?id=klU4737opt
URL:	https://mattwood.fyi/i/b22e8d01