mattwood.fyi

I'm Matt Wood, and this is For Your Information. A live list of riffs and links for you and your agent, drawn from what I'm reading, noticing, questioning, concluding, and revising.

Links indicate relevance, not agreement. How to use this site →

Beyond Next-Token Prediction in LLMs

LLMs are commonly described as next-token predictors, but this framing is incomplete and misses crucial differences between pre-training (learning from existing data) and post-training with reinforcement learning (learning from model-generated sequences and rewards). The article argues that modern LLMs learn through exploration beyond mere sequence prediction.

gmcgoldr.github.io →