Research direction
The Large World Model
Language models predict the next token. The world is not made of tokens.
A language model is trained to continue text: given what came before, return a plausible next word. Fluency about the world is the by-product. We are building for a different object of prediction — the next state of the world itself, the thing the words are only describing.
The next token is not the next state
Predicting the next token and predicting the next state are not the same task in different clothes. A token is a symbol in a description; a state is a fact about the world — where an object is, what a person knows, what an action just changed. A model can be word-perfect about a kitchen it cannot reason about: name every utensil, and still not know that moving the pan off the heat lets it cool. Fluency is a map of how people talk about the world. It is not a map of the world.
What a world model predicts
A world model’s object of prediction is the state and how it changes. Represent the things that exist — objects, people, the relations between them — and predict the transition: given a state and an action, what is the next state. Language becomes one read-out among many, because the model is no longer anchored to the text. Ask it what happens if the file is opened, the trip is booked, the pan is moved, and the question is about consequences, not about the most likely next word.
What a language model predicts
the next token in a description
What a world model predicts
Sₜ
- panon heat
- youat stove
Sₜ₊₁
- panoff heat
- youat stove
the next state of the world
Why this is the bet
The industry measures progress in tokens per second and points per benchmark. What people actually ask for is work: book the trip, fix the test, remember the studio. Work happens in the world, and the world has state — so an intelligence that does the work has to model the state, not autocomplete the description of it. That is the wager the whole program is built on. If it is right, the models that reason over state will finish what fluency alone never could.
Predict the next token and you learn to describe the world. Predict the next state and you learn to change it.
What would falsify it
A bet worth publishing is one that can lose. So the claim is stated where it can be broken: in a bounded world with explicit state, a model trained to predict transitions should out-predict a language model of the same size at what happens next — measured on transitions neither has seen, with the metric and the budget fixed in advance. If it does not, the thesis takes the hit, in public, at the same address. The first such test is DWM-1, pre-registered before it runs. No result is claimed before it exists.
Read next