TL;DR: Adaptive compute assumes more depth means better predictions. In latent world models, that isn’t always true. Across nine DeepMind Control tasks (eight seeds), depth helped on 6/9, was flat on 1/9, and hurt on 2/9 — and a training catch-22 can erase the depth advantage routing was meant to exploit.
Adaptive computation is one of those ideas that feels almost obviously correct.
If an input is easy, spend less compute. If it’s difficult, spend more. Modern machine learning is full of methods built around this intuition—from early-exit networks to adaptive transformers and mixture-of-experts models. More recently, the same idea has begun appearing in latent world models, where a planner repeatedly rolls a learned dynamics model forward to imagine future futures.
At first glance, the opportunity seems clear. A planner might call its dynamics model hundreds of times while evaluating candidate action sequences. If some prediction steps require less computation than others, routing compute adaptively could make planning substantially cheaper.
But this idea quietly assumes something that, surprisingly, has rarely been tested.
Does a deeper predictor actually produce better rollouts?
That question became the starting point for our work. The answer turned out to be much more complicated than we expected.
Paper: Adaptive Compute in Latent World Models: When Depth Helps, Hurts, or Doesn’t Matter.
The assumption behind adaptive compute
Most adaptive-compute methods assume a simple relationship between computation and prediction quality: a deeper network should produce a better prediction than a shallower one. Once that relationship exists, a router can decide when the additional computation is worthwhile.
For ordinary supervised learning, this assumption is often reasonable because each prediction is evaluated independently.
World models are different. A latent world model predicts the next latent state, then feeds that prediction back into itself to predict the following state, repeating this process many times during planning. Small prediction errors compound.
So even if a deeper predictor makes a slightly better single-step prediction, that advantage still has to survive repeated autoregressive composition, and nothing guarantees it does.
Before asking how to route computation, we first wanted to ask a more fundamental question:
Does additional predictor depth consistently improve rollout quality in latent world models?
If the answer is no, there may not be much for a router to route.
We expected multi-step training to create the answer
When we started this project, we already had an explanation in mind.
Previous experiments suggested that shallow and deep predictors often produced very similar rollouts (shallow penalty ρ ≈ 1). If true, adaptive depth would provide little benefit.
Our hypothesis was that this was a consequence of single-step training. Perhaps models simply were not being trained to preserve the benefits of depth over long autoregressive rollouts. If we instead trained them with multi-step latent overshooting (K=4), maybe a clear depth–quality tradeoff would emerge — the “created” regime in our pre-registration.
We committed to this hypothesis before running the large experimental campaign. Gate 1 was pre-registered to pass only if at least 2 of the first 3 tasks became created under multi-step training.
It failed.
Completely.
Gate 1 returned 0/3. Multi-step training did not manufacture the tradeoff. Cheetah inverted (ρK4 = 0.85), humanoid stayed flat, and walker already had a large intrinsic tradeoff under single-step training. A reduced-scale pilot that had looked encouraging (~1.25× on cheetah) collapsed to 0.85× at full scale.
Instead, the first three tasks split into three entirely different behaviors. On one, deeper predictors genuinely improved rollouts. On another, depth barely mattered. On a third, the surprising result was the opposite of what we expected:
the shallow predictor produced better rollouts than the full network — on cheetah-run, about 13% lower rollout error at the shallowest exit than at full depth (ρK4 = 0.87 [0.85, 0.89] over eight seeds).
That wasn’t supposed to happen.
Instead of abandoning the project, we changed the question. Rather than asking whether multi-step training creates a depth advantage, we asked what kinds of depth-composition behavior actually occur.
Three different regimes
Extending the study to nine DeepMind Control tasks — matched K=1 / K=4 training, later expanded to eight seeds per cell (162 healthy runs) — revealed that there wasn’t a single answer. We classify each task with a pre-registered instrument: the shallow penalty ρ = err(shallowest-exit rollout) / err(full-depth rollout).
1. Intrinsic tradeoff (6/9)
On six of nine tasks, additional depth genuinely improved rollout quality (ρK1 ≥ 1.25). The shallow predictor accumulated larger errors over the rollout horizon, while deeper predictors remained more accurate — the setting adaptive computation is designed for.
Magnitudes vary a lot. Walker-walk sits near 1.7–2.0×; acrobot-swingup reaches ρK4 ≈ 8.0 (shallow error about 8× the full-depth error). This is the majority case, and it recontextualizes the failed hypothesis: “latent rollouts don’t reward depth” was partly a task-selection artifact of the first tasks we studied.
2. Flat (1/9)
Humanoid-run showed almost no shallow-vs-full difference (ρK1 ≈ 0.90, ρK4 ≈ 0.96). Spending additional computation simply did not change prediction quality very much. Adaptive routing is largely vacuous here — not because routing fails, but because all depths behave similarly.
3. Inversion (2/9)
On two tasks, deeper predictors produced worse rollouts than shallower ones: cheetah-run (ρK4 = 0.87) and quadruped-walk (ρK4 = 0.89). That does not mean deeper networks are universally worse. It means that after repeatedly composing predictions autoregressively, the shallower predictor accumulated less error than the full predictor under the operating configuration we studied.
For adaptive computation, this is awkward. Routing toward deeper computation is no longer obviously beneficial. Instead, the router would need to learn that less computation is sometimes the better prediction operator.
Why would adding layers make rollouts worse?
At first, we suspected the inversion might simply be a property of certain environments. The experiments suggested something more interesting.
Our predictor uses deep supervision so that every early exit learns to make useful predictions — without this, adaptive routing would not be possible. We changed one thing: supervising shallow exits only at the first rollout step instead of every step, and left everything else untouched.
On cheetah — the strongest, most seed-consistent inversion — that ablation erased it. The shallow penalty moved by ∆ ≈ +0.28 over eight seeds; the standard and ablated distributions were non-overlapping, and ρ climbed back above 1. Walker, an intrinsic control, stayed essentially unchanged (~2×).
The same training procedure that makes early exits usable for adaptive routing can also train those exits into exceptionally strong rollout operators. In other words, the supervision needed to make routing possible can partially remove the depth advantage that routing was supposed to exploit.
We call this the routability catch-22.
This mechanism is not a universal explanation. It accounts for the robust cheetah inversion; quadruped’s more marginal inversion can arise for different reasons.
The story became larger than one mechanism
Once we started probing further, another pattern emerged. The regime depended on much more than the task name: evaluation metric, rollout horizon, encoder, predictor architecture, and — most strikingly — the training-data distribution. On the two tasks we retrained under competent-policy rather than random-policy data, both the inversion and the strong intrinsic tradeoff washed out toward flat (cheetah ρK4: 0.87 → 1.06; cartpole: 3.19 → 1.00), even with the catch-22 loss unchanged.
So asking “does additional compute help?” does not have a task-level answer. It depends on the entire operating configuration.
At the same time, not everything was unstable. Across many of these changes, a simpler distinction proved more robust: does depth help at all, or not? That broader intrinsic-versus-non-intrinsic split remained stable for most of the tasks we studied.
Predicting the regime before training
Once a taxonomy exists, a natural question follows: can we anticipate which regime a new task will fall into before training an adaptive-compute model?
We froze a simple logistic classifier on task dimensionality (observation and action dims), committed predictions for held-out tasks to git, and only then trained them. On genuinely novel held-out tasks the score was 2/3. The extreme extrapolation landed: dog-walk — predicted non-intrinsic at P(intrinsic)=0.00 from obs-dim 223 (~3× beyond the training range) — was the strongest inversion we measured (ρK4 = 0.70, all seeds). Cartpole-swingup was correctly intrinsic. The miss, fish-upright, turned out to be the first empirical instance of the “created” regime our original Gate 1 had failed to find.
This is still a small held-out set, but it suggests the taxonomy is more than a post-hoc description. Some aspects appear predictable before expensive experiments begin.
From rollout error to planning performance
Everything above concerns open-loop rollout prediction error. World models ultimately exist to support planning, so we asked whether ρ predicts planner behavior.
For the strongest intrinsic and inversion cases, it did. On cartpole (strong intrinsic), deep planning beat shallow by about +32 return. On acrobot, deep beat shallow on both viable seeds. On cheetah (robust inversion), the pattern flipped in closed-loop control: shallow planning returned 7.24 vs 6.58 for full depth (shallow winning on 6/8 seeds).
Under stronger planners and different training distributions the picture gets more nuanced — which is consistent with the configuration dependence above, rather than a contradiction of it.
A failed hypothesis that led somewhere more interesting
Looking back, the most valuable result of this project was not the one we originally set out to find.
We expected multi-step training to manufacture a depth–quality tradeoff. Pre-registration forced us to accept that the hypothesis was wrong. That negative result became the starting point for a broader investigation: a taxonomy of depth-composition regimes, a causal mechanism for one important inversion, evidence that the phenomenon transfers to planning under appropriate conditions, and a clearer picture of when adaptive computation is likely — or unlikely — to help.
Sometimes the most useful scientific result is not confirming an idea. It’s discovering that the assumptions behind the idea were incomplete.
Before deciding how to route computation, it may be worth asking a simpler question first:
Is additional computation actually helping this model in the first place?
