World Models as Simulators, Not Oracles

Learned world models are often evaluated as predictors: how close is the rollout to the ground-truth trajectory? For applied robotics that framing is a trap. What matters is whether a policy trained in the model transfers, and transfer tolerates a surprising amount of visual and dynamical error.

We categorize model error into three buckets. Cosmetic error — texture, lighting, minor geometry — is nearly free. Dynamical error in contact-rich regimes is expensive. Error in the reward-relevant state is fatal. Budget your model capacity accordingly.

This lets us train compact, fast world models that are visually mediocre but contact-accurate, and get more useful policy hours per GPU than a photoreal model that mispredicts friction.

Domain randomization then covers the cosmetic gap, and a small real-world calibration set corrects the systematic dynamical bias. The loop is: simulate broadly, calibrate narrowly, deploy conservatively.