PhysiCalWorld: In-Context Physics Calibration for World Models
Abstract
PhysiCalWorld calibrates action-conditioned world models to instance-specific dynamics via in-context physics descriptor tokens, improving prediction accuracy and planning without fine-tuning or physics labels.
Existing world models typically perceive a scene through current visual frame or a narrow history window. However, these observations may not reveal the physics properties that determine transition dynamics, such as an object's mass distribution, joint damping, or surface friction. Without calibrating these physics factors for world models, predictions tend to regress toward an empirical prior and induce severe drift in novel test scenes. To this end, we propose PhysiCalWorld, which calibrates action-conditioned world models to the instance-specific dynamics via in-context prompting, without fine-tuning or analytical physics modeling. Our method encodes an exploratory interaction clip collected within the test scene into a compact physics descriptor token, which grounds subsequent predictions in the scene's actual physical behavior. PhysiCalWorld is trained end-to-end with the latent prediction loss alone, without physics labels. At inference time, the descriptor token is extracted once and can be reused across rollouts with negligible compute overhead. We evaluate PhysiCalWorld across four environments covering diverse physics variations and show that it substantially improves rollout prediction accuracy and visual planning success rate. The learned descriptors also form a physically structured latent space and generalize beyond the physics ranges seen during training.
Community
Thank you for sharing your work. PhysiCalWorld improves rollout prediction and visual planning across four environments by conditioning a frozen world model on a descriptor from a short test-scene interaction, including physical parameters outside the training ranges.
We would like to discuss your work in Section 3.3 (“World-model-based control”) of our survey, In-Context Learning for Robots: Methods and Applications, to discuss instance-specific dynamics calibration as a way of conditioning action rollouts, and explain what the exploratory interaction contributes before planning begins.
If you find the survey helpful in situating physical calibration within the broader literature on robot adaptation, we would be grateful if you would consider citing it.
Models citing this paper 1
Datasets citing this paper 1
physical-world/physical-world-dataset-release
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper