TinyDreamer.
A little agent learns a world, then practices inside it.
Give the cart a push.
Watch reality surprise its imagination.
REALITY
MuJoCo physics DREAM
learned future · 0.75 s
How does this work?
This is a small Dreamer-inspired implementation for understanding latent world models and imagination-based reinforcement learning.
- The agent collects observations and rewards from real MuJoCo physics.
- A recurrent state-space model (RSSM) learns a compressed, predictive belief about that world.
- The actor practices on imagined trajectories generated by the learned model. Its critic estimates future return.
- Here, the solid pole shows what physics does. Translucent blue poles show decoded neural predictions at several future times.
- Push the cart to disturb the prediction. New observations update the belief, and a fresh dream follows.
The deployed policy runs locally on your device. Training used vector observations and CPU PyTorch; this is not a reproduction of full DreamerV3. Reward predictions can have small errors outside the real reward’s 0–1 range. The error readout compares a one-step neural prediction with the following physical observation.