Chapter 56: MuZero Intuition
Learning objectives Read a MuZero paper summary and explain how MuZero learns a model in latent space without access to the true environment dynamics. Explain how MuZero handles reward prediction and value prediction in the latent space. Contrast with AlphaZero (which uses the true game rules). Concept and real-world RL MuZero learns a latent dynamics model: instead of predicting raw next state, it predicts the next latent state and (optionally) reward and value. So the “model” is learned end-to-end for the purpose of planning; it does not need to match the true state. This allows MuZero to work in video games and domains where rules are unknown. In game AI, MuZero achieves strong results on Atari and board games without hand-coded dynamics. ...