MuZero Note
Model architecture for single agent deterministic game which can trained without prior human knowledge about the rules and strategies.. Main Contributions: Monte Carlo Tree Search (MCTS) Solve the exploitation vs exploration dilemma. The use of Representation, Prediction and Dynamic function Prediction functon $f$, predicts policy and value, $p_t$ and $v_t$ Dynamic function $g$, given the current state and action taken, $s_t$ and $a_{t+1}$ predicts the next state and immediate reward, $s_{t+1}$ and $r_{t+1}$ Representation function $h$, convert current state to latent space, $s_t$ Cons: Only can learn in an environment with relatively small action space Combination of policy evaluation and policy improvement (both can be called policy iteration) https://arxiv.org/pdf/2104.06303.pdf Brilliant summary of muzero in this paper