Count-Based Exploration

Learning objectives Implement count-based exploration for discrete state spaces using a hash table and a bonus such as \(1/\sqrt{N(s)}\). Implement pseudo-counts from a density model (e.g. PixelCNN or simpler density estimator) for image-based states. Explain why pseudo-counts are needed when the state space is huge or continuous (e.g. Atari frames). Test count-based and pseudo-count exploration on a simple Atari-style or image-based task and compare exploration coverage. Relate count-based and pseudo-count methods to game AI and recommendation (e.g. diversity). Concept and real-world RL ...