DQN | Reinforcement Learning Curriculum

Why FA, policy gradient update, DQN exploration, experience replay, and actor-critic — with explanations.

Learn convolution and pooling from scratch in NumPy. See how Atari DQN uses CNNs to process raw pixels.

5 quick questions after Chapters 21–25 of Volume 3. Check you're ready to continue.

A practical guide to reading reinforcement learning research papers: structure, notation, and three annotated examples (DQN, PPO, SAC).

10–12 questions on DQN, policy gradient, PPO, replay, target network. Solutions included.

DQN for CartPole with replay and target network.

Replay buffer class with push and sample.

Hard vs soft target updates in DQN.

Double DQN: online selects, target evaluates; compare with DQN.

Combine DDQN, Dueling, PER, NoisyNet, multi-step; train on Pong.

DQN with ε-greedy on Montezuma's Revenge; sparse rewards.

15 short drill problems for Volume 3: linear FA, semi-gradient TD, DQN, replay buffer, target network, Double DQN, and dueling networks.

Review deep learning and see why RL needs neural networks — the bridge to DQN and policy gradients.

Review Volume 2 tabular methods and preview Volume 3. From Q-tables to neural network function approximation.

Review Volume 3 (DQN and variants) and preview Volume 4 (Policy Gradients). From value-based to policy-based methods.

You have mastered the foundations. Now, combine neural networks with RL for high-dimensional problems like Atari or robotics.