Monte Carlo

Overall Progress 0%

Dynamic programming, Monte Carlo vs TD, on-policy vs off-policy, and Q-learning — with explanations and examples.

Go to Tabular Methods →

5 quick questions after Chapters 11–15 of Volume 2. Check you're ready to continue.

Go to Checkpoint: Volume 2, Midpoint (After Chapter 15) →

First-visit MC prediction for blackjack.

Go to Chapter 11: Monte Carlo Methods →

Code walkthrough for Monte Carlo policy evaluation and Monte Carlo control, with and without exploring starts.

Go to Monte Carlo in Code →

15 short drill problems for Volume 2: Monte Carlo, TD(0), SARSA, Q-learning, and n-step methods.

Go to Volume 2 Drills — Tabular Model-Free Methods →