This page points you to all places where worked solutions (step-by-step answers, derivations, and code) are available. Use it to check your work or to study from full solutions.
Math for RL
Each topic page has practice questions with full solutions in collapsible “Answer and explanation” sections:
- Probability & statistics — Sample mean, variance, expectation, law of large numbers, bandit-style problems.
- Linear algebra — Dot product, matrix-vector product, gradients, NumPy.
- Calculus — Derivatives, chain rule, partial derivatives, policy gradient.
Every practice question includes a step-by-step derivation and a short “In RL” explanation.
Preliminary assessment
The Preliminary section has worked problems with explanations for each topic (probability, linear algebra, calculus, Python, NumPy, PyTorch, RL framework, value functions, tabular methods, function approximation). Each topic page lists the assessment questions (Q1–Q25) and provides collapsible “Answer and explanation” blocks with full derivations and code where relevant.
Phase quizzes
- Phase 1 (Math) — Step-wise solutions and brief “why” explanations for all 10 questions.
- Phase 2 (Readiness) — Step-by-step answers for Python, NumPy, PyTorch, and Gym.
- Phase 3 (Foundations) and Phase 4 (Deep RL) — Expanded solutions with key steps and RL tie-ins.
Curriculum (100 chapters)
Every chapter has at least one Worked solution in a collapsible section. The solution is either for the main exercise or for the first “Warm-up” extra-practice item. Open the “Worked solution” collapse on each chapter page to see a full step-by-step answer (math, code, or both) and a short explanation. Start with Volume 1 and use the same pattern through Volume 10.
Prerequisites
- NumPy, Python, and PyTorch each have a Worked examples section with 2 short “Task → Solution” examples (collapsible) before the exercises.
Quick links: Math for RL · Preliminary · Curriculum · Assessments