Learning objectives
- Plan your path: prerequisites first, then foundations, then advanced volumes.
- Use the exercises and worked solutions effectively.
- Stay motivated and recover from getting stuck.
Follow the order
The curriculum is designed in basic-to-advanced order. Use the Course outline and Learning path as your map. Do not skip Volume 1 (foundations, bandits, MDPs, DP) or Volume 2 (MC, TD, SARSA, Q-learning) even if you are in a hurry. Later volumes (DQN, policy gradients, etc.) build on these. If you find a chapter hard, revisit the prerequisite (e.g. Math for RL or Prerequisites).
Do the exercises
Each chapter has one exercise and a worked solution (collapsible). Strategy:
- Try the exercise first. Read the spec, plan steps, write code. Give yourself at least 15–30 minutes before opening the solution.
- Compare with the solution. See where your approach matches or differs. Understand why the solution does what it does, not just the code.
- Re-implement later. After a few days, close the solution and implement again. This solidifies the algorithm.
Do not just read the solution and move on—you will forget.
Use the assessments
- Preliminary assessment: 25 questions on math, Python, NumPy, PyTorch, and basic RL. If you score well and feel comfortable, you are ready for the curriculum. If not, work through the linked prerequisites and try again.
- Phase quizzes: Phase 1 math, Phase 2 readiness, Phase 3 foundations, Phase 4 Deep RL help you check progress at each stage.
When you get stuck
- One concept: Re-read the chapter, check the Course outline for related topics, or look at the worked solution for that part only.
- Code bug: Add print statements, test on a tiny example (e.g. 2 arms, 3 steps), read the error message. See How to Code by Yourself (part 1) and part 2.
- Overwhelmed: Slow down. Do one chapter per week if needed. The goal is understanding, not speed.
- Motivation: Tie the material to a project you care about (e.g. Stock Trading, or a game/simulation). Remind yourself why you started.
Pace yourself
- Beginners: Plan for several months (e.g. 3–6) for Volumes 1–3 if you are also doing prerequisites and math. That is normal.
- With ML background: You may move faster through Volumes 1–2; spend more time on implementation and on Volumes 3–5.
- Consistency: Regular short sessions (e.g. 30 min/day) beat rare long binges.
Good luck. You have the Course outline, Learning path, Prerequisites, and this appendix; use them in order and you will get there.