Learning

Learning objectives Use spaced practice and active recall to retain concepts. Combine reading with coding and small projects. Avoid common traps (passive watching, skipping exercises). Spaced practice Do not cram. Spread your study over time: e.g. 30–60 minutes per day on the curriculum, rather than one long session per week. Revisit earlier chapters when you hit later material (e.g. when you do TD, recall what you did in Monte Carlo). Spacing strengthens long-term retention. ...

Learning objectives Read an exercise or spec and identify the inputs, outputs, and steps. Break a coding task into small, testable steps. Use documentation and error messages to fix issues without giving up. Why “by yourself” matters The curriculum gives you exercises with worked solutions. The goal is not to copy the solution but to try first, then check. Coding by yourself—even when you get stuck—builds the skill to implement algorithms and debug them later in real projects or research. This part focuses on reading and planning. ...

Learning objectives Build a habit of coding a little every day (or every session). Know when to look at the worked solution and how to use it without copying. Use print statements and small tests to debug. Practice regularly Short sessions: Even 20–30 minutes of coding (one small step: e.g. “get the bandit environment returning rewards”) counts. Consistency beats rare long sessions. One exercise at a time: Finish (or get seriously stuck on) one exercise before jumping to the next. The curriculum is ordered so that skills build. Re-do later: After reading the solution, close it and re-implement the same exercise a few days later. You will remember the idea but have to write the code again—that strengthens retention. When to look at the solution After a real attempt: Try for at least 15–30 minutes. Write something—even if it is wrong. Then open the solution. When stuck on one bug: If you have one specific bug (e.g. “my value function is all zeros”), try to fix it with print statements or a minimal example. If you are still stuck after 15 minutes, look at how the solution handles that part. Do not just copy: After reading the solution, close it and type the code yourself. You will understand it better than if you copy-paste. Proof that using Jupyter Notebook is the same as not using it You can do all exercises in Jupyter (run cells, plot inline) or in plain Python scripts (run with python script.py). The algorithms and math are the same. Jupyter is convenient for plotting and trying small pieces; scripts are convenient for running full experiments and version control. Use whichever you prefer. The curriculum does not require one or the other. ...

Learning objectives Get a quick roadmap: what to do first and how to use the course resources. Know where to find detailed advice (long version and FAQ). How to succeed in this course (short version) Follow the order. Use the Course outline and Learning path. Start with prerequisites and math if you need them; then Volume 1 (foundations, bandits, MDPs, DP), then Volume 2 (MC, TD, SARSA, Q-learning), then Volume 3 and beyond. Do not skip the foundations. ...

Learning objectives Plan your path: prerequisites first, then foundations, then advanced volumes. Use the exercises and worked solutions effectively. Stay motivated and recover from getting stuck. Follow the order The curriculum is designed in basic-to-advanced order. Use the Course outline and Learning path as your map. Do not skip Volume 1 (foundations, bandits, MDPs, DP) or Volume 2 (MC, TD, SARSA, Q-learning) even if you are in a hurry. Later volumes (DQN, policy gradients, etc.) build on these. If you find a chapter hard, revisit the prerequisite (e.g. Math for RL or Prerequisites). ...