Python

Concepts used in the curriculum and in Preliminary: Python basics: data structures (list, tuple, dict, set), classes and objects, functions, list comprehensions, loops, and conditionals. RL code is full of trajectories, configs, and custom types (agents, buffers)—all built from these basics. Data structures Choosing the right structure makes code clearer and often faster. In RL you’ll use all four constantly. List — ordered, mutable Use for sequences: trajectory of states, batch of indices, rewards per episode. ...

This is the final self-assessment step of the preliminary material. Use it to reflect on your readiness and to find gaps before starting the 100-chapter curriculum. Back to Preliminary. Why this step matters The curriculum assumes comfort with probability, linear algebra, calculus, Python, NumPy, PyTorch, and basic RL ideas. If you are weak in one area, you can still start, but you’ll progress more smoothly if you strengthen those areas first. This page helps you identify where to spend a bit more time. ...

Learning objectives Install NumPy, SciPy, Matplotlib, Pandas, and IPython (or Jupyter) for the curriculum. Optionally install Theano or TensorFlow if you follow exercises that use them; the curriculum primarily uses PyTorch for deep RL. Core libraries (required for early volumes) NumPy: pip install numpy Used for arrays, random numbers, and numerical operations in bandits, MDPs, and tabular methods. Matplotlib: pip install matplotlib Used for plotting learning curves, value functions, and heatmaps. ...

This page covers the Python you need for the preliminary assessment: writing functions, working with lists, and using list comprehensions. Back to Preliminary. Why this matters for RL RL code is full of trajectories (lists of states, actions, rewards), configs (dicts), and custom types (agents, buffers). You need to write clear functions, slice sequences, and aggregate data. Moving averages and rolling computations appear when processing reward sequences or returns. Learning objectives Write a function that returns the moving average of a list; use list comprehensions and loops; structure code for clarity and reuse. ...