Installation

Learning objectives Install NumPy, SciPy, Matplotlib, Pandas, and IPython (or Jupyter) for the curriculum. Optionally install Theano or TensorFlow if you follow exercises that use them; the curriculum primarily uses PyTorch for deep RL. Core libraries (required for early volumes) NumPy: pip install numpy Used for arrays, random numbers, and numerical operations in bandits, MDPs, and tabular methods. Matplotlib: pip install matplotlib Used for plotting learning curves, value functions, and heatmaps. ...

Learning objectives Know what software you need (Python, libraries, optional IDE). Perform a pre-installation check so you are ready for the curriculum. Pre-Installation Check Before diving into the curriculum, ensure you have: Python: Version 3.8 or higher (3.9–3.11 recommended). Check with python3 --version or python --version. pip: So you can install packages. Check with pip --version or pip3 --version. Optional but recommended: A virtual environment (venv or conda) so curriculum dependencies do not conflict with other projects. See Anaconda Setup for conda. Libraries used in the curriculum: NumPy, Matplotlib, and (for later volumes) PyTorch or TensorFlow, and Gym/Gymnasium. See Installing Libraries for how to install them. What you need For Volumes 1–2 (foundations, tabular methods): Python, NumPy, Matplotlib. You can implement gridworld, bandits, Monte Carlo, and TD in plain Python + NumPy; plotting helps for learning curves. For Volume 3+ (function approximation, deep RL): PyTorch or TensorFlow, and Gym or Gymnasium for environments (CartPole, MountainCar, etc.). Editor or IDE: Any text editor or IDE (VS Code, PyCharm, etc.). Jupyter is optional; see the FAQ on “Proof that using Jupyter Notebook is the same as not using it” (you can use scripts or notebooks—both are fine). After setup Once your environment is ready, take the Preliminary assessment to see if you are ready for the curriculum, or follow the Learning path from Phase 0 if you are new to programming.