Metrics

Useful for logging training metrics (rewards per episode, loss curves), loading small datasets, and analyzing results. Many curriculum exercises ask you to “plot the sum of rewards per episode”—storing those in a DataFrame keeps things tidy and easy to export. Why Pandas matters for RL DataFrame — Tabular data: one row per episode or per step, columns like episode, reward, length, loss. Easy to filter, aggregate, and plot. Series — 1D data (e.g. reward per episode). Rolling mean, describe(), and plot. I/O — to_csv, read_csv for saving/loading runs and sharing results. Grouping and aggregation — Mean reward per run, per algorithm, or per seed. Core concepts with examples Building a DataFrame from lists 1 2 3 4 5 6 7 8 9 10 11 import pandas as pd episodes = list(range(10)) rewards = [1, 2, 3, 2, 4, 3, 5, 4, 6, 5] lengths = [10, 12, 15, 11, 20, 14, 22, 18, 25, 21] df = pd.DataFrame({ "episode": episodes, "reward": rewards, "length": lengths }) Basic columns and selection 1 2 3 4 5 6 7 8 # Add a column (e.g. running mean) df["reward_smooth"] = df["reward"].rolling(window=3, min_periods=1).mean() # Select rows where reward > 4 df[df["reward"] > 4] # Select columns df[["episode", "reward"]] Describe and summary stats 1 2 3 df["reward"].mean() df["reward"].std() df.describe() # count, mean, std, min, 25%, 50%, 75%, max per column Save and load 1 2 df.to_csv("rewards.csv", index=False) df_loaded = pd.read_csv("rewards.csv") Plotting from a DataFrame 1 2 3 4 5 import matplotlib.pyplot as plt df.plot(x="episode", y="reward", label="reward") df.plot(x="episode", y="reward_smooth", label="smoothed", ax=plt.gca()) plt.xlabel("Episode") plt.show() Exercises Exercise 1. Create a DataFrame with columns episode (0 to 99) and reward, where reward is 10 + episode * 0.1 + noise with noise from np.random.randn(100). Add a column reward_ma5 that is the 5-step moving average of reward. Use rolling(5, min_periods=1).mean(). ...