Chain Rule

This page covers the calculus you need for RL: derivatives, the chain rule, and partial derivatives. Back to Math for RL. Core concepts Derivatives The derivative of \(f(x)\) with respect to \(x\) is \(f’(x)\) or \(\frac{df}{dx}\). It gives the rate of change (slope) of \(f\) at \(x\). Rules you will use: \(\frac{d}{dx} x^n = n x^{n-1}\) \(\frac{d}{dx} e^x = e^x\) \(\frac{d}{dx} \ln x = \frac{1}{x}\) \(\frac{d}{dx} \ln(1 + e^x) = \frac{e^x}{1+e^x} = \sigma(x)\) (sigmoid) The chart below shows the sigmoid \(\sigma(x) = \frac{e^x}{1+e^x}\): the S-shaped function whose derivative we use in policy parameterizations and softplus. ...

This page covers the calculus you need for the preliminary assessment: derivatives of common functions, the chain rule, and how they appear in logistic regression and policy gradients. Back to Preliminary. Why this matters for RL Policy gradients and loss-based updates use derivatives and the chain rule. You don’t need to derive everything by hand in practice (autograd does it), but you need to understand what a gradient is and how it’s used. The sigmoid and chain rule appear in logistic policies and in backpropagation. ...