Dl-Foundations

How the biological neuron — dendrites, soma, axon — maps onto the artificial neuron with inputs, weights, bias, and activation.

The perceptron learning rule, training on AND and OR gates, and why XOR exposes the fundamental limitation of single-layer networks.

ReLU, sigmoid, tanh, and softmax — what they compute, when to use each, and why non-linearity is essential for deep networks.

MLP architecture, parameter counting, and how stacking non-linear layers allows networks to solve XOR and approximate any function.

Layer-by-layer forward pass through an MLP — computing pre-activations, applying activations, and understanding intermediate representations.

5 questions after completing the first 6 DL Foundations pages. Check your understanding before continuing.

MSE for regression, cross-entropy for classification, and the TD error loss in DQN — how loss functions guide neural network training.

The chain rule applied backwards through a neural network — computing gradients for every weight and verifying them with numerical finite differences.

12 questions covering neural networks, backpropagation, training loops, and CNNs. Pass: 9/12.

Understand SGD, Momentum, and Adam optimizers from scratch. Implement and compare them in NumPy.

Build a full training loop in NumPy: batches, epochs, forward pass, backprop, and weight updates.

Understand overfitting and apply L2 regularization and dropout to prevent it in NumPy.

Learn convolution and pooling from scratch in NumPy. See how Atari DQN uses CNNs to process raw pixels.

Bridge NumPy implementations to PyTorch. Build QNetwork and PolicyNetwork with nn.Module for RL.

Build a 2-layer MLP to classify handwritten digits using only NumPy. Full pipeline: data, init, training, evaluation.

15 drill problems covering neural networks, forward pass, backpropagation, optimizers, and training.

Review deep learning and see why RL needs neural networks — the bridge to DQN and policy gradients.