Kiru Lab / Track
Deep Learning
Depth buys composition. Composition buys abstraction. Abstraction costs legibility.
A deep network is a stack of the linear-then-nonlinear layers you already built, trained by the chain rule you already derived. What changes with depth is that later layers get to compose the features earlier layers discovered, producing a hierarchy of abstraction nobody designed by hand. That is the power, and it is also why interpretability became necessary: the representations are learned, so no one wrote down what they mean.
What does depth give you that width does not?
By the end you can
- Implement a forward and backward pass without a framework
- Choose activations, initializations, and normalizations for reasons
- Explain what convolution and recurrence each assume about their input
- Diagnose a training run from its curves and gradient statistics
Module 1
From Neuron to Network
The artificial neuron, the nonlinearity that makes depth worthwhile, and backprop written by hand.
Module 2
Architectures as Priors
Convolution assumes locality; recurrence assumes order. Architecture is an assumption made structural.
Module 3
Training Dynamics and Failure Modes
Optimizers, normalization, regularization, and how to read a run that is going wrong.