Kiru Lab  /  Deep Learning

Training Dynamics and Failure Modes

Optimizers, normalization, regularization, and how to read a run that is going wrong.

In this module you will

  • Explain what Adam maintains beyond the gradient
  • Describe what normalization stabilizes and why
  • Name a training failure from its curves and statistics