Anatomy of a Transformer
Attention, the residual stream, and the block that repeats until you have a model.
In this module you will
- Implement scaled dot-product attention
- Explain the residual stream as a communication bus
- Describe what MLP layers contribute that attention does not
Self-Attention, Built From Scratch
Queries, keys, values — a differentiable lookup table. You already know every operation involved.
Hands-On Lab · 50m
The Residual Stream and the Transformer Block
The most useful mental model in interpretability: a shared bus that every layer reads from and writes to.
Concept · 35m