Architectures as Priors
Convolution assumes locality; recurrence assumes order. Architecture is an assumption made structural.
In this module you will
- State the assumption each architecture family encodes
- Explain weight sharing as a prior, not an optimization
- Describe why recurrence struggles with long dependencies
Convolution: Locality Made Structural
A convolutional layer is a fully connected layer with most weights forced to zero and the rest tied together.
Concept · 30m
Sequences, Memory, and the Bottleneck That Broke Recurrence
RNNs assume order matters and that a fixed-size state can carry the past. The second assumption is where they broke.
Concept · 30m