Convolution: Locality Made Structural

Hold on to

• Convolution encodes locality and translation invariance structurally
• Weight sharing is a prior about the world, not a memory optimization
• Edge-to-texture-to-object hierarchies emerge in both CNNs and visual cortex

Try each one before opening the solution. Getting it wrong first is most of where the learning happens.

1. Implement 2D convolution with explicit loops, then verify against a library version.
   Hint

   Slide the kernel over every valid position and take an elementwise product-sum.

   Solution

   The explicit version makes the two structural constraints visible: the inner loop only ever touches a small neighborhood (locality), and the same `kernel` array is used at every position (weight sharing). Note the output is smaller than the input unless you pad — that shrinkage is why deep convolutional stacks need padding to preserve spatial dimensions.

   import numpy as np
   from scipy.signal import convolve2d
   
   def conv2d(image, kernel):
       kh, kw = kernel.shape
       h, w = image.shape[0] - kh + 1, image.shape[1] - kw + 1
       out = np.zeros((h, w))
       for i in range(h):
           for j in range(w):
               out[i, j] = np.sum(image[i:i + kh, j:j + kw] * kernel)
       return out
   
   rng = np.random.default_rng(0)
   image, kernel = rng.normal(size=(20, 20)), rng.normal(size=(3, 3))
   reference = convolve2d(image, kernel[::-1, ::-1], mode="valid")  # note the flip
   assert np.allclose(conv2d(image, kernel), reference)

   Check your work

   Paste this after your own code. If it runs without raising, you have it.

   import numpy as np
   image = np.arange(16, dtype=float).reshape(4, 4)
   kernel = np.ones((2, 2))
   out = conv2d(image, kernel)
   assert out.shape == (3, 3)
   assert out[0, 0] == 0 + 1 + 4 + 5
   print("ok")

2. Visualize the first-layer filters of a pretrained CNN. Count how many are recognizable edge detectors.
   Hint

   Reshape the first conv layer weights to (out_channels, 3, kh, kw) and show them as RGB images.

   Solution

   In a trained network typically half to two-thirds of first-layer filters are recognizable oriented edge detectors — bright on one side, dark on the other, at various angles — and most of the rest are color-opponent blobs. This closely matches what Hubel and Wiesel measured in cat V1 in 1959, from a system that learned it from ImageNet rather than from evolution. Two very different processes, the same solution.

   import matplotlib.pyplot as plt
   import torchvision
   
   model = torchvision.models.resnet18(weights="DEFAULT")
   filters = model.conv1.weight.detach().numpy()      # (64, 3, 7, 7)
   
   fig, axes = plt.subplots(8, 8, figsize=(8, 8))
   for f, ax in zip(filters, axes.ravel()):
       f = (f - f.min()) / (f.max() - f.min())
       ax.imshow(f.transpose(1, 2, 0)); ax.axis("off")
   plt.tight_layout(); plt.show()

Sign in to track your progress through the lab.