The Visual Hierarchy

Edges to textures to objects, across a chain of areas — the closest thing to a confirmed prediction in this field.

The structure

Visual information passes from retina through the lateral geniculate nucleus to V1, then through V2, V4, and into inferotemporal cortex. Receptive fields grow at each stage, from a fraction of a degree in V1 to most of the visual field in IT. Hubel and Wiesel showed in the late 1950s that V1 cells respond to oriented edges at specific positions, and that this tuning develops through early visual experience.

The function it dictates

Each stage builds more abstract and more invariant representations from the previous one. Early stages encode local oriented structure; middle stages encode texture and contour combinations; late stages respond to objects and faces largely independent of position, size, and lighting. Invariance is built gradually rather than imposed at once, which is what makes the hierarchy necessary rather than merely convenient.

The mirror in AI

Convolutional networks reproduce this progression without being told to. First-layer filters in a trained CNN are oriented edge and color-opponent detectors closely resembling V1 measurements; deeper layers become texture-selective and then object-selective. Trained CNN activations also predict measured neural responses in the corresponding visual areas better than any hand-designed model. Two systems built by entirely different processes converged on the same decomposition, which argues the hierarchy is a property of the problem.

The mirror in robotics

A perception stack layered from features to objects to scene graph to affordances. The engineering motivation is identical — invariance built in stages, so each stage solves a tractable piece — and modern stacks have largely replaced hand-designed early stages with learned ones, recapitulating the biological arrangement.

Where the analogy breaks

Biological vision is active and recurrent: eyes saccade several times per second, and each area receives more feedback than feedforward connection. A feedforward CNN processing a static image is doing something structurally different from a system that samples the world and predicts what it will see next. Adversarial examples are the sharpest evidence of the divergence: imperceptible perturbations that destroy CNN classification leave human perception untouched, which means the two systems are not using the same features however similar their early layers look.

Open questions

Why are CNNs vulnerable to adversarial perturbations that humans cannot even perceive?
How much of visual recognition depends on recurrence rather than the feedforward sweep?

In Kiru Lab

Convolution: Locality Made Structural