Population Coding

Meaning lives in the pattern across many cells, not in any one of them.

The structure

Neurons in sensory and motor cortex are broadly tuned: a motor cortical cell fires for a wide range of movement directions with a preferred direction at its peak. Any given stimulus activates a large population to varying degrees, and any given cell participates in the representation of many stimuli.

The function it dictates

The represented quantity is recoverable from the population as a weighted vector sum — Georgopoulos showed in the 1980s that the direction of an arm movement can be read out of motor cortex this way, and modern brain-computer interfaces are built on exactly this readout. Distributed coding is robust to the loss of individual cells, supports graded similarity, and lets a fixed population represent far more distinct states than it has members.

The mirror in AI

This is the direct biological precedent for representing features as directions in activation space rather than as individual neurons. Broad tuning is polysemanticity; distributed representation over a fixed population is superposition. Neuroscience arrived at the conclusion decades earlier, and for the same empirical reason: recording single cells gave a confusing picture that only resolved when populations were considered together.

The mirror in robotics

Sensor fusion and probabilistic state estimation: no single sensor determines the state, and the estimate is a weighted combination whose reliability exceeds any individual input. A population vector readout and a Kalman filter are solving recognizably the same problem.

Where the analogy breaks

Biological population codes are shaped by an explicit metabolic budget and by physical wiring constraints — a neuron can only connect to cells its axon can reach. Artificial representations have neither constraint: any dimension can interact with any other at no cost. That freedom may be why artificial superposition is denser and less interpretable than the biological version.

Open questions

Does the brain use anything like superposition, or is its distributed code less densely packed?
Would wiring-cost constraints, imposed on an artificial network, produce more interpretable features?

In Kiru Lab

Vectors, Dot Products, and Alignment Embeddings and Position Features, Directions, and Superposition