Kiru Lab / Language Models / Text Becomes Numbers
Embeddings and Position
A token id becomes a learned vector, and position gets injected because attention alone is order-blind.
Concept · about 30 minutes
The embedding matrix is a lookup table with one learned vector per vocabulary entry. Training arranges this space so that directions carry meaning — the geometry is learned, not designed. This is the vector space from track two, populated by gradient descent.
Position must be added deliberately
Self-attention is permutation-invariant: shuffle the input positions and, without positional information, the outputs shuffle identically. Nothing in the mechanism knows about order. So position is injected explicitly — early models added sinusoidal or learned position vectors; most current models use rotary embeddings, which rotate query and key vectors by an angle proportional to position so that attention scores depend on relative distance.
A model's behavior beyond the positions it was trained on is extrapolation, and it degrades. This is the mechanical basis of long-context degradation and of the "lost in the middle" effect where information in the center of a long prompt is used less reliably than information at either end.
The output embedding is often tied to the input embedding — the same matrix, transposed, converts the final vector back into logits over the vocabulary. That symmetry means the model reads and writes in the same coordinate system, which is what makes the residual stream in the next lesson coherent as a shared channel.
Hold on to
- Embedding geometry is learned, and directions carry meaning
- Attention is order-blind; position is injected on purpose
- Long-context degradation is positional extrapolation
Work through
Try each one before opening the solution. Getting it wrong first is most of where the learning happens.
-
Extract embeddings for 200 words and cluster them. Name three clusters and one that resists naming.
Hint
Use k-means on the embedding matrix rows, then print the words nearest each centroid.
Solution
Clean clusters usually appear for numbers, proper names, function words, and domain vocabulary. The cluster that resists naming is the interesting result: embeddings encode distributional similarity — which contexts a word appears in — not semantic category, so you routinely get groups united by syntactic role or corpus artifact rather than by meaning. Note that resistance rather than forcing a label onto it.
import numpy as np from sklearn.cluster import KMeans E = embedding_matrix[selected_ids] # (200, d) labels = KMeans(n_clusters=8, n_init=10, random_state=0).fit_predict(E) for c in range(8): members = [words[i] for i in np.where(labels == c)[0]] print(c, members[:12]) -
Test the same factual question with the key detail at the start, middle, and end of a long prompt. Report accuracy at each position.
Hint
Keep the prompt and question identical; move only the position of the supporting fact.
Solution
Accuracy is typically highest when the fact sits at the very start or very end and measurably lower in the middle — the "lost in the middle" effect. The mechanism is positional: attention over long contexts is not uniform, and positions far beyond the training distribution are extrapolation. Practically this means retrieval systems should place the most important context at the edges, not buried.
filler = "Irrelevant background sentence. " * 200 fact = "The internal code name for the project is Kiru. " question = "\n\nWhat is the internal code name for the project?" for label, prompt in ( ("start", fact + filler + question), ("middle", filler[:len(filler)//2] + fact + filler[len(filler)//2:] + question), ("end", filler + fact + question), ): correct = sum(ask(prompt) == "Kiru" for _ in range(20)) print(f"{label:6} {correct}/20")
In the Bio Mirror
Population CodingSign in to track your progress through the lab.