Kiru Lab  /  Mechanistic Interpretability  /  Features and Representation

Circuits: Components That Compose

A circuit is a set of components whose composition implements an identifiable algorithm.

Concept  ·  about 35 minutes

A circuit is a subgraph of the model — particular attention heads and MLP neurons, in particular layers — that together implement a describable computation. The residual stream from track five is what makes this coherent: components communicate by writing to and reading from shared directions, so a circuit is a chain of writes and reads across depth.

Induction heads, the best-understood example

Induction heads implement a two-step algorithm across two layers. A previous-token head in an earlier layer writes, at each position, information about the token that preceded it. An induction head in a later layer then searches for an earlier occurrence of the current token and attends to whatever followed it, copying that token forward. Together they implement "if the sequence [A][B] appeared earlier and we just saw [A], predict [B]".

  • This is a genuine algorithm, recovered from weights, not a story fitted to outputs.
  • It appears abruptly during training, at the same point as a visible bend in the loss curve — structure forming and capability appearing together.
  • It accounts for a substantial share of in-context learning ability, which is why it is the canonical result of the field.

The methodological lesson matters more than the specific circuit. The claim was not "this head looks like it does induction". It was: predict what breaks if this head is disabled, disable it, and check. That is the standard the next module builds.

Why full reverse-engineering is hard

Superposition means circuits are entangled rather than modular; a component participates in many circuits at once. Recovering one clean algorithm is possible, has been done, and is slow. Do not mistake a handful of well-understood circuits for a solved problem.

Hold on to

  • A circuit is components composing through the residual stream
  • Induction heads implement a real, verified two-layer algorithm
  • Entanglement from superposition is the main obstacle to scaling this work

Work through

Try each one before opening the solution. Getting it wrong first is most of where the learning happens.

  1. Reproduce induction behavior: feed a random repeated token sequence and measure per-position loss. Find the drop at the repeat.
    Hint

    Build a sequence of random tokens, then concatenate it with itself.

    Solution

    Per-token loss is high through the first copy — the tokens are random and unpredictable — then drops sharply at the start of the second copy and stays low. The model is not recalling anything from training; it discovered the pattern within the prompt. This is in-context learning made measurable in about ten lines, and the drop is the induction mechanism turning on.

    import torch

rng = torch.Generator().manual_seed(0)
seq = torch.randint(1000, 5000, (1, 50), generator=rng)
repeated = torch.cat([seq, seq], dim=1)

with torch.no_grad():
    logits = model(repeated)

log_probs = torch.log_softmax(logits[0, :-1], dim=-1)
targets = repeated[0, 1:]
per_token = -log_probs[range(len(targets)), targets]

print("first copy mean loss: ", per_token[:49].mean().item())
print("second copy mean loss:", per_token[50:].mean().item())   # much lower
  2. Identify a candidate induction head by its attention pattern on the repeated sequence, then predict what its ablation will do before running it.
    Hint

    An induction head attends from the current token to the position just after its earlier occurrence.

    Solution

    Score each head by how much attention mass it places on the "offset by one from the earlier match" position; induction heads stand out sharply. Write your prediction down before ablating — something like "second-copy loss will rise back toward first-copy levels" — because a prediction recorded in advance is the only kind that can be wrong, and a hypothesis that cannot be wrong is not a finding.

    import torch

def induction_score(attn_pattern, seq_len):
    """attn_pattern: (heads, 2*seq_len, 2*seq_len) for a repeated sequence."""
    scores = []
    for head in range(attn_pattern.shape[0]):
        diagonal = [attn_pattern[head, seq_len + i, i + 1]
                    for i in range(seq_len - 1)]
        scores.append(float(torch.stack(diagonal).mean()))
    return scores

Sign in to track your progress through the lab.

← Features, Directions, and Superposition Back to track →