Kiru Lab  /  Foundations: Python as an Instrument  /  Python From First Principles

Functions: Naming a Piece of Behavior

The discipline of writing a function that does exactly one thing — and the training loop that results.

Code Walkthrough  ·  about 35 minutes

A function gives a name to a block of behavior so it can be reused, tested, and reasoned about in isolation. You now have branches and loops; functions are what keep a program made of them from becoming one long unreadable script.

A training loop is just control flow
def train(model, data, epochs: int, lr: float):
    history = []
    for epoch in range(epochs):
        total_loss = 0.0
        for x, y in data:
            prediction = model(x)
            loss = (prediction - y) ** 2
            model.step(x, prediction - y, lr)
            total_loss += loss
        history.append(total_loss / len(data))
        if epoch > 0 and history[-1] > history[-2]:
            print(f"epoch {epoch}: loss went up — check the learning rate")
    return history

That is the whole shape of deep learning training. Every framework you will use later — PyTorch, JAX, whatever replaces them — is an elaboration of this loop with better derivatives and faster hardware underneath.

Functions that fit in your head

  • One job per function. If the name needs an "and", split it.
  • Take what you need as arguments; do not reach for globals.
  • Return a value rather than printing one — printing is a decision the caller should make.
  • Fail loudly and early on bad input instead of returning something plausible and wrong.
Guard clauses beat nested ifs
def normalize(vector: list[float]) -> list[float]:
    if not vector:
        raise ValueError("cannot normalize an empty vector")
    magnitude = sum(v * v for v in vector) ** 0.5
    if magnitude == 0:
        raise ValueError("cannot normalize the zero vector")
    return [v / magnitude for v in vector]
Silent wrongness

A model that returns a confident answer to a question it cannot answer is doing the same thing as a function that returns 0.0 instead of raising on an empty input. Hold that thought until the LLM track — it is the mechanical root of confabulation.

Hold on to

  • A training loop is ordinary control flow, not magic
  • Guard clauses keep the happy path flat and readable
  • Returning a plausible wrong value is worse than raising

Work through

Try each one before opening the solution. Getting it wrong first is most of where the learning happens.

  1. Rewrite a deeply nested validation function using guard clauses. Count the lines before and after.
    Hint

    Return early on the invalid cases, then write the main logic unindented.

    Solution

    The guard-clause version handles each invalid case in two lines at the top and leaves the real work at the bottom with no indentation. It is usually a third shorter and reads as a list of preconditions followed by the actual behavior.

    # Nested version
def process(user):
    if user is not None:
        if user.is_active:
            if user.email:
                return send(user.email)
            else:
                raise ValueError("no email")
        else:
            raise ValueError("inactive")
    else:
        raise ValueError("no user")

# Guard clauses
def process(user):
    if user is None:
        raise ValueError("no user")
    if not user.is_active:
        raise ValueError("inactive")
    if not user.email:
        raise ValueError("no email")
    return send(user.email)
  2. Write `train()` above against a toy linear model of your own and plot the loss history. Deliberately set the learning rate too high and describe what the history looks like.
    Hint

    A "model" here can be as simple as a single weight and a step method.

    Solution

    With too high a learning rate the loss history rises rather than falls, often to infinity or NaN within a few epochs — the step overshoots the minimum and lands further away each time, so the overshoot compounds. You will see the identical signature on a transformer in track four.

    class ToyModel:
    def __init__(self):
        self.w = 0.0

    def __call__(self, x):
        return self.w * x

    def step(self, x, error, lr):
        self.w -= lr * error * x

data = [(1.0, 2.0), (2.0, 4.0), (3.0, 6.0)]   # true w = 2
history = train(ToyModel(), data, epochs=20, lr=0.05)   # converges
history = train(ToyModel(), data, epochs=20, lr=1.0)    # diverges

Sign in to track your progress through the lab.

← Repeating Work The Four Data Structures That Matter →