Kiru Lab  /  The Mathematics of Learning  /  The Calculus of Change

Derivatives and Gradients

A derivative answers one question: if I nudge this input, how much does the output move?

Concept  ·  about 30 minutes

A derivative is a rate: how much the output changes per unit change of the input, in the limit of a tiny nudge. For a function of many inputs, you get one derivative per input, and collecting them into a vector gives the gradient.

grad f(x) = [ df/dx_1, df/dx_2, ..., df/dx_n ]

The gradient points in the direction of steepest increase of `f`. Training a model means repeatedly stepping in the opposite direction — steepest decrease of the loss. That is the whole algorithm; everything else is engineering around it.

Numerical gradient — the definition, in code
import numpy as np

def numerical_gradient(f, x, eps=1e-6):
    grad = np.zeros_like(x, dtype=float)
    for i in range(x.size):
        step = np.zeros_like(x, dtype=float)
        step[i] = eps
        grad[i] = (f(x + step) - f(x - step)) / (2 * eps)
    return grad

f = lambda v: float(v @ v)          # f(x) = |x|^2, so grad f = 2x
x = np.array([1.0, 2.0, 3.0])
print(numerical_gradient(f, x))     # [2. 4. 6.]
Keep this function

Numerical gradients are too slow to train with, but they are the standard way to check that a hand-written analytic gradient is correct. You will use this exact helper when you write backpropagation from scratch in the deep learning track.

From gradient to Jacobian

A gradient is what you get when the output is a single number. When the output is itself a vector — as it is for a robot arm whose end-effector has a 3D position, or for a network layer producing many activations — you get one gradient per output component. Stack them as rows and you have the Jacobian matrix.

J[i][j] = d(output_i) / d(input_j) f: R^n -> R^1 => J is 1 x n (a gradient) f: R^n -> R^m => J is m x n (a Jacobian)

Remember this shape. In track six the inputs will be joint angles and the outputs will be the position of a gripper in space, and inverting that matrix is how a robot decides how to move.

Hold on to

  • The gradient points uphill; training walks the other way
  • Numerical gradients are the ground truth for checking analytic ones
  • The Jacobian is the gradient generalized to vector-valued outputs

Work through

Try each one before opening the solution. Getting it wrong first is most of where the learning happens.

  1. Verify the analytic gradient of a small quadratic against `numerical_gradient` to five decimal places.
    Hint

    For f(x) = x^T A x with symmetric A, the gradient is 2Ax.

    Solution

    Central differences agree with the analytic gradient to about 1e-9 with eps=1e-6. Do not push eps much smaller — below roughly 1e-8 floating-point cancellation in the subtraction makes the estimate worse, not better. That is a U-shaped error curve, and knowing it exists saves you from a confusing afternoon.

    import numpy as np

A = np.array([[3.0, 1.0], [1.0, 2.0]])      # symmetric
f = lambda x: float(x @ A @ x)
analytic = lambda x: 2 * A @ x

x = np.array([1.0, -2.0])
assert np.abs(numerical_gradient(f, x) - analytic(x)).max() < 1e-5
    Check your work

    Paste this after your own code. If it runs without raising, you have it. 

    import numpy as np
x = np.array([1.0, -2.0])
assert np.abs(numerical_gradient(f, x) - analytic(x)).max() < 1e-5
print("ok")
  2. Write a function from R^3 to R^2 and compute its 2x3 Jacobian numerically. Confirm the shape matches the rule above.
    Hint

    Perturb one input at a time and record how both outputs move.

    Solution

    The Jacobian is 2x3: one row per output component, one column per input. Confirm against the analytic form. Note the shape rule holding — f: R^3 -> R^2 gives a (2, 3) matrix — because in track six the inputs will be joint angles and the outputs a gripper position, and the shape is how you will sanity-check that code.

    import numpy as np

def f(v):
    x, y, z = v
    return np.array([x * y, y + z ** 2])

def jacobian(f, v, eps=1e-6):
    out = f(v)
    J = np.zeros((out.size, v.size))
    for j in range(v.size):
        step = np.zeros_like(v); step[j] = eps
        J[:, j] = (f(v + step) - f(v - step)) / (2 * eps)
    return J

v = np.array([2.0, 3.0, 4.0])
# analytic: [[y, x, 0], [0, 1, 2z]] = [[3, 2, 0], [0, 1, 8]]
    Check your work

    Paste this after your own code. If it runs without raising, you have it. 

    import numpy as np
v = np.array([2.0, 3.0, 4.0])
J = jacobian(f, v)
assert J.shape == (2, 3)
assert np.allclose(J, [[3, 2, 0], [0, 1, 8]], atol=1e-5)
print("ok")

In the Bio Mirror

The Cerebellum and Internal Models

Sign in to track your progress through the lab.

← The Calculus of Change The Chain Rule as Plumbing →