Kiru Lab  /  Embodied Control: Kinematics and the Inverse Jacobian  /  Inverse Kinematics

Learned Controllers and the Gap to Reality

When you learn the inverse map with a network instead of solving it, you inherit every failure mode from earlier tracks.

Bridge  ·  about 30 minutes

You can train a network to map desired pose directly to joint angles, skipping the iteration. It is fast at inference and it handles chains too complicated to model analytically. It also inherits every property you studied in the machine learning tracks — and one new hazard specific to this problem.

The averaging trap

Inverse kinematics is one-to-many: a single target has an elbow-up and an elbow-down solution. A network trained with squared error on both will learn their average, which is generally not a valid solution at all. This is not a training bug; it is what minimizing squared error over a multimodal target does. The fixes — mixture density outputs, conditioning on a mode, or learning the residual on top of an analytic solver — all amount to refusing to let the model average.

  • Distribution shift: a controller trained in simulation meets friction, backlash, and flex it never saw. This is the sim-to-real gap, and it is out-of-distribution generalization with physical consequences.
  • No guarantees: an analytic solver reports failure to converge. A network always outputs something, confidently — the same softmax-style problem as confabulation, now attached to an actuator.
  • Safety envelopes: because of the above, learned controllers in production are wrapped in analytically verified limits. The learned part proposes; the classical part disposes.
  • Hybrid designs win in practice: learn the residual correction on top of a model-based solver, so the worst case degrades to the classical solution rather than to nothing.

This is the point where the three sections of Kiru meet. A robot with a learned controller is a system whose failures are behavioral, reproducible, and worth cataloging — which is precisely the DEM-X premise, now with a body attached. The interpretability track gives you the tools to ask why the controller did what it did.

Hold on to

  • A network trained on a one-to-many map learns invalid averages
  • Learned controllers cannot report failure; classical solvers can
  • Hybrid architectures degrade to the classical solution rather than to nothing

Work through

Try each one before opening the solution. Getting it wrong first is most of where the learning happens.

  1. Train a network on two-link IK including both elbow configurations. Show the averaged output is unreachable, then fix it with a mixture density head.
    Hint

    Generate training data containing both elbow-up and elbow-down solutions for each target.

    Solution

    The MSE-trained network outputs roughly the average of the two valid configurations, and running forward kinematics on that average lands nowhere near the target — often outside the reachable set entirely. A mixture density head fixes it by predicting several components with weights, so the model can say "one of these two" instead of splitting the difference. Nothing about the training was buggy; squared error on a bimodal target does exactly this, by definition.

    import numpy as np

# For a two-link arm, both configurations reach the same point:
def both_solutions(target, lengths):
    x, y = target
    l1, l2 = lengths
    c2 = (x**2 + y**2 - l1**2 - l2**2) / (2 * l1 * l2)
    c2 = np.clip(c2, -1, 1)
    out = []
    for sign in (+1, -1):
        q2 = sign * np.arccos(c2)
        q1 = np.arctan2(y, x) - np.arctan2(l2 * np.sin(q2), l1 + l2 * np.cos(q2))
        out.append(np.array([q1, q2]))
    return out

lengths = np.array([1.0, 1.0])
up, down = both_solutions(np.array([1.0, 0.8]), lengths)
averaged = (up + down) / 2
print("target:   ", np.array([1.0, 0.8]))
print("averaged:", forward_kinematics(averaged, lengths))   # not the target
    Check your work

    Paste this after your own code. If it runs without raising, you have it. 

    import numpy as np
target = np.array([1.0, 0.8])
up, down = both_solutions(target, lengths)
assert np.allclose(forward_kinematics(up, lengths), target, atol=1e-6)
assert np.allclose(forward_kinematics(down, lengths), target, atol=1e-6)
assert not np.allclose(forward_kinematics((up + down) / 2, lengths), target, atol=1e-2)
print("ok")
  2. Add sensor noise and joint backlash to your simulator and measure how much accuracy the analytic and learned solvers each lose.
    Hint

    Add Gaussian noise to the measured angles and a dead zone around each commanded change.

    Solution

    The analytic solver degrades gracefully and predictably — error scales roughly with the noise, and it still reports non-convergence when it fails. The learned controller degrades faster once the perturbations push inputs outside its training distribution, and critically it keeps returning confident outputs the whole way down. That asymmetry — graceful and honest versus fast and silent — is the entire argument for wrapping learned controllers in analytic safety envelopes.

    import numpy as np

def with_backlash(q, previous, deadband=0.01):
    delta = q - previous
    delta = np.where(np.abs(delta) < deadband, 0.0, delta)
    return previous + delta

rng = np.random.default_rng(0)
for noise in (0.0, 0.01, 0.05):
    errors = []
    for _ in range(100):
        target = sample_reachable(rng, lengths)
        q, _, converged = inverse_kinematics(target, [0.3, 0.4], lengths)
        q_noisy = q + rng.normal(scale=noise, size=q.shape)
        errors.append(np.linalg.norm(forward_kinematics(q_noisy, lengths) - target))
    print(f"noise={noise}: mean error {np.mean(errors):.4f}")

In the Bio Mirror

The Cerebellum and Internal Models Proprioception: The Sense That Makes Control Possible

Related DEM-X entries

INF-HALL-01

Sign in to track your progress through the lab.

← Redundancy and the Null Space Back to track →