Kiru Lab / Foundations: Python as an Instrument / Python for Experiments
Seeing Your Data
A plot is a debugging tool, not a decoration. Look before you model.
Hands-On Lab · about 30 minutes
Most bad models are bad because nobody looked at the data. A histogram of your targets, a scatter of two features, and a plot of loss against step will catch more problems in five minutes than a week of hyperparameter search.
import matplotlib.pyplot as plt
import numpy as np
fig, axes = plt.subplots(1, 3, figsize=(12, 3.5))
axes[0].hist(y, bins=40) # target distribution — is it skewed? bimodal?
axes[0].set_title("targets")
axes[1].scatter(X[:, 0], y, s=4, alpha=0.4) # is there any signal at all?
axes[1].set_title("feature 0 vs target")
axes[2].imshow(np.corrcoef(X.T), cmap="gray") # are features duplicates of each other?
axes[2].set_title("feature correlation")
plt.tight_layout()What each plot catches
- A target histogram catches label leakage, clipped ranges, and class imbalance.
- A feature-target scatter catches the case where there is no relationship to learn.
- A correlation image catches redundant features that will make your optimization ill-conditioned — the same conditioning problem that produces singularities in the robotics track.
Later in this curriculum you will make the same kind of plot of a model's internals: an attention pattern is an image, an activation distribution is a histogram, and a probe's accuracy across layers is a line plot. The instinct to look is the transferable skill.
Hold on to
- Look at the data before choosing a model
- Correlated features cause the same conditioning problems as robot singularities
- Interpretability plots are the same three plots aimed at a model's internals
Work through
Try each one before opening the solution. Getting it wrong first is most of where the learning happens.
-
Take any dataset, make the three plots above, and write three sentences about what you learned before fitting anything.
Hint
Write the three sentences before you fit anything — that is the exercise.
Solution
There is no single right answer, but a good response names something specific: a skewed or bimodal target, a feature with no visible relationship, a pair of near-duplicate features, or a suspicious spike at a round number that suggests imputed values. If you could not write three sentences, you did not look long enough.
import matplotlib.pyplot as plt import numpy as np fig, axes = plt.subplots(1, 3, figsize=(12, 3.5)) axes[0].hist(y, bins=40); axes[0].set_title("targets") axes[1].scatter(X[:, 0], y, s=4, alpha=0.4); axes[1].set_title("feature 0 vs target") axes[2].imshow(np.corrcoef(X.T), cmap="gray"); axes[2].set_title("feature correlation") plt.tight_layout(); plt.show() -
Find a dataset where the target histogram alone tells you the metric you planned to use is a bad idea.
Hint
Think about what a heavily skewed or long-tailed target does to a mean-based metric.
Solution
House prices or income are the standard examples: the distribution has a long right tail, so mean squared error is dominated by a handful of expensive outliers and the model optimizes for them at everyone else's expense. The histogram makes this obvious in one glance, and the fix — predicting log-price, or using mean absolute error — is chosen before any training happens.
Sign in to track your progress through the lab.