Deep Learning Pitfalls

What you will learn in this lesson:

• Why repeatedly tuning against the test set quietly leaks information into your decisions
• Why uncontrolled differences between training runs can masquerade as real improvements
• Why strong benchmark performance doesn't guarantee real-world performance
• A practical mindset for treating impressive-looking results with appropriate skepticism

Tuning Against the Test Set Quietly Breaks It

Machine Learning Fundamentals established the train/validation/test split for exactly this reason: the test set is supposed to give one unbiased, final estimate of how a model performs on data it has never influenced in any way. Every time you peek at test performance, adjust something (an architecture choice, a hyperparameter, even just deciding "let's try a different seed because that one looked bad"), and check the test set again, you've used the test set's feedback to inform a decision, which means it has started to function like a second training signal.

import numpy as np

np.random.seed(0)
# Simulate "trying many random configurations" and always reporting the best
# test-set result, the exact failure mode of tuning against the test set.
n_configs = 200
n_test_examples = 50

# Every config is equally bad in reality (50% accuracy expected), pure noise
fake_test_accuracies = np.random.binomial(n_test_examples, 0.5, n_configs) / n_test_examples

best_observed = fake_test_accuracies.max()
print(f"Best 'accuracy' found by trying {n_configs} random configs: {best_observed:.3f}")
print(f"True expected accuracy of every config: 0.500")
print("The gap above is pure noise from repeated peeking, not a real improvement.")

Every one of those 200 simulated configurations is, by construction, equally mediocre. Reporting only the best one observed across many tries produces a number that looks meaningfully better than chance, purely as an artifact of trying enough times and keeping the best result. This is exactly the mechanism by which repeated test-set evaluation inflates a reported metric: tune and validate against a separate validation set, and reserve the test set for one single, final check.

Uncontrolled Variance Between Training Runs

Training is inherently stochastic: random weight initialization, the order mini-batches are drawn in, dropout's randomness, all introduce run-to-run variation in the final result, even with identical hyperparameters. If you compare a "new architecture" against a "baseline" but also changed the random seed, batch size, or number of training epochs between the two runs, any observed difference could be coming from those uncontrolled factors rather than the architectural change you actually meant to test.

import numpy as np
import matplotlib.pyplot as plt

# Simulate the SAME architecture trained with 10 different random seeds.
# The spread alone shows how much "natural" variance exists before any
# real architectural change is even introduced.
np.random.seed(0)
same_architecture_results = 0.82 + np.random.normal(0, 0.02, 10)

plt.scatter(range(10), same_architecture_results)
plt.axhline(same_architecture_results.mean(), color="gray", linestyle="--", label="mean")
plt.ylabel("Validation accuracy"); plt.xlabel("Run (same architecture, different seeds)")
plt.legend(); plt.title("Run-to-run variance with no architecture change at all")
show_plot()
print(f"Spread across identical-architecture runs: {same_architecture_results.min():.3f} to {same_architecture_results.max():.3f}")
print("A 'new architecture' would need to beat this entire natural spread")
print("to be confidently attributed to the architecture itself, not luck.")

Before attributing a performance difference to whatever you changed, ask whether that difference is larger than the natural run-to-run variance you'd see from the unchanged baseline alone. Fair comparisons hold every other factor, seed, batch size, epoch count, fixed, varying only the one thing actually being tested, ideally averaged across multiple seeds on both sides.

Benchmark Performance Isn't Real-World Performance

A model can perform excellently on a benchmark dataset while performing poorly once deployed, even with no test-set leakage and no run-to-run variance issues at all. Benchmark datasets are collected in specific ways, at specific times, by specific processes, and a model can learn shortcuts or biases specific to that collection process rather than the underlying task itself. A medical imaging benchmark sourced entirely from one hospital's specific scanner model can produce a classifier that's secretly learned to recognize that scanner's particular image artifacts rather than the actual pathology, a pattern that won't transfer to a different hospital's equipment.

A Practical Mindset

None of these pitfalls have a single mechanical fix; they call for a habit of skepticism toward any impressive-looking number. Before trusting a result: confirm the test set was genuinely held out and checked only once, confirm the comparison controlled for seeds and other confounds, and ask what specific biases or collection quirks the benchmark itself might carry that wouldn't hold in deployment.