RMSE vs. MAE: Comparing Regression Error Metrics

What you will learn in this lesson:

• What RMSE and MAE each measure, and how they differ mathematically
• Why RMSE penalizes large errors more heavily than MAE
• A worked numeric example computing both metrics by hand
• How outliers distort RMSE far more than MAE, demonstrated in code
• Practical guidance on when to prefer each metric, and why reporting both is good habit

Two Ways to Summarize Prediction Error

Once you've trained a regression model, you need a single number that summarizes how good its predictions are across an entire dataset. You can't just eyeball a list of a thousand individual errors, you need a summary statistic. The two most common choices are Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE).

Both start from the same basic ingredient, the residual for each prediction (actual value minus predicted value), but they combine those residuals into a summary number in meaningfully different ways, and that difference has real practical consequences for which model you'd choose to ship.

MAE takes the absolute value of every residual (so positive and negative errors don't cancel out when summed) and averages them:

MAE = average of |actual - predicted|

RMSE instead squares every residual, averages the squared values (this intermediate quantity is called the Mean Squared Error, or MSE), and then takes the square root of that average to bring the units back to the original scale so the result is comparable to MAE and to the target variable itself:

RMSE = sqrt( average of (actual - predicted)^2 )

Both metrics are zero only when every single prediction is exactly correct, and both grow as predictions get worse. The question is never "which one is right," it's "which one is measuring the kind of wrongness you actually care about."

Why Squaring Changes the Story

The squaring step in RMSE is the entire difference between the two metrics, and it's worth sitting with exactly what that step does. Under MAE, an error of 20 contributes exactly twice as much to the total as an error of 10, errors scale linearly.

Under RMSE, because the errors are squared before averaging, an error of 20 contributes four times as much as an error of 10 (400 versus 100), and an error of 40 contributes sixteen times as much. This means RMSE is disproportionately sensitive to a small number of large mistakes: a single very bad prediction can noticeably inflate RMSE even if every other prediction in the dataset was excellent.

MAE, by contrast, treats every unit of error equally regardless of how large the individual mistake was, which makes it more robust to outliers. If your dataset has a handful of genuinely unusual cases that any reasonable model would struggle with, MAE won't let those cases dominate the overall score the way RMSE would.

This isn't just a mathematical curiosity, it directly shapes what kind of mistakes a model trained to minimize each metric will tend to make. A model trained against an RMSE-style loss will work harder to avoid any single catastrophic miss, even if that means being slightly less accurate on the bulk of "easy" cases. A model trained against MAE-style loss treats all cases more even-handedly.

There's also a geometric way to see this. RMSE is closely related to the Euclidean (straight-line) distance between your vector of predictions and your vector of actual values; MAE is related to the Manhattan (taxicab) distance between the same two vectors. Squared distances grow faster than linear distances as any individual coordinate gets further from its target, which is exactly the outlier sensitivity described above, just stated geometrically instead of arithmetically.

A Worked Numeric Example

Consider predictions for five houses (prices in thousands of dollars):

Summing the absolute errors: 5 + 10 + 5 + 10 + 60 = 90. Dividing by 5 gives MAE = 18.

Summing the squared errors: 25 + 100 + 25 + 100 + 3600 = 3850. Dividing by 5 gives an MSE of 770. Taking the square root gives RMSE = 27.75 (in thousands of dollars).

Notice the gap: MAE says the typical error is about $18,000, while RMSE says about $27,750, over 50% larger. The reason for this gap is entirely House 5, whose $60,000 error is far larger than the others and gets squared up to 3,600, more than 14 times larger than the next-biggest squared error of 100, before it even gets averaged.

Whenever RMSE is substantially larger than MAE on the same dataset, that gap itself is a useful diagnostic signal: it tells you a small number of unusually large errors are present and worth investigating individually, rather than assuming the model is uniformly mediocre across all cases.

If House 5's error had instead been a more typical 8 (matching the other houses' scale), MAE would barely move, but RMSE would drop sharply, from 27.75 down to roughly 7.7. That sensitivity is precisely the property that makes RMSE useful in some contexts and misleading in others, depending on what you actually want your metric to reflect.

Seeing Outlier Sensitivity in Code

The cleanest way to build intuition for this difference is to watch what happens to each metric as you deliberately inject a single large outlier into an otherwise well-behaved set of predictions:

import numpy as np
from sklearn.metrics import mean_absolute_error, mean_squared_error

# A baseline set of fairly small, well-behaved errors
actual = np.array([250, 300, 320, 280, 310, 295, 330, 275])
predicted_good = np.array([255, 290, 315, 270, 305, 300, 325, 280])

mae_good = mean_absolute_error(actual, predicted_good)
rmse_good = np.sqrt(mean_squared_error(actual, predicted_good))
print("No outlier  -> MAE: {:.2f}  RMSE: {:.2f}".format(mae_good, rmse_good))

# Now corrupt just ONE prediction with a large miss, leave the rest identical
predicted_outlier = predicted_good.copy()
predicted_outlier[-1] = predicted_outlier[-1] + 150  # one house is way off

mae_bad = mean_absolute_error(actual, predicted_outlier)
rmse_bad = np.sqrt(mean_squared_error(actual, predicted_outlier))
print("One outlier -> MAE: {:.2f}  RMSE: {:.2f}".format(mae_bad, rmse_bad))

print()
print("MAE increased by a factor of {:.2f}".format(mae_bad / mae_good))
print("RMSE increased by a factor of {:.2f}".format(rmse_bad / rmse_good))

Run this and you'll see RMSE jump by a noticeably larger factor than MAE after the single outlier is introduced, even though only one of the eight predictions changed. This is the same phenomenon as the house-price example above, just made deliberately controllable so you can see the effect in isolation. Try changing the size of the injected error (150) to something smaller or much larger and watch how the gap between the two metrics widens or narrows accordingly.

When to Prefer Each

Prefer RMSE when large errors are genuinely more costly than small ones in your application, and you want your evaluation metric (and ideally your training loss too) to reflect that asymmetry.

Forecasting electricity demand for a power grid is a good example: a small under-forecast is a minor inefficiency easily absorbed by reserve capacity, but a huge under-forecast could mean blackouts, so you want a metric that punishes the big misses disproportionately, steering your model away from ever being catastrophically wrong even if that costs a little accuracy on the routine cases.

Prefer MAE when you want a metric that's robust to occasional outlier cases and easy to explain to a non-technical audience: "on average, we're off by $18,000" is intuitive in a way that RMSE's squared-then-rooted calculation isn't.

MAE is also the better choice when your dataset is known to contain a few unusual edge cases, perhaps data entry errors or genuinely unpredictable one-off events, that you don't want dominating your sense of overall model quality. If your evaluation set has noisy labels or occasional bad data, MAE will give you a more honest read on "typical" performance.

In practice, it's good habit to report both side by side rather than picking one and discarding the other. Most ML libraries provide simple helpers for this, scikit-learn's mean_absolute_error and mean_squared_error (take the square root yourself, or pass squared=False depending on your version) compute each metric directly from arrays of actual and predicted values.

If the two numbers are close together, your errors are fairly uniform in size. If RMSE is much larger than MAE, you have a small number of large misses worth digging into individually, they might reveal a systematic blind spot in your model rather than random noise.