Mean squared error
Mean squared error, or MSE, is a regression metric that averages the squared differences between predicted and actual values. By squaring each error before...
Penalizing Large Errors More Heavily
Mean squared error, or MSE, is a regression metric that averages the squared differences between predicted and actual values. By squaring each error before averaging, MSE gives disproportionately high weight to large errors. A prediction that is off by ten units contributes one hundred to the sum, while a prediction off by one unit contributes only one. This property makes MSE the natural choice when large errors are especially harmful and should be heavily discouraged during training and evaluation.
MSE is the most widely used loss function for training regression models, not only as an evaluation metric. Minimizing MSE during training pushes the model to eliminate large deviations, because the gradient of a squared loss is proportional to the error itself, meaning large errors produce large gradients and large parameter updates.
MSE is always non-negative and equals zero only when predictions match labels exactly. Its units are the square of the target variable's units, which makes it less directly interpretable than MAE. A house price model with MSE of one million means errors are being squared in dollars-squared, a unit without intuitive meaning. Root mean squared error converts back to the original units.
MSE is sensitive to outliers. A single extreme misprediction can dramatically increase MSE, potentially misrepresenting typical model performance. If your dataset contains legitimate outliers that the model should not be penalized for missing, MAE or Huber loss may be more appropriate.
A concrete example: a student grade prediction model makes errors of 2, 3, and 10 points on three test cases. MSE is the average of 4, 9, and 100, giving 37.67 squared points. The single large error of 10 points contributes 100 of those 113 squared points, illustrating how MSE is dominated by the largest mistake.
Exam tip: explain that MSE is differentiable everywhere, which makes gradient computation clean during optimization, while MAE has a non-differentiable kink at zero that requires subgradients.