Back to Study Material Machine Learning

R-squared score

R-squared, also called the coefficient of determination, measures how much of the variance in the target variable is explained by the model. It ranges from...

R-squared score

How Much Variance Does the Model Explain?

R-squared, also called the coefficient of determination, measures how much of the variance in the target variable is explained by the model. It ranges from zero to one for a model that performs at least as well as predicting the mean. A value of one means the model perfectly predicts every training example. A value of zero means the model is no better than always predicting the mean. Negative values are possible when the model performs worse than the mean baseline.

R-squared is computed as one minus the ratio of residual sum of squares to total sum of squares. The residual sum of squares measures the total squared error of the model's predictions. The total sum of squares measures the total squared deviation of the true values from their mean. The ratio is the fraction of variance not explained by the model; subtracting from one gives the fraction that is explained.

A critical caution: adding more features to a linear regression model will never decrease R-squared on the training set, even if the new features are random noise. This is because the model can always assign tiny weights to unhelpful features, achieving at least as small a residual. Adjusted R-squared corrects for this by penalizing the number of features, providing a fairer comparison across models of different complexity.

R-squared also depends on the target's variance. On a dataset where all examples are very similar, even a mediocre model can achieve high R-squared. On a highly variable dataset, a genuinely useful model might achieve only moderate R-squared. Always contextualize R-squared with domain knowledge.

A concrete example: a model predicting apartment rental prices achieves R-squared of 0.87, meaning it accounts for 87 percent of the variation in rent across apartments. The remaining 13 percent is attributed to factors not captured in the model's features, such as floor level or building age.

Exam tip: remember that R-squared is a relative metric comparing the model to a mean-only baseline, not an absolute measure of quality, and that adjusted R-squared should be used when comparing models with different numbers of features.