Multiple linear regression
Multiple linear regression extends simple linear regression to handle more than one input feature. Instead of a single weight, the model learns a separate ...
Predicting with Several Features at Once
Multiple linear regression extends simple linear regression to handle more than one input feature. Instead of a single weight, the model learns a separate weight for each feature plus one intercept term. The prediction is the weighted sum of all features, and training still minimizes the sum of squared differences between predictions and true labels.
The key insight is that each weight represents the expected change in the output for a one-unit increase in its corresponding feature, holding all other features constant. This partial effect interpretation is what makes multiple regression so useful for understanding which factors drive an outcome even when multiple variables move together.
Multicollinearity is the main complication. When two input features are highly correlated — for example, a person's height in centimetres and height in inches — the model struggles to assign stable individual weights because infinitely many combinations of the two weights produce the same predictions. Ridge regression addresses this by adding a penalty proportional to the square of the weights, shrinking them toward zero and stabilizing the solution.
Feature scaling matters more in multiple regression than in simple regression. If one feature spans values from 0 to 1 and another spans values from 0 to 100,000, the algorithm must work across vastly different scales, which can slow convergence in gradient-based optimization. Standardizing features to zero mean and unit variance avoids this problem.
A concrete example: predicting a car's fuel consumption in miles per gallon using engine displacement, vehicle weight, and model year. With three features there is one weight per feature plus an intercept. The weight on model year might be positive, reflecting that newer cars are more efficient, while the weight on engine displacement is negative.
Exam tip: understand that adding more features does not always improve a model. Adding irrelevant features inflates the number of parameters without adding useful signal, which can lead to overfitting on small datasets.