Back to Study Material Machine Learning

Baseline model

A baseline model is the simplest possible predictor you establish before building any complex system. It sets the floor: any model you deploy must outperfo...

Baseline model

The Simplest Benchmark You Must Beat

A baseline model is the simplest possible predictor you establish before building any complex system. It sets the floor: any model you deploy must outperform the baseline to justify its complexity and cost. Without a baseline, you cannot know whether a model achieving 85 percent accuracy is impressive or mediocre, because the baseline might already achieve 84 percent with zero learning.

Common classification baselines include the majority class classifier, which predicts the most frequent class for every example, and a random classifier, which assigns classes in proportion to their training frequencies. For regression, the mean predictor is standard: it predicts the mean target value for every input. These baselines are trivially implemented and require no feature engineering.

Beyond these rule-based baselines, a simple logistic regression or linear regression model trained with default hyperparameters often serves as a strong baseline before moving to complex ensemble methods or deep networks. The logic is that linear models are easy to train, easy to debug, and often surprisingly competitive on real data.

Establishing a baseline also helps estimate achievable performance. Human-level error on the task provides an upper bound. If humans achieve 95 percent accuracy and your neural network achieves 96 percent, further improvement is negligible. If the baseline achieves only 60 percent and your model achieves 61 percent, something may be wrong with the model or the data.

A concrete example: a news article categorization task has five categories with 40 percent of articles in the most common category. The majority class baseline achieves 40 percent accuracy. A logistic regression model achieves 72 percent. Only after establishing these reference points does building a transformer model become clearly motivated.

Exam tip: always state your baseline before comparing models. Interviewers expect you to know what a naive predictor would achieve on your problem, because it contextualizes whether your sophisticated model is actually adding value.