Missing values
Missing values are one of the most common data quality challenges in real-world machine learning. Data can be missing for many reasons: sensor failures, su...
Strategies for Incomplete Data
Missing values are one of the most common data quality challenges in real-world machine learning. Data can be missing for many reasons: sensor failures, survey non-response, database joins that find no matching record, or fields that simply were not collected. How you handle missing values affects both model accuracy and the validity of conclusions drawn from the data.
Understanding why values are missing matters before deciding how to handle them. Data is missing completely at random if missingness has no relationship to any variable — a random instrument failure, for instance. Data is missing at random if missingness depends on other observed variables but not on the missing value itself. Data is missing not at random if the missing value itself is related to why it is missing — for example, very sick patients refusing to report their health scores. This last case is the hardest to handle without bias.
Simple imputation replaces missing values with a summary statistic: the mean or median for numerical columns (median is more robust to outliers), and the most frequent value for categorical columns. This is fast but ignores relationships between variables and can distort variance.
Multiple imputation generates several complete datasets using a model of the data, analyzes each separately, and combines results. This better captures uncertainty but is computationally expensive. Predictive imputation trains a small model specifically to predict the missing column from other columns — a more sophisticated approach that respects relationships in the data.
Some algorithms, like certain tree-based methods, handle missing values natively by learning optimal split directions for missing cases, eliminating the need for imputation.
A concrete example: in a loan dataset, income is missing for 15 percent of applicants. Using the median income for imputation works but misses the fact that income missingness correlates with self-employment — a category with very different risk profiles.
Exam tip: always discuss the mechanism of missingness before recommending a strategy. Naive imputation of data missing not at random introduces systematic bias.