Back to Study Material Machine Learning

Label encoding

Label encoding is a preprocessing technique that converts categorical feature values into integers. Each unique category is assigned a distinct integer, ty...

Label encoding

Assigning Numbers to Categories

Label encoding is a preprocessing technique that converts categorical feature values into integers. Each unique category is assigned a distinct integer, typically starting from zero. For example, the color feature with values red, green, and blue might be encoded as 0, 1, and 2 respectively. The encoding is straightforward, requires minimal memory, and is a required step before most machine learning algorithms can process categorical data.

The primary risk with label encoding is that it introduces an artificial ordinal relationship among the categories. When the model sees 0, 1, and 2 for red, green, and blue, it may interpret green as being between red and blue in some meaningful sense, or that blue is twice as far from red as green is. For features where the categories have a natural order — such as small, medium, and large — this is perfectly appropriate. For truly unordered categories like colors or country names, the implied order is misleading.

Tree-based algorithms such as decision trees and random forests are generally immune to this problem because they partition data by checking whether a value is above or below a threshold, not by assuming linear relationships. Linear models and neural networks, however, may learn spurious relationships from the implied ordering.

For unordered categorical features used with linear or neural models, one-hot encoding is safer. For ordered categories or when memory is limited and the model is tree-based, label encoding is the practical choice.

A concrete example: in a model predicting salary, the education level feature has values high school, bachelor, master, and doctorate. These have a natural order, so label encoding them as 0, 1, 2, and 3 is appropriate and gives the model meaningful ordinal information.

Exam tip: always ask whether a categorical feature is ordinal or nominal before choosing label encoding. Ordinal features benefit from label encoding; nominal features should use one-hot encoding unless the model is tree-based.