Back to Study Material Machine Learning

Feature engineering

Feature engineering is the process of transforming raw data into representations that make it easier for machine learning algorithms to learn. It is often ...

Feature engineering

Crafting Better Inputs for Better Models

Feature engineering is the process of transforming raw data into representations that make it easier for machine learning algorithms to learn. It is often described as more art than science, requiring domain knowledge, creativity, and iterative experimentation. Good feature engineering can dramatically improve a model's performance, sometimes more than switching to a more powerful algorithm.

Raw data rarely arrives in a form that highlights the patterns a model needs. A timestamp in an e-commerce dataset, for example, contains multiple latent signals: hour of day, day of week, whether it is a public holiday, days until payday. Extracting these signals as explicit features gives the model more to work with than the raw timestamp alone.

Interaction features capture relationships between variables. Multiplying price per unit by quantity purchased creates a total spend feature that may predict customer lifetime value better than either original feature alone. Polynomial features let linear models capture curved relationships by adding squared or cubed versions of original features.

Feature engineering also includes encoding. Target encoding replaces a categorical variable with the mean target value for that category, which can be powerful but risks leakage if not done carefully within cross-validation folds. Frequency encoding replaces a category with how often it appears in the dataset.

A concrete example: in predicting flight delays, raw features include scheduled departure time and airport. Engineered features might include time since last maintenance, historical on-time rate for that route at that hour, and number of connecting flights in the airport that hour. These derived signals are far more predictive.

Exam tip: be ready to describe at least two feature engineering techniques and explain why they help. Interviewers appreciate the understanding that feature engineering encodes domain knowledge into the model.