Back to Study Material Artificial Intelligence

Text classification

Text classification is the task of automatically assigning predefined categories to text documents based on their content. It is one of the most widely dep...

Text classification

Assigning Categories to Text

Text classification is the task of automatically assigning predefined categories to text documents based on their content. It is one of the most widely deployed NLP tasks in industry, underpinning spam filtering, sentiment analysis, topic labelling, intent detection in chatbots, and content moderation systems. The input is a text string of arbitrary length; the output is one or more category labels from a fixed set.

Classical approaches represented documents as bag-of-words or TF-IDF vectors, where each dimension corresponded to a vocabulary term and its value reflected word frequency weighted by how distinctive that word is across the corpus. These high-dimensional sparse vectors were then fed into classifiers such as naive Bayes, support vector machines, or logistic regression. Naive Bayes in particular has a natural fit for text classification because it assumes conditional independence among words given the class, which despite being clearly false, tends to produce surprisingly competitive performance in practice.

Modern text classification is dominated by fine-tuning pre-trained transformer models such as BERT. The process has two stages. In pre-training, the model learns general language representations from massive unlabelled text corpora using objectives such as masked language modelling. In fine-tuning, a classification head is added on top of the pre-trained encoder and the entire model is updated on labelled examples from the target task. This transfer learning approach achieves state-of-the-art performance with orders of magnitude less labelled data than training from scratch would require.

Evaluation Metrics

Accuracy measures the fraction of correctly classified examples and is appropriate when classes are balanced. When classes are imbalanced, as is typical in spam detection where legitimate emails heavily outnumber spam, precision, recall, and F1 score are more informative. Precision is the fraction of predicted positives that are truly positive. Recall is the fraction of true positives that are correctly predicted. F1 is their harmonic mean.

Interview Tip

For text classification interviews, be prepared to discuss how you would handle class imbalance. Strategies include oversampling the minority class, undersampling the majority class, using class-weighted loss functions, and generating synthetic minority examples with data augmentation. Mentioning that the choice of evaluation metric should drive training decisions, for example optimising for recall in medical screening to minimise false negatives, shows practical judgement.