Multilayer perceptron
A multilayer perceptron (MLP) is a fully connected neural network consisting of an input layer, one or more hidden layers, and an output layer. Each neuron...
From Single Neurons to Multilayer Networks
A multilayer perceptron (MLP) is a fully connected neural network consisting of an input layer, one or more hidden layers, and an output layer. Each neuron in a layer connects to every neuron in the next layer. Adding hidden layers with non-linear activation functions overcomes the perceptron's limitation of only solving linearly separable problems.
The key insight is that each hidden layer transforms the input into a new representation. With a non-linear activation like ReLU or sigmoid, the hidden layers can reshape the input space, effectively making a non-linearly separable problem linearly separable by the final layer. This is why MLPs are called universal function approximators: given enough neurons and layers, an MLP can approximate any continuous function to arbitrary precision.
Training an MLP uses the same backpropagation and gradient descent framework as deeper networks. The loss function measures prediction error, backpropagation computes gradients through all layers, and an optimiser updates weights layer by layer. The training loop iterates over the dataset many times until the loss converges.
MLPs are used in a wide range of tasks: tabular data classification (such as predicting customer churn), regression (predicting a continuous output), and as components within more complex architectures like Transformers, where feed-forward sub-layers are essentially two-layer MLPs applied to each position independently.
A concrete example: a two-hidden-layer MLP with ReLU activations can classify the XOR problem perfectly. The first hidden layer reshapes the input space so that the two XOR classes become linearly separable, and the output layer draws the separating boundary. This is something a single perceptron cannot do.
Exam tip: interviewers sometimes ask how an MLP differs from a deep network. The term MLP typically refers to a fully connected architecture. Deep learning often uses specialised architectures like CNNs, RNNs, or Transformers that exploit structure in the data for better efficiency. An MLP ignores spatial or sequential structure, making it less sample-efficient for images or text.