Back to Study Material Deep Learning

Autoencoder

An autoencoder is a neural network trained to reproduce its input at the output, forcing information to pass through a compressed intermediate representati...

Autoencoder

Autoencoders for Representation Learning

An autoencoder is a neural network trained to reproduce its input at the output, forcing information to pass through a compressed intermediate representation called the latent space or bottleneck. The network is split into two parts: the encoder, which compresses the input into a lower-dimensional latent vector, and the decoder, which reconstructs the original input from that vector.

The training objective is reconstruction loss: minimising the difference between the input and the output, typically measured with mean squared error for continuous data or binary cross-entropy for binary data. Because the bottleneck has fewer dimensions than the input, the encoder must learn to retain only the most important information. The decoder must learn to reconstruct the full input from this compressed code.

Autoencoders are used for dimensionality reduction, anomaly detection, and data denoising. For anomaly detection, the autoencoder is trained on normal examples only. At inference time, normal examples are reconstructed accurately, producing low reconstruction error. Anomalous examples, being unlike anything in the training data, are reconstructed poorly, producing high error that flags them as anomalies.

Denoising autoencoders are trained by deliberately corrupting inputs (for example, adding random noise to images) and learning to reconstruct the clean original. This teaches the encoder to extract robust features that capture the true structure of the data rather than noise.

A concrete example: a denoising autoencoder trained on clean handwritten digits is corrupted with Gaussian noise during training. The reconstructed output is a clean digit. The latent space that enables this denoising has learned a smooth, structured representation of digit shapes that discards noise by design.

Exam tip: interviewers often compare autoencoders with PCA. Both perform dimensionality reduction, but PCA is linear whereas an autoencoder with non-linear activations can learn non-linear manifolds in the data. A linear autoencoder with no activation functions is mathematically equivalent to PCA. Knowing this connection demonstrates a strong grasp of the relationship between classical and deep learning methods.