Back to Study Material Deep Learning

Layers in a neural network

A neural network is organised into layers, each consisting of a set of neurons that process information and pass it forward. The architecture of these laye...

Layers in a neural network

The Role of Layers in a Neural Network

A neural network is organised into layers, each consisting of a set of neurons that process information and pass it forward. The architecture of these layers determines what patterns the network can learn. There are three broad categories: the input layer, hidden layers, and the output layer.

The input layer receives raw data. If the input is a 28 by 28 pixel grayscale image, the input layer has 784 neurons, one per pixel. No computation happens here; the layer simply holds the feature values that will be processed downstream.

Hidden layers sit between the input and output. They perform the actual learning by applying weighted transformations and activation functions. Shallow networks have one or two hidden layers and can model simple functions. Deep networks have many hidden layers, sometimes hundreds, and can model extremely complex relationships. For instance, a deep network classifying handwritten digits might have five hidden layers; early layers detect strokes and curves, later layers compose those into full digit shapes.

The output layer produces the final prediction. For binary classification, it typically has one neuron outputting a probability. For ten-class digit recognition, it has ten neurons, each outputting the probability of a specific class. The class with the highest probability is the prediction.

Choosing the number of layers and neurons per layer is called network architecture design. Too few layers underfit the data, meaning the model cannot capture the complexity. Too many layers overfit, memorising training examples rather than generalising. Techniques like cross-validation help find the right depth.

Exam tip: when asked about deep versus shallow networks, emphasise that depth allows the network to build hierarchical representations. Each layer transforms its input into a slightly more abstract form. This composition of simple transformations into complex ones is the mathematical foundation of deep learning's expressive power.