Back to Study Material Deep Learning

Perceptron

The perceptron is the simplest neural network model, introduced by Frank Rosenblatt in 1958. It is a single artificial neuron that learns to classify input...

Perceptron

The Perceptron: Foundation of Neural Networks

The perceptron is the simplest neural network model, introduced by Frank Rosenblatt in 1958. It is a single artificial neuron that learns to classify inputs into one of two categories by finding a linear decision boundary. Despite its simplicity, understanding the perceptron is essential because modern deep learning builds on the same fundamental principles.

A perceptron takes a vector of numerical inputs, multiplies each by a learned weight, sums the products, adds a bias, and applies a step activation function. If the result is above zero, it outputs class 1; otherwise it outputs class 0. Training adjusts the weights and bias whenever the perceptron misclassifies a training example, nudging them in the direction that would have produced the correct output.

This learning rule is the perceptron update algorithm. For each misclassified sample, each weight is updated by adding the product of the learning rate, the error (difference between true label and predicted label), and the input value. This process repeats until all training examples are correctly classified, which is guaranteed to converge if the data is linearly separable.

The limitation is that perceptrons can only solve problems where the two classes can be separated by a straight line (or hyperplane in higher dimensions). The classic example of the XOR problem cannot be solved by a single perceptron because the two classes are not linearly separable. This limitation motivated the development of multilayer networks.

A concrete example: a perceptron can learn to classify email as spam or not spam if spam can be identified by a simple linear combination of features like word count and presence of certain keywords. For more complex patterns, a multilayer network is required.

Exam tip: if asked about the perceptron convergence theorem, explain that it guarantees the algorithm will find a solution in finite steps if one exists, but makes no promises about the number of steps or what happens when the data is not linearly separable. This shows you understand both the power and the limits of the model.