Understanding the Perceptron from Scratch: Weights, Bias, Activation Function & the Learning…

Understanding the Perceptron from Scratch: Weights, Bias, Activation Function & the Learning Algorithm (Visual Guide)

Learn the perceptron — the building block of modern neural networks — with intuitive diagrams, Python examples, and visual explanations.

Tags: Machine Learning, Python, Artificial Intelligence, Neural Networks, Data Science

Why Read This Article?

If you’ve started learning Machine Learning, you’ve probably encountered code like this:

prediction = self.activation(self.weighted_sum(inputs)) At first glance, it can feel mysterious.

What are weights ?
Why do we multiply inputs by weights?
What is bias ?
Why do we need an activation function?
How does the perceptron actually learn?

This article answers all of those questions using simple analogies, diagrams, and Python examples.

What is a Perceptron?
Understanding Inputs
Understanding Weights
What is a Weighted Sum?
Why Do We Need Bias?
Understanding the Activation Function
Visualizing the Decision Boundary
The Perceptron Learning Algorithm
Python Implementation
Limitations of a Perceptron
Key Takeaways

What is a Perceptron?

The perceptron is the simplest artificial neuron.

It receives some information, processes it, and makes a decision.

Think of it as a yes/no machine.

Inputs ↓ Weights ↓ Weighted Sum ↓ Bias ↓ Activation Function ↓ Prediction Every modern neural network is built from this basic idea.

Step 1 — Inputs

Inputs are simply pieces of information.

Imagine you’re hiring an employee.

Experience = 8 years Communication = 6/10 Python:

inputs = [8,6] Nothing fancy.

They’re just numbers.

Step 2 — Weights

Not every feature matters equally.

Maybe experience matters more than communication.

weights = [3,1] Meaning:

Experience has 3× more influence .

Visual:

Experience ████████████ Communication ████ Weights simply tell the perceptron:

“Pay more attention to this feature.”

Step 3 — Weighted Sum

Now multiply each input by its weight.

Experience 8 × 3 = 24 Communication 6 × 1 = 6 Total:

24 + 6 = 30 Formula:

Weighted Sum = x₁w₁ + x₂w₂ + ... + xₙwₙ Python:

total = 0 for i in range(len(inputs)): total += inputs[i] * weights[i] Step 4 — Bias Bias is like moving the passing mark.

Without bias:

Pass if score ≥ 0 With bias:

Pass if score ≥ 50 Mathematically:

y = w₁x₁ + w₂x₂ + b where

b is the bias.

Visualizing Bias

Without bias

y ↑ / / ----/-----------→ x With bias

y ↑ / / ------/----------→ x Notice:

The line moved.

It didn’t rotate.

Step 5 — Activation Function

The weighted sum is just a number.

The activation function converts that number into a decision.

Example:

if weighted_sum >= 0: return 1 else: return -1 Output:

Positive ↓ Class 1 Negative ↓ Class -1 Visualizing the Entire Perceptron Inputs x₁ x₂ │ │ ▼ ▼ Multiply by Weights │ │ ▼ ▼ Weighted Sum ↓ + Bias ↓ Activation Function ↓ Prediction fig : Decision Boundary Decision Boundary A perceptron separates two classes using a straight line.

Example:

x + y = 45 Graph:

+ + + + + + + + ------------- - - - - - - - Everything above the line belongs to one class.

Everything below belongs to another.

Training the Perceptron

The perceptron repeats the same four steps.

Guess ↓ Compare ↓ Learn ↓ Repeat Python:

prediction = self.activation( self.weighted_sum(inputs) ) actual = training_set[inputs] error = actual - prediction Updating the Weights The learning rule:

weight += error * input Suppose:

weight = 1 error = -2 input = 3 New weight:

1 + (-2 × 3) = -5 Every mistake changes the weights slightly.

Eventually the perceptron learns.

Complete Learning Cycle

Start ↓ Guess Weights ↓ Take Training Example ↓ Weighted Sum ↓ Activation ↓ Prediction ↓ Compare ↓ Wrong? ↓ Update Weights ↓ Repeat ↓ No Errors? ↓ Training Complete Limitations A single perceptron can only learn linearly separable data.

Example:

Cats ---------------- Dogs Works perfectly.

But XOR:

X O O X Cannot be separated using one straight line.

This limitation led to multi-layer neural networks.

Key Takeaways

✅ Inputs are pieces of information.

✅ Weights measure importance.

✅ Weighted Sum combines all inputs.

✅ Bias moves the decision boundary.

✅ Activation Function makes the final decision.

✅ Training updates the weights after every mistake.

Together, these simple ideas form the foundation of neural networks and modern deep learning.

What’s Next?

Now that you understand a single perceptron, the natural next step is to explore:

Multi-Layer Perceptrons (MLPs)
Gradient Descent
Backpropagation
Sigmoid, ReLU, and Softmax activation functions
Building your first neural network with TensorFlow or PyTorch

Understanding the perceptron gives you the vocabulary and intuition you’ll use throughout your machine learning journey.