What Is Backpropagation?

Backpropagation is the process a neural network uses to determine how its weights contributed to the prediction error and how those weights should be adjusted to improve the next prediction.

In simple words

A neural network makes a prediction, calculates how wrong that prediction is, and then works backward through the network to figure out how its weights should change.

That backward process is called backpropagation.

The Big Picture

A neural network learns through a repeated process.

Input
  ↓
Neural Network
  ↓
Prediction
  ↓
Calculate Loss
  ↓
Backpropagation
  ↓
Calculate How Weights Should Change
  ↓
Update Weights
  ↓
Try Again

The important part is that backpropagation happens after the loss has been calculated.

Why Does It Go Backward?

During forward propagation, information moves from the input toward the output.

Input
  ↓
Hidden Layer
  ↓
Output
  ↓
Prediction

After the prediction is made, we calculate the loss. Now the network needs to determine which weights caused the error.

This information is worked backward from the output toward the earlier layers.

Loss
  ↓
Output Layer
  ↓
Hidden Layer
  ↓
Earlier Layers

That is where the name backpropagation comes from: the error information is propagated backward through the network.

A Simple Example

Imagine a neural network predicting whether a student will pass an exam.

The input is the number of hours studied:

Hours Studied = 5

The correct answer is:

Actual Answer = 1
1 = Pass

But the neural network predicts:

Prediction = 0.30

The model is predicting only a 30% probability of passing.

The prediction is not good, so the loss will be relatively large.

Actual = 1
Prediction = 0.30

        ↓

Calculate Loss

        ↓

Large Error

What Does Backpropagation Do?

Once the network knows that the prediction was wrong, it needs to answer another question:

Which weights should change?

And more importantly:

How much should each weight change?

Backpropagation calculates the information needed to answer these questions.

Prediction is wrong
        ↓
Calculate Loss
        ↓
Backpropagation
        ↓
Determine each weight's contribution
        ↓
Calculate gradients
        ↓
Update weights

We will study gradients in detail later in this lesson.

Why Are Weights Important?

Remember that weights control how strongly inputs influence neurons.

Input
  ↓
Weight
  ↓
Neuron

For example:

Hours Studied = 5

Weight = 0.2

5 × 0.2 = 1

If the weight changes, the neuron's calculation changes. That can change the final prediction.

Weight
   ↓
Neuron output
   ↓
Final prediction
   ↓
Loss

Therefore, changing weights is how the network learns.

How the Network Learns

Suppose the network starts with poor weights.

Weights
   ↓
Prediction = 0.30
   ↓
Large Loss

Backpropagation provides information about how those weights should be adjusted.

Large Loss
   ↓
Backpropagation
   ↓
Adjust Weights
   ↓
New Prediction = 0.60
   ↓
Smaller Loss

After more training:

Prediction = 0.60
        ↓
Adjust Weights
        ↓
Prediction = 0.80
        ↓
Adjust Weights
        ↓
Prediction = 0.90

The model gradually learns weights that produce better predictions.

Backpropagation vs Weight Update

These two ideas are related, but they are not exactly the same thing.

Backpropagation
      ↓
Calculate gradients


Weight Update
      ↓
Use gradients to change weights

For example:

Loss
 ↓
Backpropagation
 ↓
Gradient = 0.8
 ↓
Optimizer
 ↓
New Weight

Backpropagation determines the direction and sensitivity information needed for learning. The optimizer then uses that information to update the weights.

Forward Propagation vs Backpropagation

FORWARD PROPAGATION

Input
  ↓
Hidden Layers
  ↓
Output
  ↓
Prediction
  ↓
Loss

BACKPROPAGATION

Loss
  ↓
Output Layer
  ↓
Hidden Layers
  ↓
Gradients
  ↓
Weight Updates

So they perform opposite-direction jobs during training.

Real-Life Example

Imagine you are learning to shoot basketball.

You shoot the ball and miss the basket.

Shot
 ↓
Miss
 ↓
Error

You then ask:

Was my shot too strong?
Was my angle wrong?
Was my position wrong?

You use that feedback to change your next attempt.

Miss
 ↓
Understand the error
 ↓
Adjust technique
 ↓
Shoot again
 ↓
Better result

Backpropagation follows a similar learning idea: prediction error is used to determine how the internal parameters should be adjusted.

Important: Backpropagation Does Not "Fix" the Network by Itself

Backpropagation calculates gradients that tell us how the loss changes with respect to the network's weights.

The weights are then updated using an optimization method such as gradient descent or Adam.

The Mathematical Idea

Eventually, we want to know how much the loss changes when a particular weight changes.

Weight changes
      ↓
How does Loss change?

This relationship is represented by a gradient.

Gradient
    =
How much the loss changes
when a weight changes

We will calculate gradients in the next topics. For now, remember that gradients provide the direction and sensitivity information needed to improve the weights.

Complete Learning Cycle

1. Input Data
       ↓
2. Forward Propagation
       ↓
3. Prediction
       ↓
4. Calculate Loss
       ↓
5. Backpropagation
       ↓
6. Calculate Gradients
       ↓
7. Update Weights
       ↓
8. Repeat

The network repeats this process many times during training.

Over time, the weights are adjusted so the model can make better predictions.

Simple Numerical Example

Suppose a very small network has one weight:

Input = 5
Weight = 0.2

The network calculates:

5 × 0.2 = 1

Suppose the desired output is:

Target = 2

The prediction is too small.

Prediction = 1
Target     = 2

Error = 1

Backpropagation determines how changing the weight would affect the loss.

Suppose, for this simplified example, the calculated gradient is:

Gradient = -10

The exact gradient calculation will be explained later. For now, the important idea is:

Loss
 ↓
Backpropagation
 ↓
Gradient
 ↓
Weight should change

Why Is Backpropagation So Important?

A neural network can contain thousands, millions, or even billions of parameters.

Manually deciding how every weight should change would be impossible.

Millions of Weights
        ↓
One Prediction
        ↓
One Loss
        ↓
Backpropagation
        ↓
Gradients for the Weights
        ↓
Automatic Learning

Backpropagation provides an efficient mathematical method for determining how all these parameters should contribute to reducing the loss.

The Key Idea

Backpropagation is the mechanism that takes the error from the output and works backward through the neural network to calculate how the weights affected that error.

Prediction
    ↓
Loss
    ↓
Backpropagation
    ↓
Gradients
    ↓
Weight Updates
    ↓
Better Prediction

QUICK CHECK

Check Your Understanding

What is backpropagation?
It is the process of propagating information about the prediction error backward through the network to calculate gradients for the weights.

When does backpropagation happen?
After the network makes a prediction and the loss has been calculated.

Why do we need it?
Because the loss tells us how wrong the model is, but we also need to know how each weight should change.

Does backpropagation directly change the weights?
No. It calculates gradients. An optimizer uses those gradients to update the weights.

What direction does information move during backpropagation?
From the loss/output side backward toward the earlier layers.

What is the overall learning cycle?
Forward pass → prediction → loss → backpropagation → gradients → weight update → repeat.

NEXT TOPIC

Why Do We Need Backpropagation?

Next, we will look at the specific problem backpropagation solves and why calculating the loss alone is not enough for a neural network to learn.