Why Do We Need Loss?

First, the Model Makes a Prediction

During forward propagation, data moves through the neural network and produces an output.

Input
  ↓
Neural Network
  ↓
Prediction

For example, suppose we are predicting whether a student will pass an exam.

1 = Pass
0 = Fail

The model receives the student's information and predicts:

Prediction = 0.80

But we still have a question:

Compare the Prediction With the Actual Answer

Suppose the student actually passed.

Actual Answer = 1
Prediction    = 0.80

The prediction is reasonably close to the actual answer.

Now suppose another model predicts:

Actual Answer = 1
Prediction    = 0.20

This prediction is much worse.

Actual = 1

Prediction A = 0.80
      ↓
Closer to actual answer


Prediction B = 0.20
      ↓
Farther from actual answer

We need a mathematical way to represent that difference. That is where loss comes in.

What Happens Without Loss?

Imagine a neural network that produces predictions but has no way to measure its errors.

Input
  ↓
Neural Network
  ↓
Prediction
  ↓
???

The model knows what it predicted, but there is no numerical measurement telling the training process how far that prediction was from the target.

It would be difficult to systematically determine whether the model is improving or getting worse.

What Happens With Loss?

Now add a loss function.

Input
  ↓
Neural Network
  ↓
Prediction
  ↓
Compare With Actual Answer
  ↓
Loss
  ↓
Measure of Error

Now the training process has a numerical measurement of the model's prediction error.

Example: Good vs Bad Prediction

Suppose the correct answer is:

Actual = 1

Model A predicts:

Prediction = 0.95

Model B predicts:

Prediction = 0.10

Model A is much closer to the target.

Therefore, with an appropriate loss function:

Model A
0.95
 ↓
Smaller Loss


Model B
0.10
 ↓
Larger Loss

The loss gives us a numerical way to distinguish between these two situations.

Loss Gives the Training Process Feedback

Think of loss as feedback about the current prediction.

Prediction
    ↓
Loss Function
    ↓
Loss Value
    ↓
How wrong was the prediction?

For example, imagine training produces these loss values over several iterations:

Step 1 → Loss = 1.80
Step 2 → Loss = 1.20
Step 3 → Loss = 0.75
Step 4 → Loss = 0.40
Step 5 → Loss = 0.18

In this example, the loss is decreasing, which is a sign that the model is becoming better according to the chosen loss function.

How Does Loss Help the Model Learn?

Loss is one part of a larger training process.

1. Make a prediction
        ↓
2. Calculate loss
        ↓
3. Calculate gradients
        ↓
4. Update weights
        ↓
5. Make another prediction
        ↓
6. Calculate loss again
        ↓
7. Repeat

The model repeatedly adjusts its parameters in an attempt to reduce the loss.

Loss Does Not Change the Weights by Itself

This is important.

The loss function calculates the error. It does not directly update the neural network's weights.

Loss Function
      ↓
Measures Error

Backpropagation
      ↓
Calculates Gradients

Optimizer
      ↓
Updates Weights

These are different responsibilities.

Why Do We Usually Try to Reduce Loss?

During training, we generally want the model's predictions to become closer to the target.

Large Loss
   ↓
Large Prediction Error

Small Loss
   ↓
Smaller Prediction Error

Therefore, training algorithms generally try to find parameter values that minimize the loss.

However, "lower loss" only makes sense when comparing values from the same loss function under comparable conditions.

Another Example: House Price Prediction

Loss is not only used for classification.

Suppose a neural network predicts the price of a house.

Actual Price      = $300,000
Predicted Price   = $295,000

The prediction is fairly close.

Another model predicts:

Actual Price      = $300,000
Predicted Price   = $120,000

This prediction is much farther from the target.

Prediction A
$295,000
     ↓
Smaller Error


Prediction B
$120,000
     ↓
Larger Error

A suitable regression loss function can convert those prediction errors into numerical loss values.

Different Problems Need Different Loss Functions

The way we measure error depends on the type of problem.

Regression
    ↓
Predict a number
    ↓
Example: House Price
    ↓
Regression Loss


Binary Classification
    ↓
Two classes
    ↓
Example: Spam / Not Spam
    ↓
Binary Cross-Entropy


Multi-Class Classification
    ↓
Multiple classes
    ↓
Example: Cat / Dog / Horse
    ↓
Categorical Cross-Entropy

We will study these loss functions individually in the following topics.

Loss Inside the Complete Training Loop

Training Data
      ↓
Neural Network
      ↓
Forward Propagation
      ↓
Prediction
      ↓
Loss Function
      ↓
Loss
      ↓
Backpropagation
      ↓
Gradients
      ↓
Optimizer
      ↓
Updated Weights
      ↓
Repeat

The important point is that the model doesn't simply make a prediction once. During training, this process is repeated many times.

A Simple Analogy

Imagine throwing a ball toward a target.

Target
  ●

Your Throw
      ●

Distance Between Them
        ↓
     Error

If your next throw is closer to the target, the error becomes smaller.

A loss function plays a similar measurement role for a model: it quantifies prediction error according to a mathematical rule.

Prediction and Loss Are Different Things

Do not confuse the model's prediction with its loss.

Prediction
    ↓
What the model thinks


Loss
    ↓
How far that prediction is
from the target according
to the loss function

For example:

Prediction = 0.90

Actual = 1

       ↓

Loss Function

       ↓

Loss Value

The prediction and the loss answer two different questions.

What You Should Remember

Without Loss

Input
  ↓
Prediction
  ↓
No numerical error measurement


With Loss

Input
  ↓
Prediction
  ↓
Compare With Target
  ↓
Loss
  ↓
Error Measurement
  ↓
Training Can Improve Parameters

Loss gives the training process a measurable objective: reduce prediction error according to the selected loss function.