What Does Training Mean?

What Does Training Mean?

A neural network does not automatically know the correct answer when we first create it.

Its weights usually start with random or initial values. Because of this, its first predictions may be very poor.

Training is how we gradually improve those weights.

Data
  ↓
Prediction
  ↓
Compare With Correct Answer
  ↓
Calculate Loss
  ↓
Calculate Gradients
  ↓
Update Weights
  ↓
Better Prediction
  ↓
Repeat

This process is repeated many times until the neural network learns useful patterns from the training data.

A Simple Example

Imagine that we want a neural network to predict whether a student will pass an exam based on how many hours the student studies.

Study Hours    Result

1 hour         Fail
2 hours        Fail
4 hours        Pass
5 hours        Pass
7 hours        Pass

We give these examples to the neural network.

The network needs to learn the relationship between study hours and the exam result.

Before Training

Before training, the neural network does not understand the relationship between study hours and exam results.

For example, suppose we give it:

Study Hours = 5

The network might initially produce:

Prediction = 0.30

If 1 means Pass and 0 means Fail, a prediction of 0.30 is not very confident about passing.

But our training data tells us that a student studying 5 hours passed.

Correct Answer = 1

Model Prediction = 0.30

The prediction is not good enough.

Step 1 — Measure the Mistake

The neural network needs to know how wrong its prediction is.

This is where the loss function is used.

Prediction
     ↓
    0.30

Correct Answer
     ↓
    1.00

       ↓

Calculate Loss

       ↓

Measure how wrong the prediction is

A large loss means the prediction is far from the correct answer. A small loss means the prediction is closer.

We covered loss functions in the previous lesson.

Step 2 — Learn From the Error

After calculating the loss, the neural network needs to determine which weights caused the error and how those weights should change.

This is where backpropagation is used.

Prediction
     ↓
Loss
     ↓
Backpropagation
     ↓
Gradients
     ↓
Determine how weights should change

The network does not simply say "my prediction was wrong." It calculates how each weight contributed to that error.

Step 3 — Update the Weights

Once the gradients are calculated, the optimizer adjusts the weights.

Old Weights
     ↓
Calculate Gradients
     ↓
Update Weights
     ↓
New Weights

The goal is to move the weights in a direction that reduces the loss.

After the update, the network tries the example again.

Step 4 — Make a Better Prediction

After updating its weights, the network might now produce:

Study Hours = 5

Before Training:
Prediction = 0.30

After Weight Update:
Prediction = 0.65

The prediction moved closer to the correct answer:

Correct Answer = 1

0.30  →  0.65  →  closer to 1

The network has started learning.

Step 5 — Repeat the Process

One update is usually not enough.

The neural network repeats the process again and again.

1. Make prediction
2. Calculate loss
3. Calculate gradients
4. Update weights

        ↓

Repeat

        ↓

Better predictions

        ↓

Lower loss

This repeated process is what allows the neural network to learn.

See the Learning Process

Imagine that the network starts with a poor prediction. Over several training steps, its prediction can gradually move closer to the correct answer.

Correct Answer = 1

Step 1
Prediction = 0.30
Loss = High

        ↓

Step 2
Prediction = 0.55
Loss = Lower

        ↓

Step 3
Prediction = 0.72
Loss = Lower

        ↓

Step 4
Prediction = 0.86
Loss = Much Lower

        ↓

Step 5
Prediction = 0.94
Loss = Very Low

The important idea is not the exact numbers. The important idea is that the weights are repeatedly adjusted so the model can produce better predictions.

What Does the Neural Network Actually Learn?

A neural network does not memorize a simple instruction such as:

"5 hours means Pass"

Instead, it learns useful numerical parameters called weights and biases.

Those parameters allow the network to recognize patterns in the training data.

Training Data
     ↓
Neural Network
     ↓
Learn Weights + Biases
     ↓
Learn Patterns
     ↓
Make Predictions

Training Does Not Simply Mean Memorizing

This is an important distinction.

We want the neural network to learn a general pattern, not simply memorize the examples it has seen.

For example, if the network sees:

2 hours → Fail
4 hours → Pass
5 hours → Pass
7 hours → Pass

We want it to learn the underlying relationship well enough to make a reasonable prediction for a new student.

New Student

Study Hours = 6

        ↓

Trained Neural Network

        ↓

Prediction = Pass

This ability to perform well on new data is one of the most important goals of machine learning.

The Training Cycle

             Training Data
                   │
                   ▼
            ┌─────────────┐
            │   Network   │
            └──────┬──────┘
                   │
                   ▼
              Prediction
                   │
                   ▼
                Loss
                   │
                   ▼
            Backpropagation
                   │
                   ▼
               Gradients
                   │
                   ▼
            Update Weights
                   │
                   └───────────┐
                               │
                               ▼
                         Try Again

This cycle happens many times during neural-network training.

Important Terms

Training
→ The complete process of teaching the network.

Weights
→ Numbers the network adjusts during learning.

Bias
→ Another learnable value that helps shift the output.

Prediction
→ The answer produced by the network.

Loss
→ A measurement of how wrong the prediction is.

Gradient
→ Tells how the weights should change.

Optimizer
→ Uses gradients to update the weights.

We will explore epochs, batch size, iterations, and learning rate in the next topics.

How This Looks in Python

The basic idea can be represented with a simple training loop:

for step in range(100):

    prediction = model(input)

    loss = calculate_loss(prediction, target)

    gradients = calculate_gradients(loss)

    update_weights(gradients)

Do not worry about the exact Python syntax yet. We will build this concept step by step in the upcoming topics.

The important thing to understand now is the flow:

Prediction
    ↓
Loss
    ↓
Gradients
    ↓
Weight Update
    ↓
Repeat

Remember This

Training means:

1. Give data to the network.
2. Make a prediction.
3. Compare the prediction with the correct answer.
4. Calculate the loss.
5. Calculate gradients.
6. Update weights.
7. Repeat the process.

The purpose of training is simple: make the neural network's predictions better by reducing its errors over time.