Understand the Python Code

The Complete Training Code

First, look at the complete simplified training loop. Do not worry if it looks complicated. We will break it down line by line.

epochs = 5
learning_rate = 0.1

for epoch in range(epochs):

    prediction = model(x_train)

    loss = calculate_loss(
        prediction,
        y_train
    )

    gradients = calculate_gradients(loss)

    update_weights(
        gradients,
        learning_rate
    )

    print(
        "Epoch:",
        epoch + 1,
        "Loss:",
        loss
    )

The basic idea is:

Data
 ↓
Prediction
 ↓
Loss
 ↓
Gradients
 ↓
Update Weights
 ↓
Repeat

1. Setting the Number of Epochs

epochs = 5

This tells Python that we want to repeat the training process 5 times.

One complete pass through the training dataset is called an epoch.

epochs = 5

Epoch 1
Epoch 2
Epoch 3
Epoch 4
Epoch 5

More epochs give the model more opportunities to update its weights, but more epochs are not automatically better.

2. Setting the Learning Rate

learning_rate = 0.1

The learning rate controls how large the weight updates should be.

Think of it as the size of each learning step.

Small learning rate
→ Small weight changes


Large learning rate
→ Large weight changes

For example, if the gradient is 0.5:

learning_rate = 0.1

weight change
=
0.1 × 0.5

=
0.05

The actual update direction depends on the optimization rule, but the learning rate controls the scale of the change.

3. Starting the Training Loop

for epoch in range(epochs):

This tells Python to repeat the code inside the loop for each epoch.

Because:

epochs = 5

Python effectively performs:

Epoch 1
Epoch 2
Epoch 3
Epoch 4
Epoch 5

Everything indented inside the loop is executed once per epoch.

4. Making a Prediction

prediction = model(x_train)

Here, we give the training data to the neural network.

x_train
   ↓
model
   ↓
prediction

Suppose our model is trying to predict whether a student will pass an exam.

x_train = study hours

Study Hours = 5

        ↓

Neural Network

        ↓

Prediction = 0.70

The model predicts 0.70 in this simplified example.

This step is called the forward pass.

5. Calculating the Loss

loss = calculate_loss(
    prediction,
    y_train
)

Now we compare the model's prediction with the correct answer.

Prediction = 0.70
Actual     = 1.00

        ↓

Calculate Loss

        ↓

Loss = how wrong the model was

The exact numerical loss depends on which loss function is being used.

The important idea is:

Prediction
      +
Actual Answer
      ↓
Loss

A smaller loss generally means the prediction is better for the selected objective.

6. Calculating Gradients

gradients = calculate_gradients(loss)

Now the model needs to determine how its weights should change to reduce the loss.

Gradients provide the direction and sensitivity information used to adjust the parameters.

Loss
 ↓
Backpropagation
 ↓
Gradients
 ↓
Weight Updates

You can think of a gradient as answering:

"If this weight changes,
what happens to the loss?"

Backpropagation calculates these gradients by working backward through the network.

7. Updating the Weights

update_weights(
    gradients,
    learning_rate
)

The optimizer uses the gradients and learning rate to modify the model's weights.

Old Weights
     ↓
Gradients
     +
Learning Rate
     ↓
Updated Weights

A simplified gradient-descent update is:

new_weight =
    old_weight - learning_rate * gradient

For example:

Old Weight     = 0.50
Gradient       = 0.20
Learning Rate  = 0.10

New Weight
=
0.50 - (0.10 × 0.20)

=
0.48

The weight changed from 0.50 to 0.48.

8. Printing the Results

print(
    "Epoch:",
    epoch + 1,
    "Loss:",
    loss
)

This displays the current epoch and its loss.

The output might look like:

Epoch: 1 Loss: 1.20
Epoch: 2 Loss: 0.85
Epoch: 3 Loss: 0.60
Epoch: 4 Loss: 0.42
Epoch: 5 Loss: 0.30

If the loss is decreasing in a healthy training process, that can indicate that the model is improving on the training objective.

Putting Everything Together

Now we can understand the entire code as one sequence.

epochs = 5
learning_rate = 0.1

for epoch in range(epochs):

    prediction = model(x_train)

    loss = calculate_loss(
        prediction,
        y_train
    )

    gradients = calculate_gradients(loss)

    update_weights(
        gradients,
        learning_rate
    )

    print(
        "Epoch:",
        epoch + 1,
        "Loss:",
        loss
    )

Read it like this:

Repeat 5 times:

    1. Give data to the model
    2. Get prediction
    3. Calculate loss
    4. Calculate gradients
    5. Update weights
    6. Show the loss

Then repeat.

Complete Training Flow

                 Training Data
                       ↓
                 model(x_train)
                       ↓
                   Prediction
                       ↓
              calculate_loss(...)
                       ↓
                     Loss
                       ↓
          calculate_gradients(loss)
                       ↓
                   Gradients
                       ↓
       update_weights(gradients, lr)
                       ↓
                Updated Weights
                       ↓
                    Repeat

The key point is that the updated weights are used in the next forward pass.

Old Weights
     ↓
Prediction
     ↓
Loss
     ↓
Gradients
     ↓
New Weights
     ↓
Prediction again
     ↓
Loss again
     ↓
...

Simple Numerical Example

Imagine that the model has one weight.

Initial Weight = 0.50
Learning Rate  = 0.10

During the first training step, suppose the model calculates a gradient of 0.40.

New Weight
=
0.50 - (0.10 × 0.40)

=
0.46

During the next training step, suppose the gradient becomes 0.20.

New Weight
=
0.46 - (0.10 × 0.20)

=
0.44

So the simplified learning process looks like:

Weight = 0.50
     ↓
Weight = 0.46
     ↓
Weight = 0.44
     ↓
Weight = ...
     ↓
Weight = ...

The weights are repeatedly adjusted based on the gradients calculated from the current loss.

Where Does Batch Size Fit?

Real neural networks usually process data in batches rather than sending the entire dataset through the model at once.

Suppose we have 1,000 training examples and a batch size of 100.

1,000 Training Examples
          ↓
     10 Batches
          ↓

Batch 1 → Forward → Loss → Backward → Update
Batch 2 → Forward → Loss → Backward → Update
Batch 3 → Forward → Loss → Backward → Update
...
Batch 10 → Forward → Loss → Backward → Update

          ↓

      One Epoch

Therefore, a more realistic structure has a loop inside another loop.

for epoch in range(epochs):

    for batch in batches:

        prediction = model(batch)

        loss = calculate_loss(
            prediction,
            batch.target
        )

        gradients = calculate_gradients(loss)

        update_weights(gradients)

The outer loop controls epochs. The inner loop processes the batches within each epoch.

Where Does Validation Loss Fit?

After the model has updated its weights using training data, we can evaluate it on separate validation data.

Training Data
      ↓
Prediction
      ↓
Training Loss
      ↓
Gradients
      ↓
Update Weights


Validation Data
      ↓
Prediction
      ↓
Validation Loss
      ↓
Evaluate Model

Validation loss is used to monitor generalization. It is normally not used to directly update the weights in the training step.

Easy Way to Remember the Code

Imagine a student learning mathematics.

Student solves a problem
        ↓
Gets an answer
        ↓
Checks how wrong the answer is
        ↓
Finds what went wrong
        ↓
Changes the approach
        ↓
Solves another problem
        ↓
Repeat

A neural network does something similar:

Model makes prediction
        ↓
Calculate loss
        ↓
Calculate gradients
        ↓
Update weights
        ↓
Make another prediction
        ↓
Repeat

This is the core idea behind neural network training.

Do Not Confuse These Terms

Epoch
→ One complete pass through the training dataset.


Batch
→ A smaller portion of the training dataset.


Iteration
→ One training update, usually for one batch.


Loss
→ Measures how wrong the prediction is.


Gradient
→ Shows how the loss changes with respect to parameters.


Learning Rate
→ Controls the size of parameter updates.


Weight
→ A parameter learned by the neural network.

How to Read the Code Yourself

When you see a neural network training loop, look for these five things first.

1. Where is the data?
       ↓
   x_train / batch


2. Where is the prediction?
       ↓
   model(...)


3. Where is the loss?
       ↓
   calculate_loss(...)


4. Where are gradients calculated?
       ↓
   calculate_gradients(...)


5. Where are weights updated?
       ↓
   update_weights(...)

If you understand these five parts, you already understand the basic structure of a neural network training loop.

Remember This

Training Loop

for each epoch:

    1. Get training data
    2. Make prediction
    3. Calculate loss
    4. Calculate gradients
    5. Update weights
    6. Repeat

The model learns because
the weights change after each
training step.

The most important idea is: the Python training code is simply implementing the learning process we have already studied — forward pass, loss calculation, backpropagation, and weight updates, repeated many times.