Understand the Python Code

The Complete Python Code

First, look at the complete code. Then we will break it down line by line.

input_value = 2
target = 10

weight = 3.0
learning_rate = 0.1

for step in range(5):

    # Forward pass
    prediction = input_value * weight

    # Calculate loss
    loss = (prediction - target) ** 2

    # Calculate gradient
    gradient = 2 * (prediction - target) * input_value

    # Update weight
    weight = weight - learning_rate * gradient

    print("Step:", step + 1)
    print("Prediction:", prediction)
    print("Loss:", loss)
    print("Weight:", weight)
    print()

This small program contains the basic idea behind gradient-based learning.

Step 1 — Define the Input

input_value = 2

This is the input that we give to our model.

input_value
     ↓
     2

In a real machine-learning problem, the input could be something much larger, such as house size, age, height, temperature, or hundreds of other features.

Step 2 — Define the Target

target = 10

The target is the correct answer that the model is trying to learn.

Model Prediction →  ?
Correct Answer   →  10

During training, the model compares its prediction with this target.

Step 3 — Start With a Weight

weight = 3.0

The model starts with a weight of 3.

The weight controls how strongly the input affects the prediction.

Input = 2
Weight = 3

Prediction
= 2 × 3
= 6

The starting weight is not good enough because the target is 10.

Step 4 — Define the Learning Rate

learning_rate = 0.1

The learning rate controls how much the weight changes during each update.

Large learning rate
→ Larger weight changes


Small learning rate
→ Smaller weight changes

Here we use:

learning_rate = 0.1

Step 5 — Repeat the Learning Process

for step in range(5):

This tells Python to repeat the training process five times.

Step 1
Step 2
Step 3
Step 4
Step 5

Why repeat it?

Because one weight update is usually not enough. The model needs multiple updates to gradually improve.

Step 6 — Forward Pass

prediction = input_value * weight

This is the forward pass.

Python multiplies the input by the current weight.

input_value = 2
weight = 3

prediction = 2 * 3

prediction = 6

So the model currently predicts 6.

Step 7 — Calculate the Loss

loss = (prediction - target) ** 2

This calculates the squared error between the prediction and the target.

The Python operator ** means exponent.

(prediction - target) ** 2

Means:

(prediction - target) ×
(prediction - target)

With our values:

prediction = 6
target = 10

loss = (6 - 10) ** 2

loss = (-4) ** 2

loss = 16

The model's current loss is 16.

Step 8 — Calculate the Gradient

gradient = 2 * (prediction - target) * input_value

This is the most important line in the example.

It calculates how the loss changes when the weight changes.

With our values:

prediction = 6
target = 10
input_value = 2

gradient
= 2 * (6 - 10) * 2

= 2 * (-4) * 2

= -16

Therefore:

gradient = -16

The negative sign tells the optimizer that the weight should move upward in this particular example.

Step 9 — Update the Weight

weight = weight - learning_rate * gradient

This is the gradient descent update rule.

New Weight
=
Old Weight
-
Learning Rate × Gradient

Insert our values:

weight = 3
learning_rate = 0.1
gradient = -16

weight
= 3 - (0.1 × -16)

= 3 + 1.6

= 4.6

The weight changes from 3 to 4.6.

Step 10 — The Model Tries Again

The loop starts the next iteration using the new weight.

Input = 2
New Weight = 4.6

Prediction
= 2 × 4.6

Prediction = 9.2

The prediction improved significantly.

Before Update

Prediction = 6


After Update

Prediction = 9.2


Target = 10

Step 11 — Check the New Loss

The next iteration calculates the loss again.

loss = (9.2 - 10) ** 2

loss = (-0.8) ** 2

loss = 0.64

The loss has changed from:

Old Loss = 16
New Loss = 0.64

A much smaller loss means the model's prediction is now much closer to the target.

What the Loop Is Really Doing

for step in range(5):

    prediction = input_value * weight
             ↓
        Make prediction

    loss = (prediction - target) ** 2
             ↓
        Measure error

    gradient = 2 * (prediction - target) * input_value
             ↓
        Find direction of change

    weight = weight - learning_rate * gradient
             ↓
        Update weight

Then Python goes back to the beginning of the loop and repeats everything with the updated weight.

Step 12 — Print the Results

print("Step:", step + 1)
print("Prediction:", prediction)
print("Loss:", loss)
print("Weight:", weight)

These lines allow us to see what is happening during training.

For example:

Step: 1
Prediction: 6
Loss: 16
Weight: 4.6

The next iterations use the updated weight.

What Should Happen During Training?

As the model learns, we generally want the loss to move downward.

Training Step

Loss
16
 ↓
0.64
 ↓
smaller
 ↓
smaller
 ↓
smaller

The exact values depend on the data, learning rate, model, and loss function, but the goal is to reduce the loss.

Python Code With Comments

# Input value
input_value = 2

# Correct answer
target = 10

# Starting weight
weight = 3.0

# Controls how much the weight changes
learning_rate = 0.1


# Repeat the training process
for step in range(5):

    # -------------------------
    # Forward pass
    # -------------------------
    prediction = input_value * weight


    # -------------------------
    # Calculate loss
    # -------------------------
    loss = (prediction - target) ** 2


    # -------------------------
    # Calculate gradient
    # -------------------------
    gradient = 2 * (prediction - target) * input_value


    # -------------------------
    # Update weight
    # -------------------------
    weight = weight - learning_rate * gradient


    # -------------------------
    # Show results
    # -------------------------
    print("Step:", step + 1)
    print("Prediction:", prediction)
    print("Loss:", loss)
    print("Weight:", weight)
    print()

Connect the Python Code to the Concepts

Python Code
                    Concept

input_value
        →           Input

weight
        →           Model Parameter

prediction
        →           Forward Pass

loss
        →           Loss Function

gradient
        →           Backpropagation

learning_rate
        →           Step Size

weight = weight - ...
        →           Weight Update

for loop
        →           Repeated Training

Once you understand this mapping, the code becomes much easier to read.

How This Looks in Real Deep Learning

In frameworks such as PyTorch or TensorFlow, you normally do not calculate every gradient manually.

The framework automatically performs much of the backpropagation work.

Model
  ↓
Prediction
  ↓
Loss
  ↓
Backward Pass
  ↓
Gradients
  ↓
Optimizer
  ↓
Updated Weights

The manual Python example is important because it shows what is happening underneath those high-level APIs.

The Entire Lesson in One Picture

Input
  │
  ▼
Forward Pass
  │
  ▼
Prediction
  │
  ▼
Calculate Loss
  │
  ▼
Backpropagation
  │
  ▼
Calculate Gradient
  │
  ▼
Learning Rate
  │
  ▼
Update Weight
  │
  ▼
New Prediction
  │
  ▼
Lower Loss
  │
  ▼
Repeat

Remember This

prediction = input × weight

loss = (prediction - target)²

gradient
= 2 × (prediction - target) × input

new weight
= old weight - learning_rate × gradient

These four ideas are enough to understand the basic mechanics of our simple backpropagation example.