Understand the Python Code

The Complete Code

First, look at the complete program before we break it apart.

import math


def relu(x):
    return max(0, x)


def sigmoid(x):
    return 1 / (1 + math.exp(-x))


# Input

x1 = 5
x2 = 8


# Hidden Neuron 1

z1 = (x1 * 0.4) + (x2 * 0.2) + 0.5
h1 = relu(z1)


# Hidden Neuron 2

z2 = (x1 * 0.1) + (x2 * 0.5) - 0.2
h2 = relu(z2)


# Hidden Layer Output

hidden_output = [h1, h2]


# Output Neuron

z_output = (
    (h1 * 0.6)
    + (h2 * 0.4)
    - 0.5
)


# Output Activation

prediction = sigmoid(z_output)


# Final Prediction

if prediction >= 0.5:
    result = "Pass"
else:
    result = "Fail"


print("Hidden Output:", hidden_output)
print("Raw Output:", z_output)
print("Probability:", prediction)
print("Prediction:", result)

First Understand the Flow

Before looking at individual lines, understand what the program is doing overall.

Input
  ↓
Hidden Neuron 1
  ↓
Hidden Neuron 2
  ↓
Hidden Layer Output
  ↓
Output Neuron
  ↓
Sigmoid
  ↓
Prediction
  ↓
Pass / Fail

Every part of the code belongs to one of these steps.

1. Importing the Math Module

import math

Python does not provide every mathematical function directly as a built-in function.

The math module provides useful mathematical operations.

We need it here because the Sigmoid function uses the exponential function:

math.exp()

For example:

import math

print(math.exp(1))

This calculates the mathematical value of e¹.

2. Creating the ReLU Function

def relu(x):
    return max(0, x)

This creates a Python function called relu.

The variable x represents the value we want to pass through ReLU.

ReLU follows this rule:

if x is positive:
    return x

if x is negative:
    return 0

Examples:

relu(5)
→ 5

relu(-3)
→ 0

relu(0)
→ 0

So when the neural network calculates:

h1 = relu(z1)

it is applying the ReLU activation function to the hidden neuron's raw value.

3. Creating the Sigmoid Function

def sigmoid(x):
    return 1 / (1 + math.exp(-x))

This function implements the Sigmoid activation function.

The mathematical formula is:

Sigmoid(x) = 1 / (1 + e⁻ˣ)

The Python version is:

1 / (1 + math.exp(-x))

For example:

sigmoid(0)
→ 0.5

sigmoid(3.68)
→ approximately 0.976

The output of Sigmoid is always between 0 and 1.

4. Defining the Input

x1 = 5
x2 = 8

These are the two input values entering our neural network.

For our example, we can imagine:

x1 = Study Hours
x2 = Attendance

So:

Input = [5, 8]

These values are passed into the hidden layer.

5. Calculating Hidden Neuron 1

z1 = (x1 * 0.4) + (x2 * 0.2) + 0.5

This line calculates the weighted sum for the first hidden neuron.

Remember the general neuron formula:

z = (input × weight) + (input × weight) + bias

Our code uses:

x1 = 5
x2 = 8

weight1 = 0.4
weight2 = 0.2

bias = 0.5

Therefore:

z1 = (5 × 0.4) + (8 × 0.2) + 0.5

z1 = 2.0 + 1.6 + 0.5

z1 = 4.1

So:

z1 = 4.1

6. Applying ReLU to Neuron 1

h1 = relu(z1)

We already calculated:

z1 = 4.1

So Python effectively performs:

h1 = relu(4.1)

h1 = 4.1

Because 4.1 is positive, ReLU returns 4.1.

The variable h1 now stores the output of hidden neuron 1.

7. Calculating Hidden Neuron 2

z2 = (x1 * 0.1) + (x2 * 0.5) - 0.2

This is another neuron, so it has its own weights and bias.

x1 = 5
x2 = 8

weight1 = 0.1
weight2 = 0.5

bias = -0.2

Calculate:

z2 = (5 × 0.1) + (8 × 0.5) - 0.2

z2 = 0.5 + 4.0 - 0.2

z2 = 4.3

Therefore:

z2 = 4.3

8. Applying ReLU to Neuron 2

h2 = relu(z2)

Since:

z2 = 4.3

Python calculates:

h2 = relu(4.3)

h2 = 4.3

Now the hidden layer has produced two outputs:

h1 = 4.1
h2 = 4.3

9. Storing the Hidden Layer Output

hidden_output = [h1, h2]

This creates a Python list containing both hidden-neuron outputs.

hidden_output = [4.1, 4.3]

Think of this as the information being passed from the hidden layer to the output layer.

10. Calculating the Output Neuron

z_output = (
    (h1 * 0.6)
    + (h2 * 0.4)
    - 0.5
)

Notice something important here:

h1
h2

The output neuron does not use the original x1 and x2 values directly.

It uses the values produced by the hidden layer.

Substitute the values:

h1 = 4.1
h2 = 4.3

The calculation becomes:

z_output =
    (4.1 × 0.6)
    + (4.3 × 0.4)
    - 0.5

z_output =
    2.46
    + 1.72
    - 0.5

z_output = 3.68

11. Creating the Final Output

prediction = sigmoid(z_output)

We already calculated:

z_output = 3.68

So Python calls:

prediction = sigmoid(3.68)

Which produces approximately:

prediction = 0.976

For this binary-classification example, that value can be interpreted as a probability-like output for the positive class, assuming the model is designed that way.

12. Making the Final Decision

if prediction >= 0.5:
    result = "Pass"
else:
    result = "Fail"

This is normal Python if/else logic.

The program checks:

prediction >= 0.5

Since:

0.976 >= 0.5

the condition is true.

Therefore:

result = "Pass"

Notice that the neural network produced the numeric output. The if/else statement is our decision rule for converting that output into a class.

13. Printing the Results

print("Hidden Output:", hidden_output)
print("Raw Output:", z_output)
print("Probability:", prediction)
print("Prediction:", result)

These lines display the values calculated by the program.

The output will look approximately like:

Hidden Output: [4.1, 4.3]
Raw Output: 3.68
Probability: 0.9759...
Prediction: Pass

Follow the Data Through the Code

The easiest way to understand the program is to follow the values as they move through it.

x1 = 5
x2 = 8
   ↓
z1 = 4.1
z2 = 4.3
   ↓
h1 = 4.1
h2 = 4.3
   ↓
hidden_output = [4.1, 4.3]
   ↓
z_output = 3.68
   ↓
prediction = 0.976
   ↓
result = "Pass"

That is the entire program in one view.

Python Code vs Neural Network Math

Another Simple Example

Let's change only the input:

x1 = 2
x2 = 3

The rest of the network stays the same.

Python will run the same calculations again:

New Input
[2, 3]
   ↓
Hidden Layer
   ↓
New Hidden Outputs
   ↓
Output Layer
   ↓
New Prediction

This is an important property of a model: once the network is defined, different input data can be passed through the same calculations.

How This Relates to Real Neural Networks

Our example has only:

2 inputs
2 hidden neurons
1 output neuron

Real neural networks can have many more layers and neurons.

Input
  ↓
Hidden Layer 1
  ↓
Hidden Layer 2
  ↓
Hidden Layer 3
  ↓
...
  ↓
Output Layer
  ↓
Prediction

The basic idea is still the same: each layer performs calculations using the output from the previous layer.

Complete Code Flow

import math
    ↓
Create ReLU
    ↓
Create Sigmoid
    ↓
Define Input
    ↓
Calculate Hidden Neuron 1
    ↓
Apply ReLU
    ↓
Calculate Hidden Neuron 2
    ↓
Apply ReLU
    ↓
Store Hidden Outputs
    ↓
Calculate Output Neuron
    ↓
Apply Sigmoid
    ↓
Get Prediction
    ↓
Use if/else
    ↓
Pass / Fail

What This Code Does NOT Do

This program performs a forward pass only.

It does not:

Calculate loss
Calculate gradients
Update weights
Train the model

The weights in our example are fixed:

0.4
0.2
0.1
0.5
0.6
0.4

In a real training process, these values would be learned from data.

Forward Pass vs Training

FORWARD PASS

Input
  ↓
Neural Network
  ↓
Prediction


TRAINING

Input
  ↓
Neural Network
  ↓
Prediction
  ↓
Loss
  ↓
Backpropagation
  ↓
Update Weights
  ↓
Repeat

This distinction is important. A forward pass is one calculation. Training repeats forward passes while adjusting the model's parameters to reduce the error.