A Simple Forward Pass

The Complete Forward Pass

We will use a very small neural network:

Input Layer
     ↓
Hidden Layer
     ↓
Output Layer
     ↓
Prediction

Our network will have:

2 input values
2 hidden neurons
1 output neuron

Step 1 — Input Data

Suppose we want to predict whether a student will pass an exam.

We use two features:

x1 = Study Hours
x2 = Attendance

For one student:

x1 = 5
x2 = 8

So our input is:

Input = [5, 8]

Our Small Neural Network

Our network looks like this:

                 Hidden Layer
                ┌───────────────┐
                │   Neuron 1    │
Input           │               │
[5, 8] ────────→│   Neuron 2    │
                │               │
                └───────┬───────┘
                        ↓
                  Output Neuron
                        ↓
                    Prediction

We will calculate every step manually.

Step 2 — Calculate Hidden Neuron 1

Neuron 1 has these weights and bias:

w1 = 0.4
w2 = 0.2
bias = 0.5

The formula is:

z = (x1 × w1) + (x2 × w2) + bias

Put our values into the formula:

z1 = (5 × 0.4) + (8 × 0.2) + 0.5

z1 = 2.0 + 1.6 + 0.5

z1 = 4.1

Now apply ReLU:

ReLU(4.1) = 4.1

Therefore:

Hidden Neuron 1 Output = 4.1

Step 3 — Calculate Hidden Neuron 2

Neuron 2 uses different weights and bias:

w1 = 0.1
w2 = 0.5
bias = -0.2

Calculate:

z2 = (5 × 0.1) + (8 × 0.5) - 0.2

z2 = 0.5 + 4.0 - 0.2

z2 = 4.3

Apply ReLU:

ReLU(4.3) = 4.3

Therefore:

Hidden Neuron 2 Output = 4.3

Step 4 — Hidden Layer Output

We now have the output from both hidden neurons:

Neuron 1 = 4.1
Neuron 2 = 4.3

So the complete hidden-layer output is:

Hidden Output = [4.1, 4.3]

Step 5 — Calculate the Output Neuron

The output neuron receives the hidden-layer outputs:

h1 = 4.1
h2 = 4.3

Suppose the output neuron has:

w1 = 0.6
w2 = 0.4
bias = -0.5

Calculate the weighted sum:

z = (h1 × w1) + (h2 × w2) + bias

z = (4.1 × 0.6) + (4.3 × 0.4) - 0.5

z = 2.46 + 1.72 - 0.5

z = 3.68

Step 6 — Apply the Output Activation

Because our example is a binary classification problem, we can use Sigmoid in the output layer.

Sigmoid(3.68) ≈ 0.976

Therefore, the model produces:

Output ≈ 0.976

This can be interpreted as approximately a 97.6% predicted probability for the positive class, assuming the model is designed and calibrated that way.

Step 7 — Make the Prediction

Suppose our decision threshold is 0.5.

0.976 >= 0.5

Therefore:

Prediction = Pass

Complete Calculation in One Place

Let's put the entire forward pass together.

INPUT
x1 = 5
x2 = 8


HIDDEN NEURON 1

z1 = (5 × 0.4) + (8 × 0.2) + 0.5
z1 = 4.1

h1 = ReLU(4.1)
h1 = 4.1


HIDDEN NEURON 2

z2 = (5 × 0.1) + (8 × 0.5) - 0.2
z2 = 4.3

h2 = ReLU(4.3)
h2 = 4.3


OUTPUT NEURON

z = (4.1 × 0.6) + (4.3 × 0.4) - 0.5
z = 3.68

prediction = Sigmoid(3.68)
prediction ≈ 0.976


FINAL PREDICTION

0.976 >= 0.5

Pass

See the Entire Forward Pass

                 INPUT
                [5, 8]
                   │
                   ▼
          ┌─────────────────┐
          │   Hidden Layer  │
          │                 │
          │  Neuron 1 = 4.1 │
          │  Neuron 2 = 4.3 │
          └────────┬────────┘
                   │
                   ▼
              [4.1, 4.3]
                   │
                   ▼
          ┌─────────────────┐
          │  Output Neuron  │
          │                 │
          │     z = 3.68    │
          └────────┬────────┘
                   │
                   ▼
               Sigmoid
                   │
                   ▼
                0.976
                   │
                   ▼
              Prediction
                 PASS

Why Is This Called a Forward Pass?

Notice the direction of information:

Input
  ↓
Hidden Layer
  ↓
Output Layer
  ↓
Prediction

The information only moves forward through the network.

We are not changing the weights or biases during this calculation. We are simply using the current values to produce a prediction.

Forward Pass vs Backpropagation

These are two different parts of neural-network training.

FORWARD PASS

Input
  ↓
Calculate Hidden Layers
  ↓
Calculate Output
  ↓
Prediction


BACKPROPAGATION

Prediction
  ↓
Calculate Error
  ↓
Calculate Gradients
  ↓
Update Weights

The forward pass tells the model what it currently predicts. Backpropagation helps the model learn from its error.

Complete Forward Pass With Python

We can implement the same calculation in Python:

import math


def relu(x):
    return max(0, x)


def sigmoid(x):
    return 1 / (1 + math.exp(-x))


# -------------------------
# Input
# -------------------------

x1 = 5
x2 = 8


# -------------------------
# Hidden Neuron 1
# -------------------------

z1 = (x1 * 0.4) + (x2 * 0.2) + 0.5

h1 = relu(z1)


# -------------------------
# Hidden Neuron 2
# -------------------------

z2 = (x1 * 0.1) + (x2 * 0.5) - 0.2

h2 = relu(z2)


# -------------------------
# Output Neuron
# -------------------------

z_output = (
    (h1 * 0.6)
    + (h2 * 0.4)
    - 0.5
)

prediction = sigmoid(z_output)


print("Hidden Output:", [h1, h2])
print("Raw Output:", z_output)
print("Prediction:", prediction)


if prediction >= 0.5:
    print("Class: Pass")
else:
    print("Class: Fail")

Approximate output:

Hidden Output: [4.1, 4.3]
Raw Output: 3.68
Prediction: 0.9759...
Class: Pass

What Just Happened?

We started with two input values:

[5, 8]

The first hidden neuron transformed them into:

4.1

The second hidden neuron transformed them into:

4.3

Together:

[4.1, 4.3]

The output neuron then transformed those values into:

0.976

Finally, our decision rule converted that output into:

Pass