Input to Hidden Layer

How Input Reaches the Hidden Layer

Suppose our neural network receives two inputs:

x1 = 2
x2 = 3

These values enter the input layer.

The hidden neuron does not simply add the inputs. It uses a different weight for each input.

Why Do We Need Weights?

Different inputs can have different levels of importance.

For example, suppose we are predicting whether a student will pass an exam.

x1 = Study Hours
x2 = Attendance

The model may learn that study hours and attendance should not have exactly the same influence.

w1 = 0.8
w2 = 0.3

The weights control how strongly each input affects the neuron.

Calculating the Weighted Inputs

Each input is multiplied by its corresponding weight.

x1 = 2
x2 = 3

w1 = 0.8
w2 = 0.3

First input:

x1 × w1

2 × 0.8

= 1.6

Second input:

x2 × w2

3 × 0.3

= 0.9

Adding the Weighted Inputs

Now we add the weighted inputs together.

1.6 + 0.9 = 2.5

So the weighted sum is:

weighted_sum = 2.5

Adding the Bias

The neuron also has a value called a bias.

Suppose:

bias = 0.5

Add it to the weighted sum:

z = weighted_sum + bias

z = 2.5 + 0.5

z = 3.0

The complete neuron calculation is therefore:

z = (x1 × w1) + (x2 × w2) + bias

Applying the Activation Function

The value calculated by the neuron is then passed through an activation function.

Suppose we use ReLU:

z = 3.0

ReLU(3.0) = 3.0

Therefore, the hidden neuron produces:

hidden_output = 3.0

Complete Example

Let's put everything together.

Input:
x1 = 2
x2 = 3

Weights:
w1 = 0.8
w2 = 0.3

Bias:
b = 0.5

Step 1 — Multiply each input by its weight:

2 × 0.8 = 1.6

3 × 0.3 = 0.9

Step 2 — Add them:

1.6 + 0.9 = 2.5

Step 3 — Add the bias:

2.5 + 0.5 = 3.0

Step 4 — Apply ReLU:

ReLU(3.0) = 3.0

Final hidden-neuron output:

3.0

What If the Hidden Layer Has Multiple Neurons?

A hidden layer normally contains multiple neurons. Each neuron has its own weights and bias.

             ┌── Neuron 1
Input ────────┼── Neuron 2
             └── Neuron 3

For the same inputs, each neuron can produce a different result because its weights and bias are different.

Example With Two Hidden Neurons

Suppose:

x1 = 2
x2 = 3

Neuron 1:

w1 = 0.8
w2 = 0.3
bias = 0.5

z = (2 × 0.8) + (3 × 0.3) + 0.5

z = 3.0

ReLU(3.0) = 3.0

Neuron 2 has different parameters:

w1 = 0.2
w2 = 0.7
bias = -0.5

z = (2 × 0.2) + (3 × 0.7) - 0.5

z = 1.6

ReLU(1.6) = 1.6

So the hidden layer produces:

Hidden Layer Output:

[3.0, 1.6]

The Mathematical Idea

For a neuron with multiple inputs, the calculation can be written as:

z = x1w1 + x2w2 + ... + xnw_n + b

Then the activation function is applied:

output = activation(z)

For a whole layer, this is commonly written using vectors and matrices:

z = XW + b

output = activation(z)

You don't need to memorize the matrix notation yet. The important idea is still the same:

Inputs
  ↓
Weights
  ↓
Weighted Sum
  ↓
Bias
  ↓
Activation
  ↓
Hidden Layer Output

Simple Python Example

We can represent the calculation directly in Python:

x1 = 2
x2 = 3

w1 = 0.8
w2 = 0.3

bias = 0.5

z = (x1 * w1) + (x2 * w2) + bias

output = max(0, z)

print(output)

Output:

3.0

Here, max(0, z) is the ReLU operation.

Complete Input-to-Hidden Flow

Input
  │
  ├── x1 = 2
  │
  └── x2 = 3
       │
       ▼
     Hidden Neuron
       │
       ├── Multiply by weights
       │
       ├── Add weighted values
       │
       ├── Add bias
       │
       ├── Apply activation
       │
       ▼
Hidden Layer Output

This is the first major calculation performed during forward propagation.