Weights and Bias

A Neuron Uses Inputs, Weights, and Bias

A neuron receives input values. Each input is combined with a weight, and then a bias is added.

With multiple inputs, each input has its own weight.

Input 1 × Weight 1
Input 2 × Weight 2
Input 3 × Weight 3
        +
      Bias
        ↓
     Neuron

What Is a Weight?

A weight controls how strongly an input affects a neuron's calculation.

Think of a weight as an importance value.

A larger absolute weight generally means that the corresponding input has a stronger influence on the neuron's weighted sum.

Example — Student Prediction

Suppose a neuron receives information about a student:

Suppose the neuron has these weights:

The neuron multiplies each input by its corresponding weight.

7 × 0.5 = 3.5

90 × 0.2 = 18

The two weighted values are then combined.

3.5 + 18 = 21.5

This is part of the neuron's calculation. We will add the bias next.

What Is Bias?

Bias is an additional value that is added to the weighted inputs.

It gives the neuron more flexibility when producing its output.

Without bias, the neuron would have less freedom to adjust its calculation.

Example — Adding Bias

From the previous example, the weighted inputs gave us:

3.5 + 18 = 21.5

Now suppose the neuron has a bias of:

Bias = 2

The neuron adds the bias:

21.5 + 2 = 23.5

So the value before the activation function would be:

23.5

The Basic Neuron Formula

For multiple inputs, the basic calculation can be written as:

z = (x₁ × w₁) + (x₂ × w₂) + ... + b

Where:

Complete Example

Let's calculate the output of a simple neuron.

Suppose we have:

x₁ = 2
x₂ = 3

w₁ = 0.5
w₂ = 0.4

b = 1

First multiply each input by its weight:

2 × 0.5 = 1.0

3 × 0.4 = 1.2

Add the weighted values:

1.0 + 1.2 = 2.2

Finally, add the bias:

2.2 + 1 = 3.2

What Happens After Weights and Bias?

The weighted sum plus bias is usually passed through an activation function before the neuron produces its final output.

Activation functions are covered in detail in Lesson 3 — Activation Functions.

Does the Network Choose the Weights and Bias?

The important part is that we normally do not manually choose the final weights and bias.

During training, the neural network adjusts them based on its prediction errors.

The same idea applies to the bias. Training adjusts these parameters so the model can make better predictions.

A Simple Way to Think About Weights and Bias

Imagine a teacher deciding whether a student is likely to pass.

The teacher might consider different factors:

The teacher may consider some factors more strongly than others. This is similar to how weights control the influence of different inputs.

The bias is like an additional adjustment that shifts the overall calculation.

Weight vs Bias

Why Are Weights and Bias Important?

Without adjustable weights and bias, a neural network would not have the same ability to learn useful relationships from training data.

Training repeatedly adjusts these values so the network can improve its predictions.

Complete Neuron

Inputs
  ↓
Multiply by weights
  ↓
Add the results
  ↓
Add bias
  ↓
Activation function
  ↓
Neuron output