Understand the Python Code

The Complete Code

First, let's look at the complete program before breaking it into individual parts.

import numpy as np


# Training data
X = np.array([
    [0, 0],
    [0, 1],
    [1, 0],
    [1, 1]
])

y = np.array([
    [0],
    [1],
    [1],
    [1]
])


# Model parameters
weights = np.array([
    [0.0],
    [0.0]
])

bias = 0.0


# Activation function
def sigmoid(x):
    return 1 / (1 + np.exp(-x))


# Prediction function
def predict(X):
    z = np.dot(X, weights) + bias
    return sigmoid(z)


# Training
learning_rate = 0.1

for epoch in range(1000):

    predictions = predict(X)

    error = predictions - y

    gradient_weights = np.dot(
        X.T,
        error
    ) / len(X)

    gradient_bias = np.mean(error)

    weights -= learning_rate * gradient_weights
    bias -= learning_rate * gradient_bias


# Test the model
predictions = predict(X)

print("Predictions:")
print(predictions)

print("Weights:")
print(weights)

print("Bias:")
print(bias)

1. Import NumPy

import numpy as np

This imports the NumPy library.

NumPy gives us arrays and mathematical operations that we need to work with the neural network.

2. Create the Training Data

X = np.array([
    [0, 0],
    [0, 1],
    [1, 0],
    [1, 1]
])

X contains the input data.

There are four training examples, and each example contains two input values.

So:

X = inputs

3. Create the Correct Answers

y = np.array([
    [0],
    [1],
    [1],
    [1]
])

y contains the correct output for each training example.

Therefore:

X = input data
y = correct answers

4. Create the Weights

weights = np.array([
    [0.0],
    [0.0]
])

These are the initial weights of our neuron.

Each input has its own weight.

During training, these weights will change.

5. Create the Bias

bias = 0.0

The bias is another value that helps the neuron shift its output.

The basic calculation is:

weighted inputs + bias

The bias is also updated during training.

6. Create the Sigmoid Function

def sigmoid(x):
    return 1 / (1 + np.exp(-x))

This function takes a number and converts it into a value between 0 and 1.

This is useful for our simple binary prediction example.

7. Create the Prediction Function

def predict(X):
    z = np.dot(X, weights) + bias
    return sigmoid(z)

This function performs the forward calculation.

First:

z = np.dot(X, weights) + bias

This calculates the weighted sum of the inputs and adds the bias.

Then:

return sigmoid(z)

The result is passed through the sigmoid function to produce the prediction.

8. Set the Learning Rate

learning_rate = 0.1

The learning rate controls how large each weight update should be.

Choosing a good learning rate is important because updates that are too large or too small can make training difficult.

9. Start the Training Loop

for epoch in range(1000):

This repeats the training process 1,000 times.

Each complete pass through the loop gives the model another opportunity to adjust its parameters.

10. Make Predictions

predictions = predict(X)

The model uses the current weights and bias to make predictions for all training examples.

At this point, the predictions are based on the model's current knowledge.

11. Calculate the Error

error = predictions - y

This compares the model's predictions with the correct answers.

If the prediction is different from the correct answer, the error will show that the model needs to improve.

12. Calculate the Weight Gradients

gradient_weights = np.dot(
    X.T,
    error
) / len(X)

This calculates the direction in which the weights should change to reduce the error.

You do not need to memorize this formula yet. The important idea is:

13. Calculate the Bias Gradient

gradient_bias = np.mean(error)

This calculates the gradient for the bias.

Just like the weights, the bias also needs to be adjusted during training.

14. Update the Weights

weights -= learning_rate * gradient_weights

This is one of the most important lines in the program.

It changes the weights using the learning rate and the calculated gradient.

The goal is to move the weights in a direction that reduces the model's error.

15. Update the Bias

bias -= learning_rate * gradient_bias

The bias is updated in a similar way.

The Training Loop Together

for epoch in range(1000):

    predictions = predict(X)

    error = predictions - y

    gradient_weights = np.dot(
        X.T,
        error
    ) / len(X)

    gradient_bias = np.mean(error)

    weights -= learning_rate * gradient_weights
    bias -= learning_rate * gradient_bias

Every iteration follows the same process.

16. Make the Final Predictions

predictions = predict(X)

After training is complete, we use the learned weights and bias to make predictions again.

The predictions should now be closer to the expected outputs than they were at the beginning.

17. Print the Results

print("Predictions:")
print(predictions)

print("Weights:")
print(weights)

print("Bias:")
print(bias)

These statements display what the model learned and the predictions it produces.

Follow One Example Through the Network

Let's take one input:

X = [1, 0]

The model combines the inputs with their weights:

z = (1 × weight_1)
  + (0 × weight_2)
  + bias

The result is then passed through the sigmoid function:

prediction = sigmoid(z)

The Complete Mental Model