Build the Neural Network

What Are We Building?

Our goal is to create an image classification neural network.

Input Image
    ↓
28 × 28 pixels
    ↓
Flatten
    ↓
Dense Layer
    ↓
Dense Layer
    ↓
Output Layer
    ↓
10 digits

The network will eventually learn to answer a simple question:

"Which digit is this image?"

Possible answers:

0
1
2
3
4
5
6
7
8
9

Import TensorFlow

import tensorflow as tf

TensorFlow provides the tools we need to create and train the neural network.

We use the Keras API inside TensorFlow because it makes building neural networks much easier to read and write.

Create a Sequential Model

model = tf.keras.Sequential([
    ...
])

Sequential means that our layers are arranged one after another.

Layer 1
  ↓
Layer 2
  ↓
Layer 3
  ↓
Layer 4

Data enters the first layer and moves forward through the network until it reaches the output layer.

Add the Flatten Layer

tf.keras.layers.Flatten(
    input_shape=(28, 28)
)

Our image has two dimensions:

28 × 28

A basic Dense network expects a one-dimensional list of values, so Flatten converts the image into:

28 × 28
   ↓
784 values

The calculation is simple:

28 × 28 = 784

Add the First Dense Layer

tf.keras.layers.Dense(
    128,
    activation="relu"
)

This creates a fully connected layer containing 128 neurons.

784 input values
       ↓
128 neurons
       ↓
Next Layer

Each neuron receives information from the previous layer and learns weights that help it detect useful patterns.

What Is a Neuron Doing?

A neuron receives input values, multiplies them by learned weights, adds a bias, and then applies an activation function.

Inputs
  ↓
Multiply by weights
  ↓
Add bias
  ↓
Activation function
  ↓
Output

During training, the network changes these weights and biases so that its predictions become more accurate.

Why Use ReLU?

activation="relu"

ReLU is a commonly used activation function in neural networks.

In simple terms, it keeps positive values and turns negative values into zero.

ReLU(x) = max(0, x)

Example:

x = -5
ReLU(-5) = 0

x = 3
ReLU(3) = 3

This gives the network the non-linear behavior it needs to learn more complex patterns.

Add Another Dense Layer

tf.keras.layers.Dense(
    64,
    activation="relu"
)

Now we add another layer with 64 neurons.

784
 ↓
128
 ↓
64

The first Dense layer can learn useful combinations of pixel information, while later layers can combine those learned features into more useful representations.

Add the Output Layer

tf.keras.layers.Dense(
    10,
    activation="softmax"
)

Why 10 neurons?

Because MNIST has 10 possible classes:

0  1  2  3  4
5  6  7  8  9

Each output neuron represents one possible digit.

Understand Softmax

Softmax converts the output values into probabilities.

0 → 0.01
1 → 0.02
2 → 0.01
3 → 0.03
4 → 0.01
5 → 0.02
6 → 0.01
7 → 0.85
8 → 0.02
9 → 0.02

The highest probability is for digit 7.

Prediction = 7

The probabilities across all classes add up to approximately 1.

Complete Model Code

import tensorflow as tf

model = tf.keras.Sequential([

    tf.keras.layers.Flatten(
        input_shape=(28, 28)
    ),

    tf.keras.layers.Dense(
        128,
        activation="relu"
    ),

    tf.keras.layers.Dense(
        64,
        activation="relu"
    ),

    tf.keras.layers.Dense(
        10,
        activation="softmax"
    )
])

Understand the Complete Architecture

Input
28 × 28
784 values
    ↓
Flatten
    ↓
128 Neurons
ReLU
    ↓
64 Neurons
ReLU
    ↓
10 Neurons
Softmax
    ↓
Digit Prediction

This is the architecture we will train using the prepared MNIST dataset.

Compile the Model

model.compile(
    optimizer="adam",
    loss="sparse_categorical_crossentropy",
    metrics=["accuracy"]
)

Building the architecture is not enough. We also need to tell the model how it should learn.

The three important settings are:

optimizer
    ↓
How weights are updated

loss
    ↓
How prediction errors are measured

accuracy
    ↓
How often predictions are correct

Why Use Adam?

optimizer="adam"

Adam is an optimization algorithm that adjusts the network's weights during training.

Prediction
    ↓
Calculate Error
    ↓
Adam adjusts weights
    ↓
New Prediction
    ↓
Calculate Error Again
    ↓
Continue Learning

You do not manually calculate how every weight should change. The optimizer handles that process.

Why Use Sparse Categorical Crossentropy?

loss="sparse_categorical_crossentropy"

Our labels are simple integer class numbers:

0
1
2
3
...
9

Sparse categorical crossentropy is suitable when the target labels are stored this way.

For example:

Image → handwritten 7

Correct label:
7

Model prediction:
0.85 for class 7

Loss:
Measures how good or bad
that prediction was.

Why Track Accuracy?

metrics=["accuracy"]

Accuracy tells us the percentage of predictions that are correct.

100 images

Correct predictions = 95

Accuracy = 95%

Accuracy is useful for understanding how well the model is classifying the digits.

View the Model Summary

model.summary()

This prints information about the model's layers and parameters.

Model
────────────────────────────
Flatten
Dense (128)
Dense (64)
Dense (10)
────────────────────────────

It also shows how many trainable parameters the model contains.

The Complete Build Code

import tensorflow as tf

# Create neural network
model = tf.keras.Sequential([

    # Convert 28 × 28 image into 784 values
    tf.keras.layers.Flatten(
        input_shape=(28, 28)
    ),

    # First hidden layer
    tf.keras.layers.Dense(
        128,
        activation="relu"
    ),

    # Second hidden layer
    tf.keras.layers.Dense(
        64,
        activation="relu"
    ),

    # Output layer
    tf.keras.layers.Dense(
        10,
        activation="softmax"
    )
])

# Configure the model
model.compile(
    optimizer="adam",
    loss="sparse_categorical_crossentropy",
    metrics=["accuracy"]
)

# Display architecture
model.summary()

What Happens When an Image Enters?

Handwritten 7
     ↓
28 × 28 pixels
     ↓
Flatten
     ↓
784 numbers
     ↓
128 neurons
     ↓
64 neurons
     ↓
10 output neurons
     ↓
Softmax
     ↓
Highest probability
     ↓
7

During training, the network does not initially know that the image is a 7. It gradually adjusts its weights by comparing predictions with the correct labels.

Simple Example

Imagine the model receives an image containing a handwritten 3.

Input:

Handwritten 3

        ↓

Model Prediction:

0 → 0.02
1 → 0.01
2 → 0.04
3 → 0.80  ← highest
4 → 0.01
5 → 0.03
6 → 0.01
7 → 0.02
8 → 0.03
9 → 0.03

        ↓

Prediction = 3

During training, the model compares this prediction with the correct label and adjusts its weights to improve.

Quick Check