Creating a Neural Network

What Are We Creating?

Let's create a simple neural network that receives two input features and produces one output.

Input
  ↓
Hidden Layer
4 neurons
ReLU
  ↓
Output Layer
1 neuron
Sigmoid
  ↓
Output

For example, suppose our two inputs are:

Input 1 = hours studied
Input 2 = practice tests completed

The network could use these values to predict whether a student will pass an exam.

Hours Studied
       +
Practice Tests
       ↓
Neural Network
       ↓
0.92
       ↓
Likely to Pass

Step 1 — Import TensorFlow

First, we import TensorFlow.

import tensorflow as tf

We use TensorFlow because it provides Keras and the tools required to create and train the neural network.

import tensorflow as tf

tf.keras

The tf is simply a shorter name for TensorFlow.

Step 2 — Create the Model

Next, we create a Keras Sequential model.

model = tf.keras.Sequential()

This creates an empty neural network.

Empty Model
    ↓
No Layers Yet

We then add layers to this model.

Step 3 — Define the Input

Our example has two input features:

hours studied
practice tests

Therefore, the input contains two values.

Input(shape=(2,))

We can write:

model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(2,))
])

The (2,) means each training example has two features.

[hours_studied, practice_tests]

For example:

[5, 3]

means:

5 hours studied
3 practice tests completed

Step 4 — Add a Hidden Layer

Now we add a hidden layer containing four neurons.

tf.keras.layers.Dense(
    4,
    activation="relu"
)

The complete model becomes:

model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(2,)),

    tf.keras.layers.Dense(
        4,
        activation="relu"
    )
])

This means:

Input
2 features
   ↓
Dense Layer
4 neurons
ReLU

Each of the four neurons receives information from the two input features.

What Happens Inside a Neuron?

Each neuron performs a calculation using weights and a bias.

z = x₁w₁ + x₂w₂ + b

Then the activation function is applied:

output = ReLU(z)

Keras creates and manages the weights and biases for us.

For example, imagine one neuron has:

x₁ = 5
x₂ = 3

w₁ = 0.2
w₂ = 0.4
b  = 0.1

The neuron calculates:

z = (5 × 0.2) + (3 × 0.4) + 0.1

z = 1.0 + 1.2 + 0.1

z = 2.3

ReLU then produces:

ReLU(2.3) = 2.3

This is the same neuron calculation we learned earlier. Keras simply performs it automatically.

Step 5 — Add the Output Layer

Now we need an output layer.

Because our example predicts one binary result — pass or fail — we use one output neuron.

tf.keras.layers.Dense(
    1,
    activation="sigmoid"
)

Our complete model is now:

model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(2,)),

    tf.keras.layers.Dense(
        4,
        activation="relu"
    ),

    tf.keras.layers.Dense(
        1,
        activation="sigmoid"
    )
])

The structure is:

2 Inputs
   ↓
4 Hidden Neurons
   ↓
1 Output Neuron

Why Use Sigmoid in the Output?

Sigmoid converts the output into a value between 0 and 1.

0.0 → Very unlikely
0.5 → Around the middle
1.0 → Very likely

So if the model produces:

0.92

we can interpret this as a high probability of the positive class.

For our example:

0.92 → Likely to pass

This is why sigmoid is commonly used for binary classification.

Our Complete Neural Network

import tensorflow as tf

model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(2,)),

    tf.keras.layers.Dense(
        4,
        activation="relu"
    ),

    tf.keras.layers.Dense(
        1,
        activation="sigmoid"
    )
])

This creates the following network:

          Neural Network

Input 1 ─────┐
             │
Input 2 ─────┼──→ Hidden Layer ──→ Output
             │      4 neurons        1 neuron
             │      ReLU             Sigmoid
             │
             └──────────────────────────────→


Input shape = 2
Hidden neurons = 4
Output neurons = 1

Viewing the Model Structure

Keras provides summary() to show the model structure.

model.summary()

It displays information such as:

Layer
Output Shape
Number of Parameters
Total Parameters

This is useful when your network becomes larger and you need to check whether the architecture is correct.

Understanding Parameters

Neural networks learn parameters such as weights and biases.

For our first hidden layer:

Input features = 2
Neurons = 4

Each neuron needs two weights and one bias.

Weights:

2 × 4 = 8


Biases:

4


Total:

8 + 4 = 12 parameters

The output layer receives 4 values from the hidden layer and contains one neuron.

Weights:

4 × 1 = 4


Bias:

1


Total:

4 + 1 = 5 parameters

Therefore, the entire network has:

Hidden layer = 12 parameters
Output layer = 5 parameters

Total = 17 parameters

These parameters are what the neural network learns during training.

Example 2 — A Larger Network

We can create a network with more hidden neurons and another hidden layer.

model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(3,)),

    tf.keras.layers.Dense(
        8,
        activation="relu"
    ),

    tf.keras.layers.Dense(
        4,
        activation="relu"
    ),

    tf.keras.layers.Dense(
        1,
        activation="sigmoid"
    )
])

Its structure is:

3 Inputs
   ↓
8 Neurons
   ↓
4 Neurons
   ↓
1 Output

This network can learn more complex relationships than the first simple example, but that does not automatically mean it will perform better. A larger network can also overfit or become unnecessarily expensive.

What Happens When We Create the Model?

When Keras creates the Dense layers, it creates the parameters needed by those layers.

Dense Layer
    ↓
Create Weights
    ↓
Create Biases
    ↓
Ready for Training

The initial weights are not already correct. They are initialized automatically and will be adjusted during training.

This is important: creating a neural network does not mean the network has learned anything yet.

Creating vs Training a Neural Network

These are two different steps.

Creating
    ↓
Define the architecture


Training
    ↓
Learn the weights and biases

For example:

model = tf.keras.Sequential([
    ...
])

creates the structure.

Later:

model.fit(X_train, y_train)

trains that structure.

Complete Example

Here is a small example showing the complete model creation process.

import tensorflow as tf

# Input data
X_train = [
    [2, 0],
    [3, 1],
    [5, 2],
    [6, 3]
]

# Target values
y_train = [
    0,
    0,
    1,
    1
]


# Create neural network
model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(2,)),

    tf.keras.layers.Dense(
        4,
        activation="relu"
    ),

    tf.keras.layers.Dense(
        1,
        activation="sigmoid"
    )
])

# Display model structure
model.summary()

Notice that we have not trained the model yet.

Create Model
     ↓
Add Layers
     ↓
Model Ready
     ↓
Training comes next

Understand the Python Code

import tensorflow as tf

Imports TensorFlow.

model = tf.keras.Sequential([

Creates a Sequential neural network. The layers will be processed in the order in which they appear.

tf.keras.layers.Input(shape=(2,))

Defines an input containing two features.

tf.keras.layers.Dense(
    4,
    activation="relu"
)

Creates a hidden dense layer with four neurons and ReLU activation.

tf.keras.layers.Dense(
    1,
    activation="sigmoid"
)

Creates an output layer containing one neuron with sigmoid activation.

model.summary()

Displays information about the neural network's layers and parameters.

The Main Idea

Input
  ↓
Hidden Layers
  ↓
Output Layer
  ↓
Prediction

In Keras, creating this network is as simple as defining the layers.

model = tf.keras.Sequential([
    Input(...),
    Dense(...),
    Dense(...)
])

The important thing is not memorizing the syntax. You should understand what each layer is doing and why it is there.

Remember This

Input Layer
→ Receives the features


Hidden Layer
→ Learns patterns


Output Layer
→ Produces the final result


Dense()
→ Creates a fully connected layer


Sequential()
→ Connects layers in order


activation="relu"
→ ReLU activation


activation="sigmoid"
→ Output between 0 and 1

The most basic pattern to remember is:

model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(number_of_features,)),

    tf.keras.layers.Dense(
        number_of_neurons,
        activation="relu"
    ),

    tf.keras.layers.Dense(
        number_of_outputs,
        activation="sigmoid"
    )
])

In the next topic, we will go deeper into Adding Layers and understand how different layers change the structure and behavior of a neural network.