Lesson 3 · Foundations

What is Deep Learning?

Deep Learning is a type of machine learning that uses neural networks with multiple layers to learn increasingly useful patterns from data.

First: where does Deep Learning fit?

Deep Learning is not a separate world from AI and Machine Learning. It is one part of the larger AI field, and it is a powerful approach within machine learning.

Why do we need Deep Learning?

Traditional machine learning can work extremely well when we can describe the important signals in a useful way. But some problems involve raw, high-dimensional data such as images, audio, and natural language.

For these problems, deciding all the useful features by hand can be difficult. Deep learning can learn representations directly from the data, which is one reason it became so important for modern computer vision, speech, and language systems.

What is a neural network?

A neural network is a mathematical model made from connected units often called neurons. The network takes inputs, transforms them through layers, and produces an output.

What does each layer do?

• Input layer: receives the data, such as pixel values or numeric features.
• Hidden layers: transform the information and learn intermediate representations.
• Output layer: produces the final prediction, such as a class or probability.

Why is it called “deep” learning?

The word deep refers to the use of multiple computational layers. A network with more layers can build a hierarchy of representations, where later layers can combine patterns learned by earlier layers.

How does a neural network learn?

A neural network starts with parameters called weights. During training, it makes predictions, compares those predictions with the correct answers, calculates how wrong it was, and adjusts its weights to reduce that error.

This loop is repeated across many examples. Over time, the network can learn parameters that produce better predictions on data it has not seen before.

Build a small Deep Learning model in Python

Now let’s move from the idea to a real neural network. We will use TensorFlow and Keras to train a small image classifier on the MNIST handwritten-digit dataset. Each image is a 28 × 28 grayscale picture, and the model learns to predict which digit from 0 to 9 it contains.

import tensorflow as tf

# Load handwritten digit images
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()

# Scale pixel values from 0-255 to 0-1
x_train = x_train.astype("float32") / 255
x_test = x_test.astype("float32") / 255

model = tf.keras.Sequential([
    tf.keras.layers.Flatten(input_shape=(28, 28)),
    tf.keras.layers.Dense(128, activation="relu"),
    tf.keras.layers.Dense(10, activation="softmax")
])

model.compile(
    optimizer="adam",
    loss="sparse_categorical_crossentropy",
    metrics=["accuracy"]
)

model.fit(x_train, y_train, epochs=3, validation_split=0.1)

test_loss, test_accuracy = model.evaluate(x_test, y_test, verbose=0)
print("Test accuracy:", round(test_accuracy, 3))

The important part is not memorizing the code. Notice the learning process: the model receives many labeled examples, calculates a loss, updates its weights, and repeats this process over several training epochs.

Read the model from the inside out

1. Flatten: converts each 28 × 28 image into a one-dimensional sequence of values.
2. Dense(128): provides a hidden layer with 128 learned units and a ReLU activation.
3. Dense(10): produces ten output probabilities, one for each digit from 0 to 9.
4. Adam + loss: guides the training process so the weights can be updated toward better predictions.

What makes Deep Learning different from simpler ML?

Where is Deep Learning used?

One important limitation

Deep learning is powerful, but it is not automatically the best choice for every problem. Neural networks can require substantial data, compute, tuning, and careful evaluation. For a small structured dataset, a simpler machine learning model can sometimes be easier, faster, and more appropriate.

Recap

• Deep Learning is a branch of machine learning based on neural networks with multiple layers.
• Layers can learn increasingly useful representations from data.
• Training repeatedly compares predictions with correct answers and updates model weights.
• Deep learning is especially useful for complex data such as images, audio, and language.
• Deep learning is powerful, but simpler ML methods can still be the right choice for some problems.