DEEP LEARNING • LESSON 14

Pretrained Models

A pretrained model is a neural network that has already been trained on a large dataset. Instead of starting from random weights, we can reuse the knowledge learned by that model for our own problem.

SIMPLE IDEA

A pretrained model is a model that has already learned.

We can take that existing knowledge and use it as a starting point for a new machine learning task.

What Is a Pretrained Model?

Normally, when we create a neural network, its weights start with random values.

We then provide training data and allow the model to learn useful patterns.

A pretrained model has already gone through this process. Someone has already trained the model using a large dataset.

Normal Model

Random Weights
      ↓
Training Data
      ↓
Training
      ↓
Learn Patterns
      ↓
Trained Model


Pretrained Model

Large Dataset
      ↓
Training
      ↓
Learn Patterns
      ↓
Pretrained Model
      ↓
Reuse It

So, instead of starting from zero, we start with a model that already contains useful learned information.

Why Are Pretrained Models Useful?

Training a large neural network from scratch can require millions of training examples and a lot of computing resources.

A pretrained model allows us to reuse work that has already been completed.

Without Pretrained Model

Your Dataset
      ↓
Train Everything
      ↓
Learn Basic Features
      ↓
Learn Complex Features
      ↓
Learn Your Task


With Pretrained Model

Pretrained Model
      ↓
Already Learned Features
      ↓
Your Dataset
      ↓
Adapt To Your Task

This is one of the main reasons pretrained models are widely used in modern deep learning.

What Does a Pretrained Image Model Learn?

Consider a model that has been trained on millions of images.

During training, the model can learn different levels of visual patterns.

Image
  ↓
Edges
  ↓
Lines
  ↓
Shapes
  ↓
Textures
  ↓
Parts of Objects
  ↓
Objects

For example, early layers may learn simple patterns such as edges and lines, while deeper layers can learn more complex shapes and object features.

These learned features can sometimes be useful for a completely different image classification task.

Example — Image Classification

Imagine that a pretrained model was trained on a very large image dataset.

Now we want to build a model that identifies different types of flowers.

Instead of training the complete model from scratch, we can reuse the pretrained model.

Pretrained Image Model
        ↓
Learned Visual Features
        ↓
Flower Dataset
        ↓
Train New Classification Layer
        ↓
Rose
Tulip
Sunflower
Lily

The original model provides general visual knowledge, while the new training teaches it how to distinguish between the flower classes.

Example — Plant Disease Detection

Suppose we want to create a system that detects whether a plant leaf is healthy or diseased.

We may not have millions of plant images. However, we can start with a pretrained image model.

Pretrained Model
        ↓
Visual Features
        ↓
Plant Leaf Dataset
        ↓
Train For New Task
        ↓
Healthy
   OR
Diseased

The pretrained model already understands many general visual patterns. Our new dataset teaches it which patterns are associated with plant diseases.

Popular Pretrained Models

Many pretrained models are available for different types of machine learning problems.

For computer vision, examples include:

ResNet
VGG
MobileNet
EfficientNet
Inception

These models can be used as a starting point for many image-related tasks.

In Natural Language Processing, pretrained models such as BERT and other Transformer-based models can provide learned language representations.

Pretrained Model vs Model From Scratch

Model From Scratch

Random Weights
      ↓
Large Dataset
      ↓
Long Training
      ↓
Learn Features
      ↓
Learn Task
      ↓
Final Model


Pretrained Model

Pretrained Weights
      ↓
Smaller New Dataset
      ↓
Adapt Model
      ↓
Learn New Task
      ↓
Final Model

The main difference is where the model starts.

A model trained from scratch starts with random weights. A pretrained model starts with weights learned from previous training.

Pretrained Models and Transfer Learning

Pretrained models are the foundation of Transfer Learning.

Transfer Learning means taking knowledge from an existing trained model and applying that knowledge to a new task.

Pretrained Model
        ↓
Existing Knowledge
        ↓
Transfer Learning
        ↓
New Dataset
        ↓
New Task

So remember the difference:

Pretrained Model
=
Already Trained Model


Transfer Learning
=
Using That Knowledge
For A New Task

Important Point

A pretrained model is not automatically perfect for every problem.

The model should have learned features that are useful for the new task.

Related Problem
      +
Pretrained Model
      ↓
Good Starting Point


Very Different Problem
      +
Unrelated Pretrained Model
      ↓
May Not Work Well

Choosing the right pretrained model is therefore an important part of Transfer Learning.

KEY TAKEAWAY

Pretrained models give us a head start.

A pretrained model has already learned useful patterns from a large dataset. We can reuse those learned weights instead of starting from random weights.

Quick Check

What is a pretrained model?

A model that has already been trained on a dataset and has learned useful patterns.

Why use one?

It gives us a useful starting point and can reduce the amount of training needed for a new task.

What is the connection to Transfer Learning?

Transfer Learning uses knowledge from a pretrained model and applies it to a new task.