Feature Extraction

What Is Feature Extraction?

A neural network can learn different features from data. In an image model, early layers can learn simple patterns such as edges and lines, while deeper layers can learn shapes and more complex visual patterns.

With feature extraction, we take a pretrained model and reuse the features it has already learned.

Input Image
      ↓
Pretrained Model
      ↓
Learned Features
      ↓
New Classifier
      ↓
Prediction

Instead of asking the model to learn everything again, we reuse the existing feature-learning part.

Why Do We Use Feature Extraction?

Training a deep neural network from scratch can require a large amount of data and computing power.

A pretrained model has already learned many useful patterns, so we can reuse those patterns for our new problem.

Training From Scratch

Random Model
      ↓
Learn Edges
      ↓
Learn Shapes
      ↓
Learn Objects
      ↓
Learn Your Task


Feature Extraction

Pretrained Model
      ↓
Already Learned Features
      ↓
Your Dataset
      ↓
New Classifier
      ↓
Prediction

This can make training faster and can be especially useful when our new dataset is relatively small.

What Happens to the Pretrained Model?

In feature extraction, we normally freeze the pretrained layers. Freezing means that their weights are not updated during training on the new dataset.

Pretrained Layers
        ↓
    FROZEN
        ↓
Extract Features
        ↓
New Classification Layer
        ↓
    TRAINED
        ↓
Prediction

The pretrained layers keep their existing knowledge while the new classification layer learns the new task.

Simple Example — Flower Classification

Imagine we have a pretrained image model that already understands many general visual patterns.

We want to build a model that identifies different types of flowers.

Flower Image
      ↓
Pretrained Model
      ↓
Edges
Shapes
Textures
Visual Features
      ↓
New Classifier
      ↓
Rose / Tulip / Lily

We don't need to teach the pretrained model basic visual features again. We only need to train the new classifier for our flower categories.

Example — Cats and Dogs

Suppose we have a pretrained image model and want to classify images as either cats or dogs.

             Pretrained Model
                    │
                    │
          ┌─────────┴─────────┐
          │                   │
     Learned Features     Frozen Layers
          │
          ↓
     New Classifier
          │
          ↓
     Cat or Dog

The pretrained model provides general visual features. The new classifier learns how those features relate to cats and dogs.

Feature Extraction vs Training From Scratch

TRAINING FROM SCRATCH

Random Weights
      ↓
Train All Layers
      ↓
Learn Features
      ↓
Learn New Task
      ↓
Final Model


FEATURE EXTRACTION

Pretrained Weights
      ↓
Freeze Pretrained Layers
      ↓
Extract Features
      ↓
Train New Classifier
      ↓
Final Model

The biggest difference is that feature extraction starts with knowledge that has already been learned.

Feature Extraction in Python

TensorFlow and Keras provide pretrained models that can be used for feature extraction.

For example, we can use MobileNetV2 as the pretrained feature extractor.

import tensorflow as tf

base_model = tf.keras.applications.MobileNetV2(
    weights="imagenet",
    include_top=False,
    pooling="avg"
)

base_model.trainable = False

The model is loaded with pretrained ImageNet weights. Setting trainable to False freezes the pretrained layers.

Add a New Classifier

After loading the pretrained feature extractor, we can add our own classification layer.

model = tf.keras.Sequential([
    base_model,
    tf.keras.layers.Dense(2, activation="softmax")
])

The final layer has two outputs because our example has two classes.

Output 0 → Cat
Output 1 → Dog

The pretrained model extracts features, and the new classifier uses those features to make the final prediction.

Complete Simple Example

import tensorflow as tf

# Load pretrained model
base_model = tf.keras.applications.MobileNetV2(
    weights="imagenet",
    include_top=False,
    pooling="avg"
)

# Freeze pretrained layers
base_model.trainable = False

# Build new model
model = tf.keras.Sequential([
    base_model,
    tf.keras.layers.Dense(2, activation="softmax")
])

# Compile model
model.compile(
    optimizer="adam",
    loss="sparse_categorical_crossentropy",
    metrics=["accuracy"]
)

model.summary()

MobileNetV2 provides the pretrained features, while the final Dense layer is used for the new classification task.

The Main Idea to Remember

Pretrained Model
        ↓
Freeze Its Layers
        ↓
Use It To Extract Features
        ↓
Add New Classifier
        ↓
Train New Classifier
        ↓
Make Predictions

Feature extraction does not mean manually finding features. The neural network extracts the useful features for us.

Quick Check