DEEP LEARNING LESSON 11 RECURRENT NEURAL NETWORKS

Sequence Data

Sequence data is data where the order of the values matters. The current value can depend on information from previous values in the sequence.

What Is Sequence Data?

Sequence data is a collection of data points arranged in a particular order.

The important thing is that the order is meaningful.

For example:

10 → 20 → 30 → 40 → 50

This is a sequence because the values appear in a specific order.

If we change the order:

50 → 10 → 40 → 20 → 30

we have changed the sequence and potentially changed its meaning.

Why Does Order Matter?

Consider these two sequences:

Sequence A:

10 → 20 → 30
Sequence B:

30 → 20 → 10

Both sequences contain exactly the same numbers.

But they represent different patterns.

Sequence A

10 → 20 → 30
     increasing


Sequence B

30 → 20 → 10
     decreasing

Therefore, simply knowing the values is not enough. We also need to know their order.

Example 1: Text

A sentence is a sequence of words.

I → love → Python

The order creates the meaning.

Compare:

I love Python

Python love I

The same words are present, but the second sequence does not have the same meaning.

This is why text can be treated as sequence data.

Example 2: Time Series

A time series records values over time.

Monday    → 100
Tuesday   → 110
Wednesday → 120
Thursday  → 130
Friday    → 140

The values are connected to a specific time order.

If we rearrange them:

Wednesday → 120
Monday    → 100
Friday    → 140
Tuesday   → 110
Thursday  → 130

we lose the natural time sequence.

Other Examples of Sequence Data

Sequence data appears in many areas.

Text
↓
Word 1 → Word 2 → Word 3 → Word 4


Speech
↓
Sound 1 → Sound 2 → Sound 3 → Sound 4


Temperature
↓
Monday → Tuesday → Wednesday → Thursday


Stock Price
↓
Price 1 → Price 2 → Price 3 → Price 4


Sensor Data
↓
Reading 1 → Reading 2 → Reading 3 → Reading 4

The common feature is that the data has an order.

Sequence Does Not Always Mean Time

A common mistake is to think that every sequence must be time-based.

Time series are sequences, but not every sequence is a time series.

For example, this sentence is sequence data:

The → cat → is → sleeping

The words have an order, but they are not measurements taken at different times.

So:

Sequence Data
      ↓
Order matters


Time Series
      ↓
Sequence data where
the order represents time

What Is a Time Step?

When working with sequence data, each position in the sequence is often called a time step.

For example:

10 → 20 → 30 → 40 → 50
 ↑     ↑     ↑     ↑     ↑
 t1    t2    t3    t4    t5

Here we have five time steps.

t1 = 10
t2 = 20
t3 = 30
t4 = 40
t5 = 50

The word "time" is commonly used even when the sequence is not literally about clock time.

What Is a Feature in a Sequence?

At each time step, we can have one or more features.

Suppose we record temperature:

Monday    → 25
Tuesday   → 27
Wednesday → 29

There is one feature:

Temperature

But suppose we record temperature and humidity:

Monday    → [25, 60]
Tuesday   → [27, 65]
Wednesday → [29, 70]

Now each time step has two features:

Feature 1 → Temperature
Feature 2 → Humidity

Sequence Data Shape

When using neural networks, sequence data is commonly represented using three dimensions:

(samples, time_steps, features)

Each part has a simple meaning.

samples
↓
How many sequences we have


time_steps
↓
How many values are in each sequence


features
↓
How many values are available at each step

Understanding the Shape With an Example

Suppose we have 100 sequences.

Each sequence contains 5 time steps.

Each time step contains 2 features.

(100, 5, 2)

This means:

100 → sequences
5   → time steps per sequence
2   → features per time step

Visual example:

Sequence 1

Step 1 → [25, 60]
Step 2 → [27, 65]
Step 3 → [29, 70]
Step 4 → [28, 68]
Step 5 → [30, 72]

Here, every step contains two features.

Representing Sequence Data With Python

We can represent a simple sequence using a Python list:

sequence = [10, 20, 30, 40, 50]

print(sequence)

Output:

[10, 20, 30, 40, 50]

Each value appears in a specific position.

sequence[0]  # 10
sequence[1]  # 20
sequence[2]  # 30

Python uses zero-based indexing, so the first item is at index 0.

Why Previous Values Can Matter

One of the important properties of sequence data is that earlier values can provide context for later values.

Consider:

Monday    → 100
Tuesday   → 120
Wednesday → 140
Thursday  → ?

If the values have been consistently increasing, the previous values may help predict Thursday.

This is one reason sequence models such as RNNs are useful.

How Sequence Data Connects to RNNs

An RNN processes a sequence one step at a time.

Sequence

10 → 20 → 30 → 40 → 50
     ↓
    RNN
     ↓
Process one step at a time

More specifically:

10
 ↓
RNN
 ↓
Hidden State 1

20
 +
Hidden State 1
 ↓
RNN
 ↓
Hidden State 2

30
 +
Hidden State 2
 ↓
RNN
 ↓
Hidden State 3

The hidden state allows information from earlier steps to be carried forward.

Normal Data vs Sequence Data

Normal Data

[10, 20, 30]

The relationship between positions
may not depend on their order.


Sequence Data

10 → 20 → 30

The order itself carries information.

This distinction is important when choosing a neural network architecture.

Two Simple Examples

Example 1 — Sentence

I → am → learning → Python

Changing the order changes the sentence.

Example 2 — Temperature

25°C → 27°C → 29°C → 31°C

The order tells us how the temperature changed over time.

Simple Python Example

temperatures = [
    25,
    27,
    29,
    31,
    30
]

for temperature in temperatures:
    print(temperature)

The loop processes each value in sequence.

Output:

25
27
29
31
30

The important part is that Python reads the values from the first position to the last position.

The Main Idea

Sequence Data
      ↓
Data arranged in an order
      ↓
Order contains information
      ↓
Earlier values can provide context
      ↓
RNNs can process the sequence step by step

So, sequence data is not simply "a list of numbers." The important part is that the relationship between positions and their order carries meaning.

Final Summary

Sequence Data
↓
Data where order matters

Examples
↓
Text
Time Series
Speech
Sensor Data
Stock Prices

Time Step
↓
One position in the sequence

Features
↓
Values recorded at each time step

Common Shape
↓
(samples, time_steps, features)

Why Important?
↓
Previous values can provide context

RNN
↓
Processes sequence step by step
while carrying information forward

The simplest definition to remember is:

Sequence data is data where the order
of the data points carries useful information.
QUICK CHECK

Check Your Understanding

1. What is sequence data?
Data where the order of the data points carries meaning.

2. Give two examples.
Text and time-series data.

3. What is a time step?
One position or observation in a sequence.

4. What does the shape (100, 5, 2) mean?
100 sequences, 5 time steps per sequence, and 2 features at each time step.

5. Why are RNNs useful for sequence data?
They process the sequence in order and carry information from previous steps.