Training an LLM

The big picture: training is learning from prediction errors

Training an LLM is an optimization loop. The model sees token sequences, predicts what should come next, compares those predictions with the known next tokens, calculates a loss, and adjusts its parameters so that future predictions are better.

Step 1: prepare training data

Training starts with a large corpus of text. Before the model sees it, the data must be collected, cleaned, filtered, deduplicated, normalized where appropriate, and converted into a form suitable for training.

Important: “more data” is not automatically “better data.” Data quality, diversity, duplication, licensing, safety filtering, and mixture design all matter.

Step 2: convert text into training sequences

The tokenizer converts text into token IDs. Those IDs are arranged into sequences that can be processed in batches. For next-token prediction, each position provides a target for the following token.

That one-token shift is the basic teaching signal behind next-token prediction. Real training uses many sequences and many positions at once.

Step 3: run a forward pass

The model receives a batch of token IDs and computes predictions. Inside the network, embeddings, attention, feed-forward layers, normalization, and other operations transform the representations until the model produces output scores for the vocabulary.

Step 4: calculate the loss

The training system compares the model's predicted distribution with the correct target tokens. A loss function turns that difference into a number that tells us how well the model performed on the batch.

For language-model training, cross-entropy is a common loss family. You do not need to memorize the formula yet; understand what the loss is doing: turning prediction quality into a signal the optimizer can use.

Step 5: backpropagation finds how parameters contributed to the error

The model contains parameters—learned numerical values such as weights. Backpropagation uses the loss to calculate gradients, which indicate how changing those parameters would affect the loss.

Step 6: the optimizer updates the parameters

An optimizer uses gradients to change the parameters. A simplified update looks like:

The learning rate controls how large the update steps are. Too large can make training unstable; too small can make learning painfully slow.

The training loop repeats—many, many times

One batch produces one set of predictions, loss values, gradients, and updates. Training repeats this process over many batches.

See the idea in Python

You can observe the core training loop with a tiny neural network. This is not an LLM—it is deliberately small so you can see the mechanics without needing a large GPU.

import torch
from torch import nn

# Tiny dataset: learn y = 2x
x = torch.tensor([[1.0], [2.0], [3.0], [4.0]])
y = torch.tensor([[2.0], [4.0], [6.0], [8.0]])

model = nn.Linear(1, 1)
loss_fn = nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)

for step in range(500):
    prediction = model(x)          # forward pass
    loss = loss_fn(prediction, y)  # measure error

    optimizer.zero_grad()
    loss.backward()                 # gradients
    optimizer.step()                # update parameters

print(model(torch.tensor([[5.0]])))

How this scales to an LLM

An LLM training run applies the same basic loop to a vastly larger model and dataset. Instead of four tiny examples and one parameterized layer, there may be huge datasets, many layers, very large batches, distributed hardware, mixed-precision arithmetic, optimizer state, checkpoints, and carefully designed evaluation pipelines.

The algorithmic pattern is still the same: predict → measure error → compute gradients → update parameters → repeat.

Training is not just “make the loss smaller”

A model can become very good at the training data without generalizing well. Training systems therefore monitor held-out validation data and other evaluations.

Why training an LLM is expensive

Large-scale training requires substantial compute, memory, storage, networking, data processing, experimentation, and engineering. The model parameters must be updated across enormous numbers of training examples.

Training, inference, and fine-tuning are different

For example, when your Python application sends a prompt to an existing LLM API, you are normally doing inference, not training the base model.

Mini-project: watch a model learn

Use the notebook with this lesson to train the tiny regression model, print the loss during training, and change the learning rate.