How LLMs Work

The big picture

When you send a prompt to an LLM, the model does not receive your sentence as raw English characters. The system converts the text into tokens, turns those tokens into numerical representations, processes them through the model, and produces probabilities for what should come next.

Step 1: text becomes tokens

LLMs work with tokens rather than directly with words. A tokenizer splits input text into pieces and maps those pieces to token IDs.

The exact split depends on the tokenizer and model. A token may represent a whole word, part of a word, punctuation, or another frequently occurring text fragment.

Step 2: tokens become vectors

A token ID is just an identifier. The neural network needs useful numerical representations, so tokens are mapped into vectors called embeddings.

These numbers do not mean “cat = these five properties.” Real embeddings have many dimensions, and their useful structure emerges from training. Tokens with related usage can develop representations that help the network model relationships in language.

Step 3: the Transformer processes the sequence

Modern LLMs are commonly based on the Transformer architecture. At a high level, Transformer layers repeatedly transform the representations of the input sequence.

For this lesson, you do not need every matrix operation. The important idea is that the network repeatedly mixes information and transforms representations so that later predictions can use relationships across the sequence.

Step 4: attention helps connect relevant context

Attention gives the model a mechanism for relating one position in a sequence to other positions. This helps the network decide which parts of the available context are useful when building a representation.

Step 5: the model produces next-token probabilities

After processing the current sequence, the model produces scores that can be converted into probabilities over possible next tokens.

Step 6: generation repeats the process

The model does not normally generate an entire answer in one magical step. In autoregressive generation, a token is selected, added to the sequence, and the model predicts the next token again.

Real generation also involves decoding choices, stopping rules, token limits, and other settings. You will study those ideas in later lessons.

What happens when your Python app calls an LLM?

The API hides the internal neural-network calculations, but the application-level flow is still useful to understand.

from openai import OpenAI

client = OpenAI(api_key=input("Enter your API key: "))

response = client.responses.create(
    model="gpt-5.6-luna",
    input="Explain attention in one simple paragraph."
)

print(response.output_text)

A tiny Python model to understand the idea

You can demonstrate the shape of next-token prediction without pretending that a few lines of Python are an LLM. Here, a tiny frequency table chooses the most common next word from a small dataset.

from collections import Counter, defaultdict

sentences = [
    "the cat sleeps",
    "the cat eats",
    "the dog sleeps",
]

next_words = defaultdict(Counter)

for sentence in sentences:
    words = sentence.split()
    for current_word, next_word in zip(words, words[1:]):
        next_words[current_word][next_word] += 1

print(next_words["the"].most_common())

This toy program captures one narrow idea: use previous context to estimate what comes next. Real LLMs replace this tiny lookup table with a huge learned neural network operating over token sequences.

Experiment: change the context

One of the most useful ways to understand LLM behavior is to change the input and observe how the generated continuation changes.

What you should remember