Instruction Tuning

Instruction tuning in one picture

Pre-training gives a model broad language knowledge and the ability to predict tokens. Instruction tuning then uses curated examples that show an instruction, useful context when needed, and a desired response.

Before and after instruction tuning

A base language model is trained to predict what text is likely to come next. That objective does not automatically mean it will behave like a helpful assistant. Instruction examples add a clearer target: understand the request and produce an appropriate answer.

Key point: instruction tuning does not replace pre-training. It builds on the capabilities that pre-training already created.

Step 1: build instruction–response examples

Each example should make the intended task and desired response clear. A dataset can contain many different tasks as long as the responses consistently demonstrate useful instruction following.

Step 2: teach many kinds of instructions

Instruction tuning datasets often combine several task families. This helps a model learn a general pattern: read the request, infer the requested operation, and produce the expected kind of response.

Why variety matters: a single instruction pattern is not enough to create a broadly useful assistant. The dataset should cover the behaviors the model is expected to handle.

Step 3: represent the conversation

A common instruction-tuning example can be represented as messages with roles. The model sees the user's request and a target assistant response, then training encourages the model to produce the demonstrated response.

instruction_example = {
    "messages": [
        {"role": "user", "content": "Summarize: The team reduced response time by 30%."},
        {"role": "assistant", "content": "The team reduced response time by 30%."}
    ]
}

print(instruction_example)

The exact data format varies by training system. The important idea is the teaching pair: instruction → high-quality response.

Step 4: train on the target responses

During supervised instruction tuning, the model processes the example and learns to assign higher probability to the target response tokens. The optimization process still uses loss, gradients, and parameter updates, but the dataset now focuses on instruction-following examples.

Step 5: evaluate instruction following

A model can memorize the style of training examples without becoming genuinely useful. Evaluation should include instructions that were not present in training and should vary wording, difficulty, and task type.

Practical check: hold out evaluation prompts and include realistic variations rather than testing only examples that look like the training set.

Instruction tuning vs other techniques

Run a small instruction-following experiment

You can see the core idea without training a large language model. This Python example creates a tiny instruction dataset and applies the requested operation to make the instruction → response teaching signal concrete.

examples = [
    {"instruction": "uppercase", "input": "simple ai", "output": "SIMPLE AI"},
    {"instruction": "uppercase", "input": "learn llms", "output": "LEARN LLMS"},
]

new_input = "build with ai"
if examples[0]["instruction"] == "uppercase":
    result = new_input.upper()

print(result)

Change it: add examples for another instruction and make the target responses demonstrate the new behavior.