Fine-tuning

Fine-tuning in one picture

Pre-training gives a model broad capabilities. Fine-tuning starts from those learned parameters instead of starting from random values. You then expose the model to examples that represent a narrower target behavior.

Why fine-tune a model?

Fine-tuning is useful when repeated prompts are not enough to reliably produce the behavior you need. The training examples can teach consistent output structure, terminology, classification behavior, or a domain-specific response style.

Important: fine-tuning is not automatically the best way to add changing company facts. If the model needs fresh external knowledge, retrieval can be a better fit.

Step 1: design a useful fine-tuning dataset

The model learns from the examples you provide. Good examples make the desired behavior clear and consistent. Poor examples can teach the wrong behavior just as efficiently.

Step 2: structure training examples

For supervised fine-tuning, each example describes an input and the response the model should learn to produce. The exact format depends on the training API, but the teaching idea is the same: show the model the behavior, not just a description of it.

training_example = {
    "messages": [
        {"role": "user", "content": "Classify: My card was charged twice."},
        {"role": "assistant", "content": "billing"},
    ]
}

print(training_example)

This example teaches a mapping from a customer message to a target label. A real dataset would contain many carefully reviewed examples covering the variations you expect.

Step 3: train with small, careful updates

Fine-tuning usually starts from a capable model, so you generally do not need the huge updates used to learn language from scratch. Training settings such as learning rate, batch size, number of epochs, and sequence length affect how strongly the dataset changes the model.

Watch for overfitting

A model can become too specialized to the examples it saw. If training loss keeps improving while performance on held-out examples gets worse, the model may be memorizing the training set instead of learning a useful general pattern.

Fine-tuning vs other techniques

Decision rule: if your problem is “the model does not know today's policy,” think retrieval. If the problem is “the model repeatedly fails to follow this output behavior,” fine-tuning may be appropriate.

Run a small behavior experiment

You can understand the core idea without training a large model. The following Python example uses a tiny text classifier to show what focused training does: the model starts with general numeric features and learns a narrower mapping from examples to labels.

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression

texts = [
    "card charged twice",
    "refund is missing",
    "password reset link",
    "cannot sign in",
]
labels = ["billing", "billing", "account", "account"]

vectorizer = TfidfVectorizer()
X = vectorizer.fit_transform(texts)
model = LogisticRegression(max_iter=1000)
model.fit(X, labels)

new_message = ["I was charged twice"]
prediction = model.predict(vectorizer.transform(new_message))
print(prediction[0])