Supervised Fine-Tuning

A language model is first trained on huge amounts of text from the internet. Afterward, it can predict astonishingly well which word fits next. What it still can’t do: answer a question instead of simply continuing to write it. Supervised Fine-Tuning is the step that makes up for exactly that. To do this, the model is shown thousands of examples, each consisting of a task and a matching sample answer. The word “supervised” means here: for every input, the desired output is already provided.

From text continuer to assistant

Without this step, a chatbot would be practically unusable. If you ask a purely pretrained model “What is the capital of Norway?”, it might answer with “And what is the capital of Sweden?”. It learned from the internet that questions are often followed by more questions. That is statistically correct and yet useless. Supervised Fine-Tuning teaches the model that a question deserves an answer.

The second reason is economic. Pretraining a large model costs tens of millions of dollars and takes months. A fine-tuning run often gets by with a few thousand to a few hundred thousand examples. Depending on model size, it runs in hours or days. That’s why smaller companies and research groups can also adapt an openly available model to their own needs instead of starting from scratch.

That is exactly what companies do with specialized knowledge that barely appears on the open internet. A law firm trains a model on its own legal briefs, a hospital on medical reports. The result often matches the tone and technical language in this narrow domain better than a much larger general-purpose model.

What’s in the example data

The raw material consists of pairs of input and desired output. One line might contain the instruction “Summarize this text in three sentences” plus the text, along with the finished summary. Such pairs are written by paid experts, or collected from vetted sources. Some of them now also come from other, more powerful AI models. What matters is quality, not quantity: a few thousand carefully written examples often beat millions of sloppy ones.

Technically, the training proceeds like pretraining, just with different material. The model generates its own answer to an input. This is compared with the sample answer, and the deviation is measured. Afterward, a computational procedure shifts the model’s internal dials minimally in the direction that makes the error smaller. These dials are called parameters, and large models have billions of them.

Often, not all parameters are changed at all. Methods like LoRA attach only a small additional layer to the model and leave the rest untouched. This saves memory and compute time considerably. A common misconception is also to confuse Supervised Fine-Tuning with the step that follows it. In so-called Reinforcement Learning, humans rate several answers and rank them by quality. In fine-tuning, by contrast, there is exactly one predefined sample solution per task.

Where the term appears in news and products

Anyone downloading a model like Llama or Mistral usually finds two variants. One carries the suffix “base”, the other “instruct” or “chat”. The second has undergone Supervised Fine-Tuning and can be used directly as an assistant. The first is raw material for developers who want to continue training it themselves.

The major providers also sell this step as a service. OpenAI, Google and others offer to adapt their models using uploaded example files. Companies use this, for example, so that a support chatbot reliably gets the company’s own product names and characteristic tone of address right. In technical articles, the abbreviation SFT then appears, often in a list alongside other training stages.

A recurring topic in the news is the dispute over training data. When a provider fine-tunes its model using answers from a competing model, this often violates that competitor’s terms of use. Such accusations have already been raised publicly several times. They show just how valuable good example data has become.