What is fine-tuning?

Giving a general-purpose language model a new specialty takes more than clever prompting. When you need reliable, consistent performance on a specific task — drafting insurance policy summaries, classifying support tickets, extracting fields from clinical notes — you teach the model directly. Fine-tuning does exactly that: it continues training an already capable model on a curated set of examples so the model adapts its internal behaviour to the task, rather than depending on fragile word-level instructions. It’s not Retrieval-Augmented Generation (RAG), which supplies outside facts at query time without changing the model, and it’s not prompt engineering, which only guides behaviour with words. Fine-tuning changes the model itself.

How fine-tuning changes the model

A pre-trained language model arrives with broad knowledge from training on vast amounts of text. Its internal parameters — the weights that shape its predictions — are already set. Fine-tuning presents a new, task-specific dataset and updates a subset of those weights. Usually most layers stay frozen: their learned patterns remain intact while the top layers adjust to reflect the new examples. This limited update preserves general language skills and makes the model markedly better on the target task without needing an enormous dataset.

When a task is deeply domain-specific, engineers may unfreeze more layers or retrain everything. That demands more data and compute, but the model gains deeper fluency. The goal is never to erase the original training — it’s to add a new skill on a stable foundation.

When to choose fine-tuning over prompt engineering

Prompt engineering works when instructions alone define the task: “summarise this in two sentences”. But for tasks that hinge on a particular format, style, or internal logic that’s hard to capture in a prompt, fine-tuning becomes the practical choice. A brand voice that must feel consistent across many replies, or structured extraction from messy reports, often cannot depend on carefully worded prompts alone — small rewordings can break output quality. Fine-tuning shows the model many correct examples so it internalises the pattern, producing far more consistent results.

The trade-off is effort. Prompt engineering is fast to iterate; fine-tuning demands a curated training set, compute resources, and periodic retuning when the domain changes. But you get behaviour that holds steady.

Fine-tuning versus retrieval-augmented generation

RAG and fine-tuning solve different problems, and they’re often complementary. RAG gives the model access to an external knowledge base at response time — like having a reference book open. That makes it ideal for answering questions where facts change (product prices, current policies) or where citing a source matters. Fine-tuning, by contrast, bakes the skill into the model’s weights. There’s no lookup overhead, and it excels at style, tone, and structure, but it won’t automatically know new facts without retraining.

Many practical systems combine both. A support bot might be fine-tuned to follow a specific apologetic tone and an escalation flow, while a RAG pipeline fetches the latest stock levels or policy details. That gives you consistent behaviour and fresh, sourceable information.

What you’ll need in practice

Fine-tuning is not a one-click solution. You need a high-quality dataset — examples that faithfully represent the task — and enough variety to let the model generalise rather than memorise. Overfitting is a real risk with smaller datasets, and even after deployment you’ll want to monitor for drift: if the domain shifts, the model may need a fresh round of training. The effort pays off when the task demands consistency and depth that prompt tweaking alone cannot reach. For many teams, the starting point is asking: can we teach this through examples? If the answer is yes, fine-tuning is worth exploring.