What is fine-tuning?

What is fine-tuning?

Adapting a pretrained model with examples of the behavior you want

A foundation model already knows language and broad patterns. Fine-tuning adjusts that model using a curated training dataset.

Base model → Training examples → Fine-tuned model

It changes the model's learned behavior, not just the prompt.

Think of fine-tuning as teaching a model a repeatable pattern through examples.

What can fine-tuning change?

It is most useful for consistent behavior, format, style, or task patterns

Format: Produce the same structured output shape again and again.

Style: Follow a consistent tone, wording pattern, or response style.

Classification: Improve performance on a narrow labeling or routing task.

Domain behavior: Learn examples of how your organization handles a specific task.

Tool use: Improve when and how a model selects tools in a repeated workflow.

Efficiency: Sometimes replace very long instructions or many-shot examples with learned behavior.

Fine-tuning is strongest when the target behavior is stable and can be demonstrated with good examples.

What fine-tuning does not do

It does not magically give the model live business knowledge

Live data: It does not automatically know today's inventory, orders, prices, or customer records.

Document search: It does not replace retrieval when answers depend on changing manuals, policies, or files.

Permissions: It does not give the model access to systems it was never connected to.

Perfect accuracy: A fine-tuned model can still make mistakes or hallucinate.

Bad data fix: Weak, inconsistent, or incorrect training examples can teach weak behavior.

If the problem is missing or changing knowledge, retrieval or tool access is usually the better answer.

Fine-tuning vs. RAG

One changes behavior. The other supplies relevant knowledge.

Fine-tuning

Changes learned behavior

Good for repeated style or task patterns

Requires a training dataset

Knowledge can become stale

Best when the desired behavior is stable

RAG

Supplies relevant external knowledge

Good for changing documents and data

No model retraining required

Can stay current as sources update

Best when answers depend on business information

Many real systems use both: fine-tuning for behavior and RAG for current knowledge.

When does fine-tuning make sense?

Use it when the task is repeated, measurable, and well defined

Good fit

You have many high-quality examples of the desired behavior

The same task happens often enough to justify training and testing

Prompting alone is inconsistent or too cumbersome

You can evaluate whether the tuned model actually improved

Bad first fit

You mainly need current business facts or documents

The process changes every week

You do not have clean training examples

A prompt, RAG system, or tool call already solves the problem well

Fine-tuning should solve a specific measured problem, not be used just because the option exists.