Getting started
Should you fine-tune, or is it a prompt problem?
When is fine-tuning worth it compared to prompt engineering or retrieval?
Fine-tune when the behaviour you want is hard to describe in words, when a long prompt has become a real per-request cost, or when the model is right most of the time but not reliably enough. Use retrieval for facts that change, and prompts for everything you can still solve with them.
Try the prompt first, always
If a prompt solves the problem, you have solved it in an afternoon with no training data, no evaluation set and no model to maintain. That is a better outcome than a successful fine-tune, and it is worth genuinely exhausting before moving on.
The prompt is also not wasted work if it fails. A prompt that gets you to eighty per cent tells you precisely what a fine-tune needs to learn, and it usually becomes the thing that generates your first training examples.
The three signals that it is time
Any one of these justifies the work. All three together usually means the fine-tune will pay for itself within weeks.
- The prompt has grown long enough to be a real cost. A two-thousand-token system prompt on every request is money you are spending to repeat yourself, and fine-tuning moves it into the weights.
- It works most of the time but not reliably enough. Schema compliance in the low nineties is the classic case: correct values in a broken envelope, needing a repair layer you would rather delete.
- You cannot describe what you want. "Confident but not arrogant" is not actionable in a prompt and is perfectly learnable from examples of people choosing one draft over another.
Three cases where it is the wrong answer
- You want the model to know facts that change. Fine-tuned facts go stale, cannot be updated individually, and are stated with total confidence when wrong. Use retrieval.
- You have fewer than a few hundred consistent examples. Below that threshold the model learns your inconsistencies rather than your task.
- The task is genuinely open-ended. A model doing one job on one distribution shrinks well; a model doing everything does not, and no amount of training changes that.
Retrieval and fine-tuning are not alternatives
This is put as a choice far more often than it should be. Retrieval decides what the model sees. Fine-tuning decides what it does with what it sees. A production system usually wants both.
The specific thing fine-tuning contributes to a retrieval system is faithfulness: using the supplied context rather than the model’s own recollection, and declining when the context does not support an answer. That behaviour is what makes such a system trustworthy, and it is learned from examples of refusals.
Decide with a number, not an argument
The honest way to settle it is to build the fine-tune and gate it against what you are running today, blinded, on held-out cases from your own traffic. A training run on an 8B model over a few thousand examples is a small enough cost that measuring beats speculating.
A fail is not a wasted run. It is a cheap answer to an expensive question, and it usually points at data rather than at the idea.
Frequently asked questions
How much does it cost to find out?
A LoRA run on an 8B model over a few thousand examples is typically well under an hour of GPU time. At mainstream class rates that is a small fraction of what a month of the prompt-based version costs at any real volume.
Can fine-tuning make a model worse?
Yes, and it commonly does when the dataset is inconsistent or too small, or when training runs for too many epochs. This is exactly what an evaluation gate is for — it catches the regression before your users do.
Terms used here
More on getting started
Last verified 6 August 2026.
Start with the free tier
A magic link creates your account, your tenant and your first API key. No card until you ask for compute.