Supervised fine-tuning
Classification at volume
What is the cheapest way to classify millions of documents with an LLM?
Fine-tune the smallest model that clears your accuracy bar and serve it on an entry GPU class. At high volume the model size dominates the bill, and a 0.6B model that matches an 8B on your task is more than ten times cheaper to run.
| Objective | Supervised fine-tuning |
|---|---|
| Dataset shape | Document in, label out. Balanced across classes, with the rare classes deliberately over-sampled. |
| Metric that decides it | Per-class F1 on a held-out set, with the rarest class treated as the binding constraint. |
| Cheapest starting point | Qwen3 0.6B on RTX 3080$0.09 per GPU-hour |
At a million documents a month, the difference between a 0.6B and an 8B model is not a rounding error — it is the entire economics of the feature. Start at the bottom of the size ladder and move up only when the gate says you have to.
Fine-tuned tiny models are startlingly competitive on classification specifically, because the task needs discrimination rather than generation. Most of what a large model knows is irrelevant to picking one of twelve labels.
Over-sample the rare classes. Your aggregate accuracy will be dominated by the common ones, and the rare ones are usually the reason the classifier exists.
The mistake to avoid
Choosing the model size before measuring. Teams routinely deploy an 8B for a task a 1.7B handles identically, and then carry the difference every month forever.
Models to start from
In order. Start at the top and move down the list only if the evaluation gate says you have to — the smallest model that clears your bar is the one you pay for on every request forever.
Before you deploy it
Whatever the metric above says in isolation, the deployment decision is comparative: is this better than what is running today? Gate the candidate against the incumbent, blinded, on held-out cases from your own traffic. A fail is a cheap answer to an expensive question.
Frequently asked questions
How do I know when to stop going smaller?
Run the gate at each size. The smallest model that passes against your accuracy bar is the right one; the gate makes that a measurement rather than an argument.
Is scale-to-zero worth it here?
Only if the volume is bursty. Steady high volume keeps replicas warm anyway, so an always-warm endpoint is simpler and no more expensive.
Start with Qwen3 0.6B
Upload your dataset, read the validation report, and forecast the run before anything is leased.