Supervised fine-tuning
Structured data extraction
How do I fine-tune a model to return reliable JSON from messy documents?
Fine-tune on your own document-to-schema pairs. The gain is not accuracy on any single field — it is schema compliance approaching certainty, which removes the retry-and-repair layer that prompt-based extraction always needs.
| Objective | Supervised fine-tuning |
|---|---|
| Dataset shape | Raw document text in, a JSON object matching your schema out. One to three thousand pairs. |
| Metric that decides it | Schema validity rate and per-field exact match on a held-out set, measured separately. |
| Cheapest starting point | LFM2 1.2B on RTX 3080$0.09 per GPU-hour |
Prompted extraction fails in a specific and expensive way: it is usually right about the values and intermittently wrong about the shape. A missing bracket, a field rendered as a string instead of a number, an array where the schema wanted an object. Every one of those needs a retry, and retries are what make extraction pipelines slow and costly.
Fine-tuning fixes the shape problem almost completely, because the output format is the most learnable thing in the dataset. Teams routinely see schema validity move from the low nineties to above ninety-nine per cent, and the repair layer can then be deleted rather than tuned.
Measure the two things separately. A model that returns valid JSON with the wrong values is a different failure from one that returns the right values in a broken envelope, and averaging them into one score hides which problem you have.
The mistake to avoid
Training only on documents that parsed cleanly. The model never learns what to do with the malformed inputs that are the entire reason you needed a model, and it will hallucinate confidently when it meets one.
Models to start from
In order. Start at the top and move down the list only if the evaluation gate says you have to — the smallest model that clears your bar is the one you pay for on every request forever.
Before you deploy it
Whatever the metric above says in isolation, the deployment decision is comparative: is this better than what is running today? Gate the candidate against the incumbent, blinded, on held-out cases from your own traffic. A fail is a cheap answer to an expensive question.
Frequently asked questions
Does fine-tuning replace constrained decoding?
It reduces how much you need it. Constrained decoding guarantees shape but cannot help with values; a fine-tune improves both and makes the constraint rarely bind. Using them together is reasonable.
What if my schema changes?
A small additive change usually survives. A structural change means a new training set and a new run — which is cheap enough at this model size that it is not worth engineering around.
Start with LFM2 1.2B
Upload your dataset, read the validation report, and forecast the run before anything is leased.