Supervised fine-tuning
Code review assistant
How do I fine-tune a model on my team’s code review standards?
Train on your merged pull requests paired with the review comments that were acted on. The value is your team’s conventions — which a general model cannot know and a style guide in a prompt only partly conveys.
| Objective | Supervised fine-tuning |
|---|---|
| Dataset shape | Diff in, review comment out. Only comments that led to a change. |
| Metric that decides it | Reviewer agreement rate on held-out pull requests, and false-positive rate on clean diffs. |
| Cheapest starting point | DeepSeek-R1 Distill Qwen 7B on RTX 3080$0.09 per GPU-hour |
A general model reviewing your code produces generically correct observations that your team already knows and does not want repeated. The useful review comments are the ones that reference your conventions, your historical incidents and your architectural decisions.
Filter hard. Use review comments that were followed by a change in the next commit, which is a reasonable proxy for the comment having been correct and worth making. Comments that were ignored are noise, and training on them teaches the model to generate noise.
Code-oriented bases are worth the extra parameters here. The reasoning-distilled DeepSeek models start with visible step-by-step behaviour, which makes their review comments easier to evaluate and easier to trust.
The mistake to avoid
Optimising for coverage. A reviewer that comments on everything gets muted within a week; measure the false-positive rate on clean diffs and weight it heavily.
Models to start from
In order. Start at the top and move down the list only if the evaluation gate says you have to — the smallest model that clears your bar is the one you pay for on every request forever.
Before you deploy it
Whatever the metric above says in isolation, the deployment decision is comparative: is this better than what is running today? Gate the candidate against the incumbent, blinded, on held-out cases from your own traffic. A fail is a cheap answer to an expensive question.
Frequently asked questions
How many pull requests do I need?
A few thousand acted-on review comments. Teams below that threshold usually get more value from a well-written prompt and a linter than from a fine-tune.
Will it catch real bugs?
Sometimes, but that is not the argument for it. The argument is consistency on convention and context, where a general model has nothing to work from.
Start with DeepSeek-R1 Distill Qwen 7B
Upload your dataset, read the validation report, and forecast the run before anything is leased.