Supervised fine-tuning
PII redaction
How do I fine-tune a model to redact personal data in our documents?
Train on your own documents with your own redaction decisions. What counts as identifying is contextual and jurisdictional, and a general model applies a generic definition that is wrong in both directions.
| Objective | Supervised fine-tuning |
|---|---|
| Dataset shape | Original text in, span-annotated redactions out, with the category for each span. |
| Metric that decides it | Recall on each PII category, reported separately. Missing one identifier is not offset by catching a hundred. |
| Cheapest starting point | Qwen3 1.7B on RTX 3080$0.09 per GPU-hour |
Redaction is asymmetric in a way that most metrics hide. A false positive means a redacted word that did not need to be; a false negative means personal data left in a published document. Those are not comparable outcomes, and any single score that averages them is the wrong score.
Context decides. An employee number is identifying inside your organisation and meaningless outside it. A date of birth is identifying; a year of birth may not be. Those judgements are in your redaction history and nowhere else.
Small models are sufficient and preferable: this is span classification, it runs at volume, and a smaller model is easier to keep inside a controlled environment.
The mistake to avoid
Reporting one aggregate accuracy figure. Report recall per category, and treat any category below your threshold as a blocker regardless of the average.
Models to start from
In order. Start at the top and move down the list only if the evaluation gate says you have to — the smallest model that clears your bar is the one you pay for on every request forever.
Before you deploy it
Whatever the metric above says in isolation, the deployment decision is comparative: is this better than what is running today? Gate the candidate against the incumbent, blinded, on held-out cases from your own traffic. A fail is a cheap answer to an expensive question.
Frequently asked questions
Is a fine-tune enough on its own?
No. Pair it with deterministic pattern matching for structured identifiers. The model catches the contextual cases regular expressions cannot; regular expressions catch the formatted ones the model may miss.
How do I evaluate without exposing the data?
Evaluate inside the same controlled environment the model runs in. Weights and artefacts are downloadable, so the whole loop can stay where the data already is.
Start with Qwen3 1.7B
Upload your dataset, read the validation report, and forecast the run before anything is leased.