---
title: Why fine-tuned models fail, and how to tell which failure you have
description: Five failure modes with the symptom that identifies each, in the order they are worth checking.
url: https://www.onrup.com/guides/why-fine-tuned-models-fail
category: Training
published: 2026-08-06
updated: 2026-08-06
source: Onrup
---

# Why fine-tuned models fail, and how to tell which failure you have

**Why did my fine-tuned model get worse instead of better?**

Almost always one of five causes: too many epochs, inconsistent data, silent truncation, a missing end-of-sequence token, or evaluating on contaminated data. Each has a distinctive symptom, and four of the five are visible before training starts.

## The model repeats itself or will not stop

Symptom: correct answers that keep going, or output that loops until it hits the length limit.

Cause: the end-of-sequence token is missing from your training targets, so the model never learned where to stop. This is a conversion bug rather than a training one, and it is the single most common cause of a fine-tune that looks broken.

## Answers stop mid-sentence

Symptom: responses that cut off partway through, consistently on longer inputs.

Cause: truncation. Your sequence length is shorter than your examples, and the end of the target was silently discarded during training. Check the token length percentiles in the validation report — the 95th percentile against your sequence length tells you immediately.

## Excellent on training data, poor on anything else

Symptom: training loss beautifully low, held-out performance flat or worse.

Cause: overfitting, usually from too many epochs. Fine-tuning wants one to three; runs configured for ten are paying to memorise. Reduce epochs before touching anything else — it is the cheapest lever and the most commonly mis-set.

## The model hedges, or is inconsistent

Symptom: vague answers, or the same question answered differently on different runs.

Cause: inconsistent training data. Two examples giving different answers to substantively the same question teach the model that the task is ambiguous, and hedging is the correct response to ambiguity.

This is a data problem and no hyperparameter fixes it.

## Better at the task, worse at everything else

Symptom: the target metric improves and unrelated capability quietly degrades — including safety refusals.

Cause: catastrophic forgetting. It is invisible in training loss, which measures only the new data, so it is only detectable with an evaluation set covering tasks you are not training on.

Adapter methods reduce the risk substantially because the base weights are frozen. Full fine-tuning at a high learning rate over many epochs is where it appears worst.

## Great evaluation, disappointing production

Symptom: the gate passed comfortably and users disagree.

Cause: usually contamination — evaluation data that overlapped with training data, often because deduplication ran after splitting rather than before. Sometimes the evaluation set was simply drawn from cleaner data than production contains.

## Frequently asked questions

### Which should I check first?

Truncation and the end-of-sequence token, because both are visible in the validation report before you spend anything, and between them they explain a large share of confusing failures.

### The loss curve looked fine. Does that mean training worked?

No. Training loss measures fit to the training data only. A model losing general capability, overfitting, or learning from truncated targets can all produce a perfectly healthy-looking curve.