Training
Overfitting
What is overfitting?
Overfitting is when a model memorises its training data rather than learning the pattern behind it. Training loss keeps falling while performance on unseen data plateaus and then degrades, which makes the training curve alone a misleading indicator of progress.
It is diagnosed by comparison, never in isolation. A validation split held out from training gives the second curve; when training loss falls and validation loss rises, the gap between them is the overfitting.
In fine-tuning it usually shows up quickly, because the datasets are small. Two or three epochs over a few thousand examples is often already too many, and practitioners coming from training-from-scratch habits routinely run far too long.
Small datasets, high learning rates, high adapter rank and many epochs all push in the same direction. Reduce epochs first — it is the cheapest lever and the most commonly mis-set.
Related terms
Start with the free tier
A magic link creates your account, your tenant and your first API key. No card until you ask for compute.