Training
Loss
What is loss?
Loss is a single number measuring how wrong the model’s predictions are on a batch. For language models it is typically cross-entropy: how surprised the model was by the correct next token. Training minimises it; it is a proxy for quality, not quality itself.
Falling training loss means the model is fitting the training data. It does not mean the model is getting better at the task, and the two diverge exactly when overfitting begins.
Loss values are not comparable across datasets, tokenisers or model families. A loss of 1.2 is meaningful only against the same setup measured earlier — comparing it to a number from someone else’s run says nothing.
Related terms
Start with the free tier
A magic link creates your account, your tenant and your first API key. No card until you ask for compute.