Training
Full fine-tuning
What is full fine-tuning?
Full fine-tuning updates every weight in the model rather than a small added set. It offers the most capacity to change behaviour and costs the most memory — typically an order of magnitude more than a parameter-efficient method on the same model.
The memory cost is not just the weights. Optimiser state for a standard adaptive optimiser is several times the size of the parameters themselves, which is why a model that occupies sixteen gigabytes at rest can need well over a hundred to fully fine-tune.
It is worth it when the target behaviour is genuinely far from anything the base model does — a new language, a highly unusual output format, a domain whose vocabulary barely overlaps with the training corpus. For ordinary task adaptation, the quality difference against a well-configured adapter is small.
The output is also a complete model rather than a small file, which makes it heavier to store, slower to load and impossible to co-locate with siblings against a shared base.
Related terms
Start with the free tier
A magic link creates your account, your tenant and your first API key. No card until you ask for compute.