Cost and billing
VRAM
Also called GPU memory.
What is vram?
VRAM is the memory on a GPU, and it is the binding constraint on almost every fine-tuning job. It must hold the weights, the gradients, the optimiser state and the activations simultaneously — which is why training needs several times what inference does.
A rough guide for full fine-tuning in half precision is around sixteen gigabytes per billion parameters once everything is accounted for. Adapter methods cut that dramatically because optimiser state and gradients only exist for the small trained set.
Running out of memory is the most common way a run fails, and it usually fails at a random-looking step rather than at the start, because activation memory peaks with the longest sequence in a batch.
Related terms
Start with the free tier
A magic link creates your account, your tenant and your first API key. No card until you ask for compute.