Training
Batch size
What is batch size?
Batch size is the number of training examples processed before the model weights are updated. Larger batches give smoother, more reliable gradient estimates and use more memory; smaller batches are noisier but fit on cheaper hardware.
It interacts directly with the learning rate. A larger batch produces a lower-variance gradient, which supports a larger step — so changing one without considering the other changes the effective training dynamics.
When memory is the constraint, gradient accumulation gives the same effect: process several small batches, sum the gradients, and update once. The result is mathematically close to a large batch at a fraction of the peak memory.
Related terms
Start with the free tier
A magic link creates your account, your tenant and your first API key. No card until you ask for compute.