Training
Learning rate schedule
What is learning rate schedule?
A learning rate schedule varies the learning rate over the course of a run rather than holding it constant — usually a short warmup to full rate followed by a gradual decay. It improves both stability at the start and convergence at the end.
Cosine decay is the common default: the rate follows a smooth curve down toward zero across the run. Linear decay is simpler and behaves similarly in practice on short fine-tuning runs.
The schedule is defined over the total number of steps, so changing the dataset size or epoch count changes the shape of the curve. A run that was well tuned at two epochs may behave differently at three for this reason alone.
Related terms
Start with the free tier
A magic link creates your account, your tenant and your first API key. No card until you ask for compute.