Training
Checkpoint
What is checkpoint?
A checkpoint is a saved snapshot of training state — weights, optimiser state and position in the dataset — written periodically so a run can resume from it rather than restarting. It is what makes a long run survivable on interruptible capacity.
Without checkpoints, any interruption costs the whole run. With them, it costs the time since the last one, which turns a class of failure from catastrophic into merely annoying.
Frequency is a trade: writing more often reduces the worst-case loss and consumes more time and storage. On runs measured in hours rather than days, every few hundred steps is a reasonable default.
A checkpoint is also the artefact you fall back to when a run gets worse rather than better — the best checkpoint by validation loss is often not the last one.
Related terms
Start with the free tier
A magic link creates your account, your tenant and your first API key. No card until you ask for compute.