Data
Sequence length
Also called Max sequence length.
What is sequence length?
Sequence length is the maximum number of tokens in a single training example. It drives memory consumption superlinearly through the attention mechanism, and it is the setting most often responsible for a run that will not fit.
Setting it to the model’s maximum by default is a common and expensive mistake. If ninety-five per cent of your examples are under a thousand tokens, training at eight thousand pays for headroom that is almost never used.
Setting it too low is worse, because the excess is silently truncated. A dataset where a fifth of the targets are cut off will train without complaint and produce a model that stops mid-answer.
Related terms
Start with the free tier
A magic link creates your account, your tenant and your first API key. No card until you ask for compute.