Training
Rank
Also called LoRA rank, r.
What is rank?
Rank is the inner dimension of the low-rank matrices an adapter trains, and it sets how much capacity the adapter has. Higher rank means more trainable parameters, more memory and more ability to depart from the base model’s behaviour.
Ranks between 8 and 64 cover the overwhelming majority of tasks. Below 8 the adapter often lacks the capacity to represent the change; above 64 the returns are usually flat while the memory cost is not.
The scaling factor commonly paired with rank controls how strongly the adapter’s output is weighted against the frozen base. The two are usually set together, and changing one without the other alters the effective learning rate of the adaptation.
A higher rank is not a fix for insufficient data. If the adapter is not learning, more or better examples is the more reliable lever.
Related terms
Start with the free tier
A magic link creates your account, your tenant and your first API key. No card until you ask for compute.