Model architecture
Transformer
What is transformer?
The transformer is the architecture behind essentially every current language model: stacked blocks of attention and feed-forward layers. Its defining property is that it processes a whole sequence in parallel rather than one position at a time.
Parallel processing is what made training on internet-scale data feasible, and it is the reason the architecture displaced its recurrent predecessors so completely.
Generation is still sequential, because each new token depends on the last. Training is parallel over the sequence; inference is not, and that asymmetry explains much of the difference in their cost profiles.
Related terms
Start with the free tier
A magic link creates your account, your tenant and your first API key. No card until you ask for compute.