Model architecture
Active parameters
What is active parameters?
Active parameters are the subset of a mixture-of-experts model actually used to process a given token. Inference compute scales with active parameters; memory requirements scale with total parameters. Both numbers are needed to reason about cost.
A model with a hundred and nine billion total and seventeen billion active generates at roughly seventeen-billion speed while needing memory for all hundred and nine.
Quoting only one of the two numbers is how mixture-of-experts models get misrepresented in both directions — as unaffordably large, or as implausibly fast.
Related terms
Start with the free tier
A magic link creates your account, your tenant and your first API key. No card until you ask for compute.