Evaluation
Perplexity
What is perplexity?
Perplexity is the exponential of cross-entropy loss, interpretable as how many equally-likely options the model was effectively choosing between at each token. Lower is better, and like loss it is only comparable within an identical setup.
It measures how well the model predicts text, which is related to but distinct from how useful its output is. A model can have excellent perplexity on your domain and still format its answers wrongly or refuse when it should not.
It is a reasonable smoke test and a poor deployment criterion. Task-level evaluation against the model you would actually replace is the decision-grade measurement.
Related terms
Start with the free tier
A magic link creates your account, your tenant and your first API key. No card until you ask for compute.