Evaluation
Baseline model
What is baseline model?
The baseline is whatever is running today — the model a candidate would replace. Evaluating against it rather than against an absolute score answers the only question that matters at deploy time: is this an improvement on what we already have?
Absolute scores are unanchored. Knowing that a model scores 0.82 on your rubric says nothing about whether to deploy it; knowing it beats the incumbent on 58 per cent of held-out cases says everything.
The baseline changes over time as models are deployed, which means an evaluation from six months ago is not comparable to one from today. Recording which baseline a result was measured against is part of the result.
Related terms
Start with the free tier
A magic link creates your account, your tenant and your first API key. No card until you ask for compute.