Evaluation
Rubric
What is rubric?
A rubric is the explicit set of criteria a judge applies when scoring an output — what counts as correct, complete, appropriately formatted and appropriately toned. Writing it down is what makes subjective evaluation reproducible.
The act of writing a rubric usually reveals that the team disagrees about what good means. That disagreement was there all along; the rubric surfaces it before it becomes an argument about a deployment.
Rubrics should be versioned alongside the evaluation set. A score is only comparable to another score produced under the same rubric, and rubrics drift as people notice cases they had not considered.
Related terms
Start with the free tier
A magic link creates your account, your tenant and your first API key. No card until you ask for compute.