Data
Preference pair
What is preference pair?
A preference pair is a prompt with two responses, one marked better than the other. It is the training unit for preference tuning, and its value depends entirely on the comparison being genuine rather than manufactured.
Pairs harvested from a real review process — two drafts, one chosen — carry real signal. Pairs constructed by degrading a good response teach the model to detect the degradation, which is a different and useless skill.
The two responses should be plausible alternatives. A comparison between a good answer and an obviously broken one contains almost no information about what makes the good one good.
Related terms
Start with the free tier
A magic link creates your account, your tenant and your first API key. No card until you ask for compute.