Data
Special tokens
What is special tokens?
Special tokens are reserved symbols that carry structural meaning rather than content — start and end of sequence, role boundaries, padding. Mishandling them is one of the most common causes of a fine-tune that trains cleanly and behaves strangely.
The classic failure is a missing end-of-sequence token in the training targets. The model never learns where to stop and generates until it hits the length limit, which reads as a model that will not shut up.
The reverse failure is including a role marker as literal text rather than as its reserved token. The model learns to output the marker as content, which appears in production as stray markup in otherwise correct answers.
Related terms
Start with the free tier
A magic link creates your account, your tenant and your first API key. No card until you ask for compute.