Preparing data
ShareGPT, ChatML and Alpaca: which dataset format to use
What is the difference between ShareGPT, ChatML and Alpaca dataset formats?
ShareGPT and Alpaca are interchange formats for storing your data; ChatML is the trainer-ready form a model actually consumes. Store in whichever suits your data shape and let a format adapter convert — the conversion is where mistakes happen, not the choice.
Two different jobs
ShareGPT and Alpaca are storage and interchange formats. They exist so a dataset can be shared, versioned and reused independently of which model you eventually train.
ChatML is a target. It marks each message with an explicit role using reserved tokens the model was trained to recognise, and it is what the trainer actually consumes.
Choosing between the first two is a question about your data shape. Getting to the third correctly is a question about your pipeline.
Which to store in
| Your data | Format | Why |
|---|---|---|
| Multi-turn conversations | ShareGPT | Represents turn sequences natively |
| Single-turn tasks | Alpaca | Simpler; fewer ways to build it wrongly |
| Ranked pairs from review | Preference records | Chosen and rejected against one prompt |
| Prompts with a checkable answer | Prompt-only | Reinforcement learning needs no target text |
| Rows in a spreadsheet | Tabular with a column map | Avoids a hand-written conversion step |
The conversion is where it goes wrong
Three mistakes account for most of the confusing failures, and all three produce a run that completes without error.
- The system message is dropped. The model never learns the framing it will be given at inference, so training and serving disagree.
- Roles are mapped wrongly. Loss computed over the prompt as well as the response teaches the model to generate the question along with the answer.
- The end-of-sequence token is missing from targets. The model never learns where to stop and generates until it hits the length limit.
The third is the most common and the most recognisable in production: a model that answers correctly and then keeps going.
Templates are model-specific
Each model family uses different reserved tokens for role boundaries. A dataset converted for one family and trained on another produces a model that treats structural markers as ordinary text and reproduces them in its output.
This is a good argument for storing in an interchange format and converting at training time rather than storing pre-converted. The dataset then survives a change of base model.
Frequently asked questions
Can I mix formats in one dataset?
No. Convert everything to one shape first. A mixed dataset either fails validation or, worse, trains on an inconsistent structure.
Does the format affect quality?
The storage format does not. The conversion does, considerably — a correct conversion is invisible and an incorrect one is expensive to diagnose because nothing errors.
Terms used here
More on preparing data
Last verified 6 August 2026.
Start with the free tier
A magic link creates your account, your tenant and your first API key. No card until you ask for compute.