---
title: How to choose a base model
description: A filtering order that avoids the most expensive mistake — choosing a size before measuring whether you need it.
url: https://www.onrup.com/guides/choosing-a-base-model
category: Getting started
published: 2026-08-06
updated: 2026-08-06
source: Onrup
---

# How to choose a base model

**How do I choose which base model to fine-tune?**

Filter by licence first, then by the smallest size that could plausibly clear your accuracy bar, then by whether it supports the objective you need. Benchmarks are for shortlisting; the choice among the final two or three should be made on your own data.

## Licence first, because it is a hard constraint

A licence problem discovered after training is a retraining exercise. Standard permissive licences impose almost nothing. Community licences add acceptable-use policies, naming requirements and sometimes revenue thresholds. Some carry restrictions on using outputs to train other models.

The obligations survive fine-tuning: a derivative inherits the base model’s terms, and they pass to anyone you give the model to.

## Then size, from the bottom

The most expensive mistake in this whole process is picking a size before measuring whether you need it. Teams routinely deploy an 8B for a task a 1.7B handles identically, and then carry that difference on every request forever.

Start at the smallest size that could plausibly work, run the gate, and move up only if it fails. At high volume the difference between a 0.6B and an 8B is not a rounding error — it is the economics of the feature.

## Then objective support

Not every model supports every objective. Preference tuning needs the memory for a reference model alongside, which puts it out of reach on the largest sizes. Reinforcement learning is offered on a narrower set still.

If you know you will need a reward function, filter for it at the start rather than discovering the constraint after building the dataset.

## Ecosystem is a real tiebreaker

Where two models are close on the criteria above, the one with more surrounding work is usually the better choice. More published recipes, more adapters, more troubleshooting threads — when something goes wrong, somebody has already written up the answer.

This is a genuine advantage and it is worth more than a small benchmark difference on a leaderboard your task does not resemble.

## What benchmarks are for

Triage. Benchmarks narrow forty models to three. They should not choose among the three, because benchmark contamination is endemic and general capability is not what you are buying.

The choice among finalists is made by training on your data and gating against your baseline. That measurement costs a few dollars and answers the question the leaderboard cannot.

## Frequently asked questions

### Should I start from a base model or an instruction-tuned one?

Instruction-tuned, for most task-specific work — the model already knows how to be asked a question and your fine-tune only teaches the specifics. Start from a raw base when the target behaviour is unusual enough that existing instruction tuning would be something to overcome.

### How much does switching base models later cost?

More than expected. Adapters do not transfer, evaluation results are not comparable, and the chat template changes. The dataset survives, which is the main thing.