Serving many fine-tuned models without paying for each one

Ten merged models need ten copies of the weights resident. Ten adapters against one base need one copy plus ten small files. At an 8B base that is roughly sixteen gigabytes against a hundred and sixty.

This is what makes a per-customer or per-task model strategy viable at all. Without it, the memory cost forces you to consolidate into one model doing several jobs adequately.

Every co-resident adapter must share the same base model and the same version. An adapter trained against a different version produces degraded or nonsensical output, and nothing prevents you from loading it.

Plan for this at the start. Choosing a base model is not just a quality decision — it decides which of your future models can share serving capacity.

Loading an adapter against an already-resident base is far quicker than loading a whole model. An endpoint serving several variants therefore recovers from a scale-to-zero event more cheaply than one serving several merged models.

This makes scale-to-zero more attractive for multi-variant deployments than it is for a single large model.

Merge when you need a standalone artefact: handing the model to somebody who does not know or care about adapters, publishing for general use, or deploying into an environment that expects a single model directory.

The trade is co-residency, and you should make it deliberately rather than by default.

Once several variants are cheap to serve, the better architecture is often several specialised models with a routing layer rather than one general model. Small specialised models are easier to evaluate, easier to improve independently, and easier to roll back.

A small routing model in front costs a few milliseconds and can be fine-tuned too.