---
title: How much VRAM does fine-tuning actually need?
description: Where the memory actually goes, why training needs several times what inference does, and how to make a run fit without changing GPU class.
url: https://www.onrup.com/guides/how-much-vram-do-i-need
category: Cost
published: 2026-08-06
updated: 2026-08-06
source: Onrup
---

# How much VRAM does fine-tuning actually need?

**How much GPU memory do I need to fine-tune a model?**

For half-precision LoRA, budget roughly two to two and a half gigabytes per billion parameters. For four-bit QLoRA, closer to one and a half. For full fine-tuning, around sixteen. Preference tuning roughly doubles whichever figure applies, because a frozen reference model is held alongside.

## Four things compete for the same memory

This is why an adapter method saves so much: it collapses two of the four terms almost entirely. It is also why a run that fitted yesterday fails today when someone raised the sequence length.

- Weights — the model itself. Two bytes per parameter in half precision, half a byte in four-bit.
- Gradients — one value per trainable parameter. Near zero for adapters, enormous for full fine-tuning.
- Optimiser state — several values per trainable parameter. The dominant term in full fine-tuning and negligible for adapters.
- Activations — intermediate values held for the backward pass. Scales with batch size and sequence length, and it is the one that surprises people.

## Rough figures per billion parameters

These are working approximations, not guarantees. The exact figure depends on sequence length, batch size and architecture, which is why the model catalogue publishes a specific admission threshold per model rather than a formula.

| Approach | Roughly, per 1B params | 8B model | 70B model |
| --- | --- | --- | --- |
| LoRA, half precision | ~2.2 GB | ~18 GB | ~140 GB |
| QLoRA, four-bit | ~1.7 GB | ~14 GB | ~48 GB |
| Full fine-tuning | ~16 GB | ~128 GB | Multi-node |
| Inference, half precision | ~2 GB | ~16 GB | ~140 GB |

## Preference tuning doubles it

Direct preference optimisation holds a frozen reference copy of the model alongside the one being trained, to measure and penalise drift. Two models resident means roughly twice the memory.

This catches people out because supervised training on the same model fitted comfortably. A 14B pair that needs 28 GB for supervised training needs a 48 GB class for preference tuning.

## Making a run fit without paying more

Before moving to a larger class, three levers are usually available and are worth trying in this order.

- Reduce sequence length to something the data actually needs. Setting it to the model maximum by default is the most common source of waste — check the token length percentiles in the validation report first.
- Turn on gradient checkpointing. It discards activations and recomputes them, cutting the largest variable term for roughly a third more time.
- Use gradient accumulation. Several small batches summed into one update give the same effective batch size at a fraction of the peak memory.

Together these will usually bring a run back inside a class. Whether that is cheaper than the next class up depends on the rate difference against the time penalty, and it is an arithmetic question rather than a judgement one.

## Frequently asked questions

### Why does training need so much more than inference?

Inference holds weights and a cache. Training additionally holds gradients, optimiser state and activations for the backward pass. For full fine-tuning those extra terms dominate.

### Does a longer context need more memory at inference too?

Yes — the attention cache grows with sequence length and concurrency. At long context and high concurrency, that cache can exhaust memory with the weights comfortably resident.