Llama · Meta
Fine-tuning Llama 4 Scout 109B
What does it take to fine-tune Llama 4 Scout 109B?
Llama 4 Scout 109B needs 220 GB for half-precision LoRA training and 80 GB in four-bit, and 80 GB to serve. It supports supervised fine-tuning, under the Llama 4 Community licence. The cheapest qualifying class is A100 80 GB at $1.68 per GPU-hour.
Sixteen experts, seventeen billion active per token, and natively multimodal. Note the asymmetry in the numbers: serving fits in 80GB but half-precision training needs 220GB, so the practical path is four-bit training on an 80GB class and serving on the same. Experimental in this catalogue.
What to know before choosing it
Read the memory figures carefully, because they are not in the order anyone expects: serving fits in 80GB while half-precision training needs 220GB. The practical path is four-bit training and half-precision serving on the same class, which is an unusual shape and worth planning for.
Native multimodality is the reason to be here at all. If your task is text-only, a dense model a fraction of this size will be cheaper to run and easier to fine-tune, and the sixteen-expert routing will not pay for itself.
Specification
| Parameters | 109B17B active per token |
|---|---|
| Architecture | Mixture of experts |
| Size tier | Extra large |
| LoRA training memory | 220 GBHalf precision, frozen base |
| QLoRA training memory | 80 GBFour-bit base, higher-precision adapter |
| Serving memory | 80 GBHalf precision, before attention cache |
| Objectives | SFT |
| Licence | Llama 4 Community |
| Repository | meta-llama/Llama-4-Scout-17B-16E |
What it costs to train
Every class with enough memory for four-bit training, cheapest first. Total cost is the rate multiplied by wall time, so the cheapest rate is not always the cheapest run — a faster class that finishes sooner frequently wins.
| GPU class | VRAM | Training / hr | Serving / hr | Fits |
|---|---|---|---|---|
| A100 80 GB | 80 GB | $1.68 | $1.94 | QLoRA only |
| H100 80 GB | 80 GB | $2.59 | $2.99 | QLoRA only |
| H200 | 141 GB | $4.55 | $5.25 | QLoRA only |
Serving fits on A100 80 GB at $1.94 per hour — before the attention cache, which grows with context length and concurrency.
Good starting point for
- Multimodal tasks
- Frontier-scale capability at moderate serving cost
This is a mixture-of-experts model
109B total parameters with 17B active per token. Memory scales with the total; inference compute scales with the active count. That makes it attractive when memory is cheaper than compute for your workload, and unattractive when the reverse holds. Quoting only one of the two numbers is how these models get misrepresented in both directions.
Other sizes in this family
| Model | Params | QLoRA | Objectives |
|---|---|---|---|
| Llama 3.2 1B | 1B | 3 GB | SFT, DPO |
| Llama 3.2 3B | 3B | 6 GB | SFT, DPO |
| Llama 3.1 8B | 8B | 14 GB | SFT, DPO, GRPO |
| Llama 3.3 70B | 70B | 48 GB | SFT |
Comparable sizes elsewhere
Frequently asked questions
How much VRAM does it take to fine-tune Llama 4 Scout 109B?
220 GB for half-precision LoRA and 80 GB for four-bit QLoRA. Serving needs 80 GB. Preference tuning roughly doubles the training figure, because a frozen reference model is held alongside the one being trained.
What is the cheapest way to fine-tune Llama 4 Scout 109B?
Four-bit QLoRA on A100 80 GB at $1.68 per GPU-hour is the cheapest class that meets the 80 GB threshold. Note that quantised training is slower per step, so a faster class sometimes costs less over the whole run.
Can I download the weights after fine-tuning Llama 4 Scout 109B?
Yes. Every finished run exposes its trained weights for download, and publishing to a model hub is a single call with a generated model card recording the base model and version the adapter applies to.
What licence does Llama 4 Scout 109B carry?
Llama 4 Community. The licence follows the fine-tune — a derivative inherits the base model’s terms, and those terms pass to anyone you give the model to.
Last verified 6 August 2026. Memory thresholds are the platform's own admission limits.
Fine-tune Llama 4 Scout 109B
Upload a dataset, forecast the run, and see the cost before any compute is leased.