Granite · IBM
Fine-tuning Granite 3.0 2B
What does it take to fine-tune Granite 3.0 2B?
Granite 3.0 2B needs 7 GB for half-precision LoRA training and 5 GB in four-bit, and 5 GB to serve. It supports supervised fine-tuning, preference tuning (dpo), under the Apache 2.0 licence. The cheapest qualifying class is RTX 3080 at $0.09 per GPU-hour.
Tuned toward retrieval-augmented answering and tool use rather than open-ended chat, and released under Apache-2.0 with published training-data documentation — which is the reason it shows up in regulated environments more often than its benchmark scores alone would explain.
What to know before choosing it
The reason Granite appears in regulated environments more than its benchmark scores explain is documentation: IBM publishes what went into training, so a question about provenance has an answer. That is a procurement property rather than a capability one, and procurement is often what actually blocks a deployment.
Behaviourally it is tuned to decline rather than improvise when it does not know, which is the single most useful default for retrieval-augmented answering and the hardest to instil in a model that does not already have it.
Specification
| Parameters | 2B |
|---|---|
| Architecture | Dense |
| Size tier | Small |
| LoRA training memory | 7 GBHalf precision, frozen base |
| QLoRA training memory | 5 GBFour-bit base, higher-precision adapter |
| Serving memory | 5 GBHalf precision, before attention cache |
| Objectives | SFT, DPO |
| Licence | Apache 2.0 |
| Repository | ibm-granite/granite-3.0-2b-instruct |
What it costs to train
Every class with enough memory for four-bit training, cheapest first. Total cost is the rate multiplied by wall time, so the cheapest rate is not always the cheapest run — a faster class that finishes sooner frequently wins.
| GPU class | VRAM | Training / hr | Serving / hr | Fits |
|---|---|---|---|---|
| RTX 3080 | 12 GB | $0.09 | $0.11 | LoRA and QLoRA |
| RTX 4000 Ada | 20 GB | $0.09 | $0.11 | LoRA and QLoRA |
| L4 | 24 GB | $0.17 | $0.20 | LoRA and QLoRA |
| RTX 3090 | 24 GB | $0.22 | $0.26 | LoRA and QLoRA |
| RTX 4090 | 24 GB | $0.38 | $0.43 | LoRA and QLoRA |
| A40 | 48 GB | $0.42 | $0.48 | LoRA and QLoRA |
| RTX 6000 Ada | 48 GB | $0.61 | $0.71 | LoRA and QLoRA |
| A6000 | 48 GB | $0.65 | $0.75 | LoRA and QLoRA |
| L40S | 48 GB | $0.78 | $0.90 | LoRA and QLoRA |
| A100 40 GB | 40 GB | $1.17 | $1.35 | LoRA and QLoRA |
| A100 80 GB | 80 GB | $1.68 | $1.94 | LoRA and QLoRA |
| H100 80 GB | 80 GB | $2.59 | $2.99 | LoRA and QLoRA |
| H200 | 141 GB | $4.55 | $5.25 | LoRA and QLoRA |
Serving fits on RTX 3080 at $0.11 per hour — before the attention cache, which grows with context length and concurrency.
Good starting point for
- Retrieval-augmented answering
- Environments needing data provenance
- Tool-calling at small scale
Preference tuning on this model
Preference tuning holds a frozen reference copy of the model alongside the one being trained, so budget roughly 14 GB rather than 7 GB. That is the thing that catches people out — supervised training on this model fits on a class that preference tuning will overflow.
Other sizes in this family
| Model | Params | QLoRA | Objectives |
|---|---|---|---|
| Granite 3.0 8B | 8B | 14 GB | SFT, DPO |
| Granite 4.1 8B | 8B | 14 GB | SFT, DPO |
Comparable sizes elsewhere
Frequently asked questions
How much VRAM does it take to fine-tune Granite 3.0 2B?
7 GB for half-precision LoRA and 5 GB for four-bit QLoRA. Serving needs 5 GB. Preference tuning roughly doubles the training figure, because a frozen reference model is held alongside the one being trained.
What is the cheapest way to fine-tune Granite 3.0 2B?
Four-bit QLoRA on RTX 3080 at $0.09 per GPU-hour is the cheapest class that meets the 5 GB threshold. Note that quantised training is slower per step, so a faster class sometimes costs less over the whole run.
Can I download the weights after fine-tuning Granite 3.0 2B?
Yes. Every finished run exposes its trained weights for download, and publishing to a model hub is a single call with a generated model card recording the base model and version the adapter applies to.
What licence does Granite 3.0 2B carry?
Apache 2.0. The licence follows the fine-tune — a derivative inherits the base model’s terms, and those terms pass to anyone you give the model to.
Last verified 6 August 2026. Memory thresholds are the platform's own admission limits.
Fine-tune Granite 3.0 2B
Upload a dataset, forecast the run, and see the cost before any compute is leased.