Training
Finetune cost
What a LoRA or QLoRA run will need, how long it will take, and what it will cost to rent the card. Memory is modelled term by term — including the logits tensor, which is usually larger than the adapter and which almost nobody budgets for.
—
Peak VRAM
—
Wall clock
—
Rental cost
—
Throughput
—tok/s
Memory breakdown
0
Table view & working
| Component | Size | Share | How it is derived |
|---|
The run
Same run, other hardware
| GPU | Peak VRAM | Fits | Throughput | Time | Cost |
|---|
Rental prices are typical community-cloud rates and move constantly — treat them as an order of magnitude, not a quote. Time assumes a single card of that type unless you set a count above.
What this assumes
-
The logits tensor is real memory.
seq × batch × vocab × 4 bytes, and modern vocabularies are 128k+. At 2048 tokens and batch 2 that is 2.1 GB — twenty-five times the size of a rank-16 adapter. Fused cross-entropy (Unsloth, Liger) avoids materialising it, and that is most of where the "70% less VRAM" claim comes from. -
LoRA parameters are counted from real shapes.
A rank-r adapter on a din×dout projection is
r × (d_in + d_out), summed over the model's actual layers. The "LoRA is 0.5% of the model" heuristic is only true near 8B — it falls to 0.29% at 70B. - LoRA backward is 4 FLOPs per parameter, not 6. A frozen base still needs input gradients but no weight gradients. Gradient checkpointing adds one extra forward pass, taking it to 6.
- DDP replicates, FSDP shards. Adding GPUs under DDP buys throughput and no memory at all — each card still holds the whole model. If the run does not fit on one card, more cards under DDP will not help.
- Expect ±25%. Peak memory depends on allocator behaviour and on your longest batch; throughput depends on kernel versions. Leave headroom, and run one epoch before booking a week of GPU time.