Quality
Quantisation quality
What each quantisation level actually costs you, taken from the published llama.cpp perplexity runs. Unlike the rest of this site these are measurements rather than estimates — so the job here is to present them clearly and be explicit about what they do not cover.
—
What fits your budget
| Quant | Weights | Total | Fits | Quality cost |
|---|
Measured quality ladder
| Quant | Perplexity | vs FP16 | % worse | Damage |
|---|
What the numbers say
Where this comes from
- Published llama.cpp perplexity runs on wikitext-2, measured against each model's own FP16 baseline. The absolute perplexity is meaningless across models — only the delta from that model's own baseline says anything about quantisation damage, which is why this table leads with the delta.
- LLaMA-2 is used because it is the only complete ladder. It is the one family measured consistently across every level from Q2_K to Q8_0 on one harness. Newer models are measured piecemeal by different people under different settings, and stitching those together would produce a table that looks authoritative and is not.
- Perplexity is a proxy, not the thing you care about. It correlates with capability but does not capture instruction following, tool-call formatting or long-context recall — all of which degrade faster than perplexity suggests. A quant that looks fine here can still break structured output.
- Importance-matrix quants beat these numbers. The IQ formats use a calibration pass to decide which weights matter and land meaningfully better than the legacy formats at the same size. They arrived after this dataset was measured, so they are absent from the ladder — treat their entries in the fit table as sized, not scored.