Cost
API cost & the self-host crossover
What your workload costs on a hosted API, and the monthly token volume at which renting a GPU beats paying per token. The answer usually turns on utilisation rather than on hardware: a rented card bills by the hour whether or not you are sending it work.
—
API / month
—
Self-host / month
—
GPU utilisation
—
Break-even
—
tokens/month for the rig to win
Same workload, every provider
| Model | $/M in | $/M out | Per request | Per month |
|---|
The self-hosting arithmetic
What this assumes
- Utilisation decides everything. A rented GPU bills by the hour whether or not there is work for it, so cost per token is a property of how busy you keep the card rather than of the card. At 3% utilisation you are paying roughly thirty times the marginal rate.
- The self-host figure counts prefill. Prompt processing occupies the GPU too, and for a RAG-shaped workload with a long prompt and a short answer it is often most of the work. Estimates that count only generated tokens understate the bill badly.
- It ignores everything that is not the GPU. No engineering time, no on-call, no model upgrades, no spare capacity for traffic spikes, no egress, no storage for weights. A hosted API is considerably more than a rental with a markup, so the break-even volume here is a floor, not a decision.
- Throughput comes from the serving model. The self-hosted rate is the batched figure from /serve at this model, quant and context — not the single-stream number, which would understate a real server by an order of magnitude.
- Prices go stale. There is no price feed here; it is a static site. Rows marked approximate are representative market rates rather than quotes. Check before you budget.