How to Self-Host DeepSeek R2 and Cut Your Inference Bill

Self-hosting DeepSeek R2 makes sense once you push more than roughly 40 million tokens a day, or when data can’t leave your network. Below that, the managed API is almost always cheaper.

Everyone assumes self-hosting always wins on cost. It doesn’t. GPUs idle, ops time is real, and quantized reasoning models lose some accuracy. But at high, steady volume the math flips hard — and for regulated data there’s no substitute for keeping prompts on-prem.

What Hardware Does DeepSeek R2 Need?

The full model is large and mixture-of-experts, so memory is your constraint, not raw compute.

• 4-bit quantized: fits on a single 80GB card for short contexts; ~2x that for long ones.
• 8-bit: 2-4×80GB GPUs for reasonable throughput.
• Throughput reality: a 2-GPU 4-bit setup serves roughly 15-25 concurrent reasoning requests before latency climbs.

You’ll want vLLM or SGLang as the serving layer. Both handle the MoE routing and continuous batching well; a naive Hugging Face pipeline will be several times slower.

How Do You Actually Deploy It?

1. Pull the quantized weights from Hugging Face (community GGUF and AWQ builds exist).
2. Start vLLM with --tensor-parallel-size matching your GPU count and a quantization flag.
3. Point your existing OpenAI-compatible client at the local endpoint — no code changes beyond the base URL.
4. Add a KV-cache size cap so long reasoning traces don’t blow out VRAM.

Cache the system prompt server-side. Reasoning models re-read context every step, so prefix caching is the single biggest lever on both speed and cost.

When Does Self-Hosting Beat the API?

Do the arithmetic honestly. An 80GB-class GPU rents for roughly $1.50-$2.50 an hour on the spot market, call it $1,200-$1,800 a month. If your API bill for the same workload is under that and your traffic is bursty, stay hosted.

Self-host when: volume is predictable, latency matters, prompts contain sensitive data, or you already own the GPUs. Don’t do it just to save the last 15%.

What Gets Worse After You Self-Host?

Accuracy. Quantizing a reasoning model to 4-bit can shave a few points off hard math benchmarks — usually fine, sometimes not. You also own uptime, upgrades, and the pager when a serving process OOMs at 2am.

Keep one thing in mind: a hybrid setup often beats both extremes. Run R2 locally for bulk and sensitive jobs, fall back to the hosted API for overflow and for the accuracy-critical edge cases.

Quick Summary

• Self-hosting R2 pays off above roughly 40M tokens/day or for regulated data.
• 4-bit build fits an 80GB card; use vLLM or SGLang, not vanilla HF.
• A 2-GPU setup handles ~15-25 concurrent reasoning requests.
• Tradeoffs: quantized accuracy loss, uptime burden — a hybrid often wins.