How to Self-Host DeepSeek R1 Distill for Local Reasoning
You do not need a data centre to run DeepSeek R1. The team published a set of distilled models that shrink the reasoning behaviour into smaller backbones you can run locally — some on a laptop, some on one GPU. For teams that want reasoning without sending data to an API, this is the practical path.
The catch is up front: distills are weaker than the full R1, and picking the wrong size wastes either money or accuracy.
Which DeepSeek R1 Distill Should You Pick?
The distills come in sizes built on different base models — roughly 1.5B, 7B, 8B, 14B, 32B and 70B. Bigger means better reasoning and more hardware. A 7B or 8B runs on a modest GPU or a 16GB Mac; the 32B wants a 24GB GPU; the 70B wants serious VRAM.
| Size | Runs on | Good for |
|---|---|---|
| 1.5B – 7B | Laptop / 8GB GPU | Light logic, demos |
| 14B – 32B | 24GB GPU | Real reasoning, local privacy |
| 70B | Multi-GPU | Closest to full R1 |
How Do You Serve It?
- Install Ollama for the simplest path — it pulls the distill with one command.
- Choose a quantised build (4-bit) to fit your GPU.
- Expose the local endpoint to your client or app.
- Benchmark on your real tasks; measure tokens per second before deciding.
The reasoning traces still appear, so you keep the transparency that makes R1 useful — just at lower accuracy.
Where Do Distills Lose to the Full Model?
Hard maths, long multi-step logic and rare edge cases degrade first. A 32B distill is impressive for its size but will miss problems the full DeepSeek R1 solves. Use distills for privacy-sensitive, moderate-difficulty work; route the genuinely hard problems to the hosted full model or a stronger API.
Is Local Reasoning Worth It?
Yes when data cannot leave, or when you want an always-available reasoning tool with no per-call cost. No if you need top accuracy and have no privacy constraint — a hosted API is simpler and better. The honest rule: pick local for control, hosted for capability.
Key Takeaways
- R1 distills come in ~1.5B to 70B sizes; bigger needs more VRAM and reasons better.
- 7-8B runs on a laptop; 32B needs a 24GB GPU; 70B needs multi-GPU.
- Serve with Ollama or llama.cpp using 4-bit quantisation.
- Distills lose on hard maths and long logic — route those to the full model.
