The catch is up front: distills are weaker than the full R1, and picking the wrong size wastes either money or accuracy.

Which DeepSeek R1 Distill Should You Pick?

The distills come in sizes built on different base models — roughly 1.5B, 7B, 8B, 14B, 32B and 70B. Bigger means better reasoning and more hardware. A 7B or 8B runs on a modest GPU or a 16GB Mac; the 32B wants a 24GB GPU; the 70B wants serious VRAM.

How Do You Serve It?

1. Install Ollama for the simplest path — it pulls the distill with one command.
2. Choose a quantised build (4-bit) to fit your GPU.
3. Expose the local endpoint to your client or app.
4. Benchmark on your real tasks; measure tokens per second before deciding.

The reasoning traces still appear, so you keep the transparency that makes R1 useful — just at lower accuracy.

Where Do Distills Lose to the Full Model?

Hard maths, long multi-step logic and rare edge cases degrade first. A 32B distill is impressive for its size but will miss problems the full DeepSeek R1 solves. Use distills for privacy-sensitive, moderate-difficulty work; route the genuinely hard problems to the hosted full model or a stronger API.

Is Local Reasoning Worth It?

Yes when data cannot leave, or when you want an always-available reasoning tool with no per-call cost. No if you need top accuracy and have no privacy constraint — a hosted API is simpler and better. The honest rule: pick local for control, hosted for capability.

Key Takeaways

• R1 distills come in ~1.5B to 70B sizes; bigger needs more VRAM and reasons better.
• 7-8B runs on a laptop; 32B needs a 24GB GPU; 70B needs multi-GPU.
• Serve with Ollama or llama.cpp using 4-bit quantisation.
• Distills lose on hard maths and long logic — route those to the full model.

This is the deployment step after you have already decided local weights are worth it.

What Does a Self-Hosted Llama 4 Stack Look Like?

Four pieces: a GPU host, a model server (vLLM, Ollama or llama.cpp), a reverse proxy with auth, and a chat front end. vLLM is the pick for teams because it batches concurrent requests and gives much better throughput than a single-user runner.

• Server: vLLM for concurrency, Ollama for simplicity.
• Model: Llama 4 Scout, 4-bit quantised to fit one 24GB GPU.
• Proxy: nginx or Caddy in front, with API-key or SSO auth.
• UI: Open WebUI or a custom client hitting the local endpoint.

How Do You Control Cost and Access?

Compute is the cost. A 24GB GPU instance runs a few dollars an hour; always-on, that dwarfs many API bills. Two rules keep it sane: scale the instance down when idle, and set per-user rate limits so one person cannot saturate the GPU.

Lock the endpoint down. A bare model server with no auth is an open door, and the reverse proxy is where you enforce keys, spend caps and request logging.

How Do You Handle Quality Gaps?

A quantised, locally served Llama 4 is not a frontier model. It will be weaker on hard reasoning and long context. Route sensitive queries to the local model and non-sensitive ones to a hosted API — a split that gets you privacy where it matters without paying the quality tax everywhere.

When Self-Hosting Is Not Worth It

If you have no sensitive data and low volume, a hosted API is cheaper, better and has zero maintenance. Self-hosting wins only when data cannot leave, or volume is high enough to amortise the hardware. Be honest about which one you are.

Key Takeaways

• Stack: GPU host + vLLM/Ollama + auth proxy + chat UI (e.g. Open WebUI).
• vLLM gives the best throughput for multi-user serving; quantise to 4-bit.
• Always-on GPU cost usually beats API cost only at high volume.
• Route sensitive queries locally, the rest to a hosted API.