This is the deployment step after you have already decided local weights are worth it.

What Does a Self-Hosted Llama 4 Stack Look Like?

Four pieces: a GPU host, a model server (vLLM, Ollama or llama.cpp), a reverse proxy with auth, and a chat front end. vLLM is the pick for teams because it batches concurrent requests and gives much better throughput than a single-user runner.

• Server: vLLM for concurrency, Ollama for simplicity.
• Model: Llama 4 Scout, 4-bit quantised to fit one 24GB GPU.
• Proxy: nginx or Caddy in front, with API-key or SSO auth.
• UI: Open WebUI or a custom client hitting the local endpoint.

How Do You Control Cost and Access?

Compute is the cost. A 24GB GPU instance runs a few dollars an hour; always-on, that dwarfs many API bills. Two rules keep it sane: scale the instance down when idle, and set per-user rate limits so one person cannot saturate the GPU.

Lock the endpoint down. A bare model server with no auth is an open door, and the reverse proxy is where you enforce keys, spend caps and request logging.

How Do You Handle Quality Gaps?

A quantised, locally served Llama 4 is not a frontier model. It will be weaker on hard reasoning and long context. Route sensitive queries to the local model and non-sensitive ones to a hosted API — a split that gets you privacy where it matters without paying the quality tax everywhere.

When Self-Hosting Is Not Worth It

If you have no sensitive data and low volume, a hosted API is cheaper, better and has zero maintenance. Self-hosting wins only when data cannot leave, or volume is high enough to amortise the hardware. Be honest about which one you are.

Key Takeaways

• Stack: GPU host + vLLM/Ollama + auth proxy + chat UI (e.g. Open WebUI).
• vLLM gives the best throughput for multi-user serving; quantise to 4-bit.
• Always-on GPU cost usually beats API cost only at high volume.
• Route sensitive queries locally, the rest to a hosted API.

You download the weights, run them on your own machine, and pay no per-token fee. The trade is setup effort and hardware.

What Do You Need to Run Llama 4?

Scout runs with quantisation on a single GPU with 24GB of VRAM, or on a 32GB+ Apple Silicon Mac. The larger Maverick variant needs substantially more — think multiple GPUs or a beefy cloud instance. Run the numbers before you commit; a rented GPU often beats buying one for occasional use.

How Do You Actually Run It?

1. Install a runtime: Ollama or llama.cpp are the easy paths.
2. Pull a quantised build of the model (Ollama handles this with one command).
3. Point your chat client or API wrapper at the local endpoint.
4. Test a real prompt and measure speed — tokens per second tells you if it is usable.

Start with a 4-bit quantised version. Full precision rarely justifies the extra memory for everyday work.

Llama 4 vs a Hosted API: Which Should You Use?

Run it locally for privacy, offline use, or high-volume jobs where per-token fees add up. Use a hosted API when you want the strongest model, no setup and someone else handling updates. For most people, a mid-tier hosted model runs rings around a heavily quantised local Llama 4 on hard tasks.

Key Takeaways

• Llama 4 (April 2025) is open-weight with a MoE design; Scout activates ~17B of 109B params.
• Scout runs on one 24GB GPU or a 32GB Mac with quantisation.
• Start with a 4-bit quantised build via Ollama or llama.cpp.
• Choose local for privacy and volume; hosted for top quality and zero setup.