How to Run GPT-OSS Locally on Your Own Machine
GPT-OSS is OpenAI’s open-weight language model family that you download and run on your own hardware, no API key required. You can have it answering prompts locally in roughly 15 minutes if you already have Python and enough RAM.
What is GPT-OSS and what sizes does it come in?
GPT-OSS ships in two main variants: a 20B parameter model and a larger 120B one. Both use a mixture-of-experts design, which means only a slice of the network fires on each token. That keeps the 20B model usable on a single 16GB GPU or a 32GB Mac, while the 120B version realistically wants 80GB+ of VRAM or a beefy multi-GPU rig.
Weights are on Hugging Face under a permissive license, so commercial use is allowed. The catch: no official fine-tuning recipe is bundled, and the safety tuning is lighter than the hosted OpenAI models.
How do you install GPT-OSS?
The fastest route is Ollama, then llama.cpp if you want full control.
- Install Ollama:
curl -fsSL https://ollama.com/install.sh | shon Linux or macOS, or grab the Windows installer. - Pull the small model:
ollama pull gpt-oss:20b. Expect about 13GB of download. - Run it:
ollama run gpt-oss:20band type a prompt.
On the 20B model I get roughly 35-45 tokens per second on an RTX 4090 at 4-bit quantisation. A 2020 MacBook Air with 8GB will not run it — you need at least 16GB of unified memory, and 32GB feels comfortable.
Is GPT-OSS free to use?
Yes. The weights cost nothing and there is no per-token billing. Your real cost is electricity and hardware. That is the whole appeal for teams who cannot send data to a third-party API — legal, medical and defence shops especially.
What you give up: no usage dashboard, no autoscaling, and you own every outage. If your GPU driver updates and breaks CUDA, that is your problem at 2am.
How does GPT-OSS compare with hosted models?
| Factor | GPT-OSS 20B (local) | Hosted API |
|---|---|---|
| Cost | Hardware only | Per token |
| Privacy | Fully offline | Data leaves your network |
| Speed | 35-45 tok/s on a 4090 | Faster, network-bound |
| Setup | ~15 minutes | An API key |
Who should run GPT-OSS and who should skip it?
It fits you if data residency matters, if you hit API rate limits constantly, or if you want to experiment without a bill running. Skip it if you need the strongest reasoning available or multi-modal input — the open weights lag the frontier models on hard math and long-context tasks, and there is no vision input here.
Key Takeaways
- GPT-OSS is OpenAI’s free open-weight model, downloadable and runnable offline.
- The 20B variant runs on 16GB VRAM; the 120B wants 80GB+.
- Ollama gives you a working local chat in about 15 minutes.
- No API cost, but you supply the hardware and the uptime.
- Not the right pick if you need true frontier reasoning or vision.
