How to Self-Host Mistral Voxtral and Compare It to Whisper
Running Mistral Voxtral locally makes sense when audio is confidential, volume is high, or you need to control latency. Here’s the practical path: pick the right variant, set up chunking for long files, and know where it wins against Whisper before you commit.
Direct answer: The 3B Voxtral weights run on a single 16GB GPU and handle 30 minutes of audio per pass; self-hosting cuts per-minute cost to zero after hardware, but you trade away managed scaling and Whisper’s 90+ language coverage.
Which Voxtral size should I self-host?
Two options ship as open weights. The 3B model runs on a 16GB card (RTX 4080 or A10) with room to spare. The larger 24B variant needs roughly 48GB of VRAM in FP16 or about 24GB with 4-bit quantization. If your clips are under 30 minutes and you don’t need maximum accuracy, 3B is the sane default — it costs a fifth of the hardware and transcribes clean speech nearly as well.
How do I set it up step by step?
- Install runtime:
pip install vllm transformers torch. - Pull weights:
huggingface-cli download mistralai/Voxtral-Mini-3B-2507. - Serve it:
vllm serve mistralai/Voxtral-Mini-3B-2507 --max-model-len 32768. - Point your client at
http://localhost:8000/v1and send audio as you would to the hosted API.
The model context is 32k tokens. Because audio tokens are heavier than text tokens, that works out to the 30-minute ceiling you’ll hit in practice.
How do I handle audio longer than 30 minutes?
Split on silence, not on fixed timestamps. Fixed 30-minute cuts will slice a sentence or a pricing decision in half. Use something like ffmpeg’s silencedetect to find natural gaps, then patch any words that landed on a boundary by re-running a short overlap window. A 15-second overlap between chunks catches most dropped words.
Mistral Voxtral vs OpenAI Whisper: which should you pick?
| Factor | Mistral Voxtral 3B | Whisper large-v3 |
|---|---|---|
| Languages | 4 (EN/FR/ES/IT) | 90+ |
| Max audio per call | 30 min | ~30 sec segments, looped |
| Audio understanding | Yes (Q&A on audio) | Transcription only |
| Self-host VRAM | 16GB | ~10GB |
| Diarization | No | No |
Pick Voxtral when you want reasoning on top of the transcript — summarizing a call, extracting action items — in one pass. Pick Whisper when you need broad language support or you’re already tooled around its ecosystem.
What are the real downsides?
Four languages is limiting if you handle global calls. There’s no diarization, so multi-speaker meetings need pyannote added on top, which roughly doubles your pipeline complexity. And the 24B variant’s accuracy edge over 3B rarely justifies the GPU bill unless you’re doing medical or legal transcription.
Is self-hosting actually cheaper?
At $0.001 per minute on the hosted API, 10,000 minutes a month costs about $10. An A10 GPU rental runs $0.60–$1.00 an hour, or $430+ monthly if left running. Self-hosting only wins on privacy or at very high volume — not on raw cost at hobby scale.
Key Takeaways
- 3B Voxtral fits a 16GB GPU; 24B needs ~48GB in FP16.
- Chunk long audio on silence, not fixed timestamps, with a 15-second overlap.
- Voxtral adds audio Q&A; Whisper covers 90+ languages but transcribes only.
- Neither does diarization — budget for pyannote if you need speaker labels.
- Hosted API is cheaper than GPU rental below roughly 5,000 minutes a month.
