How to Cut Mistral Medium 3.1 API Costs in Production

Mistral Medium 3.1 is already cheap per token, but the bill still surprises teams at scale — because most production code sends the same instructions every single call and pays for them every single time. Fixing that is the single biggest lever.

The techniques below are ranked by how much they save, not by how clever they are. Most teams get 40-60% off with the first two alone.

Why Does Prompt Caching Matter So Much?

If your system prompt, tool definitions, or reference documents repeat across calls, cache them. Cached input tokens bill at a fraction of normal input rates, and long system prompts are exactly where token counts balloon.

Practical rules:

• Put the static part first — system prompt, schema, examples. Never interleave variable data into it.
• Keep the cached prefix byte-identical between calls. One changed character invalidates the whole cache.
• Batch requests so the cache stays warm; caches expire after idle periods.

How Do Output Caps Change the Math?

Output tokens cost several times what input tokens do. A model that rambles is an expensive model. Three fixes:

Fix	Effect
Max token cap per call	Hard ceiling on worst-case spend
“Answer in under N words” instruction	Typical cut of 30-50% on output
JSON/structured output mode	Removes boilerplate prose entirely

Structured output is the sleeper here. When Mistral Medium 3.1 returns a typed object instead of a chatty paragraph, you pay for the data and not the framing.

Should You Route Tasks to a Smaller Model?

Yes, for classification, routing, and simple extraction. A small model handles “is this email a refund request?” at a tenth of the cost. Reserve Medium 3.1 for work that needs actual reasoning: code review, multi-field extraction from messy documents, anything with a schema and edge cases.

Build a two-tier router. Send everything to the cheap model, then escalate only on low confidence or failed validation. This is the highest-leverage architecture change and the one teams delay longest.

When Is Self-Hosting Worth It?

The open weights let you run Medium 3.1 on your own GPUs. The break-even is not where people expect. Hosting costs are dominated by GPU hours, and GPU hours are billed whether you are serving traffic or not.

• Under ~50M tokens a month: API is cheaper. Idle GPU cost dominates.
• Steady high throughput: self-hosting wins, especially if you already run other models on the same hardware.
• Strict privacy requirements: self-hosting wins regardless of price.

What Should You Measure?

Log input tokens, output tokens, cached tokens, and latency per request, tagged by feature. Most teams discover one endpoint is responsible for the majority of spend — usually a chatty summarizer nobody optimized. Fix that endpoint before doing anything architectural.

A final caution: aggressive caching plus terse prompts can hurt quality on tasks that genuinely need context. Test output quality after each change. A cheaper wrong answer is not a saving.

Quick Summary

• Cache static prompt prefixes — biggest single saving.
• Cap output tokens and ask for terse or structured replies.
• Route simple tasks to a cheaper model and escalate on low confidence.
• Self-hosting only beats the API at sustained high volume or for privacy.
• Instrument spend per endpoint before making architectural changes.