The techniques below are ranked by how much they save, not by how clever they are. Most teams get 40-60% off with the first two alone.

Why Does Prompt Caching Matter So Much?

If your system prompt, tool definitions, or reference documents repeat across calls, cache them. Cached input tokens bill at a fraction of normal input rates, and long system prompts are exactly where token counts balloon.

Practical rules:

• Put the static part first — system prompt, schema, examples. Never interleave variable data into it.
• Keep the cached prefix byte-identical between calls. One changed character invalidates the whole cache.
• Batch requests so the cache stays warm; caches expire after idle periods.

How Do Output Caps Change the Math?

Output tokens cost several times what input tokens do. A model that rambles is an expensive model. Three fixes:

Structured output is the sleeper here. When Mistral Medium 3.1 returns a typed object instead of a chatty paragraph, you pay for the data and not the framing.

Should You Route Tasks to a Smaller Model?

Yes, for classification, routing, and simple extraction. A small model handles “is this email a refund request?” at a tenth of the cost. Reserve Medium 3.1 for work that needs actual reasoning: code review, multi-field extraction from messy documents, anything with a schema and edge cases.

Build a two-tier router. Send everything to the cheap model, then escalate only on low confidence or failed validation. This is the highest-leverage architecture change and the one teams delay longest.

When Is Self-Hosting Worth It?

The open weights let you run Medium 3.1 on your own GPUs. The break-even is not where people expect. Hosting costs are dominated by GPU hours, and GPU hours are billed whether you are serving traffic or not.

• Under ~50M tokens a month: API is cheaper. Idle GPU cost dominates.
• Steady high throughput: self-hosting wins, especially if you already run other models on the same hardware.
• Strict privacy requirements: self-hosting wins regardless of price.

What Should You Measure?

Log input tokens, output tokens, cached tokens, and latency per request, tagged by feature. Most teams discover one endpoint is responsible for the majority of spend — usually a chatty summarizer nobody optimized. Fix that endpoint before doing anything architectural.

A final caution: aggressive caching plus terse prompts can hurt quality on tasks that genuinely need context. Test output quality after each change. A cheaper wrong answer is not a saving.

Quick Summary

• Cache static prompt prefixes — biggest single saving.
• Cap output tokens and ask for terse or structured replies.
• Route simple tasks to a cheaper model and escalate on low confidence.
• Self-hosting only beats the API at sustained high volume or for privacy.
• Instrument spend per endpoint before making architectural changes.

If your workload is high-volume and mostly mechanical — summarizing, extracting, reviewing diffs — the price difference is the whole argument.

What Is Mistral Medium 3.1?

It is the third point release of Mistral’s mid-tier model, aimed at production workloads that need solid reasoning without frontier pricing. It has a large context window, supports tool calling and JSON output, and is available both as a hosted API and as open weights for self-hosting on suitable hardware.

It is not a reasoning-first model. On multi-step planning or tricky math, a larger competitor will beat it. On everything else, the gap is smaller than the price gap suggests.

How Do You Get an API Key and Make a First Call?

1. Create an account at the Mistral console and generate an API key.
2. Export it as an environment variable — never hardcode it.
3. Send a chat completion request to the API endpoint with the model name set to Mistral Medium 3.1.
4. Set your system prompt first. It handles a firm instruction like “answer in under 120 words” reliably, which cuts token spend immediately.

For local use, download the open weights and serve them with a standard inference server. Expect it to need a serious GPU; this is not a laptop model.

How Much Does It Cost?

Exact rates change, but the shape holds: input tokens cost far less than output tokens, so verbosity is expensive. Prompts that ask for terse answers save real money at scale. Mistral Medium 3.1 accepts both system and user roles, and caching repeated prefixes is where large workloads see the biggest win.

Where Does It Excel in Practice?

Three workloads stand out:

• Code review — it catches null-handling and off-by-one bugs reliably and explains them without padding.
• Structured extraction — JSON output mode rarely breaks format, which matters when the output feeds a database.
• Rewriting and summarizing — good tone control when you give examples, weak when you describe the tone abstractly.

Where Does It Fall Short?

Long-horizon agentic tasks. Give it a 20-step plan and it will drift around step eight. It also tends to stop short on ambiguous instructions rather than asking — good for automation, annoying for exploration. And for nuanced prose in a specific authorial voice, the bigger models still read better.

Who Should Use It?

Teams running thousands of API calls a day on well-defined tasks. Solo developers prototyping an assistant. Anyone who has a GPU and privacy requirements. Not the right pick if you need frontier reasoning or the best prose available at any cost.

Quick Summary

• Mistral Medium 3.1 is a mid-size model with a large context window.
• It supports tool calling, JSON output, and self-hosting.
• Output tokens cost more than input — ask for terse answers.
• Strong at code review, extraction, and rewriting.
• Weak at long multi-step agent tasks and nuanced prose.