How to Cut Cloudflare Workers AI Costs with Caching and Queues

Every Cloudflare Workers AI call is billed in Neurons, and a busy endpoint can quietly burn through a few hundred dollars a month. This second-stage guide is about the boring work that cuts that bill: caching, queueing, and model selection.

Short answer: the three changes that cut Cloudflare Workers AI spend hardest are cache-first lookups in KV, moving non-urgent work to Queues, and swapping 70B models for 8B where quality holds. In practice these took one workload from about $140 to $38 a month.

Which part of the bill is actually large?

Input tokens, usually. Workers AI charges for prompt and completion tokens separately, and the input is what you control. Two things inflate it: stuffing entire documents into the system prompt, and ignoring repeated prompts.

Measure first. Log response.usage alongside the model name, then sort by total Neurons per route after a week. Most teams find 60-80% of the spend comes from two or three routes — a summariser and a semantic search endpoint, typically.

How does a KV cache work in front of the AI binding?

Deterministic prompts are free money you are leaving on the table. Hash the prompt, look it up in KV, return the cached answer if present.

async function cachedRun(env, prompt) {
  const key = await sha256(prompt);
  const hit = await env.CACHE.get(key);
  if (hit) return JSON.parse(hit);
  const r = await env.AI.run("@cf/meta/llama-3.1-8b-instruct", {
    messages: [{ role: "user", content: prompt }]
  });
  await env.CACHE.put(key, JSON.stringify(r), { expirationTtl: 86400 });
  return r;
}

Bind KV as CACHE in wrangler.jsonc. KV reads are cheap and fast at the edge but writes cost more, so keep the TTL sane — 24 hours for factual content, 5 minutes for anything time-sensitive.

Skip caching for anything that leans on fresh data. A cached answer to “what is on my calendar today” is worse than no answer.

When should work move to Queues?

Whenever the user is not staring at the result. Bulk summarisation, embeddings backfill, and content moderation all belong in Queues. The producer returns immediately; the consumer worker does the model call with a longer time budget.

// producer
await env.MY_QUEUE.send({ docId, text });
return Response.json({ queued: true });

Queues also gives you retries for free, which matters because Workers AI occasionally returns a 3049 “out of capacity” error. Inline calls fail the whole request; queued calls retry with backoff.

One caveat: the free plan limits Queues to 10,000 operations per day, and each message delivery counts. Batch 20 items per message rather than sending one per document.

Which model should you pick?

Model	Neurons per 1K tokens (approx)	Best for
Llama 3.1 8B Instruct	4-9	chat, summarising, classification
Llama 3.3 70B	25-45	reasoning, long-form writing
Mistral 7B	3-7	fast drafts, cheap filters
BGE-large	2-5	embeddings
Whisper	40 per audio minute	transcription

The 8B model handles about 85% of typical product requests. Reserve the 70B for the 15% that fails a quality check — route by prompt length or by a cached classifier result rather than guessing.

A practical pattern: run the cheap model first, then escalate only when the answer is shorter than 30 tokens or contains “I don’t know.” That single rule cut one support-bot bill by half.

What about response streaming and cost?

Streaming does not reduce Neuron costs — you pay for the same tokens. It does reduce perceived latency, and more importantly it lets you stop generation early. If the user navigates away, aborting the stream saves the remaining tokens.

const controller = new AbortController();
request.signal.addEventListener("abort", () => controller.abort());

Pass { signal: controller.signal } to env.AI.run. Roughly 10-15% of chat sessions end before generation finishes, so this is a real, if modest, saving.

Where this approach falls short

Caching hides prompt drift. If you change the system message, stale entries serve old-style answers for up to 24 hours — version your cache keys (v3: prefix) and bump the version on every prompt change.

Queues add 200-800 ms latency. For a background summariser that is irrelevant; for an autocomplete it is unusable.

And no amount of caching fixes a bloated prompt. If you are sending 6,000 tokens of context per request, trim the context before optimising anything else.

Key Takeaways

• Workers AI bills in Neurons; input tokens dominate, so measure per route before optimising.
• A KV cache keyed on the prompt hash with a versioned prefix is the cheapest win available.
• Queues suit anything asynchronous and give retries for free; batch 20 items per message to stay inside the free tier.
• Llama 3.1 8B covers most product traffic at 4-9 Neurons per 1K tokens; escalate to 70B only on failures.
• Expect a 40-70% reduction with all three changes in place — not 95%. Anyone promising more is measuring wrong.