How to Deploy Your First AI Endpoint on Cloudflare Workers

Cloudflare Workers runs your JavaScript at the edge closest to the user, and its AI binding lets you call models such as Llama and Whisper without managing a GPU or an API key. This guide walks through deploying a working AI endpoint from an empty folder in roughly 20 minutes.

Direct answer: to run AI on Cloudflare Workers, create a project with npm create cloudflare@latest, add the ai binding in wrangler.jsonc, then call env.AI.run() inside your fetch handler. The free plan includes 10,000 Neurons per day, which is enough for a few hundred small chat requests.

What do you need before you start?

Three things. Node.js 18 or newer, a Cloudflare account, and the paid Workers plan if you plan to run anything real. The free tier allows 100,000 requests per day but compute time is capped at 10 ms per request, and a model call almost never fits in 10 ms.

Install the CLI first:

npm install -g wrangler
wrangler login

The login step opens a browser tab and writes a token to ~/.wrangler. On a headless server, wrangler login --browser false prints a URL you paste into any browser instead.

You also need to enable Workers AI from the dashboard once. New accounts occasionally show “binding not found” until that toggle is flipped, which is a confusing error for a one-minute fix.

How do you scaffold the project?

Run the generator and answer the prompts:

npm create cloudflare@latest ai-endpoint
cd ai-endpoint

Pick “Hello World” as the template, TypeScript if you want types, and decline Git initialisation if the repo already exists. The result is a single src/index.js with a fetch handler and a wrangler.jsonc file.

Open wrangler.jsonc and add the AI binding:

{
  "name": "ai-endpoint",
  "main": "src/index.js",
  "compatibility_date": "2026-09-01",
  "ai": { "binding": "AI" }
}

The binding name matters — env.AI in the code must match binding here. A mismatch produces undefined is not an object, and the message never says which field is wrong.

How do you call a model from the worker?

Replace the default handler with this:

export default {
  async fetch(request, env) {
    const { prompt } = await request.json();
    if (!prompt) return new Response("missing prompt", { status: 400 });
    const r = await env.AI.run("@cf/meta/llama-3.1-8b-instruct", {
      messages: [{ role: "user", content: prompt }]
    });
    return Response.json({ answer: r.response });
  }
};

That is a complete AI endpoint. It accepts POST with {"prompt": "..."}, forwards to Llama 3.1 8B, and returns JSON. No keys, no billing setup inside the code.

Deploy with npx wrangler deploy. You get a https://ai-endpoint.YOURSUBDOMAIN.workers.dev URL in the output. Test it:

curl -X POST https://ai-endpoint.yourname.workers.dev \
  -H 'content-type: application/json' \
  -d '{"prompt":"Explain edge computing in one sentence"}'

Should you stream the response instead?

Yes, for anything with a chat UI. A 300-token answer that arrives all at once feels broken; streamed, it feels instant. Workers AI supports a stream option:

const stream = await env.AI.run("@cf/meta/llama-3.1-8b-instruct", {
  messages, stream: true
});
return new Response(stream, {
  headers: { "content-type": "text/event-stream" }
});

The response is server-sent events. On the client, read it with response.body.getReader() and append each chunk. Note that streaming responses can run past the default CPU time budget, so set "limits": { "cpu_ms": 30000 } in wrangler.jsonc for anything user-facing.

What are the real limits?

Limit	Free plan	Paid plan
Requests per day	100,000	unlimited
CPU time per request	10 ms	30 s default, 5 min max
Neurons per day (AI)	10,000	billed at $0.011 per 1,000
Model catalogue	Llama, Mistral, Whisper, BGE	same plus larger models
Cold start	none reported	none reported

Neurons are Cloudflare’s unit of AI compute. A single Llama 3.1 8B request with a 200-token reply burns roughly 40-60 Neurons, so 10,000 free Neurons is about 200 requests — not 10,000. Model size changes that number fast; a 70B model costs several times more per token.

Who is this setup actually for?

Good fit: someone adding a summariser, translator, or classifier to an existing site who does not want to babysit a Python container. Cold starts are effectively zero and the global distribution is real.

Poor fit: anyone needing fine-tuning, long context windows beyond 8K, or strict model parity with OpenAI. The catalogue is decent but it is not GPT-5. If your product depends on a specific model’s behaviour, call that provider’s API from the worker instead — the binding is optional.

Common failure modes

• Forgetting "ai" in wrangler.jsonc gives env.AI is undefined at runtime, not at deploy time.
• Sending GET requests to a handler that calls request.json() throws immediately; guard the method.
• compatibility_date older than 2024-04-03 breaks some runtime APIs. Set it near today.
• Free-plan 10 ms CPU kills anything that does post-processing on a large response.

Key Takeaways

• Cloudflare Workers + the AI binding gives you a model endpoint with no API key and no server.
• Setup is three files: wrangler.jsonc, src/index.js, and a lockfile. Deploy takes under 30 seconds.
• Free tier: 10,000 Neurons/day, roughly 200 Llama 3.1 8B calls — not enough for production traffic.
• Streaming needs a higher cpu_ms limit; raise it to 30,000 ms before you ship.
• Reach for it when you want low latency and zero ops; skip it if you need a specific closed model or fine-tuning.