How to Track LLM Cost Per Customer with Portkey Observability

November 9, 2026

The question that ends most AI product pricing discussions is “what does one customer cost us?” Portkey logs every request that passes through its gateway with token counts and cost, and lets you attach your own metadata to each one — which makes per-customer attribution a filtering problem rather than a forecasting exercise.

The setup is small: a header on every request and a habit of tagging. The payoff is a number you can put next to your plan price.

Why is per-customer cost hard to get from provider dashboards?

Provider dashboards tell you the total bill. They do not know which end user made which call, so a product with a free tier, a paid tier and a heavy internal testing account all look the same.

Cost also varies wildly inside one product. A user who asks short questions costs cents; a user who uploads a 200-page PDF every week for summarisation costs dollars. Average cost per user hides both of these, and it is the average that gets quoted in pricing meetings.

How do you tag requests with a customer ID?

Portkey accepts a trace_id and arbitrary metadata on each request, passed as headers:

headers = {
  "x-portkey-metadata": json.dumps({
    "_user": "cust_8841",
    "plan": "pro",
    "feature": "summary",
  })
}

Three tags are worth the discipline:

  • User ID — your own customer identifier, not an email.

  • Feature — which product surface made the call, because features have wildly different unit economics.

  • Plan — free, pro, enterprise, so you can compute margin per tier.

Set these in one place: a request interceptor or a middleware that reads the authenticated user from the request context. Do not let individual call sites construct the tag, because the day one of them forgets is the day your report has a large “unknown” bucket.

How does Portkey know the cost?

For standard provider models it uses a price table and the token counts returned by the provider. For anything it does not know — a fine-tune, a self-hosted model, a proxy — you can pass custom_cost per request and it uses yours.

Self-hosted and fine-tuned models are exactly where teams get attribution wrong. A local Llama endpoint returns tokens but no price; without custom_cost the dashboard shows zero, and zero reads as “free” in a report nobody has questioned yet.

Compute a per-token price from your GPU-hour cost and the throughput you actually observe, and pass it. It will be wrong at first; it will be closer than zero.

How do you turn logs into a per-customer report?

The dashboard filters by metadata but is built for browsing, not for a monthly billing reconciliation. Export the data.

A workable monthly routine:

  1. Export request logs for the period with metadata included. 2. Group by user ID and feature; sum tokens in and out, and cost. 3. Join with your own usage data (documents processed, messages sent) to compute cost per unit of your product’s headline action. 4. Sort by total cost; the top 5% of users usually account for 30–50% of spend.

That fourth line is the actionable part. It tells you whether your worst-case users are the ones on your cheapest plan, which is a pricing bug, not a usage pattern.

How do you cap the damage from a single account?

Portkey supports budget and rate limits per metadata value, which means limits keyed on your customer ID rather than on a shared API key.

Practical settings for a consumer product:

  • Daily token budget per free-tier user — set it high enough to complete a normal session, low enough that a loop cannot run for hours. – Requests-per-minute per user — catches scripted abuse faster than a token budget does. – Alert, then block. Warning thresholds at 80% cost less in support tickets than hard blocks at 100%.

These limits are not usage caps for the product; they are circuit breakers. Distinguish them in your error messages so a paying user hitting a safety limit does not think they have run out of credits.

What does the gateway itself cost, and what does it miss?

Portkey’s free tier covers a meaningful monthly request volume; paid tiers scale by request count with enterprise pricing for SSO and self-hosting.

What it does not count: the tokens your app burns calling the model from a script that bypasses the gateway, and any provider-side charges that are not per token (image generation, fine-tune training, embedding storage). Route everything through one gateway or accept that your report covers 85% of spend and say so.

When is per-customer cost the wrong metric?

If your product bills by seats and each seat has a fixed monthly allowance, cost per seat matters more than cost per token. If your gross margin is dominated by infrastructure other than inference — vector search, file storage, audio minutes — then token spend is a small slice and optimising it carefully is misplaced effort.

Check the split before you spend a week on tagging.

Quick Summary

  • Pass x-portkey-metadata with user ID, feature and plan from one middleware, so no call site can forget.

  • Standard models are priced from a table; pass custom_cost for fine-tunes and self-hosted endpoints or their cost shows as zero.

  • Export logs monthly and group by user and feature; the top 5% of users commonly account for 30–50% of token spend.

  • Set per-user token budgets and request-rate limits at the gateway as circuit breakers, with alerts before hard blocks.

  • Anything that bypasses the gateway is invisible in the report, so route all model traffic through one entry point.

Tags: