How to Set Up LLM Logging with Helicone

Helicone is an observability layer for LLM apps. Point your requests through it and you get logs of every call, token counts, cost, latency and cache hits. Setup is a base-URL change, not a rewrite. This guide covers the proxy path and what to actually watch.

What problem does Helicone solve?

LLM apps fail quietly. A prompt change doubles token cost; a slow model swap adds seconds of latency; a retry loop quietly triples spend. Helicone records each request so those changes show up as numbers instead of guesses. It works with OpenAI, Anthropic and other providers, plus self-hosted models.

How do you connect your app?

1. Create a Helicone account and copy the API key.
2. Change your provider base URL to Helicone’s proxy and add the auth header.
3. Send one test request and confirm it appears in the dashboard within seconds.
4. Add custom properties — user ID, session, environment — so you can filter later.

Alternatively, use the async integration that runs as a logger alongside your SDK. The proxy is faster to set up; the async path avoids adding a network hop.

What should you watch in the dashboard?

• Cost per user or feature. Total spend hides the feature that is draining budget.
• p95 latency. Averages mask the slow tail that users actually notice.
• Cache hit rate. Helicone can cache identical prompts, cutting cost on repeated queries.
• Error rate by model. Provider outages and rate limits show up here first.

Is Helicone free?

There is a generous free tier covering a meaningful volume of requests per month, which is enough for small apps and staging. Higher volume, longer retention and team features move to paid plans. Self-hosting the open-source version avoids usage billing if you run the infrastructure yourself.

What are the trade-offs?

The proxy adds a network hop, so measure latency before and after. It also sees your prompts — decide whether sensitive data should be logged or scrubbed before it reaches the proxy. And caching can surprise you: a prompt that should be dynamic must bypass the cache.

Quick Summary

• Helicone logs LLM requests and surfaces cost, latency and cache stats.
• Connect via a base-URL proxy change or an async logger; add custom properties.
• Watch cost per feature, p95 latency, cache hit rate and per-model errors.
• Free tier covers small apps; paid adds volume, retention and team features.
• Trade-offs: extra hop, prompt visibility, and cache surprises on dynamic prompts.