How to Build an Empathic Voice Agent With Hume AI EVI
Hume AI’s Empathic Voice Interface (EVI) is a real-time voice agent that reacts to emotion in a caller’s speech. This walkthrough builds a working prototype — including the WebSocket plumbing and the latency tuning that makes it feel natural.
What Makes EVI Different From a Standard Voice Assistant?
Most voice assistants pipeline speech-to-text, a language model, then text-to-speech. Hume AI folds emotion into that loop. EVI measures vocal prosody as you speak and shapes its reply’s wording and tone accordingly, all over a single streaming connection.
How Do You Set Up the EVI Connection?
- Generate an API key and access token in the Hume dashboard.
- Open a WebSocket to the EVI endpoint and send a session config with your chosen voice and a system prompt.
- Stream microphone audio in the required PCM format (16-bit, 16kHz mono is the safe default).
- Handle three inbound message types: audio output, emotion scores, and transcripts.
- Send an end-of-turn signal when the user stops, or let EVI’s VAD detect silence.
How Do You Handle Interruptions?
Real conversations overlap. EVI supports barge-in: when the user starts talking, stop playing queued audio immediately and clear the buffer. Skip this and your agent talks over people, which feels robotic fast. Map the interruption event to a hard stop on your audio player, not a fade-out.
How Do You Pick a Voice and Keep Latency Low?
Hume offers a set of voices with different characters; preview each and match it to your brand. On latency, the biggest wins are regional endpoints, sending audio in chunks rather than waiting for a full utterance, and keeping your system prompt short. Every extra sentence in the prompt adds to time-to-first-audio.
| Setting | Suggested value | Why |
|---|---|---|
| Audio format | 16-bit PCM, 16kHz mono | Widest compatibility |
| Turn detection | VAD + explicit end | Fewer false cutoffs |
| Interruption | Hard stop buffer | Feels responsive |
What Does a Prototype Cost?
Hume’s free tier gives limited monthly credits to validate the experience. Production pricing scales with audio minutes, so a support line open 8 hours a day adds up quickly. Estimate before you ship: minutes × concurrency × your traffic pattern.
When Should You Not Use EVI?
Skip it for pure transcription, for high-noise environments where prosody is unreliable, and for regulated use cases where emotion inference raises compliance questions. It shines in coaching, companionship, and support where tone carries real information.
Quick Summary
- EVI streams audio over WebSocket and returns speech plus emotion scores.
- Send 16-bit PCM at 16kHz mono to avoid codec issues.
- Implement barge-in with a hard buffer stop for natural turns.
- Keep prompts short and use regional endpoints to cut latency.
- Pricing scales with audio minutes; prototype on the free tier first.
