How to Clone a Voice with Cartesia for Production Use
Cartesia is a voice AI API company. Its two products are text-to-speech with very low latency and voice cloning from a short audio sample. Developers use it where latency matters: live voice agents, interactive characters, real-time translation.
This walks through cloning a voice and producing audio you can ship.
What does a good reference recording need?
One to three minutes of clean speech, one speaker, no music, no echo, no other voices.
Specs in practice: 44.1 kHz or 48 kHz mono, consistent distance from the microphone, natural pace with some variation. Read a paragraph of ordinary prose rather than a list of sentences — intonation on a list is flat and clones reproduce flatness.
What ruins a clone: room echo (the biggest problem), background hum, clipping, and a reference where you are deliberately “doing a voice”. The model copies what you record, including the performance.
If you have a legal or ethical obligation to the voice owner, get written consent before uploading. Cloning without consent is the fastest way to lose an API account and, in many jurisdictions, to break the law.
How do you create and test the clone?
Upload the sample, name the voice, and generate a fixed test phrase — something with numbers, a question and a name. Save that test phrase and reuse it for every voice you create; it is your comparison baseline.
The first generation will sound close but slightly stiff. Tune two knobs:
- Stability or expressiveness. Lower stability gives more variation and more risk of odd delivery. Start mid-range.
- Speed. Most clones default slightly fast; 0.95 to 1.0 suits narration.
Regenerate the same sentence three or four times at each setting and listen for consistency, not for the best take. A voice that is great once and strange twice is unusable in a product.
How do you get low latency in a real application?
Stream the audio. Requesting a whole paragraph and waiting for the file defeats the purpose; the streaming endpoint starts returning audio within a fraction of a second.
Three practical notes from live-agent work:
- Send sentence-sized chunks, not the whole reply. The first sentence starts playing while the rest generates.
- Pre-warm the connection before the user finishes speaking, if your stack allows.
- Measure time-to-first-byte, not total synthesis time. That is what the user experiences.
What does it cost?
Priced per character of generated audio, with a free tier for development and paid tiers from small monthly commitments. Volume pricing is negotiated. Long-form narration — an audiobook, say — is expensive relative to consumer TTS apps, because you are paying for an API with latency guarantees rather than a file export.
Estimate before you commit: characters per minute of speech is roughly 900 to 1,000 in English, so an hour of audio is about 55,000 to 60,000 characters.
Where it is the wrong tool
One-off voiceovers for videos. A consumer TTS app costs less and needs no integration.
Languages and accents it handles less well. Check your target language with the test phrase before building around it.
Emotional acting. Clones reproduce the reference’s range. A monotone reference gives a monotone clone, and no tuning parameter invents performance.
Quick Summary
- Record 1–3 minutes of clean, echo-free speech at 44.1/48 kHz mono; room echo is the main destroyer of clones.
- Keep a fixed test phrase and regenerate 3–4 times per setting — consistency beats the best single take.
- Stream sentence-sized chunks and measure time-to-first-byte, not total synthesis time.
- Pricing is per character with a free development tier; about 55,000–60,000 characters per hour of English speech.
- Wrong choice for one-off video voiceovers and languages you have not validated.
