ElevenLabs' new AI model supports enhanced speech capabilities, 90+ languages
What's the story
ElevenLabs, a leading player in the field of artificial intelligence (AI), has launched its latest speech models: ElevenLabs v4 and v4 Turbo. These innovative models offer enhanced control over expression, reduced latency for voice agents, and support for over 90 languages. The company had previously unveiled its v3 model last year and teased the upcoming version at an event in Warsaw earlier this year.
Model evolution
New architecture for faster voice cloning
For the v4 generation of models, ElevenLabs is adopting a new architecture. This change will allow better control and faster cloning capabilities.
The company claims that with this new model, users can clone a voice with just 10 seconds of audio.
On the creative side, it handles voice identity over longer text chunks better than its predecessors.
Feature upgrades
Improved expression changes and language support
The v4 model also takes into account the context of the text while reading it aloud, allowing for more nuanced expression changes.
ElevenLabs introduced inline tags with its v3 model to define expression and is expanding this feature in v4.
The new version supports 90 languages, up from 70 in the previous iteration, with significant quality improvements noted in Japanese, Brazilian Portuguese, Mandarin, and Cantonese.
Enterprise focus
Model optimized for voice agents
The new model is specifically designed for voice agents, with lower latency for more fluid conversations. The v4 can also start generating audio as soon as the underlying large language model (LLM) starts generating answers.
It can handle confrontations, escalations, and holds differently to ensure better issue resolution.
This comes as competition in speech models heats up with start-ups like Cartesia and Deepgram developing expressive speech models.
Business expansion
ElevenLabs raised $500 million from Sequoia at $11 billion valuation
ElevenLabs raised $500 million from Sequoia in a funding round that valued the company at $11 billion. There are already rumors of a follow-up fundraising round that would value the company at $22 billion.
The company's annualized revenue run rate has jumped from about $330 million at the beginning of this year to over $600 million.