ElevenLabs for AI agents: give your bots a voice worth hearing

What is ElevenLabs?

ElevenLabs makes speech models and the products built on them: an AI voice generator, dubbing, an audiobook reader, and a platform for voice agents. Developers reach the same models through its API.

Its text to speech is known for sounding human, with emotion and natural pacing, in many languages. You can clone a voice from a recording, design a new one, or pick from a large shared voice library. Its Scribe models turn speech back into text. That combination is why it is often the first name people search when they want an AI agent that talks.

What you can do with ElevenLabs in SFCrew

ElevenLabs both speaks and transcribes, so it covers every voice feature in SFCrew. The voice and calls guide shows where each control lives.

• Talk instead of typing. Hold the mic in the composer, speak, and let go to send. When SFCrew transcribes on the server, as it does in a phone browser or the desktop app, it uses ElevenLabs' scribe_v2.
• Hear replies. Choose Speak this reply on any message, or turn on Read replies aloud in a bot's settings to hear everything it says.
• Call a bot. Choose Call at the top of a chat to talk in real time. You can cut in while the bot is speaking, and the transcript stays in the chat.
• Get called. Ask "call me at 5 to go over the day" and the bot rings you then. A bot may also call if you've missed the same reminder several times.
• Give each bot its own voice. The Voice next to Read replies aloud overrides your default for that bot, so your inbox bot and your sales bot don't sound alike.
• Use your own voices. The voice picker lists the voices in your ElevenLabs account, including ones you cloned or saved from its library.
• Send voice notes from chat apps. A voice note sent to a bot through Telegram, for example, is transcribed with ElevenLabs before the bot reads it. See messaging apps.

When to choose ElevenLabs

Choose ElevenLabs when the voice itself matters: you'll listen to a bot every day, you want it to sound like a person, or you want it to sound like a specific person you have permission to clone. It is also the simplest choice for full voice support, since one key speaks and transcribes.

OpenAI also speaks and transcribes, and suits you if you already have an OpenAI key, though its speech models are scheduled to shut down. Cartesia is built for fast, interruptible calls but only speaks. Fish Audio is another option with cloning and expressive speech. To keep audio on your own hardware, use a self-hosted speech server, or macOS voices if your server runs on a Mac. The comparison puts them side by side.

Limits

• SFCrew always speaks with Eleven Flash v2.5 (eleven_flash_v2_5), ElevenLabs' fast model for conversation. You can't switch to a more expressive model such as Eleven v4 from SFCrew.
• SFCrew doesn't clone voices itself. Create the clone in ElevenLabs first, then pick it in SFCrew.
• Each sentence is generated as a finished clip and played in order, rather than streamed word by word, so there is a short pause before the bot starts talking.
• Speech and transcription go through ElevenLabs' servers and use your ElevenLabs account's credit. If the account runs out, SFCrew tells you so instead of failing quietly.
• Calls and hold-to-talk are a server setting. On a self-hosted server they stay off until the admin turns them on.
• In a desktop browser such as Chrome, SFCrew may use the browser's own speech recognition for the mic instead of ElevenLabs.

How to set it up

Anyone can do this in their own settings, on SFCrew Cloud or a self-hosted server. The key belongs to your account, not the whole server.

1. Create an API key in your ElevenLabs account. Give it permission for text to speech, speech to text and reading voices.
2. In SFCrew, open Settings → Voice, choose ElevenLabs as the Provider, paste the API key and choose Connect.
3. Pick a Voice and use Hear a sample to check it.
4. Optionally, open a bot's settings, expand Advanced, turn on Read replies aloud and give it its own Voice.
5. If you run your own server and want calls and hold-to-talk, set PHONE_CALLS_ENABLED=true on the API and worker.