Self-hosted voice for your AI agents

What is a self-hosted speech server?

It's a small web service on a machine you control that answers OpenAI's audio API: POST /v1/audio/speech turns text into audio and POST /v1/audio/transcriptions turns audio into text. Because the shape is shared, one client works with many servers and models.

Kokoro-FastAPI serves the Apache-licensed Kokoro-82M model and only speaks. Speaches does both, with faster-whisper for transcription and Kokoro or Piper for speech. mlx-audio runs Kokoro and Whisper on Apple Silicon. SFCrew's default model names match mlx-audio's.

What you can do with self-hosted voice in SFCrew

SFCrew sends both speech and transcription to the one Server URL you enter. If that server does both, you get every voice feature in the voice and calls guide.

• Talk instead of typing. Hold the mic in the composer and your server transcribes what you say.
• Hear replies. Choose Speak this reply on a message, or turn on Read replies aloud in a bot's settings.
• Call a bot with Call at the top of the chat, or let a bot call you, for example when you ask it to call you at 5.
• Pick a voice from the server's own list. Kokoro voices show their language, and SFCrew passes that language to the server so a French voice reads with French pronunciation.

When to choose self-hosted voice

Choose it when you don't want a hosted vendor to process your voice, when you want voice with no vendor account, or when you already run local models. It pairs naturally with a self-hosted SFCrew and a local model, so the whole assistant can work offline.

Hosted providers are less work and usually sound more polished. ElevenLabs and Fish Audio offer cloning, OpenAI reuses a key you may already have, and Cartesia is tuned for low latency. If SFCrew runs directly on a Mac, macOS voices speak with nothing to install, but don't transcribe. See the comparison for all six.

Limits and network rules

• Quality and speed depend on your models and hardware. Whisper on a CPU can be slow on a call.
• A speech-only server such as Kokoro-FastAPI can't transcribe. Use a server that does both if you want to hold the mic or make calls.
• Addresses on your own network, localhost and host.docker.internal are refused unless the server sets MCP_ALLOW_PRIVATE_ENDPOINT=true on the API and worker. That setting applies to every user, so leave it off on a public install.
• A hostname that resolves to a private address must also be listed in SFCREW_OPENAI_COMPAT_PRIVATE_HOSTS. Cloud metadata and link-local addresses are always blocked.
• A public address must use HTTPS if you add an API key. Setting SFCREW_OPENAI_COMPAT_ALLOW_PUBLIC=0 blocks public servers, and hides the option if private ones aren't allowed either.
• On the mobile app, calls can use your phone's own voices if you pick a Device voice in the bot's settings. Otherwise replies use this provider.

How to set it up

Whoever runs the SFCrew server allows the address once; then each person connects in the app. The self-hosted model settings cover the network options in full.

1. Start a speech server, for example the Kokoro-FastAPI Docker image, Speaches, or mlx_audio.server on a Mac.
2. On the SFCrew API and worker, set MCP_ALLOW_PRIVATE_ENDPOINT=true for a server on your network, and PHONE_CALLS_ENABLED=true to turn on hold-to-talk and calls. Restart them.
3. Open Settings → Voice, set Provider to Self-hosted and enter the Server URL, such as http://192.168.1.10:8880/v1. SFCrew adds /v1 if you leave it off. If SFCrew runs in Docker and the speech server is on the same machine, use host.docker.internal instead of localhost.
4. Add an API key only if your server needs one. Change Speech model and Transcription model if your server uses other names; Kokoro-FastAPI expects kokoro.
5. Choose Connect, pick a Voice and choose Hear a sample.