Self-Hosting Text-to-Speech: What 400 MB of Kokoro Replaced

Four sites on one server were each calling a cloud text-to-speech API to narrate videos and lesson audio. It worked, it was free, and it sounded like a robot reading a phone menu. This is what happened when we replaced it with a self-hosted model — including the part where the first attempt tried to download three gigabytes of GPU libraries onto a machine with no GPU.

The starting point

Every site was shelling out to edge-tts independently. That gave us three problems, in ascending order of annoyance:

• A cloud dependency for something that should be local. Every line of narration was a network round-trip to a service we don't control and have no contract with.
• No shared voice identity. Each site had picked its own voices in isolation. Nothing was reusable.
• The voice itself. Flat, over-enunciated, unmistakably synthetic. For a talking-mascot video, the voice is the performance.

Picking a model

The requirement was narrow: good English narration, running on CPU, on a box already doing other work. Kokoro-82M fits — it's an 82-million-parameter open-weight model, which is tiny by current standards, and it sounds markedly better than the free cloud tiers.

There are two ways to run it, and the difference matters more than the docs suggest:

We started down the first path without thinking, watched the pip cache pass 2.7 GB, and killed it. The server has no GPU. It was about to install several gigabytes of CUDA libraries that could never execute a single instruction.

The ONNX route is the correct one for a CPU-only box: 277 MB of virtualenv plus 115 MB of weights (int8 model + voice pack). Just under 400 MB, and no compiled GPU dependency anywhere.

One installation, four sites

The obvious mistake here is to install the model inside whichever project you happen to be working on. We put it in a neutral location with its own virtualenv, so nothing depends on any one app's dependency tree:

/var/www/shared/tts/
  say.py            # CLI + importable module
  models/           # weights, fetched by script, not in git
  venv/             # self-contained

Any language can use it by shelling out — PHP, a cron job, a Python pipeline:

/var/www/shared/tts/venv/bin/python /var/www/shared/tts/say.py \
  --text "Retrieval practice beats rereading." \
  --voice am_michael --out /tmp/line.wav

The whole wrapper is about thirty lines. Kokoro is loaded lazily so a process that never speaks never pays for it, and the function reports which engine produced the audio:

_kokoro = None

def _engine():
    global _kokoro
    if _kokoro is None:
        from kokoro_onnx import Kokoro
        _kokoro = Kokoro(MODEL, VOICES)
    return _kokoro


def synth(text, out_path, voice="am_michael", speed=1.0, fallback_voice=None):
    """Synthesise text to out_path. Returns the engine actually used."""
    try:
        import soundfile as sf
        audio, sr = _engine().create(text, voice=voice, speed=speed, lang="en-us")
        sf.write(out_path, audio, sr)
        return "kokoro"
    except Exception as e:
        # deliberately broad: never lose the audio over a TTS failure
        sys.stderr.write(f"[say] kokoro failed ({e}); falling back to edge-tts\n")
        subprocess.run(["edge-tts", "--voice", fallback_voice or "en-US-GuyNeural",
                        "--text", text, "--write-media", mp3], check=True)
        subprocess.run(["ffmpeg", "-y", "-i", mp3, out_path], check=True)
        return "edge-tts"

That return value sounds trivial and turned out to be the most useful design decision in the whole change — see the fallback section below.

What it costs: CPU time

On 8 cores, Kokoro runs at roughly half realtime: a 6.8-second line takes about 13 seconds to synthesise. For batch work that's free — a video render goes from ~4 minutes to ~6, and nobody is waiting.

For a web request it is disqualifying, and this is the trap. One of the sites exposes an on-demand "listen to this card" endpoint with a six-second timeout. A twenty-second clip would take about forty seconds to generate and hold a PHP-FPM worker for the entire time. Enough concurrent requests and you've taken the site down with a feature nobody asked to be slower.

So the interactive endpoint stayed on the cloud engine. Batch callers opt in explicitly:

TtsSynth::synth($text, $voice, $dest, $timeout = 300, $preferKokoro = true);

The language trap

The education site supports German, Spanish, French, Italian, Japanese, Korean and Croatian. Kokoro v1.0 has no German, Korean or Croatian voices.

A blanket switch would not have thrown an error. It would have quietly read Croatian flashcards in an American accent, and the only people who'd have noticed are the users. So the voice map only covers languages the model actually speaks, and returns null for everything else:

/**
 * edge-tts voice -> Kokoro voice. English only on purpose: Kokoro v1.0 ships
 * no German, Korean or Croatian voices, so those must stay on edge-tts rather
 * than be silently swapped for a wrong-language voice.
 */
private const KOKORO_VOICES = [
    'en-US-AvaNeural'   => 'af_heart',
    'en-US-JennyNeural' => 'af_bella',
    'en-US-GuyNeural'   => 'am_michael',
    'en-GB-SoniaNeural' => 'bf_emma',
];

public static function kokoroVoiceFor(string $voice): ?string
{
    return self::KOKORO_VOICES[$voice] ?? null;   // null => keep edge-tts
}

Every language that isn't in that map falls through untouched. Adding a language is a deliberate act, not a default.

This is the general shape of the problem with swapping an infrastructure component: the failure you should fear isn't the one that crashes, it's the one that returns something plausible.

Always leave a way back

Speech synthesis sits in the middle of a pipeline. If it fails at 3am, a video renders silent and gets published silent. So the shared helper falls back to the old cloud engine on any failure, and — importantly — reports which engine it used. Degradation is visible in logs instead of being discovered on a published video.

Results

• Four sites, one engine, one installation: two video pipelines, an education platform's lesson audio, and a market-news video generator.
• ~400 MB total footprint, CPU-only, no GPU, no per-call cost, no rate limits.
• 54 voices available; each brand now has a deliberate voice identity rather than whichever default was picked first.
• The cloud engine is still there, on purpose, for the two places where it's genuinely the better tool.

Calling it from an existing PHP codebase needs no new dependency — the batch job opts in, the web endpoint doesn't:

// batch: better voice, 300s budget
TtsSynth::synth($text, $voice, $dest, 300, preferKokoro: true);

// interactive endpoint: unchanged, still the fast cloud engine
TtsSynth::synth($text, $voice, $dest, 6);

And from a Python pipeline, the prosody mapping is the only fiddly part. Kokoro exposes speed but not pitch, so a rate is converted and the pitch component is dropped rather than faked:

def rate_to_speed(rate: str) -> float:
    """edge-tts style rate ("+6%", "-4%") -> Kokoro speed multiplier."""
    try:
        pct = float(str(rate).strip().rstrip("%"))
    except (TypeError, ValueError):
        return 1.0
    return round(max(0.5, min(2.0, 1.0 + pct / 100.0)), 3)

What we'd tell you before you do this

Self-hosting a model is not automatically the right answer. It was here because the workload is batch, the quality gap was audible, and the footprint is small enough to be uninteresting. Change any one of those and the calculation flips.

Check three things before you start: is your workload batch or interactive; does the model actually cover every language you serve; and are you about to install a GPU stack on a machine that doesn't have one.