01 · AI · video · audio · 2026
A local English→Thai dubbing pipeline
Hours of finished video needed a second language. Cloud dubbing was quoted at roughly three dollars a minute. The goal: professional-feeling dubs at zero dollars a minute, on the Mac already on the desk.
PartsDemucs · MLX Whisper · LM Studio · F5-TTS-THAI · ffmpeg · Python · Apple Silicon
Problem
Hours of English-language video, a Thai audience, and a budget that said no to per-minute cloud dubbing. “Good enough” had a number: about 93% of frontier quality — not perfection, but nothing that makes a listener wince.
Pieces
- Demucs separates the voice from music and room noise, so the music survives the dub.
- MLX Whisper transcribes with timestamps on Apple Silicon.
- A local LLM (via LM Studio) translates line by line, keeping the timing.
- F5-TTS-THAI clones a reference voice from a ≤12-second sample and speaks each line.
- ffmpeg cuts long videos into ~3-minute chunks at silent gaps and stitches the dubbed parts back.
- One Python core drives all of it, with
--resumeso a fixed line doesn't mean a full re-run.
Put back together
The first real five-minute review landed “quite surprised at the quality level.” Two rules made that possible: never judge synthesized speech by listening to a few seconds — Whisper round-trips every generated line and compares it to the intended text, because fluent-sounding garbage is the failure mode — and treat the transcript review as part of the pipeline, not an optional extra. What remains is sync polish: pops, dead pauses, an occasional stretched line, each itemized in a roadmap with a diagnosis and a fix.
Would do differently
Curate one reference voice per speaker up front and store it. Auto-picking a reference clip per video cost a full round of re-dubbing.