Kokoro can now clone any voice with just ~600 MB of extra VRAM!
retained text
While looking for audio models, I came across OpenVoice. It's getting pretty old, but surprisingly still ranks high on TTS leaderboards. What caught my attention was its design: essentially **TTS + voice conversion**. The ToneColorConverter can run independently of the TTS model. That got me thinking: Can we pair it with other TTS models? I tried it with Kokoro, and the results were pretty good! So I added a small recipe `any-tts` to audio.cpp for Kokoro + OpenVoice ToneColorConverter. Recipes were introduced in v0.9.2 as predefined pipelines that combine multiple audio models. The overheads: * **\~600 MB** additional peak VRAM * **\~10% slowdown** on CUDA * \~0.5× the original speed on CPU, but still **2× real-time** You can also try pairing any tiny TTS model with ToneColorConverter. The TTS model determines the speech content, pronunciation, and prosody, while the VC model transfers the target speaker's voice characteristics. The built-in voice you choose can also affect the final output quality. A simple way to give existing TTS models voice cloning capabilities without retraining them! **Disclaimer**: This experiment is more about extending the capabilities of existing TTS models, not introducing another voice cloning model. I used synthetic voices as references, and the speaker similarity scores measured by CAM++ range from 0.7 to 0.8. So the generated voices largely match the reference samples I provided. This is one way to create custom voices for arbitrary TTS models. For better results, you may need to experiment with different built-in TTS voices and higher-quality reference audio. Both can significantly affect the final quality and speaker similarity. For example, with the same reference sample, `am_michael` scores 0.4, while `am_adam` can reach 0.7.
document fetched 2026-10-30 00:44 · canonical https://reddit.com/r/LocalLLaMA/comments/1xiplj7/kokoro_can_now_clone_any_voice_with_just_600_mb