I made VoxCPM2 5.3x smaller with TTF. Same quality, voice cloning, runs real-time on a laptop CPU with no Python
retained text
Several days ago I made this post [https://www.reddit.com/r/LocalLLaMA/comments/1xtgnh9/you\_can\_now\_train\_your\_own\_llms\_with\_less\_memory/](https://www.reddit.com/r/LocalLLaMA/comments/1xtgnh9/you_can_now_train_your_own_llms_with_less_memory/) about training your own LLMs with less memory, u/[Silver-Champion-4846](https://www.reddit.com/user/Silver-Champion-4846/) wanted me to try this on a text to speech model, VoxCPM 2B. I converted the base VoxCPM 2B model from bf16 (at about 4.96GB) to ternary 1.58b quant (\~850MB, 5x smaller). The reason its not 10x smaller is because the 850MB includes Q8 VAE and DiT. Here are the results (generated by my agent): ## Results **Size:** 827 MB model + 100 MB audio VAE (Q8_0, recommended), vs 4.96 GB for the original VoxCPM2 in bf16: **5.3x smaller**. The language models (text-semantic LM, residual LM, local encoder) are ternary (-1, 0, +1); the diffusion transformer is the original VoxCPM2 DiT rounded to 8 bits. ### Blind listening test (-1, 0, +1); the diffusion transformer is the original VoxCPM2 DiT rounded to 8 bits. ### Blind listening test 16 clips per model (8 sentences x 2 arks mapped to a 1-4 score. | model | mean score | buzzy / robotic | very good or perfect | |---|---|---|---| | VoxCPM2 (full precision) | 2.75 | 1/16 | 6/16 | | **ttf-VoxCPM2 (this release)** | **3.02** | **1/16** | **7/16** | | ttf-VoxCPM2, first release | 1.66 | 8/16 | 0/16 | ### Automatic metrics Voice cloning, 100 English + 20 Chinrecision on the same items and seeds. UTMOS and similarity: higher is bettter. | model | UTMOS en | WER en | speaker similarity | CER zh | |---|---|---|---|---| | VoxCPM2 (full precision) | 4.25 | 2.70% | 0.960 | 0.91%* | | **ttf-VoxCPM2** | **4.31** | **2.61%** | **0.956** | 1.48%* | English naturalness is level with fu-0.01, +0.15]. It replicates on 100new items at +0.01. \* On ~1000 characters. ### Zero-shot, 27 languages FLEURS, 8 sentences per language, Whr rate (lower is better). | model | median CER | mean CER | |---|---|---| | VoxCPM2 (full precision, PyTorch) | 0.031 | 0.061 | | **ttf-VoxCPM2 (ttf-home runtime)** | **0.043** | **0.057** | ### Voice cloning from a 5-20 s clip 20 LibriTTS-R speakers, WavLM speaker similarity (higher is better). | setup | similarity | |---|---| | two real clips of the same speaker | 0.936 | | **ttf-VoxCPM2 with --ref** | **0.933** | | VoxCPM2 full precision, same mode | 0.926 | | ttf-VoxCPM2 zero-shot (no reference) | 0.670 | Cloning WER is 1.9%, the same as ful ### Speed Real-time factor = generation time /r). | hardware | RTF | |---|---| | RTX 4070 Ti SUPER | 0.16 | | laptop RTX 4080 | 0.24-0.34 | | laptop CPU only, i9-14900HX (AVX-VNNI) | 0.8-1.1 (about real time) | | CPU only, i7-9800X (no VNNI) | 2.5 | It runs in our own C++ runtime builth, on an NVIDIA GPU or a CPU. ## Links - Model: https://huggingface.co/oguzeldereli/ttf-VoxCPM2 - Runtime (ttf-home): https://github.com/oguzeldereli/ttf-home - Conversion code: https://github.com/oguzeldereli/ttf-conversion - TTF ternary training: https://github.com/oguzeldereli/ttf-ternary - Base model: https://huggingface.co/openbmb/VoxCPM2 Licenses: the model is Apache-2.0, sis PolyForm Noncommercial 1.0.0. Hoping to hear feedback. Enjoy! Edit: I will go to sleep now so I won't be able to respond, I will respond to everyone though, good night!
document fetched 2026-11-19 00:43 · canonical https://reddit.com/r/LocalLLaMA/comments/1xz4msh/i_made_voxcpm2_53x_smaller_with_ttf_same_quality