NVIDIA's RE-USE Beats 500M-Parameter Models at Cleaning Speech With Just 9.6M

retained text

NVIDIA's RE-USE Beats 500M-Parameter Models at Cleaning Speech With Just 9.6M
NVIDIA's new 9.6M parameter RE-USE model strips noise, echo and distortions from speech across 7 sampling rates and beats models 25x larger.
- NVIDIA released NemotronLabs RE-USE, a 9.6M parameter open multilingual speech enhancement model.
- Beats 250M and 500M parameter rivals on PESQ, STOI, speaker similarity, and ASR accuracy.
- Handles noise, reverb, clipping, codec artifacts, packet loss, and bandwidth limits across 7 sample rates.
- Built on bi-directional Mamba (30 layers) with convolutional encoder/decoder; linear-time sequence modeling.
- Trained on 2,500 hours of synthetic data in English, German, Spanish, French, and Mandarin.
- Released under OpenMDW 1.1 license, cleared for commercial use, with a Gradio demo on Spaces.
NVIDIA’s Nemotron Labs has released RE-USE, a 9.6-million-parameter speech enhancement model that removes noise, reverberation, clipping, codec artifacts, packet loss, and microphone distortion while preserving intelligibility and speaker identity.
The published benchmark results place RE-USE ahead of open models with roughly 26 to 52 times as many parameters. Its FP32 weights occupy about 38.4 MB before packaging, making the checkpoint easier to add to transcription, media cleanup, and speech-dataset pipelines.
One model, seven sample rates
RE-USE accepts audio sampled at 8, 16, 22.05, 24, 32, 44.1, and 48 kHz without requiring a separate checkpoint for each rate. NVIDIA describes the model as language-agnostic and trained it with English, German, Spanish, French, and Mandarin speech.
During overlapping speech, the training target preserves the main speaker and suppresses competing voices. That behavior suits interviews, podcasts, and voice recordings where one speaker should remain prominent. Source separation and diarization require separate tools when every speaker must be retained or identified.
Small checkpoint, strong reported scores
On the URGENT 2025 non-blind test set, NVIDIA reports higher scores for RE-USE than for the 250-million-parameter Sidon and 500-million-parameter UniPASE models across the four metrics below.
- PESQ estimates perceived speech quality.
- STOI estimates speech intelligibility.
- Speaker similarity measures preservation of the speaker’s identity.
- ASR character accuracy measures downstream speech-recognition performance.
The reported 16 kHz results show ASR character accuracy rising from 76.5% on noisy input to 87.1% after enhancement. On the 44.1 kHz subset, RE-USE records 0.90 speaker similarity, compared with 0.57 for Resemble Enhance.
Mamba keeps sequence costs down
RE-USE places a 30-layer bidirectional Mamba core between a convolutional encoder and decoder. The core models time-frequency patterns associated with speech and recording defects, while the encoder and decoder transform audio into and out of the model’s internal representation.
Mamba is a state-space sequence architecture whose computational cost grows linearly with sequence length. Standard full self-attention grows quadratically, so Mamba can process longer sequences with a lower scaling cost.
NVIDIA trained the model on about 2,500 hours of synthetic mixtures. Clean speech came from LibriVox, LibriTTS, VCTK, Multilingual LibriSpeech, and Common Voice; noise came from AudioSet and Freesound. Training also used about 60,000 simulated room impulse responses and 70 microphone impulse responses, which reproduce acoustic spaces and microphone characteristics.
The accompanying NeurIPS 2026 paper describes the training targets, architecture, and data-quality choices in detail.
Run it from the repository
The model card publishes RE-USE under the OpenMDW 1.1 license and labels it ready for commercial use. NVIDIA’s earlier checkpoint carried noncommercial terms, so teams migrating between releases should verify that they are using the intended weights and review the current license’s distribution and derivative-work conditions.
huggingface-cli download nvidia/NVIDIA-NemotronLabs-RE-USE \
  --local-dir ./NemotronLabs_REUSE
cd ./NemotronLabs_REUSE
# Place input files in noisy_audio/
sh inference.sh
# Enhanced files appear in enhanced_audio/
The standard script processes files from noisy_audio/ and writes results to enhanced_audio/. For recordings longer than 30 seconds, the repository provides inference_chunk.sh, which divides audio into overlapping segments to control GPU memory use. The training clips were shorter than 10 seconds.
The published inference path targets NVIDIA Ampere GPUs, specifically the A100, and uses Triton acceleration. Parameter count alone does not determine latency or memory use, so CPU, mobile, and real-time performance require separate measurement. NVIDIA also provides a hosted demo for testing individual files.
Where RE-USE fits
- Podcast and video cleanup: The model can reduce room echo, background noise, clipping, and compression damage while keeping a primary speaker prominent.
- Speech-dataset preparation: Researchers can enhance training audio for text-to-speech systems and speech large language models, which process or generate spoken audio.
- ASR preprocessing: The reported character-accuracy gains support using RE-USE as a compact front end for noisy transcription pipelines.
- Shared GPU pipelines: The small checkpoint reduces weight memory when enhancement must run alongside transcription or generation models.
Limits to test before deployment
- Overlapping speakers: The main checkpoint suppresses competing voices. Workflows that need every voice should add source separation or diarization.
- Laughter and vocalizations: NVIDIA provides a sibling checkpoint, NemotronLabs RE-USE-laughter, for applications that need stronger laughter preservation.
- Metric tradeoffs: Sidon scores higher on some UTMOS and DNSMOS perceptual predictors, while RE-USE leads the reported PESQ, STOI, speaker-similarity, and ASR results. Selection should follow the production pipeline’s target metric and listening tests.
- Language coverage: Training includes five named languages. Deployments involving other languages, accents, or specialized vocabulary need representative evaluation data.
- Runtime portability: The release documents an A100-oriented Triton path. The 9.6-million-parameter size does not by itself establish edge, CPU, or real-time performance.

document fetched 2026-10-31 02:55