Audio denoising?

[Jawzper]

I see audio denoising is mentioned in the readme, but it's not clear to me how to actually do this.

Which tool should I be using? I just want to clean up voice clips with background noise so I can use it for cloning.

[0xShug0]

@Jawzper The denoisors rnnoise and zipenhancer are internal libs for model development, and currently are not exposed as runnable models.

[Jawzper]

Oh, ok. It sounds useful, any plans to provide an interface for this in the future?

[0xShug0]

Yes, there is a PR in the review queue (temporarily closed) for this.

[Disonantemus]

I also miss some denoise/enhance/restoration tools that can help to improve ASR from noisy audio recordings (as a previous pass); In the form of another CLI tool (as the PR) or maybe in another modes, like: s2s.

To get rid of noise, reverberation, improve voice, etc.

---

I think this is the PR: #367

The three models there are:

• deepfilternet2
• rnnoise
• zipenhancer

Audio.cpp (available from CLI)

There is AudioSR using s2s mode, but the model is so big (6GB) that I haven't used (too large 4 casual use), I think only upscale to 44/48kHz (no idea) without denoise (maybe I'm wrong).

Audio.cpp (not available from CLI)

In Audio Tools there is GTCRN that should do some denoise.

DeepFilterNet (CLI)

Abandoned 3 years ago, but still working.

• Rikorose/DeepFilterNet

FFmpeg

rnnoise is available to use in ffmpeg as an audio filter, example:

ffmpeg -i in.wav -af 'arnndn=model=./arnndn-models/std.rnnn:mix=0.8' out.wav

ZipEnhancer-rs

zipenhancer is available from this CLI.

More

CrispASR

Audio upscaling and restoration:

• Sidon
• VoxCPM2-vae

sukoon

URL: https://github.com/ruwadgroup/sukoon

has 3 denoise modes (fast and small):

• DeepFilterNet 3
• MDX-Net Kim Vocal 2
• MDX-Net UVR 9482

Example:

sukoon clean --engine hq in.wav

More

• https://github.com/Fovty/HushMic
• https://github.com/Allenmylath/Hush-vani
• https://github.com/DakeQQ/Audio-Denoiser-ONNX

[viebrix]

SAM-Audio from meta can do this and a lot more. But haven't had the time to test it locally. First try didn't work...
https://ai.meta.com/research/samaudio/
You need to register at meta to download it at huggingface:
https://huggingface.co/collections/facebook/sam-audio
On huggingface are also quant.-versions...

[0xShug0]

@Jawzper @Disonantemus @viebrix The built-in utilities are now exposed as a special model. https://github.com/0xShug0/audio.cpp/blob/main/docs/audio_tools.md#built-in-audio-utilities