openvino-plugins-ai-audacity - GitLite

KO | EN

gitlite — search

#python #hacktoberfest #ai #golang #android #rust #deep-learning #linux #awesome #cli #kubernetes #mcp #ios #open-source

openvino-plugins-ai-audacity

(★ 2,060)

A set of AI-enabled effects, generators, and analyzers for Audacity®.

File Explorer

Download Latest Version (.zip)

- - audio_sr.cpp
  - audio_sr.h
  - audiosr_common.h
  - autoencoder.cpp
  - autoencoder.h
  - ddim_sampler.cpp
  - ddim_sampler.h
  - ddpm_latent_diffusion.cpp
  - ddpm_latent_diffusion.h
  - distributions.h
- - ittutils.h
- - CMakeLists.txt
  - encodec_encoder.h
  - music_gen_decoder_cl.cpp
  - music_gen_decoder_cl.h
  - music_gen_decoder_full_batch1.h
  - music_gen_decoder_full_batch2.h
  - musicgen.cpp
  - musicgen.h
  - musicgen_config.h
  - musicgen_decoder_model.h
  - musicgen_for_causal_lm.cpp
  - musicgen_for_causal_lm.h
  - musicgen_for_conditional_generation.cpp
  - musicgen_for_conditional_generation.h
  - musicgen_model_static.cpp
  - musicgen_model_static.h
  - musicgen_utils.h
- - noise_suppression_df_model.cpp
  - noise_suppression_df_model.h
  - noise_suppression_model.h
  - noise_suppression_omz_model.cpp
  - noise_suppression_omz_model.h
- CMakeLists.txt
- htdemucs.cpp
- htdemucs.h
- OpenVINO.cpp
- OVAudioSR.cpp
- OVAudioSR.h
- OVMusicGenerationLLM.cpp
- OVMusicGenerationLLM.h
- OVMusicSeparation.cpp
- OVMusicSeparation.h
- OVNoiseSuppression.cpp
- OVNoiseSuppression.h
- OVStringUtils.h
- OVWhisperTranscription.cpp
- OVWhisperTranscription.h
- - add_ov_module.patch
  - audacity_ai_plugins.iss
  - audacity_no_vc_runtime_install.patch
  - build.bat
  - fetch_build_package.bat
  - LICENSE_text.txt
  - module_preferences.bmp
  - package.bat
  - prereq.bat
  - README.md
  - set_env.bat
.gitignore
CODE_OF_CONDUCT.md
CONTRIBUTING.md
LICENSE.txt
README.md
security.md

# Installation Guide

1. Get the code

git clone https://github.com/intel/openvino-plugins-ai-audacity

Downloads the entire project code from GitHub to your computer.

cd openvino-plugins-ai-audacity

Moves into the project folder you just downloaded.

2. CMake

Medium Recommended

Prerequisites

Git Needed to download the project code from GitHub.
CMake The tool used to generate build configuration.
C/C++ 컴파일러 Windows needs Visual Studio (Community edition, free), macOS needs Xcode Command Line Tools, Linux needs the gcc/g++ package.

cd mod-openvino

This project's files live in a subfolder, so move into it first.

mkdir build && cd build

Creates a folder to hold the build output and moves into it.

cmake ..

Analyzes the source code and generates build configuration files (must be run inside the build folder).

make

Compiles the code based on the generated build configuration to produce an executable.

✅ Check that an executable was created inside the build folder, then run it directly (e.g. ./build/app_name).

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# OpenVINO™ AI Plugins for Audacity* :metal:
![updated_audacity_top_level](https://github.com/user-attachments/assets/f7c911db-fb98-43ff-972d-f66473b4c921)
[![License: GPL v3](https://img.shields.io/badge/License-GPL%20v3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0) [![Mentioned in Awesome OpenVINO](https://awesome.re/mentioned-badge-flat.svg)](https://github.com/openvinotoolkit/awesome-openvino)

A set of AI-enabled effects, generators, and analyzers for [Audacity®](https://www.audacityteam.org/). These AI features run 100% locally on your PC :computer: -- no internet connection necessary! [OpenVINO™](https://github.com/openvinotoolkit/openvino) is used to run AI models on supported accelerators found on the user's system such as CPU, GPU, and NPU.

- [**Music Separation**](doc/feature_doc/music_separation/README.md):musical_note: -- Separate a mono or stereo track into individual stems -- Drums, Bass, Vocals, & Other Instruments. 
- [**Noise Suppression**](doc/feature_doc/noise_suppression/README.md):broom: -- Removes background noise from an audio sample.
- [**Music Generation & Continuation**](doc/feature_doc/music_generation/README.md):notes: -- Uses MusicGen LLM to generate snippets of music, or to generate a continuation of an existing snippet of music.
- [**Whisper Transcription**](doc/feature_doc/whisper_transcription/README.md):microphone: -- Uses [whisper.cpp](https://github.com/ggerganov/whisper.cpp) to generate a label track containing the transcription or translation for a given selection of spoken audio or vocals.
- [**Super Resolution**](doc/feature_doc/super_resolution/README.md):sparkles: -- Upscales and enriches audio for improved clarity and detail.

## Installation :floppy_disk: 
  Go [here](https://github.com/intel/openvino-plugins-ai-audacity/releases) to find installation packages & instructions for the latest Windows release.

## Build Instructions :hammer:
  - [Windows Build Instructions](doc/build_doc/windows/README.md)  
  - [Linux Build Instructions](doc/build_doc/linux/README.md)

## Help, Feedback, & Bug Reports 🙋‍♂️
  We welcome you to submit an issue [here](https://github.com/intel/openvino-plugins-ai-audacity/issues) for
  * Questions
  * Bug Reports
  * Feature Requests
  * Feedback of any kind -- how can we improve this project?
    
## Contribution :handshake:
  Your contributions are welcome and valued, no matter how big or small. Feel free to submit a pull-request!

## Acknowledgements & Citations :pray:
* Audacity® development team & Muse Group-- Thank you for your support!  
* Audacity® GitHub -- https://github.com/audacity/audacity  
* Whisper transcription & translation analyzer uses whisper.cpp (with OpenVINO™ backend): https://github.com/ggerganov/whisper.cpp  
* Music Generation & Continuation use MusicGen model, from Meta.
  - We currently have support for [MusicGen-Small](https://huggingface.co/facebook/musicgen-small), and [MusicGen-Small-Stereo](https://huggingface.co/facebook/musicgen-stereo-small)
  - The txt-to-music pipelines were ported from python to C++, referencing logic from the Hugging Face transformers project: https://github.com/huggingface/transformers 
* Music Separation effect uses Meta's Demucs v4 model (https://github.com/facebookresearch/demucs), which has been ported to work with OpenVINO™
* Noise Suppression:
  * noise-suppression-denseunet-ll:  from OpenVINO™'s Open Model Zoo: https://github.com/openvinotoolkit/open_model_zoo   
  * DeepFilterNet2 & DeepFilterNet3:
    * Ported the models & pipeline from here: https://github.com/Rikorose/DeepFilterNet  
    * We also made use of @grazder's fork / branch (https://github.com/grazder/DeepFilterNet/tree/torchDF-changes) to better understand the Rust implementation, and so we also based some of our C++ implementation on ```torch_df_offline.py``` found here.  
    * Citations:
      ```bibtex
      @inproceedings{schroeter2022deepfilternet2,
      title = {{DeepFilterNet2}: Towards Real-Time Speech Enhancement on Embedded Devices for Full-Band Audio},
      author = {Schröter, Hendrik and Escalante-B., Alberto N. and Rosenkranz, Tobias and Maier, Andreas},
      booktitle={17th International Workshop on Acoustic Signal Enhancement (IWAENC 2022)},
      year = {2022},
      }
        
      @inproceedings{schroeter2023deepfilternet3,
      title = {{DeepFilterNet}: Perceptually Motivated Real-Time Speech Enhancement},
      author = {Schröter, Hendrik and Rosenkranz, Tobias and Escalante-B., Alberto N. and Maier, Andreas},
      booktitle={INTERSPEECH},
      year = {2023},
      }
      ```
* Super Resolution feature is a port (python -> C++, pytorch -> OpenVINO IR) of this project: https://github.com/haoheliu/versatile_audio_super_resolution
  * Citations:
    ```bibtex
    @article{liu2023audiosr,
    title={{AudioSR}: Versatile Audio Super-resolution at Scale},
    author={Liu, Haohe and Chen, Ke and Tian, Qiao and Wang, Wenwu and Plumbley, Mark D},
    journal={arXiv preprint arXiv:2309.07314},
    year={2023}
    }
    ```
* [OpenVINO™ Notebooks](https://github.com/openvinotoolkit/openvino_notebooks) -- We have learned a lot from this awesome set of python notebooks, and are still using it to learn latest / best practices for implementing AI pipelines using OpenVINO™!