Audio-FLAN - GitLite

KO | EN

gitlite — search

#llm #go #android #machine-learning #deep-learning #docker #awesome #cli #vue #pytorch #security #mcp #ios #openai

Audio-FLAN

(★ 163)

Audio-FLAN

파일 탐색기

최종 버전 다운로드 (.zip)

- - compute_acc.py
  - compute_caption.py
  - compute_clap.py
  - compute_cossim.py
  - compute_der.py
  - compute_eer.py
  - compute_emosim.py
  - compute_f0rmse.py
  - compute_fad.py
  - compute_mae.py
  - compute_mcd.py
  - compute_mse.py
  - compute_pesq.py
  - compute_sdr.py
  - compute_snr.py
  - compute_ssmmse.py
  - compute_stoi.py
  - compute_tor.py
  - compute_wer.py
- - f0.py
  - hparam.py
  - util.py
- __init__.py
- calc_metrics.py
- readme
- requirements.txt
- run.sh
README.md

# 설치 가이드

1. 코드 내려받기

git clone https://github.com/lmxue/Audio-FLAN

깃허브에서 프로젝트 코드 전체를 내 컴퓨터로 내려받습니다.

cd Audio-FLAN

방금 내려받은 프로젝트 폴더 안으로 이동합니다.

2. Python

쉬움 추천

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Python 3 설치 시 'Add Python to PATH' 옵션을 꼭 체크하세요 (Windows).

pip install -r evaluation/requirements.txt

requirements.txt 등에 명시된 파이썬 라이브러리를 설치합니다.

jupyter notebook

브라우저에서 노트북(.ipynb) 파일들을 열람하고 실행할 수 있는 Jupyter 화면을 켭니다.

✅ 에러 메시지 없이 실행되고 터미널에 안내 문구가 출력되면 정상입니다.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# 🚀Audio-FLAN
**Audio-FLAN** aims to **unify audio-language models** that can seamlessly handle both **understanding** and **generation** tasks across **speech, music, sound**.

## 🔥 Audio-FLAN: A Preliminary Release
<div>
    <a href="https://huggingface.co/datasets/HKUSTAudio/Audio-FLAN-Dataset"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20Huaging_Face-HKUST_Audio-yellow"></a>
    <a href="https://arxiv.org/pdf/2502.16584"><img src="https://img.shields.io/badge/arxiv-Audio_FLAN-green">
</a>
</div>
<br>

**An Instruction-Tuning Dataset for Unified Audio Understanding and Generation Across Speech, Music, and Sound**

Recent advancements in audio tokenization have significantly enhanced the integration of audio capabilities into large language models (LLMs). However, audio understanding and generation are often treated as distinct tasks, hindering the development of truly **unified** audio-language models. While instruction tuning has demonstrated remarkable success in improving generalization and zero-shot learning across text and vision, its application to audio remains largely unexplored. A major obstacle is the lack of comprehensive datasets that unify audio understanding and generation. To address this, we introduce Audio-FLAN, a large-scale instruction-tuning dataset covering **80 diverse tasks** across **speech, music, and sound** domains, with **over 100 million instances**. **Audio-FLAN** lays the foundation for unified audio-language models that can seamlessly handle both **understanding** (e.g., transcription, comprehension) and **generation** (e.g., speech, music, sound) tasks across a wide range of audio domains in a zero-shot manner. The **Audio-FLAN** dataset is available on [HuggingFace](https://huggingface.co/datasets/HKUSTAudio/Audio-FLAN-Dataset) and [GitHub](https://github.com/lmxue/Audio-FLAN) and will be continuously updated.

![image](https://github.com/user-attachments/assets/39cb6891-1334-42d3-8d4e-8671a478d37d)

# 📌 News  
* A subset of the dataset is now available on 🤗 [HuggingFace](https://huggingface.co/datasets/HKUSTAudio/Audio-FLAN-Dataset). The full dataset will be released progressively.

# ⚠️ Disclaimer 
The Audio-FLAN dataset is released under the same license as the original datasets used in its creation. Please refer to the respective licenses of the original datasets for usage and redistribution terms. We do not claim ownership of the original data and encourage users to comply with the licensing terms of the source materials.

# 📚 Citation 
```
@article{xue2025audio,
  title={Audio-FLAN: A Preliminary Release},
  author={Liumeng Xue, Ziya Zhou, Jiahao Pan, Zixuan Li, Shuai Fan, Yinghao Ma, Sitong Cheng, Dongchao Yang, Haohan Guo, Yujia Xiao, Xinsheng Wang, Zixuan Shen, Chuanbo Zhu, Xinshen Zhang, Tianchi Liu, Ruibin Yuan, Zeyue Tian, Haohe Liu, Emmanouil Benetos, Ge Zhang, Yike Guo, Wei Xue},
  journal={arXiv preprint arXiv:2502.16584},
  year={2025}
}
```