MDLT

KO | EN

gitlite — search

#hacktoberfest #ai #typescript #llm #golang #android #machine-learning #rust #deep-learning #linux #macos #kubernetes #pytorch #mcp

(★ 9)

Official Implementation of the Transformer and Mamba models from the Music to Dance as Language Translation using Sequence Models paper. Translate music pieces to dance poses.

MDLT 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

> **Music to Dance as Language Translation using Sequence Models**\
> André Correia, Luís A. Alexandre\
> Paper: https://arxiv.org/abs/2403.15569

## Video Preview

[Video Demonstration](https://drive.google.com/file/d/1kejVe1j_n0XKT9BaeYoBFHtKxNyfR_8Y/view?usp=drive_link)

## Pre-Processing

If you want to extract audio and pose features from a data set, you can find example code for AIST++ and PhantomDance data sets in the processing directory.

Otherwise, you can download the processed AIST++ and PhantomDance data bellow:

[Google Drive link](https://drive.google.com/file/d/1YsrAkAkTjTsEswvtT9DDlWI4YipEJjZf/view?usp=drive_link)

# Install Anaconda
```
sudo apt-get install libgl1-mesa-glx libegl1-mesa libxrandr2 libxrandr2 libxss1 libxcursor1 libxcomposite1 libasound2 libxi6 libxtst6

curl -O https://repo.anaconda.com/archive/Anaconda3-2024.02-1-Linux-x86_64.sh

bash Anaconda3-2024.02-1-Linux-x86_64.sh

conda init
```

# Create and activate environment
```
conda create --name MDLT

conda activate MDLT
```

# Install Dependencies
```
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

pip3 install numpy

pip3 install transformers

pip install causal-conv1d>=1.2.0

pip install mamba-ssm
```

# Train

Train Transformer on AIST++

```
python3 main_transformer_aist.py --genre 'all' --infer_every 1000 --K 20 --n_epochs 50000 --infer_every 5000 --n_layer 6 --d_model 128 --val_index 0
```

Train Transformer on AIST++ "mLH" genre

```
python3 main_transformer_aist.py --genre 'mLH' --infer_every 1000 --K 20 --n_epochs 50000 --infer_every 5000 --n_layer 6 --d_model 128 --val_index 0
```

Train Transformer on PhantomDance

```
python3 main_transformer_phantom.py  --infer_every 1000 --K 20 --n_epochs 50000 --infer_every 5000 --n_layer 6 --d_model 128 --val_type 0
```

## Citation

If you use this codebase, or otherwise found our work valuable, please cite MDLT:
```
@misc{correia2024music,
      title={Music to Dance as Language Translation using Sequence Models}, 
      author={André Correia and Luís A. Alexandre},
      year={2024},
      eprint={2403.15569},
      archivePrefix={arXiv},
      primaryClass={cs.SD}
}
```

MDLT

MDLT 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

MDLT

MDLT 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README