VioPose

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

(★ 9)

No description available.

VioPose Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# VioPose 
This repository provides code and pretrained weights for the paper. The code is provided without any warranty. If you benefited from the work, or partially or directly use our code, please cite our work as shown below. For more information please visit our [project website](https://sj-yoo.info/viopose/)

@inproceedings{yooshrestha2024viopose,
		title     = {{VioPose}: Violin Performance 4D Pose Estimation by Hierarchical Audiovisual Inference},
		author    = {Yoo, Seong Jong and Shrestha, Snehesh and Muresanu, Irina and Fermüller, Cornelia},
		journal   = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
		year      = {2025},
	}

## Installation 
### Installation using Conda
```
conda create -n viopose python==3.9
conda activate viopose
conda install pip
pip install -r requirements.txt
./download.sh
```

- Install [tensorflow](https://www.tensorflow.org/install/pip) $\approx$ 2.13  and cuda accordingly
- Download dataset (will release soon) and locate it at `data/FullData`

## Run Demo
### VioDat Test Demo (currently unavailable)
1. Run `test.py` script. This code will generate and save *result.npz* at `./Logs/VioPose`
```
python3 test.py --folder Logs/VioPose --data violin --data_path data/FullData/mmViolin_v1.0.npz
```
2.  Run `generate_video.py` script

### In-the-wild Demo
1. Prepare video (mp4 format) and audio (wav format). For demo we prepared video at `./data/demo` 
2. Run `inference.py` script
```
python3 inference.py --folder Logs/VioPose --video_path data/demo/demo.mp4 --audio_path data/demo/demo.wav --vis True
```
- If you don't provide audio file then *VioPose_wo_audio* model will be used for inference
- The output video is saved at `./output/output_w_audio.mp4` with audio and `./output/output.mp4` without audio

## Train from Scratch
1. Prepare [*VioDat Dataset*](https://www.snehesh.com/viodat/) at `./data/FullData`
2. Run `main.py` script
```
python3 main.py --cfg ./config/viopose.yaml > ./out_log/viopose.out
python3 test.py --folder /Logs/viopose --data violin --data_path data/FullData/mmViolin_v1.0.npz
```

## Bug Report
Please raise an issue on Github for issues related to this code. If you have any questions related about the code feel free to send an email to here (yoosj@umd.edu).

## License
VioPose and [VioDat](https://www.snehesh.com/viodat/) is available for non-commercial and research use only and may not be redistributed and should follow requirements under the conditions detailed on the [license page](#). For commercial licensing or if you have any questions, please get in touch with us at yoosj@umd.edu.

VioPose

VioPose Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

VioPose

VioPose Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README