ScaMo_code - GitLite

KO | EN

gitlite — search

#javascript #hacktoberfest #typescript #llm #go #machine-learning #rust #docker #macos #nodejs #pytorch #security #ios #open-source

ScaMo_code

(★ 95)

No description available.

파일 탐색기

최종 버전 다운로드 (.zip)

- Arxiv_ScaMo.pdf
- teaser.png
- - __init__.py
  - adapter.py
  - indexed_dataset.py
  - lora.py
  - model.py
  - model_hf.py
  - quantization.py
  - tokenizer.py
  - utils.py
- encdec.py
- evaluator_wrapper.py
- facal_loss.py
- FSQ.py
- LFQ.py
- modules.py
- pos_encoding.py
- quantize_cnn.py
- resnet.py
- rotation2xyz.py
- smpl.py
- t2m_trans.py
- unigram_loss.py
- vqvae.py
- get_eval_option.py
- option_transformer.py
- option_vq.py
- download_model.sh
- download_t5.sh
- config.py
- losses.py
- motion_process.py
- paramUtil.py
- quaternion.py
- rotation_conversions.py
- skeleton.py
- utils_model.py
- word_vectorizer.py
- - .DS_Store
  - gBR_sBM_cAll_d04_mBR0_ch01.pkl
  - template_correct_head_toe.bvh
- - bvh.py
  - quat.py
- __init__.py
- Animation.py
- AnimationStructure.py
- BVH.py
- BVH_mod.py
- InverseKinematics.py
- joints2bvh.py
- plot_3d_global.py
- Quaternions.py
- remove_fs.py
.gitignore
environment.yml
inference_generation_hf.py
README.md

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# ScaMo
Pytorch implementation of paper "ScaMo: Exploring the Scaling Law in Autoregressive Motion Generation Model"

[[Project Page]](https://shunlinlu.github.io/ScaMo/) [[Paper]](https://arxiv.org/pdf/2412.14559)

<p align="center">
<img src="assets/teaser.png" width="600px" alt="teaser">
</p>

## Release Log
* 2024/12/06 Initialize the project and release the inference code.

## Table of Content
* [1. Installation](#1-installation)
* [2. Quick Start](#2-quick-start)
* [3. Transform the output vector to bvh](#3-transform-the-output-vector-to-bvh)
* [4. Contribute](#4-contribute)
* [5. Acknowledgement](#5-acknowledgement)

## 1. Installation

### 1.1. Environment

```bash
conda env create -f environment.yml
conda activate ScaMo
```

The code was tested on Python 3.8 and PyTorch 2.0.0.

### 2.2. Dependencies

```bash
bash prepare/download_model.sh
```
Download google flan-t5-xl model from huggingface.
```bash
bash prepare/download_t5.sh
```

## 3. Quick Start

A quick start guide of how to inference ScaMo.
```python
python inference_generation_hf.py --nb-code 65536 --quantizer FSQ --pretrained_llama 3B --text_encode flan-t5-xl
```

## 4. Transform the output vector to bvh

You should change the folder path correctly. Here is an example:

```bash
python3 visualization/joints2bvh.py
```
Note the process may lead to wrong bone rotation, due to the ik. We are training a new model with a rotation-based representation to solve this problem.

## 5. Contribute
We try to scale the dataset and model. However, we still do not observe the emerging abilities and have a long way to go. We believe our model can "eat" more data. If you have any open-sourced data, feel free to contact me. I can contribute to convert the motion data to same format in this paper to train a better model.

### 6. Acknowledgement
We appreciate helps from :

* public code like [t2m-gpt](https://github.com/Mael-zys/T2M-GPT), [momask](https://github.com/EricGuo5513/momask-codes), [text-to-motion](https://github.com/EricGuo5513/text-to-motion), [TM2T](https://github.com/EricGuo5513/TM2T), [MDM](https://github.com/GuyTevet/motion-diffusion-model), [MotionDiffuse](https://github.com/mingyuan-zhang/MotionDiffuse) etc.