MiE-X

KO | EN

gitlite — search

#python #typescript #go #machine-learning #deep-learning #nodejs #java #windows #awesome #awesome-list #pytorch #security #mcp #open-source

(★ 16)

Synthesis large scale trainable data for the micro-expression recognition task

MiE-X Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# MiE-X

## Introduction

This is an implementation of MiE-X by Pytorch. MiE-X is a large-scale synthetic dataset that improves data-driven micro-expression methods. MiE-X introduces three types of effective Actions Units (AUs) that constitute trainable micro-expressions. This repository provides the implementation of acquiring these AUs and using these AUs to obtain MiE-X.

![](https://github.com/liuyvchi/MiE-X/blob/master/imgs/three_types_AUs.jpg)

Related material: [Paper](https://arxiv.org/pdf/2112.01730.pdf), [Poster](https://github.com/liuyvchi/MiE-X/releases/download/eccv2022/ECCV2022_MiEX_poster.pdf), [5min Intro](https://github.com/liuyvchi/MiE-X/releases/download/eccv2022/presentation_ECCV2022.mp4).

## Dependencies
MiE-X uses the same libraries as [GANimation](https://github.com/albertpumarola/GANimation)
- python 3.7+
- pytorch 1.6+ & torchvision
- numpy
- matplotlib
- tqdm
- dlib
- face_recognition
- opencv-contrib-python

## Datasets

Baidunetdisk is available. Google drive is coming soon.

|    Variant      | MiE-X (AU_mie)        | MiE-X (AU_mae)  |  MiE-X (AU_exp)        |
|--------------|------------------|------------------|-----------|
| Access     | [Baidu](https://pan.baidu.com/s/1vcOrXyPks-T8PY_UJimGKw?pwd=42i8),[Google](https://drive.google.com/file/d/) | [Baidu](https://pan.baidu.com/s/1kfAz5W2MP1jiVIzfAU9x9g?pwd=81vm),[Google](https://drive.google.com/file/d/) | [Baidu](https://pan.baidu.com/s/1BYnLANq-mYvlqDcg_yQiEg?pwd=v1p7 ) |

## Usage

go to ganimation_replicate

```shell script
cd ganimation_replicate
```

### Extract AUs by the OpenFace toolkit
This work use OpenFace to extract Action Units from real-world expressions. If you would like to extract
AUs by yourself, please follow the official [OpenFace](https://github.com/TadasBaltrusaitis/OpenFace)
repo. We have extracted AUs from MEGC, MMEW, and Oulu and placed them in the `MER/Data` folder.

### Prepare AUs pools for simulation

E.g., prepare the AU pool for the AUs extracted from the MMEW dataset.

```shell script
python prepare_AUMMEW_pool.py
```

### Simulate MiEs

Simulate image based micro-expressions:

```shell script
python simulate.py --mode test --data_root datasets/celebA --gpu_ids 2,3 --ckpt_dir ckpts/emotionNet/ganimation/190327_160828/ --load_epoch 30
```

Preliminary: You need to download your face source dataset and place its path after `--data_root`. The pretrained
GANimation model should be placed in `--ckpt_ddir`. You can train your own GANimation model by following the 
[official](https://github.com/albertpumarola/GANimation) GANimation repo or directly downloading the pretrained model from the 
[third-party](https://github.com/donydchen/ganimation_replicate) implementation.

Simulate video based micro-expressions:

```shell script
python simulate_video_AUexp.py --mode test --data_root datasets/celebA --gpu_ids 2,3 --ckpt_dir ckpts/emotionNet/ganimation/190327_160828/ --load_epoch 30
```

## Train MiE classifers

go to MER

```shell script
cd MER
```
### Train on MiE-X and fine-tune on real-world data
```shell script
bash run_train_fold_fineT.sh
```

If you find this code useful, please kindly cite:

```
@inproceedings{liu2021synthesize,
  title={How to Synthesize a Large-Scale and Trainable Micro-Expression Dataset?},
  author={Liu, Yuchi and Wang, Zhongdao and Gedeon, Tom and Zheng, Liang},
  booktitle={ECCV},
  year={2022}
}
```

MiE-X

MiE-X Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

MiE-X

MiE-X Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README