ResidualMaskingNetwork

KO | EN

gitlite — search

#react #llm #macos #windows #awesome #cli #kubernetes #awesome-list #vue #pytorch #security #ios #openai #open-source

ResidualMaskingNetwork

(★ 513)

ICPR 2020: Facial Expression Recognition using Residual Masking Network

ResidualMaskingNetwork 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Facial Expression Recognition using Residual Masking Network

_(*) State-of-the-art Facial Expression Recognition / Emotion Detection._

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1ZEbJ6rJuGZ6UzmOFE1XCzSgacTUJiN-H?usp=sharing)
[![PyPI Downloads](https://static.pepy.tech/badge/rmn)](https://pepy.tech/projects/rmn)
[![pypi package](https://img.shields.io/badge/version-v3.3.0-blue)](https://pypi.org/project/rmn)
[![circleci](https://circleci.com/gh/phamquiluan/ResidualMaskingNetwork.svg?style=shield)](https://app.circleci.com/pipelines/github/phamquiluan/ResidualMaskingNetwork)
[![Python package](https://github.com/phamquiluan/ResidualMaskingNetwork/actions/workflows/python-package.yml/badge.svg)](https://github.com/phamquiluan/ResidualMaskingNetwork/actions/workflows/python-package.yml)
[![style](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/phamquiluan/residualmaskingnetwork)
[![HF Paper](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Paper-yellow)](https://huggingface.co/papers/2603.05937)
[![HF Model](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-yellow)](https://huggingface.co/phamquiluan/ResidualMaskingNetwork)
[![HF Space](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Demo-orange)](https://huggingface.co/spaces/phamquiluan/ResidualMaskingNetwork)

## Live Demo

Try it in your browser — [**huggingface.co/spaces/phamquiluan/ResidualMaskingNetwork**](https://huggingface.co/spaces/phamquiluan/ResidualMaskingNetwork).
Inference runs entirely client-side through ONNX Runtime Web, so images
never leave your machine. Source in [`space/`](space).

## Installation

1. Install from pip

```bash
pip install rmn
```

2. Or build from source

```bash
git clone git@github.com:phamquiluan/ResidualMaskingNetwork.git
cd ResidualMaskingNetwork
pip install -e .
```

## Quick Start

```python
from rmn import RMN
import cv2

# Initialize the model
m = RMN()

# Detect emotions from an image
image = cv2.imread("your-image.png")
results = m.detect_emotion_for_single_frame(image)
print(results)

# Draw results on the image
image = m.draw(image, results)
cv2.imwrite("output.png", image)
```

## Webcam Demo

```python
from rmn import RMN
m = RMN()
m.video_demo()
```

### Table of Contents

- <a href='#benchmarking_fer2013'>Benchmarking on FER2013</a>
- <a href='#benchmarking_imagenet'>Benchmarking on ImageNet</a>
- <a href='#datasets'>Download datasets</a>
- <a href='#train_fer'>Training on FER2013</a>
- <a href='#train_imagenet'>Training on ImageNet</a>
- <a href='#eval'>Evaluation results</a>
- <a href='#docs'>Download dissertation and slide</a>

## Benchmarking on FER2013

We benchmark our code thoroughly on two datasets: FER2013 and VEMO. Below are the results and trained weights:

| Model                                                                                                     | Accuracy |
| --------------------------------------------------------------------------------------------------------- | -------- |
| [VGG19](https://drive.google.com/file/d/196qnnULQpFK5O5Md_YfvsUC2OQRd5LrQ/view?usp=sharing)               | 70.80    |
| [EfficientNet_b2b](https://drive.google.com/file/d/1ZqvGcqYZXXbMKFwSaoHlo6LaKVYptXxx/view?usp=sharing)    | 70.80    |
| [Googlenet](https://drive.google.com/file/d/1FNiNS7qqkd4dPQX9APiFsEyylb-ZHN8f/view?usp=sharing)           | 71.97    |
| [Resnet34](https://drive.google.com/file/d/1GFSWnjA2tvsVel3kz0EyYW_el7Nyu8Gc/view?usp=sharing)            | 72.42    |
| [Inception_v3](https://drive.google.com/file/d/10LpGJEmSTikidqaiv32rcJYQc3Cx_n6-/view?usp=sharing)        | 72.72    |
| [Bam_Resnet50](https://drive.google.com/file/d/1RlVX2eiaQrUUmpng1PCyhK_hJdfJRcoU/view?usp=sharing)        | 73.14    |
| [Densenet121](https://drive.google.com/file/d/1VWZ96Ar_n_OQD4Rk5ThZbixApMQ5eeYW/view?usp=sharing)         | 73.16    |
| [Resnet152](https://drive.google.com/file/d/1A-kUqrP9u3ZCXVBtE6AdUN5afPwQjTHl/view?usp=sharing)           | 73.22    |
| [Cbam_Resnet50](https://drive.google.com/file/d/1_u_B2NcxqCMPdWmPLMYKVzAenwBlwoFi/view?usp=sharing)       | 73.39    |
| [ResMaskingNet](https://drive.google.com/file/d/1dmOycqZACsLh0eyDSR2ssn4g9xh36zMp/view?usp=sharing)       | 74.14    |
| [**ResMaskingNet + 6**](https://drive.google.com/drive/folders/1Nt7y1T99HpmF93peYxMg-i6BUqdzDBve?usp=sharing) | **76.82**    |

Results in VEMO dataset could be found in my thesis or slide (attached below)

## Benchmarking on ImageNet

We also benchmark our model on ImageNet dataset.

| Model                                                                                        | Top-1 Accuracy | Top-5 Accuracy |
| -------------------------------------------------------------------------------------------- | -------------- | -------------- |
| [Resnet34](https://drive.google.com/open?id=16lErBAk7K3WswKP0wyE9S0dNrr7AF6wd)               | 72.59          | 90.92          |
| [CBAM Resnet34](https://drive.google.com/open?id=16lErBAk7K3WswKP0wyE9S0dNrr7AF6wd)          | 73.77          | 91.72          |
| [**ResidualMaskingNetwork**](https://drive.google.com/open?id=1myjp4_XL8mNJlAbz0TFjYKUc7B0N64eb) | **74.16**          | **91.91**          |

## Datasets

- [FER2013 Dataset](https://drive.google.com/drive/folders/1Nt7y1T99HpmF93peYxMg-i6BUqdzDBve?usp=sharing) (locate it in `saved/data/fer2013` like `saved/data/fer2013/train.csv`)
- [ImageNet 1K Dataset](http://image-net.org/download-images) (ensure it can be loaded by torchvision.datasets.Imagenet)

## Training on FER2013

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1IEQ091jBeJrOKHJe4wNhodH-bUGbLHSE?usp=sharing)

To train the networks, you need to specify the model name and other hyperparameters in the config file (located at configs/\*) then ensure it is loaded in main file, then run training procedure by simply run main file, for example:

```Shell
python main_fer.py  # Example for fer2013_config.json file
```

The best checkpoints will chosen at term of best validation accuracy, located at `saved/checkpoints`. By default, it will train `alexnet` model, you can switch to another model by edit `configs/fer2013_config.json` file (to `resnet18` or `cbam_resnet50` or my network `resmasking_dropout1`.

## Training on the Imagenet dataset

To perform training resnet34 on 4 V100 GPUs on a single machine:

```Shell
python ./main_imagenet.py -a resnet34 --dist-url 'tcp://127.0.0.1:12345' --dist-backend 'nccl' --multiprocessing-distributed --world-size 1 --rank 0
```

## Ensemble method

I used the no-weighted sum average ensemble method to fuse 7 different models together, to reproduce results, you need to do some steps:

1. Download all needed trained weights and locate them on the `./saved/checkpoints/` directory. The link to download can be found in the Benchmarking section.
2. Edit file `gen_results` and run it to generate result offline for **each** model.
3. Run the `gen_ensemble.py` file to generate accuracy for example methods.

## Dissertation and Slide

- [Dissertation PDF (in Vietnamese)](https://drive.google.com/drive/folders/1Nt7y1T99HpmF93peYxMg-i6BUqdzDBve?usp=sharing)
- [Dissertation Overleaf Source](https://www.overleaf.com/read/qdyhnzjmbscd)
- [Presentation slide PDF (in English) with full appendix](https://drive.google.com/drive/folders/1Nt7y1T99HpmF93peYxMg-i6BUqdzDBve?usp=sharing)
- [Presentation slide Overleaf Source](https://www.overleaf.com/read/vxdhjvhvgwdn)
- [ICPR Paper](docs/paper.pdf)
- [Hugging Face Paper Page](https://huggingface.co/papers/2603.05937)
- [Hugging Face Model (trained weights)](https://huggingface.co/phamquiluan/ResidualMaskingNetwork)
- [ICPR Poster Overleaf Source](https://www.overleaf.com/read/jjqwfrsdcdwh#566470)

## Citation

Pham Luan, The Huynh Vu, and Tuan Anh Tran. "Facial Expression Recognition using Residual Masking Network". In: Proc. ICPR. 2020.
```
@inproceedings{pham2021facial,
  title={Facial expression recognition using residual masking network},
  author={Pham, Luan and Vu, The Huynh and Tran, Tuan Anh},
  booktitle={2020 25th International Conference on Pattern Recognition (ICPR)},
  pages={4513--4519},
  year={2021},
  organization={IEEE}
}
```

## Star History

[![Star History Chart](https://api.star-history.com/svg?repos=phamquiluan/residualmaskingnetwork&type=Date)](https://star-history.com/#phamquiluan/residualmaskingnetwork&Date)

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

ResidualMaskingNetwork

ResidualMaskingNetwork 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

ResidualMaskingNetwork

ResidualMaskingNetwork 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README