imbalanced-DL

KO | EN

gitlite — search

#typescript #ai-agents #ai #dsh-plugin #deepseek-harness #open-source #claude-code #codex #cli #developer-tools #react #windows

★ 58 Open GitHub ↗

A Python Package for Deep Imbalanced Learning

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# imbalanced-DL: Deep Imbalanced Learning in Python

## Overview
`imbalanced-DL` (imported as imbalanceddl) is a Python package designed to make deep imbalanced learning easier for researchers and real-world users. From our experiences, we observe that to tackcle deep imbalanced learning, there is a need for a strategy. That is, we may not just address this problem with one single model or approach. Thus in this package, we seek to provide several **strategies for deep imbalanced learning**. The package not only implements several popular deep imbalanced learning strategies, but also provides benchmark results on several image classification tasks. Futhermore, this package provides an interface for implementing more datasets and strategies.

## Strategy
We provide some baseline strategies as well as some state-of-the-are strategies in this package as the following:
* Empirical Risk Minimization (baseline strategy)
* [Reweighting with Class Balance (CB) Loss](https://arxiv.org/pdf/1901.05555.pdf)
* [Deferred Re-Weighting (DRW)](https://arxiv.org/pdf/1906.07413.pdf)
* [M2m: Major-to-minor translation](https://arxiv.org/pdf/2004.00431.pdf)
* [Label Distribution Aware Margin (LDAM) Loss with DRW](https://arxiv.org/pdf/1906.07413.pdf)
* [DeepSMOTE: Fusing Deep Learning and SMOTE for Imbalanced Data](https://arxiv.org/pdf/2105.02340.pdf)
* [Mixup with DRW](https://arxiv.org/pdf/1710.09412.pdf)
* [Remix with DRW](https://arxiv.org/pdf/2007.03943.pdf)
* [MAMix with DRW](https://www.csie.ntu.edu.tw/~htlin/paper/doc/taai23mamix.pdf) (Ours)

## Environments
* This package is tested on Linux OS.
* You are suggested to use a different virtual environment so as to avoid package dependency issue.
* For Pyenv & Virtualenv users, you can follow the below steps to create a new virtual environment or you can also skip this step.
##### Pyenv & Virtualenv (Optinal)
* For dependency isolation, it's better to create another virtual environment for usage.
* The following will be the demo for creating and managing virtual environment.
* Install `pyenv` & `virtualenv` first.
* `pyenv virtualenv [version] [virtualenv_name]`
    *  For example, if you'd like to use python 3.6.8, you can do: `pyenv virtualenv 3.6.8 TestEnv`
* `mkdir [dir_name]`
* `cd [dir_name]`
* `pyenv local [virtualenv_name]`
* Then, you will have a new (clean) python virtual environment for the package installation.

## Installation 
### Basic Requirement
* Python >= 3.6

```
git clone https://github.com/ntucllab/imbalanced-DL.git
cd imbalanceddl
python -m pip install -r requirements.txt
python setup.py install
```
### Note
* If you would like to further develop your own algorithm within this package, or modifying existing files, please remember to build again locally with `python setup.py install` so that the change will be updated in the package.
* Please also note that by default, `CIFAR10`, `CIFAR100`, and `SVHN` are defaulted available when you are using PyTorch, you can experiment with these three basic datasets first, and if you would like to further test your algorithm on larger images, you can follow the instruction in the `/example/README.md` for further downloading information for `CINIC10` and `Tiny-ImageNet`.

## Usage
We highlight three key features of `imbalanced-DL` as the following:

(0) Imbalanced Dataset:
* We support 5 benchmark image datasets for deep imbalanced learing.
* To create and ImbalancedDataset object, you will need to provide a `config_file` as well as the dataset name you would like to use.
* Specifically, inside the `config_file`, you will need to specify three key parameters for creating imbalanced dataset.
    * `imb_type`: you can choose from `exp` (long-tailed imbalance) or `step` imbalanced type.
    * `imb_ratio`: you can specify the imbalanceness of your data, typically researchers choose `0.1` or `0.01`.
    * `dataset_name`: you can specify 5 benchmark image datasets we provide, or you can implement your own dataset.
    * For an example of the `config_file`, you can see [example/config](https://github.com/ntucllab/imbalanced-DL/blob/main/example/config/config_cifar10.yaml).
* To contruct your own dataset, you should inherit from [`BaseDataset`](https://github.com/ntucllab/imbalanced-DL/blob/e63acaab958bf206edad9418e9c30352e9566356/imbalanceddl/dataset/dataset_base.py#L4), and you can follow [`torchvision.datasets.ImageFolder`](https://pytorch.org/vision/stable/_modules/torchvision/datasets/folder.html#ImageFolder) to construct your dataset in PyTorch format.

```python
from imbalanceddl.dataset.imbalance_dataset import ImbalancedDataset

# specify the dataset name
imbalance_dataset = ImbalancedDataset(config, dataset_name=config.dataset)

```

(1) Strategy Trainer:
* We support 6 different strategies for deep imbalance learning, and you can either choose to train from scratch, or evaluate with the best model after training. To evaluate with the best model, you can get more in-depth metrics such as per class accuracy for further evaluation on the performance of the selected strategy. We provide one trained model in [example/checkpoint_cifar10](https://github.com/ntucllab/imbalanced-DL/tree/main/example/checkpoint_cifar10).
* For each strategy trainer, it is associated with a `config_file`, `ImbalancedDataset object`, `model`, and `strategy_name`.
* Specifically, the `config_file` will provide some training parameters, where the default settings for reproducing benchmark result can be found in [example/config](https://github.com/ntucllab/imbalanced-DL/tree/main/example/config). You can also set these training parameters based on your own need.
* For `model`, we currently provide `resnet32` and `resnet18` for reproducing the benchmark results.
* We provide a `build_trainer()` function to return the specified trainer as the following.

```python
from imbalanceddl.strategy.build_trainer import build_trainer

# specify the strategy
trainer = build_trainer(config,
                        imbalance_dataset,
                        model=model,
                        strategy=config.strategy)
# train from scratch
trainer.do_train_val()

# Evaluate with best model
trainer.eval_best_model()

```
* Or you can also just select the specific strategy you would like to use as:

```python
from imbalanceddl.strategy import LDAMDRWTrainer

# pick the trainer
trainer = LDAMDRWTrainer(config,
                         imbalance_dataset,
                         model=model,
                         strategy=config.strategy)

# train from scratch
trainer.do_train_val()

# Evaluate with best model
trainer.eval_best_model()

```
* To construct your own strategy trainer, you need to inherit from [`Trainer`](https://github.com/ntucllab/imbalanced-DL/blob/e63acaab958bf206edad9418e9c30352e9566356/imbalanceddl/strategy/trainer.py#L11) class, where in your own strategy you will have to implement `get_criterion()` and `train_one_epoch()` method. After this you can choose whether to add your strategy to `build_trainer()` function or you can just use it as the above demonstration.

(2) Benchmark research environment:
* To conduct deep imbalanced learning research, we provide example codes for training with different strategies, and provide benchmark results on five image datasets. To quickly start training CIFAR-10 with ERM strategy, you can do:

```
cd example
python main.py --gpu 0 --seed 1126 --c config/config_cifar10.yaml --strategy ERM

```
* Following the example code, you can not only get results from baseline training as well as state-of-the-art performance such as [LDAM](https://arxiv.org/pdf/1906.07413.pdf) or [Remix](https://arxiv.org/pdf/2007.03943.pdf), but also use this environment to develop your own algorithm / strategy. Feel free to add your own strategy into this package.
* For more information about example and usage, please see the [Example README](https://github.com/ntucllab/imbalanced-DL/tree/main/example)

## Benchmark Results
We provide benchmark results on 5 image datasets, including **CIFAR-10**, **CIFAR-100**, **CINIC-10**, **SVHN**, and **Tiny-ImageNet**. We follow standard procedure to generate imbalanced training dataset for these 5 datasets, and provide their top 1 validation accuracy results for research benchmark. For example, below you can see the result table of **Long-tailed Imbalanced CIFAR-10** trained on different strategies. For more detailed benchmark results, please see [example/README.md](https://github.com/ntucllab/imbalanced-DL/tree/main/example).

* `Long-tailed Imbalanced CIFAR-10`

| `imb_type` |`imb_factor`|   Model   | Strategy | Validation Top 1 |
|:----------:|:----------:|:---------:|:--------:|:----------------:|
|long-tailed | 100        | ResNet32  | ERM      | 71.23            |
|long-tailed | 100        | ResNet32  | DRW      | 75.08            |
|long-tailed | 100        | ResNet32  | M2m      | 76.15            |
|long-tailed | 100        | ResNet32  | DeepSMOTE| 76.66            |
|long-tailed | 100        | ResNet32  | LDAM-DRW | 77.75            |
|long-tailed | 100        | ResNet32  | Mixup-DRW| 82.11            |
|long-tailed | 100        | ResNet32  | Remix-DRW| 81.82            |
|long-tailed | 100        | ResNet32  | MAMix-DRW| 82.29            |

## Test
* `python -m unittest -v`

## Contact
If you have any question, please don't hesitate to email `wccheng3011@gmail.com` or `maitanhaksdtvt6@gmail.com`. Thanks !

## Citing

If you find this package useful, please cite both the original works associated with each strategy and the following:
```
@InProceedings{MaMixup2024,
author="Cheng, Wei-Chao
and Mai, Tan-Ha
and Lin, Hsuan-Tien",
title="From SMOTE to Mixup for Deep Imbalanced Classification",
booktitle="Technologies and Applications of Artificial Intelligence",
year="2024",
publisher="Springer Nature Singapore",
address="Singapore",
pages="75--96",
isbn="978-981-97-1711-8"
}
```

## Acknowledgement
The authors thank members of the Computational Learning Lab at National Taiwan University for valuable discussions and various contributions to making this package better.

## Notice for Tiny200 dataset.
You might face the issue with validation dataset of Tiny200 when you reproduce on this dataset, because the structure of validate dataset is not well-organized yet. Therefore, you need to preprocess the validation dataset of it by running the `make.py` file to reorganize it before training. You need to copy the `make.py` to `tiny/val/` folder and then just run this file.

imbalanced-DL

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README