FMix - GitLite

KO | EN

gitlite — search

#javascript #ai #llm #go #golang #rust #docker #nodejs #windows #awesome #cli #awesome-list #openai #open-source

FMix

(★ 339)

Official implementation of 'FMix: Enhancing Mixed Sample Data Augmentation'

파일 탐색기

최종 버전 다운로드 (.zip)

- adversarial_linf.py
- mine.py
- README.md
- train_vaes.py
- vae.py
- vae_mi.py
- vgg.py
- - google_commands.py
  - README.md
  - sft_transforms.py
  - transforms.py
- __init__.py
- bengali.py
- datasets.py
- fashion.py
- imagenet_a.py
- imagenet_hdf5.py
- tiny_imagenet.py
- toxic.py
- toxic_bert.py
- bengali_experiment.sh
- cifar_experiment.sh
- fashion_experiments.sh
- google_commands_experiment.sh
- imagenet_experiment.sh
- modelnet_experiment.sh
- tiny_imagenet_experiment.sh
- toxic_experiment.sh
- lightning.py
- tensorflow_implementation.py
- test_lightning.py
- test_tensorflow.py
- test_torchbearer.py
- torchbearer_implementation.py
- __init__.py
- bert.py
- densenet3.py
- models.py
- pyramid.py
- resnet.py
- senet.py
- toxic_cnn.py
- toxic_lstm.py
- wide_resnet.py
- example_masks.ipynb
- grad_cam.ipynb
- - __init__.py
  - auto_augment.py
  - auto_augment_aug_list.py
- __init__.py
- bengali_evaluate.py
- convert_imagenet_model.py
- cross_val.py
- imagenet_to_hdf5.py
- lr_warmup.py
- macro_recall.py
- msda_alternator.py
- process.py
- reduced_dataset_splitter.py
- reformulated_mixup.py
.gitignore
fmix.py
fmix_3d.gif
fmix_example.png
hubconf.py
LICENSE
README.md
requirements.txt
trainer.py

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<div align="center">
  
# FMix

This repository contains the __official__ implementation of the paper ['FMix: Enhancing Mixed Sampled Data Augmentation'](https://arxiv.org/abs/2002.12047)

[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/understanding-and-enhancing-mixed-sample-data/image-classification-on-cifar-10)](https://paperswithcode.com/sota/image-classification-on-cifar-10?p=understanding-and-enhancing-mixed-sample-data)
[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/understanding-and-enhancing-mixed-sample-data/image-classification-on-fashion-mnist)](https://paperswithcode.com/sota/image-classification-on-fashion-mnist?p=understanding-and-enhancing-mixed-sample-data)

<p align="center">
  <a href="https://arxiv.org/abs/2002.12047">ArXiv</a> •
  <a href="https://paperswithcode.com/paper/understanding-and-enhancing-mixed-sample-data">Papers With Code</a> •
  <a href="#about">About</a> •
  <a href="#experiments">Experiments</a> •
  <a href="#implementations">Implementations</a> •
  <a href="#pre-trained-models">Pre-trained Models</a>
</p>

<a href="https://colab.research.google.com/github/ecs-vlc/fmix/blob/master/notebooks/example_masks.ipynb">
<img src="http://www.pytorchbearer.org/assets/img/colab_logo.png" height="28">
  
Dive in with our example notebook in Colab!
</a>

</div>

## About

FMix is a variant of MixUp, CutMix, etc. introduced in our paper ['FMix: Enhancing Mixed Sampled Data Augmentation'](https://arxiv.org/abs/2002.12047). It uses masks sampled from Fourier space to mix training examples. Take a look at our [example notebook in colab](https://colab.research.google.com/github/ecs-vlc/fmix/blob/master/notebooks/example_masks.ipynb) which shows how you can generate masks in two dimensions

<div align="center">
  <img src="./fmix_example.png", width="80%">
</div>

and in three!

<div align="center">
  <img src="./fmix_3d.gif", width="30%">
</div>

## Experiments

### Core Experiments

Shell scripts for our core experiments can be found in the [experiments folder](./experiments). For example,

```bash
bash cifar_experiment cifar10 resnet fmix ./data
```

will train a PreAct-ResNet18 on CIFAR-10 with FMix. More information can be found at the start of each of the shell files.

### Additional Experiments

All additional classification experiments can be run via [`trainer.py`](./trainer.py)

### Analyses

For Grad-CAM, take a look at the [Grad-CAM notebook in colab](https://colab.research.google.com/github/ecs-vlc/fmix/blob/master/notebooks/grad_cam.ipynb).

For the other analyses, have a look in the [analysis folder](./analysis).

## Implementations

The core implementation of `FMix` uses `numpy` and can be found in [`fmix.py`](./fmix.py). We provide bindings for this in [PyTorch](https://pytorch.org/) (with [Torchbearer](https://github.com/pytorchbearer/torchbearer) or [PyTorch-Lightning](https://github.com/PyTorchLightning/pytorch-lightning)) and [Tensorflow](https://www.tensorflow.org/).

### Torchbearer

The `FMix` callback in [`torchbearer_implementation.py`](./implementations/torchbearer_implementation.py) can be added directly to your torchbearer code:

```python
from implementations.torchbearer_implementation import FMix

fmix = FMix()
trial = Trial(model, optimiser, fmix.loss(), callbacks=[fmix])
```

See an example in [`test_torchbearer.py`](./implementations/test_torchbearer.py).

### PyTorch-Lightning

For PyTorch-Lightning, we provide a class, `FMix` in [`lightning.py`](./implementations/lightning.py) that can be used in your `LightningModule`:

```python
from implementations.lightning import FMix

class CoolSystem(pl.LightningModule):
    def __init__(self):
        ...
        
        self.fmix = FMix()
    
    def training_step(self, batch, batch_nb):
        x, y = batch
        x = self.fmix(x)

x = self.forward(x)

loss = self.fmix.loss(x, y)
        return {'loss': loss}
```

See an example in [`test_lightning.py`](./implementations/test_lightning.py).

### Tensorflow

For Tensorflow, we provide a class, `FMix` in [`tensorflow_implementation.py`](./implementations/tensorflow_implementation.py) that can be used in your tensorflow code:

```python
from implementations.tensorflow_implementation import FMix

fmix = FMix()

def loss(model, x, y, training=True):
    x = fmix(x)
    y_ = model(x, training=training)
    return tf.reduce_mean(fmix.loss(y_, y))
```

See an example in [`test_tensorflow.py`](./implementations/test_tensorflow.py).

## Pre-trained Models

We provide pre-trained models via `torch.hub` (more coming soon). To use them, run

```python
import torch
model = torch.hub.load('ecs-vlc/FMix:master', ARCHITECTURE, pretrained=True)
```

where `ARCHITECTURE` is one of the following:

### CIFAR-10

#### PreAct-ResNet-18

| Configuration    | `ARCHITECTURE`                            | Accuracy |
| ---------------- | ----------------------------------------- | -------- |
| Baseline         | `'preact_resnet18_cifar10_baseline'`      | -------- |
|   + MixUp        | `'preact_resnet18_cifar10_mixup'`         | -------- |
|   + FMix         | `'preact_resnet18_cifar10_fmix'`          | -------- |
|   + Mixup + FMix | `'preact_resnet18_cifar10_fmixplusmixup'` | -------- |

#### PyramidNet-200

| Configuration    | `ARCHITECTURE`                       | Accuracy  |
| ---------------- | ------------------------------------ | --------- |
| Baseline         | `'pyramidnet_cifar10_baseline'`      | 98.31     |
|   + MixUp        | `'pyramidnet_cifar10_mixup'`         | 97.92     |
|   + FMix         | `'pyramidnet_cifar10_fmix'`          | __98.64__ |

### ImageNet

#### ResNet-101

| Configuration    | `ARCHITECTURE`                       | Accuracy (Top-1) |
| ---------------- | ------------------------------------ | ---------------- |
| Baseline         | `'renset101_imagenet_baseline'`      | 76.51            |
|   + MixUp        | `'renset101_imagenet_mixup'`         | 76.27            |
|   + FMix         | `'renset101_imagenet_fmix'`          | __76.72__        |