deep-residual-networks-pyfunt

KO | EN

gitlite — search

#python #typescript #golang #android #machine-learning #macos #java #nodejs #windows #cli #kubernetes #vue #security #ios

deep-residual-networks-pyfunt

(★ 52)

Python implementation of "Deep Residual Learning for Image Recognition" (http://arxiv.org/abs/1512.03385 - MSRA, winner team of the 2015 ILSVRC and COCO challenges).

deep-residual-networks-pyfunt Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Deep Residual Learning for Image Recognition

Implementation of ["Deep Residual Learning for Image Recognition", Kaiming
He, Xiangyu Zhang, Shaoqing Ren, Jian Sun](http://arxiv.org/abs/1512.03385) in [PyFunt](https://github.com/dnlcrl/PyFunt) (a simple Python + Numpy DL framework).

Also inspired by [this implementation in Lua + Torch](https://github.com/gcr/torch-residual-networks).

The network operates on minibatches of data that have shape (N, C, H, W)
consisting of N images, each with height H and width W and with C input
channels. It has, like in the reference paper, (6*n)+2 layers,
composed as below:

Every convolution layer has a pad=1 and stride=1, except for the dimension
enhancning layers which has a stride of 2 to mantain the computational
complexity.
Optionally, there is the possibility of setting m affine layers immediatley before the softmax layer by setting the hidden_dims parameter, which should be a list of integers representing the numbe of neurons for each affine layer.

Each residual block is composed as below:

After every layer, a batch normalization with momentum .1 is applied.

## Requirements

- [Python 2.7](https://www.python.org/)
- [numpy](www.numpy.org/)
- [pyfunt](https://github.com/dnlcrl/PyFunt)
- [pydatset](https://github.com/dnlcrl/PyDatSet)

After you get Python, you can get [pip](https://pypi.python.org/pypi/pip) and install all requirements by running:
	
	pip install -r requirements.txt

## Usage

If you want to train the network on the CIFAR-10 dataset, simply run:

python train.py --help
	
Otherwise, you have to get the right train.py for MNIST or SFDDD datasets, they are respectively on the mnist and sfddd git branches:

- train.py for MNIST: https://github.com/dnlcrl/PyResNet/blob/mnist/train.py

- train.py for SFDDD: https://github.com/dnlcrl/PyResNet/blob/sfddd/train.py

## Experiments Results

You can view all the experiments results in the [./docs directory](https://github.com/dnlcrl/PyResNet/tree/master/docs). Main results are shown below:

###  [CIFAR-10](https://www.cs.toronto.edu/~kriz/cifar.html)

best error: 9.59 % (accuracy: 0.9041) with a 20 layers residual network (n=3):

[![CIFAR-10 results](https://github.com/dnlcrl/PyResNet/blob/master/docs/imgs/cifar.png)](https://github.com/dnlcrl/PyResNet/blob/master/docs/CIFAR-10%20Experiments.ipynb)

[CIFAR-10 Results - iPython notebook](https://github.com/dnlcrl/PyResNet/blob/master/docs/CIFAR-10%20Experiments.ipynb)

###  [MNIST](http://yann.lecun.com/exdb/mnist/)

best error: 0.36 % (accuracy: 0.9964) with a 32 layers residual network (n=5):

[![MNIST results](https://github.com/dnlcrl/PyResNet/blob/master/docs/imgs/mnistres.png)](https://github.com/dnlcrl/PyResNet/blob/master/docs/MNIST%20Experiments.ipynb)

[MNIST Results - iPython notebook](https://github.com/dnlcrl/PyResNet/blob/master/docs/MNIST%20Experiments.ipynb)

###  [SFDDD](https://www.kaggle.com/c/state-farm-distracted-driver-detection)

best error: 0.25 % (accuracy: 0.9975 %) on a subset (1000 samples) of the train data (~21k images) with a 44 layers residual network (n=7), resizing the images to 64x48, randomly cropping 32x32 images for training and cropping a 32x32 image from the center of the original images for testing. Unfortunately I got more than 2% error on Kaggle's results (composed of ~80k images).

[![SFDDD results](https://github.com/dnlcrl/deep-residual-networks-pyfunt/blob/master/docs/imgs/SFDDD.png)](https://github.com/dnlcrl/PyResNet/blob/master/docs/SFDDD%20Experiments.ipynb)
	
[SFDDD Results - iPython notebook](https://github.com/dnlcrl/PyResNet/blob/master/docs/SFDDD%20Experiments.ipynb)

<!--## TODOs:

- regenerate plots with english labels

- experimentat elastic distortion as data augmentation function on MNIST

- experiment other data augmentation functions on SFDDD

- implementation of the second version of residual networks, as explained in ["Identity Mappings in Deep Residual Networks" by Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun](http://arxiv.org/pdf/1603.05027v1.pdf) -->

deep-residual-networks-pyfunt

deep-residual-networks-pyfunt Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

deep-residual-networks-pyfunt

deep-residual-networks-pyfunt Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README