ImageNet-Training

KO | EN

gitlite — search

#javascript #hacktoberfest #ai #typescript #llm #go #machine-learning #deep-learning #linux #docker #cli #vue #security #open-source

ImageNet-Training

(★ 101)

ImageNet training using torch

ImageNet-Training Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

Deep Learning on ImageNet using Torch
=====================================
This is a complete training example for Deep Convolutional Networks on the ILSVRC classification task.

Data is preprocessed and cached as a LMDB data-base for fast reading. A separate thread buffers images from the LMDB record in the background.

Multiple GPUs are also supported by using nn.DataParallelTable (https://github.com/torch/cunn/blob/master/docs/cunnmodules.md).

This code allows training at 4ms/sample with the AlexNet model and 2ms for testing on a single GPU (using Titan Z with 1 active gpu)

## Dependencies
* Torch (http://torch.ch)
* "eladtools" (https://github.com/eladhoffer/eladtools) for optimizer.
* "lmdb.torch" (http://github.com/eladhoffer/lmdb.torch) for LMDB usage.
* "DataProvider.torch" (https://github.com/eladhoffer/DataProvider.torch) for DataProvider class.
* "cudnn.torch" (https://github.com/soumith/cudnn.torch) for faster training. Can be avoided by changing "cudnn" to "nn" in models.

To install all dependencies (assuming torch is installed) use:
```bash
luarocks install https://raw.githubusercontent.com/eladhoffer/eladtools/master/eladtools-scm-1.rockspec
luarocks install https://raw.githubusercontent.com/eladhoffer/lmdb.torch/master/lmdb.torch-scm-1.rockspec
luarocks install https://raw.githubusercontent.com/eladhoffer/DataProvider.torch/master/dataprovider-scm-1.rockspec
```

## Data
* To get the ILSVRC data, you should register on their site for access: http://www.image-net.org/
* Extract all archives and configure the data location and save dir in **Config.lua**. You can also change the saved image size by editing the default value `ImageMinSide=256`.
* LMDB records for fast read access are created by running **CreateLMDBs.lua**.
It defaults to saving the compressed jpgs (about ~24GB for training data, ~1GB for validation data when smallest image dimension is 256).
* To validate the LMDB configuration and test its loading speed, you can run **TestLMDBs.lua**.
* All data related functions used for training are available at **Data.lua**.

## Model configuration
Network model is defined by writing a <ModelName>.lua file in `Models` folder, and selecting it using the `network` flag.
The model file must return a trainable network. It can also specify additional training options such optimization regime, input size modifications.

e.g for a model file:
```lua
local model = nn.Sequential():add(...)
return  --optional: you can also simply return model
{
  model = model,
  regime = {
    epoch        = {1,    19,   30,   44,   53  },
    learningRate = {1e-2, 5e-3, 1e-3, 5e-4, 1e-4},
    weightDecay  = {5e-4, 5e-4, 0,    0,    0   }
  }
}
```
Currently available in `Models` folder are: `AlexNet`, `MattNet`, `OverFeat`, `GoogLeNet`, `CaffeRef`, `NiN`. Some are available with a batch normalized version (denoted with `_BN`)

## Training
You can start training using **Main.lua** by typing:
```lua
th Main.lua -network AlexNet -LR 0.01
```
or if you have 2 gpus availiable,
```lua
th Main.lua -network AlexNet -LR 0.01 -nGPU 2 -batchSize 256
```
A more elaborate example continuing a pretrained network and saving intermediate results
```lua
th Main.lua -network GoogLeNet_BN -batchSize 64 -nGPU 2 -save GoogLeNet_BN -bufferSize 9600 -LR 0.01 -checkpoint 320000 -weightDecay 1e-4 -load ./pretrainedNet.t7
```
Buffer size should be adjusted to suit the used hardware and configuration. Default value is 5120 (40 batches of 128) which works well when using a non SSD drive and 16GB ram. Bigger buffer size allows better sample shuffling.

## Output
Training output will be saved to folder defined with `save` flag.

The complete netowork will be saved on each epoch as **Net_<#epoch>.t7** along with
* A complete log **Log.txt**
* Error rate summary **ErrorRate.log** and accompanying�������������� **ErrorRate.log.eps** graph

## Additional flags
|Flag             | Default Value        |Description
|:----------------|:--------------------:|:----------------------------------------------
|modelsFolder     |./Models/             | Models Folder
|network          |AlexNet               | Model file - must return valid network.
|LR               |0.01                  | learning rate
|LRDecay          |0                     | learning rate decay (in # samples)
|weightDecay      |5e-4                  | L2 penalty on the weights
|momentum         |0.9                   | momentum
|batchSize        |128,                  | batch size
|optimization     |'sgd'                 | optimization method
|seed             |123                   | torch manual random number generator seed
|epoch            |-1                    | number of epochs to train, -1 for unbounded
|testonly         |false                 | Just test loaded net on validation set
|threads          |8                     | number of threads
|type             |'cuda'                | float or cuda
|bufferSize       |5120                  | buffer size
|devid            |1                     | device ID (if using CUDA)
|nGPU             |1                     | num of gpu devices used
|constBatchSize   |false                 | do not allow varying batch sizes - e.g for ccn2 kernel
|load             |''                    | load existing net weights
|save             |time-identifier       | save directory
|optState         |false                 | Save optimization state every epoch
|checkpoint       |0                     | Save a weight check point every n samples. 0 for off
|augment          |1                     | data augmentation level - {1 - simple mirror and crops, 2 +scales, 3 +rotations}
|estMeanStd       |preDef                | estimate mean and std. Options: {preDef, simple, channel, image}
|shuffle          |true                  | shuffle training samples

ImageNet-Training

ImageNet-Training Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

ImageNet-Training

ImageNet-Training Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README