VGG16-PyTorch - GitLite

KO | EN

gitlite — search

#javascript #hacktoberfest #react #ai #typescript #llm #android #machine-learning #deep-learning #linux #cli #security #mcp #openai

VGG16-PyTorch

(★ 91)

VGG16 Net implementation from PyTorch Examples scripts for ImageNet dataset

File Explorer

Download Latest Version (.zip)

- data_to_plot.xlsx
- Report_on_PyTorch_VGG16_training.pptx
- training_acc_top1.png
- training_acc_top5.png
- training_loss.png
- val_acc_top1.png
- val_acc_top5.png
- val_loss.png
- vgg16_BN_pytorch_cpu_gpu_usage.PNG
- vgg16_BN_testing_pytorch.PNG
- vgg16_BN_testing_pytorch_1.PNG
- vgg16_BN_testing_pytorch_2.PNG
- vgg16_BN_testing_pytorch_33.PNG
- vgg16_BN_testing_pytorch_39.PNG
- vgg16_BN_testing_pytorch_39_1.PNG
- vgg16_BN_testing_pytorch_49.PNG
- vgg16_BN_testing_pytorch_51.PNG
- vgg16_BN_testing_pytorch_55.PNG
- vgg16_BN_testing_pytorch_56.PNG
- vgg16_BN_testing_pytorch_57.PNG
- vgg16_BN_testing_pytorch_58.PNG
- vgg16_BN_training_pytorch.PNG
- vgg16_BN_training_pytorch_10.PNG
- vgg16_BN_training_pytorch_11.PNG
- vgg16_BN_training_pytorch_12.PNG
- vgg16_BN_training_pytorch_13.PNG
- vgg16_BN_training_pytorch_14.PNG
- vgg16_BN_training_pytorch_2.PNG
- vgg16_BN_training_pytorch_3.PNG
- vgg16_BN_training_pytorch_4.PNG
- vgg16_BN_training_pytorch_5.PNG
- vgg16_BN_training_pytorch_6.PNG
- vgg16_BN_training_pytorch_7.PNG
- vgg16_BN_training_pytorch_8.PNG
- vgg16_BN_training_pytorch_9.PNG
ILSVRC2012_test_ground_truth.txt
main.py
parallelism_tutorial.py
README.md
requirements.txt
test.py
valprep.sh
vgg.py

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

## Disclaimer
This is a modified repository from [PyTorch/examples/ImageNet](https://github.com/pytorch/examples/tree/master/imagenet). Please refer to the original repository for more details.

# ImageNet training in PyTorch

This implements training of popular model architectures, such as ResNet, AlexNet, and VGG on the ImageNet dataset.

## Requirements

- Install PyTorch ([pytorch.org](http://pytorch.org))
- `pip install -r requirements.txt`
- Download the ImageNet dataset and move validation images to labeled subfolders
    - To do this, you can use the following script: https://raw.githubusercontent.com/soumith/imagenetloader.torch/master/valprep.sh

## Training

To train a model, run `main.py` with the desired model architecture and the path to the ImageNet dataset:

```bash
python main.py -a resnet18 [imagenet-folder with train and val folders]
```

The default learning rate schedule starts at 0.1 and decays by a factor of 10 every 30 epochs. This is appropriate for ResNet and models with batch normalization, but too high for AlexNet and VGG. Use 0.01 as the initial learning rate for AlexNet or VGG:

```bash
python main.py -a alexnet --lr 0.01 [imagenet-folder with train and val folders]
```

## Multi-processing Distributed Data Parallel Training

You should always use the NCCL backend for multi-processing distributed training since it currently provides the best distributed training performance.

### Single node, multiple GPUs:

```bash
python main.py -a resnet50 --lr 0.01 --dist-url 'tcp://127.0.0.1:FREEPORT' --dist-backend 'nccl' --multiprocessing-distributed [imagenet-folder with train and val folders]
```

### Multiple nodes:

Node 0:
```bash
python main.py -a resnet50 --lr 0.01 --dist-url 'tcp://IP_OF_NODE0:FREEPORT' --dist-backend 'nccl' --multiprocessing-distributed --world-size 2 --rank 0 [imagenet-folder with train and val folders]
```

Node 1:
```bash
python main.py -a resnet50 --lr 0.01 --dist-url 'tcp://IP_OF_NODE0:FREEPORT' --dist-backend 'nccl' --multiprocessing-distributed --world-size 2 --rank 1 [imagenet-folder with train and val folders]
```

## Usage

```
usage: main.py [-h] [--arch ARCH] [-j N] [--epochs N] [--start-epoch N] [-b N]
               [--lr LR] [--momentum M] [--weight-decay W] [--print-freq N]
               [--resume PATH] [-e] [--pretrained] [--world-size WORLD_SIZE]
               [--rank RANK] [--dist-url DIST_URL]
               [--dist-backend DIST_BACKEND] [--seed SEED] [--gpu GPU]
               [--multiprocessing-distributed]
               DIR

PyTorch ImageNet Training

positional arguments:
  DIR                   path to dataset

optional arguments:
  -h, --help            show this help message and exit
  --arch ARCH, -a ARCH  model architecture: alexnet | densenet121 |
                        densenet161 | densenet169 | densenet201 |
                        resnet101 | resnet152 | resnet18 | resnet34 |
                        resnet50 | squeezenet1_0 | squeezenet1_1 | vgg11 |
                        vgg11_bn | vgg13 | vgg13_bn | vgg16 | vgg16_bn | vgg19
                        | vgg19_bn (default: resnet18)
  -j N, --workers N     number of data loading workers (default: 4)
  --epochs N            number of total epochs to run
  --start-epoch N       manual epoch number (useful on restarts)
  -b N, --batch-size N  mini-batch size (default: 256), this is the total
                        batch size of all GPUs on the current node when using
                        Data Parallel or Distributed Data Parallel
  --lr LR, --learning-rate LR
                        initial learning rate
  --momentum M          momentum
  --weight-decay W, --wd W
                        weight decay (default: 1e-4)
  --print-freq N, -p N  print frequency (default: 10)
  --resume PATH         path to latest checkpoint (default: none)
  -e, --evaluate        evaluate model on validation set
  --pretrained          use pre-trained model
  --world-size WORLD_SIZE
                        number of nodes for distributed training
  --rank RANK           node rank for distributed training
  --dist-url DIST_URL   url used to set up distributed training
  --dist-backend DIST_BACKEND
                        distributed backend
  --seed SEED           seed for initializing training.
  --gpu GPU             GPU id to use.
  --multiprocessing-distributed
                        Use multi-processing distributed training to launch N
                        processes per node, which has N GPUs. This is the
                        fastest way to use PyTorch for either single node or
                        multi node data parallel training
```

### Our case:

```bash
python main.py -a vgg16 --lr 0.01 -b 32 D:\Dataset\Imagenet2012\Images
```

For resuming:

```bash
python main.py -a vgg16 --lr 0.01 -b 32 --resume D:\Dataset\Imagenet2012\Images
```

With Batch Normalization:

```bash
python main.py -a vgg16_bn --lr 0.01 -b 16 D:\Dataset\Imagenet2012\Images
```

```bash
python main.py -a vgg16_bn --lr 0.01 -b 16 --resume . --epochs 50 --start-epoch 35 --multiprocessing-distributed  D:\Dataset\Imagenet2012\Images
```

```bash
python main.py -a vgg16_bn --lr 0.001 -b 16 --resume model_best.pth.tar --epochs 40 --gpu 0  D:\Dataset\Imagenet2012\Images
```

## Download the ImageNet dataset
The ImageNet Large Scale Visual Recognition Challenge (ILSVRC) dataset has 1000 categories and 1.2 million images. The images do not need to be preprocessed or packaged in any database, but the validation images need to be moved into appropriate subfolders.

1. Download the images from http://image-net.org/download-images

2. Extract the training data:
  ```bash
  mkdir train && mv ILSVRC2012_img_train.tar train/ && cd train
  tar -xvf ILSVRC2012_img_train.tar && rm -f ILSVRC2012_img_train.tar
  find . -name "*.tar" | while read NAME ; do mkdir -p "${NAME%.tar}"; tar -xvf "${NAME}" -C "${NAME%.tar}"; rm -f "${NAME}"; done
  cd ..
  ```

3. Extract the validation data and move images to subfolders:
  ```bash
  mkdir val && mv ILSVRC2012_img_val.tar val/ && cd val && tar -xvf ILSVRC2012_img_val.tar
  wget -qO- https://raw.githubusercontent.com/soumith/imagenetloader.torch/master/valprep.sh | bash
  ```