hnd-ghnd-object-detectors

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

hnd-ghnd-object-detectors

★ 25 Open GitHub ↗

[ICPR 2020] "Neural Compression and Filtering for Edge-assisted Real-time Object Detection in Challenged Networks" and [ACM MobiCom EMDL 2020] "Split Computing for Complex Object Detectors: Challenges and Preliminary Results"

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# HND & GHND for Object Detectors
Head Network Distillation (HND) and Generalized HND for Faster, Mask, and Keypoint R-CNNs  
- "Neural Compression and Filtering for Edge-assisted Real-time Object Detection in Challenged Networks," [ICPR 2020](https://www.micc.unifi.it/icpr2020/)  
[[PDF](https://ieeexplore.ieee.org/document/9412388)] [[Supp](https://ieeexplore.ieee.org/document/9412388/media#media)] [[Preprint](https://arxiv.org/abs/2007.15818)]
- "Split Computing for Complex Object Detectors: Challenges and Preliminary Results," [ACM MobiCom 2020 Workshop EMDL '20](https://emdl20.github.io/index.html)  
[[PDF (Open Access)](https://dl.acm.org/doi/abs/10.1145/3410338.3412338)] [[Preprint](https://arxiv.org/abs/2007.13312)]  
  
Looking for **image classification tasks**? Take a look at https://github.com/yoshitomo-matsubara/head-network-distillation as well

![GHND and Neural Filter](https://raw.githubusercontent.com/yoshitomo-matsubara/hnd-ghnd-object-detectors/master/img/ghnd_and_nf.png)

## Citations
```bibtex
@inproceedings{matsubara2021neural,
  title={Neural Compression and Filtering for Edge-assisted Real-time Object Detection in Challenged Networks},
  author={Yoshitomo Matsubara and Marco Levorato},
  booktitle={2020 25th International Conference on Pattern Recognition (ICPR)},
  pages={2272--2279},
  year={2021}
}

@inproceedings{matsubara2020split,
  title={Split Computing for Complex Object Detectors: Challenges and Preliminary Results},
  author={Matsubara, Yoshitomo and Levorato, Marco},
  booktitle={Proceedings of the 4th International Workshop on Embedded and Mobile Deep Learning},
  pages={7--12},
  year={2020}
}

```

## Requirements
- Python 3.6
- pipenv
- [myutils](https://github.com/yoshitomo-matsubara/myutils)

## How to clone
```
git clone https://github.com/yoshitomo-matsubara/hnd-ghnd-object-detectors.git
cd hnd-ghnd-object-detectors/
git submodule init
git submodule update --recursive --remote
pipenv install
```
It is not necessary to use pipenv, and you can instead manually install the required packages listed in [Pipfile](Pipfile), using pip3.

## COCO 2017 Dataset
```
mkdir -p ./resource/dataset/coco2017
cd ./resource/dataset/coco2017
wget http://images.cocodataset.org/zips/train2017.zip
wget http://images.cocodataset.org/zips/val2017.zip
wget http://images.cocodataset.org/annotations/annotations_trainval2017.zip
unzip -q train2017.zip
unzip -q val2017.zip
unzip -q annotations_trainval2017.zip
```

## Checkpoints with trained model weights
1. Download [emdl2020.zip](https://github.com/yoshitomo-matsubara/hnd-ghnd-object-detectors/releases/download/google_drive-to-github/emdl2020.zip)
2. Unzip **emdl2020.zip** at the root directory of this repository so that you can use the checkpoints with yaml config files under [config/hnd/](config/hnd/)
3. Download [icpr2020-faster_rcnn.zip](https://github.com/yoshitomo-matsubara/hnd-ghnd-object-detectors/releases/download/google_drive-to-github/icpr2020-faster_rcnn.zip), [icpr2020-mask_rcnn.zip](https://github.com/yoshitomo-matsubara/hnd-ghnd-object-detectors/releases/download/google_drive-to-github/icpr2020-mask_rcnn.zip), and [icpr2020-keypoint_rcnn.zip](https://github.com/yoshitomo-matsubara/hnd-ghnd-object-detectors/releases/download/google_drive-to-github/icpr2020-keypoint_rcnn.zip)
4. Unzip the three zip files at the root directory of this repository so that you can use the checkpoints with yaml config files under [config/hnd/](config/hnd/) and [config/ghnd/](config/ghnd/)
5. Test the trained models using the checkpoints and yaml config files  
e.g., Faster R-CNN with 3 output channels for bottleneck
```
pipenv run python src/mimic_runner.py --config config/hnd/faster_rcnn-backbone_resnet50-b3ch.yaml
pipenv run python src/mimic_runner.py --config config/ghnd/faster_rcnn-backbone_resnet50-b3ch.yaml
```

## Distilling head portion of R-CNNs
If you have already downloaded our trained model weights above, you should move the ckpt files in `resource/ckpt/` to somewhere else or change ckpt file path (`ckpt` under `student_model`) in config files.

### Bottleneck-injected Faster R-CNN with ResNet-50 and FPN
e.g., Bottleneck with 3 output channels
```
# HND
pipenv run python src/mimic_runner.py --config config/hnd/faster_rcnn-backbone_resnet50-b3ch.yaml -distill

# GHND
pipenv run python src/mimic_runner.py --config config/ghnd/faster_rcnn-backbone_resnet50-b3ch.yaml -distill
```

### Bottleneck-injected Mask R-CNN with ResNet-50 and FPN
e.g., Bottleneck with 3 output channels
```
# HND
pipenv run python src/mimic_runner.py --config config/hnd/mask_rcnn-backbone_resnet50-b3ch.yaml -distill

# GHND
pipenv run python src/mimic_runner.py --config config/ghnd/mask_rcnn-backbone_resnet50-b3ch.yaml -distill
```

### Bottleneck-injected Keypoint R-CNN with ResNet-50 and FPN
e.g., Bottleneck with 3 output channels
```
# HND
pipenv run python src/mimic_runner.py --config config/hnd/keypoint_rcnn-backbone_resnet50-b3ch.yaml -distill

# GHND
pipenv run python src/mimic_runner.py --config config/ghnd/keypoint_rcnn-backbone_resnet50-b3ch.yaml -distill
```

## Training a neural filter on top of our trained, bottleneck-injected Keypoint R-CNN
```
pipenv run python src/ext_runner.py --config config/ext/keypoint_rcnn-backbone_ext_resnet50-b3ch.yaml -train
```

## References
- [pytorch/vision/references/detection/](https://github.com/pytorch/vision/tree/master/references/detection)
- [code for visualization in the object detection tutorial](https://github.com/pytorch/vision/issues/1610)

hnd-ghnd-object-detectors

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README