KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#typescript #ai-agents #developer-tools #cli #codex #claude-code #dsh-plugin #mcp #react #agent #dsh #deepseek-harness

MSCG-Net

★ 75 Open GitHub ↗

Multi-view Self-Constructing Graph Convolutional Networks with Adaptive Class Weighting Loss for Semantic Segmentation

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# MSCG-Net for Semantic Segmentation
## Introduce
This repository contains MSCG-Net models (MSCG-Net-50 and MSCG-Net-101) for semantic segmentation in [Agriculture-Vision Challenge and Workshop](https://github.com/SHI-Labs/Agriculture-Vision) (CVPR 2020), and the pipeline of training and testing models, implemented in PyTorch. Please refer to our paper for details:  [Multi-view SelfConstructing Graph Convolutional Networks with Adaptive Class Weighting Loss for Semantic Segmentation](http://openaccess.thecvf.com/content_CVPRW_2020/papers/w5/Liu_Multi-View_Self-Constructing_Graph_Convolutional_Networks_With_Adaptive_Class_Weighting_Loss_CVPRW_2020_paper.pdf)

## Code structure

```
├── config		# config code
├── data		# dataset loader and pre-processing code
├── tools		# train and test code, ckpt and model_load
├── lib			# model block, loss, utils code, etc
└── ckpt 		# output check point, trained weights, log files, etc

```

## Environments

- python 3.5+
- pytorch 1.4.0
- opencv 3.4+
- tensorboardx 1.9
- albumentations 0.4.0
- pretrainedmodels 0.7.4
- others (see requirements.txt)

## Dataset prepare

1. change DATASET_ROOT to your dataset path in ./data/AgricultureVision/pre_process.py
```
DATASET_ROOT = '/your/path/to/Agriculture-Vision'
```

## Train with a single GPU

```
CUDA_VISIBLE_DEVICES=0 python ./tools/train_R50.py  # trained weights ckpt1
# train_R101.py 								 # trained weights, ckpt2
# train_R101_k31.py 							  # trained weights, ckpt3
```

**Please note that:** we first train these models using Adam combined with Lookahead as the optimizer
for the first 10k iterations (around 7~10 epochs) and then change the optimizer to
SGD in the remaining iterations. So you will have to **manually change the code to switch the optimizer**
**to SGD** as follows:

```
# Change line 48: Copy the file name ('----.pth') of the best ckpt trained with Adam
train_args.snapshot = '-------.pth'
...
# Comment line 92
# base_optimizer = optim.Adam(params, amsgrad=True)

# uncomment line 93
base_optimizer = optim.SGD(params, momentum=train_args.momentum, nesterov=True)
```

## Test with a single GPU

```
# To reproduce the leaderboard results (0.608), download the trained-weights ckpt1,2,3
# and save them with the original names into ./ckpt folder before run test_submission.py
CUDA_VISIBLE_DEVICES=0 python ./tools/test_submission.py
```

#### Trained weights for  3 models download (save them to ./ckpt before run test_submission)
[ckpt1](https://drive.google.com/open?id=1eVvUd4TVUtEe_aUgKamUrDdSlrIGHuH3),[ckpt2](https://drive.google.com/open?id=1vOlS4LfHGnWIUpqTFB2a07ndlpBxFmVE),[ckpt3](https://drive.google.com/open?id=1nEPjnTlcrzx0FOH__MbP3e_f9PlhjMa2)

## Results Summary

| Models | mIoU (%) | Background | Cloud shadow | Double plant | Planter skip | Standing water | Waterway | Weed cluster |
| ----------------------------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- |
| MSCG-Net-50 (ckpt1) | 54.7 | 78.0 | 50.7 | 46.6 | 34.3 | 68.8 | 51.3 | 53.0 |
| ***MSCG-Net-101 (ckpt2)*** | ***55.0*** | ***79.8*** | ***44.8*** | ***55.0*** | ***30.5*** | ***65.4*** | ***59.2*** | ***50.6*** |
| MSCG-Net-101_k31 (ckpt3) | 54.1 | 79.6 | 46.2 | 54.6 | 9.1 | 74.3 | 62.4 | 52.1 |
| Ensemble_TTA (ckpt1,2) | 59.9 | 80.1 | 50.3 | 57.6 | 52.0 | 69.6 | 56.0 | 53.8 |
| **Ensemble_TTA (ckpt1,2,3)** | 60.8 | 80.5 | **51.0** | 58.6 | 49.8 | **72.0** | 59.8 | **53.8** |
| **Ensemble_TTA (new_5model)** | **62.2** | **80.6** | 48.7 |**62.4** | **58.7** | 71.3| **60.1** | 53.4 |

Please note that all our single model's scores are computed with just single-scale (512x512) and single feed-forward inference without TTA. TTA denotes test time augmentation (e.g. flip and mirror). Ensemble_TTA (ckpt1,2) denotes two models (ckpt1, and ckpt2) ensemble with TTA, and (ckpt1, 2, 3) denotes three models ensemble.

### Model Size

| Models | Backbones | Parameters | GFLOPs | Inference time (CPU/GPU ) |
| ---------------- | ------------------- | ---------- | ------ | ------------------------------- |
| MSCG-Net-50 | Se_ResNext50_32x4d | 9.59 | 18.21 | 522 / 26 ms |
| MSCG-Net-101 | Se_ResNext101_32x4d | 30.99 | 37.86 | 752 / 45 ms |
| MSCG-Net-101_k31 | Se_ResNext101_32x4d | 30.99 | 37.86 | 752 / 45 ms |

Please note that all backbones used pretrained weights on **ImageNet** that can be imported and downloaded from the [link](https://github.com/Cadene/pretrained-models.pytorch#senet). And MSCG-Net-101_k31 has exactly the same architecture wit MSCG-Net-101, while it is trained with extra 1/3 validation set (4,431) instead of just using the official training images (12,901).

## Citation: 
Please consider citing our work if you find the code helps you

[Multi-view Self-Constructing Graph Convolutional Networks with Adaptive Class Weighting Loss for Semantic Segmentation](http://openaccess.thecvf.com/content_CVPRW_2020/papers/w5/Liu_Multi-View_Self-Constructing_Graph_Convolutional_Networks_With_Adaptive_Class_Weighting_Loss_CVPRW_2020_paper.pdf)

```
@InProceedings{Liu_2020_CVPR_Workshops,
author = {Liu, Qinghui and Kampffmeyer, Michael C. and Jenssen, Robert and Salberg, Arnt-Borre},
title = {Multi-View Self-Constructing Graph Convolutional Networks With Adaptive Class Weighting Loss for Semantic Segmentation},
booktitle = {The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {June},
year = {2020}
}
```
[Self-Constructing Graph Convolutional Networks for Semantic Labeling](https://arxiv.org/pdf/2003.06932)
```
@inproceedings{liu2020scg,
  title={Self-Constructing Graph Convolutional Networks for Semantic Labeling},
  author={Qinghui Liu and Michael Kampffmeyer and Robert Jenssen and Arnt-Børre Salberg},
  booktitle={Proceedings of IGARSS 2020 - 2020 IEEE International Geoscience and Remote Sensing Symposium},
  year={2020}
}
```

MSCG-Net

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README