KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#developer-tools #typescript #claude-code #codex #dsh #dsh-plugin #ai-agent #ai-agents #deepseek-harness #agent-skills #macos #cli

DSNet

★ 145 Open GitHub ↗

a novel real-time model in semantic segmentation

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# DSNet：A Novel Way to Use Atrous Convolutions in Semantic Segmentation
## :rocket: If it helps you, click a star! :star: ##
a novel real-time model in semantic segmentation.

Paper: ([PDF]https://arxiv.org/abs/2406.03702))
[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/dsnet-a-novel-way-to-use-atrous-convolutions/real-time-semantic-segmentation-on-cityscapes-1)](https://paperswithcode.com/sota/real-time-semantic-segmentation-on-cityscapes-1?p=dsnet-a-novel-way-to-use-atrous-convolutions) [![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/dsnet-a-novel-way-to-use-atrous-convolutions/semantic-segmentation-on-camvid)](https://paperswithcode.com/sota/semantic-segmentation-on-camvid?p=dsnet-a-novel-way-to-use-atrous-convolutions) 
[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/dsnet-a-novel-way-to-use-atrous-convolutions/semantic-segmentation-on-bdd100k-val)](https://paperswithcode.com/sota/semantic-segmentation-on-bdd100k-val?p=dsnet-a-novel-way-to-use-atrous-convolutions)

This is the implementation for DSNet. DSNetV2 is currently under development... DSNetV2 will offer a better balance between speed and accuracy, as well as a more comprehensive DSNet family ranging from small to large models.

```
git clone  https://github.com/takaniwa/DSNet.git
```

## News 2024/7/28
We extended the MSAF and MSA experiments for classification by applying them to models like ResNet18. On the ImageNet1K task, this led to a 3.3% accuracy improvement with only a 1% increase in computational cost! We will include a more detailed explanation of this module in our paper for submission to a high-quality journal.

Performance of MSA and MSAF on ResNet. "Insert" represents applying the module after the feature fusion of BasicBlock, and "Fusion" represents using MSAF or the AFF series for feature fusion. r stands for channel compression multiplier. The GFLOPs is measured at 256 x 3 x 224 x 224.
| Method       | Type   | Model   | r   | #Params | #GFLOPs | Top1. acc(%) |
|:-------------:|:------:|:-------:|:---:|:-------:|:-------:|:------------:|
| Add           | None   | ResNet18| -   | 11.7    | 434.9   | 69.7         |
| Add           | None   | ResNet34| -   | 21.8    | 877.1   | 72.9         |
| SE            | Insert | ResNet18| 16  | 11.8    | 435.1   | 71.2         |
| MSA(Ours)     | Insert | ResNet18| 16  | 12.1    | 441.1   | **72.2**         |
| MSA(Ours)     | Insert | ResNet18| 4   | 13.1    | 455.5   | **72.9**        |
| AFF           | Fusion | ResNet18| 4   | 12.4    | 448.3   | 72.0         |
| iAFF          | Fusion | ResNet18| 4   | 12.8    | 461.7   | 60.2         |
| MSAF(Ours)    | Fusion | ResNet18| 16  | 12.1    | 441.1   | **72.3**         |
| MSAF(Ours)    | Fusion | ResNet18| 4   | 13.1    | 455.5   | **73.2**         |

## **Environment**: 
PyTorch 1.10

python 3.8

4\*RTX4090 or 8\*RTX4090

```
pip install -r requirements.txt
```
## **Highlight**
<table border="0">
<tr>
    <td>
        <figure>
            <img src="./figs/params vs acc.jpg" width="90%" />
            <figcaption style="text-align: center;">Params vs mIOU on Cityscapes val set</figcaption>
        </figure>
    </td>
    <td>
        <figure>
            <img src="./figs/fps.jpg" width="90%" />
            <figcaption > ADE20K</figcaption>
        </figure>
    </td>
</tr>
</table>

• We revisited the design of atrous convolutions in CNNs,and explored three empirical guidelines for atrous convolution. Based on the above guidelines, we proposed a novel Dual-branch network.

• DSNet achieves a new state-of-the-art trade-off between accuracy and speed on ADE20K, Cityscapes,and BDD10K.

## **Overview**: 
<p align="center">
  <img src="figs/model.jpg" alt="overview-of-our-method" width="800"/></br>
  <span align="center">An overview of the basic architecture of our proposed DSNet. </span> 
</p>

<p align="center">
  <img src="figs/mafcb.jpg"  width="800"/></br>
  <span align="center">Diagram of Multi-Scale Fusion Atrous Convolutional Block (MFACB). </span> 
</p>

## **Train and Inference speed**:
This implementation is based on [HRNet-Semantic-Segmentation](https://github.com/HRNet/HRNet-Semantic-Segmentation) and [PIDNet](https://github.com/XuJiacong/PIDNet). Please refer to their repository for installation and dataset preparation. The inference speed is tested on single RTX 3090 or RTX4090. BDD10K has not been implemented in the above link. The dataset storage format is as follows. Download link: [web page](https://bdd-data.berkeley.edu/)

- bdd
  - seg
    - color_labels
      - train
      - val
    - images
      - train
      - val
      - test
    - labels
      - train
      - val

### Train
```
python -m torch.distributed.launch --nproc_per_node=4 DSNet/tools/train.py
```

### Inference speed
```
python DSNet/models/speed/dsnet_speed.py
```

## Weight
### DSNet-Base:
  
  DSNet_Base_imagenet：[ Baidu drive](https://pan.baidu.com/s/1acGfjtF1eHb3hNxyHcsJTA?pwd=a123) ,[google drive](https://drive.google.com/file/d/1LqmgL4thNJFcMWRYaXJUFNTy2y5FvZ8E/view?usp=sharing)
  
  ADE20K: 43.44%mIOU: [ Baidu drive](https://pan.baidu.com/s/1TKBFtCj6gwMq97NjYsmPOQ?pwd=a123), [google drive](https://drive.google.com/file/d/1hr9BlqgI4t4djibyj1fCW2LTMvlFFDWP/view?usp=sharing)
  
  BDD10K: 64.6%mIOU： [ Baidu drive](https://pan.baidu.com/s/13Hvi6he0hZgciff7tBUo0A?pwd=a123), [google drive]( https://drive.google.com/file/d/1IqMornjPHMVYHWdGhl-Jr1J4FvcZotoj/view?usp=sharing)
                      
  Camvid(pretrained on Cityscapes train set): 83.32%mIOU: [ Baidu drive](https://pan.baidu.com/s/1Q-e-_s-vsgn14S8GoBoTlA?pwd=a123), [google drive](https://drive.google.com/file/d/141889Jei9rcgJ9wSiFF8rNvUDUiV8SqI/view?usp=sharing)

Cityscapes : 82.0%mIOU:[google drive](https://drive.google.com/file/d/17SpPO3tsusB9JngjfnJFWzxffCoQRc9z/view?usp=sharing)
### DSNet:

DSNet_imagenet: [ Baidu drive](https://pan.baidu.com/s/1wMPH5ZNKwHIyFJ6Pp9-n2w?pwd=a123), [google drive](https://drive.google.com/file/d/1Cb3nd69IjQjjK_r8jXSMON4cHQ76MWbR/view?usp=sharing)
  
  ADE20k 40.0%mIOU: [ Baidu drive](https://pan.baidu.com/s/17CH66GTI2YEXMq7eXnK0xQ?pwd=a123), [google drive](https://drive.google.com/file/d/1J-qf5blQ71HGy4EStqMMg-NT1sO1CyUV/view?usp=sharing)
  
  BDD10K 62.8%mIOU: [ Baidu drive](https://pan.baidu.com/s/1tPQHC1LTE6tlueXvabU1-Q?pwd=a123), [google drive](https://drive.google.com/file/d/192T2dauq_cA1bBkmiRwYKWdYw27lxZIG/view?usp=sharing)

Cityscapes: 80.4%mIOU:[google drive](https://drive.google.com/file/d/1UmG7qaMz9BXR4hXxvIoJJ9k_N9JVuUQy/view?usp=sharing)

## Citation:

```
@article{guo2024dsnet,
  title={DSNet: A Novel Way to Use Atrous Convolutions in Semantic Segmentation},
  author={Guo, Zilu and Bian, Liuyang and Wei, Hu and Li, Jingyu and Ni, Huasheng and Huang, Xuan},
  journal={IEEE Transactions on Circuits and Systems for Video Technology},
  year={2024},
  publisher={IEEE}
}
```

DSNet

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README