DiFusionSeg

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

(★ 16)

[KBS 2025] This is official Pytorch implementation of "DiFusionSeg: Diffusion-Driven Semantic Segmentation with Multi-modal Fusion for Perception Optimization"

DiFusionSeg 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# DiFusionSeg
This is official Pytorch implementation of "[DiFusionSeg: Diffusion-Driven Semantic Segmentation with Multi-modal Fusion for Perception Optimization](https://www.sciencedirect.com/science/article/pii/S0950705125015205)"
 - 
```
@article{Wang2025DiFusionSegDS,
  title={DiFusionSeg: Diffusion-Driven Semantic Segmentation with Multi-Modal image Fusion for Enhanced Perception},
  author={Zhiwei Wang and Defeng He and Li Zhao and Bo Liu and Yayu Zheng and Xiaoqin Zhang},
  journal={Knowledge-Based Systems},
  year={2025},
  url={https://api.semanticscholar.org/CorpusID:281557716}
}
```
## Framework
![image](https://raw.githubusercontent.com/warren-wzw/DiFusionSeg/master/images/model_arch.png)

## Recommended Environment
 - [ ] python  3.9.1
 - [ ] torch  1.13.1
 - [ ] cudatoolkit 11.8
 - [ ] torchvision 0.14.0
 - [ ] mmcv  2.2.1
 - [ ] mmcv-full 1.7.2
 - [ ] mmsegmentation 0.30.0
 - [ ] numpy  1.26.4
 - [ ] opencv-python 4.10.0.84

## Experiments 
### Dataset & Checkpoints & Results
The checkpoints and results can be in [DiFusionSeg](https://www.dropbox.com/scl/fo/zjbyp7pml54epiz8wg4gj/AIGFfGfG8Ea_XU25WwyxQno?rlkey=47uzbn6jxn27o4bjvdvo7yk47&st=3gcxs94v&dl=0). Download MSRS dataset from [MSRS](https://pan.baidu.com/s/18q_3IEHKZ48YBy2PzsOtRQ?pwd=MSRS) and the MFNet dataset from [MFNet](https://www.mi.t.u-tokyo.ac.jp/static/projects/mil_multispectral/).
If you need to evaluate other datasets, please organize them as follows:
```
├── /dataset
    MSRS/
    ├── test
    │   ├── ir
    │   ├── Segmentation_labels
    │   ├── Segmentation_visualize
    │   └── vi
    └── train
        ├── ir
        ├── Segmentation_labels
        └── vi
    MFD/
    ├── test
    │   ├── ir
    │   ├── Segmentation_labels
    │   ├── Segmentation_visualize
    │   └── vi
    ├── test_day
    │   ├── ir
    │   ├── Segmentation_labels
    │   └── vi
    ├── test_night
    │   ├── ir
    │   ├── Segmentation_labels
    │   └── vi
    └── train
        ├── ir
        ├── Segmentation_labels
        └── vi
    ......
```
### Evaluate model
python
```
python test_model.py
```
### run sample
python
```
python test_demo.py --img="./images/00131D_vi.png" --ir="./images/00131D_ir.png" --checkpoint="./exps/Done/msrs_vi_ir_meanstd_ConvNext_fusioncomplex_8083/best.pth" --segout="./seg.png"
```
### To Train
Before training DiFusionSeg, you need to download the MSRS dataset MSRS and putting it in ./datasets.

Then running 
python
```
python train_model.py
```
### Segmentation comparison
![image](https://raw.githubusercontent.com/warren-wzw/DiFusionSeg/master/images/seg.png)
![image](https://raw.githubusercontent.com/warren-wzw/DiFusionSeg/master/images/MsrsSegTable.png)
![image](https://raw.githubusercontent.com/warren-wzw/DiFusionSeg/master/images/MfdSegResult.png)
![image](https://raw.githubusercontent.com/warren-wzw/DiFusionSeg/master/images/MfdSegTable.png)
![image](https://raw.githubusercontent.com/warren-wzw/DiFusionSeg/master/images/FmbSegResult.png)
![image](https://raw.githubusercontent.com/warren-wzw/DiFusionSeg/master/images/FmbSegTable.png)
### Fusion comparison
![image](https://raw.githubusercontent.com/warren-wzw/DiFusionSeg/master/images/fusion.png)
## If this work is helpful to you, please cite it as：
```
@article{Wang2025DiFusionSegDS,
  title={DiFusionSeg: Diffusion-Driven Semantic Segmentation with Multi-Modal image Fusion for Enhanced Perception},
  author={Zhiwei Wang and Defeng He and Li Zhao and Bo Liu and Yayu Zheng and Xiaoqin Zhang},
  journal={Knowledge-Based Systems},
  year={2025},
  url={https://api.semanticscholar.org/CorpusID:281557716}
}
```
## Acknowledgements
We use MMSegmentation, DDP, Segformer as the correspond codebase. We would like to express our sincere gratitude to the developers of these codebases.

DiFusionSeg

DiFusionSeg 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

DiFusionSeg

DiFusionSeg 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README