SAM-Swin

KO | EN

gitlite — search

#python #nextjs #react #api #typescript #python3 #nodejs #flask #vue #docker #reactjs #javascript

★ 32 Open GitHub ↗

This is an official implementation for "SAM-Swin: SAM-Driven Dual-Swin Transformers with Adaptive Lesion Enhancement for Laryngo-Pharyngeal Tumor Detection" - Medical Image Analysis (Major Revision)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# SAM-Swin: SAM-Driven Dual-Swin Transformers with Adaptive Lesion Enhancement for Laryngo-Pharyngeal Tumor Detection

This repo is the official implementation of [SAM-Swin: SAM-Driven Dual-Swin Transformers with Adaptive Lesion Enhancement for Laryngo-Pharyngeal Tumor Detection](https://arxiv.org/abs/2410.21813) (MIA Major Revision).

## Introduction

The SAM-Swin mainly consists of four pivotal components: SAM2-guided lesion location (**SAM2-GLLM**), whole image branch (**WIB**), lesion region branch (**LRB**), and multi-scale lesion-aware enhancement module (**MS-LEAM**).

- By leveraging the advanced object segmentation capabilities of the **SAM2**, we pioneerly integrate SAM2 into the SAM-Swin framework, enabling SAM-Swin to achieve highly precise segmentation of the lesion region.
- We propose MS-LAEM designed to adaptively enhance the learning of nuanced **complementary features** across various scales, improving the quality of feature extraction and representation.
- We introduce the multi-scale CAG loss, a novel approach that employs targeted supervision to facilitate the extraction of **class-specific features** within the model.

![architecture](https://raw.githubusercontent.com/VVJia/SAM-Swin/master/architecture.png)

## Fine-tune SAM2

To fine-tune SAM2 tailored for your tasks, we recommend following the guidelines provided in the original repository: [MedSAM2](https://github.com/bowang-lab/MedSAM/tree/MedSAM2)

## Dataset

Organize your datasets in the following manner:

```markdown
datasets/
├── dataset1/
│   ├── global/
│   │   ├── train/
│   │   │   ├── benign/
│   │   │   ├── normal/
│   │   │   └── tumor/
│   │   ├── val/
│   │   │   ├── benign/
│   │   │   ├── normal/
│   │   │   └── tumor/
│   │   └── test/
│   │       ├── benign/
│   │       ├── normal/
│   │       └── tumor/
│   └── local_seg/
│       ├── train/
│       │   ├── benign/
│       │   ├── normal/
│       │   └── tumor/
│       ├── val/
│       │   ├── benign/
│       │   ├── normal/
│       │   └── tumor/
│       └── test/
│           ├── benign/
│           ├── normal/
│           └── tumor/
├── dataset6/
│   └── ...
```

## Training

We train the SAM-Swin in two stages.

1. Stage 1, run:

```bash
   python -m torch.distributed.launch --nproc_per_node 2 --master_port 12345 main.py --cfg configs/dynamic.yaml --batch-size 32 --pretrained swinv2_base_patch4_window16_256.pth --cache-mode full --amp-opt-level O1 --accumulation-steps 4 --fused_window_process --fused_layernorm --tag exp
   ```

2. Stage 2, run:

```bash
   python -m torch.distributed.launch --nproc_per_node 2 --master_port 12345 main.py --cfg configs/ft_baseline.yaml --batch-size 32 --pretrained <Your path of latest checkpoint at the Stage 1> --cache-mode full --amp-opt-level O1 --accumulation-steps 4 --fused_window_process --fused_layernorm --tag exp_ft
   ```

## Testing

Using DDP, Run:

```
python -m torch.distributed.launch --nproc_per_node 2 --master_port 12345 main.py --cfg configs/dynamic.yaml --resume <Your path of the checkpoint> --cache-mode full --amp-opt-level O1 --accumulation-steps 4 --fused_window_process --fused_layernorm --tag exp --eval
```

## Acknowledgement

The code of SAM-Swin is built upon [MedSAM2](https://github.com/bowang-lab/MedSAM/tree/MedSAM2) and [Swin Transformer](https://github.com/microsoft/Swin-Transformer?tab=readme-ov-file), and we express our gratitude to these awesome projects.

## Citing SAM-Swin

```
@article{wei2024sam,
  title={Sam-swin: Sam-driven dual-swin transformers with adaptive lesion enhancement for laryngo-pharyngeal tumor detection},
  author={Wei, Jia and Li, Yun and Fan, Xiaomao and Ma, Wenjun and Qiu, Meiyu and Chen, Hongyu and Lei, Wenbin},
  journal={arXiv preprint arXiv:2410.21813},
  year={2024}
}
```

SAM-Swin

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README