MedViTV2

KO | EN

gitlite — search

#python #hacktoberfest #golang #android #machine-learning #rust #linux #macos #java #kubernetes #pytorch #mcp #ios #open-source

(★ 104)

MedViTV2: Medical Image Classification with KAN-Integrated Transformers and Dilated Neighborhood Attention (Applied Soft Computing 2025)

File Explorer

Download Latest Version (.zip)

- _init.py
- bloodmnist_MedViT_small.json
- breastmnist_MedViT_small.json
- chestmnist_MedViT_small.json
- dermamnist_MedViT_small.json
- octmnist_MedViT_small.json
- organamnist_MedViT_small.json
- organcmnist_MedViT_small.json
- organsmnist_MedViT_small.json
- pathmnist_MedViT_small.json
- pneumoniamnist_MedViT_small.json
- retinamnist_MedViT_small.json
- tissuemnist_MedViT_small.json
- ACC.png
- bloodmnist.png
- breastmnist.png
- chestmnist.png
- cover.jpg
- dermamnist.png
- init
- logo.jpg
- octmnist.png
- organamnist.png
- organcmnist.png
- organsmnist.png
- pathmnist.png
- pneumoniamnist.png
- retinamnist.png
- structure.png
- tissuemnist.png
- visualize.png
- GradCam_MedViT_large.py
- Evaluation.ipynb
- Visualization.ipynb
__init__.py
datasets.py
fasterkan.py
LICENSE
main.py
MedViT.py
README.md
requirements.txt

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

[![Paper](https://img.shields.io/badge/arXiv-Paper-<COLOR>.svg)](https://arxiv.org/abs/2502.13693)
[![Paper](https://img.shields.io/badge/Elsevier-ASOC-blue)](https://doi.org/10.1016/j.asoc.2025.114045)
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Omid-Nejati/MedViTV2/blob/main/Tutorials/Evaluation.ipynb)

<div align="center">
  <h1 style="font-family: Arial;">MedViT</h1>
  <h3>MedViTV2: Medical Image Classification with KAN-Integrated Transformers and Dilated Neighborhood Attention</h3>
</div>

## 🔥 News 
- **[2025.08.27]** We have released the pre-trained weights.
- **[2025.10.06]** Our paper accepted for publication in Applied Soft Computing.
## Train & Test --- Prepare data
To **train or evaluate** MedViT models on **17 medical datasets**, follow this ["Evaluation"](https://github.com/Omid-Nejati/MedViTV2/blob/main/Tutorials/Evaluation.ipynb).

**Important:** This code also supports training **all TIMM models**.
## Introduction
<div align="justify">
Convolutional networks, transformers, hybrid models, and Mamba-based architectures have shown strong performance in medical image classification but are typically designed for clean, labeled data. Real-world clinical datasets, however, often contain corruptions arising from multi-center studies and variations in imaging equipment. To address this, we introduce the Medical Vision Transformer (MedViTV2), the first architecture to integrate Kolmogorov–Arnold Network (KAN) layers into a transformer for generalized medical image classification. We design an efficient KAN block to lower computational cost while improving accuracy over the original MedViT. To overcome scaling fragility, we propose Dilated Neighborhood Attention (DiNA), an adaptation of fused dot-product attention that expands receptive fields and mitigates feature collapse. Additionally, a hierarchical hybrid strategy balances local and global feature perception through efficient stacking of Local and Global Feature Perception blocks. Evaluated on 17 classification and 12 corrupted datasets, MedViTV2 achieved state-of-the-art performance in 27 of 29 benchmarks, improving efficiency by 44% and boosting accuracy by 4.6% on MedMNIST, 5.8% on NonMNIST, and 13.4% on MedMNIST-C.
</div>

<div style="text-align: center">
<img src="https://github.com/Omid-Nejati/MedViT-V2/blob/main/Fig/ACC.png" title="MedViT-S" height="60%" width="60%">
</div>
Figure 1. Comparison between MedViTs (V1 and V2), MedMamba, and the baseline ResNets, in terms of Average
Accuracy vs. FLOPs trade-off over all MedMNIST datasets. MedViTV2-T/S/L significantly improves average accu-
racy by 2.6%, 2.5%, and 4.6%, respectively, compared to MedViTV1-T/S/L.</center>

## Overview

<div style="text-align: center">
<img src="https://github.com/Omid-Nejati/MedViT-V2/blob/main/Fig/structure.png" title="MedViT-S" height="75%" width="75%">
</div>
Figure 2. Overall architecture of the proposed Medical Vision Transformer (MedViTV2).</center>

## Visual Examples
You can find a tutorial for visualizing the Grad-CAM heatmap of MedViT in this repository ["visualize"](https://github.com/Omid-Nejati/MedViTV2/blob/main/Tutorials/Visualization.ipynb).
<br><br>
![MedViT-V](https://github.com/Omid-Nejati/MedViT-V2/blob/main/Fig/visualize.png)
<center>Figure 3. Grad-Cam heatmap visualization. We present heatmaps generated from the last three layers of MedViTV1-
T, MedViTV2-T, MedViTV1-L, and MedViTV2-L, respectively. Specifically, we utilize the final GFP, LGP, and
normalization layers in these models to produce the heatmaps using Grad-CAM.</center>

## Usage
First, clone the repository locally:
```
git clone https://github.com/whai362/PVT.git](https://github.com/Omid-Nejati/MedViTV2.git
cd MedViTV2
```
Install PyTorch 2.5
```
pip install torch==2.5.0 torchvision==0.20.0 torchaudio==2.5.0 --index-url https://download.pytorch.org/whl/cu124
```
Then, install natten 0.17.3
```
pip install natten==0.17.3+torch250cu124 -f https://shi-labs.com/natten/wheels/
```
Also, install requirements
```
pip install -r requirements.txt
```
## Training
To train MedViT-small on breastMNIST on a single gpu for 100 epochs run:
```
python main.py --model_name 'MedViT_small' --dataset 'breastmnist' --pretrained False
```

## 📊 Performance Overview
Below is the performance summary of MedViT on various medical imaging datasets.  
🔹 **Model weights are available now.**

| **Dataset** | **Task** | **MedViTV2-tiny (%)** |**MedViTV2-small (%)** |**MedViTV2-base (%)** |**MedViTV2-large (%)** |
|:-----------:|:--------:|:-----------------------:|:------------------:|:---------------------:|:-----------------------:|
| **[ChestMNIST](https://medmnist.com/)** | Multi-Class (14) | 96.3 ([model](https://drive.google.com/file/d/1j7HVQZ66_nbLuHepiYDfNJZURcJyyH66/view?usp=drive_link))| 96.4 ([model](https://drive.google.com/file/d/1TFtUYEqckZsa0oF7hlh5ixD9mLRXAEYv/view?usp=drive_link))| 96.4 ([model](https://drive.google.com/file/d/1biyIHSd6V4aSIz0rlhxj3OzQXql-8WzT/view?usp=drive_link))| 96.7 ([model](https://drive.google.com/file/d/1ecn2TZNGt0L3FXkaNgMqkFuU3bvh0YWr/view?usp=sharing))| 
| **[PathMNIST](https://medmnist.com/)** | Multi-Class (9) | 95.9 ([model](https://drive.google.com/file/d/1bD3fxpxzlbP3SuC8EDV3oZuBYhs5XpOl/view?usp=drive_link))| 96.5 ([model](https://drive.google.com/file/d/1Q09QdkwSLtb-au_3ip7vYzxo_VKvOyby/view?usp=drive_link))| 97.0 ([model](https://drive.google.com/file/d/1WazBW35P2sD3nnnSkuC9cjQAMH-PAICi/view?usp=drive_link))| 97.7 ([model](https://drive.google.com/file/d/1imY08j2tiBEsQAN_Du9-Ve1AdUeBmiVU/view?usp=sharing))| 
| **[DermaMNIST](https://medmnist.com/)** | Multi-Class (7) | 78.1 ([model](https://drive.google.com/file/d/1kFsxcB0L6S_WiEayDdyFoviXw8QInT71/view?usp=drive_link))| 79.2 ([model](https://drive.google.com/file/d/1b3MC7O-AeMaUeOwhcUE0NcKfInJeRMAL/view?usp=drive_link))| 80.8 ([model](https://drive.google.com/file/d/11nsf8LeQc_yZAwozXUul8jgYvLzKjeed/view?usp=drive_link))| 81.7 ([model](https://drive.google.com/file/d/1j-39VrzQII8Rgsi8htUNCrIiWJWx1LFM/view?usp=sharing))|
| **[OCTMNIST](https://medmnist.com/)** | Multi-Class (4) | 92.7 ([model](https://drive.google.com/file/d/1hNoTKG9R4QLqgeT77CVBM2WJ0Qgtgzfv/view?usp=drive_link))| 94.2 ([model](https://drive.google.com/file/d/1LyDzhY3dyutYp-Z1-uxCTpKEHwHTDPQ1/view?usp=drive_link))| 94.4 ([model](https://drive.google.com/file/d/1g7rStUsAqiXLKpt1QSULbzNSgG_madW0/view?usp=drive_link))| 95.2 ([model](https://drive.google.com/file/d/14pWK-8dXX9tw9LiCRuOlVJ_njg9JXrGR/view?usp=sharing))|
| **[PneumoniaMNIST](https://medmnist.com/)** | Multi-Class (2) | 95.1 ([model](https://drive.google.com/file/d/1EJmHGtmYqNhNlnEKpcKMUBieOaZbGFXg/view?usp=drive_link))| 96.5 ([model](https://drive.google.com/file/d/1z-NpuR-U4irhfHV5pF6w2Da5kvfdNsoE/view?usp=drive_link))| 96.9 ([model](https://drive.google.com/file/d/19IRSskO1TtVwCMzIs6NsKtVqec2aqEeb/view?usp=drive_link))| 97.3 ([model](https://drive.google.com/file/d/1oYzejaGw7UuYMXibzeP922kEgW3hio7A/view?usp=sharing))|
| **[RetinaMNIST](https://medmnist.com/)** | Multi-Class (5) | 54.7 ([model](https://drive.google.com/file/d/1rZd94-OoSZwJam8PcX53gNPPc-4FK4z-/view?usp=drive_link))| 56.2 ([model](https://drive.google.com/file/d/1gKl5LV05kUxJcR39NZckklBItND3l5tB/view?usp=drive_link))| 57.5 ([model](https://drive.google.com/file/d/1so0GEW1i6yUMc5kN5pHxqz-rf_7uagRU/view?usp=drive_link))| 57.8 ([model](https://drive.google.com/file/d/1bCwYBSsINop_JpZFDK29zsICJxRl3D0l/view?usp=sharing))|
| **[BreastMNIST](https://medmnist.com/)** | Multi-Class (2) | 88.2 ([model](https://drive.google.com/file/d/1vJAKCWTZIU3Q5gdY4Zu4k9RCzlwV1ZlY/view?usp=drive_link))| 89.5 ([model](https://drive.google.com/file/d/1VL8-ZJ1KhCZY0CELyfaP6gHZCfqRRhkf/view?usp=drive_link))| 90.4 ([model](https://drive.google.com/file/d/1T8ShD9Wcxo8tCxCKsySZ2g7XZnisIo7v/view?usp=sharing))| 91.0 ([model](https://drive.google.com/file/d/1_e11jiGdy03fDokWqIq-uLQePSLuX4RZ/view?usp=drive_link))|
| **[BloodMNIST](https://medmnist.com/)** | Multi-Class (8) | 97.9 ([model](https://drive.google.com/file/d/1v5-TyJTY14ZA4A5_3SYvfszSO_YGbBUQ/view?usp=drive_link))| 98.5 ([model](https://drive.google.com/file/d/1gn96VohPPlqsN_98ZNUwcFTNZ3wahBOt/view?usp=drive_link))| 98.5 ([model](https://drive.google.com/file/d/1gn96VohPPlqsN_98ZNUwcFTNZ3wahBOt/view?usp=sharing))| 98.7 ([model](https://drive.google.com/file/d/1NSNDOWuOOruGX3NSbEzh8N1_WqXjuwKj/view?usp=drive_link))|
| **[TissueMNIST](https://medmnist.com/)** | Multi-Class (8) | 69.9 ([model](https://drive.google.com/file/d/1n3hcdWLDU3v7YmenHXrFV91Qu6-lCAMO/view?usp=drive_link))| 70.5 ([model](https://drive.google.com/file/d/1xZ8w-ZSJnP0CLUkyPwz982Ua9LSE6Fv8/view?usp=drive_link))| 71.1 ([model](https://drive.google.com/file/d/1fdCAKKxVKFSyC6rgXdxy46J39CormgLM/view?usp=drive_link))| 71.6 ([model](https://drive.google.com/file/d/1Fgi-JSiyw6qKhI1HiJ_O8-tZACrXeH_C/view?usp=sharing))|
| **[OrganAMNIST](https://medmnist.com/)** | Multi-Class (11) | 95.8 ([model](https://drive.google.com/file/d/18lAPYy4RfwWSd3lpYhfKOSIcuAILRlOr/view?usp=drive_link))| 96.6 ([model](https://drive.google.com/file/d/1yOkNU3-WC1zBf_uPligWHmUpxUrsuZ1U/view?usp=drive_link))| 96.9 ([model](https://drive.google.com/file/d/1xFtcQMnkfgEmWtaWYNgqBFM5jvw9G3RU/view?usp=drive_link))| 97.3 ([model](https://drive.google.com/file/d/1D9XIKdJmbUbvzrKXCzXxF4nXK6RJuDfh/view?usp=sharing))|
| **[OrganCMNIST](https://medmnist.com/)** | Multi-Class (11) | 93.5 ([model](https://drive.google.com/file/d/1Rs_yH-iL2m7SXJ4X0Cyu22QQJCWp7N_a/view?usp=drive_link))| 95.0 ([model](https://drive.google.com/file/d/1d0nhYmsUVzMKul5F7pY2Tpu6eEsQd3g0/view?usp=drive_link))| 95.3 ([model](https://drive.google.com/file/d/1Qk43YyXdJrcO9OFJc1aY-E6VPefcoCih/view?usp=drive_link))| 96.1 ([model](https://drive.google.com/file/d/1jpPTbcy0ztZxo9XshfU_J0TiRV26RwFC/view?usp=sharing))|
| **[OrganSMNIST](https://medmnist.com/)** | Multi-Class (11) | 82.4 ([model](https://drive.google.com/file/d/17yvqiBt57QUQJNxpQSL-ddke-pBEytxo/view?usp=drive_link))| 83.9 ([model](https://drive.google.com/file/d/1bq_g2EstVUCo5Heb_ulRV4pL67WHIFJ1/view?usp=drive_link))| 84.4 ([model](https://drive.google.com/file/d/10NGAdpeo5hj2rqtpyAl80jGKqdmMshtv/view?usp=drive_link))| 85.1 ([model](https://drive.google.com/file/d/1kWdjz_WxCmfSM3uSIs40bYKtgt7jYyFF/view?usp=sharing))|
| **[PAD-UFES-20](https://data.mendeley.com/datasets/zr7vgbcyr2/1)** | Multi-Class (6) | 63.6 ([model](https://drive.google.com/file/d/1geL0CJoAUlR6smifzU-K9JOPUsJjrcOf/view?usp=drive_link))| |
| **[ISIC2018](https://challenge.isic-archive.com/data/)** | Multi-Class (7) | 77.1 ([model](https://drive.google.com/file/d/170V8I-Ghmvl8kj-BtGuTkC1u9LfTC6Nx/view?usp=drive_link))|
| **[CPN X-ray](https://data.mendeley.com/datasets/dvntn9yhd2/1)** | Multi-Class (3) | |  95.3 ([model](https://drive.google.com/file/d/1S3prrvtcBeNAeiGjtxlwsNYP45S4Y-_3/view?usp=drive_link))|
| **[Kvasir](https://datasets.simula.no/kvasir/)** | Multi-Class (8) |  |82.8 ([model](https://drive.google.com/file/d/1T5OSt9ngHLx3er3KPNGtKTiKsfk14tsD/view?usp=drive_link))| |
| **[Fetal-Planes-DB](https://zenodo.org/records/3904280)** | Multi-Class (6) | | |  95.3 ([model](https://drive.google.com/file/d/16bWPHWGQxvq_ynVYnRRfhANNMNlFx9O1/view?usp=drive_link))|

## License
MedViT is released under the [MIT License](LICENSE).

💖🌸 If you find my GitHub repository useful, please consider giving it a star!🌟

## References
* [FasterKAN](https://github.com/AthanasiosDelis/faster-kan)
* [Natten](https://github.com/SHI-Labs/NATTEN)
* [MedViTV1](https://github.com/Omid-Nejati/MedViT)
  
## Citation
```bibtex
@article{manzari2025medical,
  title={Medical image classification with KAN-integrated transformers and dilated neighborhood attention},
  author={Manzari, Omid Nejati and Asgariandehkordi, Hojat and Koleilat, Taha and Xiao, Yiming and Rivaz, Hassan},
  journal={Applied Soft Computing},
  pages={114045},
  year={2025},
  publisher={Elsevier}
}
@article{manzari2023medvit,
  title={MedViT: a robust vision transformer for generalized medical image classification},
  author={Manzari, Omid Nejati and Ahmadabadi, Hamid and Kashiani, Hossein and Shokouhi, Shahriar B and Ayatollahi, Ahmad},
  journal={Computers in Biology and Medicine},
  volume={157},
  pages={106791},
  year={2023},
  publisher={Elsevier}
}

```

MedViTV2

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

MedViTV2

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

Download README