viconex-med

KO | EN

gitlite — search

#python #llm #go #android #machine-learning #rust #deep-learning #docker #macos #nodejs #cli #pytorch #ios #open-source

(★ 5)

Code for the paper "ViConEx-Med: Visual Concept Explainability via Multi-Concept Token Transformer for Medical Image Analysis", 2025.

viconex-med 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# ViConEx-Med: Visual Concept Explainability via Multi-Concept Token Transformer for Medical Image Analysis

[![Cite](https://img.shields.io/badge/Cite-BibTeX-red)](https://raw.githubusercontent.com/CristianoPatricio/viconex-med/main/)
[![Paper](https://img.shields.io/badge/Paper-arXiv-blue)](https://arxiv.org/abs/2510.10174)

Official implementation of the paper **ViConEx-Med: Visual Concept Explainability via Multi-Concept Token Transformer for Medical Image Analysis**.

## Abstract
Concept-based models aim to explain model decisions with human-understandable concepts. However, most existing approaches treat concepts as numerical attributes, without providing complementary visual explanations that could localize the predicted concepts. This limits their utility in real-world applications and particularly in high-stakes scenarios, such as medical use-cases. This paper proposes ViConEx-Med, a novel transformer-based framework for visual concept explainability, which introduces multi-concept learnable tokens to jointly predict and localize visual concepts. By leveraging specialized attention layers for processing visual and text-based concept tokens, our method produces concept-level localization maps while maintaining high predictive accuracy. Experiments on both synthetic and real-world medical datasets demonstrate that ViConEx-Med outperforms prior concept-based models and achieves competitive performance with black-box models in terms of both concept detection and localization precision. Our results suggest a promising direction for building inherently interpretable models grounded in visual concepts.

## ⚡️ Highlights 🔥  
- We propose ViConEx-Med, a novel multi-concept token transformer framework that learns concept-specific localization maps directly from concept-to-patch attentions of multiple concept tokens;
- We introduce SynSkin, a large-scale synthetic image dataset with multilabel color annotations and pixel-level masks, specifically designed to facilitate robust evaluation of explainable models;
- We conduct extensive experiments on six medical imaging datasets, demonstrating that our framework jointly provides accurate concept predictions and faithful visual concept explanations, outperforming state-of-the-art baselines while achieving competitive performance with black-box models.

---

## 1. Download data
- **PH $^2$** dataset: https://www.fc.up.pt/addi/ph2%20database.html
- **Derm7pt** dataset: https://derm.cs.sfu.ca/Welcome.html
- **HAM10000** dataset: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/DBW86T
- **ISIC 2018** dataset: https://challenge.isic-archive.com/data/#2018 (**Task 2**)
- **DDR** dataset: https://github.com/nkicsl/DDR-dataset | See instructions at [Intructions for download and use of DDR dataset](assets/DDR_Download_Instructions.md)
- All metadata (train/val/test splits) are under `metadata/` folder. Please, modify the file paths in configuration files (`configs/datamodule/`) accordingly.

*Note*: To get all necessary data in a ready-to-use folder (~16 GB), please e-mail me at cristiano [dot] patricio [at] ubi [dot] pt.

1.1 After downloading data, the directory structure of `data` folder should look like this:
```bash
.
└── data/
    ├── PH2/
    │   ├── images/
    │   │   ├── IMD002.jpg
    │   │   └── ...
    |   ├── masks/
    │   │   ├── IMD002/
    |   │   │   ├── IMD002_color_blacks.png
    |   │   │   └── ...
    │   │   └── ...
    │   └── metadata/
    │       ├── PH2_train_split.csv
    │       └── ...
    ├── Derm7pt/
    │   ├── images/
    │   │   ├── Aal002.jpg
    │   │   └── ...
    |   ├── masks/
    │   │   ├── Aal002/
    |   │   │   ├── Aal002_color_blacks.png
    |   │   │   └── ...
    │   │   └── ...
    │   └── metadata/
    │       ├── derm7pt_train.csv.csv
    │       └── ...
    ├── HAM10000/
    │   ├── images/
    │   │   ├── ISIC_0024310.jpg
    │   │   └── ...
    |   ├── masks/
    │   │   ├── ISIC_0024310/
    |   │   │   ├── ISIC_0024310_color_blacks.png
    |   │   │   └── ...
    │   │   └── ...
    │   └── metadata/
    │       ├── HAM10000_train.csv
    │       └── ...
    └── ...
```

## 2. Installation

2.1 Clone this repository and navigate to the folder:
```bash
git clone https://github.com/CristianoPatricio/viconex-med.git
cd viconex-med/
```

2.2 Create a new conda environment from `viconex.yaml` file:

```bash
conda env create -f viconex.yaml
conda activate visxplain
```

## 3. Reproduce experiments

### 3.1 Run ViConEx-Med Baseline
```bash
./scripts/run_ViConEx_Med_Baseline.sh 0 # GPU ID: 0
```

### 3.2 Run ViConEx-Med Joint (Visual-Text Token-Fusion)
```bash
./scripts/run_ViConEx_Med_Joint.sh 0  # GPU ID: 0
```

### 3.3 Run ViConEx-Med Text-Guided Concepts
```bash
./scripts/run_ViConEx_Med_Text_Guided_Concepts.sh 0  # GPU ID: 0
```

### 3.4 Run ViConEx-Med Hybrid
```bash
./scripts/run_ViConEx_Med_Hybrid.sh 0  # GPU ID: 0
```

Attention maps are saved at: `logs/logs/runs/<LOG_NAME>/attention_maps/`

## Citation

If you use this repository, please cite:

```
@article{patricio2025viconexmed,
    title={ViConEx-Med: Visual Concept Explainability via Multi-Concept Token Transformer for Medical Image Analysis},
    author = {Cristiano Patrício and Luís F. Teixeira and João C. Neves},
    journal={arXiv preprint arXiv:2510.10174},
    year = {2025}
}
```

viconex-med

viconex-med 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

viconex-med

viconex-med 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README