CMCRL

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

★ 68 Open GitHub ↗

The official implementation of “Cross-Modal Causal Representation Learning for Radiology Report Generation” （IEEE T-IP 2025）

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# CMCRL

This is the implementation of [Cross-Modal Causal Intervention for Medical Report Generation](https://arxiv.org/pdf/2303.09117.pdf).
It contains the codes of the Radiological Cross-modal Alignment and Reconstruction Enhanced（RadCARE）, and fine-tuning via Visual-Linguistic Causal Intervention (VLCI) on IU-Xray/MIMIC-CXR dataset.

</div>

## Requirements
All the requirements are listed in the requirements.yaml file. Please use this command to create a new environment and activate it.

```
conda env create -f requirements.yaml
conda activate mrg
```

## Preparation
1. Datasets: 
You can download the dataset via `data/datadownloader.py`, or download from the repo of [R2Gen](https://github.com/cuhksz-nlp/R2Gen).
Then, unzip the files into `data/iu_xray` and `data/mimic_cxr`, respectively. 
2. Models: We provide the well-trained models of VLCI for inference, and you can download from [here](https://pan.baidu.com/s/1LKp_XLUVttAkgCHwm6QHpg?pwd=vsg1).
3. Please remember to change the path of data and models in the config file (`config/*.json`).

## Evaluation
- For VLCI on IU-Xray dataset

```
python main.py -c config/iu_xray/vlci.json
```

| Model             | B@1   | B@2   | B@3   | B@4   |C      |    R|   M  |
|:-----:            |:---:  |:---:  |:---:  |:---:  |:---:  |:---:|:---: |
| R2Gen             | 0.470 | 0.304 | 0.219 | 0.165 |/      |0.371|0.187 |
| CMCL              | 0.473 | 0.305 | 0.217 | 0.162 |/      |0.378|0.186 |
| PPKED             | 0.483 | 0.315 | 0.224 | 0.168 | 0.351 |0.376|0.190 |
| CA                | 0.492 | 0.314 | 0.222 | 0.169 |/      |0.381|0.193 |
| AlignTransformer  | 0.484 | 0.313 | 0.225 | 0.173 |/      |0.379|0.204 |
| M2TR              | 0.486 | 0.317 | 0.232 | 0.173 |/      |0.390|0.192 |
| MGSK              | 0.496 | 0.327 | 0.238 | 0.178 |0.382  |0.381|/     |
| RAMT              | 0.482 | 0.310 | 0.221 | 0.165 |/      |0.377|0.195 |
| MMTN              | 0.486 | 0.321 | 0.232 | 0.175 |0.361  |0.375|/     |
| DCL               | /     | /     | /     | 0.163 |**0.586**  |0.383|0.193 |
| CMCRL              | **0.505** | **0.334** | **0.245** | **0.189** |0.456  |**0.397**|**0.204** |

</div>

- For VLCI on MIMIC-CXR dataset

```
python main.py -c config/mimic_cxr/vlci.json
```

| Model             | B@1   | B@2   | B@3   | B@4   |C    |    R|   M  | CE-P  | CE-R  | CE-F1 |
|:-----:            |:---:  |:---:  |:---:  |:---:  |:---:|:---:|:---: |:---:  |:---:  |:---:  |
| R2Gen             | 0.353 | 0.218 | 0.145 | 0.103 |/    |0.277|0.142 | 0.333 | 0.273 | 0.276 |
| CMCL              | 0.334 | 0.217 | 0.140 | 0.097 |/    |0.281|0.133 | /     | /     | /     |
| PPKED             | 0.360 | 0.224 | 0.149 | 0.106 |0.237|**0.284**|0.149 | /     | /     | /     |
| CA                | 0.350 | 0.219 | 0.152 | 0.109 |/    |0.283|0.151 | 0.352 | 0.298 | 0.303 |
| AlignTransformer  | 0.378 | 0.235 | 0.156 | 0.112 |/    |0.283|0.158 | /     | /     | /     |
| M2TR              | 0.378 | 0.232 | 0.154 | 0.107 |/    |0.272|0.145 | 0.240 | 0.428 | 0.308 |
| MGSK              | 0.363 | 0.228 | 0.156 | 0.115 |0.203|**0.284**|/     | 0.458 | 0.348 | 0.371 |
| RAMT              | 0.362 | 0.229 | 0.157 | 0.113 |/    |**0.284**|0.153 | 0.380 | 0.342 | 0.335 |
| MMTN              | 0.379 | 0.238 | 0.159 | 0.116 |/    |0.283|**0.161** | /     | /     | /     |
| DCL               | /     | /     | /     | 0.109 |**0.281**|**0.284**|0.150 | 0.471 | 0.352 |0.373  |
| CMCRL              | **0.400** | **0.245** | **0.165** | **0.119** | 0.190 | 0.280   |  0.150  | **0.489** | **0.340** | **0.401** |

</div>

## Citation
If you use this code for your research, please cite our paper.

```
@ARTICLE{11005686,
  author={Chen, Weixing and Liu, Yang and Wang, Ce and Zhu, Jiarui and Li, Guanbin and Liu, Cheng-Lin and Lin, Liang},
  journal={IEEE Transactions on Image Processing}, 
  title={Cross-Modal Causal Representation Learning for Radiology Report Generation}, 
  year={2025},
  volume={34},
  pages={2970-2985},
  doi={10.1109/TIP.2025.3568746}}
```

## Contact
If you have any questions about this code, feel free to reach me (chenwx228@mail2.sysu.edu.cn)

## Acknowledges
We thank [R2Gen](https://github.com/cuhksz-nlp/R2Gen) for their open source works.

CMCRL

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README