KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #dsh-plugin #developer-tools #ai-agent #claude-code #codex #typescript #ai-agents #deepseek-harness #mcp #cli #rust

Delphi

★ 510 Open GitHub ↗

Modelling human health trajectories using generative transformers

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

## Learning the natural history of human disease with generative transformers

[[`Paper`](https://doi.org/10.1038/s41586-025-09529-3)] [[`BibTeX`](#Citation)]

Artem Shmatko*, Alexander Wolfgang Jung*, Kumar Gaurav*, Søren Brunak, Laust Mortensen, Ewan Birney, Tom Fitzgerald, Moritz Gerstung (*Equal Contribution)

[![Code License: MIT](https://img.shields.io/badge/Code%20License-MIT-blue)](https://opensource.org/license/mit)
[![Weights License: CC BY-NC-ND 4.0](https://img.shields.io/badge/Weights%20License-CC%20BY--NC--ND%204.0-lightgrey)](https://creativecommons.org/licenses/by-nc-nd/4.0/)

## Repository Overview

This repository contains the code for Delphi, the modified GPT-2 model used in the paper "Learning the natural history of human disease with generative transformers", along with the training code and analysis notebooks.

The implementation is based on Andrej Karpathy's [nanoGPT](https://github.com/karpathy/nanoGPT).

## Installation

1. Download the repository:

```bash
git clone https://github.com/gerstung-lab/Delphi.git
cd Delphi
```

2. Create a virtual conda environment and install the requirements:
```bash
conda create -n delphi python=3.11
conda activate delphi
pip install -r requirements.txt
```

Installing the requirements normally takes a few minutes.

## Data availability

Delphi-2M is trained on 400K patient health trajectories from the UK Biobank data, which is available to researchers upon [application](https://www.ukbiobank.ac.uk/). In addition, we provide a minimal synthetic dataset that was generated by the trained model itself. The synthetic data is broadly statistically similar to the real data, while not disclosing any patient information; however this data and model is for coding testing purposes only and we absolutely do not recommend using this model for any type of research - instead, we recommend applying for the full model from the UK BioBank.

## Data preparation

To reproduce Delphi's training data on the UK Biobank Research Analysis Platform (RAP), follow the end-to-end guide in [`supplementary/ukb_rap_training.md`](supplementary/ukb_rap_training.md), which runs `data/ukb_real_data/example_ukb_rap_convert_new.ipynb` (total cost under £10).

Alternatively, to convert a raw UK Biobank ".tab" basket download locally, refer to the `data/ukb_simulated_data/example_ukb_to_bin.ipynb` file.

If you are interested in using your own data, please refer to the `data/README.md` file for the details on how to transform it into a Delphi-compatible format.

The synthetic dataset that is available in the `data/ukb_simulated_data/` is already in the desired format and can be used for the model training demo.

## Training

To train the model, run:

```bash
python train.py config/train_delphi_demo.py --device=cuda --out_dir=Delphi-2M
```

If you want to train the model on a CPU, remove the `--device=cuda` argument.
For more information on the training configuration, check the `config/train_delphi_demo.py` file.

Training a demo model takes around 10 minutes on a single GPU.

Training the original model took 1 GPU-hour (NVIDIA V100, CentOS 7). Training on M1 Macbook Pro's MPS takes around 10 hours.

## Docker

We provide a Dockerfile to run the training and downstream analyses. Please refer to the `containers/Dockerfile` for the details.

## Notebooks

After training the model, the obtained checkpoints can be used to run the analysis notebooks.
Please note that in order to reproduce the results from the paper, it is necessary to train the model on the UK Biobank data.

There are the following notebooks in the root folder:

`evaluate_delphi.ipynb`: The notebook is used to evaluate Delphi-2M in terms of prediction accuracy compared to age-sex-based epidemiological baseline and calibration of the predicted risks. Furthermore, it explores the attention mechanism of the model and the structure of the latent space using UMAP of the learned disease embeddings.

`shap_analysis.ipynb`: The notebook is used to analyse the SHAP values of the model to understand the importance of the input disease events for the model's predictions for a given patient. Then, using SHAP values aggregated over the entire dataset, we identify the diseases that are most important for each possible disease in the future.

`sampling_trajectories.ipynb`: The notebook is used to analyse the synthetic data generated by Delphi-2M. It compares the synthetic data to the real data in terms of the distribution of disease events, ages and disease rates.

`data/ukb_simulated_data/example_ukb_to_bin.ipynb`: The notebook is used to convert the raw UK Biobank data into the format required for training.

## License

- **Code** (this repository): [MIT](LICENSE).
- **Trained Delphi-2M weights** (distributed separately via UK Biobank): [CC BY-NC-ND 4.0](https://creativecommons.org/licenses/by-nc-nd/4.0/) — non-commercial use only.

The model and weights are provided for research purposes only and must not be used for clinical decision-making.

For any commercial usage, please contact [DKFZ Innovation Management](mailto:innovation@dkfz-heidelberg.de).

A patent has been filed for the use of generative transformer architectures to model competing risk and timings of diseases (application number: PCT/EP2025/065771; applicants: DKFZ, EMBL), with Moritz Gerstung, Artem Shmatko, Tom Fitzgerald, Ewan Birney, Kumar Gaurav and Alex W Jung listed as inventors.

## Citation

```bibtex
@article{shmatko2025,
  title = {Learning the natural history of human disease with generative transformers},
  author = {Artem Shmatko and Alexander Wolfgang Jung and Kumar Gaurav and Søren Brunak and Laust Hvas Mortensen and Ewan Birney and Tom Fitzgerald and Moritz Gerstung},
  journal = {Nature},
  year = {2025},
  isbn = {1476-4687},
  doi = {10.1038/s41586-025-09529-3},
  url = {https://doi.org/10.1038/s41586-025-09529-3}
}
```

Delphi

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README