atombench

KO | EN

gitlite — search

#python #hacktoberfest #go #machine-learning #linux #docker #windows #awesome #cli #awesome-list #pytorch #security #ios #open-source

(★ 10)

AtomBench: A Benchmark for Generative Atomic Structure Models using GPT, Diffusion, and Flow Architectures https://arxiv.org/abs/2510.16165

atombench 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

**A Python package for benchmarking generative crystal reconstruction models**

[![arXiv](https://img.shields.io/badge/arXiv-2510.16165-FF5050.svg?logo=arxiv&logoColor=white)](https://arxiv.org/abs/2510.16165)
[![License: MIT](https://img.shields.io/badge/License-MIT-FF9999.svg)](https://raw.githubusercontent.com/atomgptlab/atombench/main/LICENSE)
[![Python](https://img.shields.io/badge/python-%E2%89%A53.9-FF5050.svg)](https://raw.githubusercontent.com/atomgptlab/atombench/main/pyproject.toml)
[![Open in Colab](https://img.shields.io/badge/Colab-tutorial-FF9999.svg)](https://github.com/crhysc/jarvis-tools-notebooks/blob/master/atombench_example.ipynb)
[![Documentation](https://img.shields.io/badge/docs-latest-FF5050.svg)](https://atomgptlab.github.io/atombench/)

</div>

**AtomBench is a Python package that automates the statistical analysis of the reconstruction performance of generative inverse materials design models.** Point it at a model's predicted structures and it scores how faithfully they reconstruct the targets. Its main use is running several models in one go, where it overlays them in shared figures and gathers their metrics into a single table.

We also used AtomBench to run our own study, benchmarking four models (AtomGPT, CDVAE, FlowMM, and MatterGen) on the JARVIS Supercon-3D and Alexandria DS-A/B superconductivity datasets. Those benchmarks are fully reproducible through the Snakemake pipeline in this repository.

**Documentation:** <https://atomgptlab.github.io/atombench/>

The repository has two independent parts: the `atombench` package, which runs anywhere with Python 3.9+, and the Snakemake pipeline that reproduces our study, which needs a Linux HPC cluster.

## AtomBench Python Package

Install the package:

```bash
pip install atombench
```

Run it on a directory of benchmark CSVs, one per model, which it overlays in shared figures and a metrics table. A single CSV also works.

```
benchmarks/
├── atomgpt.csv
├── cdvae.csv
├── flowmm.csv
└── mattergen.csv
```

```bash
atombench benchmarks/ out/
```

`out/` then holds two folders:

```
out/
├── figures/                  # plots (PNG), all models overlaid
└── numerical_calculations/   # metrics_table.{json,tex}, epic_metrics.csv
```

Each input CSV needs three columns: `id` (a unique identifier), `target` (the ground-truth structure as a POSCAR string), and `prediction` (the model's structure as a POSCAR string). A `metrics.json` is cached next to each CSV for later runs.

See the [documentation](https://atomgptlab.github.io/atombench/tutorial.html) for metric definitions, the full list of options, and the Python API.

## Submit to the JARVIS-Leaderboard

`atombench-submit` turns the same CSV into a valid `AI / AtomGen` contribution and opens a pull request.

```bash
pip install 'atombench[submit]'
atombench-submit predictions.csv --dataset dft_3d --prop Tc_supercon \
  --model-name MyModel --author-email me@example.com \
  --project-url https://example.com/paper --git-url https://github.com/me/mymodel
```

See the [documentation](https://atomgptlab.github.io/atombench/) for the GitHub token, new benchmarks, and the Python API.

## Reproducing Our Benchmarks

The Snakemake pipeline reproduces our study, training and evaluating all four models on the JARVIS Supercon-3D and Alexandria DS-A/B datasets. It needs a Linux HPC cluster (SLURM + CUDA 11.8); run `bash depcheck.sh` first.

The full guide lives in the [documentation](https://atomgptlab.github.io/atombench/reproducing.html), with a [walkthrough in Google Colab](https://github.com/crhysc/jarvis-tools-notebooks/blob/master/atombench_example.ipynb).

## Tutorials

Per-model notebooks: [AtomGPT](https://github.com/knc6/jarvis-tools-notebooks/blob/master/jarvis-tools-notebooks/atomgpt_example.ipynb), [CDVAE](https://github.com/crhysc/jarvis-tools-notebooks/blob/master/jarvis-tools-notebooks/cdvae_example.ipynb), and [FlowMM](https://github.com/crhysc/jarvis-tools-notebooks/blob/master/jarvis-tools-notebooks/flowmm_example.ipynb).

## Citation

If you use AtomBench, please cite:

```bibtex
@article{campbell2026atombench,
  title   = {AtomBench: A Benchmarking Framework for Generative Crystal Reconstruction Models in Conventional Superconductors},
  author  = {Campbell, Charles Rhys and Romero, Aldo H. and Choudhary, Kamal},
  year    = {2026},
}
```

## License

Released under the [MIT License](LICENSE).

atombench

atombench 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

atombench

atombench 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README