GFMPapers

KO | EN

gitlite — search

#python #rust #css #html #llm #python3 #javascript #reactjs #mongodb #machine-learning #kotlin #ai

★ 376 Open GitHub ↗

Must-read papers on graph foundation models (GFMs)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<h2 align="center"><b>GFMPapers: Must-read papers on graph foundation models (GFMs)</b></h2>

<a href="https://github.com/BUPT-GAMMA/GFMPapers/"><img src="https://awesome.re/badge.svg" alt="awesome"></a> <a href="https://github.com/BUPT-GAMMA/GFMPapers/pulls"><img src="https://img.shields.io/badge/PRs-Welcome-green" alt="PRs"></a>
     ![](https://img.shields.io/github/last-commit/BUPT-GAMMA/GFMpapers?color=blue)

This list is currently maintained by members in BUPT GAMMA Lab. If you like our project, please give us a star ⭐ on GitHub for the latest update.

We thank all the great [contributors](#contributors) very much.

⭐ We have held a tutorial about [graph foundation model](https://arxiv.org/pdf/2310.11829.pdf) at the [WebConf 2024](https://www2024.thewebconf.org/)! Here is the tutorial. [[part1](http://www.shichuan.org/doc/WWW24Tutorial_GFM_Part1.pdf)][[part2](http://www.shichuan.org/doc/WWW24Tutorial_GFM_Part2.pdf)][[part3](http://www.shichuan.org/doc/WWW24Tutorial_GFM_Part3.pdf)]

### Contents

- [Contents](#contents)
- [Keywords Convention](#keywords-convention)
- [0. Survey Papers](#0-survey-papers)
- [1. GNN-based Papers](#1-gnn-based-papers)
- [2. LLM-based Papers](#2-llm-based-papers)
- [3. GNN+LLM-based Papers](#3-gnnllm-based-papers)
- [4. Benchmarks and Datasets](#4-benchmarks-and-datasets)
- [5. Other papers](#5-other-papers)
- [Contributors](#contributors)

### Keywords Convention

![](https://img.shields.io/badge/xxx-DCE7F1) backbone architecture

![](https://img.shields.io/badge/yyy-EAD8D9) Pretraining

![](https://img.shields.io/badge/zzz-D8D0E1) Adaptation

The meaning of each tag can be referred to in the "Towards Graph Foundation Models: A Survey and Beyond" paper.

## 0. Survey Papers
1. [TPAMI 2025] **Graph Foundation Models: Concepts, Opportunities and Challenges**. [[pdf](https://ieeexplore.ieee.org/abstract/document/10915556)]
2. [TKDE 2024] **Large Language Models on Graphs: A Comprehensive Survey**. [[pdf](https://arxiv.org/pdf/2312.02783.pdf)][[paperlist](https://github.com/PeterGriffinJin/Awesome-Language-Model-on-Graphs)]
3. [KDD 2025] **Graph Foundation Models: Challenges, Methods, and Open Questions**. [[pdf](https://dl.acm.org/doi/pdf/10.1145/3711896.3736568)]
4. [KDD 2024] **A Survey of Large Language Models for Graphs**. [[pdf](https://arxiv.org/pdf/2405.08011.pdf)]
5. [KDD 2024] **Graph Intelligence with Large Language Models and Prompt Learning**. [[pdf](https://dl.acm.org/doi/pdf/10.1145/3637528.3671456)]
6. [arXiv 2023.8] **Graph Meets LLMs: Towards Large Graph Models**. [[pdf](https://arxiv.org/pdf/2308.14522.pdf)][[paperlist](https://github.com/THUMNLab/awesome-large-graph-model)]
7. [IEEE Intelligent Systems] **Integrating Graphs with Large Language Models: Methods and Prospects**. [[pdf](https://arxiv.org/pdf/2310.05499.pdf)]
8. [arXiv 2023.10] **Towards Graph Foundation Models: A Survey and Beyond**. [[pdf](https://arxiv.org/pdf/2310.11829.pdf)][[paperlist](https://github.com/BUPT-GAMMA/GFMPapers)]
9. [arXiv 2023.11] **A Survey of Graph Meets Large Language Model: Progress and Future Directions**. [[pdf](https://arxiv.org/pdf/2311.12399.pdf)][[paperlist](https://github.com/yhLeeee/Awesome-LLMs-in-Graph-tasks)]
10. [arXiv 2024.2] **Graph Foundation Models**. [[pdf](https://arxiv.org/pdf/2402.02216.pdf)][[paperlist](https://github.com/CurryTang/Towards-Graph-Foundation-Models-New-perspective-)][[paperlist2](https://github.com/CurryTang/Towards-graph-foundation-models)]
11. [arXiv 2024.2] **Advancing Graph Representation Learning with Large Language Models: A Comprehensive Survey of Techniques**. [[pdf](https://arxiv.org/pdf/2402.05952.pdf)]
12. [arXiv 2024.2] **Towards Versatile Graph Learning Approach: from the Perspective of Large Language Models**. [[pdf](https://arxiv.org/pdf/2402.11641.pdf)]
13. [arXiv 2024.3] **A Survey on Self-Supervised Pre-Training of Graph Foundation Models: A Knowledge-Based Perspective**. [[pdf](https://arxiv.org/pdf/2403.16137.pdf)][[paperlist](https://github.com/Newiz430/Pretext)]
14. [arXiv 2024.4] **A Survey of Large Language Models on Generative Graph Analytics: Query, Learning, and Applications**. [[pdf](https://arxiv.org/pdf/2404.14809.pdf)]
15. [TIST 2024] **Graph Machine Learning in the Era of Large Language Models (LLMs)**. [[pdf](https://arxiv.org/pdf/2404.14928.pdf)]
16. [arXiv 2024.12] **Graph Learning in the Era of LLMs: A Survey from the Perspective of Data, Models, and Tasks**. [[pdf](https://arxiv.org/pdf/2412.12456.pdf)]
17. [arXiv 2025.3] **Towards Graph Foundation Models: A Transferability Perspective**. [[pdf](https://arxiv.org/pdf/2503.09363.pdf)]
18. [arXiv 2025.5] **Graph Foundation Models: A Comprehensive Survey**. [[pdf](https://www.arxiv.org/pdf/2505.15116.pdf)]
19. [arXiv 2025.5] **Using Large Language Models to Tackle Fundamental Challenges in Graph Learning: A Comprehensive Survey**. [[pdf](https://arxiv.org/pdf/2505.18475.pdf)]
20. [arXiv 2025.6] **Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunities**. [[pdf](https://arxiv.org/pdf/2506.18019.pdf)][[paperlist](https://github.com/YuanchenBei/Awesome-Graphs-Meet-Agents)]

## 1. GNN-based Papers
1. [arXiv 2023.10] **Enhancing Graph Neural Networks with Structure-Based Prompt** [[pdf](https://arxiv.org/pdf/2310.17394.pdf)] ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
1. [arXiv 2023.11] **MultiGPrompt for Multi-Task Pre-Training and Prompting on Graphs** [[pdf](https://arxiv.org/pdf/2312.03731.pdf)] ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
1. [arXiv 2023.10] **HetGPT: Harnessing the Power of Prompt Tuning in Pre-Trained Heterogeneous Graph Neural Networks** [[pdf](https://arxiv.org/abs/2310.15318)] ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
1. [arXiv 2023.10] **Prompt Tuning for Multi-View Graph Contrastive Learning** [[pdf](https://arxiv.org/abs/2310.10362)] ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
1. [arXiv 2023.05] **PRODIGY: Enabling In-context Learning Over Graphs**. [[pdf](https://arxiv.org/pdf/2305.12600.pdf)] ![](https://img.shields.io/badge/GCN/GAT-DCE7F1) ![](https://img.shields.io/badge/Graph_Reconstruction/Supervised-EAD8D9)  ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
23. [arXiv 2023.05] **G-Adapter: Towards Structure-Aware Parameter-Efficient Transfer Learning for Graph Transformer Networks.** [[pdf](https://arxiv.org/pdf/2305.10329.pdf)] ![](https://img.shields.io/badge/Graph_Transformer-DCE7F1) ![](https://img.shields.io/badge/Supervised/Graph_Reconstruction/Property_Prediction-EAD8D9) ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
17. [arXiv 2023.04] **AdapterGNN: Efficient Delta Tuning Improves Generalization Ability in Graph Neural Networks.** [[pdf](https://arxiv.org/pdf/2304.09595.pdf)] ![](https://img.shields.io/badge/GIN-DCE7F1) ![](https://img.shields.io/badge/Cross--Scale_CL/Same--Scale_CL/Graph_Reconstruction-EAD8D9) ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
20. [arXiv 2023.02] **SGL-PT: A Strong Graph Learner with Graph Prompt Tuning.** [[pdf](https://arxiv.org/pdf/2302.12449.pdf)] ![](https://img.shields.io/badge/GIN-DCE7F1) ![](https://img.shields.io/badge/Same--Scale_CL/Graph_Reconstruction-EAD8D9) ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
1. [KDD 2023] **All in One: Multi-Task Prompting for Graph Neural Networks**. [[pdf](https://www.researchgate.net/profile/Jia-Li-127/publication/371608827_All_in_One_Multi-Task_Prompting_for_Graph_Neural_Networks/links/648c2270c41fb852dd0a4f62/All-in-One-Multi-Task-Prompting-for-Graph-Neural-Networks.pdf)] ![](https://img.shields.io/badge/GCN/GAT/Graph_Transformer-DCE7F1) ![](https://img.shields.io/badge/Same--Scale_CL-EAD8D9)  ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
18. [KDD 2023] **A Data-centric Framework to Endow Graph Neural Networks with Out-Of-Distribution Detection Ability.** [[pdf](http://shichuan.org/doc/150.pdf)] [[code](https://github.com/BUPT-GAMMA/AAGOD)] ![](https://img.shields.io/badge/GIN-DCE7F1) ![](https://img.shields.io/badge/Same--Scale_CL/Supervised-EAD8D9) ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
1. [AAAI 2023] **Ma-gcl: Model augmentation tricks for graph contrastive learning**. [[pdf](https://ojs.aaai.org/index.php/AAAI/article/download/25547/25319)] [[code](https://github.com/GXM1141/MA-GCL)] ![](https://img.shields.io/badge/GCN-DCE7F1) ![](https://img.shields.io/badge/Same--Scale_CL-EAD8D9)  ![](https://img.shields.io/badge/Vanilla_FT-D8D0E1)
8. [WWW 2023] **GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learner.** [[pdf](https://github.com/THUDM/GraphMAE2)] [[code](https://dl.acm.org/doi/pdf/10.1145/3543507.3583379)] ![](https://img.shields.io/badge/GAT-DCE7F1) ![](https://img.shields.io/badge/Graph_Reconstruction-EAD8D9) ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
14. [WWW 2023] **Graphprompt: Unifying pre-training and downstream tasks for graph neural networks.** [[pdf](https://dl.acm.org/doi/pdf/10.1145/3543507.3583386)] [[code](https://github.com/Starlien95/GraphPrompt)] ![](https://img.shields.io/badge/GIN-DCE7F1) ![](https://img.shields.io/badge/Graph_Reconstruction-EAD8D9) ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
10. [CIKM 2023] **Voucher Abuse Detection with Prompt-based Fine-tuning on Graph Neural Networks.** [[pdf](https://github.com/WenZhihao666/VPGNN)] [[code](https://dl.acm.org/doi/pdf/10.1145/3583780.3615505)] ![](https://img.shields.io/badge/GraphSAGE-DCE7F1) ![](https://img.shields.io/badge/Cross--Scale_CL-EAD8D9) ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
7. [KDD 2022] **GraphMAE: Self-supervised masked graph autoencoders.** [[pdf](https://dl.acm.org/doi/pdf/10.1145/3534678.3539321)] [[code](https://github.com/THUDM/GraphMAE)] ![](https://img.shields.io/badge/GAT-DCE7F1) ![](https://img.shields.io/badge/Graph_Reconstruction-EAD8D9) ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
9. [KDD 2022] **Gppt: Graph pre-training and prompt tuning to generalize graph neural networks.** ![](https://img.shields.io/badge/GraphSAGE-DCE7F1) ![](https://img.shields.io/badge/Graph_Reconstruction/Cross--Scale_CL-EAD8D9) ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
19. [arXiv 2022.09] **Universal Prompt Tuning for Graph Neural Networks.** [[pdf](https://www.researchgate.net/profile/Taoran-Fang/publication/364110035_Prompt_Tuning_for_Graph_Neural_Networks/links/647abf7879a722376509c6a9/Prompt-Tuning-for-Graph-Neural-Networks.pdf)] ![](https://img.shields.io/badge/GIN-DCE7F1) ![](https://img.shields.io/badge/Cross--Scale_CL/Graph_Reconstruction-EAD8D9) ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
12. [KDD 2021] **Pre-training on large-scale heterogeneous graph.** [[pdf](https://github.com/BUPT-GAMMA/PTHGNN)] [[code](https://ink.library.smu.edu.sg/cgi/viewcontent.cgi?article=7891&context=sis_research)] ![](https://img.shields.io/badge/HGT-DCE7F1) ![](https://img.shields.io/badge/Same--Scale_CL-EAD8D9) ![](https://img.shields.io/badge/Vanilla_FT-D8D0E1)
13. [CIKM 2021] **Contrastive pre-training of GNNs on heterogeneous graphs.** [[pdf](https://ink.library.smu.edu.sg/cgi/viewcontent.cgi?article=7892&context=sis_research)] [[code]([https://ink.library.smu.edu.sg/cgi/viewcontent.cgi?article=7892&context=sis_research](https://github.com/BUPT-GAMMA/CPT-HG))] ![](https://img.shields.io/badge/HGT-DCE7F1) ![](https://img.shields.io/badge/Same--Scale_CL-EAD8D9) ![](https://img.shields.io/badge/Vanilla_FT-D8D0E1)
4. [ICML 2020] **Deep graph contrastive representation learning.** [[pdf](https://arxiv.org/pdf/2006.04131.pdf)] [[code](https://github.com/CRIPAC-DIG/GRACE)] ![](https://img.shields.io/badge/GCN-DCE7F1) ![](https://img.shields.io/badge/Same--Scale_CL-EAD8D9) ![](https://img.shields.io/badge/Vanilla_FT-D8D0E1)
22. [NeurIPS 2020] **Self-supervised graph transformer on large-scale molecular data.** [[pdf](https://proceedings.neurips.cc/paper/2020/file/94aef38441efa3380a3bed3faf1f9d5d-Paper.pdf)] ![](https://img.shields.io/badge/Graph_Transformer-DCE7F1) ![](https://img.shields.io/badge/Property_Prediction-EAD8D9) ![](https://img.shields.io/badge/Vanilla_FT-D8D0E1)
16. [NeurIPS 2020] **Graph contrastive learning with augmentations.** [[pdf](https://proceedings.neurips.cc/paper_files/paper/2020/file/3fe230348e9a12c13120749e3f9fa4cd-Paper.pdf)] [[code](https://github.com/Shen-Lab/GraphCL)] ![](https://img.shields.io/badge/GIN-DCE7F1) ![](https://img.shields.io/badge/Same--Scale_CL-EAD8D9) ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
15. [KDD 2020] **Gcc: Graph contrastive coding for graph neural network pre-training.** [[pdf](https://arxiv.org/pdf/2006.09963.pdf)] [[code](https://github.com/THUDM/GCC)] ![](https://img.shields.io/badge/GIN-DCE7F1) ![](https://img.shields.io/badge/Same--Scale_CL-EAD8D9) ![](https://img.shields.io/badge/Vanilla_FT-D8D0E1)
11. [KDD 2020] **Gpt-gnn: Generative pre-training of graph neural networks.** [[pdf](https://github.com/acbull/GPT-GNN)] [[code](https://dl.acm.org/doi/pdf/10.1145/3394486.3403237)] ![](https://img.shields.io/badge/HGT-DCE7F1) ![](https://img.shields.io/badge/Graph_Reconstruction-EAD8D9) ![](https://img.shields.io/badge/Vanilla_FT-D8D0E1)
21. [arXiv 2020.01] **Graph-bert: Only attention is needed for learning graph representations.** [[pdf](https://arxiv.org/pdf/2001.05140.pdf)] [[code](https://github.com/anonymous-sourcecode/Graph-Bert)] ![](https://img.shields.io/badge/Graph_Transformer-DCE7F1) ![](https://img.shields.io/badge/Supervised/Graph_Reconstruction-EAD8D9) ![](https://img.shields.io/badge/Vanilla_FT-D8D0E1)
1. [ICLR 2019] **Deep graph infomax**. [[pdf](https://arxiv.org/pdf/1809.10341)] [[code](https://github.com/PetarV-/DGI)] ![](https://img.shields.io/badge/GCN-DCE7F1) ![](https://img.shields.io/badge/Graph_Reconstruction/Cross--Scale_CL-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
1. [arXiv 2016.11] **Variational graph auto-encoders**. [[pdf](https://arxiv.org/pdf/1611.07308.pdf)] [[code](https://github.com/tkipf/gae)] ![](https://img.shields.io/badge/GCN-DCE7F1) ![](https://img.shields.io/badge/Graph_Reconstruction/Property_Prediction-EAD8D9)  ![](https://img.shields.io/badge/Vanilla_FT-D8D0E1)
2. [TKDE 2024] **Generalized graph prompt: Toward a unification of pretraining and downstream tasks on graphs** [[pdf](https://arxiv.org/pdf/2311.15317)] [[code](https://github.com/gmcmt/graph_prompt_extension)] ![](https://img.shields.io/badge/GIN-DCE7F1) ![](https://img.shields.io/badge/Cross--Scale_CL/Same--Scale_CL/Graph_Reconstruction-EAD8D9) ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
4. [WWW 2024] **Inductive graph alignment prompt: Bridging the gap between graph pretraining and inductive fine-tuning from spectral perspective** [[pdf](https://arxiv.org/pdf/2402.13556)] ![](https://img.shields.io/badge/GCN/GAT/SimpleHGN-DCE7F1) ![](https://img.shields.io/badge/Graph_Reconstruction-EAD8D9) ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
5. [AAAI 2024] **Hgprompt: Bridging homogeneous and heterogeneous graphs for few-shot prompt learning** [[pdf](https://arxiv.org/pdf/2312.01878v5)] [[code](https://github.com/Starlien95/HGPrompt)] ![](https://img.shields.io/badge/GCN/GAT/SimpleHGN-DCE7F1) ![](https://img.shields.io/badge/Graph_Reconstruction-EAD8D9) ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
6. [NeurIPS 2023 workshop] **Its all graph to me: Single-model graph representation learning on multiple domains** [[pdf](https://openreview.net/pdf?id=i8JgQVS30K)] ![](https://img.shields.io/badge/GIN-DCE7F1) ![](https://img.shields.io/badge/Same--Scale_CL-EAD8D9) ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
7. [WWW 2024] **Graphcontrol: Adding conditional control to universal graph pre-trained models for graph domain transfer learning** [[pdf](https://arxiv.org/pdf/2310.07365)] ![](https://img.shields.io/badge/GIN-DCE7F1) ![](https://img.shields.io/badge/Same--Scale_CL-EAD8D9) ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
8. [AAAI 2024] **Fine-tuning graph neural networks by preserving graph generative patterns** [[pdf](https://arxiv.org/pdf/2312.13583)] [[code](https://github.com/zjunet/G-Tuning)] ![](https://img.shields.io/badge/GIN-DCE7F1) ![](https://img.shields.io/badge/Same--Scale_CL/Graph_Reconstruction-EAD8D9) ![](https://img.shields.io/badge/Customized--FT-D8D0E1)
9. [WSDM 2024] **S2GAE: Self-Supervised Graph Autoencoders are Generalizable Learners with Graph Masking** [[pdf](https://dl.acm.org/doi/pdf/10.1145/3539597.3570404)]
10. [NeurIPS 2024] **Replay-and-Forget-Free Graph Class-Incremental Learning: A Task Profiling and Prompting Approach** [[pdf](https://arxiv.org/pdf/2410.10341)] ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
11. [NeurIPS 2024] **FUG: Feature-Universal Graph Contrastive Pre-training for Graphs with Diverse Node Features** [[pdf](https://openreview.net/pdf?id=VUuOsBrqaw)] ![](https://img.shields.io/badge/Same--Scale_CL-EAD8D9)
12. [KDD 2024] **All in One and One for All: A Simple yet Effective Method towards Cross-domain Graph Pretraining** [[pdf](https://arxiv.org/pdf/2402.09834)] ![](https://img.shields.io/badge/Same--Scale_CL-EAD8D9)
13. [KDD 2025] **Non-Homophilic Graph Pre-Training and Prompt Learning** [[pdf](https://arxiv.org/pdf/2408.12594)] ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)

## 2. LLM-based Papers
1. [arXiv 2023.10] **Talk Like a Graph: Encoding Graphs for Large Language Models** [[pdf](https://arxiv.org/abs/2310.04560)] ![](https://img.shields.io/badge/Manual_Prompt_Tuning-D8D0E1) 
2. [arxiv 2023.10] **Graphtext: Graph reasoning in text space**. [[pdf](https://arxiv.org/pdf/2310.01089.pdf)] ![](https://img.shields.io/badge/Graph_to_text_+_GPTs-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9) ![](https://img.shields.io/badge/Manual_Prompt_Tuning-D8D0E1)
3. [arXiv 2023.09] **Can LLMs Effectively Leverage Graph Structural Information: When and Why** [[pdf](https://arxiv.org/abs/2309.16595)] ![](https://img.shields.io/badge/Manual_Prompt_Tuning-D8D0E1) 
4. [arXiv 2023.08] **Natural language is all a graph needs**. [[pdf](https://scholar.google.com/scholar_url?url=https://arxiv.org/pdf/2308.07134&hl=zh-CN&sa=T&oi=gsb-gga&ct=res&cd=0&d=14935989239849530960&ei=WgdCZbaJFtqk6rQPkv-skA8&scisig=AFWwaeZJKMJktGJOJmeusMs1l5k1)] [[code](https://github.com/agiresearch/InstructGLM)] ![](https://img.shields.io/badge/Graph_to_token_+_Flan_T5,LLaMA-DCE7F1) ![](https://img.shields.io/badge/MLM,LM-EAD8D9) ![](https://img.shields.io/badge/Manual_Prompt_Tuning-D8D0E1) 
5. [arxiv 2023.08] **Evaluating large language models on graphs: Performance insights and comparative analysis**. [[pdf](https://scholar.google.com/scholar_url?url=https://arxiv.org/pdf/2308.11224&hl=zh-CN&sa=T&oi=gsb-gga&ct=res&cd=0&d=13367291863109264530&ei=JQlCZcecFqKQ6rQPwPS0qAw&scisig=AFWwaebcL4UoKZKs-b3HfIKmzeoB)] [[code](https://github.com/ayame1006/llmtograph)] ![](https://img.shields.io/badge/Graph_to_text_+_GPTs,Vicuna-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9) ![](https://img.shields.io/badge/Manual_Prompt_Tuning-D8D0E1)
6. [arxiv 2023.07] **Can large language models empower molecular property prediction?** [[pdf](https://arxiv.org/pdf/2307.07443.pdf)] [[code](https://github.com/chnq/llm4mol)] ![](https://img.shields.io/badge/Graph_to_text_+_GPTs-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9) ![](https://img.shields.io/badge/Manual_Prompt_Tuning-D8D0E1)
7. [arxiv 2023.07] **Meta-Transformer: A Unified Framework for Multimodal Learning**. [[pdf](https://scholar.google.com/scholar_url?url=https://arxiv.org/pdf/2307.10802&hl=zh-CN&sa=T&oi=gsb-gga&ct=res&cd=0&d=11077145075852511910&ei=cgZCZZG8AcaKywSmq6XgCw&scisig=AFWwaeaAZA6AHtPdRvX0JnyNhv1F)] [[code](https://github.com/invictus717/MetaTransformer)] ![](https://img.shields.io/badge/transformer-DCE7F1)
8. [arxiv 2023.07] **Exploring the potential of large language models (llms) in learning on graphs** [[pdf](https://arxiv.org/pdf/2307.03393.pdf)] [[code](https://github.com/CurryTang/Graph-LLM)] ![](https://img.shields.io/badge/Graph_to_text_+_Bert,sBert,LLaMa,GPTs-DCE7F1) ![](https://img.shields.io/badge/LM,MLM-EAD8D9) ![](https://img.shields.io/badge/Manual_Prompt_Tuning_+_Automatic_Prompt_Tuning-D8D0E1)
9. [arxiv 2023.05] **Gimlet: A unified graph-text model for instruction-based molecule zero-shot learning**. [[pdf](https://scholar.google.com/scholar_url?url=https://www.biorxiv.org/content/biorxiv/early/2023/06/01/2023.05.30.542904.full.pdf&hl=zh-CN&sa=T&oi=gsb-gga&ct=res&cd=0&d=8390578571473859304&ei=8QRCZbaEHIz5yATYnbjoDQ&scisig=AFWwaebONdO5ia5yjK3p4wA-pOf1)] ![](https://img.shields.io/badge/transformer-DCE7F1)
10. [arxiv 2023.05] **Can language models solve graph problems in natural language?** [[pdf](https://scholar.google.com/scholar_url?url=https://arxiv.org/pdf/2305.10037&hl=zh-CN&sa=T&oi=gsb-gga&ct=res&cd=0&d=10660384245119063422&ei=UgpCZeaYLKKQ6rQPwPS0qAw&scisig=AFWwaeb83Q4qbndJ3rQeda6SsHVD)] [[code](https://github.com/arthur-heng/nlgraph)] ![](https://img.shields.io/badge/Graph_to_text_+_GPTs-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9) ![](https://img.shields.io/badge/Manual_Prompt_Tuning-D8D0E1)
11. [arxiv 2023.05] **Gpt4graph: Can large language models understand graph structured data? an empirical evaluation and benchmarking** [[pdf](https://arxiv.org/pdf/2305.15066.pdf)] ![](https://img.shields.io/badge/Graph_to_text_+_GPT_3-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9) ![](https://img.shields.io/badge/Manual_Prompt_Tuning_+_Automatic_Prompt_Tuning-D8D0E1)
12. [NeurIPS 2023] **WalkLM: A Uniform Language Model Fine-tuning Framework for Attributed Graph Embedding.** [[pdf](https://openreview.net/attachment?id=ZrG8kTbt70&name=pdf)] ![](https://img.shields.io/badge/Graph_to_text-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9) ![](https://img.shields.io/badge/Manual_Prompt_Tuning-D8D0E1)
13. [ICML 2023] **Pretrained Language Models to Solve Graph Tasks in Natural Language** [[pdf](https://openreview.net/forum?id=LfCzmmnH4L)] ![](https://img.shields.io/badge/Graph_to_text-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9) ![](https://img.shields.io/badge/Manual_Prompt_Tuning-D8D0E1)
14. [KDD 2024] **Graphwiz: An instruction-following language model for graph problems**  [[pdf](https://arxiv.org/pdf/2402.16029)] ![](https://img.shields.io/badge/Graph_to_text-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9) ![](https://img.shields.io/badge/Manual_Prompt_Tuning-D8D0E1)
15. [AAAI 2024] **Zero-shot causal graph extrapolation from text via llms** [[pdf](https://arxiv.org/pdf/2312.14670)] ![](https://img.shields.io/badge/Graph_to_text-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9) ![](https://img.shields.io/badge/Manual_Prompt_Tuning-D8D0E1)
16. [KDD 2024] **Llm4dyg: Can large language models solve problems on dynamic graphs?** [[pdf](https://arxiv.org/pdf/2310.17110)] ![](https://img.shields.io/badge/Graph_to_text-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9) ![](https://img.shields.io/badge/Manual_Prompt_Tuning-D8D0E1)
17. [arXiv 2024.6] **MolecularGPT: Open Large Language Model (LLM) for Few-Shot Molecular Property Prediction** [[pdf](https://arxiv.org/abs/2406.12950)]

## 3. GNN+LLM-based Papers
1. [ICLR 2024] **Label-free Node Classification on Graphs with Large Language Models (LLMs)** [[pdf](https://arxiv.org/abs/2310.04668)] ![](https://img.shields.io/badge/GNN--centric-DCE7F1) 
2. [ICLR 2023] **One for All: Towards Training One Graph Model for All Classification Tasks** [[pdf](https://arxiv.org/abs/2310.00149)] ![](https://img.shields.io/badge/GNN--centric-DCE7F1) 
3. [arXiv_2023.09] **Prompt-based Node Feature Extractor for Few-shot Learning on Text-Attributed Graphs**.[[pdf](https://arxiv.org/abs/2309.02848)] ![](https://img.shields.io/badge/GNN--centric-DCE7F1) 
4. [arxiv 2023.08] **Simteg: A frustratingly simple approach improves textual graph learning**. [[pdf](https://arxiv.org/pdf/2308.02565.pdf)] ![](https://img.shields.io/badge/GNN--centric-DCE7F1) ![](https://img.shields.io/badge/MLM,TTCL-EAD8D9)  ![](https://img.shields.io/badge/Parameter--EfficientFT-D8D0E1)
5. [ICLR 2024] **Harnessing explanations: Llm-to-lm interpreter for
enhanced text-attributed graph representation learning**. [[pdf](https://arxiv.org/abs/2305.19523)] ![](https://img.shields.io/badge/GNN--centric-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9)  ![](https://img.shields.io/badge/Tuning--free_Prompting+Parameter--Efficient_FT-D8D0E1)
6. [arxiv 2023.05] **Congrat: Self-supervised contrastive pretraining for joint graph and text embeddings**. [[pdf](https://arxiv.org/pdf/2305.14321.pdf)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/MLM+GTCL-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
7. [KDD 2023] **Train your own GNN teacher: Graph-aware distillation on textual graphs**. [[pdf](https://arxiv.org/pdf/2304.10668.pdf)] ![](https://img.shields.io/badge/GNN--centric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
8. [arxiv 2023.04] **Graph-toolformer: To empower llms with graph reasoning ability via prompt augmented by chatgpt**.  [[pdf](https://arxiv.org/pdf/2304.11116.pdf)] ![](https://img.shields.io/badge/LLM--centric-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9)  ![](https://img.shields.io/badge/Tuning--free_Prompting+Vanilla_FT-D8D0E1)
9. [ICLR 2023] **Learning on large-scale text-attributed graphs via variational inference**. [[pdf](https://arxiv.org/pdf/2210.14709.pdf)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/Vanilla_FT-D8D0E1)
10. [SIGIR 2023] **Augmenting low-resource text classification with graph-grounded pre-training and prompting**. [[pdf](https://arxiv.org/pdf/2305.03324.pdf)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/GTCL-EAD8D9)  ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
11. [PMLR 2023] **Enhancing activity prediction models in drug discovery with the ability to understand human language**.  [[pdf](https://arxiv.org/pdf/2303.03363.pdf)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/MLM+GTCL-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
12. [ICLR 2022] **Node feature extraction by self-supervised multi-scale neighborhood prediction**. [[pdf](https://arxiv.org/pdf/2111.00064.pdf)] ![](https://img.shields.io/badge/GNN--centric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/vanilla--FT-D8D0E1)
13. [arxiv 2022.12] **Multi-modal molecule structure-text model for text-based retrieval and editing**.  [[pdf](https://arxiv.org/pdf/2212.10789.pdf)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/MLM+GTC-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
14. [arxiv 2022.09] **A molecular multimodal foundation model associating molecule graphs with natural language**. [[pdf](https://arxiv.org/pdf/2209.05481.pdf)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/MLM+GTC-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
15. [NIPS 2021] **Graphformers: Gnn-nested transformers for representation learning on textual graph**. [[pdf](https://proceedings.neurips.cc/paper_files/paper/2021/file/f18a6d1cde4b205199de8729a6637b42-Paper.pdf)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/Vanilla--FT-D8D0E1)
16. [EMNLP 2021] **Text2mol: Cross-modal molecule retrieval with natural language queries**.  [[pdf](https://aclanthology.org/2021.emnlp-main.47.pdf)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/MLM+GTC-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
17. [arxiv 2020.08] **Graph-based modeling of online communities for fake news detection**. [[pdf](hhttps://arxiv.org/pdf/2008.06274.pdf)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
18. [arxiv 2023] **GraphAgent: Exploiting Large Language Models for Interpretable Learning on Text-attributed Graphs.** [[pdf](https://openreview.net/pdf?id=L3jATpVEGv)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9) ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
19. [Nature] **Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editing.** [[pdf](https://www.nature.com/articles/s42256-023-00759-6)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/Contrastive-EAD8D9)
20. [KDD 2024] **GAugLLM: Improving Graph Contrastive Learning for Text-Attributed Graphs with Large Language Models** [[pdf](https://arxiv.org/abs/2406.11945)]
21. [arXiv 2024.6] **UniGLM: Training One Unified Language Model for Text-Attributed Graphs** [[pdf](https://arxiv.org/abs/2406.12052)]
22. [KDD 2023] **Graph-aware language model pre-training on a large graph corpus can help multiple graph applications** [[pdf](https://arxiv.org/abs/2306.02592)] ![](https://img.shields.io/badge/GNN--centric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
23. [KDD 2023] **Heterformer: Transformer-based deep node representation learning on heterogeneous text-rich networks** [[pdf](https://arxiv.org/abs/2205.10282)] ![](https://img.shields.io/badge/GNN--centric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
24. [ICLR 2023] **Edgeformers: Graphempowered transformers for representation learning on textualedge networks** [[pdf](https://arxiv.org/abs/2302.11050)] ![](https://img.shields.io/badge/GNN--centric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
25. [WSDM 2024] **Llmrec: Large language models with graph augmentation for recommendation** [[pdf](https://arxiv.org/pdf/2311.00423)] ![](https://img.shields.io/badge/GNN--centric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
26. [NIPS 2023] **Walklm: A uniform language model fine-tuning framework for attributed graph embedding** [[pdf](https://proceedings.neurips.cc/paper_files/paper/2023/file/2ac879d1865475a7abc8dfc7a9c15c27-Paper-Conference.pdf)] ![](https://img.shields.io/badge/GNN--centric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
27. [NIPS 2023] **Learning multiplex embeddings on text-rich networks with one text encoder** [[pdf](https://arxiv.org/abs/2310.06684)] ![](https://img.shields.io/badge/GNN--centric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
28. [EMNLP 2023] **Molca: Molecular graph-language modeling with cross-modal projector and uni-modal adapter** [[pdf](https://arxiv.org/abs/2310.12798)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/Vanilla_FT-D8D0E1)
29. [ACL 2023] **Patton: Language model pretraining on text-rich networks** [[pdf](https://arxiv.org/abs/2305.12268)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/Vanilla_FT-D8D0E1)
30. [IJCAI 2024] **Efficient tuning and inference for large language models on textual graphs** [[pdf](https://arxiv.org/pdf/2401.15569)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/Vanilla_FT-D8D0E1)
31. [WWW 2024] **Graphtranslator: Aligning graph model to large language model for open-ended tasks** [[pdf](https://arxiv.org/pdf/2402.07197)] ![](https://img.shields.io/badge/LLM--centric-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9)  ![](https://img.shields.io/badge/Tuning--free_Prompting+Vanilla_FT-D8D0E1)
32. [EMNLP] **Pretraining language models with text-attributed heterogeneous graphs** [[pdf](https://arxiv.org/abs/2310.12580)] ![](https://img.shields.io/badge/LLM--centric-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9)  ![](https://img.shields.io/badge/Tuning--free_Prompting+Vanilla_FT-D8D0E1)
33. [SIGIR 2024] **Graphgpt: Graph instruction tuning for large language models** [[pdf](https://arxiv.org/pdf/2310.13023)] ![](https://img.shields.io/badge/LLM--centric-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9)  ![](https://img.shields.io/badge/Tuning--free_Prompting+Vanilla_FT-D8D0E1)
34. [ACL 2024] **Instructgraph: Boosting large language models via graph-centric instruction tuning and preference alignment** [[pdf](https://arxiv.org/pdf/2402.08785)] ![](https://img.shields.io/badge/LLM--centric-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9)  ![](https://img.shields.io/badge/Tuning--free_Prompting+Vanilla_FT-D8D0E1)
35. [EMNLP 2023] **Relm: Leveraging language models for enhanced chemical reaction prediction** [[pdf](https://arxiv.org/abs/2310.13590)] ![](https://img.shields.io/badge/LLM--centric-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9)  ![](https://img.shields.io/badge/Tuning--free_Prompting+Vanilla_FT-D8D0E1)
36. [WWW 2024] **Can we soft prompt llms for graph learning tasks** [[pdf](https://arxiv.org/pdf/2402.10359)] ![](https://img.shields.io/badge/LLM--centric-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9)  ![](https://img.shields.io/badge/Tuning--free_Prompting+Vanilla_FT-D8D0E1)
37. [NIPS 2024] **Intruding with Words: Towards Understanding Graph Injection Attacks at the Text Level** [[pdf](https://arxiv.org/abs/2405.16405)] ![](https://img.shields.io/badge/LLM--centric-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9)  ![](https://img.shields.io/badge/Tuning--free_Prompting+Vanilla_FT-D8D0E1)
38. [EMNLP 2024] **OpenGraph: Towards Open Graph Foundation Models** [[pdf](https://arxiv.org/abs/2403.01121)] ![](https://img.shields.io/badge/LLM--centric-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9)  ![](https://img.shields.io/badge/Tuning--free_Prompting+Vanilla_FT-D8D0E1)
39. [ACL 2024] **Representation Learning with Large Language Models for Recommendation**  [[pdf](https://arxiv.org/abs/2310.15950)] ![](https://img.shields.io/badge/LLM--centric-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9)  ![](https://img.shields.io/badge/Tuning--free_Prompting+Vanilla_FT-D8D0E1)

## 4. Benchmarks and Datasets 
1. [arXiv 2024.06] **GraphFM: A Comprehensive Benchmark for Graph Foundation Model.** [[pdf](https://arxiv.org/abs/2406.08310)]
2. [NeurIPS 2023] **A Comprehensive Study on Text-attributed Graphs: Benchmarking and Rethinking.** [[pdf](https://proceedings.neurips.cc/paper_files/paper/2023/hash/37d00f567a18b478065f1a91b95622a0-Abstract-Datasets_and_Benchmarks.html)]![](https://img.shields.io/badge/Text_Attributed_Graph-EAD8D9)
3. [ICLR 2024] **Towards Foundational Models for Molecular Learning on Large-Scale Multi-Task Datasets.** [[pdf](https://arxiv.org/abs/2310.04292)] ![](https://img.shields.io/badge/Molecule-DCE7F1)
4. [Nature] **Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editing.** [[pdf](https://www.nature.com/articles/s42256-023-00759-6)] ![](https://img.shields.io/badge/Molecule-DCE7F1) ![](https://img.shields.io/badge/Text_Attributed_Graph-EAD8D9)

## 5. Other papers
1. [ICLR 2024] **Thought Propagation: An Analogical Approach to Complex Reasoning with Large Language Models.** [[pdf](https://openreview.net/forum?id=SBoRhRCzM3)]

## Contributors

We thank all the contributors to this list. And more contributions are very welcome.

GFMPapers

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README