GFMPapers

KO | EN

gitlite — search

#hacktoberfest #react #ai #typescript #go #machine-learning #rust #deep-learning #docker #macos #windows #vue #awesome-list #openai

(★ 377)

Must-read papers on graph foundation models (GFMs)

File Explorer

Download Latest Version (.zip)

LICENSE
README.md

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<h2 align="center"><b>GFMPapers: Must-read papers on graph foundation models (GFMs)</b></h2>

<a href="https://github.com/BUPT-GAMMA/GFMPapers/"><img src="https://awesome.re/badge.svg" alt="awesome"></a> <a href="https://github.com/BUPT-GAMMA/GFMPapers/pulls"><img src="https://img.shields.io/badge/PRs-Welcome-green" alt="PRs"></a>
     ![](https://img.shields.io/github/last-commit/BUPT-GAMMA/GFMpapers?color=blue)

This list is currently maintained by members in BUPT GAMMA Lab. If you like our project, please give us a star ⭐ on GitHub for the latest update.

We thank all the great [contributors](#contributors) very much.

⭐ We have held a tutorial about [graph foundation model](https://arxiv.org/pdf/2310.11829.pdf) at the [WebConf 2024](https://www2024.thewebconf.org/)! Here is the tutorial. [[part1](http://www.shichuan.org/doc/WWW24Tutorial_GFM_Part1.pdf)][[part2](http://www.shichuan.org/doc/WWW24Tutorial_GFM_Part2.pdf)][[part3](http://www.shichuan.org/doc/WWW24Tutorial_GFM_Part3.pdf)]

### Contents

- [Contents](#contents)
- [Keywords Convention](#keywords-convention)
- [0. Survey Papers](#0-survey-papers)
- [1. GNN-based Papers](#1-gnn-based-papers)
- [2. LLM-based Papers](#2-llm-based-papers)
- [3. GNN+LLM-based Papers](#3-gnnllm-based-papers)
- [4. Benchmarks and Datasets](#4-benchmarks-and-datasets)
- [5. Other papers](#5-other-papers)
- [Contributors](#contributors)

### Keywords Convention

![](https://img.shields.io/badge/xxx-DCE7F1) backbone architecture

![](https://img.shields.io/badge/yyy-EAD8D9) Pretraining

![](https://img.shields.io/badge/zzz-D8D0E1) Adaptation

The meaning of each tag can be referred to in the "Towards Graph Foundation Models: A Survey and Beyond" paper.

## 0. Survey Papers
1. [TPAMI 2025] **Graph Foundation Models: Concepts, Opportunities and Challenges**. [[pdf](https://ieeexplore.ieee.org/abstract/document/10915556)]
2. [TKDE 2024] **Large Language Models on Graphs: A Comprehensive Survey**. [[pdf](https://arxiv.org/pdf/2312.02783.pdf)][[paperlist](https://github.com/PeterGriffinJin/Awesome-Language-Model-on-Graphs)]
3. [KDD 2025] **Graph Foundation Models: Challenges, Methods, and Open Questions**. [[pdf](https://dl.acm.org/doi/pdf/10.1145/3711896.3736568)]
4. [KDD 2024] **A Survey of Large Language Models for Graphs**. [[pdf](https://arxiv.org/pdf/2405.08011.pdf)]
5. [KDD 2024] **Graph Intelligence with Large Language Models and Prompt Learning**. [[pdf](https://dl.acm.org/doi/pdf/10.1145/3637528.3671456)]
6. [arXiv 2023.8] **Graph Meets LLMs: Towards Large Graph Models**. [[pdf](https://arxiv.org/pdf/2308.14522.pdf)][[paperlist](https://github.com/THUMNLab/awesome-large-graph-model)]
7. [IEEE Intelligent Systems] **Integrating Graphs with Large Language Models: Methods and Prospects**. [[pdf](https://arxiv.org/pdf/2310.05499.pdf)]
8. [arXiv 2023.10] **Towards Graph Foundation Models: A Survey and Beyond**. [[pdf](https://arxiv.org/pdf/2310.11829.pdf)][[paperlist](https://github.com/BUPT-GAMMA/GFMPapers)]
9. [arXiv 2023.11] **A Survey of Graph Meets Large Language Model: Progress and Future Directions**. [[pdf](https://arxiv.org/pdf/2311.12399.pdf)][[paperlist](https://github.com/yhLeeee/Awesome-LLMs-in-Graph-tasks)]
10. [arXiv 2024.2] **Graph Foundation Models**. [[pdf](https://arxiv.org/pdf/2402.02216.pdf)][[paperlist](https://github.com/CurryTang/Towards-Graph-Foundation-Models-New-perspective-)][[paperlist2](https://github.com/CurryTang/Towards-graph-foundation-models)]
11. [arXiv 2024.2] **Advancing Graph Representation Learning with Large Language Models: A Comprehensive Survey of Techniques**. [[pdf](https://arxiv.org/pdf/2402.05952.pdf)]
12. [arXiv 2024.2] **Towards Versatile Graph Learning Approach: from the Perspective of Large Language Models**. [[pdf](https://arxiv.org/pdf/2402.11641.pdf)]
13. [arXiv 2024.3] **A Survey on Self-Supervised Pre-Training of Graph Foundation Models: A Knowledge-Based Perspective**. [[pdf](https://arxiv.org/pdf/2403.16137.pdf)][[paperlist](https://github.com/Newiz430/Pretext)]
14. [arXiv 2024.4] **A Survey of Large Language Models on Generative Graph Analytics: Query, Learning, and Applications**. [[pdf](https://arxiv.org/pdf/2404.14809.pdf)]
15. [TIST 2024] **Graph Machine Learning in the Era of Large Language Models (LLMs)**. [[pdf](https://arxiv.org/pdf/2404.14928.pdf)]
16. [arXiv 2024.12] **Graph Learning in the Era of LLMs: A Survey from the Perspective of Data, Models, and Tasks**. [[pdf](https://arxiv.org/pdf/2412.12456.pdf)]
17. [arXiv 2025.3] **Towards Graph Foundation Models: A Transferability Perspective**. [[pdf](https://arxiv.org/pdf/2503.09363.pdf)]
18. [arXiv 2025.5] **Graph Foundation Models: A Comprehensive Survey**. [[pdf](https://www.arxiv.org/pdf/2505.15116.pdf)]
19. [arXiv 2025.5] **Using Large Language Models to Tackle Fundamental Challenges in Graph Learning: A Comprehensive Survey**. [[pdf](https://arxiv.org/pdf/2505.18475.pdf)]
20. [arXiv 2025.6] **Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunities**. [[pdf](https://arxiv.org/pdf/2506.18019.pdf)][[paperlist](https://github.com/YuanchenBei/Awesome-Graphs-Meet-Agents)]

## 1. GNN-based Papers
1. [arXiv 2023.10] **Enhancing Graph Neural Networks with Structure-Based Prompt** [[pdf](https://arxiv.org/pdf/2310.17394.pdf)] ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
1. [arXiv 2023.11] **MultiGPrompt for Multi-Task Pre-Training and Prompting on Graphs** [[pdf](https://arxiv.org/pdf/2312.03731.pdf)] ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
1. [arXiv 2023.10] **HetGPT: Harnessing the Power of Prompt Tuning in Pre-Trained Heterogeneous Graph Neural Networks** [[pdf](https://arxiv.org/abs/2310.15318)] ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
1. [arXiv 2023.10] **Prompt Tuning for Multi-View Graph Contrastive Learning** [[pdf](https://arxiv.org/abs/2310.10362)] ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
1. [arXiv 2023.05] **PRODIGY: Enabling In-context Learning Over Graphs**. [[pdf](https://arxiv.org/pdf/2305.12600.pdf)] ![](https://img.shields.io/badge/GCN/GAT-DCE7F1) ![](https://img.shields.io/badge/Graph_Reconstruction/Supervised-EAD8D9)  ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
23. [arXiv 2023.05] **G-Adapter: Towards Structure-Aware Parameter-Efficient Transfer Learning for Graph Transformer Networks.** [[pdf](https://arxiv.org/pdf/2305.10329.pdf)] ![](https://img.shields.io/badge/Graph_Transformer-DCE7F1) ![](https://img.shields.io/badge/Supervised/Graph_Reconstruction/Property_Prediction-EAD8D9) ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
17. [arXiv 2023.04] **AdapterGNN: Efficient Delta Tuning Improves Generalization Ability in Graph Neural Networks.** [[pdf](https://arxiv.org/pdf/2304.09595.pdf)] ![](https://img.shields.io/badge/GIN-DCE7F1) ![](https://img.shields.io/badge/Cross--Scale_CL/Same--Scale_CL/Graph_Reconstruction-EAD8D9) ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
20. [arXiv 2023.02] **SGL-PT: A Strong Graph Learner with Graph Prompt Tuning.** [[pdf](https://arxiv.org/pdf/2302.12449.pdf)] ![](https://img.shields.io/badge/GIN-DCE7F1) ![](https://img.shields.io/badge/Same--Scale_CL/Graph_Reconstruction-EAD8D9) ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
1. [KDD 2023] **All in One: Multi-Task Prompting for Graph Neural Networks**. [[pdf](https://www.researchgate.net/profile/Jia-Li-127/publication/371608827_All_in_One_Multi-Task_Prompting_for_Graph_Neural_Networks/links/648c2270c41fb852dd0a4f62/All-in-One-Multi-Task-Prompting-for-Graph-Neural-Networks.pdf)] ![](https://img.shields.io/badge/GCN/GAT/Graph_Transformer-DCE7F1) ![](https://img.shields.io/badge/Same--Scale_CL-EAD8D9)  ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
18. [KDD 2023] **A Data-centric Framework to Endow Graph Neural Networks with Out-Of-Distribution Detection Ability.** [[pdf](http://shichuan.org/doc/150.pdf)] [[code](https://github.com/BUPT-GAMMA/AAGOD)] ![](https://img.shields.io/badge/GIN-DCE7F1) ![](https://img.shields.io/badge/Same--Scale_CL/Supervised-EAD8D9) ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
1. [AAAI 2023] **Ma-gcl: Model augmentation tricks for graph contrastive learning**. [[pdf](https://ojs.aaai.org/index.php/AAAI/article/download/25547/25319)] [[code](https://github.com/GXM1141/MA-GCL)] ![](https://img.shields.io/badge/GCN-DCE7F1) ![](https://img.shields.io/badge/Same--Scale_CL-EAD8D9)  ![](https://img.shields.io/badge/Vanilla_FT-D8D0E1)
8. [WWW 2023] **GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learner.** [[pdf](https://github.com/THUDM/GraphMAE2)] [[code](https://dl.acm.org/doi/pdf/10.1145/3543507.3583379)] ![](https://img.shields.io/badge/GAT-DCE7F1) ![](https://img.shields.io/badge/Graph_Reconstruction-EAD8D9) ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
14. [WWW 2023] **Graphprompt: Unifying pre-training and downstream tasks for graph neural networks.** [[pdf](https://dl.acm.org/doi/pdf/10.1145/3543507.3583386)] [[code](https://github.com/Starlien95/GraphPrompt)] ![](https://img.shields.io/badge/GIN-DCE7F1) ![](https://img.shields.io/badge/Graph_Reconstruction-EAD8D9) ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
10. [CIKM 2023] **Voucher Abuse Detection with Prompt-based Fine-tuning on Graph Neural Networks.** [[pdf](https://github.com/WenZhihao666/VPGNN)] [[code](https://dl.acm.org/doi/pdf/10.1145/3583780.3615505)] ![](https://img.shields.io/badge/GraphSAGE-DCE7F1) ![](https://img.shields.io/badge/Cross--Scale_CL-EAD8D9) ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
7. [KDD 2022] **GraphMAE: Self-supervised masked graph autoencoders.** [[pdf](https://dl.acm.org/doi/pdf/10.1145/3534678.3539321)] [[code](https://github.com/THUDM/GraphMAE)] ![](https://img.shields.io/badge/GAT-DCE7F1) ![](https://img.shields.io/badge/Graph_Reconstruction-EAD8D9) ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
9. [KDD 2022] **Gppt: Graph pre-training and prompt tuning to generalize graph neural networks.** ![](https://img.shields.io/badge/GraphSAGE-DCE7F1) ![](https://img.shields.io/badge/Graph_Reconstruction/Cross--Scale_CL-EAD8D9) ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
19. [arXiv 2022.09] **Universal Prompt Tuning for Graph Neural Networks.** [[pdf](https://www.researchgate.net/profile/Taoran-Fang/publication/364110035_Prompt_Tuning_for_Graph_Neural_Networks/links/647abf7879a722376509c6a9/Prompt-Tuning-for-Graph-Neural-Networks.pdf)] ![](https://img.shields.io/badge/GIN-DCE7F1) ![](https://img.shields.io/badge/Cross--Scale_CL/Graph_Reconstruction-EAD8D9) ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
12. [KDD 2021] **Pre-training on large-scale heterogeneous graph.** [[pdf](https://github.com/BUPT-GAMMA/PTHGNN)] [[code](https://ink.library.smu.edu.sg/cgi/viewcontent.cgi?article=7891&context=sis_research)] ![](https://img.shields.io/badge/HGT-DCE7F1) ![](https://img.shields.io/badge/Same--Scale_CL-EAD8D9) ![](https://img.shields.io/badge/Vanilla_FT-D8D0E1)
13. [CIKM 2021] **Contrastive pre-training of GNNs on heterogeneous graphs.** [[pdf](https://ink.library.smu.edu.sg/cgi/viewcontent.cgi?article=7892&context=sis_research)] [[code]([https://ink.library.smu.edu.sg/cgi/viewcontent.cgi?article=7892&context=sis_research](https://github.com/BUPT-GAMMA/CPT-HG))] ![](https://img.shields.io/badge/HGT-DCE7F1) ![](https://img.shields.io/badge/Same--Scale_CL-EAD8D9) ![](https://img.shields.io/badge/Vanilla_FT-D8D0E1)
4. [ICML 2020] **Deep graph contrastive representation learning.** [[pdf](https://arxiv.org/pdf/2006.04131.pdf)] [[code](https://github.com/CRIPAC-DIG/GRACE)] ![](https://img.shields.io/badge/GCN-DCE7F1) ![](https://img.shields.io/badge/Same--Scale_CL-EAD8D9) ![](https://img.shields.io/badge/Vanilla_FT-D8D0E1)
22. [NeurIPS 2020] **Self-supervised graph transformer on large-scale molecular data.** [[pdf](https://proceedings.neurips.cc/paper/2020/file/94aef38441efa3380a3bed3faf1f9d5d-Paper.pdf)] ![](https://img.shields.io/badge/Graph_Transformer-DCE7F1) ![](https://img.shields.io/badge/Property_Prediction-EAD8D9) ![](https://img.shields.io/badge/Vanilla_FT-D8D0E1)
16. [NeurIPS 2020] **Graph contrastive learning with augmentations.** [[pdf](https://proceedings.neurips.cc/paper_files/paper/2020/file/3fe230348e9a12c13120749e3f9fa4cd-Paper.pdf)] [[code](https://github.com/Shen-Lab/GraphCL)] ![](https://img.shields.io/badge/GIN-DCE7F1) ![](https://img.shields.io/badge/Same--Scale_CL-EAD8D9) ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
15. [KDD 2020] **Gcc: Graph contrastive coding for graph neural network pre-training.** [[pdf](https://arxiv.org/pdf/2006.09963.pdf)] [[code](https://github.com/THUDM/GCC)] ![](https://img.shields.io/badge/GIN-DCE7F1) ![](https://img.shields.io/badge/Same--Scale_CL-EAD8D9) ![](https://img.shields.io/badge/Vanilla_FT-D8D0E1)
11. [KDD 2020] **Gpt-gnn: Generative pre-training of graph neural networks.** [[pdf](https://github.com/acbull/GPT-GNN)] [[code](https://dl.acm.org/doi/pdf/10.1145/3394486.3403237)] ![](https://img.shields.io/badge/HGT-DCE7F1) ![](https://img.shields.io/badge/Graph_Reconstruction-EAD8D9) ![](https://img.shields.io/badge/Vanilla_FT-D8D0E1)
21. [arXiv 2020.01] **Graph-bert: Only attention is needed for learning graph representations.** [[pdf](https://arxiv.org/pdf/2001.05140.pdf)] [[code](https://github.com/anonymous-sourcecode/Graph-Bert)] ![](https://img.shields.io/badge/Graph_Transformer-DCE7F1) ![](https://img.shields.io/badge/Supervised/Graph_Reconstruction-EAD8D9) ![](https://img.shields.io/badge/Vanilla_FT-D8D0E1)
1. [ICLR 2019] **Deep graph infomax**. [[pdf](https://arxiv.org/pdf/1809.10341)] [[code](https://github.com/PetarV-/DGI)] ![](https://img.shields.io/badge/GCN-DCE7F1) ![](https://img.shields.io/badge/Graph_Reconstruction/Cross--Scale_CL-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
1. [arXiv 2016.11] **Variational graph auto-encoders**. [[pdf](https://arxiv.org/pdf/1611.07308.pdf)] [[code](https://github.com/tkipf/gae)] ![](https://img.shields.io/badge/GCN-DCE7F1) ![](https://img.shields.io/badge/Graph_Reconstruction/Property_Prediction-EAD8D9)  ![](https://img.shields.io/badge/Vanilla_FT-D8D0E1)
2. [TKDE 2024] **Generalized graph prompt: Toward a unification of pretraining and downstream tasks on graphs** [[pdf](https://arxiv.org/pdf/2311.15317)] [[code](https://github.com/gmcmt/graph_prompt_extension)] ![](https://img.shields.io/badge/GIN-DCE7F1) ![](https://img.shields.io/badge/Cross--Scale_CL/Same--Scale_CL/Graph_Reconstruction-EAD8D9) ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
4. [WWW 2024] **Inductive graph alignment prompt: Bridging the gap between graph pretraining and inductive fine-tuning from spectral perspective** [[pdf](https://arxiv.org/pdf/2402.13556)] ![](https://img.shields.io/badge/GCN/GAT/SimpleHGN-DCE7F1) ![](https://img.shields.io/badge/Graph_Reconstruction-EAD8D9) ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
5. [AAAI 2024] **Hgprompt: Bridging homogeneous and heterogeneous graphs for few-shot prompt learning** [[pdf](https://arxiv.org/pdf/2312.01878v5)] [[code](https://github.com/Starlien95/HGPrompt)] ![](https://img.shields.io/badge/GCN/GAT/SimpleHGN-DCE7F1) ![](https://img.shields.io/badge/Graph_Reconstruction-EAD8D9) ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
6. [NeurIPS 2023 workshop] **Its all graph to me: Single-model graph representation learning on multiple domains** [[pdf](https://openreview.net/pdf?id=i8JgQVS30K)] ![](https://img.shields.io/badge/GIN-DCE7F1) ![](https://img.shields.io/badge/Same--Scale_CL-EAD8D9) ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
7. [WWW 2024] **Graphcontrol: Adding conditional control to universal graph pre-trained models for graph domain transfer learning** [[pdf](https://arxiv.org/pdf/2310.07365)] ![](https://img.shields.io/badge/GIN-DCE7F1) ![](https://img.shields.io/badge/Same--Scale_CL-EAD8D9) ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
8. [AAAI 2024] **Fine-tuning graph neural networks by preserving graph generative patterns** [[pdf](https://arxiv.org/pdf/2312.13583)] [[code](https://github.com/zjunet/G-Tuning)] ![](https://img.shields.io/badge/GIN-DCE7F1) ![](https://img.shields.io/badge/Same--Scale_CL/Graph_Reconstruction-EAD8D9) ![](https://img.shields.io/badge/Customized--FT-D8D0E1)
9. [WSDM 2024] **S2GAE: Self-Supervised Graph Autoencoders are Generalizable Learners with Graph Masking** [[pdf](https://dl.acm.org/doi/pdf/10.1145/3539597.3570404)]
10. [NeurIPS 2024] **Replay-and-Forget-Free Graph Class-Incremental Learning: A Task Profiling and Prompting Approach** [[pdf](https://arxiv.org/pdf/2410.10341)] ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
11. [NeurIPS 2024] **FUG: Feature-Universal Graph Contrastive Pre-training for Graphs with Diverse Node Features** [[pdf](https://openreview.net/pdf?id=VUuOsBrqaw)] ![](https://img.shields.io/badge/Same--Scale_CL-EAD8D9)
12. [KDD 2024] **All in One and One for All: A Simple yet Effective Method towards Cross-domain Graph Pretraining** [[pdf](https://arxiv.org/pdf/2402.09834)] ![](https://img.shields.io/badge/Same--Scale_CL-EAD8D9)
13. [KDD 2025] **Non-Homophilic Graph Pre-Training and Prompt Learning** [[pdf](https://arxiv.org/pdf/2408.12594)] ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)

## 2. LLM-based Papers
1. [arXiv 2023.10] **Talk Like a Graph: Encoding Graphs for Large Language Models** [[pdf](https://arxiv.org/abs/2310.04560)] ![](https://img.shields.io/badge/Manual_Prompt_Tuning-D8D0E1) 
2. [arxiv 2023.10] **Graphtext: Graph reasoning in text space**. [[pdf](https://arxiv.org/pdf/2310.01089.pdf)] ![](https://img.shields.io/badge/Graph_to_text_+_GPTs-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9) ![](https://img.shields.io/badge/Manual_Prompt_Tuning-D8D0E1)
3. [arXiv 2023.09] **Can LLMs Effectively Leverage Graph Structural Information: When and Why** [[pdf](https://arxiv.org/abs/2309.16595)] ![](https://img.shields.io/badge/Manual_Prompt_Tuning-D8D0E1) 
4. [arXiv 2023.08] **Natural language is all a graph needs**. [[pdf](https://scholar.google.com/scholar_url?url=https://arxiv.org/pdf/2308.07134&hl=zh-CN&sa=T&oi=gsb-gga&ct=res&cd=0&d=14935989239849530960&ei=WgdCZbaJFtqk6rQPkv-skA8&scisig=AFWwaeZJKMJktGJOJmeusMs1l5k1)] [[code](https://github.com/agiresearch/InstructGLM)] ![](https://img.shields.io/badge/Graph_to_token_+_Flan_T5,LLaMA-DCE7F1) ![](https://img.shields.io/badge/MLM,LM-EAD8D9) ![](https://img.shields.io/badge/Manual_Prompt_Tuning-D8D0E1) 
5. [arxiv 2023.08] **Evaluating large language models on graphs: Performance insights and comparative analysis**. [[pdf](https://scholar.google.com/scholar_url?url=https://arxiv.org/pdf/2308.11224&hl=zh-CN&sa=T&oi=gsb-gga&ct=res&cd=0&d=13367291863109264530&ei=JQlCZcecFqKQ6rQPwPS0qAw&scisig=AFWwaebcL4UoKZKs-b3HfIKmzeoB)] [[code](https://github.com/ayame1006/llmtograph)] ![](https://img.shields.io/badge/Graph_to_text_+_GPTs,Vicuna-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9) ![](https://img.shields.io/badge/Manual_Prompt_Tuning-D8D0E1)
6. [arxiv 2023.07] **Can large language models empower molecular property prediction?** [[pdf](https://arxiv.org/pdf/2307.07443.pdf)] [[code](https://github.com/chnq/llm4mol)] ![](https://img.shields.io/badge/Graph_to_text_+_GPTs-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9) ![](https://img.shields.io/badge/Manual_Prompt_Tuning-D8D0E1)
7. [arxiv 2023.07] **Meta-Transformer: A Unified Framework for Multimodal Learning**. [[pdf](https://scholar.google.com/scholar_url?url=https://arxiv.org/pdf/2307.10802&hl=zh-CN&sa=T&oi=gsb-gga&ct=res&cd=0&d=11077145075852511910&ei=cgZCZZG8AcaKywSmq6XgCw&scisig=AFWwaeaAZA6AHtPdRvX0JnyNhv1F)] [[code](https://github.com/invictus717/MetaTransformer)] ![](https://img.shields.io/badge/transformer-DCE7F1)
8. [arxiv 2023.07] **Exploring the potential of large language models (llms) in learning on graphs** [[pdf](https://arxiv.org/pdf/2307.03393.pdf)] [[code](https://github.com/CurryTang/Graph-LLM)] ![](https://img.shields.io/badge/Graph_to_text_+_Bert,sBert,LLaMa,GPTs-DCE7F1) ![](https://img.shields.io/badge/LM,MLM-EAD8D9) ![](https://img.shields.io/badge/Manual_Prompt_Tuning_+_Automatic_Prompt_Tuning-D8D0E1)
9. [arxiv 2023.05] **Gimlet: A unified graph-text model for instruction-based molecule zero-shot learning**. [[pdf](https://scholar.google.com/scholar_url?url=https://www.biorxiv.org/content/biorxiv/early/2023/06/01/2023.05.30.542904.full.pdf&hl=zh-CN&sa=T&oi=gsb-gga&ct=res&cd=0&d=8390578571473859304&ei=8QRCZbaEHIz5yATYnbjoDQ&scisig=AFWwaebONdO5ia5yjK3p4wA-pOf1)] ![](https://img.shields.io/badge/transformer-DCE7F1)
10. [arxiv 2023.05] **Can language models solve graph problems in natural language?** [[pdf](https://scholar.google.com/scholar_url?url=https://arxiv.org/pdf/2305.10037&hl=zh-CN&sa=T&oi=gsb-gga&ct=res&cd=0&d=10660384245119063422&ei=UgpCZeaYLKKQ6rQPwPS0qAw&scisig=AFWwaeb83Q4qbndJ3rQeda6SsHVD)] [[code](https://github.com/arthur-heng/nlgraph)] ![](https://img.shields.io/badge/Graph_to_text_+_GPTs-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9) ![](https://img.shields.io/badge/Manual_Prompt_Tuning-D8D0E1)
11. [arxiv 2023.05] **Gpt4graph: Can large language models understand graph structured data? an empirical evaluation and benchmarking** [[pdf](https://arxiv.org/pdf/2305.15066.pdf)] ![](https://img.shields.io/badge/Graph_to_text_+_GPT_3-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9) ![](https://img.shields.io/badge/Manual_Prompt_Tuning_+_Automatic_Prompt_Tuning-D8D0E1)
12. [NeurIPS 2023] **WalkLM: A Uniform Language Model Fine-tuning Framework for Attributed Graph Embedding.** [[pdf](https://openreview.net/attachment?id=ZrG8kTbt70&name=pdf)] ![](https://img.shields.io/badge/Graph_to_text-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9) ![](https://img.shields.io/badge/Manual_Prompt_Tuning-D8D0E1)
13. [ICML 2023] **Pretrained Language Models to Solve Graph Tasks in Natural Language** [[pdf](https://openreview.net/forum?id=LfCzmmnH4L)] ![](https://img.shields.io/badge/Graph_to_text-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9) ![](https://img.shields.io/badge/Manual_Prompt_Tuning-D8D0E1)
14. [KDD 2024] **Graphwiz: An instruction-following language model for graph problems**  [[pdf](https://arxiv.org/pdf/2402.16029)] ![](https://img.shields.io/badge/Graph_to_text-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9) ![](https://img.shields.io/badge/Manual_Prompt_Tuning-D8D0E1)
15. [AAAI 2024] **Zero-shot causal graph extrapolation from text via llms** [[pdf](https://arxiv.org/pdf/2312.14670)] ![](https://img.shields.io/badge/Graph_to_text-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9) ![](https://img.shields.io/badge/Manual_Prompt_Tuning-D8D0E1)
16. [KDD 2024] **Llm4dyg: Can large language models solve problems on dynamic graphs?** [[pdf](https://arxiv.org/pdf/2310.17110)] ![](https://img.shields.io/badge/Graph_to_text-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9) ![](https://img.shields.io/badge/Manual_Prompt_Tuning-D8D0E1)
17. [arXiv 2024.6] **MolecularGPT: Open Large Language Model (LLM) for Few-Shot Molecular Property Prediction** [[pdf](https://arxiv.org/abs/2406.12950)]

## 3. GNN+LLM-based Papers
1. [ICLR 2024] **Label-free Node Classification on Graphs with Large Language Models (LLMs)** [[pdf](https://arxiv.org/abs/2310.04668)] ![](https://img.shields.io/badge/GNN--centric-DCE7F1) 
2. [ICLR 2023] **One for All: Towards Training One Graph Model for All Classification Tasks** [[pdf](https://arxiv.org/abs/2310.00149)] ![](https://img.shields.io/badge/GNN--centric-DCE7F1) 
3. [arXiv_2023.09] **Prompt-based Node Feature Extractor for Few-shot Learning on Text-Attributed Graphs**.[[pdf](https://arxiv.org/abs/2309.02848)] ![](https://img.shields.io/badge/GNN--centric-DCE7F1) 
4. [arxiv 2023.08] **Simteg: A frustratingly simple approach improves textual graph learning**. [[pdf](https://arxiv.org/pdf/2308.02565.pdf)] ![](https://img.shields.io/badge/GNN--centric-DCE7F1) ![](https://img.shields.io/badge/MLM,TTCL-EAD8D9)  ![](https://img.shields.io/badge/Parameter--EfficientFT-D8D0E1)
5. [ICLR 2024] **Harnessing explanations: Llm-to-lm interpreter for
enhanced text-attributed graph representation learning**. [[pdf](https://arxiv.org/abs/2305.19523)] ![](https://img.shields.io/badge/GNN--centric-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9)  ![](https://img.shields.io/badge/Tuning--free_Prompting+Parameter--Efficient_FT-D8D0E1)
6. [arxiv 2023.05] **Congrat: Self-supervised contrastive pretraining for joint graph and text embeddings**. [[pdf](https://arxiv.org/pdf/2305.14321.pdf)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/MLM+GTCL-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
7. [KDD 2023] **Train your own GNN teacher: Graph-aware distillation on textual graphs**. [[pdf](https://arxiv.org/pdf/2304.10668.pdf)] ![](https://img.shields.io/badge/GNN--centric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
8. [arxiv 2023.04] **Graph-toolformer: To empower llms with graph reasoning ability via prompt augmented by chatgpt**.  [[pdf](https://arxiv.org/pdf/2304.11116.pdf)] ![](https://img.shields.io/badge/LLM--centric-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9)  ![](https://img.shields.io/badge/Tuning--free_Prompting+Vanilla_FT-D8D0E1)
9. [ICLR 2023] **Learning on large-scale text-attributed graphs via variational inference**. [[pdf](https://arxiv.org/pdf/2210.14709.pdf)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/Vanilla_FT-D8D0E1)
10. [SIGIR 2023] **Augmenting low-resource text classification with graph-grounded pre-training and prompting**. [[pdf](https://arxiv.org/pdf/2305.03324.pdf)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/GTCL-EAD8D9)  ![](https://img.shields.io/badge/Prompt--Tuning-D8D0E1)
11. [PMLR 2023] **Enhancing activity prediction models in drug discovery with the ability to understand human language**.  [[pdf](https://arxiv.org/pdf/2303.03363.pdf)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/MLM+GTCL-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
12. [ICLR 2022] **Node feature extraction by self-supervised multi-scale neighborhood prediction**. [[pdf](https://arxiv.org/pdf/2111.00064.pdf)] ![](https://img.shields.io/badge/GNN--centric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/vanilla--FT-D8D0E1)
13. [arxiv 2022.12] **Multi-modal molecule structure-text model for text-based retrieval and editing**.  [[pdf](https://arxiv.org/pdf/2212.10789.pdf)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/MLM+GTC-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
14. [arxiv 2022.09] **A molecular multimodal foundation model associating molecule graphs with natural language**. [[pdf](https://arxiv.org/pdf/2209.05481.pdf)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/MLM+GTC-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
15. [NIPS 2021] **Graphformers: Gnn-nested transformers for representation learning on textual graph**. [[pdf](https://proceedings.neurips.cc/paper_files/paper/2021/file/f18a6d1cde4b205199de8729a6637b42-Paper.pdf)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/Vanilla--FT-D8D0E1)
16. [EMNLP 2021] **Text2mol: Cross-modal molecule retrieval with natural language queries**.  [[pdf](https://aclanthology.org/2021.emnlp-main.47.pdf)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/MLM+GTC-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
17. [arxiv 2020.08] **Graph-based modeling of online communities for fake news detection**. [[pdf](hhttps://arxiv.org/pdf/2008.06274.pdf)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
18. [arxiv 2023] **GraphAgent: Exploiting Large Language Models for Interpretable Learning on Text-attributed Graphs.** [[pdf](https://openreview.net/pdf?id=L3jATpVEGv)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9) ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
19. [Nature] **Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editing.** [[pdf](https://www.nature.com/articles/s42256-023-00759-6)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/Contrastive-EAD8D9)
20. [KDD 2024] **GAugLLM: Improving Graph Contrastive Learning for Text-Attributed Graphs with Large Language Models** [[pdf](https://arxiv.org/abs/2406.11945)]
21. [arXiv 2024.6] **UniGLM: Training One Unified Language Model for Text-Attributed Graphs** [[pdf](https://arxiv.org/abs/2406.12052)]
22. [KDD 2023] **Graph-aware language model pre-training on a large graph corpus can help multiple graph applications** [[pdf](https://arxiv.org/abs/2306.02592)] ![](https://img.shields.io/badge/GNN--centric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
23. [KDD 2023] **Heterformer: Transformer-based deep node representation learning on heterogeneous text-rich networks** [[pdf](https://arxiv.org/abs/2205.10282)] ![](https://img.shields.io/badge/GNN--centric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
24. [ICLR 2023] **Edgeformers: Graphempowered transformers for representation learning on textualedge networks** [[pdf](https://arxiv.org/abs/2302.11050)] ![](https://img.shields.io/badge/GNN--centric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
25. [WSDM 2024] **Llmrec: Large language models with graph augmentation for recommendation** [[pdf](https://arxiv.org/pdf/2311.00423)] ![](https://img.shields.io/badge/GNN--centric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
26. [NIPS 2023] **Walklm: A uniform language model fine-tuning framework for attributed graph embedding** [[pdf](https://proceedings.neurips.cc/paper_files/paper/2023/file/2ac879d1865475a7abc8dfc7a9c15c27-Paper-Conference.pdf)] ![](https://img.shields.io/badge/GNN--centric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
27. [NIPS 2023] **Learning multiplex embeddings on text-rich networks with one text encoder** [[pdf](https://arxiv.org/abs/2310.06684)] ![](https://img.shields.io/badge/GNN--centric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/Parameter--Efficient_FT-D8D0E1)
28. [EMNLP 2023] **Molca: Molecular graph-language modeling with cross-modal projector and uni-modal adapter** [[pdf](https://arxiv.org/abs/2310.12798)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/Vanilla_FT-D8D0E1)
29. [ACL 2023] **Patton: Language model pretraining on text-rich networks** [[pdf](https://arxiv.org/abs/2305.12268)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/Vanilla_FT-D8D0E1)
30. [IJCAI 2024] **Efficient tuning and inference for large language models on textual graphs** [[pdf](https://arxiv.org/pdf/2401.15569)] ![](https://img.shields.io/badge/Symmetric-DCE7F1) ![](https://img.shields.io/badge/MLM-EAD8D9)  ![](https://img.shields.io/badge/Vanilla_FT-D8D0E1)
31. [WWW 2024] **Graphtranslator: Aligning graph model to large language model for open-ended tasks** [[pdf](https://arxiv.org/pdf/2402.07197)] ![](https://img.shields.io/badge/LLM--centric-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9)  ![](https://img.shields.io/badge/Tuning--free_Prompting+Vanilla_FT-D8D0E1)
32. [EMNLP] **Pretraining language models with text-attributed heterogeneous graphs** [[pdf](https://arxiv.org/abs/2310.12580)] ![](https://img.shields.io/badge/LLM--centric-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9)  ![](https://img.shields.io/badge/Tuning--free_Prompting+Vanilla_FT-D8D0E1)
33. [SIGIR 2024] **Graphgpt: Graph instruction tuning for large language models** [[pdf](https://arxiv.org/pdf/2310.13023)] ![](https://img.shields.io/badge/LLM--centric-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9)  ![](https://img.shields.io/badge/Tuning--free_Prompting+Vanilla_FT-D8D0E1)
34. [ACL 2024] **Instructgraph: Boosting large language models via graph-centric instruction tuning and preference alignment** [[pdf](https://arxiv.org/pdf/2402.08785)] ![](https://img.shields.io/badge/LLM--centric-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9)  ![](https://img.shields.io/badge/Tuning--free_Prompting+Vanilla_FT-D8D0E1)
35. [EMNLP 2023] **Relm: Leveraging language models for enhanced chemical reaction prediction** [[pdf](https://arxiv.org/abs/2310.13590)] ![](https://img.shields.io/badge/LLM--centric-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9)  ![](https://img.shields.io/badge/Tuning--free_Prompting+Vanilla_FT-D8D0E1)
36. [WWW 2024] **Can we soft prompt llms for graph learning tasks** [[pdf](https://arxiv.org/pdf/2402.10359)] ![](https://img.shields.io/badge/LLM--centric-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9)  ![](https://img.shields.io/badge/Tuning--free_Prompting+Vanilla_FT-D8D0E1)
37. [NIPS 2024] **Intruding with Words: Towards Understanding Graph Injection Attacks at the Text Level** [[pdf](https://arxiv.org/abs/2405.16405)] ![](https://img.shields.io/badge/LLM--centric-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9)  ![](https://img.shields.io/badge/Tuning--free_Prompting+Vanilla_FT-D8D0E1)
38. [EMNLP 2024] **OpenGraph: Towards Open Graph Foundation Models** [[pdf](https://arxiv.org/abs/2403.01121)] ![](https://img.shields.io/badge/LLM--centric-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9)  ![](https://img.shields.io/badge/Tuning--free_Prompting+Vanilla_FT-D8D0E1)
39. [ACL 2024] **Representation Learning with Large Language Models for Recommendation**  [[pdf](https://arxiv.org/abs/2310.15950)] ![](https://img.shields.io/badge/LLM--centric-DCE7F1) ![](https://img.shields.io/badge/LM-EAD8D9)  ![](https://img.shields.io/badge/Tuning--free_Prompting+Vanilla_FT-D8D0E1)

## 4. Benchmarks and Datasets 
1. [arXiv 2024.06] **GraphFM: A Comprehensive Benchmark for Graph Foundation Model.** [[pdf](https://arxiv.org/abs/2406.08310)]
2. [NeurIPS 2023] **A Comprehensive Study on Text-attributed Graphs: Benchmarking and Rethinking.** [[pdf](https://proceedings.neurips.cc/paper_files/paper/2023/hash/37d00f567a18b478065f1a91b95622a0-Abstract-Datasets_and_Benchmarks.html)]![](https://img.shields.io/badge/Text_Attributed_Graph-EAD8D9)
3. [ICLR 2024] **Towards Foundational Models for Molecular Learning on Large-Scale Multi-Task Datasets.** [[pdf](https://arxiv.org/abs/2310.04292)] ![](https://img.shields.io/badge/Molecule-DCE7F1)
4. [Nature] **Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editing.** [[pdf](https://www.nature.com/articles/s42256-023-00759-6)] ![](https://img.shields.io/badge/Molecule-DCE7F1) ![](https://img.shields.io/badge/Text_Attributed_Graph-EAD8D9)

## 5. Other papers
1. [ICLR 2024] **Thought Propagation: An Analogical Approach to Complex Reasoning with Large Language Models.** [[pdf](https://openreview.net/forum?id=SBoRhRCzM3)]

## Contributors

We thank all the contributors to this list. And more contributions are very welcome.

GFMPapers

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

GFMPapers

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

Download README