Pretext

KO | EN

gitlite — search

#javascript #python #ai #llm #go #android #machine-learning #deep-learning #docker #windows #pytorch #security #mcp #openai

(★ 41)

Resource for "A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspective"

Pretext 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspective

This is an extensive and continuously updated compilation of self-supervised GFM literature categorized by the knowledge-based taxonomy, proposed by our TKDE paper :page_facing_up:**A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspective** [arXiv (full)](https://arxiv.org/abs/2403.16137) | [IEEE Xplore](https://doi.org/10.1109/TKDE.2025.3568147). Here every pretext of each paper is listed and briefly explained. You can find all pretexts and their corresponding papers with detailed metadata below, including additional pretexts and literature not listed in our paper.

A kind reminder: to search for a certain paper, type the title or the abbreviation of the proposed method (recommended) into the browser search bar (Ctrl + F). :warning:Some papers fall under multiple sections.

![](https://raw.githubusercontent.com/Newiz430/Pretext/main/fig/framework.png)

# News

* **[7 May 2025]**: [Full version](https://arxiv.org/abs/2403.16137) (v3) of our paper uploaded to arXiv! Check now! :fire:
* **[3 May 2025]**: **Our paper is accepted in TKDE! :fire::fire::fire: Final version coming soon!**
* **[3 May 2025]**: Updated papers in AAAI'25, NAACL'25 and more. Rearranged "Graph Language Models" to be more detailed and accurate. 
* **[8 Feb 2025]**: Updated papers in ICLR'25, WWW'25 and more.
* **[5 Dec 2024]**: Updated papers in WSDM'25, LoG'24 and more.
* **[4 Oct 2024]**: Updated papers in CIKM'24 and NeurIPS'24.
* **[2 Sept 2024]**: Updated papers in IJCAI'24, SIGIR'24, and KDD'24.
* **[1 Aug 2024]**: **We have a huge update (v2) thanks to the joining of [Dr. Yixin Su](https://github.com/ethanmock)!** :fire:
* **[1 Aug 2024]**: Updated papers in ICDE'24 and MM'24.
* **[24 Mar 2024]**: Our survey has uploaded to arXiv!

# Contents

* [Relevant surveys, benchmarks & empirical studies](#relevant-surveys-benchmarks--empirical-studies)
* Microscopic pre-training
  * [Node features](#node-features)
  * [Node properties](#node-properties)
  * [Links](#links)
* Mesoscopic pre-training

* [Context](#context)
  * [Long-range similarities](#long-range-similarities)
  * [Motifs](#motifs)
  * [Clusters](#clusters)
* Macroscopic pre-training

* [Global structure](#global-structure)
  * [Manifolds](#manifolds)
* [Multi-task pre-training](#multi-task-pre-training)
* [Downstream tuning](#downstream-tuning)
* [Graph language models](#graph-language-models)

# Relevant surveys, benchmarks & empirical studies

> Note: :spider_web: graph-related; :robot: LLM-related; :books: survey; :bar_chart: benchmark; :microscope: empirical study

| Paper                                                        | Venue                             |
| ------------------------------------------------------------ | --------------------------------- |
| [Pre-trained Models for Natural Language Processing: A Survey](https://arxiv.org/abs/2003.08271):books: | SCTS'20                           |
| [Self-supervised Learning on Graphs: Deep Insights and New Direction](https://arxiv.org/abs/2006.10141):spider_web::microscope: | arXiv:2006                        |
| [Pretrained Language Models for Text Generation: A Survey](https://arxiv.org/abs/2201.05273):robot::books: | IJCAI'21                          |
| [An Empirical Study of Graph Contrastive Learning](https://arxiv.org/abs/2109.01116):spider_web::microscope: | NeurIPS'21                        |
| [Self-supervised Learning: Generative or Contrastive](https://arxiv.org/abs/2006.08218):spider_web::books: | TKDE'21                           |
| [Self-supervised Learning on Graphs: Contrastive, Generative, or Predictive](https://arxiv.org/abs/2105.07342):spider_web::books: | TKDE'21                           |
| [A Survey on Contrastive Self-Supervised Learning](https://arxiv.org/abs/2011.00362):books: | Technologies'21                   |
| [Pre-Trained Models: Past, Present and Future](https://arxiv.org/abs/2106.07139):robot::books: | AI Open'21                        |
| [On the Opportunities and Risks of Foundation Models](https://arxiv.org/abs/2108.07258):robot::books: | arXiv:2108                        |
| [A Survey of Pretrained Language Models](https://link.springer.com/chapter/10.1007/978-3-031-10986-7_36):robot::books: | KSEM'22                           |
| [Contrastive Self-Supervised Learning: A Survey on Different Architectures](https://ieeexplore.ieee.org/abstract/document/9773725):books: | ICAI'22                           |
| [Graph Self-Supervised Learning: A Survey](https://arxiv.org/abs/2103.00111):spider_web::books::bar_chart: | TKDE'22                           |
| [Self-Supervised Learning of Graph Neural Networks: A Unified Review](https://arxiv.org/abs/2102.10757):spider_web::books: | TPAMI'22                          |
| [A Survey of Pretraining on Graphs: Taxonomy, Methods, and Applications](https://arxiv.org/abs/2202.07893):spider_web::books: | arXiv:2202                        |
| [A Survey on Masked Autoencoder for Self-supervised Learning in Vision and Beyond](https://arxiv.org/abs/2208.00173):books: | arXiv:2208                        |
| [A Systematic Survey of Chemical Pre-trained Models](https://arxiv.org/abs/2210.16484):spider_web::books: | IJCAI'23                          |
| [Can Language Models Solve Graph Problems in Natural Language?](https://arxiv.org/abs/2305.10037):spider_web::robot::microscope: | NeurIPS'23                        |
| [Graph Meets LLMs: Towards Large Graph Models](https://arxiv.org/abs/2308.14522):spider_web::robot::books: | NeurIPS Workshop (GLFrontiers)'23 |
| [Beyond Text: A Deep Dive into Large Language Models’ Ability on Understanding Graph Data](https://arxiv.org/abs/2310.04944):spider_web::robot::microscope: | NeurIPS Workshop (GLFrontiers)'23 |
| [Self-supervised Learning: A Succinct Review](https://link.springer.com/article/10.1007/s11831-023-09884-2):books: | Arch. Comput. Methods Eng.'23     |
| [Self-Supervised Learning for Recommender Systems: A Survey](https://arxiv.org/abs/2203.15876):books: | TKDE'23                           |
| [To Compress or Not to Compress - Self-Supervised Learning and Information Theory: A Review](https://arxiv.org/abs/2304.09355):books: | arXiv:2304                        |
| [GPT4Graph: Can Large Language Models Understand Graph Structured Data? An Empirical Evaluation and Benchmarking](https://arxiv.org/abs/2305.15066):spider_web::robot::bar_chart::microscope: | arXiv:2305                        |
| [Evaluating Large Language Models on Graphs: Performance Insights and Comparative Analysis](https://arxiv.org/abs/2308.11224):spider_web::robot::microscope: | arXiv:2308                        |
| [Graph Prompt Learning: A Comprehensive Survey and Beyond](https://arxiv.org/abs/2311.16534):spider_web::robot::books: | arXiv:2311                        |
| [Talk like a Graph: Encoding Graphs for Large Language Models](https://arxiv.org/abs/2310.04560):spider_web::robot::microscope: | ICLR'24                           |
| [Which Modality should I use - Text, Motif, or Image? : Understanding Graphs with Large Language Models](https://arxiv.org/abs/2311.09862):spider_web::robot::bar_chart: | NAACL Findings'24                 |
| [A Survey of Graph Meets Large Language Model: Progress and Future Directions](https://arxiv.org/abs/2311.12399):spider_web::robot::books: | IJCAI'24                          |
| [Position: Graph Foundation Models are Already Here](https://arxiv.org/abs/2402.02216):spider_web::robot: | ICML'24                           |
| [VisionGraph: Leveraging Large Multimodal Models for Graph Theory Problems in Visual Context](https://arxiv.org/abs/2405.04950):spider_web::robot::bar_chart: | ICML'24                           |
| [A Survey of Large Language Models for Graphs](https://arxiv.org/abs/2405.08011):spider_web::robot::books: | KDD'24                            |
| [LLM4DyG: Can LLMs Solve Spatial-Temporal Problems on Dynamic Graphs?](https://arxiv.org/abs/2310.17110):spider_web::robot::bar_chart::microscope: | KDD'24                            |
| [Investigating Instruction Tuning Large Language Models on Graphs](https://arxiv.org/abs/2408.05457):spider_web::robot::bar_chart: | COLM'24                           |
| [Do Neural Scaling Laws Exist on Graph Self-Supervised Learning?](https://arxiv.org/abs/2408.11243):spider_web::bar_chart::microscope: | LoG'24                            |
| [ProG: A Graph Prompt Learning Benchmark](https://arxiv.org/abs/2406.05346):spider_web::bar_chart: | NeurIPS'24                        |
| [GLBench: A Comprehensive Benchmark for Graph with Large Language Models](https://arxiv.org/abs/2407.07457):spider_web::robot::bar_chart: | NeurIPS'24                        |
| [Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insights](https://arxiv.org/abs/2406.10727):spider_web::robot::bar_chart::microscope: | NeurIPS'24                        |
| [Can Large Language Models Analyze Graphs like Professionals? A Benchmark and Dataset](https://arxiv.org/abs/2409.19667):spider_web::robot::bar_chart: | NeurIPS'24                        |
| [DTGB: A Comprehensive Benchmark for Dynamic Text-Attributed Graphs](https://arxiv.org/abs/2406.12072):spider_web::robot::bar_chart: | NeurIPS'24                        |
| [Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs](https://arxiv.org/abs/2307.03393):spider_web::robot::microscope: | KDD Explor. Newsl.'24             |
| [Integrating Graphs with Large Language Models: Methods and Prospects](https://arxiv.org/abs/2310.05499):spider_web::robot: | IEEE Intell. Syst.'24             |
| [A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT](https://arxiv.org/abs/2302.09419):spider_web::robot::microscope: | IJMLC'24                          |
| [Large Language Models on Graphs: A Comprehensive Survey](https://arxiv.org/abs/2312.02783):spider_web::robot::books: | TKDE'24                           |
| [Can LLMs Effectively Leverage Graph Structural Information through Prompts, and Why?](https://arxiv.org/abs/2309.16595):spider_web::robot::microscope: | TMLR'24                           |
| [A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends](https://arxiv.org/abs/2301.05712):books: | TPAMI'24                          |
| [Masked Modeling for Self-supervised Representation Learning on Vision and Beyond](https://arxiv.org/abs/2401.00897):books: | arXiv:2401                        |
| [Advancing Graph Representation Learning with Large Language Models: A Comprehensive Survey of Techniques](https://arxiv.org/abs/2402.05952):spider_web::robot::books: | arXiv:2402                        |
| [Towards Versatile Graph Learning Approach: from the Perspective of Large Language Models](https://arxiv.org/abs/2402.11641):spider_web::robot::books: | arXiv:2402                        |
| [Overcoming Pitfalls in Graph Contrastive Learning Evaluation: Toward Comprehensive Benchmarks](https://arxiv.org/abs/2402.15680):spider_web::bar_chart: | arXiv:2402                        |
| [GraphInstruct: Empowering Large Language Models with Graph Understanding and Reasoning Capability](https://arxiv.org/abs/2403.04483):spider_web::robot::bar_chart: | arXiv:2403                        |
| [Graph Machine Learning in the Era of Large Language Models (LLMs)](https://arxiv.org/abs/2404.14928):spider_web::robot::books: | arXiv:2404                        |
| [Towards Graph Contrastive Learning: A Survey and Beyond](https://arxiv.org/abs/2405.11868):spider_web::books: | arXiv:2405                        |
| [GraphFM: A Comprehensive Benchmark for Graph Foundation Model](https://arxiv.org/abs/2406.08310):spider_web::bar_chart::microscope: | arXiv:2406                        |
| [Learning on Graphs with Large Language Models (LLMs): A Deep Dive into Model Robustness](https://arxiv.org/abs/2407.12068):spider_web::robot::bar_chart::microscope: | arXiv:2407                        |
| [Towards Graph Prompt Learning: A Survey and Beyond](https://arxiv.org/abs/2408.14520):spider_web::robot::books: | arXiv:2408                        |
| [Large Language Models Meet Graph Neural Networks: A Perspective of Graph Mining](https://arxiv.org/abs/2412.19211):spider_web::robot::books::bar_chart: | arXiv:2412                        |
| [Graph Learning in the Era of LLMs: A Survey from the Perspective of Data, Models, and Tasks](https://arxiv.org/abs/2412.12456):spider_web::robot::books: | arXiv:2412                        |
| [GraCoRe: Benchmarking Graph Comprehension and Complex Reasoning in Large Language Models](https://arxiv.org/abs/2407.02936):spider_web::robot::bar_chart: | COLING'25                         |
| [Evaluating and Exploring Large Language Models on Graph Computation](https://openreview.net/forum?id=Y1r9yCMzeA):spider_web::robot::bar_chart::microscope: | ICLR'25                           |
| [How Do Large Language Models Understand Graph Patterns? A Benchmark for Graph Pattern Comprehension](https://arxiv.org/abs/2410.05298):spider_web::robot::bar_chart::microscope: | ICLR'25                           |
| [Beyond Graphs: Can Large Language Models Comprehend Hypergraphs?](https://arxiv.org/abs/2410.10083):spider_web::robot::bar_chart::microscope: | ICLR'25                           |
| [GraphArena: Evaluating and Exploring Large Language Models on Graph Computation](https://arxiv.org/abs/2407.00379):spider_web::robot::bar_chart: | ICLR'25                           |
| [Evaluating and Improving Graph to Text Generation with Large Language Models](https://arxiv.org/abs/2501.14497):spider_web::robot::microscope: | NAACL'25                          |
| [GraphEval36K: Benchmarking Coding and Reasoning Capabilities of Large Language Models on Graph Datasets](https://arxiv.org/abs/2406.16176):spider_web::robot::bar_chart: | NAACL Findings'25                 |
| [Graph Foundation Models: Concepts, Opportunities and Challenges](https://arxiv.org/abs/2310.11829):spider_web::robot::books: | TPAMI'25                          |
| [**A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspective**](https://arxiv.org/abs/2403.16137):spider_web::robot::books: | TKDE'25                           |
| [Graph2text or Graph2token: A Perspective of Large Language Models for Graph Learning](https://arxiv.org/abs/2501.01124):spider_web::robot::books: | arXiv:2501                        |
| [Graph Foundation Models for Recommendation: A Comprehensive Survey](https://arxiv.org/abs/2502.08346):spider_web::robot::books: | arXiv:2502                        |
| [Trustworthy GNNs with LLMs: A Systematic Review and Taxonomy](https://arxiv.org/abs/2502.08353):spider_web::robot::books: | arXiv:2502                        |
| [A Comprehensive Analysis on LLM-based Node Classification Algorithms](https://arxiv.org/abs/2502.00829):spider_web::robot::microscope: | arXiv:2502                        |
| [Exploring Graph Tasks with Pure LLMs: A Comprehensive Benchmark and Investigation](https://arxiv.org/abs/2502.18771):spider_web::robot::bar_chart::microscope: | arXiv:2502                        |
| [Towards Graph Foundation Models: A Transferability Perspective](https://arxiv.org/abs/2503.09363):spider_web::robot::books: | arXiv:2503                        |
| [GraphOmni: A Comprehensive and Extendable Benchmark Framework for Large Language Models on Graph-theoretic Tasks](https://arxiv.org/abs/2504.12764):spider_web::robot::bar_chart::microscope: | arXiv:2504                        |

# Node features

<details open>
 <summary>Node features</summary>

## Feature prediction

* Feature prediction: to predict the original node features by decoding low-dimensional representations
* Feature denoising: to add (generally continuous, e.g. isotropic Gaussian) noises to the original features and try to reconstruct them
* Masked feature prediction: a special, discrete case of feature denoising, which predicts the original features of masked nodes by representations of unmasked ones. It is "autoregressive" if the predicted nodes are generated one-by-one

| Paper | Venue | Pretext | Downstream | Code |
| ------------------------------------------------------------ | --------------------------------- | --------------------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ |
| [MGAE: Marginalized Graph Autoencoder for Graph Clustering](https://dl.acm.org/doi/10.1145/3132847.3132967) | CIKM'17 | Feature prediction | Graph partitioning | [link](https://github.com/GRAND-Lab/MGAE) |
| [Symmetric Graph Convolutional Autoencoder for Unsupervised Graph Representation Learning](https://arxiv.org/abs/1908.02441) (GALA) | ICCV'19 | Feature prediction | Node clustering; link prediction; etc. | [link](https://github.com/sseung0703/GALA_TF2.0) |
| [Strategies for Pre-training Graph Neural Networks](https://arxiv.org/abs/1905.12265) (AttrMask) | ICLR'20 | Masked feature prediction | Graph classification; biological function prediction | [link](https://github.com/snap-stanford/pretrain-gnns/) |
| [Graph Representation Learning via Graphical Mutual Information Maximization](https://arxiv.org/abs/2002.01169) (GMI) | WWW'20 | Feature prediction (JS) | Node classification; link prediction | [link](https://github.com/zpeng27/GMI) |
| [When Does Self-Supervision Help Graph Convolutional Networks?](https://arxiv.org/abs/2006.09136) (GraphComp) | ICML'20 | Masked feature prediction | Node classification | [link](https://github.com/Shen-Lab/SS-GCNs) |
| [GPT-GNN: Generative Pre-Training of Graph Neural Networks](https://arxiv.org/abs/2006.15437) | KDD'20 | Masked feature prediction (autoregressive) | Node classification; (heterogeneous) link prediction; edge regression | [link](https://github.com/acbull/GPT-GNN) |
| [Graph Attention Auto-Encoders](https://arxiv.org/abs/1905.10715) (GATE) | ICTAI'20 | Feature prediction | Node classification | [link](https://github.com/amin-salehi/GATE) |
| [Graph-Bert: Only Attention is Needed for Learning Graph Representations](https://arxiv.org/abs/2001.05140) | arXiv:2001 | Feature prediction | Node classification; node clustering | [link](https://github.com/jwzhanggy/Graph-Bert) |
| [Self-supervised Learning on Graphs: Deep Insights and New Direction](https://arxiv.org/abs/2006.10141) (AttributeMask) | arXiv:2006 | Masked feature prediction | Node classification | [link](https://github.com/ChandlerBang/SelfTask-GNN) |
| [SLAPS: Self-Supervision Improves Structure Learning for Graph Neural Networks](https://arxiv.org/abs/2102.05034) | NeurIPS'21 | Masked feature prediction | Node classification; image classification | [link](https://github.com/BorealisAI/SLAPS-GNN) |
| [Motif-based Graph Self-Supervised Learning for Molecular Property Prediction](https://arxiv.org/abs/2110.00987) (MGSSL) | NeurIPS'21 | Masked feature prediction | Graph classification | [link](https://github.com/zaixizhang/MGSSL.) |
| [Multi-Scale Variational Graph AutoEncoder for Link Prediction](https://dl.acm.org/doi/abs/10.1145/3488560.3498531) (MSVGAE) | WSDM'22 | Feature prediction | Link prediction | -- |
| [Self-Supervised Representation Learning via Latent Graph Prediction](https://arxiv.org/abs/2202.08333) (LaGraph) | ICML'22 | Masked feature prediction | Node classification; graph classification | [link](https://github.com/divelab/DIG/tree/dig/examples/sslgraph/LaGraph) |
| [GraphMAE: Self-Supervised Masked Graph Autoencoders](https://arxiv.org/abs/2205.10803) | KDD'22 | Masked feature prediction | Node classification; graph classification | [link](https://github.com/THUDM/GraphMAE) |
| [Interpretable Node Representation with Attribute Decoding](https://arxiv.org/abs/2212.01682) (NORAD) | TMLR'22 | Feature prediction | Node classification; node clustering; link prediction | -- |
| [Graph Masked Autoencoders with Transformers](https://arxiv.org/abs/2202.08391) (GMAE) | arXiv:2202 | Masked feature prediction | Node classification; graph classification | [link](https://github.com/RinneSz/GMAE) |
| [Wiener Graph Deconvolutional Network Improves Graph Self-Supervised Learning](https://arxiv.org/abs/2206.12933) (WGDN) | AAAI'23 | Feature prediction | Node classification; graph classification | [link](https://github.com/jcheng66/WGDN) |
| [Heterogeneous Graph Masked Autoencoders](https://arxiv.org/abs/2208.09957) (HGMAE) | AAAI'23 | Feature prediction; masked feature prediction | (Heterogeneous) node classification; node clustering | [link](https://github.com/meettyj/HGMAE) |
| [Mole-BERT: Rethinking Pre-training Graph Neural Networks for Molecules](https://openreview.net/forum?id=jevY-DtiZTR) | ICLR'23 | Masked feature prediction | Graph classification; graph regression | [link](https://github.com/junxia97/Mole-BERT) |
| [Learning Fair Graph Representations via Automated Data Augmentations](https://openreview.net/forum?id=1_OGWcP1s9w) (Graphair) | ICLR'23 | Masked feature prediction | Node classification | [link](https://github.com/divelab/DIG/tree/dig-stable/dig/fairgraph/method/Graphair) |
| [GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learner](https://arxiv.org/abs/2304.04779) | WWW'23 | Masked feature prediction | Node classification | [link](https://github.com/THUDM/GraphMAE2) |
| [SeeGera: Self-supervised Semi-implicit Graph Variational Auto-encoders with Masking](https://arxiv.org/abs/2301.12458) | WWW'23 | Masked feature prediction | Node classification; link prediction; attribute prediction | [link](https://github.com/SeeGera/SeeGera) |
| [Patton: Language Model Pretraining on Text-Rich Networks](https://arxiv.org/abs/2305.12268) | ACL'23 | Masked feature prediction | Node classification; link prediction; etc | [link](https://github.com/PeterGriffinJin/Patton) |
| [Directional Diffusion Models for Graph Representation Learning](https://arxiv.org/abs/2306.13210) (DDM) | NeurIPS'23 | Feature denoising | Node classification; graph classification | [link](https://github.com/statsle/DDM) |
| [DiP-GNN: Discriminative Pre-Training of Graph Neural Networks](https://arxiv.org/abs/2209.07499) | NeurIPS Workshop (GLFrontiers)'23 | Masked feature prediction | Node classification; link prediction | -- |
| [Towards Effective and Robust Graph Contrastive Learning With Graph Autoencoding](https://ieeexplore.ieee.org/abstract/document/10158931) (AEGCL) | TKDE'23 | Feature prediction | Node classification; node clustering; link prediction | [link](https://github.com/AllminerLab/Code-for-AEGCL-master) |
| [RARE: Robust Masked Graph Autoencoder](https://arxiv.org/abs/2304.01507) | TKDE'23 | Masked feature prediction | Node classification; graph classification; image classification | [link](https://github.com/WxTu/RARE) |
| [Homophily-Enhanced Self-Supervision for Graph Structure Learning: Insights and Directions](https://ieeexplore.ieee.org/document/10106110) (HES-GSL) | TNNLS'23 | Feature denoising | Node classification | [link](https://github.com/LirongWu/Homophily-Enhanced-Self-supervision) |
| [SGL-PT: A Strong Graph Learner with Graph Prompt Tuning](https://arxiv.org/abs/2302.12449) | arXiv:2302 | Masked feature prediction | Node classification; graph classification | -- |
| [Incomplete Graph Learning via Attribute-Structure Decoupled Variational Auto-Encoder](https://dl.acm.org/doi/10.1145/3616855.3635769) (ASD-VAE) | WSDM'24 | Feature prediction | Node classification; etc | [link](https://github.com/ASD-VAE-Anonymous/ASD-VAE) |
| [Deep Contrastive Graph Learning with Clustering-Oriented Guidance](https://arxiv.org/abs/2402.16012) (DCGL) | AAAI'24 | Feature prediction | Node clustering | [link](https://github.com/drongwbc/DCGL-AAAI24) |
| [Rethinking Graph Masked Autoencoders through Alignment and Uniformity](https://www.arxiv.org/abs/2402.07225) (AUG-MAE) | AAAI'24 | Masked feature prediction | Node classification; graph classification | [link](https://github.com/AzureLeon1/AUG-MAE) |
| [Empowering Dual-Level Graph Self-Supervised Pretraining with Motif Discovery](https://arxiv.org/abs/2312.11927) (DGPM) | AAAI'24 | Masked feature prediction | Graph classification | [link](https://github.com/RocccYan/DGPM) |
| [Generative and Contrastive Paradigms Are Complementary for Graph Self-Supervised Learning](https://arxiv.org/abs/2310.15523) (GCMAE1) | ICDE'24 | Masked feature prediction | Node classification; node clustering; graph classification; link prediction | [link](https://github.com/wyx11112/GCMAE) |
| [Masked Graph Modeling with Multi-View Contrast](https://wujcan.github.io/papers/icde24-gcmae.pdf) (GCMAE2) | ICDE'24 | Masked feature prediction | Node classification; graph classification; link prediction | [link](https://github.com/lyc0930/GCMAE) |
| [DiscoGNN: A Sample-Efficient Framework for Self-Supervised Graph Representation Learning](https://ieeexplore.ieee.org/document/10597976) | ICDE'24 | Masked feature prediction | Graph classification; similarity search | [link](https://github.com/junxia97/DiscoGNNs) |
| [IdmGAE: Importance-Inspired Dynamic Masking for Graph Autoencoders](https://dl.acm.org/doi/10.1145/3626772.3657913) | SIGIR'24 (short) | Masked feature prediction | Node classification | -- |
| [Where to Mask: Structure-Guided Masking for Graph Masked Autoencoders](https://arxiv.org/abs/2404.15806) (StructMAE) | IJCAI'24 | Masked feature prediction | Graph classification | [link](https://github.com/LiuChuang0059/StructMAE) |
| [A Pure Transformer Pretraining Framework on Text-attributed Graphs](https://arxiv.org/abs/2406.13873) (GSPT) | LoG'24 | Masked feature prediction1 | Node classification; link prediction | [link](https://github.com/SongYYYY/GSPT) (unavailable) |
| [HC-GAE: The Hierarchical Cluster-based Graph Auto-Encoder for Graph Representation Learning](https://arxiv.org/abs/2405.14742) | NeurIPS'24 | Feature prediction | Node classification; graph classification | -- |
| [Redundancy Is Not What You Need: An Embedding Fusion Graph Auto-Encoder for Self-Supervised Graph Representation Learning](https://ieeexplore.ieee.org/document/10418497) (EFGAE) | TNNLS'24 | Feature prediction | Node classification | -- |
| [UniGraph: Learning a Unified Cross-Domain Foundation Model for Text-Attributed Graphs](https://arxiv.org/abs/2402.13630) | arXiv:2402 | Masked feature prediction | Node classification; graph classification; edge classification | [link](https://github.com/yf-he/UniGraph) |
| [Exploring Task Unification in Graph Representation Learning via Generative Approach](https://arxiv.org/abs/2403.14340) (GA2E) | arXiv:2403 | Masked feature prediction | Node classification; graph classification; link prediction | -- |
| [Training MLPs on Graphs without Supervision](https://arxiv.org/abs/2412.03864) (SimMLP) | WSDM'25 | Feature prediction | Node classification; graph classification; link prediction | [link](https://github.com/Zehong-Wang/SimMLP) |
| [Adversarial Contrastive Graph Masked AutoEncoder Against Graph Structure and Feature Dual Attacks](https://ojs.aaai.org/index.php/AAAI/article/view/33364) (ACGMAE) | AAAI'25 | Masked feature prediction | Node classification; node clustering | -- |
| [Teacher-guided Edge Discriminator for Personalized Graph Masked Autoencoder](https://ojs.aaai.org/index.php/AAAI/article/view/33448) (TEDMAE) | AAAI'25 | Masked feature prediction | Node classification; node clustering | [link](https://github.com/ZZY-GraphMiningLab/TEDMAE) |
| [MORE: Molecule Pretraining with Multi-Level Pretext Task](https://ojs.aaai.org/index.php/AAAI/article/view/34262) | AAAI'25 | Masked feature prediction | Graph classification | [link](https://github.com/IT-fatica/MORE) |
| [UniGraph2: Learning a Unified Embedding Space to Bind Multimodal Graphs](https://openreview.net/forum?id=lEQEKUpXt6) | WWW'25 | Masked feature prediction | Node classification; link prediction; edge classification | [link](https://github.com/yf-he/UniGraph2) (unavailable) |
| [Hierarchical Vector Quantized Graph Autoencoder with Annealing-Based Code Selection](https://openreview.net/forum?id=6B6AmBaWfv) (HQA-GAE) | WWW'25 | Masked feature prediction | Node classification; link prediction | [link](hqa-gae) |

## Discrimination (contrastive)

* Node instance discrimination: to minimize/maximize the distance between pairs of positive/negative node representations. Jenson-Shannon (JS), InfoNCE (incl. NT-Xent), Triplet margin, and Bootstrapping are all estimators of mutual information (MI) between nodes. Other contrastive losses:
  * MSE stands for the mean squared error ($\ell_2$ loss)
  * SP stands for the [population spectral contrastive loss](https://arxiv.org/abs/2106.04156)
  * BPR stands for Bayesian Personalized Ranking loss, mostly used in recommendation
  * Other stands for other, literally not belonging to any of the above
* Dimension discrimination: to minimize/maximize the mutual information (MI) between pairs of positive/negative representation dimensions. Could be either intra-sample or inter-sample

| Paper | Venue | Pretext | Downstream | Code |
| ------------------------------------------------------------ | ------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |
| [Deep Graph Contrastive Representation Learning](https://arxiv.org/abs/2006.04131) (GRACE) | ICML Workshop (GRL+)'20 | Node instance discrimination (InfoNCE) | Node classification | [link](https://github.com/CRIPAC-DIG/GRACE) |
| [GraphTER: Unsupervised Learning of Graph Transformation Equivariant Representations via Auto-Encoding Node-wise Transformations](https://arxiv.org/abs/1911.08142) | CVPR'20 | Node instance discrimination (MSE) | Node classification (point cloud segmentation); graph (point cloud) classification | [link](https://github.com/gyshgx868/graph-ter) |
| [Contrastive and Generative Graph Convolutional Networks for Graph-based Semi-Supervised Learning](https://arxiv.org/abs/2009.07111) (CG3) | AAAI'21 | Node instance discrimination (InfoNCE) | Node classification | [link](https://github.com/LEAP-WS/CG3) |
| [Graph Contrastive Learning with Adaptive Augmentation](https://arxiv.org/abs/2010.14945) (GCA) | WWW'21 | Node instance discrimination (InfoNCE) | Node classification | [link](https://github.com/CRIPAC-DIG/GCA) |
| [SelfGNN: Self-supervised Graph Neural Networks without Explicit Negative Sampling](https://arxiv.org/abs/2103.14958) | WWW Workshop (SSL)'21 | Node instance discrimination (Bootstrapping) | Node classification | [link](https://github.com/zekarias-tilahun/SelfGNN) |
| [Self-supervised Graph Learning for Recommendation](https://arxiv.org/abs/2010.10783) (SGL) | SIGIR'21 | Node instance discrimination (InfoNCE, BPR) | Recommendation | [link](https://github.com/wujcan/SGL-TensorFlow) |
| [Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning](https://arxiv.org/abs/2105.05682) (MERIT) | IJCAI'21 | Node instance discrimination (InfoNCE) | Node classification | [link](https://github.com/GRAND-Lab/MERIT) |
| [Pre-training on Large-Scale Heterogeneous Graph](https://dl.acm.org/doi/abs/10.1145/3447548.3467396) (PT-HGNN) | KDD'21 | Node instance discrimination (InfoNCE) | (Heterogeneous) node classification; link prediction | [link](https://github.com/BUPT-GAMMA/PTHGNN) |
| [Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learning](https://arxiv.org/abs/2105.09111) (HeCo); [Hierarchical Contrastive Learning Enhanced Heterogeneous Graph Neural Network](https://arxiv.org/abs/2304.12228) (HeCo++) | KDD'21; TKDE'23 | Node instance discrimination (InfoNCE) | (Heterogeneous) node classification; node clustering | [link](https://github.com/liun-online/HeCo) |
| [InfoGCL: Information-Aware Graph Contrastive Learning](https://arxiv.org/abs/2110.15438) | NeurIPS'21 | Node instance discrimination (Bootstrapping) | Node classification; graph classification | -- |
| [From Canonical Correlation Analysis to Self-supervised Graph Neural Networks](https://arxiv.org/abs/2106.12484) (CCA-SSG) | NeurIPS'21 | Node instance discrimination (MSE); dimension discrimination | Node classification | [link](https://github.com/hengruizhang98/CCA-SSG) |
| [Self-Supervised GNN that Jointly Learns to Augment](https://arxiv.org/abs/2108.10420) (GraphSurgeon) | NeurIPS Workshop (SSL)'21 | Node instance discrimination (MSE); dimension discrimination | Node classification | [link](https://github.com/zekarias-tilahun/graph-surgeon) |
| [Simple Unsupervised Graph Representation Learning](https://ojs.aaai.org/index.php/AAAI/article/view/20748) (SUGRL) | AAAI'22 | Node instance discrimination (Triplet margin) | Node classification | [link](https://github.com/YujieMo/SUGRL) |
| [Large-Scale Representation Learning on Graphs via Bootstrapping](https://arxiv.org/abs/2102.06514) (BGRL) | ICLR'22 | Node instance discrimination (Bootstrapping) | Node classification | [link](https://github.com/nerdslab/bgrl) |
| [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://arxiv.org/abs/2105.04906) | ICLR'22 | Node instance discrimination (MSE); dimension discrimination | Node classification; etc | [link](https://github.com/PyGCL/PyGCL/blob/main/GCL/losses/vicreg.py) |
| [Adversarial Graph Contrastive Learning with Information Regularization](https://arxiv.org/abs/2208.06956) (ARIEL) | WWW'22 | Node instance discrimination (InfoNCE) | Node classification; graph classification | [link](https://github.com/Shengyu-Feng/ARIEL) |
| [Are Graph Augmentations Necessary? Simple Graph Contrastive Learning for Recommendation](https://arxiv.org/abs/2112.08679) (SimGCL); [XSimGCL: Towards Extremely Simple Graph Contrastive Learning for Recommendation](https://arxiv.org/abs/2209.02544) | SIGIR'22; TKDE'23 | Node instance discrimination (InfoNCE, BPR) | Recommendation | [link](https://github.com/Coder-Yu/SELFRec) |
| [Self-Supervised Representation Learning via Latent Graph Prediction](https://arxiv.org/abs/2202.08333) (LaGraph) | ICML'22 | Node instance discrimination (MSE) | Node classification; graph classification | [link](https://github.com/divelab/DIG/tree/dig/examples/sslgraph/LaGraph) |
| [ProGCL: Rethinking Hard Negative Mining in Graph Contrastive Learning](https://arxiv.org/abs/2110.02027) | ICML'22 | Node instance discrimination (InfoNCE) | Node classification | [link](https://github.com/junxia97/ProGCL) |
| [COSTA: Covariance-Preserving Feature Augmentation for Graph Contrastive Learning](https://arxiv.org/abs/2206.04726) | KDD'22 | Node instance discrimination (InfoNCE) | Node classification | [link](https://github.com/yifeiacc/COSTA) |
| [Relational Self-Supervised Learning on Graphs](https://arxiv.org/abs/2208.10493) (RGRL) | CIKM'22 | Node instance discrimination (Bootstrapping) | Node classification; link prediction | [link](https://github.com/Namkyeong/RGRL) |
| [Revisiting Graph Contrastive Learning from the Perspective of Graph Spectrum](https://arxiv.org/abs/2210.02330) (SpCo) | NeurIPS'22 | Node instance discrimination (InfoNCE) | Node classification | [link](https://github.com/liun-online/SpCo) |
| [Contrastive Graph Structure Learning via Information Bottleneck for Recommendation](https://proceedings.neurips.cc/paper_files/paper/2022/hash/803b9c4a8e4784072fdd791c54d614e2-Abstract-Conference.html) (CGI) | NeurIPS'22 | Node instance discrimination (InfoNCE) | Recommendation | [link](https://github.com/weicy15/CGI) |
| [Uncovering the Structural Fairness in Graph Contrastive Learning](https://arxiv.org/abs/2210.03011) (GRADE) | NeurIPS'22 | Node instance discrimination (InfoNCE) | Node classification | [link](https://github.com/BUPT-GAMMA/Uncovering-the-Structural-Fairness-in-Graph-Contrastive-Learning) |
| [Co-Modality Graph Contrastive Learning for Imbalanced Node Classification](https://papers.nips.cc/paper_files/paper/2022/hash/65cbe3e21ac62553111d9ecf7d60c18e-Abstract-Conference.html) (CM-GCL) | NeurIPS'22 | Node instance discrimination (InfoNCE) | Node classification (imbalanced) | [link](https://github.com/graphprojects/CM-GCL) |
| [Graph Barlow Twins: A Self-supervised Representation Learning Framework for Graphs](https://arxiv.org/abs/2106.02466) (G-BT) | KBS'22 | Dimension discrimination | Node classification | [link](https://github.com/pbielak/graph-barlow-twins) |
| [Towards Graph Self-Supervised Learning with Contrastive Adjusted Zooming](https://arxiv.org/abs/2111.10698) (G-Zoom) | TNNLS'22 | Node instance discrimination (InfoNCE) | Node classification | -- |
| [GRLC: Graph Representation Learning With Constraints](https://ieeexplore.ieee.org/abstract/document/10036344) | TNNLS'22 | Node instance discrimination (Triplet margin) | Node classification; node clustering; link prediction | [link](https://github.com/LarryUESTC/GRLC) |
| [Neural Eigenfunctions Are Structured Representation Learners](https://arxiv.org/abs/2210.12637) (NeuralEF) | arXiv:2210 | Dimension discrimination | Node classification; computer vision (object detection, instance segmentation, etc) | [link](https://openreview.net/attachment?id=OTMPdMH9JL&name=supplementary_material) |
| [MA-GCL: Model Augmentation Tricks for Graph Contrastive Learning](https://arxiv.org/abs/2212.07035) | AAAI'23 | Node instance discrimination (InfoNCE) | Node classification | [link](https://github.com/GXM1141/MA-GCL) |
| [ImGCL: Revisiting Graph Contrastive Learning on Imbalanced Node Classification](https://arxiv.org/abs/2205.11332) | AAAI'23 | Node instance discrimination (InfoNCE) | Node classification (imbalanced) | -- |
| [Spectral Feature Augmentation for Graph Contrastive Learning and Beyond](https://arxiv.org/abs/2212.01026) (SFA) | AAAI'23 | Node instance discrimination (Other) | Node classification; node clustering; graph classification; image classification | [link]() |
| [Beyond Smoothing: Unsupervised Graph Representation Learning with Edge Heterophily Discriminating](https://arxiv.org/abs/2211.14065) (GREET) | AAAI'23 | Node instance discrimination (Triplet margin) | Node classification | [link](https://github.com/yixinliu233/GREET) |
| [Link Prediction with Non-Contrastive Learning](https://arxiv.org/abs/2211.14394) (T-BGRL) | ICLR'23 | Node instance discrimination (Bootstrapping) | Link prediction | [link](https://github.com/snap-research/non-contrastive-link-prediction) |
| [LightGCL: Simple Yet Effective Graph Contrastive Learning for Recommendation](https://arxiv.org/abs/2302.08191) | ICLR'23 | Node instance discrimination (InfoNCE) | Recommendation | [link](https://github.com/HKUDS/LightGCL) |
| [Learning Fair Graph Representations via Automated Data Augmentations](https://openreview.net/forum?id=1_OGWcP1s9w) (Graphair) | ICLR'23 | Node instance discrimination (InfoNCE) | Node classification | [link](https://github.com/divelab/DIG/tree/dig-stable/dig/fairgraph/method/Graphair) |
| [GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learner](https://arxiv.org/abs/2304.04779) | WWW'23 | Node instance discrimination (MSE) | Node classification | [link](https://github.com/THUDM/GraphMAE2) |
| [Graph Self-supervised Learning with Augmentation-aware Contrastive Learning](https://dl.acm.org/doi/10.1145/3543507.3583246) (ABGML) | WWW'23 | Node instance discrimination (Bootstrapping) | Node classification; node clustering; similarity search | [link](https://github.com/Maxioo/ABGML) |
| [Randomized Schur Complement Views for Graph Contrastive Learning](https://arxiv.org/abs/2306.04004) (rLap) | ICML'23 | Node instance discrimination (InfoNCE, Bootstrapping) | Node classification | [link](https://github.com/kvignesh1420/rlap) |
| [Graph Contrastive Learning with Generative Adversarial Network](https://arxiv.org/abs/2308.00535) (GACN) | KDD'23 | Node instance discrimination (InfoNCE, BPR) | Node classification; link prediction | -- |
| [Heterformer: Transformer-based Deep Node Representation Learning on Heterogeneous Text-Rich Networks](https://arxiv.org/abs/2205.10282) | KDD'23 | Node instance discrimination (InfoNCE) | (Heterogeneous) node classification; node clustering; link prediction | [link](https://github.com/PeterGriffinJin/Heterformer) |
| [Exploring Universal Principles for Graph Contrastive Learning: A Statistical Perspective](https://dl.acm.org/doi/10.1145/3581783.3612229) | MM'23 | Dimension discrimination | Node classification | -- |
| [GiGaMAE: Generalizable Graph Masked Autoencoder via Collaborative Latent Space Reconstruction](https://arxiv.org/abs/2308.09663) | CIKM'23 | Node instance discrimination (InfoNCE) | Node classification; node clustering; link prediction | [link](https://github.com/sycny/GiGaMAE) |
| [GRENADE: Graph-Centric Language Model for Self-Supervised Representation Learning on Text-Attributed Graphs](https://arxiv.org/abs/2310.15109) | EMNLP Findings'23 | Node instance discrimination (InfoNCE, KL) | Node classification; node clustering; link prediction | [link](https://github.com/bigheiniu/GRENADE) |
| [Provable Training for Graph Contrastive Learning](https://arxiv.org/abs/2309.13944) (POT) | NeurIPS'23 | Node instance discrimination (InfoNCE) | Node classification | [link](https://github.com/VoidHaruhi/POT-GCL) |
| [Graph Contrastive Learning with Stable and Scalable Spectral Encoding](https://openreview.net/forum?id=0kz5RmHxmE) (Sp2GCL) | NeurIPS'23 | Node instance discrimination (InfoNCE) | Node classification; graph classification; graph regression | [link](https://github.com/bdy9527/Sp2GCL) |
| [Certifiably Robust Graph Contrastive Learning](https://arxiv.org/abs/2310.03312) (RES) | NeurIPS'23 | Node instance discrimination (InfoNCE) | Node classification | [link](https://github.com/ventr1c/RES-GCL) |
| [RARE: Robust Masked Graph Autoencoder](https://arxiv.org/abs/2304.01507) | TKDE'23 | Node instance discrimination (MSE) | Node classification; graph classification; computer vision (image classification) | [link](https://github.com/WxTu/RARE) |
| [Multi-Scale Self-Supervised Graph Contrastive Learning With Injective Node Augmentation](https://ieeexplore.ieee.org/document/10130468) (MS-CIA) | TKDE'23 | Node instance discrimination (InfoNCE) | Node classification | -- |
| [Boosting Graph Contrastive Learning via Adaptive Sampling](https://ieeexplore.ieee.org/document/10181235) (AdaS) | TNNLS'23 | Node instance discrimination (InfoNCE) | Node classification | -- |
| [Affinity Uncertainty-Based Hard Negative Mining in Graph Contrastive Learning](https://arxiv.org/abs/2301.13340) (AUGCL) | TNNLS'23 | Node instance discrimination (InfoNCE) | Node classification | [link](https://github.com/mala-lab/AUGCL) |
| [Unsupervised Structure-Adaptive Graph Contrastive Learning](https://ieeexplore.ieee.org/document/10143707/) | TNNLS'23 | Node instance discrimination (InfoNCE) | Node classification; node clustering; graph classification | -- |
| [Hierarchically Contrastive Hard Sample Mining for Graph Self-Supervised Pretraining](https://ieeexplore.ieee.org/abstract/document/10223441) (HCHSM) | TNNLS'23 | Node instance discrimination (JS) | Node classification; node clustering | [link](https://github.com/WxTu/HCHSM) |
| [Dual Contrastive Learning Network for Graph Clustering](https://ieeexplore.ieee.org/document/10097557) (DCLN) | TNNLS'23 | Dimension discrimination | Node classification; node clustering | [link](https://github.com/XinPeng97/TNNLS_DCLN) |
| [Graph Contrastive Learning With Adaptive Proximity-Based Graph Augmentation](https://ieeexplore.ieee.org/document/10143747) (PA-GCL) | TNNLS'23 | Dimension discrimination | Node classification; link prediction | [link](https://github.com/JhuoW/PA-GCL) |
| [Augmentation-Free Graph Contrastive Learning of Invariant-Discriminative Representations](https://arxiv.org/abs/2210.08345) (iGCL) | TNNLS'23 | Node instance discrimination (MSE); dimension discrimination | Node classification | [link](https://github.com/lehaifeng/T-GCN/tree/master/iGCL) |
| [Single-Pass Contrastive Learning Can Work for Both Homophilic and Heterophilic Graph](https://arxiv.org/abs/2211.10890) (SP-GCL) | TMLR'23 | Node instance discrimination (SP) | Node classification | [link](https://github.com/haonan3/SPGCL) |
| [Calibrating and Improving Graph Contrastive Learning](https://arxiv.org/abs/2101.11525) (Contrast-Reg) | TMLR'23 | Node instance discrimination (InfoNCE) | Node classification; node clustering; link prediction | [link](https://github.com/MaKaili/Contrast-Reg) |
| [Oversmoothing: A Nightmare for Graph Contrastive Learning?](https://arxiv.org/abs/2306.02117) (BlockGCL) | arXiv:2306 | Dimension discrimination | Node classification | [link](https://github.com/EdisonLeeeee/BlockGCL) |
| [Rethinking and Simplifying Bootstrapped Graph Latents](https://arxiv.org/abs/2312.02619) (SGCL2) | WSDM'24 | Node instance discrimination (Bootstrapping) | Node classification | [link](https://github.com/ZsZsZs25/SGCL) |
| [Towards Alignment-Uniformity Aware Representation in Graph Contrastive Learning](https://dl.acm.org/doi/10.1145/3616855.3635789) (AUAR) | WSDM'24 | Node instance discrimination (InfoNCE) | Node classification; node clustering | -- |
| [ReGCL: Rethinking Message Passing in Graph Contrastive Learning](https://ojs.aaai.org/index.php/AAAI/article/view/28698) | AAAI'24 | Node instance discrimination (InfoNCE) | Node classification | [link](https://github.com/RingBDStack/ReGCL) |
| [A New Mechanism for Eliminating Implicit Conflict in Graph Contrastive Learning](https://ojs.aaai.org/index.php/AAAI/article/view/29125) (PiGCL) | AAAI'24 | Node instance discrimination (InfoNCE) | Node classification; node clustering | [link](https://github.com/hedongxiao-tju/PiGCL) |
| [ASWT-SGNN: Adaptive Spectral Wavelet Transform-Based Self-Supervised Graph Neural Network](https://arxiv.org/abs/2312.05736) | AAAI'24 | Node instance discrimination (InfoNCE) | Node classification; graph classification | -- |
| [Graph Contrastive Invariant Learning from the Causal Perspective](https://arxiv.org/abs/2401.12564) (GCIL) | AAAI'24 | Dimension discrimination | Node classification | [link](https://github.com/BUPT-GAMMA/GCIL) |
| [A Graph is Worth 1-bit Spikes: When Graph Contrastive Learning Meets Spiking Neural Networks](https://arxiv.org/abs/2305.19306) (SpikeGCL) | ICLR'24 | Node instance discrimination (Triplet margin) | Node classification | [link](https://github.com/EdisonLeeeee/SpikeGCL) |
| [Self-supervised Heterogeneous Graph Learning: a Homophily and Heterogeneity View](https://openreview.net/forum?id=3FJOKjooIj) (HERO) | ICLR'24 | Node instance discrimination (MSE) | (Heterogeneous) node classification; similarity search | [link](https://github.com/YujieMo/HERO) |
| [Generative and Contrastive Paradigms Are Complementary for Graph Self-Supervised Learning](https://arxiv.org/abs/2310.15523) (GCMAE1) | ICDE'24 | Node instance discrimination (InfoNCE) | Node classification; node clustering; graph classification; link prediction | [link](https://github.com/wyx11112/GCMAE) |
| [GradGCL: Gradient Graph Contrastive Learning](https://ieeexplore.ieee.org/document/10598149) | ICDE'24 | Node instance discrimination (InfoNCE) | Node classification; graph classification | [link](https://github.com/ranlislz/Graph) |
| [Incorporating Dynamic Temperature Estimation into Contrastive Learning on Graphs](https://ieeexplore.ieee.org/document/10598091/) (GLATE) | ICDE'24 | Node instance discrimination (InfoNCE) | Node classification; node clustering; graph classification; link prediction | [link](https://github.com/Young0222/GLATE-paper-under-review) |
| [Graph Augmentation for Recommendation](https://arxiv.org/abs/2403.16656) (GraphAug) | ICDE'24 | Node instance discrimination (InfoNCE, BPR) | Recommendation | [link](https://github.com/HKUDS/GraphAug) |
| [Graph Contrastive Learning with Cohesive Subgraph Awareness](https://arxiv.org/abs/2401.17580) (CTAug) | WWW'24 | Node instance discrimination (InfoNCE) | Node classification | [link](https://github.com/wuyucheng2002/CTAug) |
| [Towards Expansive and Adaptive Hard Negative Mining: Graph Contrastive Learning via Subspace Preserving](https://dl.acm.org/doi/10.1145/3589334.3645327) (GRAPE) | WWW'24 | Node instance discrimination (InfoNCE) | Node classification; node clustering | [link](https://github.com/zz-haooo/WWW24-GRAPE) |
| [MARIO: Model Agnostic Recipe for Improving OOD Generalization of Graph Contrastive Learning](https://arxiv.org/abs/2307.13055) | WWW'24 | Node instance discrimination (InfoNCE) | Node classification; graph classification | [link](https://github.com/ZhuYun97/MARIO) |
| [Graph Contrastive Learning via Interventional View Generation](https://dl.acm.org/doi/10.1145/3589334.3645687) (GCL-IVG) | WWW'24 | Node instance discrimination (InfoNCE) | Node classification; node clustering | -- |
| [Graph Contrastive Learning with Kernel Dependence Maximization for Social Recommendation](https://dl.acm.org/doi/10.1145/3589334.3645412) (CL-KDM) | WWW'24 | Node instance discrimination (InfoNCE, BPR) | Recommendation | -- |
| [High-Frequency-aware Hierarchical Contrastive Selective Coding for Representation Learning on Text-attributed Graphs](https://arxiv.org/abs/2402.16240) (HASH-CODE) | WWW'24 | Node instance discrimination (SP) | Node classification; link prediction | -- |
| [S3GCL: Spectral, Swift, Spatial Graph Contrastive Learning](https://openreview.net/forum?id=znKAWRZSF9) | ICML'24 | Node instance discrimination (InfoNCE) | Node classification | [link](https://github.com/GuanchengWan/S3GCL) |
| [Geometric View of Soft Decorrelation in Self-Supervised Learning](https://dl.acm.org/doi/10.1145/3637528.3671914) (LogDet) | KDD'24 | Dimension discrimination | Node classification | -- |
| [Reserving-Masking-Reconstruction Model for Self-Supervised Heterogeneous Graph Representation](https://dl.acm.org/doi/10.1145/3637528.3671719) (RMR) | KDD'24 | Node instance discrimination (Bootstrapping) | (Heterogeneous) node classification | [link](https://github.com/DuanhaoranCC/RMR) |
| [Towards Robust Recommendation via Decision Boundary-aware Graph Contrastive Learning](https://arxiv.org/abs/2407.10184) (RGCL2) | KDD'24 | Node instance discrimination (InfoNCE, BPR) | Recommendation | [link](https://drive.google.com/file/d/1MV07_PK9GwfWUVBV-G9cAdHMso9XbC8q/view) |
| [Gaussian Mutual Information Maximization for Efficient Graph Self-Supervised Learning: Bridging Contrastive-based to Decorrelation-based](https://openreview.net/forum?id=ngQ1ZBt7HT) (GMIM) | MM'24 | Dimension discrimination | Node classification | -- |
| [Exploitation of a Latent Mechanism in Graph Contrastive Learning: Representation Scattering](https://openreview.net/forum?id=R8SolCx62K) (SGRL) | NeurIPS'24 | Node instance discrimination (Bootstrapping) | Node classification; node clustering | [link](https://github.com/hedongxiao-tju/SGRL) |
| [Leveraging Contrastive Learning for Enhanced Node Representations in Tokenized Graph Transformers](https://arxiv.org/abs/2406.19258) (GCFormer) | NeurIPS'24 | Node instance discrimination (InfoNCE) | Node classification | [link](https://github.com/JHL-HUST/GCFormer) (unavailable) |
| [Unified Graph Augmentations for Generalized Contrastive Learning on Graphs](https://openreview.net/forum?id=jgkKroLxeC) (GOUDA) | NeurIPS'24 | Node instance discrimination (InfoNCE); dimension discrimination | Node classification; node clustering; graph classification | [link](https://openreview.net/attachment?id=jgkKroLxeC&name=supplementary_material) |
| [Heterogeneous Graph Contrastive Learning with Meta-path Contexts and Adaptively Weighted Negative Samples](https://arxiv.org/abs/2212.13847) (MEOW) | TKDE'24 | Node instance discrimination (InfoNCE) | (Heterogeneous) node classification; node clustering | [link](https://github.com/jianxiangyu/MEOW) |
| [Redundancy Is Not What You Need: An Embedding Fusion Graph Auto-Encoder for Self-Supervised Graph Representation Learning](https://ieeexplore.ieee.org/document/10418497) (EFGAE) | TNNLS'24 | Dimension discrimination | Node classification | -- |
| [Multilevel Contrastive Graph Masked Autoencoders for Unsupervised Graph-Structure Learning](https://ieeexplore.ieee.org/document/10423234) (MCGMAE) | TNNLS'24 | Node instance discrimination (InfoNCE) | Node classification | -- |
| [UniGraph: Learning a Unified Cross-Domain Foundation Model for Text-Attributed Graphs](https://arxiv.org/abs/2402.13630) | arXiv:2402 | Node instance discrimination (Bootstrapping) | Node classification; graph classification; edge classification | [link](https://github.com/yf-he/UniGraph) |
| [Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees](https://arxiv.org/abs/2412.16441) (GIT) | arXiv:2412 | Node instance discrimination (Bootstrapping) | Node classification; graph classification; link prediction; edge classification | [link](https://github.com/Zehong-Wang/GIT) (unavailable) |
| [Training MLPs on Graphs without Supervision](https://arxiv.org/abs/2412.03864) (SimMLP) | WSDM'25 | Node instance discrimination (MSE) | Node classification; graph classification; link prediction | [link](https://github.com/Zehong-Wang/SimMLP) |
| [UniGLM: Training One Unified Language Model for Text-Attributed Graph Embedding](https://arxiv.org/abs/2406.12052) | WSDM'25 | Node instance discrimination (InfoNCE) | Node classification; link prediction | [link](https://github.com/NYUSHCS/UniGLM) |
| [Does GCL Need a Large Number of Negative Samples? Enhancing Graph Contrastive Learning with Effective and Efficient Negative Sampling](https://arxiv.org/abs/2503.17908) (E2Neg) | AAAI'25 | Node instance discrimination (InfoNCE) | Node classification | [link](https://github.com/hedongxiao-tju/E2Neg) |
| [Graph Structure Refinement with Energy-based Contrastive Learning](https://arxiv.org/abs/2412.17856) (ECL-GSR) | AAAI'25 | Node instance discrimination (InfoNCE) | Node classification; graph classification | -- |
| [Why Does Dropping Edges Usually Outperform Adding Edges in Graph Contrastive Learning?](https://arxiv.org/abs/2412.08128) (EPAGCL) | AAAI'25 | Node instance discrimination (InfoNCE) | Node classification | [link](https://github.com/hyzhang98/EPAGCL) |
| [GNN-Transformer Cooperative Architecture for Trustworthy Graph Contrastive Learning](https://www.arxiv.org/abs/2412.16218) (GTCA) | AAAI'25 | Node instance discrimination (InfoNCE) | Node classification | [link](https://github.com/a-hou/GTCA) |
| [Teacher-guided Edge Discriminator for Personalized Graph Masked Autoencoder](https://ojs.aaai.org/index.php/AAAI/article/view/33448) (TEDMAE) | AAAI'25 | Node instance discrimination (InfoNCE) | Node classification; node clustering | [link](https://github.com/ZZY-GraphMiningLab/TEDMAE) |
| [Adversarial Contrastive Graph Augmentation with Counterfactual Regularization](https://ojs.aaai.org/index.php/AAAI/article/view/34101) (ACGA) | AAAI'25 | Node instance discrimination (Triplet margin) | Node classification; link prediction | [link](https://github.com/longtao-09/ACGA) |
| [LOHA: Direct Graph Spectral Contrastive Learning Between Low-Pass and High-Pass Views](https://arxiv.org/abs/2501.02969) | AAAI'25 | Node instance discrimination (InfoNCE) | Node classification | -- |
| [Graph Contrastive Learning with Joint Spectral Augmentation of Attribute and Topology](https://ojs.aaai.org/index.php/AAAI/article/view/35506) (GCL-JAM) | AAAI'25 | Node instance discrimination (InfoNCE) | Node classification | -- |
| [Centrality-guided Pre-training for Graph](https://openreview.net/forum?id=X8E65IxA73) (CenPre) | ICLR'25 | Node instance discrimination (InfoNCE, MSE) | Node classification; graph classification; link prediction | -- |
| [Str-GCL: Structural Commonsense Driven Graph Contrastive Learning](https://dl.acm.org/doi/10.1145/3696410.3714900) | WWW'25 | Node instance discrimination (InfoNCE); dimension discrimination | Node classification; node clustering | -- |
| [Balancing Graph Embedding Smoothness in Self-supervised Learning via Information-Theoretic Decomposition](https://openreview.net/forum?id=uNXr6f0ARA) (BSG) | WWW'25 | Node instance discrimination (MSE) | Node classification; link prediction | [link](https://anonymous.4open.science/r/BSG-2025) (private) |

</details>

# Node properties

<details open>
 <summary>Node properties</summary>

* Property prediction: a regression task to predict the property of a node, e.g. degree/clustering coefficient 
* Centrality ranking: to estimate whether the centrality score of a node is greater/lower than that of another node
* Node order matching: to match the output node order with the input order
* Property-based discrimination: to perform contrastive learning in which positive/negative samples are selected based on node properties, e.g. nodes with the same/different degrees

</details>

# Links

<details open>
 <summary>Links</summary>

* Link prediction: a generally binary classification task that predicts if two nodes are connected by a link. For heterogeneous graphs, link prediction is based on meta-paths. For hypergraphs, link prediction searchs for the missing node given other nodes in a hyperedge
* Link denoising: to add (generally continuous) noises to the original edge set and try to reconstruct it
* Masked link prediction: to predict the masked links by node representations propagated on the unmasked graph. It is "autoregressive" if the predicted links are generated one-by-one
* (Masked) edge feature prediction: to predict the original (masked) edge features by node representations
* Edge discrimination: to perform contrastive learning between edge features / representations

| Paper | Venue | Pretext | Downstream | Code |
| ------------------------------------------------------------ | --------------------------------- | ---------------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ |
| [Variational Graph Auto-Encoders](https://arxiv.org/abs/1611.07308) (GAE, VGAE) | NIPS Workshop (BDL)'16 | Link prediction | Link prediction | [link](https://github.com/tkipf/gae) |
| [Adversarially Regularized Graph Autoencoder for Graph Embedding](https://arxiv.org/abs/1802.04407) (ARGA, ARVGA) | IJCAI'18 | Link prediction | Link prediction; node clustering | [link](https://github.com/GRAND-Lab/ARGA) |
| [Unsupervised Pre-training of Graph Convolutional Networks](https://arxiv.org/abs/1905.13728) (DenoisingRecon) | ICLR Workshop (RLGM)'19 | Masked link prediction | Node classification | -- |
| [Graphite: Iterative Generative Modeling of Graphs](https://arxiv.org/abs/1803.10459) | ICML'19 | Link prediction | Node classification; link prediction | [link](https://github.com/ermongroup/graphite) |
| [Semi-Implicit Graph Variational Auto-Encoders](https://arxiv.org/abs/1908.07078) (SIG-VAE) | NeurIPS'19 | Link prediction | Node classification; link prediction; node clustering; graph generation | [link](https://github.com/sigvae/SIGraphVAE) |
| [Strategies for Pre-training Graph Neural Networks](https://arxiv.org/abs/1905.12265) (AttrMask) | ICLR'20 | Masked edge feature prediction | Graph classification; biological function prediction | [link](https://github.com/snap-stanford/pretrain-gnns/) |
| [GPT-GNN: Generative Pre-Training of Graph Neural Networks](https://arxiv.org/abs/2006.15437) | KDD'20 | Masked link prediction (autoregressive) | Node classification; link prediction; edge classification | [link](https://github.com/acbull/GPT-GNN) |
| [Self-supervised Auxiliary Learning with Meta-paths for Heterogeneous Graphs](https://arxiv.org/abs/2007.08294) (SELAR) | NeurIPS'20 | Link prediction | (Heterogeneous) node classification; link prediction | [link](https://github.com/mlvlab/SELAR) |
| [Self-supervised Learning on Graphs: Deep Insights and New Direction](https://arxiv.org/abs/2006.10141) (EdgeMask) | arXiv:2006 | Masked link prediction | Node classification | [link](https://github.com/ChandlerBang/SelfTask-GNN) |
| [Contrastive and Generative Graph Convolutional Networks for Graph-based Semi-Supervised Learning](https://arxiv.org/abs/2009.07111) (CG3) | AAAI'21 | Link prediction | Node classification | [link](https://github.com/LEAP-WS/CG3) |
| [How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervision](https://arxiv.org/abs/2204.04879) (SuperGAT) | ICLR'21 | Link prediction | Node classification; link prediction | [link](https://github.com/dongkwan-kim/SuperGAT) |
| [Permutation-Invariant Variational Autoencoder for Graph-Level Representation Learning](https://arxiv.org/abs/2104.09856) (PIGAE) | NeurIPS'21 | Link prediction; edge feature prediction | Graph classification | [link](https://github.com/jrwnter/pigvae) |
| [Motif-based Graph Self-Supervised Learning for Molecular Property Prediction](https://arxiv.org/abs/2110.00987) (MGSSL) | NeurIPS'21 | Masked edge feature prediction | Graph classification | [link](https://github.com/zaixizhang/MGSSL.) |
| [GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graph](https://arxiv.org/abs/2105.02605) | NeurIPS'21 | Link prediction | Link prediction | [link](https://github.com/microsoft/GraphFormers) |
| [Self-Supervised Graph Representation Learning via Topology Transformations](https://arxiv.org/abs/2105.11689) (TopoTER) | TKDE'21 | Masked link prediction | Node classification; graph classification; link prediction | [link](https://github.com/gyshgx868/topo-ter) |
| [Directed Graph Auto-Encoders](https://arxiv.org/abs/2202.12449) (DiGAE) | AAAI'22 | Link prediction | (Directed) link prediction | [link](https://github.com/gidiko/DiGAE) |
| [GPPT: Graph Pre-training and Prompt Tuning to Generalize Graph Neural Networks](https://dl.acm.org/doi/10.1145/3534678.3539249) | KDD'22 | Masked link prediction | Node classification | [link](https://github.com/MingChen-Sun/GPPT) |
| [Link Prediction with Contextualized Self-Supervision](https://arxiv.org/abs/2201.10069) (CSSL2) | TKDE'22 | Link prediction | Link prediction | [link](https://github.com/daokunzhang/CSSL) |
| [Interpretable Node Representation with Attribute Decoding](https://arxiv.org/abs/2212.01682) (NORAD) | TMLR'22 | Link prediction | Node classification; node clustering; link prediction | -- |
| [S2GAE: Self-Supervised Graph Autoencoders are Generalizable Learners with Graph Masking](https://dl.acm.org/doi/abs/10.1145/3539597.3570404) | WSDM'23 | Masked link prediction | Node classification; graph classification; link prediction | [link](https://github.com/qiaoyu-tan/S2GAE) |
| [Dual Low-Rank Graph Autoencoder for Semantic and Topological Networks](https://ojs.aaai.org/index.php/AAAI/article/view/25536) (DLR-GAE) | AAAI'23 | Link prediction | Node classification | [link](https://github.com/chenzl23/DLRGAE) |
| [Heterogeneous Graph Masked Autoencoders](https://arxiv.org/abs/2208.09957) (HGMAE) | AAAI'23 | Masked link prediction | (Heterogeneous) node classification; node clustering | [link](https://github.com/meettyj/HGMAE) |
| [Deep Manifold Graph Auto-Encoder for Attributed Graph Embedding](https://arxiv.org/abs/2401.06727) (DMGAE, DMVGAE) | ICASSP'23 | Link prediction | Node clustering; link prediction | -- |
| [Learning Fair Graph Representations via Automated Data Augmentations](https://openreview.net/forum?id=1_OGWcP1s9w) (Graphair) | ICLR'23 | Masked link prediction | Node classification | [link](https://github.com/divelab/DIG/tree/dig-stable/dig/fairgraph/method/Graphair) |
| [Multi-head Variational Graph Autoencoder Constrained by Sum-product Networks](https://dl.acm.org/doi/abs/10.1145/3543507.3583517) (SPN-MVGAE) | WWW'23 | Link prediction | Node classification; link prediction | [link](https://github.com/godlovexiari/MVGAE-SPN) (unavailable) |
| [SeeGera: Self-supervised Semi-implicit Graph Variational Auto-encoders with Masking](https://arxiv.org/abs/2301.12458) | WWW'23 | Masked link prediction | Node classification; link prediction; attribute prediction | [link](https://github.com/SeeGera/SeeGera) |
| [Graph-Aware Language Model Pre-Training on a Large Graph Corpus Can Help Multiple Graph Applications](https://arxiv.org/abs/2306.02592) (GALM) | KDD'23 | Link prediction | (Heterogeneous) node classification; link prediction; edge classification | -- |
| [What's Behind the Mask: Understanding Masked Graph Modeling for Graph Autoencoders](https://arxiv.org/abs/2205.10053) (MaskGAE) | KDD'23 | Masked link prediction | Node classification; link prediction | [link](https://github.com/EdisonLeeeee/MaskGAE) |
| [DiP-GNN: Discriminative Pre-Training of Graph Neural Networks](https://arxiv.org/abs/2209.07499) | NeurIPS Workshop (GLFrontiers)'23 | Masked link prediction | Node classification; link prediction | -- |
| [Maximizing Mutual Information Across Feature and Topology Views for Representing Graphs](https://arxiv.org/abs/2105.06715) (MVMI-FT) | TKDE'23 | Link prediction | Node classification; node clustering | [link](https://github.com/xiaolongo/MaxMIAcrossFT) |
| [Towards Effective and Robust Graph Contrastive Learning With Graph Autoencoding](https://ieeexplore.ieee.org/abstract/document/10158931) (AEGCL) | TKDE'23 | Link prediction | Node classification; node clustering; link prediction | [link](https://github.com/AllminerLab/Code-for-AEGCL-master) |
| [ULTRA-DP: Unifying Graph Pre-training with Multi-task Graph Dual Prompt](https://arxiv.org/abs/2310.14845) | arXiv:2310 | Link prediction | Node classification; link prediction | [link](https://github.com/Keytoyze/ULTRA-DP) |
| [Incomplete Graph Learning via Attribute-Structure Decoupled Variational Auto-Encoder](https://dl.acm.org/doi/10.1145/3616855.3635769) (ASD-VAE) | WSDM'24 | Edge feature prediction | Node classification; etc | [link](https://github.com/ASD-VAE-Anonymous/ASD-VAE) |
| [Generative and Contrastive Paradigms Are Complementary for Graph Self-Supervised Learning](https://arxiv.org/abs/2310.15523) (GCMAE1) | ICDE'24 | Link prediction | Node classification; node clustering; graph classification; link prediction | [link](https://github.com/wyx11112/GCMAE) |
| [DiscoGNN: A Sample-Efficient Framework for Self-Supervised Graph Representation Learning](https://ieeexplore.ieee.org/document/10597976) | ICDE'24 | Edge feature prediction | Graph classification; similarity search | [link](https://github.com/junxia97/DiscoGNNs) |
| [Decoupled Variational Graph Autoencoder for Link Prediction](https://dl.acm.org/doi/10.1145/3589334.3645601) (D-VGAE) | WWW'24 | Link prediction | Node classification; node clustering; link prediction | [link](https://github.com/yoonsikcho/d-vgae) |
| [Masked Graph Autoencoder with Non-discrete Bandwidths](https://arxiv.org/abs/2402.03814) (Bandana) | WWW'24 | Link denoising | Node classification; link prediction | [link](https://github.com/Newiz430/Bandana) |
| [OpenGraph: Towards Open Graph Foundation Models](https://arxiv.org/abs/2403.01121) | EMNLP Findings'24 | Masked link prediction | Node classification; link prediction | [link](https://github.com/HKUDS/OpenGraph) |
| [HC-GAE: The Hierarchical Cluster-based Graph Auto-Encoder for Graph Representation Learning](https://arxiv.org/abs/2405.14742) | NeurIPS'24 | Link prediction | Node classification; graph classification | -- |
| [Redundancy Is Not What You Need: An Embedding Fusion Graph Auto-Encoder for Self-Supervised Graph Representation Learning](https://ieeexplore.ieee.org/document/10418497) (EFGAE) | TNNLS'24 | Link prediction | Node classification | -- |
| [AnyGraph: Graph Foundation Model in the Wild](https://arxiv.org/abs/2408.10700) | arXiv:2408 | Link prediction | Node classification; graph classification; link prediction | [link](https://github.com/HKUDS/AnyGraph) |
| [Edge Contrastive Learning: An Augmentation-Free Graph Contrastive Learning Model](https://arxiv.org/abs/2412.11075) (AFECL) | AAAI'25 | Edge discrimination (InfoNCE) | Node classification; link prediction | [link](https://github.com/YujunLi361/AFECL) |
| [Adversarial Contrastive Graph Augmentation with Counterfactual Regularization](https://ojs.aaai.org/index.php/AAAI/article/view/34101) (ACGA) | AAAI'25 | Link prediction | Node classification; link prediction | [link](https://github.com/longtao-09/ACGA) |
| [Adversarial Contrastive Graph Masked AutoEncoder Against Graph Structure and Feature Dual Attacks](https://ojs.aaai.org/index.php/AAAI/article/view/33364) (ACGMAE) | AAAI'25 | Link prediction | Node classification; node clustering | -- |
| [Hierarchical Vector Quantized Graph Autoencoder with Annealing-Based Code Selection](https://openreview.net/forum?id=6B6AmBaWfv) (HQA-GAE) | WWW'25 | Link prediction | Node classification; link prediction | [link](hqa-gae) |
| [WAGE: Weight-Sharing Attribute-Missing Graph Autoencoder](https://ieeexplore.ieee.org/document/10937935) | TPAMI'25 | Masked link prediction | Node classification | [link](https://github.com/WxTu/WAGE) (unavailable) |

</details>

# Context

<details open>
 <summary>Context</summary>

* Context discrimination: to distinguish between contextual nodes and non-contextual nodes. LE stands for Laplacian Eigenmaps objective
* Contextual subgraph discrimination: to distinguish between representations aggregated from different contextual subgraphs (maybe from different receptive fields). CE stands for cross-entropy
* Context feature prediction: node feature prediction but to reconstruct the features of k-hop neighbors instead
* Contextual property prediction: to predict the properties of contextual subgraphs (e.g. node / edge types contained, total node / edge counts, structural coefficient)

| Paper | Venue | Pretext | Downstream | Code |
| ------------------------------------------------------------ | ----------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |
| [Inductive Representation Learning on Large Graphs](https://arxiv.org/abs/1706.02216) (GraphSAGE) | NIPS'17 | Context discrimination (JS) | Node classification | [link](https://github.com/williamleif/GraphSAGE) |
| [Strategies for Pre-training Graph Neural Networks](https://arxiv.org/abs/1905.12265) (ContextPred) | ICLR'20 | Contextual subgraph discrimination (CE) | Graph classification; biological function prediction | [link](https://github.com/snap-stanford/pretrain-gnns/) |
| [GraphZoom: A Multi-level Spectral Approach for Accurate and Scalable Graph Embedding](https://arxiv.org/abs/1910.02370) | ICLR'20 | Contextual subgraph discrimination (CE) | Node classification; link prediction | [link](https://github.com/cornell-zhang/GraphZoom) |
| [GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training](https://arxiv.org/abs/2006.09963) | KDD'20 | Contextual subgraph discrimination (InfoNCE) | Node classification; graph classification; similarity search | [link](https://github.com/THUDM/GCC) |
| [Graph Attention Auto-Encoders](https://arxiv.org/abs/1905.10715) (GATE) | ICTAI'20 | Context discrimination (JS) | Node classification | [link](https://github.com/amin-salehi/GATE) |
| [Sub-Graph Contrast for Scalable Self-Supervised Graph Representation Learning](https://arxiv.org/abs/2009.10273) (Subg-Con) | ICDM'20 | Context discrimination (Triplet margin) | Node classification | [link](https://github.com/yzjiao/Subg-Con) |
| [Self-Supervised Graph Transformer on Large-Scale Molecular Data](https://arxiv.org/abs/2007.02835) (GROVER) | NeurIPS'20 | Contextual property prediction | Graph classification; graph regression | [link](https://github.com/tencent-ailab/grover) |
| [Self-supervised Graph-level Representation Learning with Local and Global Structure](https://arxiv.org/abs/2106.04113) (GraphLoG) | ICML'21 | Contextual subgraph discrimination (Other) | Graph classification; biological function prediction | [link](https://github.com/DeepGraphLearning/GraphLoG) |
| [Pre-training on Large-Scale Heterogeneous Graph](https://dl.acm.org/doi/abs/10.1145/3447548.3467396) (PT-HGNN) | KDD'21 | Context discrimination (InfoNCE) | (Heterogeneous) node classification; link prediction | [link](https://github.com/BUPT-GAMMA/PTHGNN) |
| [Transfer Learning of Graph Neural Networks with Ego-graph Information Maximization](https://arxiv.org/abs/2009.05204) (EGI) | NeurIPS'21 | Context discrimination (JS) | Link prediction; structural role identification | [link](https://github.com/GentleZhu/EGI) |
| [Contrastive Laplacian Eigenmaps](https://arxiv.org/abs/2201.05493) (COLES) | NeurIPS'21 | Context discrimination (LE) | Node classification; node clustering | [link](https://github.com/allenhaozhu/COLES) |
| [Graph-MLP: Node Classification without Message Passing in Graph](https://arxiv.org/abs/2106.04051) | arXiv:2106 | Context discrimination (InfoNCE) | Node classification | [link](https://github.com/yanghu819/Graph-MLP) |
| [Augmentation-Free Self-Supervised Learning on Graphs](https://arxiv.org/abs/2112.02472) (AFGRL) | AAAI'22 | Context discrimination (Bootstrapping) | Node classification; node clustering; similarity search | [link](https://github.com/Namkyeong/AFGRL) |
| [Simple Unsupervised Graph Representation Learning](https://ojs.aaai.org/index.php/AAAI/article/view/20748) (SUGRL) | AAAI'22 | Context discrimination (Triplet margin) | Node classification | [link](https://github.com/YujieMo/SUGRL) |
| [SAIL: Self-Augmented Graph Contrastive Learning](https://arxiv.org/abs/2009.00934) | AAAI'22 | Neighbor feature prediction (BPR) | Node classification; node clustering; link prediction | -- |
| [Robust Self-Supervised Structural Graph Neural Network for Social Network Prediction](https://dl.acm.org/doi/abs/10.1145/3485447.3512182) | WWW'22 | Contextual subgraph discrimination (InfoNCE) | Node classification; graph classification; similarity search | -- |
| [Node Representation Learning in Graph via Node-to-Neighbourhood Mutual Information Maximization](https://arxiv.org/abs/2203.12265) (N2N) | CVPR'22 | Context discrimination (InfoNCE) | Node classification | [link](https://github.com/dongwei156/n2n) |
| [RoSA: A Robust Self-Aligned Framework for Node-Node Graph Contrastive Learning](https://arxiv.org/abs/2204.13846) | IJCAI'22 | Contextual subgraph discrimination (InfoNCE) | Node classification | [link](https://github.com/ZhuYun97/RoSA) |
| [Graph Auto-Encoder Via Neighborhood Wasserstein Reconstruction](https://arxiv.org/abs/2202.09025) (NWR-GAE) | ICLR'22 | Context feature prediction | Node classification; structural role identification | [link](https://github.com/mtang724/NWR-GAE) |
| [Towards Self-supervised Learning on Graphs with Heterophily](https://dl.acm.org/doi/abs/10.1145/3511808.3557478) (HGRL) | CIKM'22 | Context discrimination (InfoNCE) | Node classification; node clustering | [link](https://github.com/yifanQi98/HGRL) |
| [Unifying Graph Contrastive Learning with Flexible Contextual Scopes](https://arxiv.org/abs/2210.08792) (UGCL) | ICDM'22 | Context discrimination (InfoNCE) | Node classification | [link](https://github.com/zyzisastudyreallyhardguy/UGCL) |
| [Generalized Laplacian Eigenmaps](https://proceedings.neurips.cc/paper_files/paper/2022/hash/c6b71f8d79d0b2d7bdac66ff3a3ba243-Abstract-Conference.html) (GLEN) | NeurIPS'22 | Context discrimination (LE) | Node classification; node clustering | [link](https://github.com/allenhaozhu/GLEN) |
| [Decoupled Self-supervised Learning for Graphs](https://arxiv.org/abs/2206.03601) (DSSL) | NeurIPS'22 | Context discrimination (Other) | Node classification | [link](https://openreview.net/attachment?id=Bwh6XmDEDe&name=supplementary_material) |
| [Towards Graph Self-Supervised Learning with Contrastive Adjusted Zooming](https://arxiv.org/abs/2111.10698) (G-Zoom) | TNNLS'22 | Context discrimination (JS) | Node classification | -- |
| [Link Prediction with Contextualized Self-Supervision](https://arxiv.org/abs/2201.10069) (CSSL2) | TKDE'22 | Context discrimination (CE) | Link prediction | [link](https://github.com/daokunzhang/CSSL) |
| [Graph Soft-Contrastive Learning via Neighborhood Ranking](https://arxiv.org/abs/2209.13964) (GSCL) | arXiv:2209 | Context discrimination (InfoNCE) | Node classification; node clustering | -- |
| [Localized Graph Contrastive Learning](https://arxiv.org/abs/2212.04604) (Local-GCL) | arXiv:2212 | Context discrimination (InfoNCE) | Node classification | [link](https://openreview.net/attachment?id=dSYkYNNZkV&name=supplementary_material) |
| [Deep Graph Structural Infomax](https://ojs.aaai.org/index.php/AAAI/article/view/25618) (DGSI) | AAAI'23 | Context discrimination (JS) | Node classification | [link](https://github.com/wtzhao1631/dgsi) |
| [Neighbor Contrastive Learning on Learnable Graph Augmentation](https://arxiv.org/abs/2301.01404) (NCLA) | AAAI'23 | Context discrimination (InfoNCE) | Node classification | [link](https://github.com/shenxiaocam/NCLA) |
| [Eliciting Structural and Semantic Global Knowledge in Unsupervised Graph Contrastive Learning](https://arxiv.org/abs/2202.08480) (S3-CL) | AAAI'23 | Contextual subgraph discrimination (InfoNCE) | Node classification; node clustering | [link](https://github.com/kaize0409/S-3-CL) |
| [GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networks](https://arxiv.org/abs/2302.08043); [Generalized Graph Prompt: Toward a Unification of Pre-Training and Downstream Tasks on Graphs](https://arxiv.org/abs/2311.15317) (GraphPrompt+) | WWW'23; TKDE'24 | Context discrimination (InfoNCE), etc | Node classification; graph classification | [link](https://github.com/Starlien95/GraphPrompt) |
| [Contrastive Learning Meets Homophily: Two Birds with One Stone](https://proceedings.mlr.press/v202/he23c.html) (NeCo) | ICML'23 | Context discrimination (InfoNCE) | Node classification | -- |
| [Contrastive Cross-scale Graph Knowledge Synergy](https://dl.acm.org/doi/abs/10.1145/3580305.3599286) (CGKS) | KDD'23 | Context discrimination (LE); contextual subgraph discrimination (InfoNCE) | Node classification; graph classification | -- |
| [Pretraining Language Models with Text-Attributed Heterogeneous Graphs](https://arxiv.org/abs/2310.12580) (THLM) | EMNLP Findings'23 | Context discrimination (InfoNCE) | (Heterogeneous) node classification; link prediction | [link](https://github.com/Hope-Rita/THLM) |
| [GRENADE: Graph-Centric Language Model for Self-Supervised Representation Learning on Text-Attributed Graphs](https://arxiv.org/abs/2310.15109) | EMNLP Findings'23 | Context discrimination (InfoNCE, KL) | Node classification; node clustering; link prediction | [link](https://github.com/bigheiniu/GRENADE) |
| [Simple and Asymmetric Graph Contrastive Learning without Augmentations](https://arxiv.org/abs/2310.18884) (GraphACL) | NeurIPS'23 | Context discrimination (InfoNCE) | Node classification | [link](https://github.com/tengxiao1/GraphACL) |
| [Better with Less: A Data-Active Perspective on Pre-Training Graph Neural Networks](https://arxiv.org/abs/2311.01038) (APT) | NeurIPS'23 | Context discrimination (InfoNCE) | Node classification; graph classification | [link](https://github.com/galina0217/APT) |
| [Dual Contrastive Learning Network for Graph Clustering](https://ieeexplore.ieee.org/document/10097557) (DCLN) | TNNLS'23 | Context discrimination (InfoNCE) | Node classification; node clustering | [link](https://github.com/XinPeng97/TNNLS_DCLN) |
| [Hierarchical Topology Isomorphism Expertise Embedded Graph Contrastive Learning](https://arxiv.org/abs/2312.14222) (HTML) | AAAI'24 | Contextual property prediction (structural coefficient) | Graph classification | [link](https://github.com/jyf123/HTML) |
| [HGPROMPT: Bridging Homogeneous and Heterogeneous Graphs for Few-shot Prompt Learning](https://arxiv.org/abs/2312.01878) | AAAI'24 | Contextual subgraph discrimination (InfoNCE) | (Heterogeneous) node classification; graph classification | [link](https://github.com/Starlien95/HGPrompt) |
| [Graph Contrastive Learning Reimagined: Exploring Universality](https://dl.acm.org/doi/10.1145/3589334.3645480) (ROSEN) | WWW'24 | Context discrimination (InfoNCE) | Node classification; node clustering | -- |
| [High-Frequency-aware Hierarchical Contrastive Selective Coding for Representation Learning on Text-attributed Graphs](https://arxiv.org/abs/2402.16240) (HASH-CODE) | WWW'24 | Context discrimination (SP); contextual subgraph discrimination (SP) | Node classification; link prediction | -- |
| [HeterGCL: Graph Contrastive Learning Framework on Heterophilic Graph](https://www.ijcai.org/proceedings/2024/265) | IJCAI'24 | Context discrimination (InfoNCE) | Node classification; node clustering | [link](https://github.com/Incendio1/HeterGCL) |
| [S3GCL: Spectral, Swift, Spatial Graph Contrastive Learning](https://openreview.net/forum?id=znKAWRZSF9) | ICML'24 | Context discrimination (InfoNCE) | Node classification | [link](https://github.com/GuanchengWan/S3GCL) |
| [Efficient Contrastive Learning for Fast and Accurate Inference on Graphs](https://openreview.net/forum?id=vsy21Xodrt) (GraphECL) | ICML'24 | Context discrimination (InfoNCE) | Node classification | [link](https://github.com/tengxiao1/GraphECL) |
| [Self-Pro: A Self-Prompt and Tuning Framework for Graph Neural Networks](https://arxiv.org/abs/2310.10362) | ECML-PKDD'24 | Context discrimination (InfoNCE) | Node classification; link prediction | [link](https://github.com/gongchenghua/Self-Pro) |
| [Smoothed Graph Contrastive Learning via Seamless Proximity Integration](https://arxiv.org/abs/2402.15270) (SGCL4) | LoG'24 | Context discrimination (cosine similarity) | Node classification; graph classification | [link](https://github.com/maysambehmanesh/SGCL) |
| [FUG: Feature-Universal Graph Contrastive Pre-training for Graphs with Diverse Node Features](https://openreview.net/forum?id=VUuOsBrqaw) | NeurIPS'24 | Context discrimination (MSE) | Node classification | [link](https://github.com/hedongxiao-tju/FUG) |
| [TAGA: Text-Attributed Graph Self-Supervised Learning by Synergizing Graph and Text Mutual Transformations](https://arxiv.org/abs/2405.16800) | arXiv:2405 | Contextual subgraph discrimination (cosine similarity) | Node classification | -- |
| [Single-View Graph Contrastive Learning with Soft Neighborhood Awareness](https://arxiv.org/abs/2412.09261) (SIGNA) | AAAI'25 | Context discrimination (JS) | Node classification; node clustering | [link](https://github.com/sunisfighting/SIGNA) |
| [Refine then Classify: Robust Graph Neural Networks with Reliable Neighborhood Contrastive Refinement](https://ojs.aaai.org/index.php/AAAI/article/view/33471) (GRANCE) | AAAI'25 | Context discrimination (InfoNCE) | Node classification | [link](https://github.com/shumanzhuang/GRANCE) (unavailable) |
| [Balancing Graph Embedding Smoothness in Self-supervised Learning via Information-Theoretic Decomposition](https://openreview.net/forum?id=uNXr6f0ARA) (BSG) | WWW'25 | Context discrimination (MSE, cosine similarity) | Node classification; link prediction | [link](https://anonymous.4open.science/r/BSG-2025) (private) |

</details>

# Long-range similarities

<details open>
 <summary>Long-range similarities</summary>

* Similarity prediction: to predict a similarity matrix between nodes. The pairwise similarity can be defined by shortest path distance, PageRank similarity, Katz index, Jaccard coefficient, $\ell_2$ distance & cosine similarity between output representations / input-output, etc
* Similarity-based discrimination: instance discrimination that is node similarity-aware
* Similarity graph alignment: to construct an additional similarity graph based on pairwise similarities of node features or graph topology, and minimize the distance of representation distributions between them (the original and similarity graph, or two different similarity graphs)

| Paper | Venue | Pretext | Downstream | Code |
| ------------------------------------------------------------ | ---------------------------- | ------------------------------------------------------------ | ----------------------------------------------------------- | ------------------------------------------------------------ |
| [Adaptive Graph Encoder for Attributed Graph Embedding](https://arxiv.org/abs/2007.01594) (AGE) | KDD'20 | Similarity prediction (cosine similarity) | Node clustering; link prediction | [link](https://github.com/thunlp/AGE) |
| [AM-GCN: Adaptive Multi-channel Graph Convolutional Networks](https://arxiv.org/abs/2007.02265) | KDD'20 | Similarity graph alignment | Node classification | [link](https://github.com/zhumeiqiBUPT/AM-GCN) |
| [Graph-Bert: Only Attention is Needed for Learning Graph Representations](https://arxiv.org/abs/2001.05140) | arXiv:2001 | Similarity prediction (PageRank, etc.) | Node classification; node clustering | [link](https://github.com/jwzhanggy/Graph-Bert) |
| [Self-supervised Learning on Graphs: Deep Insights and New Direction](https://arxiv.org/abs/2006.10141) (PairwiseDistance, PairwiseAttrSim) | arXiv:2006 | Similarity prediction (shortest path distance; cosine similarity) | Node classification | [link](https://github.com/ChandlerBang/SelfTask-GNN) |
| [SAIL: Self-Augmented Graph Contrastive Learning](https://arxiv.org/abs/2009.00934) | AAAI'22 | Similarity prediction (cosine similarity) | Node classification; node clustering; link prediction | -- |
| [Co-Modality Graph Contrastive Learning for Imbalanced Node Classification](https://papers.nips.cc/paper_files/paper/2022/hash/65cbe3e21ac62553111d9ecf7d60c18e-Abstract-Conference.html) (CM-GCL) | NeurIPS'22 | Similarity-based discrimination (cosine similarity) | Node classification (imbalanced) | [link](https://github.com/graphprojects/CM-GCL) |
| [Self-Supervised Graph Representation Learning via Global Context Prediction](https://arxiv.org/abs/2003.01604); [A New Self-supervised Task on Graphs: Geodesic Distance Prediction](https://www.sciencedirect.com/science/article/abs/pii/S0020025522006375) (S2GRL) | Information Sciences'22 | Similarity prediction (shortest path distance) | Node classification; node clustering; link prediction | -- |
| [Dual Low-Rank Graph Autoencoder for Semantic and Topological Networks](https://ojs.aaai.org/index.php/AAAI/article/view/25536) (DLR-GAE) | AAAI'23 | Similarity graph alignment | Node classification | [link](https://github.com/chenzl23/DLRGAE) |
| [Attribute and Structure Preserving Graph Contrastive Learning](https://ojs.aaai.org/index.php/AAAI/article/view/25858) (ASP) | AAAI'23 | Similarity graph alignment | Node classification | [link](https://github.com/ JialuChenChina/ASP) |
| [Beyond Smoothing: Unsupervised Graph Representation Learning with Edge Heterophily Discriminating](https://arxiv.org/abs/2211.14065) (GREET) | AAAI'23 | Similarity-based discrimination (cosine similarity) | Node classification | [link](https://github.com/yixinliu233/GREET) |
| [Deep Manifold Graph Auto-Encoder for Attributed Graph Embedding](https://arxiv.org/abs/2401.06727) (DMGAE, DMVGAE) | ICASSP'23 | Similarity prediction ($\ell_2$ distance) | Node clustering; link prediction | -- |
| [Self-Supervised Teaching and Learning of Representations on Graphs](https://dl.acm.org/doi/abs/10.1145/3543507.3583441) (GraphTL) | WWW'23 | Similarity-based discrimination (cosine similarity) | Node classification | -- |
| [Graph Self-supervised Learning via Proximity Divergence Minimization](https://proceedings.mlr.press/v216/zhang23d.html) (PDM) | UAI'23 | Similarity prediction (heat kernel, personalized PageRank, SimRank) | Node classification | [link](https://github.com/tonyzhang617/pdm) |
| [Maximizing Mutual Information Across Feature and Topology Views for Representing Graphs](https://arxiv.org/abs/2105.06715) (MVMI-FT) | TKDE'23 | Similarity graph alignment | Node classification; node clustering | [link](https://github.com/xiaolongo/MaxMIAcrossFT) |
| [Towards Effective and Robust Graph Contrastive Learning With Graph Autoencoding](https://ieeexplore.ieee.org/abstract/document/10158931) (AEGCL) | TKDE'23 | Similarity graph alignment | Node classification; node clustering; link prediction | [link](https://github.com/AllminerLab/Code-for-AEGCL-master) |
| [ULTRA-DP: Unifying Graph Pre-training with Multi-task Graph Dual Prompt](https://arxiv.org/abs/2310.14845) | arXiv:2310 | Similarity prediction (cosine similarity) | Node classification; link prediction | [link](https://github.com/Keytoyze/ULTRA-DP) |
| [Deep Contrastive Graph Learning with Clustering-Oriented Guidance](https://arxiv.org/abs/2402.16012) (DCGL) | AAAI'24 | Similarity graph alignment | Node clustering | [link](https://github.com/drongwbc/DCGL-AAAI24) |
| [E2GCL: Efficient and Expressive Contrastive Learning on Graph Neural Networks](https://ieeexplore.ieee.org/document/10598084/) | ICDE'24 | Similarity-based discrimination | Node classification; graph classification; link prediction | -- |
| [Improving Graph Contrastive Learning via Adaptive Positive Sampling](https://openaccess.thecvf.com/content/CVPR2024/html/Zhuo_Improving_Graph_Contrastive_Learning_via_Adaptive_Positive_Sampling_CVPR_2024_paper.html) (HEATS) | CVPR'24 | Similarity-based discrimination (block diagonal affinity) | Node classification; image classification | -- |
| [ConGraT: Self-Supervised Contrastive Pretraining for Joint Graph and Text Embeddings](https://arxiv.org/abs/2305.14321) | ACL Workshop (TextGraphs)'24 | Similarity-based discrimination (common neighbors, SimRank) | Node classification; link prediction | [link](https://github.com/wwbrannon/congrat) |
| [Enhancing Graph Contrastive Learning with Node Similarity](https://arxiv.org/abs/2208.06743) (SimEnhancedGCL) | KDD'24 | Similarity-based discrimination (cosine similarity, personalized PageRank) | Node classification | [link](https://github.com/frankhlchi/SimEnhancedGCL) |
| [Select Your Own Counterparts: Self-Supervised Graph Contrastive Learning With Positive Sampling](https://ieeexplore.ieee.org/abstract/document/10507017) (GPS) | TNNLS'24 | Similarity-based discrimination (cosine similarity, personalized PageRank, etc) | Node classification | -- |
| [Beyond Homophily: Graph Contrastive Learning with Macro-Micro Message Passing](https://ojs.aaai.org/index.php/AAAI/article/view/33751) (M3P-GCL) | AAAI'25 | Similarity graph alignment | Node classification | -- |
| [Adversarial Contrastive Graph Masked AutoEncoder Against Graph Structure and Feature Dual Attacks](https://ojs.aaai.org/index.php/AAAI/article/view/33364) (ACGMAE) | AAAI'25 | Similarity graph alignment | Node classification; node clustering | -- |
| [UniGraph2: Learning a Unified Embedding Space to Bind Multimodal Graphs](https://openreview.net/forum?id=lEQEKUpXt6) | WWW'25 | Similarity prediction (shortest path distance) | Node classification; link prediction; edge classification | [link](https://github.com/yf-he/UniGraph2) (unavailable) |
| [Distill & Contrast: A New Graph Self-Supervised Method With Approximating Nature Data Relationships](https://ieeexplore.ieee.org/document/10938656/) | TKDE'25 | Similarity graph alignment | Node classification; node clustering | [link](https://github.com/hedongxiao-tju/DC) (unavailable) |

</details>

# Motifs

<details open>
 <summary>Motifs</summary>

* Motif prediction: to assign each node (or supernode in the fragment graph) a motif pseudo-label given by unsupervised motif discovery algorithms (e.g. RDKit) and learn to predict them. It is "autoregressive" if the predicted supernodes are generated one-by-one
* Motif-based masked feature prediction: similar to masked feature prediction, but the features are masked in motifs
* Motif-based discrimination: to perform contrast between the original graph view and the fragment graph view

| Papers                                                       | Venue                             | Pretext                                                | Downstream                             | Code                                                         |
| ------------------------------------------------------------ | --------------------------------- | ------------------------------------------------------ | -------------------------------------- | ------------------------------------------------------------ |
| [Self-Supervised Graph Transformer on Large-Scale Molecular Data](https://arxiv.org/abs/2007.02835) (GROVER) | NeurIPS'20                        | Motif prediction         | Graph classification; graph regression | [link](https://github.com/tencent-ailab/grover)              |
| [Motif-based Graph Self-Supervised Learning for Molecular Property Prediction](https://arxiv.org/abs/2110.00987) (MGSSL) | NeurIPS'21                        | Motif prediction (autoregressive)                      | Graph classification          | [link](https://github.com/zaixizhang/MGSSL)                  |
| [Fragment-based Pretraining and Finetuning on Molecular Graphs](https://arxiv.org/abs/2310.03274) (GraphFP) | NeurIPS'23                        | Motif prediction; motif-based discrimination (InfoNCE) | Graph classification; graph regression | [link](https://github.com/lvkd84/GraphFP)                    |
| [Motif-aware Riemannian Graph Neural Network with Generative-Contrastive Learning](https://arxiv.org/abs/2401.01232) (MotifRGC) | AAAI'24                           | Motif-based discrimination (InfoNCE) | Node classification; link prediction   | [link](https://github.com/RiemannGraph/MotifRGC)             |
| [Empowering Dual-Level Graph Self-Supervised Pretraining with Motif Discovery](https://arxiv.org/abs/2312.11927) (DGPM) | AAAI'24 | Motif prediction | Graph classification | [link](https://github.com/RocccYan/DGPM) |
| [Graph Contrastive Learning with Cohesive Subgraph Awareness](https://arxiv.org/abs/2401.17580) (CTAug) | WWW'24 | Motif-based discrimination (InfoNCE) | Graph classification | [link](https://github.com/wuyucheng2002/CTAug) |
| [Motif-aware Attribute Masking for Molecular Graph Pre-training](https://arxiv.org/abs/2309.04589) (MoAMa) | LoG'24 | Motif-based masked feature prediction                  | Graph classification          | [link](https://github.com/einae-nd/MoAMa-dev)                |
| [Motif-Driven Contrastive Learning of Graph Representations](https://arxiv.org/abs/2012.12533) (MICRO-Graph) | TKDE'24       | Motif-based discrimination (InfoNCE)                   | Graph classification          | [link](https://drive.google.com/file/d/1b751rpnV-SDmUJvKZZI-AvpfEa9eHxo9) |
| [Fine-grained Semantics Enhanced Contrastive Learning for Graphs](https://ieeexplore.ieee.org/abstract/document/10693352) (FSGCL) | TKDE'24 | Motif-based discrimination (Bootstrapping) | Node classification | -- |
| [MORE: Molecule Pretraining with Multi-Level Pretext Task](https://ojs.aaai.org/index.php/AAAI/article/view/34262) | AAAI'25 | Motif prediction | Graph classification | [link](https://github.com/IT-fatica/MORE) |

</details>

# Clusters

<details open>
 <summary>Clusters</summary>

* Synthetic graph discrimination: binary classification between two synthetic graphs with different synthesizers (Erdős-Rényi generator / SBM generator)
* Node clustering: to assign each node a cluster centroid (prototype) and - i) minimize the distance between nodes and their corresponding centroids in the latent space; or ii) minimize the distance between the learned centroids and the ground-truth centroids given by unsupervised feature clustering algorithms (e.g. K-means, DeepCluster)
* Graph partitioning: to assign each node a cluster centroid (prototype) and - i) predict the quality of the learned partitions evaluated by some metrics, e.g. maximizing modularity or minimizing the normalized edge weights of a graph cut (spectral clustering); or ii) predict the cluster membership of each node given by unsupervised graph partitioning algorithms (structure-based, e.g. METIS, Louvain)
* Cluster/partition-based instance discrimination: instance discrimination that is aware of graph clustering/partitioning memberships
* Cluster/partition-conditioned link prediction: to maximize the log-likelihood of existing links, but conditioned by the graph cluster/partition distributions

| Paper | Venue | Pretext | Downstream | Code |
| ------------------------------------------------------------ | ----------------------- | --------------------------------------------------------- | ------------------------------------------------------- | ------------------------------------------------------------ |
| [SGR: Self-Supervised Spectral Graph Representation Learning](https://arxiv.org/abs/1811.06237) | KDD Workshop (DLD)'18 | Synthetic graph discrimination | Graph classification | -- |
| [Unsupervised Pre-training of Graph Convolutional Networks](https://arxiv.org/abs/1905.13728) (ClusterDetect) | ICLR Workshop (RLGM)'19 | Graph partitioning | Node classification | -- |
| [Multi-Stage Self-Supervised Learning for Graph Convolutional Networks on Graphs with Few Labeled Nodes](https://arxiv.org/abs/1902.11038) (M3S) | AAAI'20 | Node clustering | Node classification | [link](https://github.com/datake/M3S) |
| [Collaborative Graph Convolutional Networks: Unsupervised Learning Meets Semi-Supervised Learning](https://ojs.aaai.org/index.php/AAAI/article/view/5843) (CGCN) | AAAI'20 | Partition-conditioned link prediction | Node classification; node clustering | [link](https://github.com/AISKYEYE-TJU/CGCN-AAAI2020) (unavailable) |
| [When Does Self-Supervision Help Graph Convolutional Networks?](https://arxiv.org/abs/2006.09136) (NodeCluster, GraphPar) | ICML'20 | Node clustering; graph partitioning | Node classification | [link](https://github.com/Shen-Lab/SS-GCNs) |
| [CommDGI: Community Detection Oriented Deep Graph Infomax](https://dl.acm.org/doi/10.1145/3340531.3412042) | CIKM'20 | Cluster-based discrimination (JS); graph partitioning | Node clustering | [link](https://github.com/FDUDSDE/CommDGI) |
| [Dirichlet Graph Variational Autoencoder](https://arxiv.org/abs/2010.04408) (DGVAE) | NeurIPS'20 | Partition-conditioned link prediction | Node clustering; graph generation | [link](https://github.com/xiyou3368/DGVAE) |
| [Self-supervised Learning on Graphs: Deep Insights and New Direction](https://arxiv.org/abs/2006.10141) (Distance2Clusters) | arXiv:2006 | Graph partitioning | Node classification | [link](https://github.com/ChandlerBang/SelfTask-GNN) |
| [Mask-GVAE: Blind Denoising Graphs via Partition](https://arxiv.org/abs/2102.04228) | WWW'21 | Graph partitioning; partition-conditioned link prediction | Node clustering; etc | [link](https://github.com/halimiqi/www21) |
| [Self-supervised Graph-level Representation Learning with Local and Global Structure](https://arxiv.org/abs/2106.04113) (GraphLoG) | ICML'21 | Node clustering | Graph classification; biological function prediction | [link](https://github.com/DeepGraphLearning/GraphLoG) |
| [Graph Communal Contrastive Learning](https://arxiv.org/abs/2110.14863) (gCooL) | WWW'22 | Partition-based discrimination (InfoNCE) | Node classification; node clustering | [link](https://github.com/lblaoke/gCooL) |
| [Self-supervised Heterogeneous Graph Pre-training Based on Structural Clustering](https://arxiv.org/abs/2210.10462) (SHGP) | NeurIPS'22 | Graph partitioning | (Heterogeneous) node classification; node clustering | [link](https://github.com/kepsail/SHGP) |
| [Eliciting Structural and Semantic Global Knowledge in Unsupervised Graph Contrastive Learning](https://arxiv.org/abs/2202.08480) (S3-CL) | AAAI'23 | Cluster-based discrimination (InfoNCE) | Node classification; node clustering | [link](https://github.com/kaize0409/S-3-CL) |
| [CSGCL: Community-Strength-Enhanced Graph Contrastive Learning](https://arxiv.org/abs/2305.04658) | IJCAI'23 | Partition-based discrimination (InfoNCE) | Node classification; node clustering; link prediction | [link](https://github.com/HanChen-HUST/CSGCL) |
| [HomoGCL: Rethinking Homophily in Graph Contrastive Learning](https://arxiv.org/abs/2306.09614) | KDD'23 | Node clustering; cluster-based discrimination (InfoNCE) | Node classification; node clustering | [link](https://github.com/wenzhilics/HomoGCL) |
| [CARL-G: Clustering-Accelerated Representation Learning on Graphs](https://arxiv.org/abs/2306.06936) | KDD'23 | Node clustering | Node classification; node clustering; similarity search | [link](https://github.com/willshiao/carl-g) |
| [Towards Alignment-Uniformity Aware Representation in Graph Contrastive Learning](https://dl.acm.org/doi/10.1145/3616855.3635789) (AUAR) | WSDM'24 | Node clustering | Node classification; node clustering | -- |
| [Deep Contrastive Graph Learning with Clustering-Oriented Guidance](https://arxiv.org/abs/2402.16012) (DCGL) | AAAI'24 | Cluster-based discrimination (InfoNCE) | Node clustering | [link](https://github.com/drongwbc/DCGL-AAAI24) |
| [StructComp: Substituting propagation with Structural Compression in Training Graph Contrastive Learning](https://arxiv.org/abs/2312.04865) | ICLR'24 | Partition-based discrimination (JS, InfoNCE, etc.) | Node classification | [link](https://github.com/szzhang17/StructComp) |
| [MARIO: Model Agnostic Recipe for Improving OOD Generalization of Graph Contrastive Learning](https://arxiv.org/abs/2307.13055) | WWW'24 | Cluster-based discrimination | Node classification; graph classification | [link](https://github.com/ZhuYun97/MARIO) |
| [Graph Contrastive Learning with Kernel Dependence Maximization for Social Recommendation](https://dl.acm.org/doi/10.1145/3589334.3645412) (CL-KDM) | WWW'24 | Partition-based discrimination (BPR) | Recommendation | -- |
| [HeterGCL: Graph Contrastive Learning Framework on Heterophilic Graph](https://www.ijcai.org/proceedings/2024/265) | IJCAI'24 | Cluster-based discrimination (MSE) | Node classification; node clustering | [link](https://github.com/Incendio1/HeterGCL) |
| [Community-Invariant Graph Contrastive Learning](https://arxiv.org/abs/2405.01350) (CI-GCL) | ICML'24 | Partition-based discrimination (InfoNCE) | Graph classification; graph regression | [link](https://github.com/ShiyinTan/CI-GCL) |
| [From Coarse to Fine: Enable Comprehensive Graph Self-supervised Learning with Multi-granular Semantic Ensemble](https://openreview.net/forum?id=JnA9IveEwg) (MGSE) | ICML'24 | Node clustering | Graph classification | [link](https://github.com/HoytWen/MGSE) |
| [Revisiting Self-Supervised Heterogeneous Graph Learning from Spectral Clustering Perspective](https://arxiv.org/abs/2412.00742) (SCHOOL) | NeurIPS'24 | Partition-based discrimination (MSE) | (Heterogeneous) node classification; node clustering | [link](https://github.com/YujieMo/SCHOOL) |
| [Motif-Driven Contrastive Learning of Graph Representations](https://arxiv.org/abs/2012.12533) (MICRO-Graph) | TKDE'24 | Graph partitioning | Graph classification | [link](https://drive.google.com/file/d/1b751rpnV-SDmUJvKZZI-AvpfEa9eHxo9) |
| [SOLA-GCL: Subgraph-Oriented Learnable Augmentation Method for Graph Contrastive Learning](https://arxiv.org/abs/2503.10100) | AAAI'25 | Partition-based discrimination (InfoNCE) | Graph classification | -- |

</details>

# Global structure

<details open>
 <summary>Global structure</summary>

* Graph instance discrimination: to discriminate between global representations of different graph views (generally for small-scale graphs)
* Graph dimension discrimination: dimension discrimination of different graph representations
* Node-graph discrimination: instance discrimination between the representation of each node and a global representation vector, usually aggregated from the whole graph by a readout function
* Group discrimination: a simplified node-graph discrimination that binarily classifies if a node belongs to the original or the perturbed graph
* Graph similarity prediction: to predict various kinds of similarity functions between pairs of graphs, e.g. graph kernels (graphlet kernel, random walk kernel, graph edit distance kernel, etc)
* Half-graph matching: to divide each graph into two halves and predict if two halves are from the same original graph

| Paper | Venue | Pretext | Downstream | Code |
| ------------------------------------------------------------ | ---------------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |
| [Pre-training Graph Neural Networks with Kernels](https://arxiv.org/abs/1811.06930) (KernelPred) | arXiv:1811 | Graph similarity prediction | Graph classification | -- |
| [Deep Graph InfoMax](https://arxiv.org/abs/1809.10341) (DGI) | ICLR'19 | Node-graph discrimination (JS) | Node classification | [link](https://github.com/PetarV-/DGI) |
| [InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization](https://arxiv.org/abs/1908.01000) | ICLR'20 | Node-graph discrimination (JS) | Graph classification | [link](https://github.com/fanyun-sun/InfoGraph) |
| [Graph Contrastive Learning with Augmentations](https://arxiv.org/abs/2010.13902) (GraphCL) | NeurIPS'20 | Graph instance discrimination (InfoNCE) | Graph classification | [link](https://github.com/Shen-Lab/GraphCL) |
| [Contrastive Multi-View Representation Learning on Graphs](https://arxiv.org/abs/2006.05582) (MVGRL) | ICML'20 | Node-graph discrimination (JS) | Node classification; graph classification | [link](https://github.com/kavehhassani/mvgrl) |
| [Contrastive Self-supervised Learning for Graph Classification](https://arxiv.org/abs/2009.05923) (CSSL1) | AAAI'21 | Graph instance discrimination (InfoNCE) | Graph classification | -- |
| [SUGAR: Subgraph Neural Network with Reinforcement Pooling and Self-Supervised Mutual Information Mechanism](https://arxiv.org/abs/2101.08170) | WWW'21 | Node-graph discrimination (JS) | Graph classification | [link](https://github.com/RingBDStack/SUGAR) |
| [Pairwise Half-graph Discrimination: A Simple Graph-level Self-supervised Strategy for Pre-training Graph Neural Networks](https://arxiv.org/abs/2110.13567) (PHD); [An Effective Self-Supervised Framework for Learning Expressive Molecular Global Representations to Drug Discovery](https://academic.oup.com/bib/article/22/6/bbab109/6262238?login=false) (MPG) | IJCAI'21; Briefings in Bioinformatics'21 | Half-graph matching | Graph classification | [link](https://github.com/pyli0628/MPG) |
| [Graph Contrastive Learning Automated](https://arxiv.org/abs/2106.07594) (JOAO) | ICML'21 | Graph instance discrimination (InfoNCE) | Graph classification | [link](https://github.com/Shen-Lab/GraphCL_Automated) |
| [Self-supervised Graph-level Representation Learning with Local and Global Structure](https://arxiv.org/abs/2106.04113) (GraphLoG) | ICML'21 | Graph instance discrimination (Other) | Graph classification; biological function prediction | [link](https://github.com/DeepGraphLearning/GraphLoG) |
| [Adversarial Graph Augmentation to Improve Graph Contrastive Learning](https://arxiv.org/abs/2106.05819) (AD-GCL) | NeurIPS'21 | Graph instance discrimination (InfoNCE) | Graph classification | [link](https://github.com/susheels/adgcl) |
| [InfoGCL: Information-Aware Graph Contrastive Learning](https://arxiv.org/abs/2110.15438) | NeurIPS'21 | Graph instance discrimination (Bootstrapping); node-graph discrimination (Bootstrapping) | Node classification; graph classification | -- |
| [Graph Adversarial Self-Supervised Learning](https://proceedings.neurips.cc/paper/2021/hash/7d3010c11d08cf990b7614d2c2ca9098-Abstract.html) (GASSL) | NeurIPS'21 | Graph instance discrimination (Bootstrapping) | Graph classification | [link](https://github.com/LuckyZebra/GASSL) (unavailable) |
| [Disentangled Contrastive Learning on Graphs](https://proceedings.neurips.cc/paper/2021/hash/b6cda17abb967ed28ec9610137aa45f7-Abstract.html) (DGCL) | NeurIPS'21 | Graph instance discrimination (Other) | Graph classification | [link](https://haoyang.li/assets/code/DGCL.zip) |
| [Bringing Your Own View: Graph Contrastive Learning without Prefabricated Data Augmentations](https://arxiv.org/abs/2201.01702) (GraphCL-LP) | WSDM'22 | Graph instance discrimination (InfoNCE) | Graph classification | [link](https://github.com/Shen-Lab/GraphCL_Automated) |
| [Self-Supervised Graph Neural Networks via Diverse and Interactive Message Passing](https://ojs.aaai.org/index.php/AAAI/article/view/20353) (DIMP) | AAAI'22 | Node-graph discrimination (JS) | Node classification; node clustering; graph classification | [link](https://github.com/chensnail/DIMP) |
| [Unsupervised Adversarially Robust Representation Learning on Graphs](https://arxiv.org/abs/2012.02486) (GRV) | AAAI'22 | Node-graph discrimination (JS) | Node classification; node clustering; link prediction | [link](https://github.com/galina0217/robustgraph) |
| [AutoGCL: Automated Graph Contrastive Learning via Learnable View Generators](https://arxiv.org/abs/2109.10259) | AAAI'22 | Graph instance discrimination (InfoNCE) | Graph classification | [link](https://github.com/Somedaywilldo/AutoGCL) |
| [Group Contrastive Self-Supervised Learning on Graphs](https://arxiv.org/abs/2107.09787) (GroupCL; GroupIG) | TPAMI'22 | Graph instance discrimination (JS; contrastive log-ratio upper bound (CLUB)) | Graph classification | -- |
| [Towards Graph Self-Supervised Learning with Contrastive Adjusted Zooming](https://arxiv.org/abs/2111.10698) (G-Zoom) | TNNLS'22 | Node-graph discrimination (JS) | Node classification | -- |
| [SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation](https://arxiv.org/abs/2202.03104) | WWW'22 | Graph instance discrimination (InfoNCE, Bootstrapping) | Graph classification | [link](https://github.com/junxia97/SimGRACE) |
| [Let Invariant Rationale Discovery Inspire Graph Contrastive Learning](https://arxiv.org/abs/2206.07869) (RGCL1) | ICML'22 | Graph instance discrimination (InfoNCE) | Graph classification | [link](https://github.com/lsh0520/RGCL) |
| [M-Mix: Generating Hard Negatives via Multi-sample Mixing for Contrastive Learning](https://dl.acm.org/doi/10.1145/3534678.3539248) | KDD'22 | Graph instance discrimination (InfoNCE) | Node classification; node clustering; graph classification; graph edit distance prediction | [link](https://github.com/Sherrylone/m-mix) |
| [AdaGCL: Adaptive Subgraph Contrastive Learning to Generalize Large-scale Graph Training](https://dl.acm.org/doi/abs/10.1145/3511808.3557228) | CIKM'22 | Node-graph discrimination (JS) | Node classification | [link](https://github.com/YL-wang/CIKM_AdaGCL) |
| [Rethinking and Scaling Up Graph Contrastive Learning: An Extremely Efficient Approach with Group Discrimination](https://arxiv.org/abs/2206.01535) (GGD) | NeurIPS'22 | Group discrimination | Node classification | [link](https://github.com/zyzisastudyreallyhardguy/Graph-Group-Discrimination) |
| [Graph Self-supervised Learning with Accurate Discrepancy Learning](https://arxiv.org/abs/2202.02989) (D-SLA) | NeurIPS'22 | Group discrimination; graph similarity prediction | Graph classification; link prediction | [link](https://github.com/DongkiKim95/D-SLA) |
| [Deep Graph Structural Infomax](https://ojs.aaai.org/index.php/AAAI/article/view/25618) (DGSI) | AAAI'23 | Node-graph discrimination (JS) | Node classification | [link](https://github.com/wtzhao1631/dgsi) |
| [Spectral Augmentation for Self-Supervised Learning on Graphs](https://arxiv.org/abs/2210.00643) (SPAN) | ICLR'23 | Node-graph discrimination (InfoNCE) | Node classification; graph classification; graph regression | [link](https://github.com/Louise-LuLin/GCL-SPAN) |
| [Mole-BERT: Rethinking Pre-training Graph Neural Networks for Molecules](https://openreview.net/forum?id=jevY-DtiZTR) | ICLR'23 | Graph instance discrimination (InfoNCE; Triplet margin) | Graph classification; graph regression | [link](https://github.com/junxia97/Mole-BERT) |
| [Spectral Augmentations for Graph Contrastive Learning](https://arxiv.org/abs/2302.02909) (SGCL1) | AISTATS'23 | Graph instance discrimination (InfoNCE) | Node classification; graph classification; similarity search | -- |
| [Generating Counterfactual Hard Negative Samples for Graph Contrastive Learning](https://arxiv.org/abs/2207.00148) (CGC) | WWW'23 | Graph instance discrimination (InfoNCE) | Graph classification | [link](https://www.dropbox.com/sh/kyf8p9unkhn0r99/AABd33jFBfjGYIkvIqWpuNwYa?dl=0) |
| [Multi-Scale Subgraph Contrastive Learning](https://www.ijcai.org/proceedings/2023/246) (MSSGCL) | IJCAI'23 | Node-graph discrimination (InfoNCE); graph instance discrimination (InfoNCE) | Graph classification | [link](https://github.com/ZhaoYuTJPU/MSSGCL) |
| [Boosting Graph Contrastive Learning via Graph Contrastive Saliency](https://proceedings.mlr.press/v202/wei23c.html) (GCS) | ICML'23 | Graph instance discrimination (InfoNCE) | Graph classification | [link](https://github.com/weicy15/GCS) |
| [SEGA: Structural Entropy Guided Anchor View for Graph Contrastive Learning](https://arxiv.org/abs/2305.04501) | ICML'23 | Graph instance discrimination (InfoNCE) | Graph classification | [link](https://github.com/Wu-Junran/SEGA) |
| [Randomized Schur Complement Views for Graph Contrastive Learning](https://arxiv.org/abs/2306.04004) (rLap) | ICML'23 | Node-graph discrimination (InfoNCE); graph instance discrimination (InfoNCE) | Graph classification | [link](https://github.com/kvignesh1420/rlap) |
| [Graph Self-Contrast Representation Learning](https://arxiv.org/abs/2309.02304) (GraphSC) | ICDM'23 | Graph instance discrimination (Triplet margin); graph dimension discrimination | Graph classification | -- |
| [Graph Contrastive Learning with Stable and Scalable Spectral Encoding](https://openreview.net/forum?id=0kz5RmHxmE) (Sp2GCL) | NeurIPS'23 | Graph instance discrimination (InfoNCE) | Node classification; graph classification; graph regression | [link](https://github.com/bdy9527/Sp2GCL) |
| [Certifiably Robust Graph Contrastive Learning](https://arxiv.org/abs/2310.03312) (RES) | NeurIPS'23 | Graph instance discrimination (InfoNCE) | Graph classification | [link](https://github.com/ventr1c/RES-GCL) |
| [Maximizing Mutual Information Across Feature and Topology Views for Representing Graphs](https://arxiv.org/abs/2105.06715) (MVMI-FT) | TKDE'23 | Node-graph discrimination (JS) | Node classification; node clustering | [link](https://github.com/xiaolongo/MaxMIAcrossFT) |
| [Multi-Scale Self-Supervised Graph Contrastive Learning With Injective Node Augmentation](https://ieeexplore.ieee.org/document/10130468) (MS-CIA) | TKDE'23 | Node-graph discrimination (JS) | Node classification | -- |
| [Hierarchically Contrastive Hard Sample Mining for Graph Self-Supervised Pretraining](https://ieeexplore.ieee.org/abstract/document/10223441) (HCHSM) | TNNLS'23 | Node-graph discrimination (JS) | Node classification; node clustering | [link](https://github.com/WxTu/HCHSM) |
| [Dual Contrastive Learning Network for Graph Clustering](https://ieeexplore.ieee.org/document/10097557) (DCLN) | TNNLS'23 | Node-graph discrimination (JS) | Node classification; node clustering | [link](https://github.com/XinPeng97/TNNLS_DCLN) |
| [HeGCL: Advance Self-Supervised Learning in Heterogeneous Graph-Level Representation](https://ieeexplore.ieee.org/document/10135109) | TNNLS'23 | Node-graph discrimination (JS) | (Heterogeneous) node classification; graph classification | [link](https://github.com/shigen97/HeGCL) |
| [Affinity Uncertainty-Based Hard Negative Mining in Graph Contrastive Learning](https://arxiv.org/abs/2301.13340) (AUGCL) | TNNLS'23 | Graph instance discrimination (InfoNCE) | Graph classification | [link](https://github.com/mala-lab/AUGCL) |
| [Hierarchical Topology Isomorphism Expertise Embedded Graph Contrastive Learning](https://arxiv.org/abs/2312.14222) (HTML) | AAAI'24 | Graph instance discrimination (InfoNCE); graph similarity prediction (Jaccard coef-based isomorphic similarity) | Graph classification | [link](https://github.com/jyf123/HTML) |
| [TopoGCL: Topological Graph Contrastive Learning](https://ojs.aaai.org/index.php/AAAI/article/view/29026) | AAAI'24 | Graph instance discrimination (InfoNCE) | Graph classification | [link](https://github.com/topogclaaai24/TopoGCL) |
| [DiscoGNN: A Sample-Efficient Framework for Self-Supervised Graph Representation Learning](https://ieeexplore.ieee.org/document/10597976) | ICDE'24 | Graph instance discrimination (InfoNCE) | Graph classification; similarity search | [link](https://github.com/junxia97/DiscoGNNs) |
| [Masked Graph Modeling with Multi-View Contrast](https://wujcan.github.io/papers/icde24-gcmae.pdf) (GCMAE2) | ICDE'24 | Graph instance discrimination (InfoNCE) | Node classification; graph classification; link prediction | [link](https://github.com/lyc0930/GCMAE) |
| [SGCL: Semantic-aware Graph Contrastive Learning with Lipschitz Graph Augmentation](https://ieeexplore.ieee.org/document/10597692) (SGCL3) | ICDE'24 | Graph instance discrimination (InfoNCE) | Graph classification | -- |
| [Graph Contrastive Learning with Reinforcement Augmentation](https://www.ijcai.org/proceedings/2024/246) (GA2C) | IJCAI'24 | Graph instance discrimination (InfoNCE) | Graph classification | -- |
| [Disentangled Graph Self-supervised Learning for Out-of-Distribution Generalization](https://openreview.net/forum?id=OS0szhkPmF) (OOD-GCL) | ICML'24 | Graph instance discrimination (InfoNCE) | Graph classification | -- |
| [Uncovering Capabilities of Model Pruning in Graph Contrastive Learning](https://openreview.net/forum?id=N670zjFHLz) (LAMP1) | MM'24 | Graph instance discrimination (InfoNCE) | Graph classification | -- |
| [A Sample-driven Selection Framework: Towards Graph Contrastive Networks with Reinforcement Learning](https://openreview.net/forum?id=AsAbXzRt0a) (GraphSaSe) | MM'24 | Graph instance discrimination (InfoNCE) | Graph classification | [link](https://anonymous.4open.science/r/RL-CDC5/) (unavailable) |
| [Graph Contrastive Learning with Personalized Augmentation](https://arxiv.org/abs/2209.06560) (GPA) | TKDE'24 | Graph instance discrimination (InfoNCE) | Graph classification | [link](https://github.com/qiaoyu-tan/GPA) |
| [Graph Contrastive Learning with Min-Max Mutual Information]() (GCLMI) | Information Sciences'24 | Graph instance discrimination (InfoNCE) | Graph classification | [link](https://github.com/xyhappy/GCLMI) |
| [Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees](https://arxiv.org/abs/2412.16441) (GIT) | arXiv:2412 | Graph instance discrimination (Bootstrapping) | Node classification; graph classification; link prediction; edge classification | [link](https://github.com/Zehong-Wang/GIT) (unavailable) |
| [SAMGPT: Text-free Graph Foundation Model for Multi-domain Pre-training and Cross-domain Adaptation](https://openreview.net/forum?id=eRljAllb2e) | WWW'25 | Graph instance discrimination (InfoNCE) | Node classification; graph classification | [link](https://github.com/blue-soda/SAMGPT) |
| [Graph Self-Supervised Learning with Learnable Structural and Positional Encodings](https://openreview.net/forum?id=fX3UjnmtTt) (StructPosGSSL) | WWW'25 | Graph instance discrimination (InfoNCE); graph dimension discrimination | Graph classification | [link](https://anonymous.4open.science/r/StructPosGSSL-73D0) (private) |

</details>

# Manifolds

<details open>
 <summary>Manifolds</summary>

* Cross-manifold discrimination: to perform instance discrimination between different manifolds (e.g. Euclidean vs. Hyperbolic)
* Hyperbolic masked prediction: to perform masked feature/link prediction in hyperbolic space
* Hyperbolic angle prediction: to pool representations to 2-dimensional angle vectors in a unit hyperbola. These vectors serve as pseudo-labels for regression

</details>

# Multi-task pre-training

<details open>
 <summary>Multi-task pre-training</summary>

* Multi-task learning: to combine a set of different pre-training tasks with bespoke algorithms/architectures

| Paper                                                        | Venue      | Strategy            | Downstream                                                   | Code                                             |
| ------------------------------------------------------------ | ---------- | ------------------- | ------------------------------------------------------------ | ------------------------------------------------ |
| [Adaptive Transfer Learning on Graph Neural Networks](https://arxiv.org/abs/2107.08765) (AUX-TS) | KDD'21     | Multi-task learning | Node classification; link prediction                         | [link](https://github.com/Sean-Huang65/AUX-TS)   |
| [Automated Self-Supervised Learning for Graphs](https://arxiv.org/abs/2106.05470) (AutoSSL) | ICLR'22    | Multi-task learning | Node classification; node clustering                         | [link](https://github.com/ChandlerBang/AutoSSL)  |
| [Automated Graph Self-supervised Learning via Multi-teacher Knowledge Distillation](https://arxiv.org/abs/2210.02099) (AGSSL) | arXiv:2210 | Multi-task learning | Node classification                                          | --                                               |
| [Multi-task Self-supervised Graph Neural Networks Enable Stronger Task Generalization](https://arxiv.org/abs/2210.02016) (ParetoGNN) | ICLR'23    | Multi-task learning | Node classification; node clustering; graph partitioning; link prediction | [link](https://github.com/jumxglhf/ParetoGNN)    |
| [ULTRA-DP: Unifying Graph Pre-training with Multi-task Graph Dual Prompt](https://arxiv.org/abs/2310.14845) | arXiv:2310 | Multi-task learning | Node classification; link prediction                         | [link](https://github.com/Keytoyze/ULTRA-DP)     |
| [Decoupling Weighing and Selecting for Integrating Multiple Graph Pre-training Tasks](https://arxiv.org/abs/2403.01400) (WAS) | ICLR'24    | Multi-task learning | Node classification; graph classification                    | [link](https://github.com/TianyuFan0504/WAS)     |
| [MultiGPrompt for Multi-Task Pre-Training and Prompting on Graphs](https://arxiv.org/abs/2312.03731) | WWW'24     | Multi-task learning | Node classification; graph classification                    | [link](https://github.com/Nashchou/MultiGPrompt) |
| [Exploring Correlations of Self-Supervised Tasks for Graphs](https://arxiv.org/abs/2405.04245) (GraphTCM) | ICML'24    | Multi-task learning | Node classification; link prediction                         | [link](https://github.com/zjunet/GraphTCM)       |
| [UniGM: Unifying Multiple Pre-trained Graph Models via Adaptive Knowledge Aggregation](https://openreview.net/forum?id=BtVxv8wvbj) | MM'24      | Multi-task learning | Graph classification                                         | [link](https://github.com/monica309673/UniGM)    |

</details>

# Downstream tuning

<details open>
 <summary>Downstream tuning</summary>

* Fine-tuning: to jointly learn downstream branches as well as the original pre-trained model. Parameter-efficient fine-tuning (PEFT) only updates part of the pre-trained model, e.g. adapter layers or pruned networks
* Prompting: to construct task-specific prompts as model input for downstream tuning/prompting

| Paper | Venue | Strategy | Downstream | Code |
| ------------------------------------------------------------ | --------------- | ----------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ |
| [Learning to Pre-train Graph Neural Networks](https://ojs.aaai.org/index.php/AAAI/article/view/16552) (L2P-GNN) | AAAI'21 | Fine-tuning | Graph classification | [link](https://github.com/rootlu/L2P-GNN) |
| [Adaptive Transfer Learning on Graph Neural Networks](https://arxiv.org/abs/2107.08765) (AUX-TS) | KDD'21 | Fine-tuning | Node classification; link prediction | [link](https://github.com/Sean-Huang65/AUX-TS) |
| [Fine-Tuning Graph Neural Networks via Graph Topology induced Optimal Transport](https://arxiv.org/abs/2203.10453) (GTOT-Tuning) | IJCAI'22 | Fine-tuning | Graph classification | [link](https://github.com/youjibiying/GTOT-Tuning) |
| [GPPT: Graph Pre-training and Prompt Tuning to Generalize Graph Neural Networks](https://dl.acm.org/doi/10.1145/3534678.3539249) | KDD'22 | Prompting | Node classification | [link](https://github.com/MingChen-Sun/GPPT) |
| [Towards Effective and Generalizable Fine-tuning for Pre-trained Molecular Graph Models](https://www.biorxiv.org/content/10.1101/2022.02.03.479055v1) (MolAug, WordReg) | bioRxiv:2202 | Fine-tuning | Graph classification; graph regression | -- |
| [GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networks](https://arxiv.org/abs/2302.08043); [Generalized Graph Prompt: Toward a Unification of Pre-Training and Downstream Tasks on Graphs](https://arxiv.org/abs/2311.15317) (GraphPrompt+) | WWW'23; TKDE'24 | Prompting | Node classification; graph classification | [link](https://github.com/Starlien95/GraphPrompt) |
| [All in One: Multi-task Prompting for Graph Neural Networks](https://arxiv.org/abs/2307.01504) | KDD'23 | Prompting | Node classification; graph classification; link prediction; edge regression; graph regression | [link](https://github.com/sheldonresearch/ProG) |
| [Virtual Node Tuning for Few-shot Node Classification](https://arxiv.org/abs/2306.06063) (VNT) | KDD'23 | Prompting | Node classification; node clustering | -- |
| [When to Pre-Train Graph Neural Networks? From Data Generation Perspective!](https://arxiv.org/abs/2303.16458) (W2PGNN) | KDD'23 | Fine-tuning | Node classification; graph classification | [link](https://github.com/caoyxuan/W2PGNN) |
| [Universal Prompt Tuning for Graph Neural Networks](https://arxiv.org/abs/2209.15240) (GPF) | NeurIPS'23 | Prompting | Node classification; graph classification; link prediction | [link](https://github.com/zjunet/GPF) |
| [PRODIGY: Enabling In-context Learning Over Graphs](https://arxiv.org/abs/2305.12600) | NeurIPS'23 | Prompting | Node classification; link prediction | [link](https://github.com/snap-stanford/prodigy) |
| [An Empirical Study Towards Prompt-Tuning for Graph Contrastive Pre-Training in Recommendations](https://openreview.net/forum?id=XyAP8ScqLV) (CPTPP) | NeurIPS'23 | Prompting | Recommendation | [link](https://github.com/Haoran-Young/CPTPP) |
| [Contrastive Graph Prompt-tuning for Cross-domain Recommendation](https://arxiv.org/abs/2308.10685) (PGPRec) | TOIS'23 | Prompting | Recommendation | -- |
| [SGL-PT: A Strong Graph Learner with Graph Prompt Tuning](https://arxiv.org/abs/2302.12449) | arXiv:2302 | Prompting | Node classification; graph classification | -- |
| [Deep Prompt Tuning for Graph Transformers](https://arxiv.org/abs/2309.10131) (DeepGPT) | arXiv:2309 | Prompting | Graph classification; graph regression | [link](https://github.com/rezashkv/DeepGPT) |
| [ULTRA-DP: Unifying Graph Pre-training with Multi-task Graph Dual Prompt](https://arxiv.org/abs/2310.14845) | arXiv:2310 | Prompting | Node classification; link prediction | [link](https://github.com/Keytoyze/ULTRA-DP) |
| [Fine-tuning Graph Neural Networks by Preserving Graph Generative Patterns](https://arxiv.org/abs/2312.13583) (G-Tuning) | AAAI'24 | Fine-tuning | Graph classification | [link](https://github.com/zjunet/G-Tuning) |
| [Measuring Task Similarity and Its Implication in Fine-Tuning Graph Neural Networks](https://ojs.aaai.org/index.php/AAAI/article/view/29156) (Bridge-Tune) | AAAI'24 | Fine-tuning | Node classification; link prediction | [link](https://github.com/zjunet/Bridge-Tune) |
| [AdapterGNN: Parameter-Efficient Fine-Tuning Improves Generalization in GNNs](https://arxiv.org/abs/2304.09595) | AAAI'24 | Fine-tuning (PEFT) | Graph classification | [link](https://github.com/Lucius-lsr/AdapterGNN) |
| [G-Adapter: Towards Structure-Aware Parameter-Efficient Transfer Learning for Graph Transformer Networks](https://arxiv.org/abs/2305.10329) | AAAI'24 | Fine-tuning (PEFT) | Graph classification | -- |
| [HGPROMPT: Bridging Homogeneous and Heterogeneous Graphs for Few-shot Prompt Learning](https://arxiv.org/abs/2312.01878) | AAAI'24 | Prompting | (Heterogeneous) node classification; graph classification | [link](https://github.com/Starlien95/HGPrompt) |
| [One for All: Towards Training One Graph Model for All Classification Tasks](https://arxiv.org/abs/2310.00149) (OFA) | ICLR'24 | Prompting | Node classification; graph classification; link prediction | [link](https://github.com/LechengKong/OneForAll) |
| [Search to Fine-tune Pre-trained Graph Neural Networks for Graph-level Tasks](https://arxiv.org/abs/2308.06960) (S2PGNN) | ICDE'24 | Fine-tuning | Graph classification; graph regression | [link](https://anonymous.4open.science/r/code_icde2024-A9CB/) (unavailable) |
| [Endowing Pre-trained Graph Models with Provable Fairness](https://www.arxiv.org/abs/2402.12161) (GraphPAR) | WWW'24 | Fine-tuning (PEFT) | Node classification | [link](https://github.com/BUPT-GAMMA/GraphPAR) |
| [GraphControl: Adding Conditional Control to Universal Graph Pre-trained Models for Graph Domain Transfer Learning](https://arxiv.org/abs/2310.07365) | WWW'24 | Fine-tuning; prompting | Node classification | [link](https://github.com/wykk00/GraphControl) |
| [MultiGPrompt for Multi-Task Pre-Training and Prompting on Graphs](https://arxiv.org/abs/2312.03731) | WWW'24 | Prompting | Node classification; graph classification | [link](https://github.com/Nashchou/MultiGPrompt) |
| [HetGPT: Harnessing the Power of Prompt Tuning in Pre-Trained Heterogeneous Graph Neural Networks](https://arxiv.org/abs/2310.15318) | WWW'24 | Prompting | (Heterogeneous) node classification | -- |
| [GraphPro: Graph Pre-training and Prompt Learning for Recommendation](https://arxiv.org/abs/2311.16716) | WWW'24 | Prompting | Recommendation | [link](https://github.com/HKUDS/GraphPro) |
| [Inductive Graph Alignment Prompt: Bridging the Gap between Graph Pre-training and Inductive Fine-tuning From Spectral Perspective](https://arxiv.org/abs/2402.13556) (IGAP) | WWW'24 | Prompting | Node classification; graph classification | -- |
| [All in One and One for All: A Simple yet Effective Method towards Cross-domain Graph Pretraining](https://arxiv.org/abs/2402.09834) (GCOPE) | KDD'24 | Prompting | Node classification | [link](https://github.com/cshhzhao/GCOPE) |
| [A Novel Prompt Tuning for Graph Transformers: Tailoring Prompts to Graph Topologies](https://dl.acm.org/doi/10.1145/3637528.3671804) (TGPT) | KDD'24 | Prompting | Graph classification; graph regression | [link](https://github.com/jikjn/TGPT) (unavailable) |
| [Pre-Training and Prompting for Few-Shot Node Classification on Text-Attributed Graphs](https://arxiv.org/abs/2407.15431) (P2TAG) | KDD'24 | Prompting | Node classification | [link](https://github.com/THUDM/P2TAG) |
| [Killing Two Birds with One Stone: Cross-modal Reinforced Prompting for Graph and Language Tasks](https://dl.acm.org/doi/10.1145/3637528.3671742) (CMRP) | KDD'24 | Prompting | Node classification; link prediction; graph classification; graph question answering; etc | [link](https://github.com/JohnJiang12138/CMRP) |
| [Self-Pro: A Self-Prompt and Tuning Framework for Graph Neural Networks](https://arxiv.org/abs/2310.10362) | ECML-PKDD'24 | Prompting | Node classification; link prediction | [link](https://github.com/gongchenghua/Self-Pro) |
| [PSP: Pre-Training and Structure Prompt Tuning for Graph Neural Networks](https://arxiv.org/abs/2310.17394) | ECML-PKDD'24 | Prompting | Node classification; graph classification | [link](https://github.com/gqq1210/PSP) |
| [Scalable Multi-Source Pre-training for Graph Neural Networks](https://openreview.net/forum?id=rFRerAPdwI) (LAMP2) | MM'24 | Fine-tuning; prompting | Node classification; link prediction | -- |
| [GFT: Graph Foundation Model with Transferable Tree Vocabulary](https://arxiv.org/abs/2411.06070) | NeurIPS'24 | Fine-tuning | Node classification; graph classification; link prediction | [link](https://github.com/Zehong-Wang/GFT) |
| [Uncovering the Redundancy in Graph Self-supervised Learning Models](https://openreview.net/forum?id=7Ntft3U7jj) (SLIDE) | NeurIPS'24 | Fine-tuning (PEFT) | Node classification | [link](https://github.com/zhlgg/SLIDE) |
| [A Prompt-Based Knowledge Graph Foundation Model for Universal In-Context Reasoning](https://arxiv.org/abs/2410.12288) (KG-ICL) | NeurIPS'24 | Prompting | Knowledge graph reasoning | [link](https://github.com/nju-websoft/KG-ICL) |
| [Delayed Bottlenecking: Alleviating Forgetting in Pre-trained Graph Neural Networks](https://ieeexplore.ieee.org/document/10795193/) (DBP) | TKDE'24 | Fine-tuning | Graph classification; biological function prediction | [link](https://anonymous.4open.science/r/TKDE-DBP) (unavailable) |
| [Subgraph-level Universal Prompt Tuning](https://arxiv.org/abs/2402.10380) (SUPT) | arXiv:2402 | Prompting | Graph classification | -- |
| [UniGraph: Learning a Unified Cross-Domain Foundation Model for Text-Attributed Graphs](https://arxiv.org/abs/2402.13630) | arXiv:2402 | Prompting | Node classification; graph classification; edge classification | [link](https://github.com/yf-he/UniGraph) |
| [GraphAlign: Pretraining One Graph Neural Network on Multiple Graphs via Feature Alignment](https://arxiv.org/abs/2406.02953) | arXiv:2406 | Fine-tuning | Node classification; link prediction | [link](https://github.com/THUDM/GraphAlign) |
| [Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees](https://arxiv.org/abs/2412.16441) (GIT) | arXiv:2412 | Fine-tuning | Node classification; graph classification; link prediction; edge classification | [link](https://github.com/Zehong-Wang/GIT) (unavailable) |
| [Non-Homophilic Graph Pre-Training and Prompt Learning](https://arxiv.org/abs/2408.12594) (ProNoG) | KDD'25 | Prompting | Node classification; graph classification | [link](https://github.com/Jaygagaga/ProNoG) |
| [HePa: Heterogeneous Graph Prompting for All-Level Classification Tasks](https://ojs.aaai.org/index.php/AAAI/article/view/33297) | AAAI'25 | Prompting | (Heterogeneous) node classification; graph classification; edge classification | -- |
| [HeterGP: Bridging Heterogeneity in Graph Neural Networks with Multi-View Prompting](https://ojs.aaai.org/index.php/AAAI/article/view/35496) | AAAI'25 | Prompting | (Heterogeneous) node classification; graph classification | -- |
| [Edge Prompt Tuning for Graph Neural Networks](https://openreview.net/forum?id=92vMaHotTM) (EdgePrompt) | ICLR'25 | Prompting | Node classification; graph classification | [link](https://anonymous.4open.science/r/EdgePrompt-4905) (private) |
| [HG-Adapter: Improving Pre-Trained Heterogeneous Graph Neural Networks with Dual Adapters](https://arxiv.org/abs/2411.01155) | ICLR'25 | Fine-tuning (PEFT) | (Heterogeneous) node classification; node clustering | [link](https://github.com/YujieMo/HG-Adapter) (unavailable) |
| [GOFA: A Generative One-For-All Model for Joint Graph Language Modeling](https://arxiv.org/abs/2407.09709) | ICLR'25 | Prompting | Node classification; link prediction | [link](https://github.com/JiaruiFeng/GOFA) |
| [GraphCLIP: Enhancing Transferability in Graph Foundation Models for Text-Attributed Graphs](https://arxiv.org/abs/2410.10329) | WWW'25 | Prompting | Node classification; link prediction | [link](https://github.com/ZhuYun97/GraphCLIP) |
| [DAGPrompT: Pushing the Limits of Graph Prompting with a Distribution-aware Graph Prompt Tuning Approach](https://arxiv.org/abs/2501.15142) | WWW'25 | Prompting | Node classification; graph classification | [link](https://github.com/Cqkkkkkk/DAGPrompT) |
| [SAMGPT: Text-free Graph Foundation Model for Multi-domain Pre-training and Cross-domain Adaptation](https://arxiv.org/abs/2502.05424) | WWW'25 | Prompting | Node classification; graph classification | [link](https://github.com/blue-soda/SAMGPT) |
| [Fairness-aware Prompt Tuning for Graph Neural Networks](https://openreview.net/forum?id=5T5lHzF8FM) (FPrompt) | WWW'25 | Fine-tuning (PEFT); prompting | Node classification | -- |
| [Instance-Aware Graph Prompt Learning](https://arxiv.org/abs/2411.17676) (IA-GPL) | TMLR'25 | Prompting | Graph classification | [link](https://github.com/lijiazheng0917/IA-GPL) |
| [Graph Prompt Clustering](https://ieeexplore.ieee.org/document/10935718) (GPC) | TPAMI'25 | Prompting | Node clustering | -- |
| [GCoT: Chain-of-Thought Prompt Learning for Graphs](https://arxiv.org/abs/2502.08092) | arXiv:2502 | Prompting | Node classification; graph classification | -- |
</details>

# Graph language models

<details open>
 <summary>Graph language models</summary>

* Autoregressive language modeling (AR): to predict the next token in a sequence based on previous tokens
* Masked language modeling (MLM): to predicts masked tokens based on the context
* Node-text discrimination: to discriminate between textual and graph representations
* Graph-instruction matching: to reorder the list of node text to match the corresponding textual representations
* Label-free node classification: node classification with LLM-generated node pseudo-labels
* SFT: various kinds of supervised fine-tuning strategies, incl. task-specific instruction tuning

> Note: :spider_web: graph model (GNN/GT/...); :robot: language model (LM/LLM); :chart_with_upwards_trend: parameter-efficient fine-tuning module (adapter);  :memo: pure prompting without any parameter update
>
> " :train: Training" refers to the **self-supervised** pre-training phase, mostly for LLMs or other large model components. An empty "Training strategy" mostly indicates that pre-trained LLMs are used and kept frozen. Alternatively, " :notes: Tuning" can refer to all kinds of post-training processes, including fine-tuning/instruction tuning/training of some smaller, auxiliary modules, both supervised and self-supervised.
>
> Example 1: training of TAPE can be divided into **3 phases**:
>
> * LLM pre-training (" :robot::train: Training"): a pre-trained GPT-3.5 is directly utilized
> * LM fine-tuning (" :robot::notes: Tuning"): a pre-trained DeBERTa is fine-tuned with LLM-generated explanations
> * Downstream GNN training (" :spider_web::notes: Tuning"): a RevGAT is trained using the enhanced textual features
>
> Example 2: training of GraphAdapter can also be divided into **3 phases**:
>
> * LM pre-training (" :robot::train: Training"): a pre-trained BERT/RoBERTa/GPT-2/LLaMA 2 is directly utilized
>
> * GNN adapter "pre-training" (" :spider_web::notes: Tuning"): a GraphSAGE and a MLP fusion module are trained with LM frozen
>
>   ( :warning: Side note: "GNN adapters" are considered as part of LLM+GNN architectures instead of a PEFT component.)
>   
> * Instruction tuning (" :spider_web::notes: Tuning"): the GraphSAGE and fusion module are fine-tuned by task-specific prompts

| Paper | Venue | :train:Training strategy | :notes:Tuning strategy | Downstream | Code |
| ------------------------------------------------------------ | ---------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |
| [GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graph](https://arxiv.org/abs/2105.02605) | NeurIPS'21 | :spider_web::robot: Link prediction | -- | Link prediction | [link](https://github.com/microsoft/GraphFormers) |
| [Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Prediction](https://arxiv.org/abs/2111.00064) (GIANT) | ICLR'22 | -- | :robot:Neighborhood prediction | Node classification | [link](https://github.com/amzn/pecos/tree/mainline/examples/giant-xrt) |
| [Learning on Large-scale Text-attributed Graphs via Variational Inference](https://arxiv.org/abs/2210.14709) (GLEM) | ICLR'23 | -- | :spider_web::robot:SFT | Node classification | [link](https://github.com/AndyJZhao/GLEM) |
| [Augmenting Low-Resource Text Classification with Graph-Grounded Pre-training and Prompting](https://arxiv.org/abs/2305.03324) (G2P2) | SIGIR'23 | :spider_web::robot:Node-text discrimination; context discrimination | :robot:SFT | Node classification | [link](https://github.com/WenZhihao666/G2P2) |
| [Patton: Language Model Pretraining on Text-Rich Networks](https://arxiv.org/abs/2305.12268) | ACL'23 | :spider_web::robot:MLM; link prediction | -- | Node classification; link prediction; etc | [link](https://github.com/PeterGriffinJin/Patton) |
| [Graph-Aware Language Model Pre-Training on a Large Graph Corpus Can Help Multiple Graph Applications](https://arxiv.org/abs/2306.02592) (GALM) | KDD'23 | :spider_web::robot:Link prediction | :spider_web::robot:SFT | (Heterogeneous) node classification; link prediction; edge classification | -- |
| [Pretraining Language Models with Text-Attributed Heterogeneous Graphs](https://arxiv.org/abs/2310.12580) (THLM) | EMNLP Findings'23 | :spider_web::robot:MLM; context discrimination | :robot:SFT | (Heterogeneous) node classification; link prediction | [link](https://github.com/Hope-Rita/THLM) |
| [Can Language Models Solve Graph Problems in Natural Language?](https://arxiv.org/abs/2305.10037) (NLGraph) | NeurIPS'23 | -- | :memo: | Graph question answering | [link](https://github.com/Arthur-Heng/NLGraph) |
| [WalkLM: A Uniform Language Model Fine-tuning Framework for Attributed Graph Embedding](https://openreview.net/forum?id=ZrG8kTbt70) | NeurIPS'23 | -- | :robot:MLM | Node classification; link prediction | [link](https://github.com/Melinda315/WalkLM) |
| [Graph-ToolFormer: To Empower LLMs with Graph Reasoning Ability via Prompt Augmented by ChatGPT](https://arxiv.org/abs/2304.11116) | arXiv:2304 | -- | :robot:SFT | Graph question answering | [link](https://github.com/jwzhanggy/Graph_Toolformer) |
| [Can Large Language Models Empower Molecular Property Prediction?](https://arxiv.org/abs/2307.07443) (LLM4Mol) | arXiv:2307 | -- | :robot:SFT | Graph classification | [link](https://github.com/ChnQ/LLM4Mol) |
| [SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learning](https://arxiv.org/abs/2308.02565) | arXiv:2308 | -- | :spider_web::chart_with_upwards_trend:SFT | Node classification; link prediction | [link](https://github.com/vermouthdky/SimTeG) |
| [Prompt-based Node Feature Extractor for Few-shot Learning on Text-Attributed Graphs](https://arxiv.org/abs/2309.02848) (G-Prompt) | arXiv:2309 | -- | :spider_web:MLM | Node classification | -- |
| [GraphText: Graph Reasoning in Text Space](https://arxiv.org/abs/2310.01089) | arXiv:2310 | -- | :memo: | Node classification | [link](https://github.com/AndyJZhao/GraphText) |
| [Disentangled Representation Learning with Large Language Models for Text-Attributed Graphs](https://arxiv.org/abs/2310.18152) (DGTL) | arXiv:2310 | -- | :spider_web:SFT | Node classification | -- |
| [GraphLLM: Boosting Graph Reasoning Ability of Large Language Model](https://arxiv.org/abs/2310.05845) | arXiv:2310 | -- | :spider_web::chart_with_upwards_trend:SFT | Graph question answering | [link](https://github.com/mistyreed63849/Graph-LLM) |
| [Efficient Large Language Models Fine-Tuning On Graphs](https://arxiv.org/abs/2312.04737) (LEADING) | arXiv:2312 | -- | :spider_web::robot:Miscellaneous | Node classification | -- |
| [Language is All a Graph Needs](https://arxiv.org/abs/2308.07134) (InstructGLM) | EACL Findings'24 | -- | :robot:Link prediction; SFT | Node classification | [link](https://github.com/agiresearch/InstructGLM) |
| [Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning](https://arxiv.org/abs/2305.19523) (TAPE) | ICLR'24 | -- | :spider_web:Miscellaneous :robot:SFT | Node classification; link prediction | [link](https://github.com/XiaoxinHe/TAPE) |
| [Label-free Node Classification on Graphs with Large Language Models (LLMs)](https://arxiv.org/abs/2310.04668) (LLM-GNN) | ICLR'24 | -- | :spider_web:Label-free node classification | Node classification; link prediction | [link](https://github.com/CurryTang/LLMGNN) |
| [One for All: Towards Training One Graph Model for All Classification Tasks](https://arxiv.org/abs/2310.00149) (OFA) | ICLR'24 | -- | :spider_web:SFT | Node classification; graph classification; link prediction | [link](https://github.com/LechengKong/OneForAll) |
| [Can GNN be Good Adapter for LLMs?](https://arxiv.org/abs/2402.12984) (GraphAdapter) | WWW'24 | -- | :spider_web:AR; SFT | Node classification | [link](https://github.com/zjunet/GraphAdapter) |
| [GraphTranslator: Aligning Graph Model to Large Language Model for Open-ended Tasks](https://arxiv.org/abs/2402.07197) | WWW'24 | :spider_web:Link prediction | :chart_with_upwards_trend:Node-text discrimination; MLM | Node classification; graph question answering | [link](https://github.com/alibaba/GraphTranslator) |
| [Can we Soft Prompt LLMs for Graph Learning Tasks?](https://arxiv.org/abs/2402.10359) (GraphPrompter) | WWW'24 (short) | -- | :spider_web:SFT | Node classification; link prediction | [link](https://github.com/franciscoliu/graphprompter) |
| [GraphGPT: Graph Instruction Tuning for Large Language Models](https://arxiv.org/abs/2310.13023) | SIGIR'24 | :spider_web::robot:Node-text discrimination | :chart_with_upwards_trend:Graph-instruction matching; SFT | Node classification; link prediction | [link](https://github.com/HKUDS/GraphGPT) |
| [Instruction-based Hypergraph Pretraining](https://arxiv.org/abs/2403.19063) (IHP) | SIGIR'24 | -- | :chart_with_upwards_trend:Link prediction; SFT | Node classification; link prediction | -- |
| [Efficient Tuning and Inference for Large Language Models on Textual Graphs](https://arxiv.org/abs/2401.15569) (ENGINE) | IJCAI'24 | -- | :spider_web:SFT | Node classification; link prediction | [link](https://github.com/ZhuYun97/ENGINE) |
| [LLaGA: Large Language and Graph Assistant](https://arxiv.org/abs/2402.08170) | ICML'24 | -- | :chart_with_upwards_trend:SFT | Node classification; link prediction | [link](https://github.com/VITA-Group/LLaGA) |
| [InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignment](https://arxiv.org/abs/2402.08785) | ACL Findings'24 | -- | :chart_with_upwards_trend:SFT | Node classification; link prediction; graph question answering; recommendation; etc | [link](https://github.com/wjn1996/InstructGraph) |
| [ConGraT: Self-Supervised Contrastive Pretraining for Joint Graph and Text Embeddings](https://arxiv.org/abs/2305.14321) | ACL Workshop (TextGraphs)'24 | :spider_web::robot:Node-text discrimination | -- | Node classification; link prediction | [link](https://github.com/wwbrannon/congrat) |
| [GAugLLM: Improving Graph Contrastive Learning for Text-Attributed Graphs with Large Language Models](https://arxiv.org/abs/2406.11945) | KDD'24 | -- | :spider_web:Node instance discrimination / masked feature prediction :robot:Neighborhood prediction | Node classification | [link](https://github.com/NYUSHCS/GAugLLM) |
| [HiGPT: Heterogeneous Graph Language Model](https://arxiv.org/abs/2402.16024) | KDD'24 | :spider_web::robot:Node-text discrimination | :chart_with_upwards_trend:Graph-instruction matching; SFT | (Heterogeneous) node classification | [link](https://github.com/HKUDS/HiGPT) |
| [GraphWiz: An Instruction-Following Language Model for Graph Problems](https://arxiv.org/abs/2402.16029) | KDD'24 | -- | :robot:SFT | Graph question answering | [link](https://github.com/nuochenpku/Graph-Reasoning-LLM) |
| [Pre-Training and Prompting for Few-Shot Node Classification on Text-Attributed Graphs](https://arxiv.org/abs/2407.15431) (P2TAG) | KDD'24 | :spider_web::robot:MLM | :spider_web:SFT | Node classification | [link](https://github.com/THUDM/P2TAG) |
| [Killing Two Birds with One Stone: Cross-modal Reinforced Prompting for Graph and Language Tasks](https://dl.acm.org/doi/10.1145/3637528.3671742) (CMRP) | KDD'24 | -- | :spider_web::chart_with_upwards_trend:SFT | Node classification; link prediction; graph classification; graph question answering; etc | [link](https://github.com/JohnJiang12138/CMRP) |
| [Distilling Large Language Models for Text-Attributed Graph Learning](https://arxiv.org/abs/2402.12022) | CIKM'24 | -- | :spider_web:Node instance discrimination; label-free node classification | Node classification | -- |
| [OpenGraph: Towards Open Graph Foundation Models](https://arxiv.org/abs/2403.01121) | EMNLP Findings'24 | -- | :spider_web:Masked link prediction | Node classification; link prediction | [link](https://github.com/HKUDS/OpenGraph) |
| [Let’s Ask GNN: Empowering Large Language Model for Graph In-Context Learning](https://arxiv.org/abs/2410.07074) (AskGNN) | EMNLP Findings'24 | -- | :spider_web:Node instance discrimination; SFT | Node classification | -- |
| [A Pure Transformer Pretraining Framework on Text-attributed Graphs](https://arxiv.org/abs/2406.13873) (GSPT) | LoG'24 | :robot:Masked feature prediction | -- | Node classification; link prediction | [link](https://github.com/SongYYYY/GSPT) (unavailable) |
| [LLMs as Zero-shot Graph Learners: Alignment of GNN Represetantions with LLM Token Embeddings](https://www.arxiv.org/abs/2408.14512) (TEA-GLM) | NeurIPS'24 | -- | :spider_web:Node instance discrimination; dimension discrimination :chart_with_upwards_trend:SFT | Node classification; link prediction | -- |
| [Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs](https://arxiv.org/abs/2307.03393) (KEA) | KDD Explor. Newsl.'24 | -- | :spider_web:Miscellaneous :robot:SFT | Node classification | [link](https://github.com/CurryTang/Graph-LLM) |
| [Let Your Graph Do the Talking: Encoding Structured Data for LLMs](https://arxiv.org/abs/2402.05862) (GraphToken) | arXiv:2402 | -- | :spider_web:SFT | Graph question answering | [link](https://github.com/google-research/talk-like-a-graph) |
| [UniGraph: Learning a Unified Cross-Domain Foundation Model for Text-Attributed Graphs](https://arxiv.org/abs/2402.13630) | arXiv:2402 | :spider_web::robot:MLM; node instance discrimination | :chart_with_upwards_trend:SFT | Node classification; graph classification; edge classification | [link](https://github.com/yf-he/UniGraph) |
| [GraphEdit: Large Language Models for Graph Structure Learning](https://arxiv.org/abs/2402.15183) | arXiv:2402 | -- | :spider_web::chart_with_upwards_trend:SFT | Node classification | [link](https://github.com/HKUDS/GraphEdit) |
| [Similarity-based Neighbor Selection for Graph LLMs](https://arxiv.org/abs/2402.03720) (SNS) | arXiv:2402 | -- | :memo: | Node classification | [link](https://github.com/ruili33/SNS) |
| [GraphInstruct: Empowering Large Language Models with Graph Understanding and Reasoning Capability](https://arxiv.org/abs/2403.04483) (GraphLM; GraphLM+) | arXiv:2403 | -- | :chart_with_upwards_trend:SFT | Graph question answering | [link](https://github.com/CGCL-codes/GraphInstruct) |
| [TAGA: Text-Attributed Graph Self-Supervised Learning by Synergizing Graph and Text Mutual Transformations](https://arxiv.org/abs/2405.16800) | arXiv:2405 | -- | :spider_web:Node-text discrimination | Node classification | -- |
| [Improving Molecule-Language Alignment with Hierarchical Graph Tokenization](https://arxiv.org/abs/2406.14021) (HIGHT) | arXiv:2406 | -- | :chart_with_upwards_trend:Masked feature prediction; SFT | Graph classification; etc | [link](https://higraphllm.github.io/) (unavailable) |
| [Path-LLM: A Shortest-Path-based LLM Learning for Unified Graph Representation](https://arxiv.org/abs/2408.05456) | arXiv:2408 | :chart_with_upwards_trend:AR | -- | Node classification; link prediction | -- |
| [Graph Reasoning with Large Language Models via Pseudo-code Prompting](https://arxiv.org/abs/2409.17906) | arXiv:2409 | -- | :memo: | Graph question answering | [link](https://github.com/y3nk0/graph-reasoning-llms) |
| [GUNDAM: Aligning Large Language Models with Graph Understanding](https://arxiv.org/abs/2409.20053) | arXiv:2409 | -- | :robot:Similarity prediction; SFT | Graph question answering | [link](https://anonymous.4open.science/r/GUNDAM) (unavailable) |
| [How to Make LLMs Strong Node Classifiers?](https://arxiv.org/abs/2410.02296) (AuGLM) | arXiv:2410 | -- | :robot:SFT | Node classification | [link](https://openreview.net/attachment?id=GURRWHkPtx&name=supplementary_material) |
| [LLaSA: Large Language and Structured Data Assistant](https://arxiv.org/abs/2411.14460) | arXiv:2411 | :spider_web::robot:AR; node-text discrimination | :chart_with_upwards_trend:SFT | Graph question answering | -- |
| [GraphAgent: Agentic Graph Language Assistant](https://arxiv.org/abs/2412.17029) | arXiv:2412 | -- | :chart_with_upwards_trend:Graph-instruction matching; SFT | Node classification; graph question answering | [link](https://github.com/HKUDS/GraphAgent) |
| [LOGIN: A Large Language Model Consulted Graph Neural Network Training Framework](https://arxiv.org/abs/2405.13902) | WSDM'25 | -- | :spider_web:SFT | Node classification | [link](https://github.com/QiaoYRan/LOGIN) |
| [Leveraging Large Language Models for Node Generation in Few-Shot Learning on Text-Attributed Graphs](https://arxiv.org/abs/2310.09872) (ENG; LLM4NG) | AAAI'25 | -- | :spider_web:Miscellaneous :chart_with_upwards_trend:Link prediction | Node classification | [link](https://github.com/jianxiangyu/LLM4NG) |
| [Bootstrapping Heterogeneous Graph Representation Learning via Large Language Models: A Generalized Approach](https://arxiv.org/abs/2412.08038) (GHGRL) | AAAI'25 | -- | :spider_web:SFT | (Heterogeneous) node classification | [link](https://github.com/zch65458525/GHGRL) |
| [Bridging Molecular Graphs and Large Language Models](https://arxiv.org/abs/2503.03135) (Graph2Token) | AAAI'25 | :spider_web::robot:Node-text discrimination | :spider_web:SFT | Graph classification; graph regression | [link](https://github.com/GraphMoLab/Graph2Token) |
| [Large Language Model Meets Graph Neural Network in Knowledge Distillation](https://arxiv.org/abs/2402.05894) (LinguGKD) | AAAI'25 | -- | :spider_web:Node instance discrimination; SFT :chart_with_upwards_trend:SFT | Node classification | -- |
| [TGLsta: Low-resource Textual Graph Learning with Semantic and Topological Awareness via LLMs](https://ojs.aaai.org/index.php/AAAI/article/view/34408) | AAAI'25 | -- | :spider_web::robot:Node-text discrimination | Node classification | -- |
| [Can Large Language Models Improve the Adversarial Robustness of Graph Neural Networks?](https://arxiv.org/abs/2408.08685) (LLM4RGNN) | KDD'25 | -- | :spider_web:Miscellaneous :chart_with_upwards_trend:Link prediction | Node classification | [link](https://github.com/BUPT-GAMMA/LLM4RGNN) |
| [Large Language Models as Topological Structure Enhancers for Text-Attributed Graphs](https://arxiv.org/abs/2311.14324) (LLM4GraphTopology) | DASFAA'25 | -- | :spider_web::robot:SFT | Node classification | [link](https://github.com/sunshy-1/LLM4GraphTopology) |
| [GOFA: A Generative One-For-All Model for Joint Graph Language Modeling](https://arxiv.org/abs/2407.09709) | ICLR'25 | -- | :spider_web:AR; similarity prediction; common neighbor prediction; SFT | Node classification; link prediction | [link](https://github.com/JiaruiFeng/GOFA) |
| [Scale-Free Graph-Language Models](https://arxiv.org/abs/2502.15189) (SFGL) | ICLR'25 | -- | :spider_web:Miscellaneous :robot:SFT | Node classification | -- |
| [Can LLMs Convert Graphs to Text-Attributed Graphs?](https://arxiv.org/abs/2412.10136) (TANS) | NAACL'25 | -- | :spider_web:Miscellaneous | Node classification | [link](https://github.com/Zehong-Wang/TANS) |
| [Exploring the Potential of Large Language Models for Heterophilic Graphs](https://arxiv.org/abs/2408.14134) (LLM4HeG) | NAACL'25 | -- | :spider_web::chart_with_upwards_trend:SFT | Node classification | [link](https://github.com/honey0219/LLM4HeG) |
| [GraphICL: Unlocking Graph Learning Potential in LLMs through Structured Prompt Design](https://arxiv.org/abs/2501.15755) | NAACL Findings'25 | -- | :memo: | Node classification; link prediction | -- |
| [GraphCLIP: Enhancing Transferability in Graph Foundation Models for Text-Attributed Graphs](https://arxiv.org/abs/2410.10329) | WWW'25 | -- | :spider_web:Node-text discrimination; SFT | Node classification; link prediction | [link](https://github.com/ZhuYun97/GraphCLIP) |
| [Leveraging Large Language Models for Effective Label-free Node Classification in Text-Attributed Graphs](https://arxiv.org/abs/2412.11983) (Cella; Locle) | SIGIR'25 | -- | :spider_web:Label-free node classification | Node classification; node clustering | [link](https://github.com/HKBU-LAGAS/Locle) |
| [HierPromptLM: A Pure PLM-based Framework for Representation Learning on Heterogeneous Text-rich Networks](https://arxiv.org/abs/2501.12857) | arXiv:2501 | -- | :robot:MLM; link prediction | (Heterogeneous) node classification; link prediction | -- |
| [Model Generalization on Text Attribute Graphs: Principles with Large Language Models](https://arxiv.org/abs/2502.11836) (LLM-BP) | arXiv:2502 | -- | :memo: | Node classification; link prediction | [link](https://github.com/Graph-COM/LLM_BP) |
| [Are Large Language Models In-Context Graph Learners?](https://arxiv.org/abs/2502.13562) (QueryRAG, LabelRAG, FewshotRAG) | arXiv:2502 | -- | :memo: | Node classification | [link](https://github.com/EdisonLeeeee/In-Context-RAG) |
| [GraphiT: Efficient Node Classification on Text-Attributed Graphs with Prompt Optimized LLMs](https://arxiv.org/abs/2502.10522) | arXiv:2502 | -- | :memo: | Node classification | -- |
| [Exploring the Potential of Large Language Models as Predictors in Dynamic Text-Attributed Graphs](https://arxiv.org/abs/2503.03258) (GAD) | arXiv:2503 | -- | :chart_with_upwards_trend:SFT | Node classification; edge classification; node retrieval | -- |
| [LLM as GNN: Graph Vocabulary Learning for Text-Attributed Graph Foundation Models](https://arxiv.org/abs/2503.03313) (PromptGFM) | arXiv:2503 | -- | :robot:SFT | Node classification; link prediction | [link](https://github.com/agiresearch/PromptGFM) (unavailable) |
| [Unifying Text Semantics and Graph Structures for Temporal Text-attributed Graphs with Large Language Models](https://arxiv.org/abs/2503.14411) (Cross) | arXiv:2503 | -- | :spider_web::robot:Link prediction | Node classification; (temporal) link prediction | -- |
| [Rethinking Graph Structure Learning in the Era of LLMs](https://arxiv.org/abs/2503.21223) (LLaTA) | arXiv:2503 | -- | :spider_web:SFT | Node classification; node clustering | -- |
| [Toward General and Robust LLM-enhanced Text-attributed Graph Learning](https://arxiv.org/abs/2504.02343) (UltraTAG, UltraTAG-S) | arXiv:2504 | -- | :spider_web::robot:SFT | Node classification | -- |
</details>

# Have we fully understood graphs?

:heart: Contributions by issues and pull requests to this source list are always welcome! Feel free to initiate a discussion with me, or give me a reminder if there are oversights of papers/hyperlinks or categorical mistakes.

Pretext

Pretext 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Pretext

Pretext 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README