KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #dsh-plugin #developer-tools #ai-agent #claude-code #codex #typescript #ai-agents #deepseek-harness #mcp #cli #rust

Awesome-Diffusion-LLM

★ 170 Open GitHub ↗

paper list, tutorial, and nano code snippet for Diffusion Large Language Models.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<div align="center">
  
# Awesome Diffusion Large Language Models

[![Awesome](https://awesome.re/badge.svg)](https://github.com/XiaoYee/Awesome_Efficient_LRM_Reasoning) 
[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)
![](https://img.shields.io/github/last-commit/yczhou001/Awesome-Diffusion-LLM?color=green)

</div>

This repository contains：
  - A regularly updated paper list for Diffusion Large Language Models.
  - A tutorial for Diffusion Large Language Models.
  - A nano code snippet for Diffusion Large Language Models.

## Nano code
We have released a simple implementation in `code` folder. It contains two types of diffusion language models, presented as Jupyter notebooks:
- `continuous_diff.ipynb`: This notebook demonstrates a Continuous Diffusion Language Model, illustrating how diffusion modeling and sampling operate in a continuous space, such as word vector space. It could be a start point for grasping the fundamental principles of continuous diffusion language models.
- `masked_diff.ipynb`: This notebook implements a Mask-based Discrete Diffusion Language Model, which works in the discrete token space and supports text generation tasks. We draw inspiration from [GUIDELINES.md](https://github.com/ML-GSAI/LLaDA/blob/main/GUIDELINES.md) of [LLaDA](https://github.com/ML-GSAI/LLaDA) for parts of our implementation. Thanks for their valuable contributions!

## Playground
- [Meucury](https://chat.inceptionlabs.ai/)
- [Dream7B](https://huggingface.co/spaces/multimodalart/Dream)
- [LLaDA8B](https://huggingface.co/spaces/multimodalart/LLaDA)

## Product/ Blog / Technical Report
- Gemini Diffusion [blog](https://deepmind.google/models/gemini-diffusion/) 
- Mercury: Ultra-Fast Language Models Based on Diffusion [tech report](https://drive.google.com/file/d/1xrqTqF88OZblf0NgMjr1REU4doYlkNXf/view)
- Dream7B [blog](https://hkunlp.github.io/blog/2025/dream/)

## Diffusion Language Model for Multimodal Understanding

- [LaViDa: A Large Diffusion Language Model for Multimodal Understanding](https://arxiv.org/pdf/2505.16839) ![](https://img.shields.io/badge/abs-2025.05-red)
- [MMaDA: Multimodal Large Diffusion Language Models](https://arxiv.org/pdf/2505.15809) ![](https://img.shields.io/badge/abs-2025.05-red)
- [LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuning](https://arxiv.org/pdf/2505.16933) ![](https://img.shields.io/badge/abs-2025.05-red)
- [Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding](https://arxiv.org/pdf/2505.16990) (arxiv)![](https://img.shields.io/badge/abs-2025.05-red)
- [Unified Multimodal Discrete Diffusion](https://arxiv.org/pdf/2503.20853) (arxiv)![](https://img.shields.io/badge/abs-2025.03-red)  [[ **CODE** ](https://github.com/alexanderswerdlow/unidisc)]
- [DiffusionVL: Translating Any Autoregressive Models into Diffusion Vision Language Models](https://arxiv.org/pdf/2512.15713) (arxiv)![](https://img.shields.io/badge/abs-2025.12-red)  [[ **CODE** ](https://github.com/hustvl/DiffusionVL)]

## Discrete Diffusion Model
- [Esoteric Language Models](https://arxiv.org/pdf/2506.01928) (arxiv)![](https://img.shields.io/badge/abs-2025.06-red) [[ **CODE** ](https://github.com/s-sahoo/Eso-LMs)]
- [Accelerating Diffusion LLMs via Adaptive Parallel Decoding](https://arxiv.org/pdf/2506.00413) (arxiv)![](https://img.shields.io/badge/abs-2025.06-red)
- [DINGO: Constrained Inference for Diffusion LLMs](https://arxiv.org/pdf/2505.23061) (arxiv)![](https://img.shields.io/badge/abs-2025.05-red)
- [Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding](https://arxiv.org/pdf/2505.22618) (arxiv)![](https://img.shields.io/badge/abs-2025.05-red) [[ **CODE** ](https://github.com/NVlabs/Fast-dLLM)]
- [LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models](https://arxiv.org/pdf/2505.19223) (arxiv)![](https://img.shields.io/badge/abs-2025.05-red)
- [Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion](https://arxiv.org/pdf/2505.21467) (arxiv)![](https://img.shields.io/badge/abs-2025.05-red)
- [Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective](https://arxiv.org/pdf/2505.15045) (arxiv)![](https://img.shields.io/badge/abs-2025.05-red)
- [dKV-Cache: The Cache for Diffusion Language Models](https://arxiv.org/pdf/2505.15781) (arxiv)![](https://img.shields.io/badge/abs-2025.05-red)
- [CtrlDiff: Boosting Large Diffusion Language Models with Dynamic Block Prediction and Controllable Generation](https://arxiv.org/pdf/2505.14455) (arxiv) ![](https://img.shields.io/badge/abs-2025.05-red)
- [Insertion Language Models: Sequence Generation with Arbitrary-Position Insertions](https://arxiv.org/pdf/2505.05755) (arxiv) ![](https://img.shields.io/badge/abs-2025.05-red)
- [d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning](https://arxiv.org/abs/2504.12216) (arxiv) ![](https://img.shields.io/badge/abs-2025.04-red)
- [Target Concrete Score Matching: A Holistic Framework for Discrete Diffusion](https://arxiv.org/pdf/2504.16431) (arxiv) ![](https://img.shields.io/badge/abs-2025.04-red)
- [Remasking Discrete Diffusion Models with Inference-Time Scaling](https://arxiv.org/pdf/2503.00307) (arxiv)![](https://img.shields.io/badge/abs-2025.03-red)
- [Large Language Diffusion Models](https://arxiv.org/pdf/2502.09992) (arxiv) ![](https://img.shields.io/badge/abs-2025.02-red) [<span style="color: red;">code</span>](https://github.com/ML-GSAI/LLaDA)
- [Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models](https://arxiv.org/pdf/2503.09573) (ICLR 2025 Oral) - [[💻]] [**Code**](https://github.com/kuleshov-group/bd3lms)
- [Beyond Autoregression: Fast LLMs via Self-Distillation Through Time](https://arxiv.org/pdf/2410.21035) (ICLR 2025)
- [Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data](https://arxiv.org/pdf/2406.03736) (ICLR 2025)
- [Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling](https://arxiv.org/pdf/2409.02908) (ICLR 2025)
- [Scaling Diffusion Language Models via Adaptation from Autoregressive Models](https://arxiv.org/pdf/2410.17891) (ICLR 2025)
- [Scaling up Masked Diffusion Models on Text](https://arxiv.org/pdf/2410.18514) (ICLR 2025)
- [Speculative Diffusion Decoding: Accelerating Language Generation through Diffusion](https://arxiv.org/pdf/2408.05636) (NAACL 2025)
- [Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution](https://arxiv.org/pdf/2310.16834) (ICML 2024 best paper)
- [Discrete Flow Matching](https://arxiv.org/pdf/2407.15595) (NeurIPS 2024)
- [Simple and Effective Masked Diffusion Language Models](https://arxiv.org/pdf/2406.07524) (NeurIPS 2024)
- [Simplified and Generalized Masked Diffusion for Discrete Data](https://arxiv.org/pdf/2406.04329) (NeurIPS 2024)
- [A Reparameterized Discrete Diffusion Model for Text Generation](https://arxiv.org/pdf/2302.05737) (COLM 2024)
- [Diffusion-NAT: Self-Prompting Discrete Diffusion for Non-Autoregressive Text Generation](https://aclanthology.org/2024.eacl-long.86.pdf) (EACL 2024)
- [Diffusion Glancing Transformer for Parallel Sequence-to-Sequence Learning](https://aclanthology.org/2024.naacl-long.271.pdf) (NAACL 2024)
- [Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning](https://arxiv.org/pdf/2308.12219) (arxiv 2024)
- [DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models](https://arxiv.org/pdf/2211.15029) (ACL 2023)
- [DiffusER: Discrete Diffusion via Edit-based Reconstruction](https://arxiv.org/pdf/2210.16886) (ICLR 2023)
- [Likelihood-Based Diffusion Language Models](https://openreview.net/pdf?id=e2MCL6hObn) (NeurIPS 2023)
- [A Cheaper and Better Diffusion Language Model with Soft-Masked Noise](https://arxiv.org/pdf/2304.04746) (EMNLP 2023)
- [A Continuous Time Framework for Discrete Denoising Models](https://arxiv.org/pdf/2205.14987) (NeurIPS 2022)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://arxiv.org/pdf/2107.03006) (NeurIPS 2021)
- [Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions](https://arxiv.org/pdf/2102.05379) (NeurIPS 2021)

## Continuous-Time Diffusion Model
- [Unifying Continuous and Discrete Text Diffusion with Non-simultaneous Diffusion Processes](https://arxiv.org/pdf/2505.22165) (ACL2025)
- [TESS 2: A Large-Scale Generalist Diffusion Language Model](https://arxiv.org/pdf/2502.13917) (arxiv) ![](https://img.shields.io/badge/abs-2025.02-red)
- [Difformer: Empowering Diffusion Models on the Embedding Space for Text Generation](https://aclanthology.org/2024.naacl-long.261.pdf) (NAACL2024)
- [Diffusion Guided Language Modeling](https://aclanthology.org/2024.findings-acl.887.pdf) (ACL 2024)
- [TESS: Text-to-Text Self-Conditioned Simplex Diffusion](https://aclanthology.org/2024.eacl-long.144.pdf) (EACL2024)
- [Transfer Learning for Text Diffusion Models](https://arxiv.org/pdf/2401.17181) (arxiv 2024)
- [Latent Diffusion for Language Generation](https://openreview.net/pdf?id=NKdtztladR) (NeurIPS 2023)
- [AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation](https://arxiv.org/pdf/2305.09515) (NeurIPS 2023)
- [Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoise](https://arxiv.org/pdf/2212.11685) (ICML 2023)
- [DiffuSeq-v2: Bridging Discrete and Continuous Text Spaces for Accelerated Seq2Seq Diffusion Models](https://aclanthology.org/2023.findings-emnlp.660.pdf) (EMNLP 2023)
- [DeTiME: Diffusion-Enhanced Topic Modeling using Encoder-decoder based LLM](https://arxiv.org/pdf/2310.15296) (EMNLP 2023)
- [How Does Diffusion Influence Pretrained Language Models on Out-of-Distribution Data?](https://arxiv.org/pdf/2307.13949) (ECAI 2023)
- [SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control](https://aclanthology.org/2023.acl-long.647.pdf) (ACL 2023)
- [Glyphdiffusion: Text generation as image generation](https://arxiv.org/pdf/2304.12519) (arxiv 2023)
- [Dinoiser: Diffused conditional sequence learning by manipulating noises](https://arxiv.org/pdf/2302.10025) (arxiv 2023)
- [Diffusion-LM Improves Controllable Text Generation](https://openreview.net/pdf?id=3s9IrEsjLyk) (NeurIPS 2022)
- [Latent Diffusion Energy-Based Model for Interpretable Text Modeling](https://proceedings.mlr.press/v162/yu22h/yu22h.pdf) (ICML2022)
- [DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models](https://openreview.net/pdf?id=jQj-_rLVXsj) (ICLR 2022)
- [Continuous diffusion for categorical data](https://arxiv.org/pdf/2211.15089) (arxiv 2022)
- [Seqdiffuseq: Text diffusion with encoder-decoder transformers](https://arxiv.org/pdf/2212.10325) (arxiv 2022)

## Application
- [Reinforcing the Diffusion Chain of Lateral Thought with Diffusion Language Models](https://arxiv.org/pdf/2505.10446) (arxiv)![](https://img.shields.io/badge/abs-2025.05-red)
- [Table-to-Text Generation with Pretrained Diffusion Models](https://arxiv.org/pdf/2409.13739) (IEEE 2024)
- [Diffusion of Thought: Chain-of-Thought Reasoning in Diffusion Language Models](https://arxiv.org/pdf/2402.07754) (NeurIPS 2024)
- [DiffusionNER: Boundary Diffusion for Named Entity Recognition](https://aclanthology.org/2023.acl-long.215.pdf) (ACL 2023)
- [Fine-grained Text Style Transfer with Diffusion-Based Language Models](https://aclanthology.org/2023.repl4nlp-1.6.pdf) (RepL4NLP 2023)

Awesome-Diffusion-LLM

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README