FlowBlock

KO | EN

gitlite — search

#javascript #python #typescript #golang #deep-learning #linux #macos #windows #awesome #vue #pytorch #security #openai #open-source

(★ 119)

FlowBlock: Wavefront-Parallel Decoding Framework for Self-Correcting Diffusion Language Models

FlowBlock Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<h3>Wavefront-Parallel Decoding for Self-Correcting Diffusion Language Models</h3>

</div>

**FlowBlock** is a **training-free** parallel decoding framework for block-wise
diffusion language models (dLLMs). Block-wise dLLMs such as LLaDA-2.x decode
one block at a time, which makes inter-block decoding strictly serial.
FlowBlock observes that the **token-to-token (T2T)** self-correction ability of
LLaDA-2.1 series enable a downstream block start from an *informative draft* instead of
a *finalized* predecessor — so **block finality becomes a scheduling resource
rather than a hard dependency**.

FlowBlock is built on two mechanisms:

- **Gated Wavefront Decoding (GWD)** — admits blocks into a bounded wavefront
  only when a readiness gate (`SPAWN_TH`) is satisfied, jointly refines active
  blocks via T2T editing, and commits blocks in order under a windowed
  block-causal mask that preserves **exact** frozen-prefix KV-cache reuse.
- **Heterogeneous Wavefront Packing (HWP)** — gives each request its own
  wavefront while packing all active windows into dense, shape-stable batched
  forwards, so asynchronous progress stays GPU-efficient in batched serving.

## ✨ Highlights

- **Up to 2.95× / 4.01× higher TPS** than LLaDA-2.1 / LLaDA-2.0 under batched
  serving (1.57× / 3.17× at batch size 1), with **up to 53.6% / 77.1% lower
  latency**.
- **Accuracy preserved or improved**: +1.3 points on average across eight
  math / code benchmarks.
- **No training**: runs the unmodified LLaDA-2.1 series
  weights.

## 🛠️ Installation

Requirements: Linux, NVIDIA GPU(s) with ~80 GB memory for the 16B-A1B MoE
checkpoints, CUDA 12.x driver.

```bash
git clone <this-repo-url> FlowBlock
cd FlowBlock

conda create -n flowblock python=3.11 -y
conda activate flowblock

# 1) Install PyTorch matching your CUDA version (validated: 2.8.0 + cu128)
pip install torch==2.8.0 --index-url https://download.pytorch.org/whl/cu128

# 2) Install the base dependencies (sglang, lm-eval, ...)
pip install -r requirements.txt

# 3) Install vLLM last — this is a separate step on purpose
pip install vllm==0.10.2
```

The `dinfer` package is used directly from source — the evaluation scripts set
`PYTHONPATH` automatically, no `pip install` step is needed. Verify the setup:

```bash
PYTHONPATH="$PWD/dInfer/python" python -c \
  "from dinfer.decoding import FlowBlockJointDiffusionLLM, flowblock_packed; print('FlowBlock import OK')"
```

### Models

| Method | Checkpoint |
| --- | --- |
| `flowblock`, `llada2.1` | [inclusionAI/LLaDA2.1-mini](https://huggingface.co/inclusionAI/LLaDA2.1-mini) |
| `llada2.0` | [inclusionAI/LLaDA2.0-mini](https://huggingface.co/inclusionAI/LLaDA2.0-mini) |

```bash
python dInfer/evaluations/download_hf_model.py \
  --repo_id inclusionAI/LLaDA2.1-mini --local_dir /path/to/LLaDA2.1-mini
```

### Datasets

Benchmark data is fetched automatically on first run (GSM8K is downloaded to
`test_datasets/`, the other tasks load from the Hugging Face Hub). For
air-gapped machines, prefetch the code benchmarks once on a connected host:

```bash
TEST_DATASETS_ROOT=./test_datasets/code bash dInfer/evaluations/prefetch_code_datasets.sh
```

## 🚀 Running Evaluations

`run_eval.sh` is the single entry point. Pick a **method**, a **task family**
(or a specific benchmark), and a **model path**:

```bash
# FlowBlock on math (defaults: GSM8K, W=2, SPAWN_TH=0.6)
bash run_eval.sh --method flowblock --task math --model-path /path/to/LLaDA2.1-mini

# FlowBlock on code (defaults: HumanEval, W=2, SPAWN_TH=0.8)
bash run_eval.sh --method flowblock --task code --model-path /path/to/LLaDA2.1-mini

# Serial baselines
bash run_eval.sh --method llada2.1 --benchmark math500 --model-path /path/to/LLaDA2.1-mini
bash run_eval.sh --method llada2.0 --task math --model-path /path/to/LLaDA2.0-mini

# FlowBlock and LLaDA-2.1 back-to-back on the same seed-fixed subset,
# 4 GPUs (data-parallel), batch size 8 per GPU
bash run_eval.sh --method flowblock,llada2.1 --task math \
  --model-path /path/to/LLaDA2.1-mini --gpus 0,1,2,3 --batch-size 8
```

### Benchmarks

| `--task` | `--benchmark` options | Default |
| --- | --- | --- |
| `math` | `gsm8k`, `math500`, `algebra`, `asdiv` | `gsm8k` |
| `code` | `humaneval`, `humaneval_plus`, `mbpp`, `mbpp_plus` | `humaneval` |

### Key options and defaults

| Option | Default | Meaning |
| --- | --- | --- |
| `--method` | `flowblock` | `flowblock` \| `llada2.1` \| `llada2.0` (comma-separated to combine) |
| `--window` | `2` | FlowBlock wavefront window size `W` |
| `--spawn-th` | `0.6` math / `0.8` code | Readiness gate threshold `θ_spawn` |
| `--batch-size` | `4` | Per-GPU batch size |
| `--gen-length` | `2048` | Max generation length |
| `--block-length` | `32` | Diffusion block length |
| `--threshold` | `0.95` (2.0) / `0.5` math, `0.9` code (2.1 & FlowBlock) | Decode / commit threshold |
| `--gpus` | `0` | GPU ids; one data-parallel shard per GPU |
| `--num-samples` | full benchmark | Evaluate a seed-shuffled subset |
| `--output-dir` | `outputs/<benchmark>_<timestamp>` | Result directory |

Run `bash run_eval.sh --help` for the full list. Results land in:

```
<output-dir>/<method>/summary.json # accuracy, TPS, latency, TPF per method
<output-dir>/global_summary.log # side-by-side comparison table
```

For multi-configuration sweeps (block lengths, batch sizes, thresholds), edit
`RUN_COMBOS` in `dInfer/evaluations/run_flowblock_joint_batch_compare_loop.sh`
and run it directly; `aggregate_global_results.py` merges everything into one
table afterwards.

> ⚠️ **Code benchmarks execute model-generated Python** (`HF_ALLOW_CODE_EVAL=1`).
> Run them only in an isolated environment (container / VM without secrets).

## 📊 Results Preview

Under the paper settings (block length 32), FlowBlock consistently dominates
the serial baselines in the throughput–latency plane while matching or
improving accuracy.

### Single-request decoding (batch size 1)

Main results at batch size 1 on a single 80 GB GPU (block length 32,
generation length 2048; FlowBlock uses `W=2` with per-task spawn thresholds).
Best value per benchmark in **bold**.

<table>
 <thead>
 <tr>
 <th rowspan="2" align="left" style="text-align:left;">Benchmark</th>
 <th colspan="4" align="center" style="text-align:center;">LLaDA-2.0</th>
 <th colspan="4" align="center" style="text-align:center;">LLaDA-2.1</th>
 <th colspan="4" align="center" style="text-align:center;">FlowBlock</th>
 </tr>
 <tr>
 <th align="center" style="text-align:center;">Acc</th><th align="center" style="text-align:center;">TPF</th><th align="center" style="text-align:center;">TPS</th><th align="center" style="text-align:center;">Lat</th>
 <th align="center" style="text-align:center;">Acc</th><th align="center" style="text-align:center;">TPF</th><th align="center" style="text-align:center;">TPS</th><th align="center" style="text-align:center;">Lat</th>
 <th align="center" style="text-align:center;">Acc</th><th align="center" style="text-align:center;">TPF</th><th align="center" style="text-align:center;">TPS</th><th align="center" style="text-align:center;">Lat</th>
 </tr>
 </thead>
 <tbody>
 <tr>
 <td align="left">GSM8K</td>
 <td align="center" style="text-align:center;">92.49</td><td align="center" style="text-align:center;">2.04</td><td align="center" style="text-align:center;">82.6</td><td align="center" style="text-align:center;">4.17</td>
 <td align="center" style="text-align:center;">92.49</td><td align="center" style="text-align:center;">4.46</td><td align="center" style="text-align:center;">176.6</td><td align="center" style="text-align:center;">2.05</td>
 <td align="center" style="text-align:center;">92.65</td><td align="center" style="text-align:center;">6.73</td><td align="center" style="text-align:center;">254.2</td><td align="center" style="text-align:center;">1.34</td>
 </tr>
 <tr>
 <td align="left">MATH500</td>
 <td align="center" style="text-align:center;">73.40</td><td align="center" style="text-align:center;">2.59</td><td align="center" style="text-align:center;">104.6</td><td align="center" style="text-align:center;">11.25</td>
 <td align="center" style="text-align:center;">77.00</td><td align="center" style="text-align:center;">5.34</td><td align="center" style="text-align:center;">211.7</td><td align="center" style="text-align:center;">5.40</td>
 <td align="center" style="text-align:center;">76.20</td><td align="center" style="text-align:center;">8.86</td><td align="center" style="text-align:center;">332.1</td><td align="center" style="text-align:center;">3.44</td>
 </tr>
 <tr>
 <td align="left">Minerva-Algebra</td>
 <td align="center" style="text-align:center;">91.24</td><td align="center" style="text-align:center;">3.00</td><td align="center" style="text-align:center;">120.9</td><td align="center" style="text-align:center;">6.50</td>
 <td align="center" style="text-align:center;">93.60</td><td align="center" style="text-align:center;">6.29</td><td align="center" style="text-align:center;">247.6</td><td align="center" style="text-align:center;">2.87</td>
 <td align="center" style="text-align:center;">93.51</td><td align="center" style="text-align:center;">10.14</td><td align="center" style="text-align:center;">378.9</td><td align="center" style="text-align:center;">1.81</td>
 </tr>
 <tr>
 <td align="left" style="border-bottom:2px solid #999;">ASDIV</td>
 <td align="center" style="text-align:center;border-bottom:2px solid #999;">92.36</td><td align="center" style="text-align:center;border-bottom:2px solid #999;">2.03</td><td align="center" style="text-align:center;border-bottom:2px solid #999;">81.9</td><td align="center" style="text-align:center;border-bottom:2px solid #999;">2.94</td>
 <td align="center" style="text-align:center;border-bottom:2px solid #999;">92.89</td><td align="center" style="text-align:center;border-bottom:2px solid #999;">4.43</td><td align="center" style="text-align:center;border-bottom:2px solid #999;">174.5</td><td align="center" style="text-align:center;border-bottom:2px solid #999;">1.50</td>
 <td align="center" style="text-align:center;border-bottom:2px solid #999;">92.80</td><td align="center" style="text-align:center;border-bottom:2px solid #999;">6.36</td><td align="center" style="text-align:center;border-bottom:2px solid #999;">239.7</td><td align="center" style="text-align:center;border-bottom:2px solid #999;">1.01</td>
 </tr>
 <tr>
 <td align="left">HumanEval</td>
 <td align="center" style="text-align:center;">84.76</td><td align="center" style="text-align:center;">4.37</td><td align="center" style="text-align:center;">159.6</td><td align="center" style="text-align:center;">2.16</td>
 <td align="center" style="text-align:center;">82.93</td><td align="center" style="text-align:center;">4.93</td><td align="center" style="text-align:center;">180.1</td><td align="center" style="text-align:center;">1.79</td>
 <td align="center" style="text-align:center;">86.59</td><td align="center" style="text-align:center;">8.41</td><td align="center" style="text-align:center;">279.4</td><td align="center" style="text-align:center;">1.15</td>
 </tr>
 <tr>
 <td align="left">HumanEval+</td>
 <td align="center" style="text-align:center;">79.27</td><td align="center" style="text-align:center;">4.37</td><td align="center" style="text-align:center;">162.5</td><td align="center" style="text-align:center;">2.13</td>
 <td align="center" style="text-align:center;">77.44</td><td align="center" style="text-align:center;">4.93</td><td align="center" style="text-align:center;">182.8</td><td align="center" style="text-align:center;">1.76</td>
 <td align="center" style="text-align:center;">82.32</td><td align="center" style="text-align:center;">8.41</td><td align="center" style="text-align:center;">282.4</td><td align="center" style="text-align:center;">1.14</td>
 </tr>
 <tr>
 <td align="left">MBPP</td>
 <td align="center" style="text-align:center;">79.16</td><td align="center" style="text-align:center;">2.75</td><td align="center" style="text-align:center;">105.1</td><td align="center" style="text-align:center;">2.24</td>
 <td align="center" style="text-align:center;">82.20</td><td align="center" style="text-align:center;">3.26</td><td align="center" style="text-align:center;">124.2</td><td align="center" style="text-align:center;">2.09</td>
 <td align="center" style="text-align:center;">83.61</td><td align="center" style="text-align:center;">4.43</td><td align="center" style="text-align:center;">162.1</td><td align="center" style="text-align:center;">1.53</td>
 </tr>
 <tr>
 <td align="left" style="border-bottom:2px solid #999;">MBPP+</td>
 <td align="center" style="text-align:center;border-bottom:2px solid #999;">84.39</td><td align="center" style="text-align:center;border-bottom:2px solid #999;">2.78</td><td align="center" style="text-align:center;border-bottom:2px solid #999;">108.7</td><td align="center" style="text-align:center;border-bottom:2px solid #999;">2.18</td>
 <td align="center" style="text-align:center;border-bottom:2px solid #999;">87.04</td><td align="center" style="text-align:center;border-bottom:2px solid #999;">3.30</td><td align="center" style="text-align:center;border-bottom:2px solid #999;">125.8</td><td align="center" style="text-align:center;border-bottom:2px solid #999;">1.96</td>
 <td align="center" style="text-align:center;border-bottom:2px solid #999;">88.36</td><td align="center" style="text-align:center;border-bottom:2px solid #999;">4.47</td><td align="center" style="text-align:center;border-bottom:2px solid #999;">161.3</td><td align="center" style="text-align:center;border-bottom:2px solid #999;">1.39</td>
 </tr>
 <tr>
 <td align="left">Average</td>
 <td align="center" style="text-align:center;">84.63</td><td align="center" style="text-align:center;">2.99</td><td align="center" style="text-align:center;">115.7</td><td align="center" style="text-align:center;">4.20</td>
 <td align="center" style="text-align:center;">85.70</td><td align="center" style="text-align:center;">4.62</td><td align="center" style="text-align:center;">177.9</td><td align="center" style="text-align:center;">2.43</td>
 <td align="center" style="text-align:center;">87.00</td><td align="center" style="text-align:center;">7.23</td><td align="center" style="text-align:center;">261.2</td><td align="center" style="text-align:center;">1.60</td>
 </tr>
 </tbody>
</table>

Acc = accuracy (%) · TPF = tokens per forward pass · TPS = tokens per
second · Lat = per-request latency (s).

On average FlowBlock reaches **87.00** accuracy — 1.3 points above LLaDA-2.1
(85.70) — while delivering **1.45× / 2.35×** the throughput of LLaDA-2.1 /
LLaDA-2.0 (up to 1.57× / 3.17×).

### Batched serving

**Math benchmarks — batched serving (TPS ↑ / latency ↓ vs. batch size):**

**Code benchmarks — batched serving:**

## 📁 Repository Layout

```
FlowBlock
├── run_eval.sh                        # ← user-facing evaluation entry point
├── requirements.txt
├── assets/                            # logo & result figures
└── dInfer/
    ├── python/dinfer/
    │   ├── decoding/                  # FlowBlock + LLaDA-2.1 decoders
    │   │   ├── generate_uniform.py    #   GWD: gated wavefront, joint M2T/T2T refine, in-order commit
    │   │   ├── flowblock_packed.py    #   HWP: per-request wavefronts packed into dense forwards
    │   │   ├── parallel_strategy.py   #   threshold / joint-threshold decode policies
    │   │   ├── diffusion_runner.py    #   SGLang model runner (compile / CUDA-graph)
    │   │   └── utils.py               #   KV cache, block iterators, token arrays
    │   ├── decoding_llada_origin/     # original LLaDA-2.0 decoding (serial baseline)
    │   └── model/                     # LLaDA-2 MoE modeling (HF + SGLang)
    └── evaluations/
        ├── run_flowblock_joint_batch_compare.sh       # comparison harness (used by run_eval.sh)
        ├── run_flowblock_joint_batch_compare_loop.sh  # multi-config sweep driver
        ├── eval_dinfer_sglang.py                      # lm-eval entry with dInfer backend
        ├── prepare_task_subset.py                     # seed-fixed subset & GPU sharding
        ├── aggregate_dp8_results.py / aggregate_global_results.py
        ├── val_gsm8k.py / val_math.py / val_asdiv.py / val_code.py
        └── tasks/                                     # lm-eval task configs
```

## 🙏 Acknowledgements

This project is built upon [DMax](https://github.com/czg1225/DMax) and the
[dInfer](https://github.com/inclusionAI/dInfer) inference framework, and uses
[SGLang](https://github.com/sgl-project/sglang),
[lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness),
and the [LLaDA-2.x](https://huggingface.co/inclusionAI) model family. Many
thanks to the authors of these projects.

## 📄 License & Citation

Released under the [Apache 2.0 License](LICENSE).

**Paper**: **FlowBlock: Wavefront-Parallel Decoding for Self-Correcting Diffusion Language Models** — Bing Tian, Haikun Liu, Xiaocheng Zhong, Zhuohui Duan, Zhaokai Luo, Huayi Jin, Zhiyong Wang, Xiaofei Liao (<https://arxiv.org/abs/2607.17652>)

FlowBlock

FlowBlock Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

FlowBlock

FlowBlock Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README