KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#developer-tools #typescript #dsh #dsh-plugin #ai-agent #codex #ai-agents #deepseek-harness #claude-code #cli #react #mcp

ComfyUI-PiD

★ 121 Open GitHub ↗

ComfyUI custom node for NVIDIA PiD pixel diffusion decoding and upscale workflows

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# ComfyUI-PiD

Compact ComfyUI nodes for **NVIDIA PiD / PixelDiT** using ComfyUI-native `Comfy-Org/PixelDiT` model loading.

PiD is a latent-conditioned pixel diffusion decoder/upscaler:

```text
LATENT + caption + sigma -> PiD -> IMAGE
```

## Install

```bash
cd ComfyUI/custom_nodes
git clone https://github.com/Merserk/ComfyUI-PiD.git
cd ComfyUI-PiD
python -m pip install -r requirements.txt
```

Restart ComfyUI.

Requirements: recent ComfyUI with native PixelDiT/PiD support, Python `>=3.10`, NVIDIA CUDA GPU recommended. PiD v1.5 requires ComfyUI `0.28.0` or newer.

## Models

Most nodes can download required files automatically when `auto_download=true`.

| Use | Source | Local folder |
| --- | --- | --- |
| PiD diffusion + Gemma text encoder | `Comfy-Org/PixelDiT` | `ComfyUI/models/diffusion_models/nvidia_pid/` and `ComfyUI/models/text_encoders/nvidia_pid/` |
| PixelDiT text-to-image generation | `Comfy-Org/PixelDiT` | `ComfyUI/models/diffusion_models/nvidia_pid/` and `ComfyUI/models/text_encoders/nvidia_pid/` |
| Caption Creator | `Qwen/Qwen3.5-0.8B` | `ComfyUI/models/text_encoders/nvidia_pid/qwen35_caption/` |
| Upscale VAEs | Flux/Z-Image, Flux2, SD3 VAE files | `ComfyUI/models/vae/nvidia_pid/` |

PiD v1 and v1.5 files are downloaded from `diffusion_models/`. They are stored in the flat local `ComfyUI/models/diffusion_models/nvidia_pid/` folder; v1.5 filenames already include their version.

Use `model_precision=bf16` for best quality. PiD v1 `fp8` is available only for Flux1-family `2k/2kto4k` and Flux2-family `2k`; Flux2 `2kto4k`, SD3, SDXL, and Qwen-Image must use `bf16`. PiD v1.5 supports `bf16` and `int8`; its INT8 ConvRot diffusion model uses the FP8 Gemma text encoder. Unsupported combinations produce an error and never fall back to another version.

PiD selectors are compatibility-aware in the ComfyUI interface. Changing `version`, `backbone`, or `pid_ckpt_type` immediately removes unsupported downstream choices and normalizes an invalid current value to a supported one. For example, selecting `v1.5` removes `2k`, SD3/SDXL, and `fp8`, while enabling `int8`. `PiD Empty Latent Image` similarly hides SDXL and Qwen backbones in `2k` mode. Backend validation remains active for API prompts and older workflows.

## Nodes

| Node | Output | Purpose |
| --- | --- | --- |
| **PiD Decode** | `IMAGE` | One-node PiD decode from latent + caption + sigma. |
| **PiD Text Prompt** | `text`, `caption` | One prompt for normal text encoding and PiD caption input. |
| **PiD Caption Creator** | `text`, `caption` | Creates a caption from an input image with local Qwen. |
| **PiD Empty Latent Image** | `LATENT` | Backbone-aware empty latent with correct channels/downscale. |
| **PiD KSampler Capture** | `final_latent`, `pid_latent`, `pid_sigma` | KSampler-compatible sampler that captures the PiD latent and sigma. |
| **PiD Prepare** | `PID_PREP` | Moves/validates latent data and resolves PiD model assets. |
| **PiD Sample** | `PID_SAMPLES` | Runs native PiD sampling. |
| **PiD Finalize** | `IMAGE` | Converts PiD samples to a ComfyUI image. |
| **PiD Upscale** | `IMAGE` | Image-only tiled PiD upscaler with `2x/4x/6x/8x` output. |
| **PixelDiT Generate** | `IMAGE` | VAE-free 1024-class text-to-image generation with native PixelDiT. |

Recommended PiD sampling: `pid_steps=4`, `cfg_scale=1.0`, `scale=0` or `4`.

## PixelDiT Generate

`PixelDiT Generate` creates images directly in RGB pixel space. It does not load or run a traditional VAE. Connect the `text` output from `PiD Text Prompt`, select a precision and resolution, then connect the generated `IMAGE` to `Save Image`.

| Setting | Values / behavior |
| --- | --- |
| `model_precision` | `bf16` uses the BF16 diffusion model and Gemma encoder; `fp8` uses the MXFP8 diffusion model and FP8 Gemma encoder. |
| `resolution` | Eight approximately one-megapixel presets from `1024x1024` through portrait, widescreen, and ultrawide formats. |
| `steps` | Default `30`, matching the official ComfyUI PixelDiT workflow. |
| `cfg_scale` | Default `4.0`. |
| `sampler_name` / `scheduler` | Default `er_sde` / `simple`. |
| `negative_prompt` | Defaults to NVIDIA/ComfyUI's recommended quality exclusions. |

Generation models are downloaded automatically when enabled:

```text
pixeldit_1300m_1024px_bf16.safetensors
pixeldit_1300m_1024px_mxfp8.safetensors
```

PixelDiT generation requires ComfyUI `0.28.0` or newer. The NVIDIA model weights use the NSCLv1 license and are limited to non-commercial research/evaluation use.

## Supported Backbones

| Backbone value | PiD family | Versions | Checkpoints | Latent | PiD Upscale |
| --- | --- | --- | --- | --- | --- |
| `zimage` | Flux1 | v1, v1.5 | v1: `2k`/`2kto4k`; v1.5: `2kto4k` | 16ch / 8x | yes |
| `zimage-turbo` | Flux1 | v1, v1.5 | v1: `2k`/`2kto4k`; v1.5: `2kto4k` | 16ch / 8x | yes |
| `flux` | Flux1 | v1, v1.5 | v1: `2k`/`2kto4k`; v1.5: `2kto4k` | 16ch / 8x | yes |
| `flux2` | Flux2 | v1, v1.5 | v1: `2k`/`2kto4k`; v1.5: `2kto4k` | 128ch / 16x | yes |
| `flux2-klein-4b` | Flux2 | v1, v1.5 | v1: `2k`/`2kto4k`; v1.5: `2kto4k` | 128ch / 16x | yes |
| `flux2-klein-9b` | Flux2 | v1, v1.5 | v1: `2k`/`2kto4k`; v1.5: `2kto4k` | 128ch / 16x | yes |
| `sd3` | SD3 | v1 | `2k`, `2kto4k` | 16ch / 8x | yes |
| `sdxl` | SDXL | v1 | `2kto4k` only | 4ch / 8x | no |
| `qwenimage` | Qwen-Image | v1, v1.5 | `2kto4k` only | 16ch / 8x | no |
| `qwenimage-2512` | Qwen-Image | v1, v1.5 | `2kto4k` only | 16ch / 8x | no |

`dinov2` and `siglip` are not supported by the native Comfy-Org PiD model set.

## Output Size Guide

Released PiD checkpoints use native `4x` scale.

PiD v1 supports the rows shown below. PiD v1.5 provides only the `2kto4k` profile.

| `pid_ckpt_type` | Base latent/image size | Final PiD output | Valid base presets |
| --- | --- | --- | --- |
| `2k` | 512-class | base × 4, e.g. `512x512 -> 2048x2048` | `512x512`, `576x432`, `432x576`, `624x416`, `416x624`, `672x384`, `384x672`, `784x336`, `336x784` |
| `2kto4k` | 1024-class | base × 4, e.g. `1024x1024 -> 4096x4096` | `1024x1024`, `1024x768`, `768x1024`, `1008x672`, `672x1008`, `1024x576`, `576x1024`, `1008x432`, `432x1008` |

Latent size depends on backbone downscale. Example: Flux2 `1024x1024` uses a `128 × 64 × 64` latent.

## PiD Upscale

`PiD Upscale` accepts `IMAGE` and returns `IMAGE`. It is separate from latent decode: the node cuts the image into tiles, encodes each tile with the matching VAE, runs native 4-step PiD, blends tiles, then resizes to the selected final factor.

| Setting | Values / behavior |
| --- | --- |
| `pid_ckpt_type` | `2k` uses 512px tiles; `2kto4k` uses 1024px tiles. |
| `version` | `v1` supports `2k` and `2kto4k`; `v1.5` supports only `2kto4k` and only Flux1/Flux2-family backbones. |
| `backbone` | `zimage`, `zimage-turbo`, `flux`, `flux2`, `flux2-klein-4b`, `flux2-klein-9b`, `sd3`. |
| `model_precision` | v1 uses `bf16`/supported `fp8`; v1.5 uses `bf16` or INT8 ConvRot. Use `bf16` for best quality. |
| `upscale_factor` | Final output size: `2x`, `4x`, `6x`, or `8x`. |
| `strength` | PiD detail regeneration sigma, `0.0` to `1.0`; default `0.4`. |
| `caption` | Optional string input; connect `PiD Caption Creator` or `PiD Text Prompt`. |

| Profile | Tile size | Overlap | Small-image prepass |
| --- | ---: | ---: | ---: |
| `2k` | 512 | 64 | Resize long edge to 512, PiD once, then tiled upscale. |
| `2kto4k` | 1024 | 128 | Resize long edge to 1024, PiD once, then tiled upscale. |

Upscale VAEs are required because image tiles must be encoded into each backbone latent format:

| Backbone family | Accepted VAE names |
| --- | --- |
| Flux1 / Z-Image | `ae.safetensors` |
| Flux2 / Flux2-Klein | `flux2_ae.safetensors`, `flux2-vae.safetensors` |
| SD3 | `sd3_vae.safetensors`, `diffusion_pytorch_model.safetensors` |

Final upscale size is always based on the original input image: `width × factor`, `height × factor`. SDXL and Qwen-Image are not available in `PiD Upscale` because this implementation only maps image VAEs for Flux1/Z-Image, Flux2/Flux2-Klein, and SD3.

## Recommended Capture Settings

| Backbone | LDM steps | Capture step | Sampler / scheduler |
| --- | ---: | ---: | --- |
| `flux`, `sd3` | 28 | 24 | `euler` / `flowmatch_euler_discrete` |
| `sdxl` | 30 | 26 | `euler` / `normal` |
| `flux2` | 50 | 46 | `euler` / `flowmatch_euler_discrete` |
| `flux2-klein-4b`, `flux2-klein-9b` | 4 | 4 | `euler` / `flowmatch_euler_discrete` |
| `qwenimage`, `qwenimage-2512` | 50 | 44 | `euler` / `flowmatch_euler_discrete` |
| `zimage` | 50 | 46 | `euler` / `flowmatch_euler_discrete`, `flowmatch_shift=3.0` |
| `zimage-turbo` | 9 | 9 | `euler` / `flowmatch_euler_discrete`, `flowmatch_shift=3.0` |

## Main Workflows

### Text-to-image / generation

```text
PiD Text Prompt -> normal text encode + PiD caption
PiD Empty Latent Image -> model sampler
PiD KSampler Capture pid_latent + pid_sigma -> PiD Prepare
PiD Prepare -> PiD Sample -> PiD Finalize -> Save Image
```

### VAE-free PixelDiT generation

```text
PiD Text Prompt -> PixelDiT Generate -> Save Image
```

### Direct decode

```text
LATENT + caption + sigma -> PiD Decode -> Save Image
```

### Image-to-image clean decode

```text
Load Image -> Resize -> VAE Encode -> PiD Prepare -> PiD Sample -> PiD Finalize -> Save Image
```

### Tiled upscale

```text
Load Image -> PiD Caption Creator -> PiD Upscale -> Save Image
```

## Example Workflows

Included in `example_workflows/`:

```text
pid_flux_complete.json
pid_flux2_complete.json
pid_flux2_klein_4b_complete.json
pid_flux2_klein_9b_complete.json
pid_qwenimage_complete.json
pid_qwenimage_2512_complete.json
pid_sd3_complete.json
pid_sdxl_complete.json
pid_zimage_complete.json
pid_zimage_turbo_complete.json
pid_image_to_image_2k_complete.json
pid_image_to_image_2kto4k_complete.json
pid_upscale_complete.json
pid_pixeldit_generate_complete.json
```

## License

The ComfyUI-PiD node code is MIT licensed. PixelDiT/PiD model weights are distributed separately under NVIDIA's NSCLv1 license.

ComfyUI-PiD

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README