minimax-m27-nvfp4-gb10-benchmark

KO | EN

gitlite — search

#javascript #python #go #php #c #c-plus-plus #android #c-sharp #ruby #css #node #extension

minimax-m27-nvfp4-gb10-benchmark

★ 13 Open GitHub ↗

MiniMax M2.7 NVFP4 dual-GB10 Blackwell benchmark: vLLM FlashInfer-CUTLASS, public data, HTML canvas report, and Docker runtime.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# MiniMax M2.7 NVFP4 on dual GB10 / Blackwell

Publication-ready benchmark and reproducibility bundle for official `nvidia/MiniMax-M2.7-NVFP4` on NVIDIA Blackwell SM120/SM121 using vLLM.

Credit: @mr-r0b0t on X — r0b0tlab

## Headline

| Metric | Final | Public TP=2 baseline | Final / public |
|---|---:|---:|---:|
| tg128 | 25.06 tok/s | 24.30 tok/s | 103.1% |
| tg128 @ d4096 | 24.16 tok/s | 23.75 tok/s | 101.7% |
| tg128 @ d8192 | 24.05 tok/s | 22.92 tok/s | 104.9% |
| tg128 @ d16384 | 21.99 tok/s | 21.78 tok/s | 101.0% |

Mean decode vs public: 102.7%.

## Final runtime

- Model: `nvidia/MiniMax-M2.7-NVFP4`
- Quantization: NVIDIA ModelOpt NVFP4
- Runtime: vLLM source build with SM120/SM121 FP4 support
- Hardware: 2x NVIDIA GB10 / SM121
- Topology: Ray tensor parallel size 2
- Backend: `VLLM_NVFP4_GEMM_BACKEND=flashinfer-cutlass` and `--moe-backend flashinfer_cutlass`
- Compilation: torch.compile mode 3, `cudagraph_mode=none`
- Fallbacks: MARLIN selected-backend hits = 0; EMULATION selected-backend hits = 0

## Why CUDA graphs are off

For the official NVIDIA checkpoint, CUDA graph capture succeeds but graph replay deadlocks vLLM's cross-node SHM broadcast path during TP=2 inference. Compile-only mode keeps Inductor optimization without graph replay and is the production-stable path measured here.

## Container

Published runtime image:

```text
ghcr.io/r0b0tlab/vllm-minimax-m27-nvfp4-blackwell:cu130-sm120-arm64-v0.21.1
ghcr.io/r0b0tlab/vllm-minimax-m27-nvfp4-blackwell:latest-sm120-arm64
ghcr.io/r0b0tlab/vllm-minimax-m27-nvfp4-blackwell@sha256:9ee93af94a508bf7ef6125f6bf655c27b9082a892296d9db20ecdee6e360cb03
```

The image does not include model weights. See `docker/README.md`.

## Compatible devices

Validated:

- Dual NVIDIA GB10 / DGX Spark class systems, SM121, enough aggregate memory for the 130 GiB NVFP4 checkpoint plus KV cache.

Expected compatible:

- Blackwell SM120/SM121 systems with sufficient aggregate memory.
- RTX PRO 6000 Blackwell class systems.
- Multi-GPU Blackwell systems with reduced context if memory-constrained.

Not claimed:

- Ampere/Hopper optimized NVFP4 tensor-core execution.
- Marlin or emulation fallback as an optimized result.

## Files

- `publication/html/index.html` — self-contained HTML canvas benchmark report.
- `results/results.csv` — sanitized benchmark table.
- `results/results.json` — benchmark data and summary.
- `evidence/backend_evidence.txt` — sanitized backend proof.
- `docker/` — Dockerfile, entrypoint, container instructions.
- `scripts/launch_gb10_dual_tp2_optimized.sh` — generic launch script.
- `scripts/check_public_safety.py` — deterministic public-safety scanner.

## Reproduce

1. Accept the model license and download `nvidia/MiniMax-M2.7-NVFP4` locally.
2. Use Blackwell SM120/SM121 hardware with enough aggregate memory.
3. Verify container FP4 support as shown in `docker/README.md`.
4. Launch with the same backend and compile flags in `scripts/launch_gb10_dual_tp2_optimized.sh`.
5. Run llama-benchy with `pp=2048`, `tg=128`, depths `0,4096,8192,16384`, runs `3`, latency mode `api`.

## License

Code and documentation in this repo: MIT. Model weights remain subject to NVIDIA and MiniMax license terms.

minimax-m27-nvfp4-gb10-benchmark

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README