ncnn-benchmark

KO | EN

gitlite — search

#python #hacktoberfest #go #machine-learning #linux #docker #windows #awesome #cli #awesome-list #pytorch #mcp #ios #open-source

(★ 39)

Using ncnn to test the reasoning performance of neural network

ncnn-benchmark Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# ncnn-benchmark

[中文](README_zh.md) | English

Using ncnn to test the reasoning performance of neural network

## Introduction

**ncnn-benchmark** is a project to test the reasoning performance of neural network using ncnn framework, which includes many common target detection models and tests their reasoning time and accuracy performance on different devices. What is shown now is the test result under the platform of `i5-12600k` CPU Win.

## Test Environment

- **CPU**: Intel i5-12600K
- **OS**: Windows 10
- **ncnn version**: 20250916
- **Compiler**: Visual Studio 2022
- **Build config**: Release, x64

## Evaluated Models

- [yolov5](https://github.com/ultralytics/yolov5)
- [yolox](https://github.com/Megvii-BaseDetection/YOLOX)
- [yolov6](https://github.com/meituan/YOLOv6)
- [yolov7](https://github.com/WongKinYiu/yolov7)
- [yolov8](https://github.com/ultralytics/ultralytics)
- [yolov9](https://github.com/WongKinYiu/yolov9)
- [yolov10](https://github.com/THU-MIG/yolov10)
- [yolo11](https://github.com/ultralytics/ultralytics)
- [yolov12](https://github.com/sunsmarterjie/yolov12)
- [yolov13](https://github.com/iMoonLab/yolov13)
- [yolo26](https://github.com/ultralytics/ultralytics)

## Benchmark Results

![Latency vs mAP](https://raw.githubusercontent.com/mpj1234/ncnn-benchmark/main/plots/latency_avg_vs_map.png)

### Important Notes

> Due to OS scheduling and background processes, inference latency may fluctuate. Results shown are **averaged over multiple runs** and intended for **reference only**.

> **Only pure inference time** (forward pass in ncnn) is measured — **preprocessing and postprocessing are excluded**.

> The comparison of the models in this project does not include the model entered in 1024*1024.

> For **YOLOv7-tiny**, mAP (50–95) is taken from the original paper’s `AP_val`.

> **YOLOv12** uses the **Turbo** variant.

> For **YOLOX**, mAP values are based on COCO `val2017` results.

## Build Instructions (Windows)

> If it is Windows, start the x64 Native Tools command line of Visual Studio 2022 and perform the following steps:

1. Build ncnn
   Clone ncnn warehouse and compile ncnn library, refer to [ncnn official document](https://github.com/Tencent/ncnn).

2. Clone this repository

```bash
git clone https://github.com/mpj1234/ncnn-benchmark.git
cd ncnn-benchmark
```

3. Use CMake to configure the project and specify the ncnn library path.

```bash
mkdir build
cd build
cmake -G "Ninja" -DNCNN_DIR=path_to_ncnn_install -DCMAKE_BUILD_TYPE=Release ..
```

4. Build the project

```bash
cmake --build .
```

5. Run the benchmark executable to see the results.

Run the generated executable file on the command line:

```bash
# Test all models.
benchncnn.exe  all

# Test a single series of models, such as yolov5.
# Optional parameters are any one of [Evaluated Models](#Evaluated Models).
benchncnn.exe  yolov5

# Test specific path model
benchncnn.exe test 4 8 2 -1 1 param=path/to/model.param shape=[640,640,3],...
```

## Detailed Results

You can view the contents of `csv` files in various param directories.

### YOLOv5
| Model   |   Param(MB) |   mAP(50-95) |   Min Speed(ms) |   Max Speed(ms) |   Avg Speed(ms) |
|:--------|------------:|-------------:|----------------:|----------------:|----------------:|
| 5n      |         1.9 |         28   |           28.26 |           31.32 |           30.45 |
| 5s      |         7.2 |         37.4 |           57.24 |           59.55 |           58.07 |
| 5m      |        21.2 |         45.4 |          115.59 |          126.48 |          118.83 |
| 5l      |        46.5 |         49   |          214.44 |          236.44 |          222.02 |
| 5x      |        86.7 |         50.7 |          259.01 |          414.35 |          379.06 |

### YOLOx
| Model   |   Param(MB) |   mAP(50-95) |   Min Speed(ms) |   Max Speed(ms) |   Avg Speed(ms) |
|:--------|------------:|-------------:|----------------:|----------------:|----------------:|
| xs      |         9   |         40.5 |           69.36 |           70.61 |           69.91 |
| xm      |        25.3 |         46.9 |          143.48 |          146.31 |          144.96 |
| xl      |        54.2 |         49.7 |          323.59 |          370.52 |          348.56 |
| xx      |        99.1 |         51.1 |          458.84 |          499.97 |          483.23 |

### YOLOv6
| Model   |   Param(MB) |   mAP(50-95) |   Min Speed(ms) |   Max Speed(ms) |   Avg Speed(ms) |
|:--------|------------:|-------------:|----------------:|----------------:|----------------:|
| 6n      |         4.7 |         37.5 |           24.9  |           26.49 |           25.75 |
| 6s      |        18.5 |         45   |           73.17 |           85.68 |           78.27 |
| 6m      |        34.9 |         50   |          143.28 |          213.07 |          168.32 |
| 6l      |        59.6 |         52.8 |          276.01 |          317.17 |          293.68 |

### YOLOv7
| Model   |   Param(MB) |   mAP(50-95) |   Min Speed(ms) |   Max Speed(ms) |   Avg Speed(ms) |
|:--------|------------:|-------------:|----------------:|----------------:|----------------:|
| 7t      |         6.2 |         30.8 |           35.75 |           36.67 |           36.31 |
| 7       |        36.9 |         51.4 |          271.68 |          275.47 |          273.27 |
| 7x      |        71.3 |         53.1 |          413    |          423.59 |          416.81 |

### YOLOv8
| Model   |   Param(MB) |   mAP(50-95) |   Min Speed(ms) |   Max Speed(ms) |   Avg Speed(ms) |
|:--------|------------:|-------------:|----------------:|----------------:|----------------:|
| 8n      |         3.2 |         37.3 |           31.27 |           36.34 |           33.18 |
| 8s      |        11.2 |         44.9 |           66.45 |           71.05 |           67.97 |
| 8m      |        25.9 |         50.2 |          142.96 |          144.22 |          143.67 |
| 8l      |        43.7 |         52.9 |          289.46 |          303.25 |          294.05 |
| 8x      |        68.2 |         53.9 |          416.52 |          466.18 |          429.31 |

### YOLOv9
| Model   |   Param(MB) |   mAP(50-95) |   Min Speed(ms) |   Max Speed(ms) |   Avg Speed(ms) |
|:--------|------------:|-------------:|----------------:|----------------:|----------------:|
| 9t      |         2   |         38.3 |           38.72 |           40.91 |           39.77 |
| 9s      |         7.1 |         46.8 |           74.61 |           74.94 |           74.81 |
| 9m      |        20   |         51.4 |          180.76 |          182.21 |          181.75 |
| 9c      |        25.3 |         53   |          227.75 |          278.58 |          242.97 |
| 9e      |        57.3 |         55.6 |          504.36 |          562.25 |          535.77 |

### YOLOv10
| Model   |   Param(MB) |   mAP(50-95) |   Min Speed(ms) |   Max Speed(ms) |   Avg Speed(ms) |
|:--------|------------:|-------------:|----------------:|----------------:|----------------:|
| 10n     |         2.3 |         38.5 |           30.49 |           31.73 |           31.13 |
| 10s     |         7.2 |         46.3 |           67.61 |           68.06 |           67.76 |
| 10m     |        15.4 |         51.1 |          139.7  |          148.22 |          142.55 |
| 10b     |        19.1 |         52.5 |          217.56 |          225.31 |          221.71 |
| 10l     |        24.4 |         53.2 |          271.22 |          314.77 |          291.16 |
| 10x     |        29.5 |         54.4 |          390.67 |          407.5  |          397.65 |

### YOLO11
| Model   |   Param(MB) |   mAP(50-95) |   Min Speed(ms) |   Max Speed(ms) |   Avg Speed(ms) |
|:--------|------------:|-------------:|----------------:|----------------:|----------------:|
| 11n     |         2.6 |         39.5 |           34.33 |           36.52 |           35.43 |
| 11s     |         9.4 |         47   |           76.33 |           77.96 |           77.29 |
| 11m     |        20.1 |         51.5 |          191.02 |          192.92 |          191.59 |
| 11l     |        25.3 |         53.4 |          240.71 |          250.44 |          243.91 |
| 11x     |        56.9 |         54.7 |          430.65 |          451.88 |          436.72 |

### YOLOv12
| Model   |   Param(MB) |   mAP(50-95) |   Min Speed(ms) |   Max Speed(ms) |   Avg Speed(ms) |
|:--------|------------:|-------------:|----------------:|----------------:|----------------:|
| 12n     |         2.5 |         40.4 |           67.02 |           69.04 |           67.99 |
| 12s     |         9.1 |         47.6 |          147.26 |          150.6  |          149.5  |
| 12m     |        19.6 |         52.5 |          316.97 |          331.3  |          322.29 |
| 12l     |        26.5 |         53.8 |          510.84 |          545.82 |          521.76 |
| 12x     |        59.3 |         55.4 |          838.7  |          857.92 |          850.17 |

### YOLOv13
| Model   |   Param(MB) |   mAP(50-95) |   Min Speed(ms) |   Max Speed(ms) |   Avg Speed(ms) |
|:--------|------------:|-------------:|----------------:|----------------:|----------------:|
| 13n     |         2.5 |         41.6 |           75.42 |           77.34 |           76.54 |
| 13s     |         9   |         48   |          166.27 |          185.09 |          171.54 |
| 13l     |        27.6 |         53.4 |          586.47 |          616.65 |          600.61 |
| 13x     |        64   |         54.8 |          964.17 |          990.59 |          978.48 |

### YOLO26
| Model   |   Param(MB) | mAP(50-95) |   Min Speed(ms) |   Max Speed(ms) |   Avg Speed(ms) |
|:--------|------------:|-----------:|----------------:|----------------:|----------------:|
| 26n     |         2.4 |       40.9 |           37.81 |           43.81 |           40.35 |
| 26s     |         9.5 |       48.6 |           79.32 |           83.51 |           81.10 |
| 26m     |        20.4 |       53.1 |          197.57 |          217.18 |          204.67 |
| 26l     |        24.8 |       55.0 |          240.18 |          295.47 |          255.28 |
| 26x     |        55.7 |       57.5 |          428.96 |          446.55 |          435.71 |

ncnn-benchmark

ncnn-benchmark Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

ncnn-benchmark

ncnn-benchmark Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README