model_optimizer

KO | EN

gitlite — search

#python #java #python3 #arduino #golang #machine-learning #rust #html #flask #javascript #seismology #nodejs

model_optimizer

★ 24 Open GitHub ↗

No description available.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

model optimizer
===========================

## 1. Overview

model optimizer is a tool to make model quantize, optimize, deploy easier. including functions:

- quantize model
- optimize model
- deploy model

## 2. Getting Started

The code was tested in the following environments

- Ubuntu 24.04.03 LTS
- NVIDIA GPU `3090/4070/5090/Thor` driver version 580.95.05 with CUDA 13.0
- Docker nvcr.io/nvidia/tensorrt:25.10-py3

### 2.1 Installation

```bash
python setup.py sdist
```

#### 2.1.1 `download code`

```bash
git clone git@gitlab.anyverse.work:model/model_optimizer.git
git checkout development
git submodule update --init --recursive
git lfs pull
```

#### 2.1.2 `build docker image`

+ you may need install `docker-buildx`

```shell
sudo apt update && apt install docker-buildx
```

`x86`:

```shell
make build_x86
```

`jetson thor`:

```shell
make build_thor
```

#### 2.1.2.1 本地安装 (非 Docker)

```bash
# 完整功能 (量化 / 导出 / WebUI)
bash scripts/install.sh all

# 含 π0.5 模型
bash scripts/install.sh pi05

# 使用国内镜像
PIP_INDEX=https://pypi.tuna.tsinghua.edu.cn/simple bash scripts/install.sh all
```

依赖说明见 [requirements/README.md](requirements/README.md)。

#### 2.1.3 `run docker`

+ `CAUTION:` You should use `-v` to mount your `openpi` codes path into docker to use pi05 model.

+ `CAUTION:` To use host .cache huggingface or modelscope you should use `-v` to mount your `~/.cache` into docker.

###### you can see `Makefile` in codes to do these.

```Makefile
run_x86:
	docker run -it --network host \
	--gpus all \
	-v ${PWD}:/workspace \
	-v ${HOME}:/srcs \
	-v ${HOME}/.cache:/srcs/.cache \
	--shm-size=4g \
	--env PYTHONPATH=/usr/local/lib/python3.12/dist-packages/:/srcs/sources/opensrc/robot/openpi/src/:/srcs/sources/opensrc/robot/openpi/packages/openpi-client/src/ \
	model_optimizer:x86 \
	/bin/bash
```

+ Then use make run_x86 to run docker.

```shell
make run_x86
```

## 3. Supported Models

#### 3.1 supported model arch.

|**Architecture**|**Model**||
|-|-|-|
|YOLO11|yolo||
|π0.5|pi05_libero||
|π0.5|pi05_libero/vit||
|π0.5|pi05_libero/llm||
|π0.5|pi05_libero/expert||

|**Architecture**|**Detial usage**||
|-|-|-|
|π0.5|[manual](./docs/pi05.md)||
|π0.5 quantize|[manual](./docs/pi05_quantize.md)||
|YOLO11|[manual](./docs/yolo.md)||

#### 3.2 Tested and passed in the following environments

+ 3090

+ Jetson Thor

```shell
NVIDIA Jetson AGX Thor Developer Kit - Jetpack 7.0 [L4T 38.2.1]

## 4. Usage

|**Architecture**|**Detial usage**||
|-|-|-|
|π0.5|[manual](./docs/pi05.md)||
|π0.5 quantize|[manual](./docs/pi05_quantize.md)||
|YOLO11|[manual](./docs/yolo.md)||

### 4.1 CLI (Product Ready)

+ All the following operations are performed within Docker.

perform `make run_x86` or `make run_thor` enter the docker first.

#### 4.1.1 `π0.5` inference
```shell
python scripts/deployment/pi05/standalone_inference_script.py --model_path /openpi/pytorch_pi05_libero/ --inference_mode tensorrt  --perf 
```

`output`:
```shell
Inference time: 0.4213 seconds
e2e 419.43 ± 4.14 ms (shared)
action 25.14 ± 0.77 ms (shared)
vit 62.79 ± 0.49 ms (shared)
llm 100.47 ± 1.55 ms (shared)
```

+ `model_path`: pi05 model path, must a `pytorch` model
+ `inference_mode`: `tensorrt` or `pytorch`, choose tensorrt or pytorch as inference backend.
+ `--perf`: perf stats

#### 4.2 For more cli usages

[ALL CLI usage docs](./docs/cli.md)

##### 4.2.1 Models examples.
[`YOLO` usage example docs](./docs/yolo.md)
[`π0.5` usage example docs](./docs/pi05.md)

### 4.2 WebPage (Not all ready yet.)
use it in the web browser: `http://ip:7860`

![index.png](https://raw.githubusercontent.com/happyAnger66-an/model_optimizer/main/img/index.png)

## 5. Contributing

## 6. License

## 7. Contact

+ `email:` `happyAnger66@163.com`

model_optimizer

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README