KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #dsh-plugin #developer-tools #ai-agent #claude-code #codex #typescript #ai-agents #deepseek-harness #mcp #cli #rust

emx-onnx-cgen

★ 36 Open GitHub ↗

Deterministic ONNX-to-C compiler for embedded and safety-critical systems, generating static, auditable C code with no dynamic memory or runtime dependencies.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<div align="center">
  
[![PyPI - Version](https://img.shields.io/pypi/v/emx-onnx-cgen.svg)](https://pypi.org/project/emx-onnx-cgen)
[![Python](https://img.shields.io/badge/python-3.10%2B-blue)](https://pypi.org/project/emx-onnx-cgen)
[![CI](https://github.com/emmtrix/emx-onnx-cgen/actions/workflows/tests.yml/badge.svg)](https://github.com/emmtrix/emx-onnx-cgen/actions/workflows/tests.yml)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
[![Vulture](https://img.shields.io/badge/dead%20code-vulture-blue?logo=python&logoColor=white)](https://github.com/jendrikseipp/vulture)

</div>

**emmtrix ONNX-to-C Code Generator (emx-onnx-cgen)** compiles ONNX models to portable, deterministic C code for deeply embedded systems. The generated code runs without operating-system services or external runtimes and can be generated fully allocation-free, making it suitable for safety-critical and resource-constrained targets.

It now targets **full standard ONNX opset 26 support** based on **ONNX v1.21.0** and supports **nearly all microsoft ONNX operators** based on **ONNX Runtime 1.27.0**.

Key characteristics:

- **Static, compile-time known memory layout** for parameters, activations, and temporaries — no allocator is needed to size buffers
- **Optional allocation-free code generation** via `--large-temp-threshold 0` (by default, temporaries above 1024 bytes use `malloc`/`free` to avoid stack overflows)
- **Deterministic control flow** (explicit loops, no hidden dispatch or callbacks)
- **No OS dependencies**, using only standard C headers (for example, `stdint.h` and `stddef.h`)
- **Single-threaded execution model**
- **Bitwise-stable code generation** for reproducible builds
- **Readable, auditable C code** suitable for certification and code reviews
- **Generated C output format spec:** [`docs/output-format.md`](docs/output-format.md)
- **Operator-specific implementation notes:** [`docs/operator-notes.md`](docs/operator-notes.md)
- Designed for **bare-metal and RTOS-based systems**

Current coverage highlights:

- **ONNX opset 26** support for the standard operator set shipped with **ONNX 1.21.0**
- **> 99%** operator coverage in the generated support report at [`SUPPORT_OPS.md`](SUPPORT_OPS.md)
- **> 99%** official ONNX backend model coverage in [`ONNX_SUPPORT.md`](ONNX_SUPPORT.md)
- **> 99%** ONNX Runtime-derived artifact coverage in [`ONNX_SUPPORT.md`](ONNX_SUPPORT.md)
- Listed on the official [ONNX Backend Scoreboard](https://onnx.ai/backend-scoreboard/)

For PyTorch models, see the related project [`emx-pytorch-cgen`](https://github.com/emmtrix/emx-pytorch-cgen).

## Goals

- Correctness-first compilation with outputs comparable to ONNX Runtime.
- Deterministic and reproducible C code generation.
- Clean, pass-based compiler architecture (import → normalize → optimize → lower → emit).
- Minimal C runtime with explicit, predictable data movement.
- Serve as the correctness-first ONNX front end for the [emmtrix Edge AI Compiler](https://www.emmtrix.com/tools/emmtrix-edge-ai-compiler), which takes this C code and optimizes it for a concrete target (see [Usage Scenario 3](#3-target-optimized-code-generation-with-the-emmtrix-edge-ai-compiler)).

## Non-goals

- Aggressive performance optimizations in generated C (that is the job of the [emmtrix Edge AI Compiler](https://www.emmtrix.com/tools/emmtrix-edge-ai-compiler)).
- Implicit runtime dependencies or dynamic loading.
- Training/backpropagation support.

## Features

- CLI for ONNX-to-C compilation and verification.
- Deterministic codegen with explicit tensor shapes and loop nests.
- Minimal C runtime templates in `src/emx_onnx_cgen/templates/`.
- ONNX Runtime comparison for end-to-end validation.
- Full standard ONNX opset 26 support on top of ONNX v1.21.0.
- Auto-generated operator and model coverage tracking (see [`SUPPORT_OPS.md`](SUPPORT_OPS.md), [`ONNX_SUPPORT.md`](ONNX_SUPPORT.md), and [`ONNX_ERRORS.md`](ONNX_ERRORS.md)).
- Broad support for ONNX Runtime test artifacts beyond the core standard operator set.
- Supported data types:
  - `bfloat16`, `float16`, `float`, `double`
  - `float8e4m3fn`, `float8e4m3fnuz`, `float8e5m2`, `float8e5m2fnuz`, `float8e8m0` (stored as `uint8_t` with manual conversion to/from `float`)
  - `float4e2m1` (stored as `uint8_t` with manual conversion to/from `float`)
  - `int2`, `uint2`, `int4`, `uint4` (using C23 `_BitInt` types)
  - `int8`, `uint8`, `int16`, `uint16`, `int32`, `uint32`, `int64`, `uint64`
  - `bool`
  - `string` (fixed-size `'\0'`-terminated C strings; see [`docs/output-format.md`](docs/output-format.md))
  - `sequence(<tensor type>)` (fixed-capacity tensor sequences with presence/length metadata; see [`docs/output-format.md`](docs/output-format.md))
  - `optional(<tensor type>)` (optional tensors represented via an extra `_Bool <name>_present` flag; see [`docs/output-format.md`](docs/output-format.md))
  - Not supported: `complex64/complex128`, and ONNX `map/sparse_tensor/opaque` value types.
- Optional support for dynamic dimensions using C99 variable-length arrays (VLAs), when the target compiler supports them.

## Usage Scenarios

### 1. Fully Embedded, Standalone C Firmware

The generated C code can be embedded directly into a bare-metal C firmware or application where **all model weights and parameters are compiled into the C source**.

Typical characteristics:

* No file system or OS required.
* All weights stored as `static const` arrays in flash/ROM.
* Deterministic memory usage with no runtime allocation (combine with `--large-temp-threshold 0` so that temporaries stay on the stack and no `malloc`/`free` is emitted).
* Suitable for:
  * Microcontrollers
  * Safety-critical firmware
  * Systems with strict certification requirements

This scenario is enabled via --large-weight-threshold 0, forcing all weights to be embedded directly into the generated C code.

### 2. Embedded or Host C/C++ Application with External Weights

The generated C code can be embedded into C or C++ applications where **large model weights are stored externally and loaded from a binary file at runtime**.

Typical characteristics:

* Code and control logic compiled into the application.
* Large constant tensors packed into a separate `.bin` file.
* Explicit, generated loader functions handle weight initialization.
* Suitable for:
  * Embedded Linux or RTOS systems
  * Applications with limited flash but available external storage
  * Larger models where code size must be minimized

This scenario is enabled automatically once the cumulative weight size exceeds `--large-weight-threshold` (default: 102400 bytes).

### 3. Target-Optimized Code Generation with the emmtrix Edge AI Compiler

In both of the above scenarios, the generated C code can serve as **input to the
[emmtrix Edge AI Compiler](https://www.emmtrix.com/tools/emmtrix-edge-ai-compiler)**, our
source-to-source compiler that specializes portable C for a concrete embedded target while
preserving functional correctness.

Examples of applied transformations include:

* Kernel fusion and loop restructuring (loop normalization, fusion, simplified control flow)
* Constant propagation, dead code elimination, and pointer resolution
* Memory layout optimization and buffer reuse
* Reduction of internal temporary memory
* Utilization of SIMD / vector instruction sets and dedicated accelerators
* Offloading of large weights to external memory
* Dynamic loading of weights or activations via DMA

Targets supported by the emmtrix Edge AI Compiler include Infineon AURIX TC4x TriCore, Arm
Cortex-A (NEON/SVE), x86 (AVX), and RISC-V (RVV), with code emitted for toolchains such as
TASKING SmartCode, Synopsys ARC MetaWare, GCC, and Clang.

This workflow allows a clear separation between:

* **Correctness-first, deterministic ONNX lowering**, and
* **Target-specific performance and memory optimization**,

while keeping the generated C code readable, auditable, and traceable.

The generated C code is intentionally structured to make such transformations explicit and analyzable, rather than relying on opaque backend-specific code generation.

## Installation

Install the package directly from PyPI (recommended):

```bash
pip install emx-onnx-cgen
```

To use the verification workflow with ONNX Runtime, install the verification extra:

```bash
pip install "emx-onnx-cgen[verify]"
```

The pinned verification runtime is:

- `onnxruntime==1.27.0` on Python 3.11+
- `onnxruntime==1.23.2` on Python 3.10

Minimum Python version: **3.10**.

## Development

For local setup, testing, and contributor workflows, see [`docs/development.md`](docs/development.md).

## Quickstart

Compile an ONNX model into a C source file:

```bash
emx-onnx-cgen compile path/to/model.onnx build/model.c
```

Verify an ONNX model end-to-end against ONNX Runtime (default):

```bash
emx-onnx-cgen verify path/to/model.onnx
```

Models that require extra representative inputs to resolve dynamic shapes are not
supported for code generation. Export them with static shapes instead.

`--test-data-dir` is verification input/output data only. It does not change the
generated C code.

Use `emx-onnx-cgen` as an importable ONNX backend:

```python
import onnx
from onnx.backend import prepare

import emx_onnx_cgen.onnx_backend as emx_backend

model = onnx.load("path/to/model.onnx")
rep = prepare(model, backend=emx_backend)
outputs = rep.run(inputs)
```

The backend module is `emx_onnx_cgen.onnx_backend`. It compiles the ONNX model
to C on demand, builds a temporary executable, and runs that executable through
the standard ONNX backend interface.

You can also call it directly without `onnx.backend.prepare`:

```python
import onnx

from emx_onnx_cgen.onnx_backend import run_model

model = onnx.load("path/to/model.onnx")
outputs = run_model(model, inputs)
```

## CLI Reference

`emx-onnx-cgen` provides two subcommands: `compile` and `verify`.

### Common options

These options are accepted by both `compile` and `verify`:

- `--model-base-dir`: Base directory for resolving the model path (and related paths).
- `--color`: Colorize CLI output (`auto`, `always`, `never`; default: `auto`).
- `--verbose` / `-v`: Enable verbose logging (includes codegen timing).
- `--truncate-weights-after`: Truncate inline weight initializers after `N` values and insert `...` placeholders.
- `--large-weight-threshold`: Store weights in a binary file once the cumulative byte size exceeds this threshold (default: `102400`; set to `0` to disable).
- `--large-temp-threshold`: Heap-allocate (via `malloc`/`free`) temporary buffers larger than this byte threshold; smaller buffers stay on the stack (default: `1024`; set to `0` to disable heap allocation and keep all temporaries on the stack). This applies to generated model temporaries and to generated testbench input/output buffers.
- `--restrict-arrays` / `--no-restrict-arrays`: Enable or disable `restrict` qualifiers on generated array parameters.
- `--fp32-accumulation-strategy`: Accumulation strategy for float32 inputs (`simple` uses float32, `fp64` uses double; default: `simple`).
- `--fp16-accumulation-strategy`: Accumulation strategy for float16 inputs (`simple` uses float16, `fp32` uses float; default: `fp32`).
- `--replicate-ort-bugs`: Compatibility switch for verification/debugging. Enables emulation of known behavior differences of the ONNX Runtime version pinned in `requirements-ci.txt`.
- `--sequence-element-shape`: Declare rank and per-axis maxima for sequence inputs with variable element shapes.
- `--image-decoder-libs`: Comma-separated priority list of decoding libraries backing the `ImageDecoder` operator (default: `stb`). The generated C code detects the image format at runtime via magic bytes and dispatches to the decoder assigned to that format; the first listed library that supports a format wins. Known libraries: `stb` (bundled `stb_image.h`, emitted next to the generated C file; supports `bmp`, `jpeg`, `png`, `pnm`), `libjpeg-turbo` (`jpeg`, bit-exact with the ONNX reference outputs — stb's JPEG decoder is not), `libwebp` (`webp`), `libtiff` (`tiff`), and `openjpeg` (`jpeg2000`). Non-stb libraries require their system development packages and add linker flags to the build (for example `-ljpeg`); `verify` applies them automatically, `compile` reports them. Inputs in a format without a configured decoder produce a zero-filled output tensor at runtime.
- `--input-dim`: Pin a dynamic input dimension to a fixed value so the generated code uses a static array extent instead of a runtime parameter. Repeatable. Use a dim-parameter name (`--input-dim batch=1`) to fix every axis carrying that name across the graph, or an input position (`--input-dim images:0=1`) to fix a single axis. The CLI prints the model's dynamic input dimensions as part of the model report; a fixed dimension is shown inline as `in0[0]=batch -> 4`.

Pinning happens immediately after the model is loaded and **before shape inference**, so the fixed value propagates through the rest of the graph. This has three consequences worth knowing:

- **Avoiding C99 VLAs:** by default a dynamic dimension is emitted as a runtime parameter and the buffers become C99 variable-length arrays (e.g. `const float in[batch][3][4]`). Some targets cannot use these — MSVC does not support VLAs at all, C11 makes them optional (`__STDC_NO_VLA__`), and safety-critical guidelines such as MISRA C forbid them because their stack usage is unbounded and not statically analyzable. Pinning the dimension with `--input-dim` turns the extent into a compile-time constant, so the generated code is plain fixed-size arrays portable to any C compiler.
  - **Resolving dynamic-shape problems:** if a model otherwise fails to generate static C because an input dimension is symbolic or unknown (`tensor 'X' has dynamic dimensions`), pinning that input makes it concrete and lets shape inference derive the dependent intermediate and output shapes. (This only helps for dimensions *derived from the input dimension*; shapes computed from runtime tensor *values*, e.g. `AffineGrid`'s `size`, are recovered separately and are unaffected by `--input-dim`.)
  - **Possible inconsistencies:** `--input-dim` only checks that the target is a dynamic input dimension; it does not validate that the chosen value is consistent with the rest of the graph. Contradictory or partial pinning can therefore make a model that previously stayed dynamic fail during shape inference or lowering — for example pinning two operands of an `Add` to incompatible extents (`--input-dim a:0=3 --input-dim b:0=5` → "Broadcasting mismatch"), or pinning only one of two axes that an operator requires to be equal. Such failures are reported as a normal error, not a crash. Named `dim_param`s are always pinned graph-wide, so `--input-dim batch=N` stays consistent by construction; the risk is mainly with the positional form on unnamed (`?`) axes.

### `compile`

```bash
emx-onnx-cgen compile <model.onnx> [output.c] [options]
```

Options:

- `--model-name`: Override the generated model name (default: output file stem).
- `--emit-testbench`: Emit a JSON-producing `main()` testbench for validation.
- `--testbench-output-format`: Choose the generated testbench output format (`json`, `txt`, `txt-emmtrix`, or `txt-emmtrix:<float>`).
- `--testbench-file`: Emit the testbench into a separate C file at the given path (implies `--emit-testbench`). If not set, the testbench is embedded in the main output C file (legacy behavior).
- `--emit-data-file`: Emit constant data arrays into a companion `_data` C file.
- `--shape-inference-json`: Write a JSON report of the shape inference results to the given path. For every named tensor of the model — inputs, outputs, and internal node outputs — the report contains the shape declared in the model file (`model`, with `dim_param` names and `null` for unknown dims) and the shape computed by the compiler (`inferred`). Weight initializers and compiler-internal temporary names are excluded, so the file is stable across runs and can be used as a shape reference. Shapes reflect the model after `--input-dim` pinning. See the golden test reference [`tests/golden/mixed_ops_dynamic_batch_shapes.json`](tests/golden/mixed_ops_dynamic_batch_shapes.json) for an example report.

### `verify`

```bash
emx-onnx-cgen verify <model.onnx> [options]
```

Options:

- `--cc`: Explicit C compiler command for building the testbench binary.
- `--sanitize`: Enable sanitizer instrumentation when compiling the verification binary (`-fsanitize=address,undefined`). If `EMX_ENABLE_SANITIZE` is set, it overrides this flag.
- `--per-node-accuracy`: Also compare intermediate tensor outputs and print max error per node.
- `--test-data-dir`: Seed verification inputs from `input_*.pb` files instead of generating random testbench inputs.
- `--test-data-inputs-only`: Read only `input_*.pb` from `--test-data-dir` and still compare outputs against the selected runtime.
- `--max-ulp`: Maximum allowed ULP distance for floating outputs (default: `100`).
- `--atol-eps`: Absolute tolerance as a multiple of machine epsilon for floating outputs (default: `1.0`).
- `--runtime`: Runtime backend for verification (`onnxruntime` or `onnx-reference`, default: `onnxruntime`).
- `--expected-checksum`: Exit early with `CHECKSUM` when the generated C checksum matches the expected SHA-256.
- `--replicate-ort-bugs`: Verification-only compatibility mode to reproduce known behavior differences of the ONNX Runtime version pinned in `requirements-ci.txt`.
- `--temp-dir-root`: Root directory in which to create a temporary verification directory (default: system temp dir).
- `--temp-dir`: Exact directory to use for temporary verification files (default: create a temporary directory).
- `--keep-temp-dir`: Keep the temporary verification directory instead of deleting it.

How verification works:

1. **Compile with a testbench**: the compiler is invoked with `--emit-testbench`,
   generating a C program that runs the model and prints inputs/outputs as JSON.
2. **Build and execute**: the testbench is compiled with the selected C compiler
   (`--cc`, `CC`, or a detected `cc/gcc/clang`) and executed in a temporary
   directory.
3. **Run runtime backend**: the JSON inputs from the testbench are fed to the
   selected runtime (`onnxruntime` or `onnx-reference`) using the same model.
   The compiler no longer ships a Python runtime evaluator.
4. **Compare outputs**: floating outputs are compared by maximum ULP distance.
   Floating-point verification first ignores very small differences up to
   **--atol-eps × [machine epsilon](https://en.wikipedia.org/wiki/Machine_epsilon) of
   the evaluated floating-point type**, treating such values as equal. For
   values with a larger absolute difference, the ULP distance is computed, and
   the maximum ULP distance is reported; non-floating outputs must match
   exactly.
   Missing outputs or mismatches are treated as failures.
5. **ORT unsupported models**: when using `onnxruntime`, if ORT reports
   `NOT_IMPLEMENTED`, verification is skipped with a warning (exit code 0).

## Troubleshooting

Common problems and how to resolve them.

### Dynamic shapes and shape-inference failures

A model whose inputs have dynamic dimensions (a symbolic `dim_param` such as
`batch`, or a fully unknown axis) is compiled with those axes as C99
variable-length-array runtime parameters by default, e.g.
`void model(int batch, const float in[batch][3][4], ...)`. The model report
lists what is dynamic:

```
  Dynamic input dimensions (1): in[0]=batch
```

**Avoiding VLAs:** even when the dynamic build succeeds, the runtime-parameter
form relies on C99 variable-length arrays. If your toolchain cannot use them —
MSVC has no VLA support, C11 makes them optional (`__STDC_NO_VLA__`), and
MISRA C / safety-critical guidelines forbid them due to unbounded stack usage —
pin the dimensions with `--input-dim` (below) so the generated code uses plain
fixed-size arrays instead.

Dynamic input dimensions are also the root cause of a whole family of failures,
because the code generator (and ONNX shape inference itself) often cannot
resolve the rest of the graph while an input axis is still symbolic. These
surface in several different ways, for example:

- `Code generation requires static shapes. Reason: tensor 'X' has dynamic
  dimensions ...` — a dynamic dimension reaches a place that needs a concrete
  size (a buffer, a reshape, ...).
- `ONNX shape inference failed` — inference cannot make progress with the
  unresolved input shapes.
- Operator-specific lowering errors that mention `-1`/dynamic extents,
  unresolved `Reshape`/`MatMul`/broadcast shapes, or a shape that "could not be
  inferred".

The general fix is to make the input shapes concrete *before* shape inference
runs, by pinning the dynamic dimensions with [`--input-dim`](#common-options).
Because pinning happens before inference, the fixed values propagate through the
graph and usually let inference resolve everything downstream:

```bash
# Fix every axis named "batch" graph-wide:
emx-onnx-cgen compile model.onnx --input-dim batch=1
# Or fix a single axis positionally (use this for unnamed "?" axes):
emx-onnx-cgen compile model.onnx --input-dim images:0=1
```

Caveats:

- `--input-dim` only helps for dimensions *derived from* the pinned input
  dimension. Shapes computed from runtime tensor *values* (e.g. `AffineGrid`'s
  `size`) are recovered automatically and are not affected by `--input-dim`.
- It is a sharp tool: `--input-dim` only checks that the target is a dynamic
  input dimension, not that the chosen value agrees with the rest of the graph.
  Contradictory or partial choices can therefore *introduce* shape-inference or
  lowering failures (e.g. pinning two operands of an `Add` to incompatible
  extents reports a "Broadcasting mismatch"). Named `dim_param`s are pinned
  graph-wide and stay consistent by construction; the risk is mainly the
  positional form on unnamed (`?`) axes.

### "Code generation requires explicit ragged-sequence bounds"

A sequence input with variable element shapes needs explicit per-axis maxima.
Declare them with [`--sequence-element-shape`](#common-options), e.g.
`--sequence-element-shape boxes=[<=100,4]`.

### "No C compiler found" (verify)

`verify` builds and runs a testbench, so it needs a C compiler. Provide one via
`--cc`, the `CC` environment variable, or install a `cc`/`gcc`/`clang` on `PATH`.

## Official ONNX test coverage

`emx-onnx-cgen` tracks support using generated coverage reports checked into the repository and is listed on the official [ONNX Backend Scoreboard](https://onnx.ai/backend-scoreboard/).

- **Standard ONNX operator support:** [`SUPPORT_OPS.md`](SUPPORT_OPS.md) consistently reports **> 99%** verified operator coverage. The remaining unsupported entry is the non-standard contrib operator `com.microsoft::SparseToDenseMatMul`.
- **Official ONNX backend models:** [`ONNX_SUPPORT.md`](ONNX_SUPPORT.md) reports **> 99%** verified ONNX file coverage against **ONNX 1.21.0**.
- **ONNX Runtime artifact corpus:** [`ONNX_SUPPORT.md`](ONNX_SUPPORT.md) also reports **> 99%** verified coverage for the exported ONNX Runtime artifact set.

- [`ONNX_SUPPORT.md`](ONNX_SUPPORT.md): overview of ONNX models and their current verification status.
- [`ONNX_ERRORS.md`](ONNX_ERRORS.md): summary of the most common verification outcomes and failure reasons.
- [`SUPPORT_OPS.md`](SUPPORT_OPS.md): list of ONNX operators and whether they are currently supported, with links to operator-specific implementation notes and relevant CLI options.

## Related Projects

- **emmtrix Edge AI Compiler**  
  Source-to-source compiler that turns the portable C generated here into target-optimized C for embedded processors (kernel fusion, SIMD/vector intrinsics, memory-footprint reduction).  
  https://www.emmtrix.com/tools/emmtrix-edge-ai-compiler
- **emx-pytorch-cgen**  
  A PyTorch-to-C compiler following the same design principles as emx-onnx-cgen, but operating directly on PyTorch models instead of ONNX graphs.  
  https://github.com/emmtrix/emx-pytorch-cgen
- **onnx2c**  
  An ONNX-to-C code generator with a different design focus and code generation approach.  
  https://github.com/kraiskil/onnx2c

## Supporting Projects

- **emx-regex-cgen**  
  A regex-to-C code generator used to implement the ONNX `RegexFullMatch` operator in emx-onnx-cgen.  
  https://github.com/emmtrix/emx-regex-cgen
- **emx-ort-test-artifacts**  
  Repository containing exported ONNX test artifacts (`*.onnx` / `*.pb` files) produced by the ONNX Runtime test infrastructure.  
  https://github.com/emmtrix/emx-ort-test-artifacts
  
## Maintained by

This project is maintained by [emmtrix Technologies GmbH](https://www.emmtrix.com).

emx-onnx-cgen

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README