aria2mano

KO | EN

gitlite — search

#javascript #python #typescript #go #android #deep-learning #linux #java #cli #vue #security #ios #openai #open-source

(★ 11)

Aria Gen 1/2 toolkit: MANO hand fitting, stereo depth, and real-time streaming.

aria2mano Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# aria2mano

Aria [Gen 1](https://facebookresearch.github.io/projectaria_tools/docs/intro) / [Gen 2](https://facebookresearch.github.io/projectaria_tools/gen2/) toolkit with two workflows:

1) [**Offline Processing (Gen 1/2)**](#offline-processing-gen-12) - VRS recordings → ML-ready data with anatomically correct [MANO](https://mano.is.tue.mpg.de/) hands.
2) [**Real-time Streaming (Gen 2)**](#real-time-streaming-gen-2) - Live stereo depth (up to 30 FPS) + MANO fitting over USB or untethered WiFi.

## Installation

Installation tested on Ubuntu 22.04 LTS / 24.04 LTS with CUDA 12.8, PyTorch 2.9.1, and Python 3.10.

```bash
# 1) Clone and install dependencies
git clone git@github.com:KevinyWu/aria2mano.git && cd aria2mano
conda env create -f environment.yaml && conda activate aria2mano
pip install torch==2.9.1 torchvision==0.24.1 torchaudio==2.9.1 --index-url https://download.pytorch.org/whl/cu128
pip install --no-build-isolation git+https://github.com/mattloper/chumpy.git@580566e

# 2) Install aria2mano
pip install -e .

# 3) Download required assets listed below
```

<details><summary>📦 Requirements</summary>

**MANO Models**

- [Register](https://mano.is.tue.mpg.de/) → download and unzip the MANO models → copy contents of `mano_v*_*/` to `assets/mano_models/`
- ***Required*** for MANO hand mesh fitting

**S2M2 Models**

- Download weights from [S2M2 repository](https://github.com/junhong-3dv/s2m2) → copy all four model weights to `assets/s2m2/`
- ***Required*** for depth estimation

**Aria Devignetting Masks**

- Download from [Aria Gen 1 Docs](https://facebookresearch.github.io/projectaria_tools/docs/data_utilities/advanced_code_snippets/image_utilities#image-devignetting) → unzip and copy `new_isp` and `old_isp` to `assets/aria_devignetting_masks/`
- ***Optional*** for Gen 1 image enhancement

</details>

## Offline Processing (Gen 1/2)

<img src="assets/img/aria2mano.gif" alt="aria2mano" width="100%"/>
 
 Process Aria recordings with MANO hand fitting and metric stereo depth.

- ***MANO hand fitting*** - Anatomically correct hand meshes from Aria hand tracking landmarks.
- ***Stereo depth*** - Dense depth maps via [S2M2](https://junhong-3dv.github.io/s2m2-project/) (Gen 2 only).
  - > S2M2 XL (30 FPS w/ TensorRT) outperforms FoundationStereo (2 FPS; similar quality) and Fast-FoundationStereo (45 FPS w/ TensorRT; lower quality).
- ***Easy data access*** - `AriaLoader` class for simple frame-by-frame data retrieval.
- ***Web visualization*** - Interactive 3D viewer with [Viser](https://viser.studio/main/).

### Quick Start

After downloading VRS recordings to your local machine:

```bash
# 1) Sort VRS files
./scripts/sort_vrs.sh "<path/to/all/vrs/files>"

# 2) Run MPS
aria_mps single -i "<path/to/all/vrs/files>"

# 3) Process Aria recordings (~3.8GB/job, set `--max-jobs-per-gpu` accordingly)
./scripts/process_aria.sh "<path/to/all/vrs/files>" --max-jobs-per-gpu 1

# 4) Viser visualization
python -m aria2mano.viser_viewer --aria-path "<path/to/aria/recording/folder>"
```

### Detailed Usage

<details><summary>🗂️ 1) Sort VRS Files</summary>

Organize recordings and run health checks. For Gen 2, also extracts on-device data.

```bash
./scripts/sort_vrs.sh "<path/to/all/vrs/files>" [OPTIONS]

# See all options
./scripts/sort_vrs.sh --help
```

> **Note**: As of 03/16/2026, a firmware bug causes sporadic health check failures. The script trims the first second of affected recordings by default (turn off with `--no-trim`).

</details>

Run Aria's [Machine Perception Services](https://facebookresearch.github.io/projectaria_tools/gen2/technical-specs/mps/mps_cli_guide) for cloud-based SLAM and hand tracking.

```bash
aria_mps single -i "<path/to/all/vrs/files>"
```

For Gen 2 recordings, running MPS is optional. Use `--on-device` flag in step 3 to process with on-device data instead. However, **MPS data is more accurate**. See [Troubleshooting MPS Errors](#appendix) for common error codes.

> **⚠️ MPS Warning**: SLAM poses and hand tracking are not accurate for the first ~1-2 seconds of the recording.

</details>

<details><summary>⚙️ 3) Process Aria Recordings</summary>

Recordings with existing outputs are skipped by default. Pass `--reprocess` to overwrite.

**Batch processing**

```bash
./scripts/process_aria.sh "<path/to/all/vrs/files>" --max-jobs-per-gpu 1 [OPTIONS]

# See all options
./scripts/process_aria.sh --help
```

> **⚠️ GPU Memory Warning**: Each job uses ~3.8GB VRAM with PyTorch or ~2.0GB with TensorRT. Set `--max-jobs-per-gpu N` (default: 1) based on your GPU capacity.

**Single recording**

```bash
# Full processing (default)
python -m aria2mano.aria_processor --aria-path "<path/to/aria/recording/folder>"

# Fast mode (no MANO optimization or depth)
python -m aria2mano.aria_processor --aria-path "<path/to/aria/recording/folder>" --no-optimize-mano --no-get-depth
```

Each run prints a performance summary table. See [Benchmarking](#appendix) for details.

**Common overrides**

Both batch and single recording commands accept the same processing flags.

| Flag | Effect |
| ---- | ------ |
| `--reprocess` | Overwrite existing outputs (skipped by default) |
| `--no-optimize-shape` | Skip per-frame shape optimization ([MANO Optimization](#appendix)) |
| `--frozen-shape` | Use a pre-defined MANO shape ([MANO Optimization](#appendix)) |
| `--tensorrt` | Use the TensorRT depth engine instead of PyTorch; off by default |
| `--on-device` | Gen 2 only: force processing to use on-device data |
| `--downsample-rgb N` | Shrink saved RGB JPEGs; stored `rgb_camera.K`/`width`/`height` scale accordingly |

**Summary CSV**

This might be slow for large datasets, pass `--simple` to get a quick summary.

```bash
./scripts/summarize_data.sh "<path/to/all/vrs/files>"
```

> **Note**: Wrist 6DoF (`T_device_wrist`) comes from Aria, not MANO; MANO only solves finger `pose` and `shape`. See [Hand Tracking Conventions](#appendix) for details.

</details>

<details><summary>🎨 4) Viser Visualization</summary>

Launch the [Viser](https://viser.studio/main/) web viewer. The floor plane is detected with RANSAC on the MPS semidense point cloud.

```bash
python -m aria2mano.viser_viewer --aria-path "<path/to/aria/recording/folder>"

# See all options
python -m aria2mano.viser_viewer --help
```

Open `http://localhost:8080` in your browser.

> **Tip**: For long recordings, you may want to set `--max-frames`, `--start-frame` and `--end-frame` (both support negative indices), or apply downsampling to the depth point cloud and images with `--downsample`.
>
> **⚠️ MPS Warning**: SLAM poses and hand tracking are not accurate for the first ~1-2 seconds of the recording, hence `--start-frame` defaults to 20.

</details>

<details><summary>💿 Aria Data Loader</summary>

Access processed data programmatically:

```python
from aria2mano.aria_loader import AriaLoader

loader = AriaLoader(aria_path="path/to/aria/recording/folder")

# Metadata
num_frames = loader.get_num_frames()
fps = loader.get_fps()

# Per-frame data (see aria_data_classes.py for structure)
frame = loader.get_frame_data(index=0)
rgb = loader.get_rgb_image(index=0)
point_cloud = loader.get_point_cloud()

# Gen 2 only
stereo_left = loader.get_stereo_left_image(index=0)
stereo_right = loader.get_stereo_right_image(index=0)
depth = loader.get_depth_image(index=0)
depth_mask = loader.get_depth_mask_image(index=0)
```

</details>

<details><summary>📁 Data Format</summary>

After processing, each recording has this structure. See [`aria_data_classes.py`](src/utils/aria_data_classes.py) for the contents of the `aria_data.pkl` file, [Data Format Details](#appendix) for coordinate conventions, and [Hand Tracking Conventions](#appendix) for landmark indexing.

```text
recording_name/
├── video.vrs                    # Original Aria recording
├── vrs_health_check.json
├── mps_video_vrs/               # Cloud MPS results
│   ├── hand_tracking/
│   │   ├── hand_tracking_results.csv
│   │   └── summary.json
│   └── slam/
│       ├── closed_loop_trajectory.csv
│       ├── open_loop_trajectory.csv
│       ├── semidense_observations.csv.gz
│       ├── semidense_points.csv.gz
│       ├── online_calibration.jsonl
│       └── summary.json
├── on_device/                   # Gen 2 only: on-device results
│   ├── eye_gaze/
│   ├── hand_tracking/
│   └── slam/
└── processed/                   # Output from aria_processor.py
    ├── aria_data.pkl            # Metadata + per-frame hand tracking
    ├── point_cloud.npz          # World-frame 3D points (Nx3)
    ├── rgb/{:08d}.jpg           # Undistorted RGB images
    ├── stereo_left/{:08d}.png   # Gen 2 only: scanline-rectified stereo left
    ├── stereo_right/{:08d}.png  # Gen 2 only: scanline-rectified stereo right
    ├── depth/{:08d}.png         # Gen 2 only: depth maps (uint16, 1mm units)
    └── depth_mask/{:08d}.png    # Gen 2 only: depth validity mask (255=valid)
```

</details>

<details><summary>⚡ TensorRT Acceleration</summary>

We can speed up stereo depth estimation with [TensorRT](https://developer.nvidia.com/tensorrt), NVIDIA's inference optimizer. Engines are tied to the GPU architecture and TensorRT version they were built on. For best performance, build the engine on the same GPU that will run inference.

**Build the engine**

See the [TensorRT docs](https://docs.nvidia.com/deeplearning/tensorrt/latest/installing-tensorrt/installing.html) for installation instructions. We use CUDA 12.8 and install with pip, hence `tensorrt-cu12`.

```bash
# Install dependencies
pip install onnxruntime-gpu onnxscript tensorrt-cu12

# Build the engine
python -m aria2mano.build_tensorrt
```

This exports S2M2 to ONNX at `assets/s2m2/onnx/`, then builds a TensorRT engine at `assets/s2m2/trt/`.

**Using the TensorRT engine**

Enable/disable TensorRT in [`config.yaml`](src/config.yaml), where you can also specify engine parameters. Alternatively, override the config per run with `--tensorrt` or `--no-tensorrt`.

```bash
# Batch offline processing
./scripts/process_aria.sh "<path/to/all/vrs/files>" --tensorrt

# Single recording / streaming subscriber
python -m aria2mano.aria_processor --aria-path "<path/to/aria/recording/folder>" --tensorrt
python -m aria2mano.stream_sub --pub-host "<publisher-ip>" --tensorrt
```

**Inference speed comparison on NVIDIA RTX 5090**

`model_type: XL`, `num_refine: 3`, `img_width: 512`, `img_height: 512`, `precision: fp16`

| Backend | FPS | Memory |
| ------- | --- | ------ |
| PyTorch | ~12 | 4744M (3716M with expandable segments \*) |
| TensorRT | ~30 | 1994M |

> \* Enable with `export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True`.

</details>

## Real-time Streaming (Gen 2)

<img src="assets/img/aria2mano_live.gif" alt="aria2mano_live" width="100%"/>
 
 30 FPS stereo depth and MANO fitting, streaming wirelessly over WiFi.

- ***Wireless streaming with pub/sub*** - Start streaming on a laptop from anywhere; [commlink](https://github.com/jeffacce/commlink) TCP streams to a powerful remote machine for stereo depth/MANO fitting.
- ***Subscriber can run anywhere*** - Only the publisher machine needs Aria Client SDK; the subscriber runs on any platform even if Client SDK is unsupported (e.g. ARM devices: Jetson, Raspberry Pi, etc).
- ***Teleop mode*** - Streams only hands + VIO at native 30 Hz for low-thermal, long-duration sessions.

### Quick Start

For WiFi streaming (mode B), connect the glasses to the same WiFi network as the publisher machine, then:

```bash
# 1) On the publisher (glasses plugged in via USB)
aria_gen2 streaming install-certs # For the first stream on this device
python -m aria2mano.stream_pub --wifi

# 2) Unplug the glasses (streaming may pause for a few seconds)

# 3) On the subscriber (omit --pub-host if on the same machine)
python -m aria2mano.stream_sub --pub-host "<publisher-ip>"

# 4) Stop streaming: press the action button on the glasses
#    Ctrl+C doesn't stop streaming in WiFi mode
```

> [!TIP]
> For best real-time depth performance (~30 FPS inference), build a TensorRT engine with S2M2 XL on the subscriber machine, then enable it per run with `--tensorrt`.

### Detailed Usage

<details><summary>📨 Pub/Sub Streaming</summary>

Streaming is split into a publisher and subscriber that talk over [commlink](https://github.com/jeffacce/commlink), an RPC layer built on ZeroMQ. We use custom streaming profiles with reduced RGB resolution (2560x1920 → 640x480) to ensure stable streaming without thermal issues. See [Custom Streaming Profiles](#appendix) for details.

***Two main motivations for this setup:***

1) Start streaming from anywhere with a laptop; run stereo depth and MANO fitting on a remote machine with a GPU.
2) The publisher owns all `aria` and `projectaria_tools` imports and must run on a [Aria Client SDK supported device](https://facebookresearch.github.io/projectaria_tools/gen2/ark/client-sdk/start#supported-platforms); the subscriber has no Aria dependencies and runs anywhere even if Client SDK is unsupported (e.g. ARM devices: Jetson, Raspberry Pi).

![aria_streaming](https://raw.githubusercontent.com/KevinyWu/aria2mano/main/assets/img/aria_streaming.png)

There are two links to set up:

1) ***Glasses ↔ Publisher*** - USB (default) or WiFi (`--wifi`, glasses and publisher must be on the same WiFi network).
2) ***Publisher ↔ Subscriber*** - TCP on `pub_port` (default `5555`); pass the publisher's reachable IP to the subscriber (LAN, [Tailscale](https://tailscale.com/), `localhost` if on the same machine, etc.).

Open `http://localhost:8080` from the subscriber machine to view. The publisher sends an initial `meta` handshake so the subscriber auto-configures its Viser GUI to match the active streams.

```bash
# See all options
python -m aria2mano.stream_pub --help
python -m aria2mano.stream_sub --help
```

</details>

Plug the glasses into the publisher machine via USB. The subscriber can either be on the same machine as the publisher or a remote machine reachable over TCP (LAN, [Tailscale](https://tailscale.com/), etc.).

```bash
# 1) On the publisher (glasses plugged in via USB)
python -m aria2mano.stream_pub

# 2) On the subscriber (omit --pub-host if on the same machine)
python -m aria2mano.stream_sub --pub-host "<publisher-ip>"

# 3) Ctrl+C on the publisher to stop streaming
```

> **Quality Tips**: When pub/sub are on the same machine, streaming quality can be increased:
>
> - Raise image frequency in [`streaming_profile.json`](assets/streaming_profile.json) (i.e. 10 Hz → 15 Hz, **must bump SLAM/RGB cameras together**).
> - Raise RGB image size in [`streaming_profile.json`](assets/streaming_profile.json) (i.e. 640x480 → 2560x1920, must bump `rgbCamera.resolution` to `R_4032x3024` and preserve 4:3 aspect ratio).
> - Raise `--jpeg-quality` on the publisher (i.e. 80 → 90).
> - Use the built-in streaming [profile9](https://facebookresearch.github.io/projectaria_tools/gen2/technical-specs/device/profile) via `python -m aria2mano.stream_pub --profile-name profile9`.
> - Quality has tradeoffs. E.g. increase image size but reduce the frequency 10 Hz → 5 Hz.

</details>

Use WiFi when you want the glasses untethered. The glasses and the publisher machine must be on the same WiFi network - see [WiFi router streaming](https://facebookresearch.github.io/projectaria_tools/gen2/ark/client-sdk/streaming#wifi-router-streaming).

> **⚠️ Streaming Warning**: If the glasses overheat on WiFi, bump `--batch-period-ms` (e.g. `python -m aria2mano.stream_pub --wifi --batch-period-ms 200`).

```bash
# 1) On the publisher (glasses plugged in via USB)
python -m aria2mano.stream_pub --wifi

# 2) Unplug the glasses (streaming may pause for a few seconds)

# 3) On the subscriber (omit --pub-host if on the same machine)
python -m aria2mano.stream_sub --pub-host "<publisher-ip>"

# 4) Stop streaming: press the action button on the glasses
#    Ctrl+C doesn't stop streaming in WiFi mode
```

> **Latency Tips**: To reduce end-to-end glasses → viewer delay:
>
> - Run `stream_pub` and `stream_sub` on the same machine (skips the network hop).
> - Lower image frequency in [`streaming_profile.json`](assets/streaming_profile.json) (i.e. 10 Hz → 5 Hz).
> - Lower RGB image size in [`streaming_profile.json`](assets/streaming_profile.json) (i.e. 640x480 → 320x240, must preserve 4:3 aspect ratio).
> - Lower `--jpeg-quality` on the publisher (i.e. 80 → 65).

</details>

<details><summary>🤖 Mode C - Teleop</summary>

30 Hz hands + VIO only using [`teleop_profile.json`](assets/teleop_profile.json), designed with robot teleoperation in mind. Combined with subscriber `--no-optimize-mano`, this is the lightest path, which avoids the thermal load of long image streams. Both USB and WiFi modes are supported, WiFi example below.

```bash
# 1) On the publisher (glasses plugged in via USB)
python -m aria2mano.stream_pub --hands-only --wifi

# 2) Unplug the glasses (streaming may pause for a few seconds)

# 3) On the subscriber (omit --pub-host if on the same machine)
python -m aria2mano.stream_sub --pub-host "<publisher-ip>" --no-optimize-mano

# 4) Stop streaming: press the action button on the glasses
#    Ctrl+C doesn't stop streaming in WiFi mode
```

</details>

## Appendix

<details><summary>📐 Data Format Details</summary>

- ***Transforms***: Use `T_destination_source` notation. All hand landmarks are in [device frame](https://facebookresearch.github.io/projectaria_tools/gen2/technical-specs/coordinate/cpf-coor#nuance-1-coordinate-frames-as-se3-are-actually-poses-from-an-arbitrary-reference-device-frame).
- ***Camera frames***: OpenCV convention (+X right, +Y down, +Z forward), origin at optical center. See [Aria camera coordinate system](https://facebookresearch.github.io/projectaria_tools/gen2/technical-specs/coordinate/3d-coor#camera-coordinate-system-convention).
- ***`--on-device` mode***: `T_world_device = T_odometry_device` (from on-device VIO, Gen 2 only).
- ***RGB images***: Undistorted to linear camera model.
- ***Stereo images***: Scanline-rectified to linear camera model (Gen 2 only).
- ***Depth maps***: `uint16` format, 1 unit = 1mm, in stereo left camera frame (Gen 2 only).
- ***Depth masks***: `uint8` format, 255=valid, 0=invalid (Gen 2 only).

</details>

<details><summary>🖐️ Hand Tracking Conventions</summary>

The canonical Aria wrist frame and the canonical MANO wrist frame coincide.

| Direction | Left Aria/MANO | Right Aria/MANO |
| --- | --- | --- |
| Wrist → middle | +X | -X |
| Palm normal | -Y | -Y |
| Wrist → thumb | +Z | +Z |

> **Note**: The image below shows the left hand. For the right hand, the wrist frame's Z and X axes are flipped, i.e. rotated 180 degrees around the Y axis. The +Z axis points towards the thumb on both hands.

Aria and MANO use different indexing for their 21 hand landmarks:

- **Aria**: Has palm center (idx 20), no thumb CMC
- **MANO**: Has thumb CMC (idx 1), no palm center

<table align="center">
 <tr>
 <td width="50%" align="center"><img src="assets/img/aria_hand.png" alt="Aria hand landmarks" width="100%"/></td>
 <td width="50%" align="center"><img src="assets/img/mano_hand.png" alt="MANO hand landmarks" width="100%"/></td>
 </tr>
 <tr>
 <td align="center">Aria Hand Landmarks</td>
 <td align="center">MANO Hand Landmarks</td>
 </tr>
</table>

In `aria_data.pkl`, landmarks are stored in their original conventions. MANO optimization ignores thumb CMC and palm landmarks, using the 19 shared landmarks. See [`mano_hand_utils.py`](src/utils/mano_hand_utils.py) for conversion utilities.

</details>

<details><summary>🔧 MANO Optimization</summary>

MANO optimization code is located in [`mano_optimizer.py`](src/utils/mano_optimizer.py) and uses the MANO implementation from [manotorch](https://github.com/lixiny/manotorch). Note that wrist 6DoF (`T_device_wrist`) is taken from Aria hand tracking, not MANO. Landmarks are transformed into the wrist frame before fitting, so MANO only solves the 15 finger joint rotations (`pose`) and shape (`shape`) relative to a wrist at the origin.

> **Tip**: For temporal data, initializing the `ManoOptimizer.optimize_mano` with the previous frame's pose and shape (`init_pose` and `init_shape`) vastly improves optimization speed and quality.

Two CLI flags control shape behavior:

| Flag | Default | Effect |
| ---- | ------- | ------ |
| `optimize_shape` | `true` | Optimize betas per frame. If `false`, betas are initialized from the previous frame (only optimized at first frame). |
| `frozen_shape` | `false` | Load betas from `assets/mano_rhand_shape.npy` and freeze them. Overrides `optimize_shape`. |

- **`optimize_shape: false`** - Useful when you only care about pose fitting and want faster optimization or want to keep the shape fixed.
- **`frozen_shape: true`** - Useful when you have a calibrated **right hand** shape and want that exact shape used consistently across all frames/recordings. Save a NumPy array of 10 MANO **right hand** beta coefficients to `assets/mano_rhand_shape.npy` (accepted formats: 1D `(10,)` or 2D `(1, 10)`). The left hand will use the same shape.

</details>

<details><summary>🎛️ Custom Streaming Profiles</summary>

The publisher picks a profile JSON as shown in the *Select with* row below. You can also pass your own profile JSON with `--profile-path`.

| Stream | Key | [`streaming`](assets/streaming_profile.json) | [`teleop`](assets/teleop_profile.json) | [`profile9`](https://facebookresearch.github.io/projectaria_tools/gen2/technical-specs/device/profile) |
| ------ | --- | ----------- | ------------ | ------------ |
| ***Select with*** | -- | default | `--hands-only` | `--profile-name profile9` |
| Stereo SLAM (L+R) | `slamCameras` | 10 Hz, 512x512 | -- | 10 Hz, 512x512 |
| RGB camera | `rgbCamera` | 10 Hz, 640x480 | -- | 5 Hz, 2560x1920 |
| Hand tracking | `ht` | 30 Hz | 30 Hz | 30 Hz |
| VIO (for online calibration) | `vio` | 10 Hz | 10 Hz | 10 Hz |
| High-frequency pose | `vio_high_frequency_pose` | 800 Hz | 800 Hz | -- |

RGB resolution is controlled by `rgbCamera.resolution` (camera sensor mode, e.g. `R_2016x1512`) plus the `width`/`height` fields (downscale target). SLAM cameras stream at their native resolution of 512x512. High-freq pose rate is reduced by `--vio-skip 10` (default every 10th sample → ~80 Hz); pass `--vio-skip 1` to disable.

> **Note**: Any custom `width`/`height` must preserve the sensor's native 4:3 aspect ratio (e.g. 2560x1920, 2016x1512, 640x480, 320x240). Non-4:3 values will not result in linear rectification.

</details>

<details><summary>📊 Benchmarking</summary>

MANO optimization (CPU) and depth estimation (GPU) performance highly depends on your hardware. Depth estimation can be sped up with [TensorRT](#detailed-usage) while MANO optimization trades off speed and accuracy by adjusting optimization parameters in [`config.yaml`](src/config.yaml).

The following benchmarks are performed on a AMD Ryzen Threadripper PRO 7975WX 32-Cores CPU and NVIDIA RTX 5090 GPU.

**Offline processing**

Batch processing allocates multiple `aria2mano.aria_processor` jobs on the machine, so per-recording performance will be affected by the number of jobs. We suggest processing a single recording with `aria2mano.aria_processor` and checking the performance table terminal output. Here is an example on a 20 second recording (200 frames).

| Stage | FPS | ms/frame |
| --------------------------- | -----: | -------: |
| MANO optimization (CPU) | 27.2 | 36.8 |
| Stereo rectification (CPU) | 103.4 | 9.7 |
| Depth estimation (GPU, PyTorch) | 12.1 | 82.9 |
| Depth estimation (GPU, TensorRT) | 30.9 | 32.3 |

**Real-time streaming**

Real-time streaming performance depends on both hardware and the network. In this setup, Aria glasses stream to an M4 Macbook Pro over WiFi, which publishes to the aforementioned machine located **3 miles away**. This table is a snapshot of the streaming performance with TensorRT enabled for depth estimation.

| Stream | FPS | p50 latency (ms) | p95 latency (ms) | Bandwidth |
| ------------ | ---: | -----: | -----: | ---------: |
| Online calibration | 2.0 | 96.3 | 225.2 | 12.8 kbps |
| High-frequency VIO | 57.2 | 30.6 | 44.2 | 123.1 kbps |
| Hand tracking | 29.2 | 35.9 | 53.3 | 478.6 kbps |
| Images (RGB + stereo left/right) | 10.0 | 52.5 | 65.5 | 7.09 Mbps |
| MANO optimization (CPU) | 37.5 | -- | -- | -- |
| Depth estimation (GPU, TensorRT) | 30.1 | -- | -- | -- |

</details>

<details><summary>🚨 Troubleshooting MPS Errors</summary>

See all [MPS error codes](https://facebookresearch.github.io/projectaria_tools/gen2/ark/support/mps#error-codes).

| Code | Description | Action |
| ------ | ------------- | -------- |
| 102 | Health check failed | Discard recording |
| 108 | Failed to query server | Retry |
| 115 | Error checking processing status | Retry |
| 50XX | Internal server error | Force re-upload: `aria_mps single -i "<path>" -s a --force` |

</details>

## Citation

If you use aria2mano, please cite our paper [Human Universal Grasping](https://grasping.io/), which aria2mano is originally developed for:

```bibtex
@article{wu2026hug,
    title={Human Universal Grasping},
    author={Kevin Yuanbo Wu and Tianxing Zhou and Isaac Tu and Billy Yan and Irmak Guzey and David Fouhey and Dandan Shan and Lerrel Pinto},
    journal={arXiv preprint arXiv:2606.17054},
    year={2026}
}
```

Additionally, we thank the authors of [MANO](https://mano.is.tue.mpg.de/), [manotorch](https://github.com/lixiny/manotorch), and [S2M2](https://github.com/junhong-3dv/s2m2) for their excellent work.

aria2mano

aria2mano Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

aria2mano

aria2mano Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README