Open-Rig3R

KO | EN

gitlite — search

#python #hacktoberfest #react #ai #llm #android #rust #deep-learning #docker #java #cli #kubernetes #awesome-list #ios

(★ 12)

The reimplementation of the Rig3R paper

Open-Rig3R 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# 🛰️ Rig3R: Rig-Aware Conditioning for Learned 3D Reconstruction

**Rig3R** is a transformer-based model for multiview 3D reconstruction and camera pose estimation that incorporates **rig-aware conditioning**.
Unlike prior models (e.g., DUSt3R, Fast3R), Rig3R learns to leverage **rig metadata** (camera ID, timestamp, and rig pose) when available, and can **infer rig structure** when not — enabling robust 3D reconstruction across unstructured and rig-based image sets.

This repository is a **reimplementation** of the original [Rig3R paper (Li et al., 2025)](https://arxiv.org/abs/2506.02265), built for clarity, reproducibility, and extensibility.

---

## 🚀 Key Features

* **Rig-aware transformer architecture** using ViT-Large encoder-decoder.
* **Joint prediction** of:
  * Pointmaps (dense 3D coordinates),
  * Pose raymaps (global frame),
  * Rig raymaps (rig-relative frame).
* **Rig discovery**: infers rig calibration from unordered image collections.
* **Closed-form camera pose recovery** from raymaps. 
* **Multi-task learning** with dropout-based metadata conditioning.

---

## 🧠 Model Overview

![Rig3R Model Architecture](https://raw.githubusercontent.com/engichang1467/Open-Rig3R/main/img/rig3r_architecture.png)

---

## ⚙️ Dependencies

### Environment Set Up

Set up environment with [`uv`](https://docs.astral.sh/uv/)

```bash
make setup-env
```

### Activate the Environment

```bash
source .venv/bin/activate
```

### Install Dependencies

```bash
make install
```

---

## 🧩 Training

### Prepare Dataset

For complete setup instructions:

- Waymo: [Dataset Preparation – Waymo](docs/DATA_PREP.md#waymo)
- CO3D: [Dataset Preparation – CO3D](docs/DATA_PREP.md#co3d)

### Download the Pretrained DUSt3R Model

```bash
make setup-train
```

### Start Training

```bash
make train
```

- **Note**: for debugging or training on a less powerful machine, you can run the command below

```bash
make train-debug
```

Recommended configuration values:

* Batch size: 128
* Frames per sample: 24
* Image size: 512×512
* Optimizer: AdamW (lr=1e-4)
* Scheduler: Cosine annealing
* Dropout on metadata: 0.5

- **Note**: Check out here for [more training details](docs/TRAINING_DETAILS.md)

---

## 🧪 Evaluation

**Note**: Make sure to have trained models ready

### Prepare Wayve Scene 101 datasets

```bash
make download-wayve101
```

### Evaluate a pretrained Rig3R model:

```bash
make evaluate
```

- **Note**: for debugging or evaluating on a less powerful machine, you can run the command below

```bash
make evaluate-debug
```

Metrics:

* **RRA / RTA @5°/15°/30°**
* **mAA (mean angular accuracy)**
* **Chamfer distance**
* **Rig discovery accuracy (Hungarian assignment)**

---

## 🔍 Rig Discovery

Run unsupervised rig calibration discovery:

```bash
python scripts/infer_rig_discovery.py --config configs/evaluate.yaml
```

- **Note**: for debugging or running it on a less powerful machine, you can run the command below

```bash
python scripts/infer_rig_discovery.py --config configs/evaluate_mini.yaml
```

### Outputs

The script calculates and reports:
* **Rig ID Accuracy**: How well frames are clustered into cameras.
* **Rig mAA**: Mean Angular Accuracy of the recovered rig extrinsics.
* **Chamfer Distance**: Quality of the reconstructed 3D pointcloud.

It also produces:
* Clustered rig raymaps (internal representation)
* Reconstructed 3D pointclouds
* Estimated rig configurations (extrinsics)

---

## 🖼️ Visualization

Inspect Rig3R predictions interactively with the Open3D visualization pipeline:

```bash
python scripts/visualize.py --config configs/evaluate.yaml
```

- **Note**: on headless machines or WSL, skip the viewer and export instead:

```bash
python scripts/visualize.py --config configs/evaluate.yaml \
    --export outputs/scene.ply --no-show
```

Supports confidence-based filtering, four coloring modes (`rgb`, `per-camera`, `rig-cluster`, `confidence`), camera frustum overlays, and ray segment overlays.

- **Note**: Check out here for [full visualization documentation](docs/VISUALIZATION.md)

---

## 🧰 Citation

If you use this reimplementation, please cite the original paper:

```bibtex
@article{li2025rig3r,
  title={Rig3R: Rig-Aware Conditioning for Learned 3D Reconstruction},
  author={Li, Samuel and Kachana, Pujith and Chidananda, Prajwal and Nair, Saurabh and Furukawa, Yasutaka and Brown, Matthew},
  journal={arXiv preprint arXiv:2506.02265},
  year={2025}
}
```

Open-Rig3R

Open-Rig3R 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Open-Rig3R

Open-Rig3R 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README