KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #dsh-plugin #developer-tools #ai-agent #claude-code #codex #typescript #ai-agents #deepseek-harness #mcp #cli #rust

RGBManip

★ 42 Open GitHub ↗

Official implementation of RGBManip (ICRA2024)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Adapose

## Install

Install dependencies using `pip install -r requirements.txt`. The instalation is tested with `setuptools==65.6.3`. Some later versions may not be compatible.

*Potential Dependency Inssues:* Users have reported `Module not Found` error with mplib. This project is developed when the newest version of mplib is `mplib==0.0.8`. Newer versions if mplib may not be compatible with this project. We recommend using `mplib<=0.1.0`

## Download Dataset and Checkpoints

We provided scripts to fully automate dataset and checkpoint downloading from google drive. Just run `sh install.sh` to do the stuff and wait. The total file size to download is around 1GB.

You can also manually download them following the steps:

First, the dataset can be downloaded from [google drive](https://drive.google.com/file/d/154g8SzGFWOcLLjes40aTXFoREX47-ZPk/view?usp=sharing). Download it and decompress it into `downloads/dataset`

Second, download checkpoints for Pose Estimator from [google drive](https://drive.google.com/drive/folders/1WshZaRVllWxHfUFK1--e1hQd_64dAKMz?usp=sharing), and move all four checkpoints into `downloads/pose_estimator/<name>.pth`.

Third, download checkpoints for Global Scheduling Policy from [google drive](https://drive.google.com/drive/folders/1YMI38jeLkPJQa_HJ-RnbQV36zPk4_OvH?usp=sharing), and save all checkpoints into `downloads/global_sheduling_policy/<name>_0.pt`. Notice the `_0` is necessary since our code examines this number. Use `controller.load=CheckpointPath` option to load checkpoints.

## Run Experiments

To run experiments, use command line options: `python train.py dataset=?? task=?? pose_estimator=?? manipulation=?? controller=?? train=??`

Possible options include:

- dataset:
  - cabinet_train
  - cabinet_test
  - drawer_train
  - drawer_test
  - mug_train
  - mug_test
  - pot_train
  - pot_test
  - real_world
- task:
  - open_cabinet
  - open_cabinet_45
  - open_cabinet_no_dr
  - open_drawer
  - open_drawer_30
  - open_drawer_no_dr
  - open_pot
  - pick_mig
  - real_world
- pose_extimator:
  - adapose_cabinet
  - adapose_drawer
  - adapose_mug
  - adapose_pot
  - ground_truth
- manipulation:
  - open_cabinet
  - open_cabinet_open_loop
  - open_drawer
  - open_drawer_open_looop
  - open_pot
  - open_pot_open_loop
  - pick_mug
  - pick_mug_open_loop
- controller
  - gt_pose
  - heuristic_pose
  - rl
- train
  - controller
  - test
  - collect: Used to collect offline data for some baselines.
  - test_baselines: Test with offline actions generated by some baselines.

Some other possible arguments are:

- task.num_envs=X: Number of parallel environments.
- headless=True/False: Whether to show graphics.
- viewerless=True/False: Whether to disable on-hand camera to accelerate some non-vision tasks.
- exp_name=XXX: The name of the experiment (used for saving models and logging).
- controller.load=Checkpoint: Load the saved controller checkpoint.

All other configurations can be modified using the [hydra](https://hydra.cc/docs/intro/) style command line options.

## Miscs

The scripts for drawing the plots included in our paper is also provided in `miscs/plot`.

## Citations

To cite us at

```latex
@article{an2023rgbmanip,
  title={RGBManip: Monocular Image-based Robotic Manipulation through Active Object Pose Estimation},
  author={An, Boshi and Geng, Yiran and Chen, Kai and Li, Xiaoqi and Dou, Qi and Dong, Hao},
  journal={arXiv preprint arXiv:2310.03478},
  year={2023}
}
```

RGBManip

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README