NetCalib-Lidar-Camera-Auto-calibration

KO | EN

gitlite — search

#react #ai #go #golang #android #machine-learning #deep-learning #linux #docker #windows #cli #vue #awesome-list #pytorch

NetCalib-Lidar-Camera-Auto-calibration

(★ 16)

NetCalib: A Novel Approach for Lidar-Camera Auto-calibration Based on Deep Learning

NetCalib-Lidar-Camera-Auto-calibration 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# A new updated version is published in a separate [repo](https://github.com/simonwu53/NetCalib2-Sensors-Auto-Calibration)

# NetCalib: A Novel Approach for LiDAR-Camera Auto-calibration Based on Deep Learning

This work is accepted for publication in ICPR 2020.

In this paper, we demonstrated a novel approach to calibrate the LiDAR and Stereo cameras using a deep neural network.
Firstly, LiDAR point cloud is projected to the depth map in the left camera's view, and the depth map for the corresponding stereo pair is calculated as well.
Next, two modality-specific feature extraction modules are used for two depth maps respectively to preprocess and extract features from each modality.
Then, the output feature maps are concatenated and they are fed to the global regression module that will learn the geometry correspondencies among modalities. 
A SPP layer is used after the output from previous module to provide a fixed length feature vector.
Finally, two sets of output layers are used for rotation errors and translation errors separately.

## Motivation
We are releasing our approach's source code for autocalibrating camera and lidar sensors to share with the scientific community and industry with the aim of the collaboration with people around the world to push the boundaries in the field of autonomous driving and intelligent transportation systems.

## Experiment results
### 1. Losses
<img src="assets/train_loss.png" width="300" alt="train-loss"> <img src="assets/valid_loss.png" width="300" alt="valid-loss"> <img src="assets/test_loss2.png" width="300" alt="valid-loss">

### 2. Model inference error
<img src="assets/error-rotation.png" width="400" alt="rotation-error"> <img src="assets/error-translation.png" width="400" alt="translation-error">

## Deployment
The codes were tested on Ubuntu 19.10 & 20.04 with Python 3.7 & 3.8. Generally, a Ubuntu 18.04+ with Python3+ will be compatible as long as you can install those required libraries.
For a list of all source code dependencies, please see [here](https://github.com/simonwu53/NetCalib-Lidar-Camera-Auto-calibration/blob/master/requirements.txt).

1. Clone the repo to your local drive. 
2. Setup your own Python environment and install the `requirements.txt` by command `pip3 install -r requirements.txt`. 
3. The installation of `pytorch` and `torchvision` may vary in the future, you can install manually via the [official website](https://pytorch.org/get-started/locally/).

## Train the model
Modify the configurations in `src/config.py`. 
You can start training the model from the beginning using `--train` argument. 
`--err` specifies the error range to use, which is defined in the configuration.

```shell script
python main.py --train --err 3
```

In order to continue training, you can add `--ckpt` argument to specify the pytorch checkpoint `tar` file.

## Test the model
You can test the trained model by the command below, where `--win-scale` changes the opencv window size.

```shell script
python main.py --test --err 4 --ckpt /home/user/NetCalib-Lidar-Camera-Auto-calibration/results/test1/ckpt/ckpt.tar --win-scale 2.5
```
## Licence 
NetCalib is released under a [MIT License](https://github.com/simonwu53/NetCalib-Lidar-Camera-Auto-calibration/blob/master/LICENSE) license.

For a closed-source version of NetCalib for commercial purposes, please contact the authors: [Wu](mailto:Shan.Wu@ut.ee) and [Hadachi](mailto:hadachi@ut.ee)

## Contributors
Shan Wu; Amnir Hadachi; Damien Vivet; Yadu Prabhakar.

## Citation 
If you use NetCalib in an academic work, please cite:
```
@inproceedings{Shan2020NetCalib,
  title={NetCalib: A Novel Approach for LiDAR-Camera Auto-calibration Based on Deep Learning},
  author={Shan, Wu; Hadachi, Amnir; Vivet, Damien; Prabhakar, Yadu},
  booktitle={Proceedings of the 25th International Conference on Pattern recognition 2020},
  year={2020},
  organization={IEEE}
}
```

Preprint version of the paper is [here](https://www.researchgate.net/publication/344694742_NetCalib_A_Novel_Approach_for_LiDAR-Camera_Auto-calibration_Based_on_Deep_Learning).

NetCalib-Lidar-Camera-Auto-calibration

NetCalib-Lidar-Camera-Auto-calibration 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

NetCalib-Lidar-Camera-Auto-calibration

NetCalib-Lidar-Camera-Auto-calibration 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README