OpenTrafficCam3D

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

OpenTrafficCam3D

★ 17 Open GitHub ↗

Source code for paper Toward Planet-Wide Traffic Camera Calibration (WACV 2024)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# OpenTrafficCam3D

**Toward Planet-Wide Traffic Camera Calibration**  
[Khiem Vuong](https://www.khiemvuong.com/), [Robert Tamburo](https://www.ri.cmu.edu/ri-people/robert-joseph-tamburo/), [Srinivasa G. Narasimhan](https://www.cs.cmu.edu/~srinivas/)  
IEEE Winter Conference on Applications of Computer Vision (WACV), 2024

[[`arXiv`](https://arxiv.org/abs/2311.04243)]
[[`Project Page`](https://www.khiemvuong.com/OpenTrafficCam3D/)]
[[`Bibtex`](#reference)]

![teaser](https://raw.githubusercontent.com/kvuong2711/OpenTrafficCam3D/main/media/teaser.png)

## Introduction   
You have access to a traffic camera stream but want to know its intrinsics and/or extrinsics? OpenTrafficCam3D is a pipeline that can help you calibrate traffic cameras using only the traffic camera image and a few GPS-tagged images (e.g., Google Street View, for metric scale).

The pipeline consists of three main steps: scene reconstruction, traffic camera localization, and geo-registration/ground plane fitting.

## Prepare Environment

### Clone the repository
```
git clone https://github.com/kvuong2711/OpenTrafficCam3D.git
cd OpenTrafficCam3D
```

### Build Docker

Either build the docker: `sudo docker build . -t kvuong2711/otc3d:latest`

or pull the pre-built docker: `sudo docker pull kvuong2711/otc3d:latest`

### Run the docker

```
sudo docker run --gpus 'all' -v /:/mars --shm-size=64GB -it kvuong2711/otc3d:latest
```
where `/` is the directory in the local machine (in this case, the root folder), and `/mars` is the reflection of that directory in the docker.

Inside the docker, change the working directory to this repository: 
```
cd /mars/PATH/TO/THIS/REPO/OpenTrafficCam3D
```

### Additional Dependencies
Assuming you are at `OpenTrafficCam3D` directory, we want to install `detectron2` and `Mask2Former`:

```
# Install detectron2
cd trafficreloc/third_party/detectron2/
pip install .
pip install git+https://github.com/cocodataset/panopticapi.git
pip install git+https://github.com/mcordts/cityscapesScripts.git

# Install Mask2Former
cd ../Mask2Former/
pip install -r requirements.txt
cd mask2former/modeling/pixel_decoder/ops/
sh make.sh
cd ../../../../../../..
```
After these steps, you should be back at the `OpenTrafficCam3D` directory.

## Data Preparation and Hierarchy

Here, we show how to prepare a small sample set of data for Jackson Hole, WY. To modify the location/extent/number of panoramas and where the data is saved, please modify the locations and default parameters in [trafficreloc/data/gsv/prepare.py](trafficreloc/data/gsv/prepare.py). By default, the data will be saved in `./sample_data`.

1. You should obtain the API_KEY from [Google Maps API](https://developers.google.com/maps/documentation/javascript/get-api-key).

2. Run the following command to download the data:
      ```
      cd trafficreloc/data/gsv
      python prepare.py --location jhts --api_key YOUR_API_KEY
      cd ../../../
      ```

3. The data will be saved in `./sample_data`. The hierarchy of the data for location `jhts` is as follows:
      ```
      sample_data
      |
      ├── raw_data
      │   ├── jhts
      │   │   ├── pano_data  # panorama images downloaded from GSV
      │   │   ├── perspective_data  # extracted perspective images from panos
      |
      ├── processed_data
      │   ├── jhts
      │   │   ├── database
      │   │   |  ├── images  # images for scene reconstruction
      |   |   |
      │   │   ├── query
      │   │   |  ├── images  # images for localization (e.g., traffic cam)
      ```
      In short, you should put images you want to use for scene reconstruction in `./sample_data/processed_data/jhts/database/images` and any images you want to use for localization in `./sample_data/processed_data/jhts/query/images`.

If you just want to quickly try the code out without setting up the data downloading pipeline, you can download the sample data from [here](https://huggingface.co/datasets/kvuong2711/OpenTrafficCam3D/resolve/main/sample_data.zip?download=true) and extract it to the root directory of the repository (i.e., `OpenTrafficCam3D/sample_data`).

**Disclaimer**: *The data provided in the sample data is for demonstration purposes only and not to be used for commercial purposes. We do not own the data and we do not distribute it. Please refer to the [Google Maps Platform Terms of Service](https://cloud.google.com/maps-platform/terms) for more information.*

## Scene Reconstruction
You can modify the data paths and parameters in the configuration file [gsv_loc_jhts_release.yml](trafficreloc/configs/gsv_loc_jhts_release.yml) to suit your needs.

Then, run the following command to reconstruct the scene:
```
python trafficreloc/pipeline_reconstruction.py --config_file trafficreloc/configs/gsv_loc_jhts_release.yml
```

## Traffic Camera Localization
Assuming your data is prepared and the scene has been reconstructed, you can now localize traffic cameras in the scene. You can modify the data paths and parameters in the configuration file [gsv_loc_jhts_release.yml](trafficreloc/configs/gsv_loc_jhts_release.yml) to suit your needs.

Then, run the following command to localize traffic cameras:
```
python trafficreloc/pipeline_localization.py --config_file trafficreloc/configs/gsv_loc_jhts_release.yml
```

## Geo-Registration and Ground Plane Fitting
As a post-processing step, you can geo-register the scene (so that the scene is in metric scale) based on the GPS of the database images and (optionally) fit a ground plane to the scene.

Run the following command to geo-register the scene and fit a ground plane:
```
python trafficreloc/pipeline_postprocess.py --config_file trafficreloc/configs/gsv_loc_jhts_release.yml --api_key YOUR_API_KEY
```

You can find all the results in `./sample_data/processed_data/jhts/database/recon_output/sfm_superpoint+superglue`, containing the `latest_metric_model` and `groundplane_equation.txt` (in COLMAP global coordinate system).

**Note**:
- This step requires the API_KEY to query the Google Maps API for the GPS coordinates of the database images. Optionally, you can provide your own GPS coordinates by modifying the `extract_wgs84_txtfile` function in [trafficreloc/georef_colmap.py](trafficreloc/georef_colmap.py).

- This code also visualizes the scene using Open3D which usually requires a GUI.

## Reference

If you use find this code helpful, please cite:

```BibTeX
@inproceedings{vuong2024trafficcalib,
      title = {Toward Planet-Wide Traffic Camera Calibration},
      author = {Vuong, Khiem and Tamburo, Robert and Narasimhan, Srinivasa G.},
      booktitle = {IEEE Winter Conference on Applications of Computer Vision},
      year = {2024},
}
```

## Acknowledgements
We would like to thank the authors of [hloc](https://github.com/cvg/Hierarchical-Localization/tree/master) with their codebase as a starting point for our project.

## Contact
If you have any questions, please feel free to open an issue or contact [Khiem Vuong](mailto:kvuong@andrew.cmu.edu).

OpenTrafficCam3D

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README