BKI_ROS

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #llm #golang #go #linux #java #windows #cli #kubernetes #security #mcp

(★ 21)

Full ROS implementation of BKI with dynamic BKI and convolutional BKI

BKI_ROS Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Welcome!
Thank you for your interest in Convolutional Bayesian Kernel Inference (ConvBKI).
ConvBKI is an optimized semantic mapping algorithm which combines the best of 
probabilistic mapping and learning-based mapping. ConvBKI was previously presented
at ICRA 2023, which you can read more about [here](https://arxiv.org/abs/2209.10663) or 
explore the code from [here](https://github.com/UMich-CURLY/NeuralBKI).

In this repository and subsequent paper, we further accelerate ConvBKI and test
on more challenging test cases with perceptually difficult scenarios including
real-world military vehicles. An example from the real-world testing is playing below.

![Alt Text](https://raw.githubusercontent.com/UMich-CURLY/BKI_ROS/main/video.gif)

ConvBKI runs as an end-to-end network which you can test using this repository! To test ConvBKI,
clone the repository and download pre-processed ROS bags from the following links:

[KITTI](https://curly-dataset-public.s3.us-east-2.amazonaws.com/Joey/ConvBKI/Kitti/kitti.bag)

[RELLIS-3D](https://curly-dataset-public.s3.us-east-2.amazonaws.com/Joey/ConvBKI/Rellis/rellis.bag)

## Installation
ConvBKI has basic requirements since the network runs as a simple convolution layer, requiring primarily PyTorch and ROS (we use ROS noetic, however see this [pull request](https://github.com/UMich-CURLY/BKI_ROS/pull/3/commits/d3e01fecf778f14fa7752b7a3cd195d6e03e4372) for ROS 2). Most of the requirements come from our segmentation network SPVCNN, which we provide updated installation for below.

1. Install ROS (we use noetic)

2. ```conda create -n ConvBKIRos python=3.8.10```

3. ```conda activate ConvBKIRos```

4. ```conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.3 -c pytorch```

5. ```python -m pip install numpy==1.17.4```

6. ```python -m pip install spconv-cu113==2.1.21```

7. ```python -m pip install --upgrade git+https://github.com/mit-han-lab/torchsparse.git@v1.4.0```

8. ```python -m pip install torch-scatter -f https://data.pyg.org/whl/torch-2.1.2+${CUDA}.html```

9. ```python -m pip install pyyaml```

10. ```python -m pip install rospkg```

11. ```conda activate ConvBKIROS```

The ROS node publishes variance and semantic labels, which can be visualized as a MarkerArray (slow), or point cloud (fast) by editing the config file.

For more information, please see the below sections on how we preprocessed poses,
and more information on parameters. Unfortunately, we are unable to publish 
the perceptually challenging data due to proprietary restrictions. However, all code
used in the process is made public along with samples on open source data sets
which we create in the notebook CreateBag.ipynb.

## Mapping with rosbag

#### Run mapping

You can run the mapping module which will create a ros publisher that publish the map and can be visualized on rviz.

1. Run ros_node.py:
```
cd ~/BKI_ROS/EndToEnd
python ros_node.py
```
2. Play processed rosbag:
```
rosbag play your-bag.bag
```

#### YAML Parameters

Parameters can be set in the yaml config file, and it can be found in EndtoEnd/Configs

* pc_topic - the name of the pointcloud topic to subscribe to
* pose_topic - the name of the pose topic to subscribe to
* num_classes - number of semantic classes

* grid_size, min_bound, max_bound, voxel_sizes - parameters for convbki layer
* f - convbki layer kernel size
* res, cr - parameters for SPVNAS segmentation net
* seg_path - saved weights for SPVNAS segmentation net
* model_path - saved weights for convbki layer

#### Model Weights

Weights for SPVNAS segmentation network and convbki layer are located in EndtoEnd/weights, currently the weights are trained on [Rellis3D dataset](https://github.com/unmannedlab/RELLIS-3D) for off-road driving and Semantic KITTI [1] for on-road driving. If you have other pretrained weights, you should store them here and change the seg_path and model_path in the config file accordingly.

## Preprocess Poses
We are unable to release ROS bags for the military off-road driving to proprietary reasons. If you want to create ROS bags for your own data, below is the process we used to test on our custom data.

We use LIO-SAM to preprocess poses - https://github.com/TixiaoShan/LIO-SAM

## Install
See LIO-SAM documentation for software and hardware dependency information.
Use the following commands to download and compile the package.

```
git clone https://github.com/UMich-CURLY/BKI_ROS.git
mv ~/BKI_ROS/lio-sam/liorf ~/catkin_ws/src
cd ~/catkin_ws
catkin_make
```

We also provide an environment.yml which you can use to create a conda environment (This is only tested on Ubuntu 20)
```
cd ~/BKI_ROS/EndToEnd
conda env create -f environment.yml
conda activate bkiros
```

## Run the package

1. Run the launch file:
```
cd ~/catkin_ws/src/liorf/launch
roslaunch liorf run_lio_sam_ouster.launch
```

2. Play existing bag files:
```
rosbag play your-bag.bag
```

3. Call the save map service to create new rosbag:
```
rosservice call /liorf/save_map "{}"
```

**Before creating rosbag** change line 392 in ~/catkin_ws/src/liorf/src/mapOptimization.cpp to bag.open("/your/directory/lidarPoses.bag", rosbag::bagmode::Write);
 
For a more detailed setup guide to LIO-SAM, please see https://github.com/TixiaoShan/LIO-SAM and https://github.com/YJZLuckyBoy/liorf

## Acknowledgement
We utilize data and code from: 
- [1] [SemanticKITTI](http://www.semantic-kitti.org/)
- [2] [RELLIS-3D](https://arxiv.org/abs/2011.12954)
- [3] [SPVNAS](https://github.com/mit-han-lab/spvnas)
- [4] [LIO-SAM](https://github.com/YJZLuckyBoy/liorf)
- [5] [Semantic MapNet](https://github.com/vincentcartillier/Semantic-MapNet)

## Reference
If you find our work useful in your research work, consider citing [our paper](https://arxiv.org/abs/2209.10663)
```
@ARTICLE{wilson2022convolutional,
  title={Convolutional Bayesian Kernel Inference for 3D Semantic Mapping},
  author={Wilson, Joey and Fu, Yuewei and Zhang, Arthur and Song, Jingyu and Capodieci, Andrew and Jayakumar, Paramsothy and Barton, Kira and Ghaffari, Maani},
  journal={arXiv preprint arXiv:2209.10663},
  year={2022}
}
```
Bayesian Spatial Kernel Smoothing for Scalable Dense Semantic Mapping ([PDF](https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8954837))
```
@ARTICLE{gan2019bayesian,
author={L. {Gan} and R. {Zhang} and J. W. {Grizzle} and R. M. {Eustice} and M. {Ghaffari}},
journal={IEEE Robotics and Automation Letters},
title={Bayesian Spatial Kernel Smoothing for Scalable Dense Semantic Mapping},
year={2020},
volume={5},
number={2},
pages={790-797},
keywords={Mapping;semantic scene understanding;range sensing;RGB-D perception},
doi={10.1109/LRA.2020.2965390},
ISSN={2377-3774},
month={April},}

```

BKI_ROS

BKI_ROS Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

BKI_ROS

BKI_ROS Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README