xfeatSLAM - GitLite

KO | EN

gitlite — search

#python #react #ai #typescript #machine-learning #rust #deep-learning #linux #java #windows #awesome #pytorch #ios #open-source

xfeatSLAM

(★ 186)

Real-time SLAM with deep features (XFeat + ORB-SLAM3)

파일 탐색기

최종 버전 다운로드 (.zip)

- xfeatSLAM_compressed.gif
- - EuRoC.yaml
  - KITTI00-02.yaml
  - KITTI03.yaml
  - KITTI04-12.yaml
  - mono_euroc.cc
  - mono_kitti.cc
  - mono_realsense_D435i.cc
  - mono_realsense_t265.cc
  - mono_tum.cc
  - mono_tum_vi.cc
  - RealSense_D435i.yaml
  - RealSense_T265.yaml
  - TUM-VI.yaml
  - TUM1.yaml
  - TUM2.yaml
  - TUM3.yaml
- - RealSense_D435i.yaml
  - rgbd_realsense_D435i.cc
  - rgbd_tum.cc
  - TUM1.yaml
  - TUM2.yaml
  - TUM3.yaml
- - GeometricCamera.h
  - KannalaBrandt8.h
  - Pinhole.h
- Atlas.h
- Config.h
- Converter.h
- Frame.h
- FrameDrawer.h
- G2oTypes.h
- GeometricTools.h
- ImuTypes.h
- KeyFrame.h
- KeyFrameDatabase.h
- LocalMapping.h
- LoopClosing.h
- Map.h
- MapDrawer.h
- MapPoint.h
- MLPnPsolver.h
- OptimizableTypes.h
- Optimizer.h
- ORBextractor.h
- ORBmatcher.h
- ORBVocabulary.h
- SerializationUtils.h
- Settings.h
- Sim3Solver.h
- System.h
- Tracking.h
- TwoViewReconstruction.h
- Viewer.h
- XFeat.h
- XFextractor.h
- tum_rgbd_association.py
- - KannalaBrandt8.cpp
  - Pinhole.cpp
- Atlas.cc
- Config.cc
- Converter.cc
- Frame.cc
- FrameDrawer.cc
- G2oTypes.cc
- GeometricTools.cc
- ImuTypes.cc
- KeyFrame.cc
- KeyFrameDatabase.cc
- LocalMapping.cc
- LoopClosing.cc
- Map.cc
- MapDrawer.cc
- MapPoint.cc
- MLPnPsolver.cpp
- OptimizableTypes.cpp
- Optimizer.cc
- ORBextractor.cc
- ORBmatcher.cc
- Settings.cc
- Sim3Solver.cc
- System.cc
- Tracking.cc
- TwoViewReconstruction.cc
- Viewer.cc
- XFeat.cc
- XFextractor.cc
- - CMakeLists.txt
  - README.txt
- - CMakeLists.txt
  - config.h.in
  - license-bsd.txt
  - README.txt
- - .clang-format
  - .gitignore
  - .travis.yml
  - appveyor.yml
  - CMakeLists.txt
  - doxyrest-config.lua
  - LICENSE.txt
  - make_docs.sh
  - package.xml
  - README.rst
  - run_format.sh
  - SophusConfig.cmake.in
- ORBvoc.txt.tar.gz
- xfeat.pt
.gitignore
CMakeLists.txt
dependencies.txt
LICENSE.txt
project_setup.sh
README.md

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# xfeatSLAM

## Introduction

xfeatSLAM is a Visual SLAM system that integrates the lightweight and efficient [XFeat](https://github.com/verlab/accelerated_features) architecture into the [ORB-SLAM3](https://github.com/UZ-SLAMLab/ORB_SLAM3) pipeline. This project is an experimental combination of ORB-SLAM3 with the XFeat model to create a SLAM system utilizing deep learning-based image descriptors.

Typically, models used to obtain deep learning-based local features provide accurate descriptions but are highly resource-intensive. The lightweight nature of XFeat makes it particularly well-suited for environments with limited processing power, such as mobile robots and embedded systems, where real-time performance is critical.

First, the entire XFeat model is implemented in C++ using PyTorch C++ API. I have separated the code for this implementation into [this repo](https://github.com/udaysankar01/xfeat_cpp). This implementation is then integrated into the ORB-SLAM3 pipeline, following an approach similar to that used in [GCNv2 SLAM](https://github.com/jiexiong2016/GCNv2_SLAM).

![xfeatSLAM in action](https://raw.githubusercontent.com/udaysankar01/xfeatSLAM/main/doc/xfeatSLAM_compressed.gif)

## Dependencies

### C++17 Compiler

xfeatSLAM uses libtorch and requires a compiler that supports at least C++17. It has been tested with GCC and G++ compilers v11.4.0.

### PyTorch

xfeatSLAM uses [PyTorch](https://github.com/pytorch/pytorch) C++ API (libtorch) for running the C++ Implementation of XFeat model.

Please avoid using the pre-built version of libtorch as it may cause linking issues ([CXX11 ABI issue](https://github.com/pytorch/pytorch/issues/13541))

### Pangolin

[Pangolin](https://github.com/stevenlovegrove/Pangolin) is used for visualization and UI.

### OpenCV

[OpenCV](http://opencv.org/) is used for image and feature processing.

The version of OpenCV used for testing is v4.5.4.

### Eigen3

[Eigen3](http://eigen.tuxfamily.org/) is requried for matrix operations.

Project was tested using Eigen3 v3.4.0.

### DBoW2, Sophus and g2o (included in _thirdparty_ folder)

The project use modified versions of the [DBoW2](https://github.com/dorian3d/DBoW2) library to perform place recognition, [Sophus](https://github.com/strasdat/Sophus) for Lie groups, and [g2o](https://github.com/RainerKuemmerle/g2o) library to perform non-linear optimizations. All three modified libraries (which are BSD) are included in the _thirdparty_ folder.

## Getting Started

To download xfeatSLAM:

```bash
git clone https://github.com/udaysankar01/xfeatSLAM
cd xfeatSLAM
```

To install all the necessary packages:

```bash
chmod +x project_setup.sh
./project_setup.sh
```

To build the project:

```bash
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make -j8
```

## RGB-D Example

To run deep RGB-D SLAM:

```bash
./examples/RGB-D/rgbd_tum Vocabulary/ORBvoc.txt \
                          examples/RGB-D/TUMX.yaml \
                          PATH_TO_SEQUENCE_FOLDER \
                          ASSOCIATIONS_FILE
```

To use ORB feature descriptor instead of XFeat, set the environment variable `USE_ORB` as 1.

```bash
USE_ORB=1 ./examples/RGB-D/rgbd_tum Vocabulary/ORBvoc.txt \
                                    examples/RGB-D/TUMX.yaml \
                                    PATH_TO_SEQUENCE_FOLDER \
                                    ASSOCIATIONS_FILE
```

## Monocular Example

To run deep Monocular SLAM:

```bash
./examples/Monocular/mono_tum Vocabulary/ORBvoc.txt \
                              examples/Monocular/TUMX.yaml \
                              PATH_TO_SEQUENCE_FOLDER
```

Support for additional sensors will be added soon.

## Bibtex Citation

```tex
@misc{potje2024xfeatacceleratedfeatureslightweight,
      title={XFeat: Accelerated Features for Lightweight Image Matching},
      author={Guilherme Potje and Felipe Cadar and Andre Araujo and Renato Martins and Erickson R. Nascimento},
      year={2024},
      eprint={2404.19174},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2404.19174},
}
```

```tex
@article{ORBSLAM3_TRO,
  title={{ORB-SLAM3}: An Accurate Open-Source Library for Visual, Visual-Inertial
           and Multi-Map {SLAM}},
  author={Campos, Carlos AND Elvira, Richard AND G\´omez, Juan J. AND Montiel,
          Jos\'e M. M. AND Tard\'os, Juan D.},
  journal={IEEE Transactions on Robotics},
  volume={37},
  number={6},
  pages={1874-1890},
  year={2021}
 }
```

```tex
@unknown{unknown,
author = {Tang, Jiexiong and Ericson, Ludvig and Folkesson, John and Jensfelt, Patric},
year = {2019},
month = {02},
pages = {},
title = {GCNv2: Efficient Correspondence Prediction for Real-Time SLAM}
}
```