Stereo-Visual-SLAM-Odometry

KO | EN

gitlite — search

#python #react #typescript #golang #machine-learning #rust #linux #docker #nodejs #windows #awesome #awesome-list #openai #open-source

Stereo-Visual-SLAM-Odometry

(★ 37)

Real-time Stereo Visual SLAM Pipeline with Bundle Adjustment

Stereo-Visual-SLAM-Odometry Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# **Real-time Stereo Visual Odometry with Local Non-Linear Least Square Optimisation**

## **About**

This project implements 6-DOF camera pose estimation from a calibrated stereo camera (with local non-linear least square optimisation) for various scenarios in KITTI dataset

![Result on 00](https://raw.githubusercontent.com/sakshamjindal/Stereo-Visual-SLAM-Odometry/master/assets/KITTI_VO_00_cut.gif)

## **Installation**

```bash
$ conda env create -f setup/environment.yml
$ pip install -e .
```

## **Usage**

For simulation of visual odometry, run the followig command

```bash
$ python main.py --config_path configs/params.yaml
```

The `params.yaml` needs to be edited to configure the sequence to run the simulation.

## **About the Problem !**

**1. What is Visual Odometry ?** 
**2. Problem Formulation** 
**3. Algorith Implemented**

### **Visual Odometry**

Visual Odometry is the process of incrementally estimating the pose and trajectory of a robot or a vehicle (orientation and translation of a camera configuration rigidly attached to it) using video stream from the camera.

### **Problem Formulation**

An agent is moving through an environment and taking
images with a rigidly attached camera system at discrete
time instants. Let the stream of images coming from the pair of camera
(assumed stereo configuration) be denoted by IL, k and IR, k at time instant *k* . We assume that we have prior knowledge of all the intrinsic as well as extrinsic calibration parameters of the stereo rig.

We need to estimate the relative rotation *R* and translation *t* between stereo configuration at time instants *k-1* and *k* and then to concatenate the transformation to incrementally recover the full trajectory Ck of the camera

### **Algorithm Implemented**
3D-to-2D: Structure to feature correspondences (Source : [1]) 
- &nbsp;Compute the first stereo image frames IL,K and IR,K 
- &nbsp;Extract and match stereo features fL,K and fR,K 
- &nbsp;Triangulate features to build point cloud Xk 
- &nbsp;Set initial camera pose Ck 
- &nbsp;Store information from the first frame as IL,k-1, IR,k-1, fL,k-1, fR,k-1, Xk-1 
&nbsp;While exists a new image frame: 
 - Compute the new stereo image pair IL,K and R,K 
 - Extract and match stereo features fL,K and fR,K 
 - Triangulate features to build point cloud Xk 
 - Track 2D features fL,k-1 at IL,k-1 to fL,k at IL,K and thus obtain tfk-1,k 
 - Compute correspondence for the tracked features tfk-1,k and Xk-1 
 - Compute camera pose estimation (P3P), thus T = [R|t]
 - Concatenate transformation by Ck = Ck-1 Tk
 - If Optimisation is enabled, do non-linear least squares optimisation of T 
 - Store informaton from first frame as IL,k-1, IR,k-1, fL,k-1, fR,k-1, Xk-1 and Ck-1 
 
 
<img src="assets/SVO_Diagram.png" width="400" height="300" /> 
 Relative Camera Pose and Concatenation of Transformations (Source: E. F. Aguilar Calzadillas [1] )

## Future Work
- [ ] Implement Windowed Bundle Adjustment
- [ ] Implement Graph Based Optimisation in Visual SLAM
- [ ] Visualise 3D point cloud of scene using Point Cloud Library (PCL)

## **References**

[1] ****E. F. Aguilar Calzadillas****, **"Sparse Stereo Visual Odometry with Local Non-Linear Least-Squares Optimization for Navigation of Autonomous Vehicles"**, M. A. Sc. Thesis, Department of Mechanical and Aerospace Engineering, Carleton University, Ottawa ON, Canada, 2019

[2] **D. Scaramuzza, F. Fraundorfer**, *"Visual Odometry: Part I - The First 30 Years and Fundamentals"*, IEEE Robotics and Automation Magazine, Volume 18, issue 4, 2011

[3]  **F. Fraundorfer, D. Scaramuzza**, *"Visual odometry: Part II - Matching, robustness, optimization, and applications"*, IEEE Robotics and Automation Magazine, Volume 19, issue 2, 2012

[4] **Avi Singh**, *Visual Odmetry from scratch - A tutorial for beginners*, Avi Singh's Blog

## Contact
Saksham Jindal (saksham.jindal@outlook.com)

Stereo-Visual-SLAM-Odometry

Stereo-Visual-SLAM-Odometry Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Stereo-Visual-SLAM-Odometry

Stereo-Visual-SLAM-Odometry Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README