Final Project for EECE5554 performing analysis on various packages using Visual SLAM.
By Blake McHale, Ziyi Yang, and Nanday Kerkar
Refer to the powerpoint Final Project - VSLAM for results from package analysis. Please read notes at the bottom of each slide to understand project better.
Packages used for this project were OpenVSLAM, RTAB-Map, and SOFT-Odometry. Descriptions regarding each package and how to run them are below in their respective sections.
- EECE5554_FinalProject
The OpenVSLAM files are located in the openvslam directory of this repository. The installation procedure along with other details are mentioned locally in that directory. You can find a README.md specific to openvslam at openvslam/README.md.
Download the kitti dataset kitti_2011_09_26_drive_0022_synced.bag and place it inside src/vslam_user/resources.
Also, download the modified NEU dataset with corrected camera_info topics called final_100.bag and place it inside src/vslam_user/resources.
cd src/vslam_user/resources
# Bag file for kitti dataset
wget https://drive.google.com/open?id=1TJcH-Aw9yD5G5J0doLqCpa9ca22gw0cI
# Bag file for final_100.bag (NEU dataset)
wget https://drive.google.com/open?id=1FOw9sHZXFzR8m16sh4c17T6liJyaFkjg
These will be the rosbags referenced when using launch files for RTAB-Map.
rtabmap_ros: Main package used for performing visual SLAM.
Note: These are already installed in this repository
image_undistort: Stereo package to perform equidistant distortion model rectification. ROS Image Processing package only supports plumb bob and radial tangent distortion models. Image undistort depends on ROS, OpenCV and Eigen.
static_transform_mux: Static tf republisher to correct issues replaying tf frames from /tf_static in rosbag.
Make sure OpenCV and Eigen are built on your computer. If they are not they are built inside the openvslam instructions.
# First line needed for image_undistort
sudo apt install libnlopt-dev
sudo apt-get install ros-melodic-rtabmap-ros
Build the workspace with catkin at the root of the repository.
cd /path/to/eece5554_finalproject
catkin_make
RTAB-Map needs to first build a map by running without localization for one complete cycle of the data. This map will then be used when running with localization in the second cycle. Follow the steps below in order, first building the map and playing the bag once, then localizing pose and playing the bag again. If you want to record pose estimations pass the argument record to start_bag.launch and set it to true.
Make sure to call source devel/setup.bash inside every new terminal created.
It is recommended to run rtabmap before playing the bag data, so that the entire dataset is captured.
NOTE: Follow these steps first and then jump to Perform Localization section.
roslaunch vslam_user analyze.launch neu:=false localization:=false
or for NEU data
roslaunch vslam_user analyze.launch neu:=true localization:=false
roslaunch vslam_user start_bag.launch neu:=false
or for NEU data
roslaunch vslam_user start_bag.launch neu:=true
roslaunch vslam_user analyze.launch neu:=false localization:=true
or for NEU data
roslaunch vslam_user analyze.launch neu:=true localization:=true
roslaunch vslam_user start_bag.launch neu:=false
or for NEU data
roslaunch vslam_user start_bag.launch neu:=true
Run matlab file analyze_rtab.m in analysis folder to visualize poses from collected data.
Created package vslam_user that contains launch files for starting rosbag and running RTAB-Map processes. Also, has launch files for creating disparity images and point clouds from NEU dataset.
Modified files in image_undistort to manipulate images after they are rectified. Specifically, added lines 287-319 in stereo_undistort.cpp.
Two main launch files exist start_bag.launch and analyze.launch.
start_bag.launch arguments
neu - boolean flag that plays neu dataset when set to true and the KITTI dataset when set to false
record - boolean flag that records a rosbag of all pose topics when set to true
analyze.launch arguments
neu - boolean flag that reads from neu dataset when set to true and the KITTI dataset when set to false
localization - boolean flag that tells rtabmap to perform only pose localization using map and visual odometry when set to true, when false it builds a map and does not localize pose
use_viso - boolean flag that uses viso2 to perform visual odometry instead of default GFTT algorithm in rtab when set to true
rtabmap.ini: contains set of parameters to modify how rtabmap processes images and performs SLAM. Tested various configurations with different feature detection methods such as optical flow, SURF, and harris corner detection. Also, tested using a mask on the image and changing the distribution of feature detections. Could not get rtabmap to predict pose successfully on NEU dataset. KITTI dataset works without changing these parameters. Modified and tested various combinations of parameters under the groups Vis, GFTT, and SURF.
replace_camera_info.py: python file to modify and correct original camera_info topics in NEU dataset. Uses updated camera intrinsics and extrinsics given to us.
Parallel Processing Toolbox
Computer Vision Toolbox
First download kittiSLAM and SOFT folders separately, and then download KITTI stereo images dataset at here for the first dataset and here for the second one, which is the dataset we used for openvslam and RTAB map.
Unzip the zipfile under SOFT/data folders.
unzip 2011_09_26_drive_0009_sync.zip -d /your_path/SOFT/data
unzip 2011_09_26_drive_0022_sync.zip -d /your_path/SOFT/data
For running kittiSLAM, run kittiSLAM.m in matlab from analysis/SOFT/kittiSLAM folder. For running SOFT, run softSLAM.m under /SOFT/code/softSLAM.m(the original setting is to run with 2011_09_26_drive_0009 dataset) from inside that folder. To change the dataset, change the file path at line 17,18 kittiSLAM.m and line 39,40 of softSLAM.m
(note: increase vo_params.feature.nms_tau at line 17 of softSLAM.m if computation power is a limitation).
Packages were compared with their generated poses. A plot can be generated comparing the SOFT Odometry results to
RTAB-Map's results by running analyze_soft_rtab.m in Matlab from the analysis folder. Pose's were not
compared for OpenVSLAM because there was an issue retrieving poses outside the Pangolin viewer.