ETH Zurich - D-INFK - IVC - CVG - Research - VSO: Visual Semantic Odometry

VSO: Visual Semantic Odometry

Konstantinos-Nektarios Lianos       Johannes L. Schönberger      
Marc Pollefeys       Torsten Sattler
Computer Vision and Geometry Group
Department of Computer Science
ETH Zurich, Switzerland,,,


Robust data association is a core problem of visual odometry, where image-to-image correspondences provide constraints for camera pose and map estimation. Current state-of-the-art direct and indirect methods use short-term tracking to obtain continuous frame-to-frame constraints, while long-term constraints are established using loop closures. In this paper, we propose a novel visual semantic odometry (VSO) framework to enable medium-term continuous tracking of points using semantics. Our proposed framework can be easily integrated into existing direct and indirect visual odometry pipelines. Experiments on challenging real-world datasets demonstrate a significant improvement over state-of-the-art baselines in the context of autonomous driving simply by integrating our semantic constraints.

Paper / Supplementary / Bibtex

© CVG, ETH Zürich