Real-time 3D reconstruction and 6-DoF tracking with an event camera

Hanme Kim, Stefan Leutenegger, Andrew J. Davison

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

258 Scopus citations

Abstract

We propose a method which can perform real-time 3D reconstruction from a single hand-held event camera with no additional sensing, and works in unstructured scenes of which it has no prior knowledge. It is based on three decoupled probabilistic filters, each estimating 6-DoF camera motion, scene logarithmic (log) intensity gradient and scene inverse depth relative to a keyframe, and we build a real-time graph of these to track and model over an extended local workspace. We also upgrade the gradient estimate for each keyframe into an intensity image, allowing us to recover a real-time video-like intensity sequence with spatial and temporal super-resolution from the low bit-rate input event stream. To the best of our knowledge, this is the first algorithm provably able to track a general 6D motion along with reconstruction of arbitrary structure including its intensity and the reconstruction of grayscale video that exclusively relies on event camera data.

Original languageEnglish
Title of host publicationComputer Vision - 14th European Conference, ECCV 2016, Proceedings
EditorsBastian Leibe, Jiri Matas, Nicu Sebe, Max Welling
PublisherSpringer Verlag
Pages349-364
Number of pages16
ISBN (Print)9783319464657
DOIs
StatePublished - 2016
Externally publishedYes
Event14th European Conference on Computer Vision, ECCV 2016 - Amsterdam, Netherlands
Duration: 8 Oct 201616 Oct 2016

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9910 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference14th European Conference on Computer Vision, ECCV 2016
Country/TerritoryNetherlands
CityAmsterdam
Period8/10/1616/10/16

Keywords

  • 3D reconstruction
  • 6-DoF tracking
  • Event-based camera
  • Intensity reconstruction
  • SLAM
  • Visual odometry

Fingerprint

Dive into the research topics of 'Real-time 3D reconstruction and 6-DoF tracking with an event camera'. Together they form a unique fingerprint.

Cite this