Fusion++: Volumetric object-level SLAM

John McCormac, Ronald Clark, Michael Bloesch, Andrew Davison, Stefan Leutenegger

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

195 Scopus citations


We propose an online object-level SLAM system which builds a persistent and accurate 3D graph map of arbitrary reconstructed objects. As an RGB-D camera browses a cluttered indoor scene, Mask-RCNN instance segmentations are used to initialise compact per-object Truncated Signed Distance Function (TSDF) reconstructions with object size-dependent resolutions and a novel 3D foreground mask. Reconstructed objects are stored in an optimisable 6DoF pose graph which is our only persistent map representation. Objects are incrementally refined via depth fusion, and are used for tracking, relocalisation and loop closure detection. Loop closures cause adjustments in the relative pose estimates of object instances, but no intra-object warping. Each object also carries semantic information which is refined over time and an existence probability to account for spurious instance predictions. We demonstrate our approach on a hand-held RGB-D sequence from a cluttered office scene with a large number and variety of object instances, highlighting how the system closes loops and makes good use of existing objects on repeated loops. We quantitatively evaluate the trajectory error of our system against a baseline approach on the RGB-D SLAM benchmark, and qualitatively compare reconstruction quality of discovered objects on the YCB video dataset. Performance evaluation shows our approach is highly memory efficient and runs online at 4-8Hz (excluding relocalisation) despite not being optimised at the software level.

Original languageEnglish
Title of host publicationProceedings - 2018 International Conference on 3D Vision, 3DV 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages10
ISBN (Electronic)9781538684252
StatePublished - 12 Oct 2018
Externally publishedYes
Event6th International Conference on 3D Vision, 3DV 2018 - Verona, Italy
Duration: 5 Sep 20188 Sep 2018

Publication series

NameProceedings - 2018 International Conference on 3D Vision, 3DV 2018


Conference6th International Conference on 3D Vision, 3DV 2018


  • 3D Reconstruction
  • Instance Segmentation
  • Object detection
  • Object-oriented Maps
  • Objects
  • SLAM
  • Scene Understanding


Dive into the research topics of 'Fusion++: Volumetric object-level SLAM'. Together they form a unique fingerprint.

Cite this