Visual Navigation for Autonomous Vehicles: An Open-source Hands-on Robotics Course at MIT

Luca Carlone, Kasra Khosoussi, Vasileios Tzoumas, Golnaz Habibi, Markus Ryll, Rajat Talak, Jingnan Shi, Pasquale Antonante

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

This paper reports on the development, execution, and open-sourcing of a new robotics course at MIT. The course is a modern take on 'Visual Navigation for Autonomous Vehicles' (VNAV) and targets first-year graduate students and senior undergraduates with prior exposure to robotics. VNAV has the goal of preparing the students to perform research in robotics and vision-based navigation, with emphasis on drones and self-driving cars. The course spans the entire autonomous navigation pipeline; as such, it covers a broad set of topics, including geometric control and trajectory optimization, 2D and 3D computer vision, visual and visual-inertial odometry, place recognition, simultaneous localization and mapping, and geometric deep learning for perception. VNAV has three key features. First, it bridges traditional computer vision and robotics courses by exposing the challenges that are specific to embodied intelligence, e.g., limited computation and need for just-in-time and robust perception to close the loop over control and decision making. Second, it strikes a balance between depth and breadth by combining rigorous technical notes (including topics that are less explored in typical robotics courses, e.g., on-manifold optimization) with slides and videos showcasing the latest research results. Third, it provides a compelling approach to hands-on robotics education by leveraging a physical drone platform (mostly suitable for small residential courses) and a photo-realistic Unity-based simulator (open-source and scalable to large online courses). VNAV has been offered at MIT in the Falls of 2018-2021 and is now publicly available on MIT OpenCourseWare (OCW) and at vnav.mit.edu/.

Original languageEnglish
Title of host publication2022 IEEE Integrated STEM Education Conference, ISEC 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages177-184
Number of pages8
ISBN (Electronic)9781665484299
DOIs
StatePublished - 2022
Event12th IEEE Integrated STEM Education Conference, ISEC 2022 - Virtual, Online, United States
Duration: 26 Mar 2022 → …

Publication series

Name2022 IEEE Integrated STEM Education Conference, ISEC 2022

Conference

Conference12th IEEE Integrated STEM Education Conference, ISEC 2022
Country/TerritoryUnited States
CityVirtual, Online
Period26/03/22 → …

Keywords

  • Open-source Course Material
  • Project-based Learning
  • Robotics and Computer Vision Education

Fingerprint

Dive into the research topics of 'Visual Navigation for Autonomous Vehicles: An Open-source Hands-on Robotics Course at MIT'. Together they form a unique fingerprint.

Cite this