Inspiring healthy food choices in a virtual reality supermarket by adding a tangible dimension in the form of an augmented virtuality smartphone

We want to understand the changing shopping behaviour, influenced by health-targeting nutrition apps on mobile devices. To achieve that, we have built a realistic Virtual Reality (VR) supermarket simulation and addressed core aspects such as handling virtual products, prevention of cybersickness, rounded off with an initial pilot study. On top of that, we built a virtual replica smartphone in VR with nutritionrelated functionality. This has been extended with an Augmented Virtuality (AV) feature, that enables us to track the screen of a participant's own smartphone, hence allowing us to integrate real-world apps and letting the user interact with them during the simulation. To achieve a high-quality tracking, we propose a hybrid approach, utilizing an add-on RGB camera on the VR headset fused with data provided through a WLAN connection in the case of self-made apps. This enables the users to manipulate the simulation from within the smartphone app, introducing a versatile, highly usability-centered controller because they can handle their phone naturally.