Human perception model for multi-modal feedback in telepresence systems

Advanced physical and virtual telepresence systems are expected to display information to the human operator in different modalities of human perception. The coaction of stimuli to each single modality is to mediate sensory impressions and perceptions to the human operator in a manner that represents his/her interaction with the remote or virtual environment. Model-based synthesis of appropriate stimuli to the operator is an important problem particularly for the haptic modalities. For this purpose future research in telepresence and virtual reality requires a mathematical model of human perception yielded by external stimuli from multiple feedback devices in a multi-modal telepresence system. In this paper we present a mathematical framework describing the principles of human perception in the terminology of systems theory. The viability and benefit from the presented theory is shown by formal descriptions of sensory substitution and experiments performed in a virtual environment.