Perceptually Oriented Ego-Motion Simulation

This results section represents all the data, which has been collected during the experiments of the POEMS project. In general, all experimental procedures and data are described in deliverables and scientific papers, however the data is presented as a separate result here to underline the possibility for others to make use of this. See result "simulation parameters influencing ego-motion and spatial presence" for further details.

This result represents the essence of POEMS and describes the optimal parameters of a lean and elegant ego motion simulator, whose goal is to allow creating a convincing sense of presence in a virtual world, without the drawbacks of the usual full motion simulators that require vast amounts of space, money, and expertise. To achieve these optimal parameters, we studied how each of the visual, vestibular, auditory, and somato-sensory cues help give us more information about our motions through environments, and identified, for each of them, the minimum amount of realism required to significantly enhance a virtual experience. We have found that all of them are useful, but to different extents. Visual cues, for example, are the most important, as they give us most information about our position relative to our environment. We found a clear advantage to the use of realistic, coherent 3D scenes. Furthermore, we deduce that in order to build a compelling sense of presence, a significant portion of the budget needs to be spent on the projection system, such as the incorporation of a D-ILA projector. We also recommend using a curved screen with a field of view as large as possible. Audio is the second most critical sensory system in simulators. The use of proper localized sounds is highly recommended, as it has proved to improve the sense of vection and convincingness. Furthermore, proper sound can be used in place of a large FOV, as specialized sound yields the same effects as mono sound with a larger field of view. HRTFs based 3D sound positioning is now available on very cheap consumer sound cards, and should by all means be incorporated in an lean and elegant, yet low cost simulator. Regarding somato-sensory information, the installation of transducers under the seat is a great and rather cheap way of greatly improving the feeling of self-motion. Complementary to this, short physical accelerations by abruptly displacing the seat of the observer by distances as short as a centimeter can be used to replace the large hydraulic systems that are often used to physically move large simulators. There is preliminary evidence that such information can possibly be used to trade off some of the visual field of view, while obtaining a similar overall convincingness and vection intensity and onset time.

At the PRESENCE 2005 conference in London, we presented a portable version of our prototype demonstrator, which is the focus of this result. This demonstrator consists of a frame which holds a standard TFT screen, a rubber-suspended base plate with two shakers attached and headphone-based sound. The purpose of the frame is to mask the edges of the screen and to increase immersion. Different masks can be applied to adjust visual FOV. Furthermore, speed wind is simulated by the demonstrator by two ventilators located below the screen. These ventilators may be controlled by any audio signal. For simple demonstrations, one can e.g. control the speed wind by the same audio track as is presented via the headphones. This demonstrator displays the main principles of POEMS and illustrates the fundamental ideas gathered from the different experiments in order to come up with a technical design of a lean, perceptually oriented ego-motion simulator. The demonstrator itself is based on the experimental outcome of the whole project as well as technical investigations and truly represent POEMS original intentions: It is small and transportable (it actually fits in a suitcase), it uses low-cost, off-the-shelf components, has no moving potentially dangerous parts and uses simple but perceptually efficient cues to induce self-motion and presence. In addition, guidelines and ideas on how to extend this prototype and how adapt it to different applications is given in the related documents (primarily those listed as deliverables). Furthermore, larger-scale, non-portable simulators exist at both partners' labs, which can demonstrate the full range of experiments conducted within the POEMS project.

The POEMS project has developed and utilized two types of visual computer models: One type consists of complete geometrical models, where users can navigate around and explore the virtual environment in real time using different input devices (e.g. joystick, buttons, VR-bike etc.). The second type are 360° panoramic photographs, or 'roundshots' of real places. The roundshot models were created by wrapping the 360° roundshots onto a virtual cylinder. This creates an undistorted view for the observer positioned in the centre of the cylinder. Roundshot models have very high graphical fidelity while model complexity is small (typically 1200 polygons with 25MB textures), so they do not demand high computational power. They are well suited for experiments where only rotational motions are applied. There exist several roundshot models, both from open spaces (e.g. Tübingen Market Place) or interior rooms (e.g. Motion Lab at the Max Planck Institute). On the other hand, complete geometrical models are much more complex: Virtual Tübingen for example consists of about 80.000 polygons and 1 GB textures, and thus requires high performance computers for the experiments. The advantage is that they allow for unrestricted large space navigation, with any kind of motion (translation, rotation, roll, pitch etc.). Roundshot models and visual geometrical models have acoustic counterparts in the POEMS models database; HeadScape and Binscape models. These models in turn originate from CATT-Acoustic GEO models. A HeadScape model is created by simulating 128 different binaural room impulse responses (BRIRs, in Lake SIM format) corresponding to different orientations of the listener's head in a CATT-Acoustic model. User interaction (head / camera rotation) is then enabled by the Lake DSP hardware HeadScape convolver tool. As only rotational motion is allowed, this model format will be used in conjunction with the visual roundshot models. HeadScape simulations have very high acoustic fidelity but as they are restricted to rotational motion, we need another type of format to complement the complete visual geometrical models. For this purpose, BinScape models, consisting of GEO-files for early reflection calculation and SIM-files for late reverberation rendering, will be used. As only the early parts of the BRIR (the direct sound + the 1st early reflections) are updated in real time, the fidelity of these models is lower than HeadScape models but on the other hand they allow any kind of motion. When full user interaction is not required or when it is necessary to have very high fidelity acoustic simulations, e.g. in some of the basic experiments and pilot studies, acoustic walkthroughs can also be rendered offline directly in CATT-Acoustic by using the 'Walkthrough convolver'. Means to run these models are provided by the veLib and the HRTF aquisition software and system. The veLib is a software library developed by MPI to meet the requirements of the psychophysical experiments conducted within POEMS. In order to display VR models on a given computer hardware, we need to have the programs that interpret the user's input and provide the appropriate output for the given situation. This simplified description fits most VE/VR applications. However, the overall similarity between different applications can be summarized in a software library, which provides most common features. Many different VR software libraries and packages are publicly available on the Internet or professionally sold by companies. To our knowledge, however, none of them combines all the specific needs of our experiments. This was the reason to develop a library on our own that meets those requirements. Most important of all, we have required a very high degree of control of timing and usage of simplifying features for our psychophysical experiments. The HRTF aquisition system and software was developed for the purpose of investigating the importance of idiosyncratic head-related auditory cues in acoustic motion simulation. HRTFs (Head Related Transfer Functions) are an integral part of many virtual acoustic engines today; however, the importance of using personalized HRTFs is rather unclear for motion simulation scenarios. The system consists of: - 32 loudspeakers; - 3.2m high loudspeaker stand of stainless steel to hold the array; - 1 chair for test person support; - 2 microphones ; - 1 microphone holder; - 1 laser pointer; - 1 PC computer with sound card; - 1 audio power amplifier; - 1 stereo microphone preamplifier; - 1 loudspeaker 32-channel multiplexer; - Dedicated software developed by the Chalmers group for this project. A main goal was to design the system to measure as fast as possible, since previous systems for acquiring HRTFs have been rather time consuming and would have made the experimental process ineffective.

In order to establish the design parameters that lead to good spatial presence and ego-motion sensations the POEMS project has aimed at developing multilevel methods to quantify spatial presence and vection consistently and reliably. These methods combine classical questionnaire-based techniques (i.e., subjective measures) with psychophysical as well as physiological measures (such as GSR) and behavioural measures (such as speeded pointing and forward drift while running in place). The methods are thoroughly described in the deliverables D2.2 - D.6.2 (see list in this results document description). The current general understanding, which has emerged only in recent years, is that presence is best measured through a multilevel approach combining different types of measures, so in this respect POEMS has been on the forefront of research. Also, investigations on corroborating presence sensations with more objective vection measures, examined e.g. in our D4.3, is something which is timely and of great interest to presence research. The multilevel approach as with the inclusion of several modalities however comes with the cost of increased complexity, both technologically and methodologically. It is possible that focusing on one type of measure would have given more clear cut results; yet it is possible that it is only through multilevel measures with which one can take a significant leap forward in the understanding of the perception of mediated environments. Our recommendation is rather to expand the currently proposed measures to include even more aspects; physiological measures of e.g. nystagmus and the psychophysical multimodal nulling paradigm seems to be promising candidates for measures of vection.

Leistungen

Diese Seite teilen

Herunterladen