Final Report Summary - VR-HYPERSPACE (The innovative use of virtual and mixed reality to increase human comfort by changing the perception of self and space)
Executive Summary:
VR-HYPERSPACE (FP7-AAT-285681) was an ambitious 3 year European project (2011-2014) investigating the use of innovative technologies such as virtual reality (VR) and mixed reality (MR) to enhance passenger comfort in future aircraft 2050 and beyond. The goal was to present a radically new approach by creating “positive illusions” with VR/MR, drawing from the latest technology innovations and findings from neuroscience and the psychology of perception, to extend the experience of self and space. These illusions investigated the passenger seeing themselves looking comfortable, feeling like they have more space around them and in the cabin, or being in an alternative, more comfortable place, and able to carry out desired activities and social interaction with others. The consortium consisted of nine Internationally-leading partners: the University of Nottingham, UK (Coordinator); Fraunhofer Institute for Industrial Engineering, Germany; Valtion Teknillien Tutkimuskeskus, Finland; Bauhaus-Universität Weimar, Germany; Institute of Communications and Computer Systems, Greece; University of Barcelona, Spain; Max Planck Institute for Biological Cybernetics, Germany; Thales Alenia Space Italia S.p.A Italy; and Airbus Group, Germany.
The final results of VR-HYPERSPACE provided: (i) Greater understanding of how VR/MR technologies could be used in future aircraft travel to improve the comfort and experience of Passenger 2050 and beyond, with future scenarios proposed based on 14 types of passengers and 4 types of travel requirements; (ii) Scientific and technical foundations on how to create “positive illusions” of self, space, others and the cabin environment using VR/MR to influence in-flight comfort leading to the first set of recommendations; (iii) Concept demonstrators which can be used to investigate different in-flight research topics providing alternative realities, an enhanced cabin experience and collaborative social spaces; and (iv) A roadmap detailing the future scientific and technological breakthroughs required for our vision to be realised by 2050.
The impact on the scientific community is to provide a paradigm shift in relation to passenger comfort with a focus on psychological factors (e.g. changing self-perception, using illusions and distraction, and enabling social interaction). The impact is evident by the 22 publications (journals and conference papers) resulting from the project, with many more in review or in preparation. The impact on the technological community is greater advancements in VR/MR technologies for creating and evaluating positive illusions with novel virtual environments (VEs) (e.g. flying carpets over tropical islands and cityscapes, extended aircraft interiors, invisible planes). VR-HYPERSPACE provided the first steps towards understanding how the interior of an air cabin could be developed and used as an extended display providing VEs in our enhanced cabin demonstrator, and also technical novelties in collaborative virtual environments and telepresence for collocated and remote collaboration with real-time 3D capturing and reconstruction capabilities.
The impact on Air Transport beyond 2050 strategic requirements include: the ability to increase the number of passengers in an aircraft whilst providing a higher standard of comfort, thus addressing the needs of mass transportation and congested infrastructure; the ability to scale advanced technologies depending on the availability of space/surfaces, therefore our concepts are suitable for all types of air vehicles; enabling passengers to fly for longer times, thus our concepts are for all flight durations at all altitudes, addressing the need for long range transport to connect markets and people and for space travel; and providing new functionalities within a shared environment to support business and social mobility needs, thus increasing customer satisfaction and creating new markets for services. It is our hope that air travel will not just be about arriving at a destination but rather will be about experiencing the VR-HYPERSPACE interior cabin. VR-HYPERSPACE has opened up many new and exciting opportunities for research and development in VR/MR to meet future passengers’ expectations of using the aircraft to continue their business and social activities.
Project Context and Objectives:
VR-HYPERSPACE (AAT-285681) was an ambitious three year project (2011-2014) funded under the FP7 Aeronautics and Air Transport (AAT) programme to investigate the use of innovative technologies such as virtual reality (VR) and mixed reality (MR) to enhance passenger comfort in future aircraft 2050 and beyond. The consortium consisted of nine Internationally-leading partners from six European countries: the University of Nottingham, U.K. (Coordinator); Fraunhofer Institute for Industrial Engineering, Germany; Valtion Teknillien Tutkimuskeskus, Finland; Bauhaus-Universität Weimar, Germany; Institute of Communications and Computer Systems, Greece; University of Barcelona, Spain; Max Planck Institute for Biological Cybernetics, Germany; Thales Alenia Space Italia S.p.A Italy; and Airbus Group, Germany.
Results of surveys on passenger experience during air travel revealed that passengers were least satisfied with the effectiveness and amount of personal space available and their ability to work, sleep or rest. Considering current trends, it was likely that the amount of space and therefore the passenger’s comfort during a flight was unlikely to improve and perhaps significantly worsen. Therefore, the challenge was how to maintain a high level of passenger comfort and satisfaction whilst confined in a restrictive physical space.
The goal of VR-HYPERSPACE was to present a radically new approach for future aircraft by creating “positive illusions” with VR/MR, drawing from the latest technology innovations and findings from neuroscience and the psychology of perception, to extend the experience of self and space. These illusions were aimed to allow the passenger to see themselves looking comfortable, feeling like they have more space around them and in the cabin, or that they are in an alternative, more comfortable environment, carrying out desired activities and engaged in social interaction with others.
The main objectives were therefore as follows:
Objective 1: Passenger 2050 and beyond - to provide an understanding of future passengers in the second half of the century in terms of the business and social activities they are likely to want to engage in and how they would like their comfort to be enhanced.
Objective 2: Comfort and self-representation – to investigate the relationship between a person and their virtual representation in order to explore whether changing a person’s virtual self to appear more comfortable could change a person’s perceived level of comfort and physiological state.
Objective 3: Comfort and perception of space – to investigate whether we could change a person’s perception of volume and provide the illusion of a more spacious cabin by altering visual cues whilst in a confined space and even during the presence of motion cues such as turbulence.
Objective 4: Comfort and perception of others and their environment - to investigate new functionalities to improve passengers’ level of comfort by enabling them to personalise their space and interact with others on the plane and on the ground.
Objective 5: Evaluation of current and future technological approaches – to develop and apply innovative evaluation approaches to test emerging and future breakthrough immersive technological concepts using virtual, physical and mixed-mode prototyping.
Objective 6: Research Roadmap to provide a Research Roadmap to support the competitiveness and sustainability of Europe’s Aerospace industry by providing the steps required to develop current and future immersive technologies and applications for use in aircraft cabins.
Our vision was that regardless of future variations in aircraft cabin interiors, we would be able to use state of the art VR and MR technologies and apply findings from neuroscience and psychology to achieve high levels of passenger comfort.
Project Results:
The overall aim of VR-HYPERSPACE was to investigate the use of innovative technologies such as, virtual reality (VR) and mixed reality (MR), to enhance passenger comfort in future aircraft 2050 and beyond. The project had six main objectives, a summary of the key results by objective are described as follows.
1.1 Passenger 2050 and beyond
To understand Passenger 2050 and beyond, we initially defined four scenarios of Passenger 2012 (Patel et al. 2012) through literature, questionnaires, workshops, focus groups and interviews. These passenger scenarios described different current flight settings and experiences, featuring 14 personas of Passenger 2012 and identifying current perceptions of personal comfort, how they affect the comfort of other passengers, how they are affected by other people and the general environment, and the types of activities they currently perform (Patel & Lewis, submitted). A review of future relevant technologies in displays (e.g. 3D, image multiplexing, volumetric, holographic, head mounted displays), illumination (e.g. daylight in cabin, surface lighting) and other media was completed (Cappitelli, Kulik & Bues, 2012). This early work was used to generate a vision of Cabin 2050 and to produce future scenarios of air travel Passenger 2050 and beyond (Tedone, Patel & D’Cruz, 2012).
Regarding the vision of future travel, as stated in a number of reports (most notably in ACARE 2010) there is still an expectation that air travel will continue to increase despite social, technological and environmental changes over the next 40+ years. Business passengers do not believe that tele- or virtual conferencing will replace face-to-face meetings however, there may be more opportunities than currently available to work more efficiently while flying. This is driven by the changes in society to a “real-time world” which demands connectivity at all times, even while in flight. Tourism will always provide opportunities especially if the flight experience becomes one of the attractions (e.g. as in a plane cruiser like a ship cruiser). The majority of Passenger 2050 and beyond are expected to be older as life expectancy increases, and perhaps more women will be travelling as female life expectancy is higher. The impact on air travel is the possibility that more people may travel to other countries for their health or more economical medical assistance. Or people may consider retiring abroad or even further, in outer space(!) and will still want to visit or be connected in some way to their families and friends.
Cabin interior 2050 and beyond, could include innovations in bionic structures, neural technologies, smart, ecological, self-reliant materials, biopolymer membranes, 3D printing, holographic communication, energy harvesting technologies and social seating. The trend towards mobility-enabled personalised services based on high broadband capabilities provides greater opportunities for direct information to a person (termed as foundational services); direct marketing, flight booking, ticketing and use as a credit card (termed as rich media mobility services); and context-aware location-based knowledge such as a personal travel assistant or augmenting reality (termed as future horizons).
The aviation industry are also considering new ways of flying (e.g. airplane carriers, formation flying, etc.), and the increase of space tourism. Space tourism provides an extreme version of limited and confined space which impacts significantly on comfort physically, psychologically, socially and environmentally. With such small groups travelling a long way under these restrictions the following issues have been highlighted as important to consider: monotony and boredom, separation from family and friends, lack of privacy, and restricted and enforced interpersonal contacts. The concepts being explored in VR-HYPERSPACE related to interaction with others and the environment have been considered to address these issues.
Our findings highlighted that the main cause of discomfort appeared to be the lack of perceived control of the passenger to adapt to the environment, to meet needs or change physical and psychological states. Therefore increasing the level of perceived control of the passenger to influence their environment could enhance comfort e.g. enabling them to customise and create a microclimate within their personal space, escape to a virtual world, access personal content with no technical restriction, choose whether/how to interact with others, or expand personal space through individual or shared areas using virtual environments (VEs).
Eighty-two cases of how VR and advanced technologies which could be used to increase passenger comfort were listed from our future scenarios. These were prioritised to 30 cases by the project consortium and discussed with our Aviation Advisory Board (AAB). The AAB consisted of exceptional individuals from industry (Airbus, Eurocopter, Lufthansa, Munich Airport, IDS-Hamburg) and the scientific community (Delft University, Vienna University) with an interest in aircraft interiors, aircraft passengers and comfort. Three meetings were held during the project; Weimar and two meetings jointly hosted in Tὒbingen/Stuttgart (Olbert et al. 2012, 2013a, 2014). The AAB members reviewed progress of the research conducted; offered feedback on the design concepts presented, and provided guidance on the design of our concept demonstrators and final research roadmap (Frangakis et al. 2014).
The use cases and functionalities were related back to our concepts of self, space, others and the environment, and a number of fundamental research questions to be investigated were agreed. These studies (see Section 2.2 2.3 2.4) enabled us to have a better understanding as to how to integrate the functions into our concept prototypes (see Section 2.5) and whether conceptually some of these ideas have the required impact on comfort.
1.2 Comfort and self-representation
An important approach to the problem of overcoming discomfort that we explored in VR-HYPERSPACE was to give passengers a real sensation of comfort through illusory input. By ‘illusory’ we mean powerful illusions that the brain itself generates given appropriate multi-sensory stimuli, and ‘powerful’ in the sense that the illusions cannot be ‘switched off’ simply by knowing that they are illusions - but only through shutting off sensory input. VR-HYPERSPACE carried out research leading to the application of the concept of body representation to the illusion of flight comfort. Body representation is part of the field of research that attempts to understand how the brain represents the body, a current significant area of research in cognitive neuroscience. Contrary to common sense it appears that the brain exhibits significant plasticity with respect to the representation of the body - in other words it can take on radical transformations of the body and accept a changed body as being the body of the ‘self’, attributing to it ownership (this is my body) and agency (I am the cause of actions through this body).
A review of relevant literature on self-embodiment in VR was carried out early in the project (Kokkinara, Leyrer & Saulton, 2012). The review considered studies on how the brain forms a representation of the body and under what conditions would people act and respond to events and situations within VR as if they were real. Previous studies have shown that multisensory stimulation, such as visuotactile or visuomotor stimulation can induce the illusion of owning a fake body. However, in a realistic virtual environment, we need to integrate sensory feedback from vision, motion and touching under the same scenarios. Little work had been done on testing the relative importance of visuotactile and visuomotor correlations when all three sensory inputs were present. Imagine for instance the following scenario: an airplane passenger of 2050 is trying to be immersed in a given virtual world, but unfortunately, due to the limitations on the physical space, bumps constantly against the wall or on their neighbour’s arm. Most probably that would break presence and the illusion of owning the virtual body. The question is whether perfectly correlated visuomotor feedback can alleviate this issue by means of a recalibration of visuotactile representations. If so, this would demonstrate that multisensory recalibration could occur as a result of internal simulation of action and its sensory consequences.
The importance of accuracy of multisensory synchronisation in VR had not yet been investigated under the concept of body ownership. Combining methods for manipulating perceived action or physical state, and hence measuring the limits at which participants experience loss of embodiment, was a major goal of our research. The use of first person perspective along with other techniques of changing self-perception were combined in order to influence the perception of passengers’ physical state, as well as field of actions, and finally evoke a sense of comfort.
Based on these previous studies the following summarises the research topics investigated within VR-HYPERSPACE and the final results and implications for future flights (Kokkinara, Leyrer & Saulton, 2012; Kilteni et al. 2013; Kokkinara et al. 2013).
• Attribution and Persistence of body ownership illusion: Measuring the effects through time of the influence of visuomotor and visuotactile synchronous stimulation on a virtual body ownership illusion Kokkinara & Slater (2014) Results: While synchronous visuomotor correlations contribute the greatest to the attainment of the illusion, a disruption of either (through asynchronous stimulation) contributes equally to the probability of a break in the illusion. The conclusion is that if there are limited resources to use visuomotor, otherwise as much multisensory feedback as possible.
• Psychological, behavioural and attitudinal consequences in relation to the appearance of the virtual body people feel embodied in: Drumming in immersive virtual reality: the body shapes the way we play Kilteni, Bergstrom & Slater (2013) Results: Full virtual body ownership illusions, elicited through rich multisensory and sensorimotor correlations, can lead to substantial behavioural changes in the context of musical performativity, depending on the appearance the ‘new body representation’ disposes
• Mapping small arm movements to large unconstrained movements using adaptation techniques: Amplified arm movements Kokkinara, Slater & López-Moliner (under review). Results: Mapping small arm movements to amplified movements can change participants’ proprioceptive judgments of space around us, without affecting the perceived body-ownership, though through velocity-depended manipulation agency can be affected.
• Improving comfort by virtual posture: Virtual Comfort Bergstrom et al. (in preparation) Results: Subjective experience of comfort and discomfort are modulated by the illusion of embodiment. In uncomfortable position there is a Decrease in Heart Rate Variability, which indicates stress (or illness). There was no change for the comfortable posture.
• Exploring the illusion of walking in an extended space through a moving avatar: Virtual Walking Kokkinara et al. (in preparation). Results: A walking experience through embodiment towards a walking avatar, while the real body is statically sitting on a chair, it is possible to give people the illusion that they are walking. Moreover, the illusory walking seems to influence their physiological state.
• Modulating time perception through music tempo: Time Perception Pizzaro et al. (in preparation). Results: Time perception can be manipulated through controlling the tempo parameter of the music stimuli inside a virtual environment.
• Pain perception by changing the virtual self: When the body fades away: investigating the effects of transparency of an embodied virtual body on pain perception and ownership Martini et al. (in preparation). Results: Body ownership illusion decreases when the body gets more transparent, while the pain threshold is not directly modulated by the level of transparency. Nevertheless, owning a transparent body does modulate pain but in a way that it increases the pain sensitivity. This may have implications for the richness of the way that the body is represented, to make the body more analgesic.
• Space perception by changing virtual Self: Scaled hand Piryankova et al. (2013); Linkenauger et al. (2013) Results: The virtual own hand (and the whole body) is used to scale aspects of the surrounding environment. Thus, with manipulations to the virtual self, space perception can likely be influenced in a desired direction
A key requirement to achieve body ownership illusions are multisensory and sensorimotor correlations, e.g. visuomotor and visuotactile correlation have been shown essential for the induction of the illusion. The comparative contributions of different modalities have been studied within VR-HYPERSPACE. This is an important aspect, because during realistic VR applications such as the ones in a future flight, we want to enable multiple sensory input streams, in order to maximize the effect of the experience. Given the results, we could assume that by providing both visuotactile and visuomotor feedback, we could preserve an overall feeling of owning the virtual body, by keeping at all times at least one of the two stimuli correlated.
Investigation has been also carried out in order to assess the importance of accuracy of multisensory synchronization and the adaptation procedures to mismatched sensory information, under the concept of body ownership. The theoretical background of this study could provide combined knowledge, beneficial for developing application examples for future travellers; given a VR experience, they could be able to carry out tasks in a large and unconstrained space, despite moving only slightly in the confined space of their seat. Physical and psychological state could possibly be altered in a positive way, through similar applications. Using a similar setup, evidence shows that by manipulating the auditory stimuli, such as the tempo of the music track, it is possible to also distort the perception of time passed on a long flight. Perception of posture comfort might also be possible to be alternated through embodiment to a comfortably seated virtual body. Our study however only provided significant evidence that the opposite was possible, i.e. that seeing a virtual body more uncomfortable than the real body, resulted in an adverse experience. The results from our study were inconclusive however, towards achieving the opposite, i.e. inducing a greater experience of comfort from showing a body that is more comfortable than the real one.
Lastly, we have found that by manipulating how the virtual body is represented, in this case by displaying it as if it were semi-transparent, it is possible to increase the pain sensitivity. This may have implications for the richness of the way that the body is represented, to make the body more analgesic.
In conclusion, the VR-HYPERSPACE research on ‘perception of self’ provides the first evidence that full virtual body ownership illusions can lead to substantial behavioural changes in the context of a task that requires continuous body motion, in this case musical performance. Results demonstrate how self-perception can change depending on the body appearance towards which passengers feel embodied. This suggests that by manipulating their appearance, it is also possible to manipulate their experience during a flight.
1.3 Comfort and perception of space
A review of relevant literature on human perception of space from different perspectives was carried out (Kokkinara, Leyrer & Saulton, 2012). The review suggested that there might be multiple promising options to create visual illusions for the perceived volume of space by changing specific aspects of the spatial layout of the environment. However, specifically the concept of perceived volume in human space perception had received less attention in empirical research in contrast to the other areas of human space perception presented. Due to this fact, it was felt necessary to learn from the existing research on distance and size perception in order to explore and understand visual volume perception.
As distance and size contribute both to volume (at least at a geometrical point of view), a promising start is to use a combination of well-established methodologies of investigating distance and size perception for identifying the specific visual cues contributing to perceived volume. If the visual cues which contribute to perceived volume could be identified, it might be easy to manipulate specific cues to generate a desired percept of a virtual world to induce positive illusions for future airplane passengers in 2050.
Besides the visual cues, the most important aspect in the context of VR-HYPERSPACE was to investigate the contribution of body-based cues and the contribution of a virtual body on volume perception. As presented in the review some research had shown that the scaling of visual information likely occurs by using the action capabilities of the body of the observer – which might also be true for a virtual body of an observer. As we expect advancement in VR-technology in the coming decades it might be very easy to incorporate a virtual avatar representing the observer in a virtual environment, without the need of a sophisticated laboratory. Additionally it might be very easy to manipulate the virtual body in a controlled way to influence visual perception, because altering the body’s size and action capabilities in virtual environments should influence the perceived sizes of and distances within a location and therefore also change the perceived volume.
Based on these previous studies the following summarises the research issues which were investigated within VR-HYPERSPACE and the final results and implications for future flights (Kokkinara, Leyrer & Saulton, 2012; Kilteni et al. 2013; Kokkinara et al. 2014).
• Cultural influences on perceived room size: Experiments were conducted with Korean and German populations on strategies for perceiving room size. Saulton et al. (2013). Results: Culture (Asian vs. European) influences perceived room size (volume), therefore passenger populations should be considered when manipulating visual cues to create positive illusions of space.
• Perception of vast spaces: Experiments were and are being conducted with 2D images and 3D virtual environments to determine what types of outdoor environments lead people to feel awe and as if they are in a large space. Thompson et al. (in preparation), Mohler et al. (in preparation). Results: European and American populations report ocean and mountain images with a clear horizon to be vast. These vast environments are candidates for conveying an illusion of a spacious interior cabin.
• Investigating the role of eye height for perceiving space: Multiple experiments investigating perceived distance were conducted to investigate and understand the role of eye height in perceiving space in virtual and real environments. Leyrer et al. 2014a; Leyrer et al. 2014b, Leyrer, 2014. Results: A simple manipulation of eye height can drastically alter perceived space. In virtual scenarios in the airplane this can easily be used to expand the perceived space on an individualized basis.
• Investigating the influence of gravity on perceived space: Experiments investigating distance perception are/were conducted in the real and virtual worlds using different postures (i.e. lying – not aligned with gravity) Leyrer et al. in preparation. Results: Perceived distance/space is independent of the posture of the observer, as long the posture is aligned with gravity. When the gravitational and visual information about the head orientation is in conflict (e.g. passengers lying on their back using an HMD), an illusion of larger perceived distances occurs. For first class passengers or crew this could provide a positive illusion of space during lying postures.
• Motion sickness and the sense of presence during simulated aerial flight with turbulences: Investigating the positive illusion of space and self while being immersed in virtual reality (VR) during a simulated aerial flight with turbulences. Soyka et al. in preparation. Results: Brief exposure to turbulent physical motion during the experience of HMD-based immersive VR does not lead to severe motion sickness. We found evidence in presence and motion sickness measures indicating that it is beneficial to have virtual environments matching the brief physical motions. However, the differences between conditions were marginal. For actual use of HMDs in flight these differences could be negligible and it might not be necessary to incorporate turbulence into the virtual world. This is important since it facilitates the use of HMD-based VR in future transportation.
• Distance perception in large screen immersive displays: Systematic investigation of distance perception in large screen immersive displays, specifically the role of distance to the target, stereoscopic projection and motion parallax on distance perception. Piryankova et al. 2013. Results: Our findings indicate that verbal reports of egocentric distances are nearly veridical in the real world, while verbal reports of egocentric distances are underestimated in all three LSIDs. We found an effect of distance in the real and virtual worlds. Our findings indicate that stereoscopic depth cues create less underestimation but only for the nearest distance.
• Investigating the influence of the visual/virtual body shape on perceived space: Affordance judgments (e.g. can I pass through this aperture) were investigated as a function of visual body shape (from low to high BMI´s). Piryankova et al. (2014). Results: The visual/virtual body can have an influence on perceived space as indicated by affordance judgments. Thus, manipulating a virtual representation of the user might be a useful approach to generate positive illusions of space
• Using our body as a perceptual ruler to perceive the world: Judgments of perceived hand size under minification and corresponding affordance judgments. Linkenauger et al. (2014): Results: The findings suggest that the hand is perceived as having a more constant size and, consequently, can serve as a reliable metric with which to measure objects of commensurate size.
• Investigating the importance of the body for perceiving our surrounding space Multiple experiments using manipulated virtual avatars and size judgments. Linkenauger et al. 2013. Results: Manipulations to a self-avatar (not other avatars) change perceived object size in virtual environments, indicating that manipulations to the virtual self can be used to manipulate perceived space.
In conclusion, a common problem the airplane passenger faces today is limited space, which in turn affects the perceived comfort during a flight. We investigated how visual space is perceived and what possibilities there are to create positive illusions of space to increase the perceived comfort of the passenger. During VR-HYPERSPACE we explored basic questions like how people perceive spaciousness, in particular, how different sensory cues (e.g. auditory cues, visual shape of the environment, visual lighting) affect the perception of spaciousness of the surrounding environment. With the gained knowledge about the processes used to perceive enclosed spaciousness from the viewpoint of an aircraft passenger, it is possible to develop virtual and real interior spaces, which can evoke the illusion of increased space and as such lead to an increased comfort level during flight.
Specifically, we conducted multiple studies using head-mounted displays (HMDs), which allow the user to block out the surrounding environment during flight. Using such setups, we were able to investigate the interplay between different sensory cues like body-based signals and visual information presented in the virtual environment. We found that the body of the user contributes important information to perceived space, more specifically perceived distance (i.e. the eye height), which in turn enables us to create illusions of a larger space by altering simple parameters like the virtual eye height in the virtual environments (VEs). Furthermore, our studies also provide preliminary evidence that the perception of space within HMDs is not negatively affected by the posture of the passenger, at least as long as the posture is aligned with gravity. This means, HMDs can be used throughout a flight without undesired changes in perceived space due to posture changes, which suggests that HMD-based VEs are a valuable tool to increase perceived comfort during long-haul flights.
However, we also discovered a novel illusion of increased space in real and virtual environments, when the user is seeing the visual surrounding environment when lying on his back. Due to this misalignment with gravity, perceived distances appear to be larger than they actually are. This might be a valuable addition to expand perceived space for a passenger as lying down might become more important with future aircraft designs, when HMDs become widely available on aircrafts. Furthermore, this illusion can be coupled with the previously described manipulations of simple parameters like virtual eye height to generate even greater illusions. Nevertheless, those concepts need to be tested in an actual airplane environment during real flying conditions.
Within the context of HMD-based VEs, we also explored whether a virtual self-representation can alter the perceived space in a given VE. With advances in technology, it is likely that every passenger who decides to block out his real surrounding environment and replace it with a virtual reality can have a virtual self-representation in his environment, which can be easily manipulated. We found that specific manipulations of this virtual self-representation can have an effect on how the virtual space is perceived, if the virtual self is moving in synchrony with the real body. Thus, virtual self-representations provide a promising starting point to alter virtual space perception by altering the virtual self, which is, in contrast to our real body, easy in the virtual world.
We also investigated perceived space not only in HMD-based VEs, but also in large screen immersive displays. Multiple concepts within the VR-HYPERSPACE consortium plan to use large display surfaces within the aircraft for multi-user interactions, private workspaces and telepresence. Thus, we investigated how space is perceived in such displays, considering different factors like indicated distances in the virtual worlds, stereoscopic viewing and motion parallax. In future airplanes there will likely be large display surfaces, and thus it is important to understand how VEs are perceived in these display technologies. Our results suggest that with current state-of-the-art display technologies a veridical perception of distance is unlikely, however further research is necessary to investigate how this translates to special environments like an aircraft. However, some of the previously discussed methods to alter perceived virtual space might also apply to large screen immersive displays, alleviating this issue.
Finally, we tested some of the proposed concepts within the VR-HYPERSPACE consortium in the Cybermotion Simulator, a state-of-the-art multi-purpose simulator, during a simulated flight to investigate the feasibility of using HMDs during a real flight scenario.
Preliminary results suggest that the use of HMDs during a flight-like scenario does not cause motion sickness during turbulences (an important requirement). Furthermore, when teleported out of the airplane into a vast VE, the users reported that they completely forgot about the physical constraints of the enclosed cabin of our simulator, and that they really enjoyed the experience of visually being in a vast space. These findings need to be validated during actual flight scenarios.
Thus, in summary, the tested concepts of altering perceived space and using HMDs within an airplane context seems like a promising opportunity for future air travel. With the fast developments of HMDs during the last few years, it is possible that very light-weight, large field-of-view and high resolution HMDs will be widely available in the near future and a promising tool to increase perceived space and comfort during flight conditions. However, further advances in HMD technology and capturing technology (creating the virtual self) are necessary to fully use the potential of the investigated concepts. In addition, corresponding content needs to be created to provide the passengers with appropriate choices during long-haul flights.
1.4 Comfort and perception of others and their environment
Changing the perception of fellow passengers in the airplane cabin with virtual-reality technologies is a challenging task. Masking the presence of annoying others is an intriguing idea (Lewis et al. under review) but also a risky one. Even the most sophisticated technologies could only decrease our peripheral awareness of others, but this would also increase the chance of directly bumping into each other – physically. Thus, a tangible wall would work better and there are good reasons that such available technology did not make it into airplane cabins yet. During VR-HYPERSPACE we developed alternative approaches.
Instead of changing the appearance of others, we developed novel entertainment and communication technologies that encourage social interaction. The perception of fellow passengers changes as a function of our attitude towards each other. VR-Hyperspace suggests large multi-user 3D screens spanning over the backrests of several neighbouring seats. These displays create the visual illusion of virtual environments that can be travelled together. Novel 3D navigation techniques enable groups of related and unrelated passenger groups to explore travel destinations during the flight. Multi-user 3D display technologies as envisioned for the backrests of seats could also be used on the cabin walls to realize virtual windows and thereby create an enhanced experience of the flight.
When travelling alone, people often feel uncomfortable and lose their self-confidence, which in turn can be a reason for being annoyed by the presence of others. During VR-HYPERSPACE we thus developed advanced communication technologies that help staying in touch with friends, family and colleagues on the ground. Our group-to-group telepresence system is a breakthrough technological innovation that enables remote interaction with others at unprecedented fidelity. For the group of passengers sitting next to each other in the plane, the connected people on the ground appear just as standing in front of them. Our multi-user interaction techniques enable joint activities in shared virtual environments. We explored the concept for meeting friends in a casual atmosphere and business meetings that involve the discussion of 3D artefacts.
The real-time 3D acquisition and reconstruction techniques developed for telepresence applications allowed us furthermore to reconsider the role of crew members during the flight. Crew members should certainly be physically present on the plane, but their workload could be reduced through support from virtual crew members that can maintain continuous contact to passengers who need assistance. We explored the concept in terms of passenger acceptance using real demos in our lab and video presentations for online questionnaires.
The following summarises the research topics investigated within VR-HYPERSPACE and the final results and implications for future flights (Kokkinara, Leyrer & Saulton, 2012; Kilteni et al. 2013; Kokkinara et al. 2014).
• Large multi-user screens for joint explorations of travel destinations (super there): Acceptance studies in the lab of BUW. Formal analysis of shared spatial perception. Video presentation for on-line surveys. Beck et al., 2013. Results: Very promising concept, almost readily applicable
• Improvements of display technology required: Large multi-user screens for shared illusions of virtual windows (super here). Appearance studies in the lab of BUW. Video presentation for online surveys. Results: Promising concept, Image quality and lighting (colour gamut, intensity) must be improved to mimic windows
• Casual meetings with remote family and friends (super together): Qualitative assessment during live demos in the lab of BUW. Formal analysis of shared 3D perception. Online survey with video presentation. Beck et al., 2013. Results: Promising concept, but technologies still not mature for use in airplanes. Improvements needed in the fields of 3D capturing, 3D displays and data transmission.
• 3D Telepresence for productive business meetings and online discussions (super together): Qualitative assessment during live demos in the lab of BUW. Formal analysis of shared 3D perception. Online survey with video presentation. Beck et al., 2013 Results: Promising concept, but technologies still not mature for use in airplanes. Improvements needed in the fields of 3D capturing, 3D displays and data transmission.
• Virtual crew: Acceptance studies in the lab of BUW. Online survey with video presentation. Results: Promising concept, but technologies still not mature for use in airplanes. Improvements needed in the fields of 3D capturing, 3D displays and data transmission.
• Personal space and other passengers: Study on the ways other people affect personal space and the use of virtual environments. Lewis et al. under review. Results: A list of positive illusions which could be developed to overcome personal space invasions
Few airplanes are currently equipped with “circadian” Light. Circadian light in this context is supposed to reduce jet-lag on long-haul flights by changing the colour and brightness of the light. Very little is known about the correct parameters and appropriate timing of light on planes. On ground, current research suggests that blue-enriched bright light during the day and warm-white dark light in the evening can increase sleep quality during the night and performance during the day. In a windowless cabin it is very important to provide good lighting quality. This includes the time of day perception. A lot of dynamic light concepts are thinkable to not only reduce jet-lag but also make the passenger believe time is passing more quickly. All these concepts need to be investigated in flight simulators or on board during long-haul flights. The following summarises the research topics investigated within VR-HYPERSPACE and the final results and implications for future flights (Kokkinara, Leyrer & Saulton, 2012; Kilteni et al. 2013; Kokkinara et al. 2014).
• VEs simulating different thermal environments (i.e. cold, hot and room temperature) and the effect on experiences of pain: Effects of different virtual environments on tolerance to, and experiences of, a cold stimulus. Patel et al. in preparation. Results: Evidence to show that VEs can have an impact on perceived discomfort compared to a baseline no VE condition, i.e. that they can increase pain threshold.
• People’s perceived discomfort can be lowered by giving them control over the stimuli they see and hear, i.e. visual and auditory stimuli that they like could reduce discomfort: Distracting from discomfort with an “invisible plane”. Studies to investigate the effectiveness of an apparently invisible aircraft at distracting from sources of discomfort (i.e. physical - restricted legroom or auditory - sound of a baby crying). Lewis (in preparation) Results: A “Transparent Plane” can reduce discomfort experienced. Effect is stronger for restricted legroom than the sound of a crying baby. The VE was particularly effective during more interesting elements i.e. flying over the urban areas rather than countryside.
• Distracting from sources of discomfort with a tropical island: Study to investigate the effectiveness of a tropical island with a beach, palm trees and the sound of waves on distracting from discomfort. Lewis (in preparation) Results: The sound of water and waves seems to be the most important stimulus. Since there was not so much to explore in the virtual tropical island, it didn’t distract from discomfort as much as the invisible plane. For some participants, the relaxing element of the VE counteracted the stressfulness of the crying baby, for other participants, the VE did not match the crying baby sound and therefore it did not help.
• Enabling teleoperations while in a confined space: Use of VR technologies to remotely control robot-rover system remotely to study performance of a physical work task while seated in an airplane. Karaseitanidis (in preparation) Results: Despite technical challenges, such as drifting camera image, the system was estimated to be easy to learn. The participants could consider using the system for performing real work tasks if suitable tasks were available.
HMDs and earphones enable full detachment of our visual and auditory perception from the surrounding environment. For in-flight entertainment, such body-worn gear offers the advantage that passengers can bring their own devices on board and thus avoid frequent and expensive updates of built-in media technology. However, such a detachment from the surrounding environment with these two important senses can also be problematic. We can lose control of our own actions in the actual physical environment, which may be dangerous and socially unacceptable. Moreover, the remaining haptic and olfactory perception of the physical environment is likely to create perceptual conflicts. Preparing solutions which integrate these stimuli to the VE, for the infinite diversity of conflicting stimuli that may occur during air travel is an enormous challenge.
During VR-HYPERSPACE we explored an alternative approach that defines the undisturbed perception of others and the cabin environment as a fundamental constraint for trouble-free interaction with the cabin situation and other passengers. The human perception of self in a given space can nonetheless be positively altered. The shapes and colours of the interior as well as the lighting situation are well-known aspects that have a strong influence on our attitude and well-being, but during several hours of flight, viewing the back of a cabin seat can become an annoying experience – independent of its beauty and elegance. It is no wonder that most passengers eventually reduce their focus to the tiny moving pictures provided by backrest monitors or private tablet PCs. This behaviour is already an obvious decoupling from the actual cabin environment and fellow passengers. HMDs appear to be the logical next step in this direction of passenger entertainment. We note, however, that the small backrest monitors retain at least sufficient context information that allows appropriate passenger reactions to changing situations, e.g. if the seat neighbour needs to pass.
We analysed emerging display technologies and identified other possible advancements from the current situation. Flexible and lightweight OLED display foils are about to become available. In future aircraft interiors they will allow replacements of the tiny backrest monitors with freeform display surfaces covering the cabin walls, floors and seats. As a result, the visual appearance of the cabin environment can be dynamically altered – rather as an extension of the physical situation than a substitute.
Covering the seat backrests with such display technology provides a less public information layer that can be accommodated to the requirements of three to four passengers sitting next to each other in groups. Using the whole space of a row of seat backrests allows covering a large visual field as known from cinemas. Watching movies can become a much more social and engaging past-time in future aircrafts. Moreover, the displays can extend the virtual windows to the seat backrest (“the transparent seat”) or create illusions of alternative 3D spaces, both of which can be shared with fellow passengers. For example, the passengers could explore a virtual model of the target destination for planning their activities. Since each user would need to be provided with an individual dynamic perspective out of the virtual window or at the alternative virtual environment, the applied displays would need to provide individual views to each passenger.
1.5 Evaluation of current and future technological approaches
To test emerging and future breakthrough concepts, VR-HYPERSPACE developed four demonstrators based on virtual, physical and mixed-mode components. A description of each concept demonstrator is as follows.
1.5.1 Self-perception system
In order to induce body-ownership and study self-perception, the self-perception concept demonstrator (Bergstrom, 2014) consisted of the following setup. Participants were immersed in the virtual reality scenario by wearing a stereo NVIS nVisor SX111 head-mounted-display (HMD). This had dual SXGA displays with 76H64V degrees field of view (FOV) per eye, totaling a wide field-of-view of 111 horizontal and 60 vertical, with a resolution of 12801024 per eye displayed at 60 Hz. Head tracking was performed by a 6- degrees of freedom (DOF) Intersense IS-900 device, hence camera’s rotations were calculated from the tracked head location and orientation. The experimenter used a 6-DOF Wand device to deliver tactile stimulation to the real limbs of the participant, in order to induce the illusion of body ownership. The tracked Wand was represented in the virtual reality by a small red ball following the real tracker movements. Limbs are tracked with 12 infrared Optitrack cameras, operating at sub-millimeter precision. Each limb joint can be tracked separately, alternatively the inverse kinematics animation technique can be used to realistically animate limbs with only their extremities tracked. The virtual environments are implemented using the Unity3D platform, while either MiddleVR or a custom plug-in is used in order to incorporate 3D tracker and stereoscopy functionality. The virtual models of the rooms and of the virtual bodies are modelled in 3D Studio Max 2010. The ECG and SCR signals are sampled at 256 Hz using the g.tec’s portable bio-signal acquisition device g.MOBIlab+ while recordings and storage of the data are handled by a Simulink model in Matlab.
1.5.2 Perception of space system
The concept demonstrator used to investigate issues related to space includes the integration of an HMD into the MPS motion simulator based on a serial robot arm (see http://www.cyberneum.com/RoboLab_en.html Mohler & Leyrer, 2013; Soyka, Mohler & Kokkinara, 2014). The simulator has a cabin, which can be used mounted and unmounted. Included in the cabin is a curved projection screen at approximately 1.14m viewing distance, which provides a large field-of-view visual stimulation. The cabin is fully operational with a moving simulator. The CyberMotion Simulator can move participants linearly over a range of several meters and can rotate them around any axis, thus offering a high level of freedom to produce motion. The MPI CyberMotion Simulator is one of the few motion simulators in the world where participants can be passively moved along pre-defined trajectories (open-loop) and be given complete interactive control of their own movements (i.e. closed loop) via a variety of input devices, including a helicopter cyclic stick and a steering wheel. With recent modifications even greater movement is possible, including the rotation of the passenger about the seat axis, the unlimited rotation of the 1st axis and in 2012/early 2013 also movement along a 14m linear axis. Including the new linear axis, the CyberMotion Simulator can produce movements over eight different axes. The mountable cabin for the MPI CyberMotion Simulator Cabin provides a projection on the large curved screen realized with two eyevis DLP projectors, which provide a resolution of 1920x1200. A variety of visual stimuli can be projected on the front and the sides of the curved screen, providing a large visible Field-of-View of 140° horizontal and 70° vertical. This enables us for example to closely simulate for example in-flight scenarios or test the stability of visual illusions during motion.
The addition of a tracked Oculus Rift HMD (http://www.oculusvr.com/) enables a participant to view a number of environments, e.g. a magic carpet flying over an open island scene while the sound of waves are played through headphones in order to enhance the experience. The system can be used to investigate different illusions while in motion.
1.5.3 Acquisition and virtual representation of passengers
For the real time 3D acquisition of virtual passengers we used a cluster of Microsoft Kinect cameras which provide depth and colour images (Beck et al., 2013). Our test setup consists of 5-10 Kinects that are placed in such a way that they can cover the interaction space in front of a large stereoscopic multi-user projection system. The Kinects are connected to PC workstations that act as servers. The servers pre-process the image streams and send them to rendering clients over a network connection where 3D reconstruction is performed. The overall processing pipeline from capturing to the virtual representation works as follows: On the server side the images of each Kinect are pulled at 30 Hz from the driver. The colour images are rectified and compressed. The depth frames are converted to metric distances, rectified and temporally filtered in parallel to the colour streams. Both streams are then transferred to the graphics cards of the rendering clients over a network connection. On the GPU a bilateral filter is applied to the depth frames for noise reduction and finally a 3D view is reconstructed using a 3D warping step. For a good registration of the Kinects to a joint coordinate system it is crucial that the Kinects are well calibrated. This involves spatial rectification of both colour and depth images as well the calibration of the depth sensor itself. To achieve this we introduced the acquisition and use of a depth correction volume that is applied during runtime for each individual Kinect. In order to reduce interference between multiple Kinects we employ the rumble technique described by Maimone and Fuchs: “Reducing interference between multiple structured light depth sensors using motion”. As a test scenario we set up a “Group-to-Group Telepresence” system based on two large projection-based multi-user displays. The people in front of each display are captured by multiple Kinect cameras. The captured data on both sides is sent to the remote groups and there the respective users are reconstructed and displayed as 3D avatars.
1.5.4 Physical cabin environment
The cabin-related technologies concentrated on the aspect of environmental perception for both, individual and cabin-global experiences (Stefani et al. 2013). The prototype mock-up realizes the passenger experience in two main directions. The first we termed “Super-here”, where the passenger is provided with an “enhanced” flying experience (e.g. real-time flight information, location, etc.). The second we termed “Super-there” where the virtual environment displayed is not related to the flight experience and could take the passenger away from the airplane environment (e.g. on the ground).
1.6 Research Roadmap
From the start of the project it was always recognised that it would be impossible to investigate all the issues related to the use of VR/MR for comfort in future aircraft. Given that in 2011 the use of VR/MR was not even considered as an in-flight option, our research was always aimed at a potential future scenario 2050 and beyond where there were no technical restrictions on-board a plane and any personal future devices could work seamlessly and unhindered with the in-flight environment.
The VR-HYPERSPACE research roadmap (D6.4 Frangakis et al. 2014) presents the main research categories of perception of self, space, others and environment, the current state of the art, as shaped in the duration of VR-HYPERSPACE, and the steps needed in a timeline for realizing the our vision. It includes the results of our experiments and potential future experiments that should be performed; key technologies and enablers, possible products and services, and finally the drivers and barriers. The key technologies and enablers are technologies that are needed for achieving this roadmap and are developed outside the research domain of perception (e.g. visual displays, aural, haptic, tracking, gesture, interaction, etc.). In the products and services, we list a number of potential future in-flight entertainment and communication facilities on-board a plane (e.g. superhere (enhancing the flight experience), super there (enhancing the experience), virtual cabin crew, reducing turbulence in flight, etc.). Finally in the drivers and barriers we present external influences which might either drive the further development (e.g. future cabin designs, increase in personal smart devices, changing passenger demographics, social media, space tourism, etc.) or delay it (e.g. cost to airline, passenger, manufacturer; design and acceptance of illusions; VR induced symptoms and effects (VRISE); security and privacy, etc.).
This roadmap is considered as on-going work, which will continuously be updated beyond the project as the technologies evolve and the research progresses. An interactive version can be found on the project website: www.VR-Hyperspace.eu/roadmap.
In summary, VR-HYPERSPACE provides the first scientific and technological evidence of the potential for VR/MR in-flight to enhance passenger comfort and experience. The next steps have already been proposed through our roadmap. We have demonstrated that VR/MR can be the future of novel in-flight entertainment and communication systems.
Potential Impact:
The final results of VR-HYPERSPACE are as follows.
1. Greater understanding of how VR/MR technologies could be used in future aircraft travel to improve the comfort and experience of Passenger 2050 and beyond (Patel et al. 2012; D1.2 Tedone, Patel & D’Cruz, 2012; Patel & D’Cruz under review)
2. Scientific and technical foundations and first recommendations on how to create “positive illusions” of self, space, others and the cabin environment to influence comfort (Kokkinara, Leyrer & Saulton, 2012; Kilteni et al. 2013; Kokkinara et al. 2013; Aaltonen et al. 2014, Bergstrom, 2013; Mohler & Leyrer, 2013; Beck & Kulik, 2013; Stefani et al. 2013; Olbert et al. 2012, 2013a, 2013b, 2014)
3. Concept demonstrators which can be used to investigate different illusions by providing alternative realities, an enhanced cabin experience and collaborative social spaces (Kunert et al. 2013; Beck, Kunert & Kulik, 2014; Stefani, Grobler & Bues, 2014; Soyka, Mohler & Kokkinara, 2014)
4. A roadmap detailing the future technological breakthroughs required for these illusions to be delivered via, affordable, commercially available systems and devices; either built into the aircraft interior or through personal devices that passengers may bring with them (Frangakis et al. 2014)
Impact on Scientific Community
The impact on the scientific community is to provide a paradigm shift in relation to passenger comfort with a focus on psychological factors (e.g. changing self-perception, using illusions and distraction, and enabling social interaction). The results provide the first step towards a greater understanding of perceived comfort, novel concepts for increasing comfort in confined spaces and novel methods for evaluating future concept designs. The impact is evident by the 22 publications resulting from the project so far including: journal papers (Piryankova et al. (2013, 2014); Linkenauger et al. (2013); Kilteni, Bergstrom & Slater (2013); Beck et al. (2013); Kokkinara & Slater, (2013); Aromaa, Viitaniemi & Helin (2012), Saulton et al. (2013); Slater & Sanchez-Vives (2014); Kokkinara & Slater, (2014); Leyrer et al. (2014a, 2014b)); conference proceedings (Kulik et al. 2011; Kunert et al. 2014; Psonis, Frangakis & Karaseitanidis (2012); Karaseitanidis et al. (2013); D’Cruz & Patel (2013); Slater (2013); Bues et al. (2013); Beck et al. (2013); D’Cruz et al. (2014a, 2014b)) with many more in review or in preparation (e.g. Kokkinara, Slater & López-Moliner, Leyrer et al, Patel & Lewis, Patel & D’Cruz, Patel et al. Slater et al).
Impact on the Technological community
The impact on the technological community is greater advancements in VR/MR technologies and in virtual environment (VE) design, specifically in the following areas: methods for creating and evaluating positive illusions in VEs; interaction concepts between people and virtual humans; approaches to high-end technologies in confined spaces; technological advances in the design and development of virtual humans and multi-user, collaborative VEs. VR-HYPERSPACE provided the first steps towards understanding how the interior of an air cabin could be developed and used as an extended display providing novel VEs in our enhanced cabin demonstrator. Also there were several technical novelties in the realm of collaborative virtual environments and telepresence. Our joint research promoted the application of large multi-user 3D displays, which we believe to become a future standard for technical and medical visualization as well as entertainment applications. The Bauhaus-Universität Weimar (BUW) extended this system for collocated collaboration with real-time 3D capturing and reconstruction capabilities. The combination of both technical innovations enabled the realization of an immersive group-to-group telepresence system – for the first time ever. VR-HYPERSPACE demonstrated the potential benefits of such technology for telecommunication during the flight (Beck et al. 2013) and was awarded the prestigious best paper award. The availability of these technologies created novel opportunities and challenges for collaborative 3D interaction. On the basis of lessons learned from the research community concerned with computer-supported cooperative work (CSCW), BUW developed novel interaction paradigms for smooth and efficient collaboration in immersive virtual environments and recently published early results (Kunert et al. 2014). With these developments spanning from novel 3D display technology, over innovative 3D capturing and reconstruction systems, and up to intuitive interfaces and interaction techniques for collaborative engagement with virtual environments, VR-HYPERSPACE has set milestones on the path towards future interactive media technologies and applications.
Socio-economic impact on the Aviation and Aerospace community
The impact on Air Transport beyond 2050 and on ‘ACARE beyond 2050’ strategic requirements are as follows:
• the ability to increase the number of passengers in an aircraft whilst providing a higher standard of comfort, thus addressing the needs of mass transportation and congested infrastructure;
• the ability to scale advanced technologies depending on the availability of space/surfaces, therefore our concepts are suitable for all types of air vehicles;
• enabling passengers to fly for longer times, thus our concepts are for all flight durations at all altitudes, addressing the need for long range transport to connect markets and people and for space travel; and
• providing new functionalities within a shared environment to support business and social mobility needs, thus increasing customer satisfaction and creating new markets for services.
VR-HYPERSPACE provides the enabling technologies and concepts to meet future passengers’ expectations of using the aircraft to continue their business and social activities. It is our hope that air travel will not just be about arriving at a destination but rather will be about experiencing the VR-HYPERSPACE interior cabin.
Increasing cabin capacity
Over the past few decades, the growth of air traffic has been immense and is expected to continue to grow with forecasts predicting a potential demand for 25,000 new passenger and freight aircraft between 2008 and 2028. Over this period, world passenger traffic is expected to increase by 4.7% per annum, and the numbers of flights per year offered on passenger routes will more than double . While enabling more air travel is not seen as a positive for the environment, enabling more passengers to be carried in an aircraft can reduce the number of flights required. In this sense the use of VR/MR can enable the testing of more fuel efficient cabin designs such as the Blend Wing Body (BWB) which can support up to 700 passengers (e.g. the X48B BWB has been characterized by Time magazine as among the top-10 inventions for 2007 ). To allow for such novel concepts to be developed and actually implemented in passenger flights, VR-HYPERSPACE investigated ways in which we could improve social acceptability and implement VR/MR technologies in a novel way. For example, BUW provided a vision of their collaborative social spaces as part of the BWB. The aerodynamics of this airborne cruise ship offer improved fuel efficiency, however its interior is likely to resemble much more a cinema hall or an auditorium than today’s aircraft cabins. The structuring element of the interior will be columns that support the plane against varying air pressures during the flight. Their large surfaces can ideally be used as interactive displays for entertainment and communication. The BWB could accommodate many more passengers than today’s airplanes – all of which require personal attendance. Telepresence technology such as that developed in VR-HYPERSPACE could increase the availability of flight attendants, while requiring less on-board personnel. The virtual air steward could be semi-automated, for example to provide consistent safety instructions in native languages simultaneously at multiple locations in the cabin. The actual crew members could remain on the ground and serve the passengers on demand through their on-board 3D-video avatars.
The physical structure of the BWB does not allow the integration of real windows in the passenger’s seating area however, light-field displays will be used to create wide-angle views at the airplane’s surrounding during the flight and as demonstrated in VR-HYPERSPACE, the whole cabin can be considered as a display that could provide virtual windows or a transparent exterior. The interactive displays can also be used to explore the flight’s destination ahead of arrival and passengers may plan their visit together or use their displays for individual purposes.
The large extent of the BWB air cabin will allow passengers to walk around conveniently for amusement and recreation. The contents displayed at the support columns could be used to create engaging virtual environments, or they may simply be used for advertising and shopping opportunities. VR-HYPERSPACE also demonstrated how telepresence technology may be used on board to stay in contact with family, friends and colleagues - offering an extremely direct and perceptually rich form of staying in contact – even during long-haul flights. In addition, 3D-telepresence technology can enable collaborative working while in transit - groups could meet for online business meetings and jointly explore environments or discuss 3D objects, which are real or digital models. The airplane cabin can effectively become a flying office space for nomadic workers.
Enabling ancillary revenue
Several of the concepts explored through VR-HYPERSPACE form the basis for a series of services provided on-board in the future air cabin as suggested in our final VR-HYPERSPACE Roadmap (Frangakis et al, 2014). Within the aviation industry, such services are expected to grow at a Compound Annual Growth Rate (CAGR) of 6.67% during the period 2012-2017. Currently in-flight services encompass high quality audio and video services (also on-demand), multi- or single player gaming , while wireless entertainment and internet capabilities are gradually being introduced in some airlines. Over the past few years, several breakthrough technologies such as tablets, seat-centric systems, and wireless services have entered the market, transforming the in-flight passenger experience. Through the introduction of VR-HYPERSPACE concepts, airlines could increase their ancillary revenue opportunities by increasing the innovative products and services available on-board.
Improving the passenger experience
VR-HYPERSPACE has shown that passenger experience is perceived differently by each passenger (e.g. some passengers like to escape the cabin environment if they experience flight anxiety, or wish for a quieter and more peaceful environment while others would like to bring back the excitement of flying). We envision that passengers will be able to customize their experience by choosing between a variety of different scenes like beaches, mountains or desert nights with star filled skies, providing every passenger with a unique experience, tailored to their specific preferences. VR-HYPERSPACE has shown that virtual worlds can positively change a person’s perception, resulting in a feeling of spaciousness and a relaxed body. Other passengers might like to make productive use of the transit time or stay in contact with others in order to enhance their flight experience. VR-HYPERSPACE can assist, especially during long-haul flights, in enhancing the experience and productive time spent in transit and maintaining strong connectivity with family, friends and colleagues on board and on the ground. The economic viability of all possible implementations is a focus of future thorough research.
Cabin crew efficiency
Cabin crews carry a number of responsibilities to ensure passengers’ safety and well-being. VR-HYPERSPACE has the potential to support cabin crew on and off the plane. Currently offered on-board services include: sitting assistance, flight information, safety procedures demonstration, destination related information, newspaper and magazine delivery and services related to passengers’ refreshment needs. VR-HYPERSPACE has shown that using telepresence technologies could increase the quality of the service by enabling virtual crew on-board to provide personalized assistance related to information and safety. Further research and development is required to make significant advances to these services.
The wider societal implications
By 2050, the world population may reach nine billion people, and two fifths of that population will be over 50 years old . The aging population is increasingly travelling by air because of increased affluence, cheaper airfares and demographic changes , and older people have specific needs with regard to their health and comfort. In addition, flying is no longer a luxury reserved for holidays: it has become part of our business mobility. The aforementioned imply that more time will be spent on-board and by groups of people for which personalised services would be of importance. VR-HYPERSPACE has shown how VR/AR technologies can be used to provide shared and personalised interactive experiences to increase the efficiency and collaboration of working while in transit, and better connectivity with family and friends for leisure and business activities.
In addition, VR has already shown potential benefits for addressing a number of fears (for example spider or height phobias). Regarding flights it is a fact that fear of flying appears in different levels to approximately 10-25% of people that are flying . Different sources suggest that the fear of flying (Aerophobia) affects either between 1 in 3 or 1 in 5 people . The main reasons have been grouped roughly into several areas of which the key ones are: turbulence, enclosed spaces, crashing and lack of control. VR-HYPERSPACE has shown that we can provide positive illusions to address turbulence, extend space and provide real-time information to give back the control to the passenger and therefore has the potential to help people overcome aerophobia. If proved successful, this could increase the number of people selecting to fly and will greatly enhance passenger experience.
Main dissemination and exploitation of results
VR-HYPERSPACE had ambitious targets in terms of innovative research results; as such this led to a highly successful and prolific dissemination strategy. A description of all activities can be found in Amditis, Georgiou & Brousta (2014) . Highlights include the following.
Public Website
The VR-HYPERSPACE website www.vr-hyperspace.eu was launched in the very beginning of the project and will be sustained for at least five years after the end of the project in order to provide all interested stakeholders with information on project achievements and results and details on contact persons (see . Improvements to the project’s website were constantly proposed and implemented. Important changes included for example: increasing the number of videos; addition of a press announcement section; addition of a LinkedIn registration button; newsletter registration button; and latest projects news clearly visible on the front page.
Poster, Leaflet and Project Newsletters
The design of the poster and leaflet was based upon the vision of the cabin 2050 and beyond, including a flavour of VR/MR technologies that could be employed. The whole idea behind the leaflet was to give a glimpse regarding the 2050 airplane cabin in a futuristic yet convincing way. The project leaflet was distributed at 12 major events including conferences, seminars and tradeshows such as Farnborough Airshow, 2014.
A newsletter had 5 editions and summarised all project news and achievements for each reporting period. The newsletter was a means to address a wider audience and the general public. The newsletter targeted both expert and non-expert audiences and distributed via our LinkedIn site, the euroVR Association mailing list and other contacts.
Paper presentations, panels or demonstration sessions
VR-HYPERSPACE consortium participated in 16 paper presentations, panels and/or demonstration sessions at key events including: the Joint Virtual Reality Conference EGVE-ICAT-EuroVR, JVRC (2011, Nottingham, UK; 2012 Madrid, Spain; 2013 Paris, France; 2014 Bremen, Germany); IEEE VR (2013, Orlando, Florida; 2014, Minneapolis, Minnesota; Laval Virtual Conference (2012, 2014, Laval, France); 17th ACM Conference on Computer supported cooperative work & social computing – CSCW 2014, Baltimore, Massachusetts. Specific highlights are as follows.
IEEEVR2014, Minneapolis, Minnesota on 29th March - 2nd April, 2014 - VR-HYPERSPACE participated as part of the demonstration session, panel session and video session. As the largest and key conference for the VR scientific community, the project was able to reach over 350 participants (D’Cruz et al. 2014a). It was also a great opportunity to deliver and demonstrate our concept prototype results and to elicit feedback from the participants at the conference (D’Cruz et al. 2014b). During the three days of the conference more than 200 people passed by the demonstration booth. In addition the keynote of the conference Henry Fuchs provided the VR-Hyperspace partners with highly positive feedback when visiting the project’s stand.
AIRBUS Workshop, 6th November, 2013, Hamburg, Germany - Held at the Airbus facilities in Hamburg, a wide spread list of over 50 participants from the Cabin Innovation Center, Human Factors and Cabin Engineering departments, EADS Global Innovation Network and Eurocopter participated (Olbert et al., 2014). It was felt by the consortium and participants that the whole event was highly positive and interesting exchanges were made regarding views and expectations around the VR-HYPERSPACE concepts and the future of air cabin designs.
Comfort in Transit Symposium, 2014, Delft, The Netherlands - VR-HYPERSPACE were invited to present and demonstrate at the “Comfort in Transit” Symposium which was organized in Delft, the Netherlands. The symposium brought together a number of experts to discuss recent advances and trends in the fields of aviation, aircraft manufacturing, airport and airline operation and identify the major challenges ahead and discuss possible solutions and ideas. VR-HYPERSPACE concepts and results were well received especially by the industry participants including Airbus, Boeing and Bombardier.
VR-HYPERSPACE Press event - A press event in Germany, hosted by our partners MPG and FhG-IAO on 5th – 6th February, 2014. The purpose was to introduce the project goals and achieved results to the wider public through extended press coverage. Both local and international media participated in the event giving a transnational coverage. Press included journalists from newspapers and magazines including The UK Times newspaper, German radio and BBC TV. The project was shown on the BBC National news http://www.bbc.co.uk/news/technology-26096761 and following the report we were on several regional TV and radio channels with many international publications picking up the story. Given that the UK viewing figures for BBC news per show is around 28 million viewers and we were on for 2 days, the coverage could have exceeded a billion viewers worldwide.
Youtube channel “Computerphile - “Computerphile” is an on-line Youtube educational channel aimed at computer enthusiasts with over 300,000 subscribers but generally millions of viewers. Several of the consortia were interviewed with the following videos related to VR-HYPERSPACE posted during the project:
• In Flight Virtual Reality, June 2014, (http://www.youtube.com/watch?v=PJ5LJkpwBAI)
• Real Life Holodeck, July 2014 http://www.youtube.com/watch?v=7ZPs7knvs7M
• The (pink) VR Simulator, August 2014, http://www.youtube.com/watch?v=Lm0lA0enPSk
• Avatars & In-flight VR, August 2014, http://www.youtube.com/watch?v=TLKqKlrQv4s
• A New Perspective (Multi-Person 3D system), November 2014, https://www.youtube.com/watch?v=qnnEIrWQJdo
Publications in journals and conference proceedings - A major effort towards publishing scientific papers to well respected and highly rated journals had been deployed. This was considered very important in order to reach the scientific target group. Reflecting the scope of VR-HYPERSPACE, there were several scientific fields that would benefit from the project experiences and results. By the close of the project VR-HYPERSPACE partners have had 16 publications, including: eight journal papers (Piryankova et al. (2013); Linkenauger et al. (2013); Kilteni, Bergstrom & Slater (2013); Beck et al. (2013); Kokkinara & Slater, (2013); Aromaa, Viitaniemi & Helin (2012), Saulton et al. (2013); Slater & Sanchez-Vives (2014); and eight papers in conferences proceedings (Kunert et al. (2014); Psonis, Frangakis & Karaseitanidis (2012); Karaseitanidis et al. (2013); D’Cruz & Patel (2013); Slater (2013); Bues et al. (2013); Beck et al. (2013); D’Cruz et al. (2014)).
More publications are expected after the end of the project (three more papers have been submitted already to journals for review and others will be submitted in the near future). In general, the project publication work has been impressive and has reached a level well above the average of a three year European project. This is clear indication of the high quality of both the consortium partners and project’s work. In the list below one can find the existing project publications in journals and conferences.
Furthermore, during the whole duration of the project more than 20 electronic articles and more than 10 videos related to the VR-HYPERSPACE work have been released, e.g. Drummers drum smarter with a laid-back avatar, New Scientist Magazine, March 2013, http://www.newscientist.com; Virtual traveller: Beam a live, 3D you into the world, , New Scientist Magazine, April 2013, http://www.newscientist.com; Flying in virtual reality: Counterfeit comfort, GO! Exploring the world of transportation, June 2013, http://www.go-explore-trans.org/; VR-HYPERSPACE – Flugkomfort in Virtueller Realität, Innovations report , February 2014, http://www.innovations-report.de; Virtual-reality 'assisted' flying? Helping airline passengers experience comfort, space and altered self-perception, Science Daily, Science Daily, http://www.sciencedaily.com; Passagiere fliegen virtuell durch traumhafte Lüfte, DIE WELT, April 2014, http://www.welt.de; Forschungsprojekt: Virtuelle Realität soll für mehr Flugkomfort sorgen, Konstruktions praxis, February 2014, http://www.konstruktionspraxis.vogel.de
Social Media - In the final year the project created a LinkedIn group which currently has more than 90 members so far. This enabled updates and final questionnaires to be distributed and will continue beyond the project as the consortium continues to develop ideas in this area.
References
Aaltonen, L., Aromaa, S., Beck, S., Bergstrom, I., D'Cruz, M., Frangakis, N., Fröhlich, B., Helin, K., Kulik, A., Kunert, A., Lewis, L., Leyrer, M., Mohler, B., Patel, H., Soyka, F., Tedone, D., Viitaniemi, J. (2014) Final user evaluation report. Public Report D5.4 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
ACARE (2010) Aeronautics and air transport: beyond vision 2020 (towards 2050). http://www.acare4europe.com/
Amditis, A., Georgiou, N., and Brousta, E. (2014) Dissemination manual, Month 36. Public Deliverable D6.3.3 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Aromaa, S., Viitaniemi, J., Helin, K. (2012). Virtual Reality Systems Specifications for Human Machine Interaction Design in Industry. Work: A Journal of Prevention, Assessment and Rehabilitation
Beck, S. and Kulik, A. (2013) Acquisition and virtual representation of passengers – technical set up. Confidential Prototype D3.3 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Beck, S. et al. (2013). Airborne Avatars - Inflight Immersive Telepresence. Joint Virtual Reality Conference EGVE-ICAT-EuroVR, JVRC 2013, Paris, France
Beck, S., Kunert, A. and Kulik, A. (2014) Virtual Prototype. Confidential Prototype D4.2 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Beck, S., Kunert, A., Kulik, A. and Froehlich, B. (2013) Immersive Group-to-Group Telepresence. IEEE Transactions on Visualization and Computer Graphics 19, 4 (April 2013), 616-625.
Beck, S., Kunert, A., Kulik, A., Froehlich B. (2013) Immersive Group-to-Group Telepresence
IEEE Transactions on Visualization and Computer Graphics, 19(4):616-25, March 2013 (Proceedings of IEEE Virtual Reality 2013, Orlando, Florida).
Bergstrom, I. (2013) Self-Perception system – technical set up. Confidential Prototype D3.1 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Bues, M., et al. (2013). Enhancing the flight experience: The Transparent Seat and beyond. Joint Virtual Reality Conference EGVE-ICAT-EuroVR, JVRC 2013, Paris, France
Cappitelli, M., Kulik, A. and Bues, M. (2012) Report of future light, display and new media technologies interior spaces of airborne vehicles. Restricted Deliverable D1.2 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
D’Cruz M., Patel H., Lewis L., Cobb S., Bues M., Stefani O., Grobler T., Helin K., Viitaniemi J., Aromaa S., Frohlich B., Beck S., Kunert A., Kulik A., Karaseitanidis I., Psonis P., Frangakis N., Slater M., Bergstrom I., Kilteni K., Kokkinara E., Mohler B.J. Leyrer M., Soyka F., Gaia E., Tedone D., Olbert M. und Cappitelli M. (2014b). VR-HYPERSPACE—The innovative use of virtual reality to increase comfort by changing the perception of self and space. IEEE Virtual Reality (VR 2014), Minneapolis, Minnesota
D’Cruz, M. & Patel, H. (2013) VR-HYPERSPACE: Exploring new ways of enhancing comfort in future air travel using Virtual Reality (VR) and Mixed Reality (MR). Joint Virtual Reality Conference EGVE-ICAT-EuroVR, JVRC 2013, Paris, France
D’Cruz, M., Patel, H., Lewis, L. and Cobb, S. (2014a) Feedback on in-flight applications of virtual reality to enhance comfort in future aircraft. EuroVR 2014, Bremen, Germany, 8-10th December, 2014
Frangakis, N., Karaseitanidis, G., D’Cruz, M., Patel, H., Mohler, B., Bues, M. and Helin, K. (2014) VR-HYPERSPACE Research Roadmap. Public Deliverable D6.4 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Karaseitanidis, G. (2013) Studying the effects of relaxation and stress on the feeling of presence within a VE. Joint Virtual Reality Conference EGVE-ICAT-EuroVR, JVRC 2013, Paris, France
Kilteni, K., Bergstrom, I. and Slater, M. (2013). Drumming in immersive virtual reality: the body shapes the way we play. IEEE Transactions on Visualization and Computer Graphics, 19(4), 597 - 605.
Kilteni, K., Kokkinara, E., Bergstrom, I., López-Moliner, J., Slater, M., Dodds, T., Leyrer, M. Saulton, A., Viitaniemi, J., Helin, K., Aromaa, S., Beck, S., Kunert, A., Kulik. A., Patel, H., Lewis, L., D’Cruz, M., Cobb, S., Karaseitanidis, G., Psonis, P., Frangakis, N., Stefani, O., Bues, M. (2013) Experimental input for integration into prototype. Confidential Deliverable D2.2 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Kokkinara, E. and Slater, M. (2014). Measuring the effects through time of the influence of visuomotor and visuotactile synchronous stimulation on a virtual body ownership illusion. Perception, 43(1), 43–58.
Kokkinara, E., Bergstrom, I., Kilteni, K., López-Moliner, J., Slater, M., Dodds, T., Leyrer, M., Saulton, A., Mohler, B., Viitaniemi, J., Helin, K., Aromaa, S., Beck, S., Kunert, A., Kulik, A., Patel, H., Lewis, L., D’Cruz, M., Cobb, S., Karaseitanidis, G., Psonis, P., Frangakis, N., Stefani, O., Bues, M. (2013) Final overall report of experimental results. Confidential Deliverable D2.3 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Kokkinara, E., Leyrer, M. and Saulton, A. (2012) Literature Review of self embodiment and extended volume space. Public Deliverable D2.1 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Kokkinara, E., Slater, M. and López-Moliner, J. (under review) Spatiotemporal and spatial distortions of arm movements in immersive virtual reality alter judgments of the space around us. ACM TAP
Kulik A., Kunert A., Beck S., Reichel R., Blach R., Zink A., Froehlich B. (2011) C1x6: A Stereoscopic Six-User Display for Co-located Collaboration in Shared Virtual Environments. ACM Transactions on Graphics 30, 6, Article 188 (December 2011)
Kulik, A., Kunert, A., Beck, S., Reichel, R., Blach, R., Zink, A. and Froehlich, B. (2011) C1x6: a stereoscopic six-user display for co-located collaboration in shared virtual environments. In Proceedings of the 2011 SIGGRAPH Asia Conference (SA '11). ACM, New York, NY, USA, Article 188, 12 pages
Kunert, A., Beck, S., Kulik, A., Fröhlich, B. (2013) 3D models of a future airplane and various interior parts. Confidential Prototype D4.1 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Kunert, A., Kulik, A., Beck, S. and Froehlich, B. (2014) Photoportals: shared references in space and time. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing (CSCW '14). ACM, New York, NY, USA, 1388-1399.
Lewis, L. (in preparation) Investigating the ways in which VR can be used to enhance passenger comfort and experience on future aircraft. PhD thesis, The University of Nottingham, UK
Lewis, L., Patel, H., Cobb, S., D’Cruz, M. (under review) Understanding in-flight personal space invasions.
Leyrer, M. (2014) Understanding and Manipulating Eye Height to Change the User's Experience of Perceived Space in Virtual Reality. PhD Thesis, Max Planck Institute of Biological Cybernetics.
Leyrer, M., Linkenauger, SA., Bülthoff, HH and Mohler, B. (2014b). Eye height manipulations: A possible solution to counter underestimation of egocentric distances in head-mounted displays. ACM Transactions on Applied Perception.
Leyrer, M., Linkenauger, SA., Bülthoff, HH. and Mohler, B. (2014a). Eye height for perceiving egocentric distances can be determined by non-visual, body-based cues. PLOS ONE.
Linkenauger SA., Leyrer M., Bülthoff HH., Mohler BJ. (2013) Welcome to Wonderland: The Influence of the Size and Shape of a Virtual Hand on the Perceived Size and Shape of Virtual Objects. PLoS ONE 8(7).
Mohler, B. (2014) Ethical Issues Report. Confidential Report D7.3 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Mohler, B., and Leyrer, M. (2013) Volume system – technical set up. Confidential Prototype D3.2 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Olbert, M., Cappitelli, M, Tedone, D., and Gaia, E. (2013b) Feasibility study of implementation of initial concepts and technologies in real aircraft. Restricted Report D5.2 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Olbert, M., Cappitelli, M., D’Cruz, M. and Tedone, D. (2012) AAB Annual Report. Public Deliverable D6.2 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Olbert, M., Cappitelli, M., D’Cruz, M. and Tedone, D. (2013a) Second AAB Annual Report. Public Deliverable D6.2.2 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Olbert, M., Cappitelli, M., D’Cruz, M. and Tedone, D. (2014) Third AAB Annual Report. Public Deliverable D6.2.3 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Patel, H. and D’Cruz, M. (under review) Factors influencing the experience of aircraft passengers’ comfort. Transportation Research Part A. Policy and Practice.
Patel, H. and Lewis, L. (under review) In-flight experiences of comfort and discomfort – it’s not just about the seat. Applied Ergonomics.
Patel, H., Lewis, L., Cobb, S., D’Cruz, M, Tedone, D., Hakulinen, J. (2012) Report of current scenarios and case definition. Public Deliverable D1.1 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Piryankova, I. V., Wong, H. Y.,Linkenauger, S., Stinson, C Longo, M., Buelthoff, H. H., and Mohler, B. J. 2014. Owning an overweight or underweight body: distinguishing the physical, the experienced and the virtual body. PLoS ONE (August)
Piryankova, IV., de la Rosa, S., Kloos, U., Bülthoff, HH. and Mohler, BJ. (2013) Egocentric distance perception in large screen immersive displays. Displays, 34 (2), 153 - 164.
Piscataway, NJ, USA, 167-168
Psonis, P., Frangakis, N., Karaseitanidis, G. (2012). Using airplane seats as front projection screens. Joint Virtual Reality Conference EGVE-ICAT-EuroVR, JVRC 2012 Madrid, Spain
Saulton, A., Dodds, T., Tesch, J., Mohler, B. and Bȕlthoff, HH. (2013) The influence of shape and culture on visual volume perception of virtual rooms. ACM SAP 142.
Slater, M. & Sanchez-Vives, MV. (2014) Transcending the Self in Immersive Virtual Reality IEEE Computer 47: 7. 24-30 July.
Slater, M. (2013). Fooling the brain about the body using immersive virtual reality. Joint Virtual Reality Conference EGVE-ICAT-EuroVR, JVRC 2013, Paris, France
Soyka, F., Mohler, B. and Kokkinara, E. (2014) Integration of virtual reality equipment and mobile devices into MPS motion simulator. Confidential Prototype D4.4 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Stefani, O., Grobler, T. and Bues, M. (2014) Physical prototype. Confidential Prototype D4.3 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Stefani, O., Haselberger, F., Grobler, T. and Bues, M. (2013) Cabin environment – technical set up. Confidential Prototype D3.4 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Tedone, D., Patel, H. and D’Cruz, M. (2012) Report of future scenarios and case definition. Restricted Deliverable D1.1 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
List of Websites:
http://www.vr-hyperspace.eu
Dr. Mirabelle D'Cruz (Mirabelle.dcruz@nottingham.ac.uk)
VR-HYPERSPACE (FP7-AAT-285681) was an ambitious 3 year European project (2011-2014) investigating the use of innovative technologies such as virtual reality (VR) and mixed reality (MR) to enhance passenger comfort in future aircraft 2050 and beyond. The goal was to present a radically new approach by creating “positive illusions” with VR/MR, drawing from the latest technology innovations and findings from neuroscience and the psychology of perception, to extend the experience of self and space. These illusions investigated the passenger seeing themselves looking comfortable, feeling like they have more space around them and in the cabin, or being in an alternative, more comfortable place, and able to carry out desired activities and social interaction with others. The consortium consisted of nine Internationally-leading partners: the University of Nottingham, UK (Coordinator); Fraunhofer Institute for Industrial Engineering, Germany; Valtion Teknillien Tutkimuskeskus, Finland; Bauhaus-Universität Weimar, Germany; Institute of Communications and Computer Systems, Greece; University of Barcelona, Spain; Max Planck Institute for Biological Cybernetics, Germany; Thales Alenia Space Italia S.p.A Italy; and Airbus Group, Germany.
The final results of VR-HYPERSPACE provided: (i) Greater understanding of how VR/MR technologies could be used in future aircraft travel to improve the comfort and experience of Passenger 2050 and beyond, with future scenarios proposed based on 14 types of passengers and 4 types of travel requirements; (ii) Scientific and technical foundations on how to create “positive illusions” of self, space, others and the cabin environment using VR/MR to influence in-flight comfort leading to the first set of recommendations; (iii) Concept demonstrators which can be used to investigate different in-flight research topics providing alternative realities, an enhanced cabin experience and collaborative social spaces; and (iv) A roadmap detailing the future scientific and technological breakthroughs required for our vision to be realised by 2050.
The impact on the scientific community is to provide a paradigm shift in relation to passenger comfort with a focus on psychological factors (e.g. changing self-perception, using illusions and distraction, and enabling social interaction). The impact is evident by the 22 publications (journals and conference papers) resulting from the project, with many more in review or in preparation. The impact on the technological community is greater advancements in VR/MR technologies for creating and evaluating positive illusions with novel virtual environments (VEs) (e.g. flying carpets over tropical islands and cityscapes, extended aircraft interiors, invisible planes). VR-HYPERSPACE provided the first steps towards understanding how the interior of an air cabin could be developed and used as an extended display providing VEs in our enhanced cabin demonstrator, and also technical novelties in collaborative virtual environments and telepresence for collocated and remote collaboration with real-time 3D capturing and reconstruction capabilities.
The impact on Air Transport beyond 2050 strategic requirements include: the ability to increase the number of passengers in an aircraft whilst providing a higher standard of comfort, thus addressing the needs of mass transportation and congested infrastructure; the ability to scale advanced technologies depending on the availability of space/surfaces, therefore our concepts are suitable for all types of air vehicles; enabling passengers to fly for longer times, thus our concepts are for all flight durations at all altitudes, addressing the need for long range transport to connect markets and people and for space travel; and providing new functionalities within a shared environment to support business and social mobility needs, thus increasing customer satisfaction and creating new markets for services. It is our hope that air travel will not just be about arriving at a destination but rather will be about experiencing the VR-HYPERSPACE interior cabin. VR-HYPERSPACE has opened up many new and exciting opportunities for research and development in VR/MR to meet future passengers’ expectations of using the aircraft to continue their business and social activities.
Project Context and Objectives:
VR-HYPERSPACE (AAT-285681) was an ambitious three year project (2011-2014) funded under the FP7 Aeronautics and Air Transport (AAT) programme to investigate the use of innovative technologies such as virtual reality (VR) and mixed reality (MR) to enhance passenger comfort in future aircraft 2050 and beyond. The consortium consisted of nine Internationally-leading partners from six European countries: the University of Nottingham, U.K. (Coordinator); Fraunhofer Institute for Industrial Engineering, Germany; Valtion Teknillien Tutkimuskeskus, Finland; Bauhaus-Universität Weimar, Germany; Institute of Communications and Computer Systems, Greece; University of Barcelona, Spain; Max Planck Institute for Biological Cybernetics, Germany; Thales Alenia Space Italia S.p.A Italy; and Airbus Group, Germany.
Results of surveys on passenger experience during air travel revealed that passengers were least satisfied with the effectiveness and amount of personal space available and their ability to work, sleep or rest. Considering current trends, it was likely that the amount of space and therefore the passenger’s comfort during a flight was unlikely to improve and perhaps significantly worsen. Therefore, the challenge was how to maintain a high level of passenger comfort and satisfaction whilst confined in a restrictive physical space.
The goal of VR-HYPERSPACE was to present a radically new approach for future aircraft by creating “positive illusions” with VR/MR, drawing from the latest technology innovations and findings from neuroscience and the psychology of perception, to extend the experience of self and space. These illusions were aimed to allow the passenger to see themselves looking comfortable, feeling like they have more space around them and in the cabin, or that they are in an alternative, more comfortable environment, carrying out desired activities and engaged in social interaction with others.
The main objectives were therefore as follows:
Objective 1: Passenger 2050 and beyond - to provide an understanding of future passengers in the second half of the century in terms of the business and social activities they are likely to want to engage in and how they would like their comfort to be enhanced.
Objective 2: Comfort and self-representation – to investigate the relationship between a person and their virtual representation in order to explore whether changing a person’s virtual self to appear more comfortable could change a person’s perceived level of comfort and physiological state.
Objective 3: Comfort and perception of space – to investigate whether we could change a person’s perception of volume and provide the illusion of a more spacious cabin by altering visual cues whilst in a confined space and even during the presence of motion cues such as turbulence.
Objective 4: Comfort and perception of others and their environment - to investigate new functionalities to improve passengers’ level of comfort by enabling them to personalise their space and interact with others on the plane and on the ground.
Objective 5: Evaluation of current and future technological approaches – to develop and apply innovative evaluation approaches to test emerging and future breakthrough immersive technological concepts using virtual, physical and mixed-mode prototyping.
Objective 6: Research Roadmap to provide a Research Roadmap to support the competitiveness and sustainability of Europe’s Aerospace industry by providing the steps required to develop current and future immersive technologies and applications for use in aircraft cabins.
Our vision was that regardless of future variations in aircraft cabin interiors, we would be able to use state of the art VR and MR technologies and apply findings from neuroscience and psychology to achieve high levels of passenger comfort.
Project Results:
The overall aim of VR-HYPERSPACE was to investigate the use of innovative technologies such as, virtual reality (VR) and mixed reality (MR), to enhance passenger comfort in future aircraft 2050 and beyond. The project had six main objectives, a summary of the key results by objective are described as follows.
1.1 Passenger 2050 and beyond
To understand Passenger 2050 and beyond, we initially defined four scenarios of Passenger 2012 (Patel et al. 2012) through literature, questionnaires, workshops, focus groups and interviews. These passenger scenarios described different current flight settings and experiences, featuring 14 personas of Passenger 2012 and identifying current perceptions of personal comfort, how they affect the comfort of other passengers, how they are affected by other people and the general environment, and the types of activities they currently perform (Patel & Lewis, submitted). A review of future relevant technologies in displays (e.g. 3D, image multiplexing, volumetric, holographic, head mounted displays), illumination (e.g. daylight in cabin, surface lighting) and other media was completed (Cappitelli, Kulik & Bues, 2012). This early work was used to generate a vision of Cabin 2050 and to produce future scenarios of air travel Passenger 2050 and beyond (Tedone, Patel & D’Cruz, 2012).
Regarding the vision of future travel, as stated in a number of reports (most notably in ACARE 2010) there is still an expectation that air travel will continue to increase despite social, technological and environmental changes over the next 40+ years. Business passengers do not believe that tele- or virtual conferencing will replace face-to-face meetings however, there may be more opportunities than currently available to work more efficiently while flying. This is driven by the changes in society to a “real-time world” which demands connectivity at all times, even while in flight. Tourism will always provide opportunities especially if the flight experience becomes one of the attractions (e.g. as in a plane cruiser like a ship cruiser). The majority of Passenger 2050 and beyond are expected to be older as life expectancy increases, and perhaps more women will be travelling as female life expectancy is higher. The impact on air travel is the possibility that more people may travel to other countries for their health or more economical medical assistance. Or people may consider retiring abroad or even further, in outer space(!) and will still want to visit or be connected in some way to their families and friends.
Cabin interior 2050 and beyond, could include innovations in bionic structures, neural technologies, smart, ecological, self-reliant materials, biopolymer membranes, 3D printing, holographic communication, energy harvesting technologies and social seating. The trend towards mobility-enabled personalised services based on high broadband capabilities provides greater opportunities for direct information to a person (termed as foundational services); direct marketing, flight booking, ticketing and use as a credit card (termed as rich media mobility services); and context-aware location-based knowledge such as a personal travel assistant or augmenting reality (termed as future horizons).
The aviation industry are also considering new ways of flying (e.g. airplane carriers, formation flying, etc.), and the increase of space tourism. Space tourism provides an extreme version of limited and confined space which impacts significantly on comfort physically, psychologically, socially and environmentally. With such small groups travelling a long way under these restrictions the following issues have been highlighted as important to consider: monotony and boredom, separation from family and friends, lack of privacy, and restricted and enforced interpersonal contacts. The concepts being explored in VR-HYPERSPACE related to interaction with others and the environment have been considered to address these issues.
Our findings highlighted that the main cause of discomfort appeared to be the lack of perceived control of the passenger to adapt to the environment, to meet needs or change physical and psychological states. Therefore increasing the level of perceived control of the passenger to influence their environment could enhance comfort e.g. enabling them to customise and create a microclimate within their personal space, escape to a virtual world, access personal content with no technical restriction, choose whether/how to interact with others, or expand personal space through individual or shared areas using virtual environments (VEs).
Eighty-two cases of how VR and advanced technologies which could be used to increase passenger comfort were listed from our future scenarios. These were prioritised to 30 cases by the project consortium and discussed with our Aviation Advisory Board (AAB). The AAB consisted of exceptional individuals from industry (Airbus, Eurocopter, Lufthansa, Munich Airport, IDS-Hamburg) and the scientific community (Delft University, Vienna University) with an interest in aircraft interiors, aircraft passengers and comfort. Three meetings were held during the project; Weimar and two meetings jointly hosted in Tὒbingen/Stuttgart (Olbert et al. 2012, 2013a, 2014). The AAB members reviewed progress of the research conducted; offered feedback on the design concepts presented, and provided guidance on the design of our concept demonstrators and final research roadmap (Frangakis et al. 2014).
The use cases and functionalities were related back to our concepts of self, space, others and the environment, and a number of fundamental research questions to be investigated were agreed. These studies (see Section 2.2 2.3 2.4) enabled us to have a better understanding as to how to integrate the functions into our concept prototypes (see Section 2.5) and whether conceptually some of these ideas have the required impact on comfort.
1.2 Comfort and self-representation
An important approach to the problem of overcoming discomfort that we explored in VR-HYPERSPACE was to give passengers a real sensation of comfort through illusory input. By ‘illusory’ we mean powerful illusions that the brain itself generates given appropriate multi-sensory stimuli, and ‘powerful’ in the sense that the illusions cannot be ‘switched off’ simply by knowing that they are illusions - but only through shutting off sensory input. VR-HYPERSPACE carried out research leading to the application of the concept of body representation to the illusion of flight comfort. Body representation is part of the field of research that attempts to understand how the brain represents the body, a current significant area of research in cognitive neuroscience. Contrary to common sense it appears that the brain exhibits significant plasticity with respect to the representation of the body - in other words it can take on radical transformations of the body and accept a changed body as being the body of the ‘self’, attributing to it ownership (this is my body) and agency (I am the cause of actions through this body).
A review of relevant literature on self-embodiment in VR was carried out early in the project (Kokkinara, Leyrer & Saulton, 2012). The review considered studies on how the brain forms a representation of the body and under what conditions would people act and respond to events and situations within VR as if they were real. Previous studies have shown that multisensory stimulation, such as visuotactile or visuomotor stimulation can induce the illusion of owning a fake body. However, in a realistic virtual environment, we need to integrate sensory feedback from vision, motion and touching under the same scenarios. Little work had been done on testing the relative importance of visuotactile and visuomotor correlations when all three sensory inputs were present. Imagine for instance the following scenario: an airplane passenger of 2050 is trying to be immersed in a given virtual world, but unfortunately, due to the limitations on the physical space, bumps constantly against the wall or on their neighbour’s arm. Most probably that would break presence and the illusion of owning the virtual body. The question is whether perfectly correlated visuomotor feedback can alleviate this issue by means of a recalibration of visuotactile representations. If so, this would demonstrate that multisensory recalibration could occur as a result of internal simulation of action and its sensory consequences.
The importance of accuracy of multisensory synchronisation in VR had not yet been investigated under the concept of body ownership. Combining methods for manipulating perceived action or physical state, and hence measuring the limits at which participants experience loss of embodiment, was a major goal of our research. The use of first person perspective along with other techniques of changing self-perception were combined in order to influence the perception of passengers’ physical state, as well as field of actions, and finally evoke a sense of comfort.
Based on these previous studies the following summarises the research topics investigated within VR-HYPERSPACE and the final results and implications for future flights (Kokkinara, Leyrer & Saulton, 2012; Kilteni et al. 2013; Kokkinara et al. 2013).
• Attribution and Persistence of body ownership illusion: Measuring the effects through time of the influence of visuomotor and visuotactile synchronous stimulation on a virtual body ownership illusion Kokkinara & Slater (2014) Results: While synchronous visuomotor correlations contribute the greatest to the attainment of the illusion, a disruption of either (through asynchronous stimulation) contributes equally to the probability of a break in the illusion. The conclusion is that if there are limited resources to use visuomotor, otherwise as much multisensory feedback as possible.
• Psychological, behavioural and attitudinal consequences in relation to the appearance of the virtual body people feel embodied in: Drumming in immersive virtual reality: the body shapes the way we play Kilteni, Bergstrom & Slater (2013) Results: Full virtual body ownership illusions, elicited through rich multisensory and sensorimotor correlations, can lead to substantial behavioural changes in the context of musical performativity, depending on the appearance the ‘new body representation’ disposes
• Mapping small arm movements to large unconstrained movements using adaptation techniques: Amplified arm movements Kokkinara, Slater & López-Moliner (under review). Results: Mapping small arm movements to amplified movements can change participants’ proprioceptive judgments of space around us, without affecting the perceived body-ownership, though through velocity-depended manipulation agency can be affected.
• Improving comfort by virtual posture: Virtual Comfort Bergstrom et al. (in preparation) Results: Subjective experience of comfort and discomfort are modulated by the illusion of embodiment. In uncomfortable position there is a Decrease in Heart Rate Variability, which indicates stress (or illness). There was no change for the comfortable posture.
• Exploring the illusion of walking in an extended space through a moving avatar: Virtual Walking Kokkinara et al. (in preparation). Results: A walking experience through embodiment towards a walking avatar, while the real body is statically sitting on a chair, it is possible to give people the illusion that they are walking. Moreover, the illusory walking seems to influence their physiological state.
• Modulating time perception through music tempo: Time Perception Pizzaro et al. (in preparation). Results: Time perception can be manipulated through controlling the tempo parameter of the music stimuli inside a virtual environment.
• Pain perception by changing the virtual self: When the body fades away: investigating the effects of transparency of an embodied virtual body on pain perception and ownership Martini et al. (in preparation). Results: Body ownership illusion decreases when the body gets more transparent, while the pain threshold is not directly modulated by the level of transparency. Nevertheless, owning a transparent body does modulate pain but in a way that it increases the pain sensitivity. This may have implications for the richness of the way that the body is represented, to make the body more analgesic.
• Space perception by changing virtual Self: Scaled hand Piryankova et al. (2013); Linkenauger et al. (2013) Results: The virtual own hand (and the whole body) is used to scale aspects of the surrounding environment. Thus, with manipulations to the virtual self, space perception can likely be influenced in a desired direction
A key requirement to achieve body ownership illusions are multisensory and sensorimotor correlations, e.g. visuomotor and visuotactile correlation have been shown essential for the induction of the illusion. The comparative contributions of different modalities have been studied within VR-HYPERSPACE. This is an important aspect, because during realistic VR applications such as the ones in a future flight, we want to enable multiple sensory input streams, in order to maximize the effect of the experience. Given the results, we could assume that by providing both visuotactile and visuomotor feedback, we could preserve an overall feeling of owning the virtual body, by keeping at all times at least one of the two stimuli correlated.
Investigation has been also carried out in order to assess the importance of accuracy of multisensory synchronization and the adaptation procedures to mismatched sensory information, under the concept of body ownership. The theoretical background of this study could provide combined knowledge, beneficial for developing application examples for future travellers; given a VR experience, they could be able to carry out tasks in a large and unconstrained space, despite moving only slightly in the confined space of their seat. Physical and psychological state could possibly be altered in a positive way, through similar applications. Using a similar setup, evidence shows that by manipulating the auditory stimuli, such as the tempo of the music track, it is possible to also distort the perception of time passed on a long flight. Perception of posture comfort might also be possible to be alternated through embodiment to a comfortably seated virtual body. Our study however only provided significant evidence that the opposite was possible, i.e. that seeing a virtual body more uncomfortable than the real body, resulted in an adverse experience. The results from our study were inconclusive however, towards achieving the opposite, i.e. inducing a greater experience of comfort from showing a body that is more comfortable than the real one.
Lastly, we have found that by manipulating how the virtual body is represented, in this case by displaying it as if it were semi-transparent, it is possible to increase the pain sensitivity. This may have implications for the richness of the way that the body is represented, to make the body more analgesic.
In conclusion, the VR-HYPERSPACE research on ‘perception of self’ provides the first evidence that full virtual body ownership illusions can lead to substantial behavioural changes in the context of a task that requires continuous body motion, in this case musical performance. Results demonstrate how self-perception can change depending on the body appearance towards which passengers feel embodied. This suggests that by manipulating their appearance, it is also possible to manipulate their experience during a flight.
1.3 Comfort and perception of space
A review of relevant literature on human perception of space from different perspectives was carried out (Kokkinara, Leyrer & Saulton, 2012). The review suggested that there might be multiple promising options to create visual illusions for the perceived volume of space by changing specific aspects of the spatial layout of the environment. However, specifically the concept of perceived volume in human space perception had received less attention in empirical research in contrast to the other areas of human space perception presented. Due to this fact, it was felt necessary to learn from the existing research on distance and size perception in order to explore and understand visual volume perception.
As distance and size contribute both to volume (at least at a geometrical point of view), a promising start is to use a combination of well-established methodologies of investigating distance and size perception for identifying the specific visual cues contributing to perceived volume. If the visual cues which contribute to perceived volume could be identified, it might be easy to manipulate specific cues to generate a desired percept of a virtual world to induce positive illusions for future airplane passengers in 2050.
Besides the visual cues, the most important aspect in the context of VR-HYPERSPACE was to investigate the contribution of body-based cues and the contribution of a virtual body on volume perception. As presented in the review some research had shown that the scaling of visual information likely occurs by using the action capabilities of the body of the observer – which might also be true for a virtual body of an observer. As we expect advancement in VR-technology in the coming decades it might be very easy to incorporate a virtual avatar representing the observer in a virtual environment, without the need of a sophisticated laboratory. Additionally it might be very easy to manipulate the virtual body in a controlled way to influence visual perception, because altering the body’s size and action capabilities in virtual environments should influence the perceived sizes of and distances within a location and therefore also change the perceived volume.
Based on these previous studies the following summarises the research issues which were investigated within VR-HYPERSPACE and the final results and implications for future flights (Kokkinara, Leyrer & Saulton, 2012; Kilteni et al. 2013; Kokkinara et al. 2014).
• Cultural influences on perceived room size: Experiments were conducted with Korean and German populations on strategies for perceiving room size. Saulton et al. (2013). Results: Culture (Asian vs. European) influences perceived room size (volume), therefore passenger populations should be considered when manipulating visual cues to create positive illusions of space.
• Perception of vast spaces: Experiments were and are being conducted with 2D images and 3D virtual environments to determine what types of outdoor environments lead people to feel awe and as if they are in a large space. Thompson et al. (in preparation), Mohler et al. (in preparation). Results: European and American populations report ocean and mountain images with a clear horizon to be vast. These vast environments are candidates for conveying an illusion of a spacious interior cabin.
• Investigating the role of eye height for perceiving space: Multiple experiments investigating perceived distance were conducted to investigate and understand the role of eye height in perceiving space in virtual and real environments. Leyrer et al. 2014a; Leyrer et al. 2014b, Leyrer, 2014. Results: A simple manipulation of eye height can drastically alter perceived space. In virtual scenarios in the airplane this can easily be used to expand the perceived space on an individualized basis.
• Investigating the influence of gravity on perceived space: Experiments investigating distance perception are/were conducted in the real and virtual worlds using different postures (i.e. lying – not aligned with gravity) Leyrer et al. in preparation. Results: Perceived distance/space is independent of the posture of the observer, as long the posture is aligned with gravity. When the gravitational and visual information about the head orientation is in conflict (e.g. passengers lying on their back using an HMD), an illusion of larger perceived distances occurs. For first class passengers or crew this could provide a positive illusion of space during lying postures.
• Motion sickness and the sense of presence during simulated aerial flight with turbulences: Investigating the positive illusion of space and self while being immersed in virtual reality (VR) during a simulated aerial flight with turbulences. Soyka et al. in preparation. Results: Brief exposure to turbulent physical motion during the experience of HMD-based immersive VR does not lead to severe motion sickness. We found evidence in presence and motion sickness measures indicating that it is beneficial to have virtual environments matching the brief physical motions. However, the differences between conditions were marginal. For actual use of HMDs in flight these differences could be negligible and it might not be necessary to incorporate turbulence into the virtual world. This is important since it facilitates the use of HMD-based VR in future transportation.
• Distance perception in large screen immersive displays: Systematic investigation of distance perception in large screen immersive displays, specifically the role of distance to the target, stereoscopic projection and motion parallax on distance perception. Piryankova et al. 2013. Results: Our findings indicate that verbal reports of egocentric distances are nearly veridical in the real world, while verbal reports of egocentric distances are underestimated in all three LSIDs. We found an effect of distance in the real and virtual worlds. Our findings indicate that stereoscopic depth cues create less underestimation but only for the nearest distance.
• Investigating the influence of the visual/virtual body shape on perceived space: Affordance judgments (e.g. can I pass through this aperture) were investigated as a function of visual body shape (from low to high BMI´s). Piryankova et al. (2014). Results: The visual/virtual body can have an influence on perceived space as indicated by affordance judgments. Thus, manipulating a virtual representation of the user might be a useful approach to generate positive illusions of space
• Using our body as a perceptual ruler to perceive the world: Judgments of perceived hand size under minification and corresponding affordance judgments. Linkenauger et al. (2014): Results: The findings suggest that the hand is perceived as having a more constant size and, consequently, can serve as a reliable metric with which to measure objects of commensurate size.
• Investigating the importance of the body for perceiving our surrounding space Multiple experiments using manipulated virtual avatars and size judgments. Linkenauger et al. 2013. Results: Manipulations to a self-avatar (not other avatars) change perceived object size in virtual environments, indicating that manipulations to the virtual self can be used to manipulate perceived space.
In conclusion, a common problem the airplane passenger faces today is limited space, which in turn affects the perceived comfort during a flight. We investigated how visual space is perceived and what possibilities there are to create positive illusions of space to increase the perceived comfort of the passenger. During VR-HYPERSPACE we explored basic questions like how people perceive spaciousness, in particular, how different sensory cues (e.g. auditory cues, visual shape of the environment, visual lighting) affect the perception of spaciousness of the surrounding environment. With the gained knowledge about the processes used to perceive enclosed spaciousness from the viewpoint of an aircraft passenger, it is possible to develop virtual and real interior spaces, which can evoke the illusion of increased space and as such lead to an increased comfort level during flight.
Specifically, we conducted multiple studies using head-mounted displays (HMDs), which allow the user to block out the surrounding environment during flight. Using such setups, we were able to investigate the interplay between different sensory cues like body-based signals and visual information presented in the virtual environment. We found that the body of the user contributes important information to perceived space, more specifically perceived distance (i.e. the eye height), which in turn enables us to create illusions of a larger space by altering simple parameters like the virtual eye height in the virtual environments (VEs). Furthermore, our studies also provide preliminary evidence that the perception of space within HMDs is not negatively affected by the posture of the passenger, at least as long as the posture is aligned with gravity. This means, HMDs can be used throughout a flight without undesired changes in perceived space due to posture changes, which suggests that HMD-based VEs are a valuable tool to increase perceived comfort during long-haul flights.
However, we also discovered a novel illusion of increased space in real and virtual environments, when the user is seeing the visual surrounding environment when lying on his back. Due to this misalignment with gravity, perceived distances appear to be larger than they actually are. This might be a valuable addition to expand perceived space for a passenger as lying down might become more important with future aircraft designs, when HMDs become widely available on aircrafts. Furthermore, this illusion can be coupled with the previously described manipulations of simple parameters like virtual eye height to generate even greater illusions. Nevertheless, those concepts need to be tested in an actual airplane environment during real flying conditions.
Within the context of HMD-based VEs, we also explored whether a virtual self-representation can alter the perceived space in a given VE. With advances in technology, it is likely that every passenger who decides to block out his real surrounding environment and replace it with a virtual reality can have a virtual self-representation in his environment, which can be easily manipulated. We found that specific manipulations of this virtual self-representation can have an effect on how the virtual space is perceived, if the virtual self is moving in synchrony with the real body. Thus, virtual self-representations provide a promising starting point to alter virtual space perception by altering the virtual self, which is, in contrast to our real body, easy in the virtual world.
We also investigated perceived space not only in HMD-based VEs, but also in large screen immersive displays. Multiple concepts within the VR-HYPERSPACE consortium plan to use large display surfaces within the aircraft for multi-user interactions, private workspaces and telepresence. Thus, we investigated how space is perceived in such displays, considering different factors like indicated distances in the virtual worlds, stereoscopic viewing and motion parallax. In future airplanes there will likely be large display surfaces, and thus it is important to understand how VEs are perceived in these display technologies. Our results suggest that with current state-of-the-art display technologies a veridical perception of distance is unlikely, however further research is necessary to investigate how this translates to special environments like an aircraft. However, some of the previously discussed methods to alter perceived virtual space might also apply to large screen immersive displays, alleviating this issue.
Finally, we tested some of the proposed concepts within the VR-HYPERSPACE consortium in the Cybermotion Simulator, a state-of-the-art multi-purpose simulator, during a simulated flight to investigate the feasibility of using HMDs during a real flight scenario.
Preliminary results suggest that the use of HMDs during a flight-like scenario does not cause motion sickness during turbulences (an important requirement). Furthermore, when teleported out of the airplane into a vast VE, the users reported that they completely forgot about the physical constraints of the enclosed cabin of our simulator, and that they really enjoyed the experience of visually being in a vast space. These findings need to be validated during actual flight scenarios.
Thus, in summary, the tested concepts of altering perceived space and using HMDs within an airplane context seems like a promising opportunity for future air travel. With the fast developments of HMDs during the last few years, it is possible that very light-weight, large field-of-view and high resolution HMDs will be widely available in the near future and a promising tool to increase perceived space and comfort during flight conditions. However, further advances in HMD technology and capturing technology (creating the virtual self) are necessary to fully use the potential of the investigated concepts. In addition, corresponding content needs to be created to provide the passengers with appropriate choices during long-haul flights.
1.4 Comfort and perception of others and their environment
Changing the perception of fellow passengers in the airplane cabin with virtual-reality technologies is a challenging task. Masking the presence of annoying others is an intriguing idea (Lewis et al. under review) but also a risky one. Even the most sophisticated technologies could only decrease our peripheral awareness of others, but this would also increase the chance of directly bumping into each other – physically. Thus, a tangible wall would work better and there are good reasons that such available technology did not make it into airplane cabins yet. During VR-HYPERSPACE we developed alternative approaches.
Instead of changing the appearance of others, we developed novel entertainment and communication technologies that encourage social interaction. The perception of fellow passengers changes as a function of our attitude towards each other. VR-Hyperspace suggests large multi-user 3D screens spanning over the backrests of several neighbouring seats. These displays create the visual illusion of virtual environments that can be travelled together. Novel 3D navigation techniques enable groups of related and unrelated passenger groups to explore travel destinations during the flight. Multi-user 3D display technologies as envisioned for the backrests of seats could also be used on the cabin walls to realize virtual windows and thereby create an enhanced experience of the flight.
When travelling alone, people often feel uncomfortable and lose their self-confidence, which in turn can be a reason for being annoyed by the presence of others. During VR-HYPERSPACE we thus developed advanced communication technologies that help staying in touch with friends, family and colleagues on the ground. Our group-to-group telepresence system is a breakthrough technological innovation that enables remote interaction with others at unprecedented fidelity. For the group of passengers sitting next to each other in the plane, the connected people on the ground appear just as standing in front of them. Our multi-user interaction techniques enable joint activities in shared virtual environments. We explored the concept for meeting friends in a casual atmosphere and business meetings that involve the discussion of 3D artefacts.
The real-time 3D acquisition and reconstruction techniques developed for telepresence applications allowed us furthermore to reconsider the role of crew members during the flight. Crew members should certainly be physically present on the plane, but their workload could be reduced through support from virtual crew members that can maintain continuous contact to passengers who need assistance. We explored the concept in terms of passenger acceptance using real demos in our lab and video presentations for online questionnaires.
The following summarises the research topics investigated within VR-HYPERSPACE and the final results and implications for future flights (Kokkinara, Leyrer & Saulton, 2012; Kilteni et al. 2013; Kokkinara et al. 2014).
• Large multi-user screens for joint explorations of travel destinations (super there): Acceptance studies in the lab of BUW. Formal analysis of shared spatial perception. Video presentation for on-line surveys. Beck et al., 2013. Results: Very promising concept, almost readily applicable
• Improvements of display technology required: Large multi-user screens for shared illusions of virtual windows (super here). Appearance studies in the lab of BUW. Video presentation for online surveys. Results: Promising concept, Image quality and lighting (colour gamut, intensity) must be improved to mimic windows
• Casual meetings with remote family and friends (super together): Qualitative assessment during live demos in the lab of BUW. Formal analysis of shared 3D perception. Online survey with video presentation. Beck et al., 2013. Results: Promising concept, but technologies still not mature for use in airplanes. Improvements needed in the fields of 3D capturing, 3D displays and data transmission.
• 3D Telepresence for productive business meetings and online discussions (super together): Qualitative assessment during live demos in the lab of BUW. Formal analysis of shared 3D perception. Online survey with video presentation. Beck et al., 2013 Results: Promising concept, but technologies still not mature for use in airplanes. Improvements needed in the fields of 3D capturing, 3D displays and data transmission.
• Virtual crew: Acceptance studies in the lab of BUW. Online survey with video presentation. Results: Promising concept, but technologies still not mature for use in airplanes. Improvements needed in the fields of 3D capturing, 3D displays and data transmission.
• Personal space and other passengers: Study on the ways other people affect personal space and the use of virtual environments. Lewis et al. under review. Results: A list of positive illusions which could be developed to overcome personal space invasions
Few airplanes are currently equipped with “circadian” Light. Circadian light in this context is supposed to reduce jet-lag on long-haul flights by changing the colour and brightness of the light. Very little is known about the correct parameters and appropriate timing of light on planes. On ground, current research suggests that blue-enriched bright light during the day and warm-white dark light in the evening can increase sleep quality during the night and performance during the day. In a windowless cabin it is very important to provide good lighting quality. This includes the time of day perception. A lot of dynamic light concepts are thinkable to not only reduce jet-lag but also make the passenger believe time is passing more quickly. All these concepts need to be investigated in flight simulators or on board during long-haul flights. The following summarises the research topics investigated within VR-HYPERSPACE and the final results and implications for future flights (Kokkinara, Leyrer & Saulton, 2012; Kilteni et al. 2013; Kokkinara et al. 2014).
• VEs simulating different thermal environments (i.e. cold, hot and room temperature) and the effect on experiences of pain: Effects of different virtual environments on tolerance to, and experiences of, a cold stimulus. Patel et al. in preparation. Results: Evidence to show that VEs can have an impact on perceived discomfort compared to a baseline no VE condition, i.e. that they can increase pain threshold.
• People’s perceived discomfort can be lowered by giving them control over the stimuli they see and hear, i.e. visual and auditory stimuli that they like could reduce discomfort: Distracting from discomfort with an “invisible plane”. Studies to investigate the effectiveness of an apparently invisible aircraft at distracting from sources of discomfort (i.e. physical - restricted legroom or auditory - sound of a baby crying). Lewis (in preparation) Results: A “Transparent Plane” can reduce discomfort experienced. Effect is stronger for restricted legroom than the sound of a crying baby. The VE was particularly effective during more interesting elements i.e. flying over the urban areas rather than countryside.
• Distracting from sources of discomfort with a tropical island: Study to investigate the effectiveness of a tropical island with a beach, palm trees and the sound of waves on distracting from discomfort. Lewis (in preparation) Results: The sound of water and waves seems to be the most important stimulus. Since there was not so much to explore in the virtual tropical island, it didn’t distract from discomfort as much as the invisible plane. For some participants, the relaxing element of the VE counteracted the stressfulness of the crying baby, for other participants, the VE did not match the crying baby sound and therefore it did not help.
• Enabling teleoperations while in a confined space: Use of VR technologies to remotely control robot-rover system remotely to study performance of a physical work task while seated in an airplane. Karaseitanidis (in preparation) Results: Despite technical challenges, such as drifting camera image, the system was estimated to be easy to learn. The participants could consider using the system for performing real work tasks if suitable tasks were available.
HMDs and earphones enable full detachment of our visual and auditory perception from the surrounding environment. For in-flight entertainment, such body-worn gear offers the advantage that passengers can bring their own devices on board and thus avoid frequent and expensive updates of built-in media technology. However, such a detachment from the surrounding environment with these two important senses can also be problematic. We can lose control of our own actions in the actual physical environment, which may be dangerous and socially unacceptable. Moreover, the remaining haptic and olfactory perception of the physical environment is likely to create perceptual conflicts. Preparing solutions which integrate these stimuli to the VE, for the infinite diversity of conflicting stimuli that may occur during air travel is an enormous challenge.
During VR-HYPERSPACE we explored an alternative approach that defines the undisturbed perception of others and the cabin environment as a fundamental constraint for trouble-free interaction with the cabin situation and other passengers. The human perception of self in a given space can nonetheless be positively altered. The shapes and colours of the interior as well as the lighting situation are well-known aspects that have a strong influence on our attitude and well-being, but during several hours of flight, viewing the back of a cabin seat can become an annoying experience – independent of its beauty and elegance. It is no wonder that most passengers eventually reduce their focus to the tiny moving pictures provided by backrest monitors or private tablet PCs. This behaviour is already an obvious decoupling from the actual cabin environment and fellow passengers. HMDs appear to be the logical next step in this direction of passenger entertainment. We note, however, that the small backrest monitors retain at least sufficient context information that allows appropriate passenger reactions to changing situations, e.g. if the seat neighbour needs to pass.
We analysed emerging display technologies and identified other possible advancements from the current situation. Flexible and lightweight OLED display foils are about to become available. In future aircraft interiors they will allow replacements of the tiny backrest monitors with freeform display surfaces covering the cabin walls, floors and seats. As a result, the visual appearance of the cabin environment can be dynamically altered – rather as an extension of the physical situation than a substitute.
Covering the seat backrests with such display technology provides a less public information layer that can be accommodated to the requirements of three to four passengers sitting next to each other in groups. Using the whole space of a row of seat backrests allows covering a large visual field as known from cinemas. Watching movies can become a much more social and engaging past-time in future aircrafts. Moreover, the displays can extend the virtual windows to the seat backrest (“the transparent seat”) or create illusions of alternative 3D spaces, both of which can be shared with fellow passengers. For example, the passengers could explore a virtual model of the target destination for planning their activities. Since each user would need to be provided with an individual dynamic perspective out of the virtual window or at the alternative virtual environment, the applied displays would need to provide individual views to each passenger.
1.5 Evaluation of current and future technological approaches
To test emerging and future breakthrough concepts, VR-HYPERSPACE developed four demonstrators based on virtual, physical and mixed-mode components. A description of each concept demonstrator is as follows.
1.5.1 Self-perception system
In order to induce body-ownership and study self-perception, the self-perception concept demonstrator (Bergstrom, 2014) consisted of the following setup. Participants were immersed in the virtual reality scenario by wearing a stereo NVIS nVisor SX111 head-mounted-display (HMD). This had dual SXGA displays with 76H64V degrees field of view (FOV) per eye, totaling a wide field-of-view of 111 horizontal and 60 vertical, with a resolution of 12801024 per eye displayed at 60 Hz. Head tracking was performed by a 6- degrees of freedom (DOF) Intersense IS-900 device, hence camera’s rotations were calculated from the tracked head location and orientation. The experimenter used a 6-DOF Wand device to deliver tactile stimulation to the real limbs of the participant, in order to induce the illusion of body ownership. The tracked Wand was represented in the virtual reality by a small red ball following the real tracker movements. Limbs are tracked with 12 infrared Optitrack cameras, operating at sub-millimeter precision. Each limb joint can be tracked separately, alternatively the inverse kinematics animation technique can be used to realistically animate limbs with only their extremities tracked. The virtual environments are implemented using the Unity3D platform, while either MiddleVR or a custom plug-in is used in order to incorporate 3D tracker and stereoscopy functionality. The virtual models of the rooms and of the virtual bodies are modelled in 3D Studio Max 2010. The ECG and SCR signals are sampled at 256 Hz using the g.tec’s portable bio-signal acquisition device g.MOBIlab+ while recordings and storage of the data are handled by a Simulink model in Matlab.
1.5.2 Perception of space system
The concept demonstrator used to investigate issues related to space includes the integration of an HMD into the MPS motion simulator based on a serial robot arm (see http://www.cyberneum.com/RoboLab_en.html Mohler & Leyrer, 2013; Soyka, Mohler & Kokkinara, 2014). The simulator has a cabin, which can be used mounted and unmounted. Included in the cabin is a curved projection screen at approximately 1.14m viewing distance, which provides a large field-of-view visual stimulation. The cabin is fully operational with a moving simulator. The CyberMotion Simulator can move participants linearly over a range of several meters and can rotate them around any axis, thus offering a high level of freedom to produce motion. The MPI CyberMotion Simulator is one of the few motion simulators in the world where participants can be passively moved along pre-defined trajectories (open-loop) and be given complete interactive control of their own movements (i.e. closed loop) via a variety of input devices, including a helicopter cyclic stick and a steering wheel. With recent modifications even greater movement is possible, including the rotation of the passenger about the seat axis, the unlimited rotation of the 1st axis and in 2012/early 2013 also movement along a 14m linear axis. Including the new linear axis, the CyberMotion Simulator can produce movements over eight different axes. The mountable cabin for the MPI CyberMotion Simulator Cabin provides a projection on the large curved screen realized with two eyevis DLP projectors, which provide a resolution of 1920x1200. A variety of visual stimuli can be projected on the front and the sides of the curved screen, providing a large visible Field-of-View of 140° horizontal and 70° vertical. This enables us for example to closely simulate for example in-flight scenarios or test the stability of visual illusions during motion.
The addition of a tracked Oculus Rift HMD (http://www.oculusvr.com/) enables a participant to view a number of environments, e.g. a magic carpet flying over an open island scene while the sound of waves are played through headphones in order to enhance the experience. The system can be used to investigate different illusions while in motion.
1.5.3 Acquisition and virtual representation of passengers
For the real time 3D acquisition of virtual passengers we used a cluster of Microsoft Kinect cameras which provide depth and colour images (Beck et al., 2013). Our test setup consists of 5-10 Kinects that are placed in such a way that they can cover the interaction space in front of a large stereoscopic multi-user projection system. The Kinects are connected to PC workstations that act as servers. The servers pre-process the image streams and send them to rendering clients over a network connection where 3D reconstruction is performed. The overall processing pipeline from capturing to the virtual representation works as follows: On the server side the images of each Kinect are pulled at 30 Hz from the driver. The colour images are rectified and compressed. The depth frames are converted to metric distances, rectified and temporally filtered in parallel to the colour streams. Both streams are then transferred to the graphics cards of the rendering clients over a network connection. On the GPU a bilateral filter is applied to the depth frames for noise reduction and finally a 3D view is reconstructed using a 3D warping step. For a good registration of the Kinects to a joint coordinate system it is crucial that the Kinects are well calibrated. This involves spatial rectification of both colour and depth images as well the calibration of the depth sensor itself. To achieve this we introduced the acquisition and use of a depth correction volume that is applied during runtime for each individual Kinect. In order to reduce interference between multiple Kinects we employ the rumble technique described by Maimone and Fuchs: “Reducing interference between multiple structured light depth sensors using motion”. As a test scenario we set up a “Group-to-Group Telepresence” system based on two large projection-based multi-user displays. The people in front of each display are captured by multiple Kinect cameras. The captured data on both sides is sent to the remote groups and there the respective users are reconstructed and displayed as 3D avatars.
1.5.4 Physical cabin environment
The cabin-related technologies concentrated on the aspect of environmental perception for both, individual and cabin-global experiences (Stefani et al. 2013). The prototype mock-up realizes the passenger experience in two main directions. The first we termed “Super-here”, where the passenger is provided with an “enhanced” flying experience (e.g. real-time flight information, location, etc.). The second we termed “Super-there” where the virtual environment displayed is not related to the flight experience and could take the passenger away from the airplane environment (e.g. on the ground).
1.6 Research Roadmap
From the start of the project it was always recognised that it would be impossible to investigate all the issues related to the use of VR/MR for comfort in future aircraft. Given that in 2011 the use of VR/MR was not even considered as an in-flight option, our research was always aimed at a potential future scenario 2050 and beyond where there were no technical restrictions on-board a plane and any personal future devices could work seamlessly and unhindered with the in-flight environment.
The VR-HYPERSPACE research roadmap (D6.4 Frangakis et al. 2014) presents the main research categories of perception of self, space, others and environment, the current state of the art, as shaped in the duration of VR-HYPERSPACE, and the steps needed in a timeline for realizing the our vision. It includes the results of our experiments and potential future experiments that should be performed; key technologies and enablers, possible products and services, and finally the drivers and barriers. The key technologies and enablers are technologies that are needed for achieving this roadmap and are developed outside the research domain of perception (e.g. visual displays, aural, haptic, tracking, gesture, interaction, etc.). In the products and services, we list a number of potential future in-flight entertainment and communication facilities on-board a plane (e.g. superhere (enhancing the flight experience), super there (enhancing the experience), virtual cabin crew, reducing turbulence in flight, etc.). Finally in the drivers and barriers we present external influences which might either drive the further development (e.g. future cabin designs, increase in personal smart devices, changing passenger demographics, social media, space tourism, etc.) or delay it (e.g. cost to airline, passenger, manufacturer; design and acceptance of illusions; VR induced symptoms and effects (VRISE); security and privacy, etc.).
This roadmap is considered as on-going work, which will continuously be updated beyond the project as the technologies evolve and the research progresses. An interactive version can be found on the project website: www.VR-Hyperspace.eu/roadmap.
In summary, VR-HYPERSPACE provides the first scientific and technological evidence of the potential for VR/MR in-flight to enhance passenger comfort and experience. The next steps have already been proposed through our roadmap. We have demonstrated that VR/MR can be the future of novel in-flight entertainment and communication systems.
Potential Impact:
The final results of VR-HYPERSPACE are as follows.
1. Greater understanding of how VR/MR technologies could be used in future aircraft travel to improve the comfort and experience of Passenger 2050 and beyond (Patel et al. 2012; D1.2 Tedone, Patel & D’Cruz, 2012; Patel & D’Cruz under review)
2. Scientific and technical foundations and first recommendations on how to create “positive illusions” of self, space, others and the cabin environment to influence comfort (Kokkinara, Leyrer & Saulton, 2012; Kilteni et al. 2013; Kokkinara et al. 2013; Aaltonen et al. 2014, Bergstrom, 2013; Mohler & Leyrer, 2013; Beck & Kulik, 2013; Stefani et al. 2013; Olbert et al. 2012, 2013a, 2013b, 2014)
3. Concept demonstrators which can be used to investigate different illusions by providing alternative realities, an enhanced cabin experience and collaborative social spaces (Kunert et al. 2013; Beck, Kunert & Kulik, 2014; Stefani, Grobler & Bues, 2014; Soyka, Mohler & Kokkinara, 2014)
4. A roadmap detailing the future technological breakthroughs required for these illusions to be delivered via, affordable, commercially available systems and devices; either built into the aircraft interior or through personal devices that passengers may bring with them (Frangakis et al. 2014)
Impact on Scientific Community
The impact on the scientific community is to provide a paradigm shift in relation to passenger comfort with a focus on psychological factors (e.g. changing self-perception, using illusions and distraction, and enabling social interaction). The results provide the first step towards a greater understanding of perceived comfort, novel concepts for increasing comfort in confined spaces and novel methods for evaluating future concept designs. The impact is evident by the 22 publications resulting from the project so far including: journal papers (Piryankova et al. (2013, 2014); Linkenauger et al. (2013); Kilteni, Bergstrom & Slater (2013); Beck et al. (2013); Kokkinara & Slater, (2013); Aromaa, Viitaniemi & Helin (2012), Saulton et al. (2013); Slater & Sanchez-Vives (2014); Kokkinara & Slater, (2014); Leyrer et al. (2014a, 2014b)); conference proceedings (Kulik et al. 2011; Kunert et al. 2014; Psonis, Frangakis & Karaseitanidis (2012); Karaseitanidis et al. (2013); D’Cruz & Patel (2013); Slater (2013); Bues et al. (2013); Beck et al. (2013); D’Cruz et al. (2014a, 2014b)) with many more in review or in preparation (e.g. Kokkinara, Slater & López-Moliner, Leyrer et al, Patel & Lewis, Patel & D’Cruz, Patel et al. Slater et al).
Impact on the Technological community
The impact on the technological community is greater advancements in VR/MR technologies and in virtual environment (VE) design, specifically in the following areas: methods for creating and evaluating positive illusions in VEs; interaction concepts between people and virtual humans; approaches to high-end technologies in confined spaces; technological advances in the design and development of virtual humans and multi-user, collaborative VEs. VR-HYPERSPACE provided the first steps towards understanding how the interior of an air cabin could be developed and used as an extended display providing novel VEs in our enhanced cabin demonstrator. Also there were several technical novelties in the realm of collaborative virtual environments and telepresence. Our joint research promoted the application of large multi-user 3D displays, which we believe to become a future standard for technical and medical visualization as well as entertainment applications. The Bauhaus-Universität Weimar (BUW) extended this system for collocated collaboration with real-time 3D capturing and reconstruction capabilities. The combination of both technical innovations enabled the realization of an immersive group-to-group telepresence system – for the first time ever. VR-HYPERSPACE demonstrated the potential benefits of such technology for telecommunication during the flight (Beck et al. 2013) and was awarded the prestigious best paper award. The availability of these technologies created novel opportunities and challenges for collaborative 3D interaction. On the basis of lessons learned from the research community concerned with computer-supported cooperative work (CSCW), BUW developed novel interaction paradigms for smooth and efficient collaboration in immersive virtual environments and recently published early results (Kunert et al. 2014). With these developments spanning from novel 3D display technology, over innovative 3D capturing and reconstruction systems, and up to intuitive interfaces and interaction techniques for collaborative engagement with virtual environments, VR-HYPERSPACE has set milestones on the path towards future interactive media technologies and applications.
Socio-economic impact on the Aviation and Aerospace community
The impact on Air Transport beyond 2050 and on ‘ACARE beyond 2050’ strategic requirements are as follows:
• the ability to increase the number of passengers in an aircraft whilst providing a higher standard of comfort, thus addressing the needs of mass transportation and congested infrastructure;
• the ability to scale advanced technologies depending on the availability of space/surfaces, therefore our concepts are suitable for all types of air vehicles;
• enabling passengers to fly for longer times, thus our concepts are for all flight durations at all altitudes, addressing the need for long range transport to connect markets and people and for space travel; and
• providing new functionalities within a shared environment to support business and social mobility needs, thus increasing customer satisfaction and creating new markets for services.
VR-HYPERSPACE provides the enabling technologies and concepts to meet future passengers’ expectations of using the aircraft to continue their business and social activities. It is our hope that air travel will not just be about arriving at a destination but rather will be about experiencing the VR-HYPERSPACE interior cabin.
Increasing cabin capacity
Over the past few decades, the growth of air traffic has been immense and is expected to continue to grow with forecasts predicting a potential demand for 25,000 new passenger and freight aircraft between 2008 and 2028. Over this period, world passenger traffic is expected to increase by 4.7% per annum, and the numbers of flights per year offered on passenger routes will more than double . While enabling more air travel is not seen as a positive for the environment, enabling more passengers to be carried in an aircraft can reduce the number of flights required. In this sense the use of VR/MR can enable the testing of more fuel efficient cabin designs such as the Blend Wing Body (BWB) which can support up to 700 passengers (e.g. the X48B BWB has been characterized by Time magazine as among the top-10 inventions for 2007 ). To allow for such novel concepts to be developed and actually implemented in passenger flights, VR-HYPERSPACE investigated ways in which we could improve social acceptability and implement VR/MR technologies in a novel way. For example, BUW provided a vision of their collaborative social spaces as part of the BWB. The aerodynamics of this airborne cruise ship offer improved fuel efficiency, however its interior is likely to resemble much more a cinema hall or an auditorium than today’s aircraft cabins. The structuring element of the interior will be columns that support the plane against varying air pressures during the flight. Their large surfaces can ideally be used as interactive displays for entertainment and communication. The BWB could accommodate many more passengers than today’s airplanes – all of which require personal attendance. Telepresence technology such as that developed in VR-HYPERSPACE could increase the availability of flight attendants, while requiring less on-board personnel. The virtual air steward could be semi-automated, for example to provide consistent safety instructions in native languages simultaneously at multiple locations in the cabin. The actual crew members could remain on the ground and serve the passengers on demand through their on-board 3D-video avatars.
The physical structure of the BWB does not allow the integration of real windows in the passenger’s seating area however, light-field displays will be used to create wide-angle views at the airplane’s surrounding during the flight and as demonstrated in VR-HYPERSPACE, the whole cabin can be considered as a display that could provide virtual windows or a transparent exterior. The interactive displays can also be used to explore the flight’s destination ahead of arrival and passengers may plan their visit together or use their displays for individual purposes.
The large extent of the BWB air cabin will allow passengers to walk around conveniently for amusement and recreation. The contents displayed at the support columns could be used to create engaging virtual environments, or they may simply be used for advertising and shopping opportunities. VR-HYPERSPACE also demonstrated how telepresence technology may be used on board to stay in contact with family, friends and colleagues - offering an extremely direct and perceptually rich form of staying in contact – even during long-haul flights. In addition, 3D-telepresence technology can enable collaborative working while in transit - groups could meet for online business meetings and jointly explore environments or discuss 3D objects, which are real or digital models. The airplane cabin can effectively become a flying office space for nomadic workers.
Enabling ancillary revenue
Several of the concepts explored through VR-HYPERSPACE form the basis for a series of services provided on-board in the future air cabin as suggested in our final VR-HYPERSPACE Roadmap (Frangakis et al, 2014). Within the aviation industry, such services are expected to grow at a Compound Annual Growth Rate (CAGR) of 6.67% during the period 2012-2017. Currently in-flight services encompass high quality audio and video services (also on-demand), multi- or single player gaming , while wireless entertainment and internet capabilities are gradually being introduced in some airlines. Over the past few years, several breakthrough technologies such as tablets, seat-centric systems, and wireless services have entered the market, transforming the in-flight passenger experience. Through the introduction of VR-HYPERSPACE concepts, airlines could increase their ancillary revenue opportunities by increasing the innovative products and services available on-board.
Improving the passenger experience
VR-HYPERSPACE has shown that passenger experience is perceived differently by each passenger (e.g. some passengers like to escape the cabin environment if they experience flight anxiety, or wish for a quieter and more peaceful environment while others would like to bring back the excitement of flying). We envision that passengers will be able to customize their experience by choosing between a variety of different scenes like beaches, mountains or desert nights with star filled skies, providing every passenger with a unique experience, tailored to their specific preferences. VR-HYPERSPACE has shown that virtual worlds can positively change a person’s perception, resulting in a feeling of spaciousness and a relaxed body. Other passengers might like to make productive use of the transit time or stay in contact with others in order to enhance their flight experience. VR-HYPERSPACE can assist, especially during long-haul flights, in enhancing the experience and productive time spent in transit and maintaining strong connectivity with family, friends and colleagues on board and on the ground. The economic viability of all possible implementations is a focus of future thorough research.
Cabin crew efficiency
Cabin crews carry a number of responsibilities to ensure passengers’ safety and well-being. VR-HYPERSPACE has the potential to support cabin crew on and off the plane. Currently offered on-board services include: sitting assistance, flight information, safety procedures demonstration, destination related information, newspaper and magazine delivery and services related to passengers’ refreshment needs. VR-HYPERSPACE has shown that using telepresence technologies could increase the quality of the service by enabling virtual crew on-board to provide personalized assistance related to information and safety. Further research and development is required to make significant advances to these services.
The wider societal implications
By 2050, the world population may reach nine billion people, and two fifths of that population will be over 50 years old . The aging population is increasingly travelling by air because of increased affluence, cheaper airfares and demographic changes , and older people have specific needs with regard to their health and comfort. In addition, flying is no longer a luxury reserved for holidays: it has become part of our business mobility. The aforementioned imply that more time will be spent on-board and by groups of people for which personalised services would be of importance. VR-HYPERSPACE has shown how VR/AR technologies can be used to provide shared and personalised interactive experiences to increase the efficiency and collaboration of working while in transit, and better connectivity with family and friends for leisure and business activities.
In addition, VR has already shown potential benefits for addressing a number of fears (for example spider or height phobias). Regarding flights it is a fact that fear of flying appears in different levels to approximately 10-25% of people that are flying . Different sources suggest that the fear of flying (Aerophobia) affects either between 1 in 3 or 1 in 5 people . The main reasons have been grouped roughly into several areas of which the key ones are: turbulence, enclosed spaces, crashing and lack of control. VR-HYPERSPACE has shown that we can provide positive illusions to address turbulence, extend space and provide real-time information to give back the control to the passenger and therefore has the potential to help people overcome aerophobia. If proved successful, this could increase the number of people selecting to fly and will greatly enhance passenger experience.
Main dissemination and exploitation of results
VR-HYPERSPACE had ambitious targets in terms of innovative research results; as such this led to a highly successful and prolific dissemination strategy. A description of all activities can be found in Amditis, Georgiou & Brousta (2014) . Highlights include the following.
Public Website
The VR-HYPERSPACE website www.vr-hyperspace.eu was launched in the very beginning of the project and will be sustained for at least five years after the end of the project in order to provide all interested stakeholders with information on project achievements and results and details on contact persons (see . Improvements to the project’s website were constantly proposed and implemented. Important changes included for example: increasing the number of videos; addition of a press announcement section; addition of a LinkedIn registration button; newsletter registration button; and latest projects news clearly visible on the front page.
Poster, Leaflet and Project Newsletters
The design of the poster and leaflet was based upon the vision of the cabin 2050 and beyond, including a flavour of VR/MR technologies that could be employed. The whole idea behind the leaflet was to give a glimpse regarding the 2050 airplane cabin in a futuristic yet convincing way. The project leaflet was distributed at 12 major events including conferences, seminars and tradeshows such as Farnborough Airshow, 2014.
A newsletter had 5 editions and summarised all project news and achievements for each reporting period. The newsletter was a means to address a wider audience and the general public. The newsletter targeted both expert and non-expert audiences and distributed via our LinkedIn site, the euroVR Association mailing list and other contacts.
Paper presentations, panels or demonstration sessions
VR-HYPERSPACE consortium participated in 16 paper presentations, panels and/or demonstration sessions at key events including: the Joint Virtual Reality Conference EGVE-ICAT-EuroVR, JVRC (2011, Nottingham, UK; 2012 Madrid, Spain; 2013 Paris, France; 2014 Bremen, Germany); IEEE VR (2013, Orlando, Florida; 2014, Minneapolis, Minnesota; Laval Virtual Conference (2012, 2014, Laval, France); 17th ACM Conference on Computer supported cooperative work & social computing – CSCW 2014, Baltimore, Massachusetts. Specific highlights are as follows.
IEEEVR2014, Minneapolis, Minnesota on 29th March - 2nd April, 2014 - VR-HYPERSPACE participated as part of the demonstration session, panel session and video session. As the largest and key conference for the VR scientific community, the project was able to reach over 350 participants (D’Cruz et al. 2014a). It was also a great opportunity to deliver and demonstrate our concept prototype results and to elicit feedback from the participants at the conference (D’Cruz et al. 2014b). During the three days of the conference more than 200 people passed by the demonstration booth. In addition the keynote of the conference Henry Fuchs provided the VR-Hyperspace partners with highly positive feedback when visiting the project’s stand.
AIRBUS Workshop, 6th November, 2013, Hamburg, Germany - Held at the Airbus facilities in Hamburg, a wide spread list of over 50 participants from the Cabin Innovation Center, Human Factors and Cabin Engineering departments, EADS Global Innovation Network and Eurocopter participated (Olbert et al., 2014). It was felt by the consortium and participants that the whole event was highly positive and interesting exchanges were made regarding views and expectations around the VR-HYPERSPACE concepts and the future of air cabin designs.
Comfort in Transit Symposium, 2014, Delft, The Netherlands - VR-HYPERSPACE were invited to present and demonstrate at the “Comfort in Transit” Symposium which was organized in Delft, the Netherlands. The symposium brought together a number of experts to discuss recent advances and trends in the fields of aviation, aircraft manufacturing, airport and airline operation and identify the major challenges ahead and discuss possible solutions and ideas. VR-HYPERSPACE concepts and results were well received especially by the industry participants including Airbus, Boeing and Bombardier.
VR-HYPERSPACE Press event - A press event in Germany, hosted by our partners MPG and FhG-IAO on 5th – 6th February, 2014. The purpose was to introduce the project goals and achieved results to the wider public through extended press coverage. Both local and international media participated in the event giving a transnational coverage. Press included journalists from newspapers and magazines including The UK Times newspaper, German radio and BBC TV. The project was shown on the BBC National news http://www.bbc.co.uk/news/technology-26096761 and following the report we were on several regional TV and radio channels with many international publications picking up the story. Given that the UK viewing figures for BBC news per show is around 28 million viewers and we were on for 2 days, the coverage could have exceeded a billion viewers worldwide.
Youtube channel “Computerphile - “Computerphile” is an on-line Youtube educational channel aimed at computer enthusiasts with over 300,000 subscribers but generally millions of viewers. Several of the consortia were interviewed with the following videos related to VR-HYPERSPACE posted during the project:
• In Flight Virtual Reality, June 2014, (http://www.youtube.com/watch?v=PJ5LJkpwBAI)
• Real Life Holodeck, July 2014 http://www.youtube.com/watch?v=7ZPs7knvs7M
• The (pink) VR Simulator, August 2014, http://www.youtube.com/watch?v=Lm0lA0enPSk
• Avatars & In-flight VR, August 2014, http://www.youtube.com/watch?v=TLKqKlrQv4s
• A New Perspective (Multi-Person 3D system), November 2014, https://www.youtube.com/watch?v=qnnEIrWQJdo
Publications in journals and conference proceedings - A major effort towards publishing scientific papers to well respected and highly rated journals had been deployed. This was considered very important in order to reach the scientific target group. Reflecting the scope of VR-HYPERSPACE, there were several scientific fields that would benefit from the project experiences and results. By the close of the project VR-HYPERSPACE partners have had 16 publications, including: eight journal papers (Piryankova et al. (2013); Linkenauger et al. (2013); Kilteni, Bergstrom & Slater (2013); Beck et al. (2013); Kokkinara & Slater, (2013); Aromaa, Viitaniemi & Helin (2012), Saulton et al. (2013); Slater & Sanchez-Vives (2014); and eight papers in conferences proceedings (Kunert et al. (2014); Psonis, Frangakis & Karaseitanidis (2012); Karaseitanidis et al. (2013); D’Cruz & Patel (2013); Slater (2013); Bues et al. (2013); Beck et al. (2013); D’Cruz et al. (2014)).
More publications are expected after the end of the project (three more papers have been submitted already to journals for review and others will be submitted in the near future). In general, the project publication work has been impressive and has reached a level well above the average of a three year European project. This is clear indication of the high quality of both the consortium partners and project’s work. In the list below one can find the existing project publications in journals and conferences.
Furthermore, during the whole duration of the project more than 20 electronic articles and more than 10 videos related to the VR-HYPERSPACE work have been released, e.g. Drummers drum smarter with a laid-back avatar, New Scientist Magazine, March 2013, http://www.newscientist.com; Virtual traveller: Beam a live, 3D you into the world, , New Scientist Magazine, April 2013, http://www.newscientist.com; Flying in virtual reality: Counterfeit comfort, GO! Exploring the world of transportation, June 2013, http://www.go-explore-trans.org/; VR-HYPERSPACE – Flugkomfort in Virtueller Realität, Innovations report , February 2014, http://www.innovations-report.de; Virtual-reality 'assisted' flying? Helping airline passengers experience comfort, space and altered self-perception, Science Daily, Science Daily, http://www.sciencedaily.com; Passagiere fliegen virtuell durch traumhafte Lüfte, DIE WELT, April 2014, http://www.welt.de; Forschungsprojekt: Virtuelle Realität soll für mehr Flugkomfort sorgen, Konstruktions praxis, February 2014, http://www.konstruktionspraxis.vogel.de
Social Media - In the final year the project created a LinkedIn group which currently has more than 90 members so far. This enabled updates and final questionnaires to be distributed and will continue beyond the project as the consortium continues to develop ideas in this area.
References
Aaltonen, L., Aromaa, S., Beck, S., Bergstrom, I., D'Cruz, M., Frangakis, N., Fröhlich, B., Helin, K., Kulik, A., Kunert, A., Lewis, L., Leyrer, M., Mohler, B., Patel, H., Soyka, F., Tedone, D., Viitaniemi, J. (2014) Final user evaluation report. Public Report D5.4 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
ACARE (2010) Aeronautics and air transport: beyond vision 2020 (towards 2050). http://www.acare4europe.com/
Amditis, A., Georgiou, N., and Brousta, E. (2014) Dissemination manual, Month 36. Public Deliverable D6.3.3 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Aromaa, S., Viitaniemi, J., Helin, K. (2012). Virtual Reality Systems Specifications for Human Machine Interaction Design in Industry. Work: A Journal of Prevention, Assessment and Rehabilitation
Beck, S. and Kulik, A. (2013) Acquisition and virtual representation of passengers – technical set up. Confidential Prototype D3.3 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Beck, S. et al. (2013). Airborne Avatars - Inflight Immersive Telepresence. Joint Virtual Reality Conference EGVE-ICAT-EuroVR, JVRC 2013, Paris, France
Beck, S., Kunert, A. and Kulik, A. (2014) Virtual Prototype. Confidential Prototype D4.2 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Beck, S., Kunert, A., Kulik, A. and Froehlich, B. (2013) Immersive Group-to-Group Telepresence. IEEE Transactions on Visualization and Computer Graphics 19, 4 (April 2013), 616-625.
Beck, S., Kunert, A., Kulik, A., Froehlich B. (2013) Immersive Group-to-Group Telepresence
IEEE Transactions on Visualization and Computer Graphics, 19(4):616-25, March 2013 (Proceedings of IEEE Virtual Reality 2013, Orlando, Florida).
Bergstrom, I. (2013) Self-Perception system – technical set up. Confidential Prototype D3.1 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Bues, M., et al. (2013). Enhancing the flight experience: The Transparent Seat and beyond. Joint Virtual Reality Conference EGVE-ICAT-EuroVR, JVRC 2013, Paris, France
Cappitelli, M., Kulik, A. and Bues, M. (2012) Report of future light, display and new media technologies interior spaces of airborne vehicles. Restricted Deliverable D1.2 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
D’Cruz M., Patel H., Lewis L., Cobb S., Bues M., Stefani O., Grobler T., Helin K., Viitaniemi J., Aromaa S., Frohlich B., Beck S., Kunert A., Kulik A., Karaseitanidis I., Psonis P., Frangakis N., Slater M., Bergstrom I., Kilteni K., Kokkinara E., Mohler B.J. Leyrer M., Soyka F., Gaia E., Tedone D., Olbert M. und Cappitelli M. (2014b). VR-HYPERSPACE—The innovative use of virtual reality to increase comfort by changing the perception of self and space. IEEE Virtual Reality (VR 2014), Minneapolis, Minnesota
D’Cruz, M. & Patel, H. (2013) VR-HYPERSPACE: Exploring new ways of enhancing comfort in future air travel using Virtual Reality (VR) and Mixed Reality (MR). Joint Virtual Reality Conference EGVE-ICAT-EuroVR, JVRC 2013, Paris, France
D’Cruz, M., Patel, H., Lewis, L. and Cobb, S. (2014a) Feedback on in-flight applications of virtual reality to enhance comfort in future aircraft. EuroVR 2014, Bremen, Germany, 8-10th December, 2014
Frangakis, N., Karaseitanidis, G., D’Cruz, M., Patel, H., Mohler, B., Bues, M. and Helin, K. (2014) VR-HYPERSPACE Research Roadmap. Public Deliverable D6.4 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Karaseitanidis, G. (2013) Studying the effects of relaxation and stress on the feeling of presence within a VE. Joint Virtual Reality Conference EGVE-ICAT-EuroVR, JVRC 2013, Paris, France
Kilteni, K., Bergstrom, I. and Slater, M. (2013). Drumming in immersive virtual reality: the body shapes the way we play. IEEE Transactions on Visualization and Computer Graphics, 19(4), 597 - 605.
Kilteni, K., Kokkinara, E., Bergstrom, I., López-Moliner, J., Slater, M., Dodds, T., Leyrer, M. Saulton, A., Viitaniemi, J., Helin, K., Aromaa, S., Beck, S., Kunert, A., Kulik. A., Patel, H., Lewis, L., D’Cruz, M., Cobb, S., Karaseitanidis, G., Psonis, P., Frangakis, N., Stefani, O., Bues, M. (2013) Experimental input for integration into prototype. Confidential Deliverable D2.2 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Kokkinara, E. and Slater, M. (2014). Measuring the effects through time of the influence of visuomotor and visuotactile synchronous stimulation on a virtual body ownership illusion. Perception, 43(1), 43–58.
Kokkinara, E., Bergstrom, I., Kilteni, K., López-Moliner, J., Slater, M., Dodds, T., Leyrer, M., Saulton, A., Mohler, B., Viitaniemi, J., Helin, K., Aromaa, S., Beck, S., Kunert, A., Kulik, A., Patel, H., Lewis, L., D’Cruz, M., Cobb, S., Karaseitanidis, G., Psonis, P., Frangakis, N., Stefani, O., Bues, M. (2013) Final overall report of experimental results. Confidential Deliverable D2.3 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Kokkinara, E., Leyrer, M. and Saulton, A. (2012) Literature Review of self embodiment and extended volume space. Public Deliverable D2.1 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Kokkinara, E., Slater, M. and López-Moliner, J. (under review) Spatiotemporal and spatial distortions of arm movements in immersive virtual reality alter judgments of the space around us. ACM TAP
Kulik A., Kunert A., Beck S., Reichel R., Blach R., Zink A., Froehlich B. (2011) C1x6: A Stereoscopic Six-User Display for Co-located Collaboration in Shared Virtual Environments. ACM Transactions on Graphics 30, 6, Article 188 (December 2011)
Kulik, A., Kunert, A., Beck, S., Reichel, R., Blach, R., Zink, A. and Froehlich, B. (2011) C1x6: a stereoscopic six-user display for co-located collaboration in shared virtual environments. In Proceedings of the 2011 SIGGRAPH Asia Conference (SA '11). ACM, New York, NY, USA, Article 188, 12 pages
Kunert, A., Beck, S., Kulik, A., Fröhlich, B. (2013) 3D models of a future airplane and various interior parts. Confidential Prototype D4.1 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Kunert, A., Kulik, A., Beck, S. and Froehlich, B. (2014) Photoportals: shared references in space and time. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing (CSCW '14). ACM, New York, NY, USA, 1388-1399.
Lewis, L. (in preparation) Investigating the ways in which VR can be used to enhance passenger comfort and experience on future aircraft. PhD thesis, The University of Nottingham, UK
Lewis, L., Patel, H., Cobb, S., D’Cruz, M. (under review) Understanding in-flight personal space invasions.
Leyrer, M. (2014) Understanding and Manipulating Eye Height to Change the User's Experience of Perceived Space in Virtual Reality. PhD Thesis, Max Planck Institute of Biological Cybernetics.
Leyrer, M., Linkenauger, SA., Bülthoff, HH and Mohler, B. (2014b). Eye height manipulations: A possible solution to counter underestimation of egocentric distances in head-mounted displays. ACM Transactions on Applied Perception.
Leyrer, M., Linkenauger, SA., Bülthoff, HH. and Mohler, B. (2014a). Eye height for perceiving egocentric distances can be determined by non-visual, body-based cues. PLOS ONE.
Linkenauger SA., Leyrer M., Bülthoff HH., Mohler BJ. (2013) Welcome to Wonderland: The Influence of the Size and Shape of a Virtual Hand on the Perceived Size and Shape of Virtual Objects. PLoS ONE 8(7).
Mohler, B. (2014) Ethical Issues Report. Confidential Report D7.3 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Mohler, B., and Leyrer, M. (2013) Volume system – technical set up. Confidential Prototype D3.2 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Olbert, M., Cappitelli, M, Tedone, D., and Gaia, E. (2013b) Feasibility study of implementation of initial concepts and technologies in real aircraft. Restricted Report D5.2 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Olbert, M., Cappitelli, M., D’Cruz, M. and Tedone, D. (2012) AAB Annual Report. Public Deliverable D6.2 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Olbert, M., Cappitelli, M., D’Cruz, M. and Tedone, D. (2013a) Second AAB Annual Report. Public Deliverable D6.2.2 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Olbert, M., Cappitelli, M., D’Cruz, M. and Tedone, D. (2014) Third AAB Annual Report. Public Deliverable D6.2.3 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Patel, H. and D’Cruz, M. (under review) Factors influencing the experience of aircraft passengers’ comfort. Transportation Research Part A. Policy and Practice.
Patel, H. and Lewis, L. (under review) In-flight experiences of comfort and discomfort – it’s not just about the seat. Applied Ergonomics.
Patel, H., Lewis, L., Cobb, S., D’Cruz, M, Tedone, D., Hakulinen, J. (2012) Report of current scenarios and case definition. Public Deliverable D1.1 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Piryankova, I. V., Wong, H. Y.,Linkenauger, S., Stinson, C Longo, M., Buelthoff, H. H., and Mohler, B. J. 2014. Owning an overweight or underweight body: distinguishing the physical, the experienced and the virtual body. PLoS ONE (August)
Piryankova, IV., de la Rosa, S., Kloos, U., Bülthoff, HH. and Mohler, BJ. (2013) Egocentric distance perception in large screen immersive displays. Displays, 34 (2), 153 - 164.
Piscataway, NJ, USA, 167-168
Psonis, P., Frangakis, N., Karaseitanidis, G. (2012). Using airplane seats as front projection screens. Joint Virtual Reality Conference EGVE-ICAT-EuroVR, JVRC 2012 Madrid, Spain
Saulton, A., Dodds, T., Tesch, J., Mohler, B. and Bȕlthoff, HH. (2013) The influence of shape and culture on visual volume perception of virtual rooms. ACM SAP 142.
Slater, M. & Sanchez-Vives, MV. (2014) Transcending the Self in Immersive Virtual Reality IEEE Computer 47: 7. 24-30 July.
Slater, M. (2013). Fooling the brain about the body using immersive virtual reality. Joint Virtual Reality Conference EGVE-ICAT-EuroVR, JVRC 2013, Paris, France
Soyka, F., Mohler, B. and Kokkinara, E. (2014) Integration of virtual reality equipment and mobile devices into MPS motion simulator. Confidential Prototype D4.4 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Stefani, O., Grobler, T. and Bues, M. (2014) Physical prototype. Confidential Prototype D4.3 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Stefani, O., Haselberger, F., Grobler, T. and Bues, M. (2013) Cabin environment – technical set up. Confidential Prototype D3.4 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
Tedone, D., Patel, H. and D’Cruz, M. (2012) Report of future scenarios and case definition. Restricted Deliverable D1.1 of the FP7 VR-HYPERSPACE (AAT-285681) project (www.vr-hyperspace.eu)
List of Websites:
http://www.vr-hyperspace.eu
Dr. Mirabelle D'Cruz (Mirabelle.dcruz@nottingham.ac.uk)
