EU scientists make virtual reality touchable
It sounds like science fiction, but a team of European researchers has 'virtually' teleported real objects through cyberspace, 'touched' virtual reality (VR) and even felt the movements of a virtual dance partner thanks to advances in haptic technology and a new approach to generating VR content. EU support for the research came from the IMMERSENCE ('Immersive multi-modal interactive presence') project, which received EUR 5.5 million from the 'information society technologies' (IST) Thematic area of the Sixth Framework Programme (FP6). The scientists built on advances in haptic technology - technology that interfaces with the user through the sense of touch - and a new approach for generating VR content to create virtual experiences that are far more realistic and immersive than anything ever achieved before. In addition to seeing and hearing their virtual surroundings, objects and avatars, users can now touch them, paving the way for new applications in telepresence, telemedicine, industrial design, gaming and entertainment. 'The audiovisual aspects of VR have come a long way in recent years, so adding a sense of touch is the next step,' said Andreas Schweinberger, a researcher at Technische Universität München in Germany and coordinator of IMMERSENCE. 'We know that the more senses that can be used, the more interaction, the greater the sense of presence. And a stronger sense of presence means the experience is more immersive and realistic.' Mr Schweinberger led a team from nine universities and research groups across Europe to create technology that would make VR objects and characters touchable. They developed innovative haptic and multi-modal interfaces, new signal processing techniques and a pioneering method to generate VR objects from real-world objects in real time. The latter technology, developed at the Computer Vision Laboratory of the Eidgenoessische Technische Hochschule (ETH) Zürich in Switzerland, uses a 3D (three-dimensional) scanner and advanced modelling system to create a virtual representation of a real object, which can then be transmitted to someone at a remote location, who, by wearing VR goggles and touching a haptic interface, can move, prod and poke it. 'Haptic technology is still in the early stages,' Mr Schweinberger explained. 'For the haptic interface, we used a robotic arm called a PHANTOM that has one contact point. This gives the sense of touching an object, but you can't pick it up or handle it.' However, he added that researchers at the Universidad Politécnica in Madrid in Spain are 'developing a haptic device with two contact points that should make it possible to grasp an object with a virtual hand'. Meanwhile, scientists at Technische Universität München focused on enabling human-to-human interaction in a virtual environment. They used a mobile robotic platform as the dance partner for a real human dancer. By wearing VR goggles, the user could see a dancer of the opposite sex and dance with them by holding the robot's hands, according to Mr Schweinberger. Researchers at the University of Evry near Paris in France went one step further, studying how to virtually create the sensation of two people handling an object. The researchers said gamers would be 'delighted' by the developments and insisted that the technology could also be used for more serious applications, allowing doctors, for example, to treat patients remotely, or industrial designers to collaborate remotely by virtually teleporting touchable digital mock-ups of designs over the Internet. 'The research will also help in the development of cognitive robots that are better able to interact with humans,' said Mr Schweinberger. His team will continue to research this aspect of the project, while ETH Zürich will develop its virtual teleporter. Other partners will continue their work under the EU-funded BEAMING '(Beaming through augmented media for natural networked gatherings') project, which received EUR 9.2 million from the 'Information and communication technologies' (ICT) Thematic area of the Seventh Framework Programme (FP7).