Feature Stories - Beam me to my meeting!
Teleconferences, video conferencing and web exchanges have become the norm for most multinational businesses. But nobody really likes it; nothing beats meeting people face-to-face. 'When we meet people in the flesh we can pick up on subtle cues - facial expressions, quirks, who is looking at who,' explains Stephen Dunne from Starlab in Spain. 'There is so much non-verbal communication that you miss, even with the highest quality video conferencing technology. You can't shake hands or decide to look around the room, for example.' Working in an EU-funded research project, scientists from across Europe have now combined a set of digital technologies that could effectively transport someone into a meeting room on the other side of the world. For the first time ever people have been able to shake hands on a deal, separated by thousands of miles. Of course there is no teleportation involved, but the 'Beaming through augmented media for natural networked gatherings' (Beaming) project uses immersive virtual reality techniques and technologies to make you feel like you are actually somewhere else. In that 'somewhere else', you are embodied by your avatar - a real-life robot, for example - which becomes your eyes, ears and mouth. As the Beaming 'traveller' you are rigged up with multiple sensors and have your head covered in a full head-mounted display. Yet that display means you see and hear what your robotic-self can see through its video camera eyes and microphone ears - and you can respond just as you would if you were really there. If you move your head, the robot's head moves the same way; if you speak, the robot talks with your voice. Movement sensors on your arms detect how they move and the robot's arms match these actions. And if someone touches the robot's hands, this pressure is relayed back to pressure pads on your hand. The technology has been demonstrated in a groundbreaking interview between a project scientist in Spain and a BBC journalist in University College London. The journalist was able to interview the scientist and even 'high five' them at the end of the demonstration (see video link below). Today's technology, tomorrow's market? 'What we have achieved here is a real demonstration of what can be done with the technologies we have available to us today,' remarks Mr Dunne, project manager for Beaming. 'We took "off-the-shelf" products and combined them in this unusual way. This is not a breakthrough in technology development, but is certainly a breakthrough in demonstrating the powerful ways in which media can be networked to achieve amazing things.' One of the biggest tasks for the Beaming partners has been to develop a framework data architecture for the system; this defines how all the visual, audio, motion and pressure data is packaged and transmitted between the traveller and their remote environment. It also establishes how the 3D model of the remote location is generated for the 'traveller' to create a strong sense of presence for them. 'The purpose of the framework is to make Beaming entirely independent of the hardware or software involved,' explains Mr Dunne. 'You'll be able to use any robot or any sensor, for example. We have also tried to define the minimum amount of data you need but still make the Beaming experience fully immersive.' Emotional clues Although head-mounted displays, pressure pads and 3D graphical interfaces can give you a real sense of being elsewhere, what is it like for people at the other end, those sitting around a table, for example, having to interact with your avatar? Talking to a robot who talks like you could be quite unnerving! 'It was important for us to make the interaction as natural as possible. Whether people are interacting with a robot or some kind of virtual avatar, we want the experience to feel natural and not get in the way of normal communication,' says Mr Dunne. One simple improvement was to give the robot a more expressive head. The team replaced its robot's mechanical eyes and mouth with an LCD display. 'We used graphics for the eyes and mouth; they look much more natural. People seem to relax more if they can look a robot in the eye and the eyes look right. It is easier to forget that they are dealing with a machine!' says Mr Dunne. Research still continues to investigate how data about the Beaming traveller's physiological and emotional state could be recorded and relayed to the people in the remote location. The project has experimented to see how the traveller's heart rate, facial expressions and even brain waves could be clues to their emotional state. So far, technology can be trained to use these signals to recognise basic emotional states, for example differentiate between relaxed and stress. The challenge now is to work out how to communicate their signals to everyone else through the avatar. Is travel a thing of the past? 'We are not claiming that Beaming will revolutionise meetings or save the world millions of tonnes of carbon dioxide because people won't need to travel anymore,' Mr Dunne stresses, 'but we do think this idea of "beaming" individuals to locations where they can move and interact through a robot could really work for some very specific applications initially.' 'It is ideal if you want to transport a person with specific skills or knowledge to another location,' Mr Dunne continues. 'It gives that person total physical immersion in their remote location. They can touch and interact with their remote environment. We think it might work for top surgeons who could "beam" into operating theatres all over the world and share their expertise and knowledge, even perform operations. 'As a project, our inspiration was strictly to demonstrate that this kind of virtual travel and interaction could be done. Now it is the job of individual partners to translate this into the real world and real products. We are moving from the realm of scientific curiosity and looking at services we could offer.' The Beaming project received EUR 9.2 million (of total EUR 12.4 million project budget) in research funding under the EU's Seventh Framework Programme (FP7). Links to projects on CORDIS: - FP7 on CORDIS - Beaming project factsheet on CORDIS Link to project's website: - 'Beaming through augmented media for natural networked gatherings' website Links to related videos/audios: - Robot avatar beaming put to the test, BBC - Real-world beaming: The risk of avatar and robot crime, BBC - Beaming project videos Other links: - European Commission's Digital Agenda website