Better robotics exploiting visual attention
Many approaches have been proposed, but achieving real-time processing of closed-loop perception-action cycles has necessitated severely simplified system models. This poses a restriction in terms of identifying sensory information of meaning to the task (sensory-semantic communication). Scientists developed novel methods to build and update a detailed 3D scene representation based on sensory information extracted from the scene in real time. With EU support of the project FASTDEFORM, researchers delivered a pioneering system that surpassed performance of state-of-the-art methods. FASTDEFORM exploited the massive parallelism of graphics processing units, programmable logic chips that perform rapid computations primarily for the purpose of rendering images, animations or video. Concepts based on visual attention enable focusing on scene items of interest at a level of abstraction supported by both real-time sensory and prior information. Researchers achieved significant enhancements in the speed, robustness and accuracy in perceiving and understanding a complex dynamic scene. This was accomplished through real-time integration of visual simulation and visual perception on the same hardware. The highly robust and real-time system operation was demonstrated in three different tasks. These covered manipulation of a large number of rigid objects, manipulation of a complex foldable cardboard brochure and vision-based robot control using feedback from a vision sensor. More information on each is available online. FASTDEFORM technology provides robotic systems with the ability to perceive and understand complex manipulations of non-rigid and dynamically changing objects. This enables them to move out of the realm of highly constrained industrial settings to novel applications in manufacturing, medicine and domestic environments. It has the potential to usher in a new era of devices exploiting human abilities in the support of human endeavours.
Keywords
Robotics, visual information, object manipulation, real-time processing, graphics processing unit
