Major computing challenge completed
The EU-funded DILIGENT (test bed digital library infrastructure on grid enabled technology) project has enabled one of the world's largest collections of multimedia metadata to be available to all for research purposes after completing a data challenge on image feature extraction. During the data challenge, 37 million images from the public online photo database Flickr were processed in 16 weeks using the Enabling Grids for E-sciencE (EGEE) computing grid, a project co-funded by the European Commission that provides computing resources to academia and industry. Reaching an average processing capacity of 300,000 images per day, nearly five terabytes of data - 112 million text and image objects and more than 150 million extracted features - were generated in the process. In order to extract the data from the Flickr database, researchers combined a Java application with Perl scripts and a C application: The Java application helped to contact the database, download a set of Flickr users and the images those users were sharing. The Perl scripts and C application then extracted features from the images, created thumbnails and stored the results. The data retrieved will now be used by the EU project SAPIR, which aims at developing web search architectures that enable a user to search audiovisual content by example. For instance, an image taken by a cell phone would then make it possible to find information about the monument the photographer came across while sightseeing, or singing a melody could be used to search for a full song. Funded under the Sixth Framework Programme (FP6), it was the main objective of the DILIGENT project to create an advanced test bed that would allow members of virtual e-science organisations to facilitate knowledge sharing and collaboration in a cost-effective way. The total project cost amounted to over €9 million, two thirds of which were borne by the EU. Some 14 partners from France, Italy, Switzerland, Austria, the UK, Hungary, Greece, Norway and Germany were involved.