Periodic Report Summary - MIESON (Multimedia information extraction from social networks)
MIESON proposes a user-centric research approach, focused on leveraging technology to enhance information retrieval and user experience. The objective is to increase the effectiveness of tools to access, discover or create multimedia content in real-world applications.
So far, the project has covered the technological objectives set for the outgoing phase. Methods to learn statistical models from the combination of content and context information have been studied for the proposed scenario of modelling aesthetics. Emphasis has been placed in carrying out training activities in the topics of statistical data analysis, machine learning, and multimedia content analysis. A significant number of communication activities, including publications in scientific journals and conferences, talks and participation in panels have been carried out to disseminate the results generated within the scope of the project in the outgoing phase. Additionally, the fellow has attended a number of courses and tutorials aiming obtaining the skills required to complete the project objectives.
Three important results have been derived from this work. Firstly, a methodological approach has been proposed that leverages unstructured contextual information from user generated content (specifically, user comments) for revealing domain knowledge. The proposed method is application independent. We have used it to learn about aspects that influence human aesthetics perception. Secondly, a large-scale evaluation of the performance of content and context-based features for modelling the aesthetic value of photographs, which reveals a clear dominance of contextual information over content in terms of classification accuracy. This study also considers a unified content / context representation that outperforms both of the aforementioned approaches. Thirdly, a method to leverage aesthetic models to enhance image search results by incorporating aesthetics in the ranking model, which effectively extends relevance to improve search user experience as demonstrated by a user study.
Two scientific articles have been published so far, a journal article in ACM Transactions on Information Systems, a journal of reference in the Information Retrieval field with one of the highest impact factors in its field, and a full conference paper at ACM International World Wide Web Conference, WWW'11. Two additional articles have been submitted to conferences and have been accepted for publication during spring 2012 (ACM International World Wide Web Conference, WWW'12 and ACM Conference on 'Human factors in computing', CHI'12). All these conferences are top-tier scientific events and are extremely competitive in terms of acceptance rate.
Regarding the training objectives of the fellowship, the fellow has attended 6 talks by prestigious members of the fields related to MIESON (Multimedia information retrieval, cloud computing, HCI), as well as 5 tutorials and 4 courses that have provided him with the research skills defined in his career development plan. In addition to publications, the fellow has interacted with several research groups at the outgoing host institution, including the CiteSeer group led by Prof. Lee Giles and the DTRA group led by Prof. Frank Ritter, mentored 2 students from Prof. James Wangs group, gave several talks at both host institutions and was an invited panellist at the IST Symposium 2011. Also, the fellow attended several scientific research events, including ACM Conference on 'Human factors in computing systems' and ACM Conference on 'Multimedia' that provided him with networking opportunities.
So far, the project has covered the technological objectives set for the outgoing phase. Methods to learn statistical models from the combination of content and context information have been studied for the proposed scenario of modelling aesthetics. Emphasis has been placed in carrying out training activities in the topics of statistical data analysis, machine learning, and multimedia content analysis. A significant number of communication activities, including publications in scientific journals and conferences, talks and participation in panels have been carried out to disseminate the results generated within the scope of the project in the outgoing phase. Additionally, the fellow has attended a number of courses and tutorials aiming obtaining the skills required to complete the project objectives.
Three important results have been derived from this work. Firstly, a methodological approach has been proposed that leverages unstructured contextual information from user generated content (specifically, user comments) for revealing domain knowledge. The proposed method is application independent. We have used it to learn about aspects that influence human aesthetics perception. Secondly, a large-scale evaluation of the performance of content and context-based features for modelling the aesthetic value of photographs, which reveals a clear dominance of contextual information over content in terms of classification accuracy. This study also considers a unified content / context representation that outperforms both of the aforementioned approaches. Thirdly, a method to leverage aesthetic models to enhance image search results by incorporating aesthetics in the ranking model, which effectively extends relevance to improve search user experience as demonstrated by a user study.
Two scientific articles have been published so far, a journal article in ACM Transactions on Information Systems, a journal of reference in the Information Retrieval field with one of the highest impact factors in its field, and a full conference paper at ACM International World Wide Web Conference, WWW'11. Two additional articles have been submitted to conferences and have been accepted for publication during spring 2012 (ACM International World Wide Web Conference, WWW'12 and ACM Conference on 'Human factors in computing', CHI'12). All these conferences are top-tier scientific events and are extremely competitive in terms of acceptance rate.
Regarding the training objectives of the fellowship, the fellow has attended 6 talks by prestigious members of the fields related to MIESON (Multimedia information retrieval, cloud computing, HCI), as well as 5 tutorials and 4 courses that have provided him with the research skills defined in his career development plan. In addition to publications, the fellow has interacted with several research groups at the outgoing host institution, including the CiteSeer group led by Prof. Lee Giles and the DTRA group led by Prof. Frank Ritter, mentored 2 students from Prof. James Wangs group, gave several talks at both host institutions and was an invited panellist at the IST Symposium 2011. Also, the fellow attended several scientific research events, including ACM Conference on 'Human factors in computing systems' and ACM Conference on 'Multimedia' that provided him with networking opportunities.