Final Report Summary - SOMIR (Semantic Oriented Multimedia Indexation and Retrieval)
Mihaela Brut (Mihaela.Brut@irit.fr) was the principal investigator of the SOMIR project. The main research results are presented at http://www.irit.fr/~Mihaela.Brut/somir/index.php.
The multimedia information retrieval problem relies upon providing the most relevant results for a user's query executed over a multimedia content collection, that is generally heterogeneous and distributed. The efficient content indexation is a key issue for the management and retrieval of relevant information. Indexing is based on a set of algorithms, which generate diverse and heterogeneous multimedia metadata.
The SOMIR project assumed the challenge to employ a semantic-oriented and interoperable modelling of the three main actors: the multimedia metadata, the indexing algorithms' descriptions, as well as the user queries, in order to convey more efficiently the information retrieval process. This effort involved the harmonisation and integration of multiple metadata formats, both XML and RDF based, the definition of generic interfaces and annotation models for indexing algorithms, together with establishment of formal expressions for the queries. Supplementary effort was accomplished for developing methods to automatically obtain ontology-based metadata for textual documents, to convert and uniformly integrate the metadata generated by the multimedia indexing algorithms, as well as to formally transform the user query through a natural language-based and Wordnet-based processing.
Three issues were considered for improving the relevance of the query results:
- query enhancement and / or rewriting;
- relevant algorithms selection in order to accomplish an explicit indexation, tailored to the current user query;
- algorithms sequences definition in order to be successively applied for indexing the multimedia content in case of the user complex queries.
For gaining a user-centred dimension, our approach defined and developed a user profile accumulating the history of his queries. The profile was considered for refining the user query's results and to provide him with personalised recommendations.
The work in the SOMIR project provided us a lot of collaboration and personal development opportunities.
The research results were published in18 articles: 3 journal articles, 1 book chapter, 14 conference papers. Among these, seven are published by IEEE Computer Society, five by Springer, one by IGI Global, one by Elsevier, one by Inderscience Pubishers.
As illustrated by these publications, Mihaela Brut had the opportunity to directly collaborate with nine consecrated researchers, with a post-doc researcher and with five PhD students. Based on these collaborations, she explored and attacked new research ideas contributing to the general project theme, while developing her personal approach that consist in adopting ontologies to model multiple resource types: users, documents, services and to develop different personalised functionalities. She developed a technique to extract information from multiple sources and uniformly handle it through the ontology-based modelling support. She adopted new research's application areas (health, aerospace, video surveillance).
She benefitted by numerous training activities (summer schools, professional days, thematic days, symposiums, language courses).
- Mihaela Brut gave multiple presentations inside the hosting IRIT laboratory, as well as seven invited talks in research groups from different universities.
- She participated in the writing and the development of four project proposals (two FP7, one ECOS-Nord, one ANR) that brought her collaboration with partners from five universities, four research centres, and 10 industry companies.
- She wanted also to have a teaching experience, giving an object oriented programming course and some practical works.
- In 2009, she obtained the qualification for the Associate Professor position in the 27th section (computer science) of the National Council of Universities.
The multimedia information retrieval problem relies upon providing the most relevant results for a user's query executed over a multimedia content collection, that is generally heterogeneous and distributed. The efficient content indexation is a key issue for the management and retrieval of relevant information. Indexing is based on a set of algorithms, which generate diverse and heterogeneous multimedia metadata.
The SOMIR project assumed the challenge to employ a semantic-oriented and interoperable modelling of the three main actors: the multimedia metadata, the indexing algorithms' descriptions, as well as the user queries, in order to convey more efficiently the information retrieval process. This effort involved the harmonisation and integration of multiple metadata formats, both XML and RDF based, the definition of generic interfaces and annotation models for indexing algorithms, together with establishment of formal expressions for the queries. Supplementary effort was accomplished for developing methods to automatically obtain ontology-based metadata for textual documents, to convert and uniformly integrate the metadata generated by the multimedia indexing algorithms, as well as to formally transform the user query through a natural language-based and Wordnet-based processing.
Three issues were considered for improving the relevance of the query results:
- query enhancement and / or rewriting;
- relevant algorithms selection in order to accomplish an explicit indexation, tailored to the current user query;
- algorithms sequences definition in order to be successively applied for indexing the multimedia content in case of the user complex queries.
For gaining a user-centred dimension, our approach defined and developed a user profile accumulating the history of his queries. The profile was considered for refining the user query's results and to provide him with personalised recommendations.
The work in the SOMIR project provided us a lot of collaboration and personal development opportunities.
The research results were published in18 articles: 3 journal articles, 1 book chapter, 14 conference papers. Among these, seven are published by IEEE Computer Society, five by Springer, one by IGI Global, one by Elsevier, one by Inderscience Pubishers.
As illustrated by these publications, Mihaela Brut had the opportunity to directly collaborate with nine consecrated researchers, with a post-doc researcher and with five PhD students. Based on these collaborations, she explored and attacked new research ideas contributing to the general project theme, while developing her personal approach that consist in adopting ontologies to model multiple resource types: users, documents, services and to develop different personalised functionalities. She developed a technique to extract information from multiple sources and uniformly handle it through the ontology-based modelling support. She adopted new research's application areas (health, aerospace, video surveillance).
She benefitted by numerous training activities (summer schools, professional days, thematic days, symposiums, language courses).
- Mihaela Brut gave multiple presentations inside the hosting IRIT laboratory, as well as seven invited talks in research groups from different universities.
- She participated in the writing and the development of four project proposals (two FP7, one ECOS-Nord, one ANR) that brought her collaboration with partners from five universities, four research centres, and 10 industry companies.
- She wanted also to have a teaching experience, giving an object oriented programming course and some practical works.
- In 2009, she obtained the qualification for the Associate Professor position in the 27th section (computer science) of the National Council of Universities.