Periodic Reporting for period 2 - TROMPA (Towards Richer Online Music Public-domain Archives)
Periodo di rendicontazione: 2019-05-01 al 2021-04-30
European cultural heritage enjoys considerable public interest. In the first half of 2017, the collective Europeana collections enjoyed around 1.2 million page views per month and 11 million views per month through Wikimedia projects. For classical music, we observe as well both considerable availability and engagement at large scale. Large volumes of scanned out-of-copyright scores and audio clips are reachable through online portals. Moreover, a wealth of musical interpretations. These resources are not only played, shared and contributed by professional experts, but also by amateur musicians. However, mass availability does not automatically imply mass accessibility. Resources in various modalities may relate to the same musical object, but are rarely presented in linked ways, and always ‘as is’, just presenting the digital files.
At the moment, most of the classical repertoire is in the public domain, massive numbers of scores and recordings are available in online community-contributed repositories, which are actively used by scholars and musicians. They offer opportunities for multimodal enrichment and contextualisation: making relevant associations across repositories and modalities, that will allow users to better understand and appreciate repository content. While digital technology has been proposed to support this, it is not yet deployed at scale in community-contributed repositories. Relevant connections across repositories and modalities therefore still are manually made by people with musical knowledge. Given the data deluge, they will likely miss out on interesting connections in doing this.
TROMPA general goal is to broaden and deepen the enrichment of online public-domain classical music resources. This will be done by engaging a broad audience of music-loving citizens, who will strategically cooperate with novel ICT technologies. Our target is to have unprecedented volumes of musical resources being enriched, explicitly including aspects of human interpretation at various skill levels. Our aim is to have a very strong component related to community involvement. Following a co-creation philosophy, TROMPA employs crowdsourcing mechanisms to connect the large audience of music-loving citizens more deeply with public-domain music resources, combining human knowledge with advanced music information retrieval technologies.
At the moment, most of the classical repertoire is in the public domain, massive numbers of scores and recordings are available in online community-contributed repositories, which are actively used by scholars and musicians. They offer opportunities for multimodal enrichment and contextualisation: making relevant associations across repositories and modalities, that will allow users to better understand and appreciate repository content. While digital technology has been proposed to support this, it is not yet deployed at scale in community-contributed repositories. Relevant connections across repositories and modalities therefore still are manually made by people with musical knowledge. Given the data deluge, they will likely miss out on interesting connections in doing this.
TROMPA general goal is to broaden and deepen the enrichment of online public-domain classical music resources. This will be done by engaging a broad audience of music-loving citizens, who will strategically cooperate with novel ICT technologies. Our target is to have unprecedented volumes of musical resources being enriched, explicitly including aspects of human interpretation at various skill levels. Our aim is to have a very strong component related to community involvement. Following a co-creation philosophy, TROMPA employs crowdsourcing mechanisms to connect the large audience of music-loving citizens more deeply with public-domain music resources, combining human knowledge with advanced music information retrieval technologies.
All work has been carried out following management procedures that were established at the beginning of the project. Organization and coordination of the actions amongst the consortium has been done effectively, including a Data Management Plan and ethical guidelines for all experiments requiring human participants.
During its first year, the TROMPA project has focused on defining its pilot scenarios for music scholars, orchestras, instruments players, singers and music enthusiasts (WP2). The TROMPA team has worked on the elicitation and prioritisation of the requirements for target user groups and use cases. Application mockups were developed based on the requirements elicited and tested with users of each target corresponding user group.
We worked on specifications in terms of automatic music processing technologies and human annotations and crowdsourcing technologies, including developing data-driven algorithms for automatic music description (in terms of rhythm, melody, emotion, etc) and processing (including singing synthesis and source separation). We investigated current state-of-the-art technologies that are relevant in the scope of the project defined pilots, we adapted them to TROMPA needs and proposed novel approaches and contributions to the state of the art. We established mechanisms to involve the crowd for the description and quality validation of multimodal music information and we developed models for the representation of relevant properties of crowd contributors, the design of incentivisation mechanisms for crowd contributors, the design and development of hybrid workflows combining the crowd automatic processing and the design of the building blocks of a framework for the continuous evaluation and improvement of music processing technologies.
Moreover we developed the Data Infrastructure (WP5) that will be used by all partners to store or reference data. The Data Infrastructure consists of the Contributor Environment (CE), which is essentially a web API that will expose the entirety of TROMPA content and functionalities and a collection of four components, each offering functionalities to consume and enrich the TROMPA dataset: score edition component, TROMPA Processing Library, Annotation Tool and Music Performance Assessment.
The work under WP6 has focused on concretizing and integrating insights and achievements from WP2 into user-facing pilots. In the first year, we have reached out first audiences in order to gather and validate requirements and carry out user-centered design studies.
Finally, there has been a continuous programme of Dissemination activities during the first year, involving all project partners. In terms of academic dissemination there have been 15 academic publications, 5 publicly accessible collections of datasets and/or source code, 5 invited talks, 10 contributions with academic service, 19 professional outreach events and activities and 7 dissemination activities aimed at the general audience.
During its first year, the TROMPA project has focused on defining its pilot scenarios for music scholars, orchestras, instruments players, singers and music enthusiasts (WP2). The TROMPA team has worked on the elicitation and prioritisation of the requirements for target user groups and use cases. Application mockups were developed based on the requirements elicited and tested with users of each target corresponding user group.
We worked on specifications in terms of automatic music processing technologies and human annotations and crowdsourcing technologies, including developing data-driven algorithms for automatic music description (in terms of rhythm, melody, emotion, etc) and processing (including singing synthesis and source separation). We investigated current state-of-the-art technologies that are relevant in the scope of the project defined pilots, we adapted them to TROMPA needs and proposed novel approaches and contributions to the state of the art. We established mechanisms to involve the crowd for the description and quality validation of multimodal music information and we developed models for the representation of relevant properties of crowd contributors, the design of incentivisation mechanisms for crowd contributors, the design and development of hybrid workflows combining the crowd automatic processing and the design of the building blocks of a framework for the continuous evaluation and improvement of music processing technologies.
Moreover we developed the Data Infrastructure (WP5) that will be used by all partners to store or reference data. The Data Infrastructure consists of the Contributor Environment (CE), which is essentially a web API that will expose the entirety of TROMPA content and functionalities and a collection of four components, each offering functionalities to consume and enrich the TROMPA dataset: score edition component, TROMPA Processing Library, Annotation Tool and Music Performance Assessment.
The work under WP6 has focused on concretizing and integrating insights and achievements from WP2 into user-facing pilots. In the first year, we have reached out first audiences in order to gather and validate requirements and carry out user-centered design studies.
Finally, there has been a continuous programme of Dissemination activities during the first year, involving all project partners. In terms of academic dissemination there have been 15 academic publications, 5 publicly accessible collections of datasets and/or source code, 5 invited talks, 10 contributions with academic service, 19 professional outreach events and activities and 7 dissemination activities aimed at the general audience.
One main goal of TROMPA is to create and advance technologies of music description focused on classical music. An amount of work towards this direction has been carried out especially under WP3, focused on automated music processing on audio and video, alignment between musical resources, choir singing analysis and synthesis, score analysis algorithms and multimodal cross linking. In the first year of the project, there are already 11 publications in peer-reviewed conferences and workshops.
Another impact of TROMPA is the ambition of realizing a richer, broader and more inclusive mass engagement of relevant audiences, to enable scalable and sustainable crowd feedback mechanisms for digital musical resources enrichment. WP4 focused specifically on this challenge, with activities aimed at creating the foundations -in terms of both knowledge and technology- for the development of methods for music knowledge creation from anonymous crowd drawn from microtask crowdsourcing platforms, and social media communities. The scientific work focused on the development of automatic methods for music knowledge extraction from YouTube video comments, and in a series of experiments aimed at testing the effectiveness of multimodal interfaces for the analysis and correction of digital scores produced by state-of-the-art Object Music Recognition (OMR) solutions.
Another main goal is to enrich public domain repositories with new data and metadata. This will be done under the user pilots, which are planned to start by the end of the 2nd year of the project. However, in the first year of TROMPA the consortium released scientific datasets and source code according to the Data Management Plan.
Regarding socio-economic impact, TROMPA is contributing to the current debate on how to find a good balance between human and machine intelligence and has proposed a cooperation (instead of competition) approach. In addition, the project has links with citizen science.
Another impact of TROMPA is the ambition of realizing a richer, broader and more inclusive mass engagement of relevant audiences, to enable scalable and sustainable crowd feedback mechanisms for digital musical resources enrichment. WP4 focused specifically on this challenge, with activities aimed at creating the foundations -in terms of both knowledge and technology- for the development of methods for music knowledge creation from anonymous crowd drawn from microtask crowdsourcing platforms, and social media communities. The scientific work focused on the development of automatic methods for music knowledge extraction from YouTube video comments, and in a series of experiments aimed at testing the effectiveness of multimodal interfaces for the analysis and correction of digital scores produced by state-of-the-art Object Music Recognition (OMR) solutions.
Another main goal is to enrich public domain repositories with new data and metadata. This will be done under the user pilots, which are planned to start by the end of the 2nd year of the project. However, in the first year of TROMPA the consortium released scientific datasets and source code according to the Data Management Plan.
Regarding socio-economic impact, TROMPA is contributing to the current debate on how to find a good balance between human and machine intelligence and has proposed a cooperation (instead of competition) approach. In addition, the project has links with citizen science.