Periodic Reporting for period 1 - DaphNet (Dynamic Preservation of Interactive Art: The next frontier of Multimedia Cultural Heritage)
Période du rapport: 2017-02-01 au 2019-01-31
This project addresses the theoretical and methodological questions of embodied interaction with multimedia installation art through body movement and the use of mediation technology. The goal of the action is to contribute to the definition of an effective model for the preservation of interactive art, ensuring future re-use and flexible access. In particular, the action focuses on the interactive aspect of interactive installations. The taxonomy of interaction thus defined is translated into a metadata set for the description of interaction in artistic contexts, that would enable the exploitation of a wealth of data that is currently not captured or searchable, and that therefore doesn’t contribute to the building of new knowledge.
Most interactive artwork produced since the 1990s is lost due to the lack of adequate preservation methodologies: as a result the European cultural asset is impoverished and its potential for economic growth is drastically diminished. The intrinsic interconnection of interactive artworks with computer-based technology requires a multidisciplinary approach to preservation, involving high-profile cultural and scientific competences. With this project I have integrated my years of experience in the field of audio preservation with strategic competences that come from user-oriented studies in embodied cognition research, in order to bridge the gap between the world of the arts and the cutting-edge technology for digital preservation.
This project contributes to the Work Programme H2020 by (1) canalizing advanced competences on the safeguard and the promotion of European cultural heritage, and by (2) bridging the gap between cutting-edge scientific-technological knowledge and the world of the arts, facilitating interdisciplinary innovation and strengthening the competitiveness of the European cultural and creative sectors (Official Journal of the European Union, L 347/225, Artt. 3a and 5.1).
Most interactive artwork produced since the 1990s is lost due to the lack of adequate preservation methodologies: as a result the European cultural asset is impoverished and its potential for economic growth is drastically diminished. The intrinsic interconnection of interactive artworks with computer-based technology requires a multidisciplinary approach to preservation, involving high-profile cultural and scientific competences. With this project I have integrated my years of experience in the field of audio preservation with strategic competences that come from user-oriented studies in embodied cognition research, in order to bridge the gap between the world of the arts and the cutting-edge technology for digital preservation.
This project contributes to the Work Programme H2020 by (1) canalizing advanced competences on the safeguard and the promotion of European cultural heritage, and by (2) bridging the gap between cutting-edge scientific-technological knowledge and the world of the arts, facilitating interdisciplinary innovation and strengthening the competitiveness of the European cultural and creative sectors (Official Journal of the European Union, L 347/225, Artt. 3a and 5.1).
Most multimedia artwork produced since the 1990s is lost due to the lack of adequate preservation methodologies: as a result, the European cultural asset is impoverished and its potential for economic growth is drastically diminished. The intrinsic interconnection of multimedia art with information technology requires a multidisciplinary approach to preservation, involving high-profile cultural and scientific competences. In order to save and exploit the wealth of knowledge and economic potential locked into interactive installation art, it is necessary to define a taxonomy of these interactive installations, and subsequently to find a language to talk about the modes of interaction – as they evolve along with the technology, and the progressive absorption of these technologies into society. The methodology used during this action has mostly been qualitative, although a preliminary quantitative had been carried out with the same research group during one of my visits in preparation for this project. The main step ahead was possible after connecting with the research team that is mostly based at Concordia University in Montréal, Canada, who takes physical interaction to the next level and involves all the human senses applying physiological computing to the arts. The working framework around this type of installation deeply informed my approach to modeling interaction. Also, thanks to the grant funding, I could meet top scholars and artists especially in North America and discuss their approach to interaction, again informing my approach in a way that rendered the big picture more complex but much more accurate and fascinating at the same time. In particular, the concept that interaction is hardly an on/off thing, just like consciousness was believed to be, and that it is best represented by a spectrum of possible “degrees” of interaction. Despite simpler categories for installations like one user vs. multi user, this fluid vision of interaction abolished the idea that discrete categories are applicable if we only analyze a sufficient set of case studies. So the main result achieved so far, during the two years of intense study and traveling, is a new framework that does not focus only on modes of interaction (in terms of action-reaction between the user and system), but considers interaction as a multi-layered phenomenon where all the human senses (including electrophysiological signals like brain waves, etc.) are coupled with the technology currently used to detect, process, and display these signals (e.g. wearable sensors, etc.). The next step is a deeper categorization of concepts such as “agency”, actively debated in neighbor disciplines like Human Computer Interaction and Artificial Intelligence, intentionality, and at a higher level a hermeneutic hypercycle that helps us understand our ability to form meaning during the process of interaction.
A plain text description of how an installation works may be useful not to lose the memory of the work completely, but it’s hardly searchable, not so much because we can’t search for words in a text, but because concepts are not organized in a structured way that the system can operate with, i.e. semantic operations are not possible. Due to its influence on other art forms as well as on the contemporary culture, the evolution of interactive art may be considered as important as the evolution of opera in the XVII century. That is why the work of structuring this type of information is important: it can be a preliminary step to understanding what actually happens in the process of interaction, but it can be an effective way to describe how interaction looks like (understanding vs. description), and prevent the loss of today’s vast artistic production, with an impact that goes well beyond the artistic domain. This project has been concerned with defining a way to produce optimal metadata for interactive art, in order to facilitate the archiving, access and re-purposing of interactive artworks. In practical terms, it has theorized the possibility to submit queries such as “show me all the [interactive] installations that use full body movement”, “show me all the installations that use upper body movement to control a sonic output” o “show me all the installations that require the use to move his/her right arm to trigger a sonic and light output”. Such system did not exist to date and it is, in a way, the translation of the concept of “query by content” already studied in music, where songs (to pick a narrow but clear example) cannot just be retrieved by extra-musical features like title and author, but also by intrinsically musical features like tonality, melody fragments, etc. Interaction is obviously a key aspect of interactive art, therefore it is about time that we can adequately store and most importantly retrieve these works by features that intrinsically relate to interaction and not only author (name of the artist), year of production, etc. The importance and scope of this work will reveal itself more in a decade from now and further down the road, when contemporary modes of interaction and the technology used to detect, process, and display information will have evolved. With the rapid pace at which new technology is released on the market, it is only reasonable to assume that the landscape of interactive art will look like a completely different world from now, especially with the very interesting experimentation currently going on with mixed reality.