Periodic Reporting for period 4 - STYDS (Seeing things you don't see: Unifying the philosophy, psychology and neuroscience of multimodal mental imagery)
Période du rapport: 2022-03-01 au 2023-08-31
When I am looking at my coffee machine that makes funny noises, this is an instance of multisensory perception – I perceive this event by means of both vision and audition. But very often we only receive sensory stimulation from a multisensory event by means of one sense modality. If I hear the noisy coffee machine in the next room (without seeing it), then how do I represent the visual aspects of this multisensory event?
The aim of this research project is to bring together empirical findings about multimodal perception and empirical findings about (visual, auditory, tactile) mental imagery and argue that on occasions like the one described in the last paragraph, we have multimodal mental imagery: perceptual processing in one sense modality (here: vision) that is triggered by sensory stimulation in another sense modality (here: audition).
Multimodal mental imagery is rife. The vast majority of what we perceive are multisensory events: events that can be perceived in more than one sense modality – like the noisy coffee machine. And most of the time we are only acquainted with these multisensory events via a subset of the sense modalities involved – all the other aspects of these events are represented by means of multisensory mental imagery. This means that multisensory mental imagery is a crucial element of almost all instances of everyday perception, which has wider implications to philosophy of perception and beyond, to epistemological questions about whether we can trust our senses.
Focusing on multimodal mental imagery can help us to understand a number of puzzling perceptual phenomena, like sensory substitution and synaesthesia. Further, manipulating mental imagery has recently become an important clinical procedure in various branches of psychiatry as well as in counteracting implicit bias – using multimodal mental imagery rather than voluntarily and consciously conjured up mental imagery can lead to real progress in these experimental paradigms.
The aim of this research project is to bring together empirical findings about multimodal perception and empirical findings about (visual, auditory, tactile) mental imagery and argue that on occasions like the one described in the last paragraph, we have multimodal mental imagery: perceptual processing in one sense modality (here: vision) that is triggered by sensory stimulation in another sense modality (here: audition).
Multimodal mental imagery is rife. The vast majority of what we perceive are multisensory events: events that can be perceived in more than one sense modality – like the noisy coffee machine. And most of the time we are only acquainted with these multisensory events via a subset of the sense modalities involved – all the other aspects of these events are represented by means of multisensory mental imagery. This means that multisensory mental imagery is a crucial element of almost all instances of everyday perception, which has wider implications to philosophy of perception and beyond, to epistemological questions about whether we can trust our senses.
Focusing on multimodal mental imagery can help us to understand a number of puzzling perceptual phenomena, like sensory substitution and synaesthesia. Further, manipulating mental imagery has recently become an important clinical procedure in various branches of psychiatry as well as in counteracting implicit bias – using multimodal mental imagery rather than voluntarily and consciously conjured up mental imagery can lead to real progress in these experimental paradigms.
Work on the Seeing Things You Don’t See project unfolded as expected, with a couple of very minor and logistical exceptions (the most important of which was that one of the postdocs got a permanent job in the middle of his contract and had to be replaced – see below).
The monograph by the PI on the topic of the ERC project was accepted for publication and is now under contract with Oxford University Press. The working title is the same as the title of the ERC project: Seeing Things You Don’t See. I am now exploring the possibility of publishing this book fully open access, in accordance with the open access requirements of the European Research Council.
I am also editing a special issue of Philosophical Transactions of the Royal Society B on the topic of the grant (working title: Offline perception) together with Peter Fazekas and Joel Pearson.
I published papers on the topic of the ERC project in some of the top journals in philosophy (and also in some very good psychology/neuroscience journals), such as British Journal for the Philosophy of Science, Analysis, Erkenntnis, Synthese, Cortex, Perception, i-Perception, European Journal of Philosophy, Inquiry, Ergo, Consciousness and Cognition. The support of ERC was acknowledged in all of these venues and, where possible, the publication was made open access.
I have given talks on the topic of the ERC project at the following universities: University of Torino (Italy), University of Glasgow (UK), Freie Universitat Berlin (Germany), NYU Abu Dhabi (UAE), University of Copenhagen (Denmark), University of Murcia (Spain), Bilkent University (Turkey), University of Warwick (UK), Central European University (Hungary), University of Oxford (UK), University of Cyprus (Cyprus), University of Crete (Greece), University of Ghent (Belgium), Humboldt University, Berlin (Germany), University of Toronto (Canada), University of Milan (Italy), Ruhr University, Bochum (Germany), University of Firenze (Italy), University of Nijmegen (The Netherlands), University of Southampton (UK), University of Louvain la Neuve (Belgium), University of Haifa (Israel), Hebrew University, Jerusalem (Israel), Universita Roma Tre (Italy), among others. The funding of the European Research Council was acknowledged throughout.
The results of the research were also communicated to non-expert audiences, for example, by giving talks at the Hay-on-Wye festival (UK) or the National Portrait Gallery (UK), by interviews in the magazine Der Spiegel (Germany), TalkRadio (UK) or the Radio W (Colombia), by a TED-Ed video (translated to 16 languages) as well as various articles (translated altogether to 12 languages) to the general public in Psychology Today, Quartz, iai.tv TheConversation, Aeon, The Week, The Wire, Esquire, Big Think, and so on.
The monograph by the PI on the topic of the ERC project was accepted for publication and is now under contract with Oxford University Press. The working title is the same as the title of the ERC project: Seeing Things You Don’t See. I am now exploring the possibility of publishing this book fully open access, in accordance with the open access requirements of the European Research Council.
I am also editing a special issue of Philosophical Transactions of the Royal Society B on the topic of the grant (working title: Offline perception) together with Peter Fazekas and Joel Pearson.
I published papers on the topic of the ERC project in some of the top journals in philosophy (and also in some very good psychology/neuroscience journals), such as British Journal for the Philosophy of Science, Analysis, Erkenntnis, Synthese, Cortex, Perception, i-Perception, European Journal of Philosophy, Inquiry, Ergo, Consciousness and Cognition. The support of ERC was acknowledged in all of these venues and, where possible, the publication was made open access.
I have given talks on the topic of the ERC project at the following universities: University of Torino (Italy), University of Glasgow (UK), Freie Universitat Berlin (Germany), NYU Abu Dhabi (UAE), University of Copenhagen (Denmark), University of Murcia (Spain), Bilkent University (Turkey), University of Warwick (UK), Central European University (Hungary), University of Oxford (UK), University of Cyprus (Cyprus), University of Crete (Greece), University of Ghent (Belgium), Humboldt University, Berlin (Germany), University of Toronto (Canada), University of Milan (Italy), Ruhr University, Bochum (Germany), University of Firenze (Italy), University of Nijmegen (The Netherlands), University of Southampton (UK), University of Louvain la Neuve (Belgium), University of Haifa (Israel), Hebrew University, Jerusalem (Israel), Universita Roma Tre (Italy), among others. The funding of the European Research Council was acknowledged throughout.
The results of the research were also communicated to non-expert audiences, for example, by giving talks at the Hay-on-Wye festival (UK) or the National Portrait Gallery (UK), by interviews in the magazine Der Spiegel (Germany), TalkRadio (UK) or the Radio W (Colombia), by a TED-Ed video (translated to 16 languages) as well as various articles (translated altogether to 12 languages) to the general public in Psychology Today, Quartz, iai.tv TheConversation, Aeon, The Week, The Wire, Esquire, Big Think, and so on.
I expect the research to unfold as planned for the rest of the grant period - the monograph will be sent to Oxford University Press and the special issue of Philosophical Transactions of the Royal Society B on the topic of the grant (working title: Offline perception) will be published, together with a number of other scientific papers in a variety of philosophy and interdisciplinary journals.