Periodic Reporting for period 1 - INCREASE (Innovations in Neural Conceptual Representation: Exploring Aspects of Semantics)
Berichtszeitraum: 2016-10-15 bis 2018-10-14
Our world knowledge, semantic representations, can be revealed by similarity between concepts. The capacity to recognize semantic similarity underlies many cognitive processes such as categorization, pattern recognition, memory retrieval, reasoning and problem solving, allowing us to apply experience to new problems, facilitating innovation. Many theories of semantic representation exist, but tend to focus on very different measures, some based on sensorimotor experience with the world and others based on regularities in language. Recent approaches show the value of combining similarity measures, but many more remain that could allow us better understanding of meaning, such as word association, false memory, priming and rating tasks. Because of this, basic questions like how meanings are represented and how they are organised in our brains are still unanswered, hot topics of debate in the field. Our project “INCREASE” simultaneously investigated this question at three different levels: behaviour, models, and brain activity, considering many similarity dimensions together to test their roles in establishing meaning. Our results provide a better characterization of different similarity measures adjudicating between the different semantic models and identifying their neural underpinnings in representation of abstract and concrete concepts.
1. Do different measures of similarity vary in predicting semantic priming?
We carried out behavioural experiments to test whether relatedness by verbal features, co-occurrence or word association predicts semantic priming (faster responses to target word after a related one), selecting word pairs related only by one measure. Priming occurred only for pairs related by association, indicating importance of this measure for theories of semantic representation.
2. Does featural similarity affect pupil responses in false memory and recognition for words?
We measured pupil dilation in an old/new recognition task, controlling for word association and co-occurrence to allow us to specifically investigate featural similarity. New words in the recognition test with high similarity to studied words tended to be mistakenly identified as old, and had more pupil dilation than those with low similarity, suggesting that featural similarity is relevant, beyond association and co-occurrence, for semantic representation.
3. How independent are different measures of similarity?
The studies above show that these measures can predict behavioural effects. To test their joint contribution in reflecting semantic representations, we compared similarity measures between all pairs of words, including new association and co-occurrence norms we collected. We found high similarity between representations: despite some degree of independence in predicting behavioural effects, these measures might reflect a common underlying semantic system.
To test this question more directly we compared an even wider variety of relatedness measures. Consistent with the previous study, very different data sources are employed similarly in semantic representations and surprisingly, they reflect a gradient of depth of information encoding.
4. Does similarity in brain activity correlate with similarity derived from different measures?
We used functional magnetic resonance imaging (fMRI) and functional near-infrared spectroscopy (fNIRS) in two experiments to test this question. Preliminary analyses show differences in how the left-lateralized semantic network responds to different similarity measures, varying as a function of concreteness, consistent with the idea that different organisational principles govern representations of abstract and concrete words.
5. Is processing of semantic dimensions affected by divided attention?
In a behavioural study using a feature selection task, we tested effects of featural relevance on semantic performance under conditions of divided attention. As the relevance of the distracter feature increased, there was greater dual-task disruption, indicating that semantic relevance is an organizing principle of semantic representation, and that domain-general control processes are more relevant to the capacity to efficiently avoid distracting information during semantic decision-making.
6. Do participant-level variables modulate semantic and conceptual processing?
We tested the extent to which grammatical gender in Italian and German affected speakers' affective ratings (valence, arousal and dominance), and whether this might be affected by individual variation in sex and age. We found that grammatical gender did not affect affective ratings within or across languages, nor were there any interactions with speakers’ age and sex. These findings suggest that grammatical gender does not primarily affect conceptual representations, at least where affect is concerned.
We carried out the following dissemination activities so far:
- Two empirical papers published in international peer-reviewed journals, and one as a conference proceeding. Eight more papers arising from the project are in preparation or under revision for publication.
- Several presentations at international conferences and workshops targeting experts in language, semantic and conceptual processing.
- Public engagement: promotion of women's participation in science; online activity bringing together researchers and school students; European Researcher's night to share news about research in neuroscience to members of the public.
We carried out behavioural experiments to test whether relatedness by verbal features, co-occurrence or word association predicts semantic priming (faster responses to target word after a related one), selecting word pairs related only by one measure. Priming occurred only for pairs related by association, indicating importance of this measure for theories of semantic representation.
2. Does featural similarity affect pupil responses in false memory and recognition for words?
We measured pupil dilation in an old/new recognition task, controlling for word association and co-occurrence to allow us to specifically investigate featural similarity. New words in the recognition test with high similarity to studied words tended to be mistakenly identified as old, and had more pupil dilation than those with low similarity, suggesting that featural similarity is relevant, beyond association and co-occurrence, for semantic representation.
3. How independent are different measures of similarity?
The studies above show that these measures can predict behavioural effects. To test their joint contribution in reflecting semantic representations, we compared similarity measures between all pairs of words, including new association and co-occurrence norms we collected. We found high similarity between representations: despite some degree of independence in predicting behavioural effects, these measures might reflect a common underlying semantic system.
To test this question more directly we compared an even wider variety of relatedness measures. Consistent with the previous study, very different data sources are employed similarly in semantic representations and surprisingly, they reflect a gradient of depth of information encoding.
4. Does similarity in brain activity correlate with similarity derived from different measures?
We used functional magnetic resonance imaging (fMRI) and functional near-infrared spectroscopy (fNIRS) in two experiments to test this question. Preliminary analyses show differences in how the left-lateralized semantic network responds to different similarity measures, varying as a function of concreteness, consistent with the idea that different organisational principles govern representations of abstract and concrete words.
5. Is processing of semantic dimensions affected by divided attention?
In a behavioural study using a feature selection task, we tested effects of featural relevance on semantic performance under conditions of divided attention. As the relevance of the distracter feature increased, there was greater dual-task disruption, indicating that semantic relevance is an organizing principle of semantic representation, and that domain-general control processes are more relevant to the capacity to efficiently avoid distracting information during semantic decision-making.
6. Do participant-level variables modulate semantic and conceptual processing?
We tested the extent to which grammatical gender in Italian and German affected speakers' affective ratings (valence, arousal and dominance), and whether this might be affected by individual variation in sex and age. We found that grammatical gender did not affect affective ratings within or across languages, nor were there any interactions with speakers’ age and sex. These findings suggest that grammatical gender does not primarily affect conceptual representations, at least where affect is concerned.
We carried out the following dissemination activities so far:
- Two empirical papers published in international peer-reviewed journals, and one as a conference proceeding. Eight more papers arising from the project are in preparation or under revision for publication.
- Several presentations at international conferences and workshops targeting experts in language, semantic and conceptual processing.
- Public engagement: promotion of women's participation in science; online activity bringing together researchers and school students; European Researcher's night to share news about research in neuroscience to members of the public.
The results of INCREASE advance our understanding of semantic representation, showing that word association is an important principle, and that pupillary response is a reliable semantic measure. The similarity between very different measures provides evidence for a common underlying semantic system even though different measures can contribute separately to meaning-based effects. We also found that cognitive control modulates the way semantic dimensions affect concept processing, providing evidence for a new theoretical framework, "controlled semantic cognition" that clarifies the relation between semantics and control processes. Most importantly, we found that abstract and concrete concepts tend to depend on different kinds of information, allowing us to adjudicate between different semantic theories and identifying their neural underpinnings. The project has also provided new resources for language researchers, including Italian age of acquisition, lexical co-occurrence and association norms.
Deeper understanding how similarity dimensions underlie human knowledge arising from INCREASE has further practical value, such as information retrieval systems, automated online assistants, and question answering systems, which have tended to adopt less flexible, syntactic approaches. By providing human-like knowledge representations based on semantic similarity, they may improve query recognition and provide more relevant responses. More importantly, as >30% of patients with stroke have some kind of aphasia, better understanding of semantic information in our brains may provide the basis to develop more effective diagnostic and rehabilitation tests for patients with specific language and semantic deficits.
Deeper understanding how similarity dimensions underlie human knowledge arising from INCREASE has further practical value, such as information retrieval systems, automated online assistants, and question answering systems, which have tended to adopt less flexible, syntactic approaches. By providing human-like knowledge representations based on semantic similarity, they may improve query recognition and provide more relevant responses. More importantly, as >30% of patients with stroke have some kind of aphasia, better understanding of semantic information in our brains may provide the basis to develop more effective diagnostic and rehabilitation tests for patients with specific language and semantic deficits.