Final Report Summary - INTERACT (The Interaction Engine: Interactive foundations for communication)
The goal of this project has been to bring multiple methods to bear on our understanding of the interactional foundations for language. The ‘interaction engine’ hypothesis suggests that these foundations will be visible early in ontogeny, independent of language or modality, and universal across cultures. We have concentrated on the origins and nature of turn-taking in conversation. Here are some major outcomes:
• Turn-taking timings are very similar across languages and cultures, regardless for example whether the language basic word order is Subject-Object-Verb or Verb-Object-Subject. This is surprising because the latencies in producing Verb-initial sentences might be expected to be longer, but in fact turn-transitions are as fast or faster.
• Exactly the same sort of timings are found in sign languages. This is not obvious because whereas simultaneous speaking and hearing may mask each other, the gestural-visual modality might allow more extensive overlap, but this is found only in the preparation and decay of gestures not the lexical content.
• There is an acute sensitivity to timing, so that delays in production are signaled with e.g. audible in-breaths, clicks and particles which are produced at the expected start-time.
• Vocal turn-taking is established early in infancy, the 6-month infant producing less than a third of vocalizations in overlap, with response times around 750 ms or less. The speed of response however diminishes as the infant struggles with its first words (12 months) and first phrases (18 months), and is still not at adult speed till after 3.5 years.
Having established that the cross-linguistic average gap between turns is of the order of 2-300ms, we now have a major puzzle for understanding the psychology of language use: it takes at least 600ms to plan the production of a single word, and over 1500ms to produce a simple transitive sentence regardless of language. That implies that speakers must design their responses while the other is still speaking, beginning on average half way through the other’s turn. We have been checking the evidence for this and asking: How can this overlap in comprehension and production actually work? We hypothesized that two distinct processes should be detectable in the speaker-to-be while she is listening to the current speaker:
(1) initiation of conceptualization of the response,
(2) beginning of the launch of the response.
With regards to (1):
• We have found a distinct EEG signal in the addressee linked to the point at which a speech act becomes clear, and thus to the preparation of a response – a signal not found in a matching comprehension-only task. In ancillary EEG work, we have also found that speech acts in dialogue are recognized very early.
With regards to (2) we have found:
• Launch of the articulators (the hands) in sign language is timed precisely at the end of the other’s turn
• In vocal language, breathing (measured by plethysmography) is initiated before the end of the other’s turn
• In triadic communication (using eye-trackers) the non-addressed participant switches gaze from the speaker to the addressee c. 50ms before the end of the speaker’s turn
These measures show that participants can predict the end of the other’s turn and launch their own accordingly. How do they do this? It can’t just be syntax, because syntactic units can be embedded within larger units. We have tested whether prosody is used, and have shown experimentally that it is.
In sum, we have gone a long way to showing that the ‘interaction engine’ hypothesis has a firm foundation. Many publications are in press or in preparation from this period of the project’s work (for a summary see Levinson, S. C. (2016). Turn-taking in human communication, origins, and implications for language processing. Trends in Cognitive Sciences, 20(1), 6-14. doi:10.1016/j.tics.2015.10.010.)
• Turn-taking timings are very similar across languages and cultures, regardless for example whether the language basic word order is Subject-Object-Verb or Verb-Object-Subject. This is surprising because the latencies in producing Verb-initial sentences might be expected to be longer, but in fact turn-transitions are as fast or faster.
• Exactly the same sort of timings are found in sign languages. This is not obvious because whereas simultaneous speaking and hearing may mask each other, the gestural-visual modality might allow more extensive overlap, but this is found only in the preparation and decay of gestures not the lexical content.
• There is an acute sensitivity to timing, so that delays in production are signaled with e.g. audible in-breaths, clicks and particles which are produced at the expected start-time.
• Vocal turn-taking is established early in infancy, the 6-month infant producing less than a third of vocalizations in overlap, with response times around 750 ms or less. The speed of response however diminishes as the infant struggles with its first words (12 months) and first phrases (18 months), and is still not at adult speed till after 3.5 years.
Having established that the cross-linguistic average gap between turns is of the order of 2-300ms, we now have a major puzzle for understanding the psychology of language use: it takes at least 600ms to plan the production of a single word, and over 1500ms to produce a simple transitive sentence regardless of language. That implies that speakers must design their responses while the other is still speaking, beginning on average half way through the other’s turn. We have been checking the evidence for this and asking: How can this overlap in comprehension and production actually work? We hypothesized that two distinct processes should be detectable in the speaker-to-be while she is listening to the current speaker:
(1) initiation of conceptualization of the response,
(2) beginning of the launch of the response.
With regards to (1):
• We have found a distinct EEG signal in the addressee linked to the point at which a speech act becomes clear, and thus to the preparation of a response – a signal not found in a matching comprehension-only task. In ancillary EEG work, we have also found that speech acts in dialogue are recognized very early.
With regards to (2) we have found:
• Launch of the articulators (the hands) in sign language is timed precisely at the end of the other’s turn
• In vocal language, breathing (measured by plethysmography) is initiated before the end of the other’s turn
• In triadic communication (using eye-trackers) the non-addressed participant switches gaze from the speaker to the addressee c. 50ms before the end of the speaker’s turn
These measures show that participants can predict the end of the other’s turn and launch their own accordingly. How do they do this? It can’t just be syntax, because syntactic units can be embedded within larger units. We have tested whether prosody is used, and have shown experimentally that it is.
In sum, we have gone a long way to showing that the ‘interaction engine’ hypothesis has a firm foundation. Many publications are in press or in preparation from this period of the project’s work (for a summary see Levinson, S. C. (2016). Turn-taking in human communication, origins, and implications for language processing. Trends in Cognitive Sciences, 20(1), 6-14. doi:10.1016/j.tics.2015.10.010.)