Location: 46-3310

Zoom: https://mit.zoom.us/j/92495348437

Speaker: Tamar Regev

Affiliation: Fedorenko lab​​​​​​​

Title: Characterizing the relationship between language and prosody using neural and computational approaches


Abstract: Although much past work on human communication has focused on the verbal component of language, communication also relies on a variety of non-verbal cues, including visual cues, such as facial expressions and gestures, and auditory cues, such as non-verbal vocalizations and prosody. In the recent surge of interest in multimodal communication, these different cues have often been discussed on par with one another. However, I will here argue and provide empirical evidence that prosody occupies a privileged position among non-verbal communication signals, showing a uniquely tight relationship with language processing. First, I will present an fMRI investigation of brain areas that process prosody, revealing that prosody brain areas are similar in topography to, and show partial overlap with, language brain areas. This overlap is in sharp contrast to past work that has established that facial expressions, gestures, and non-verbal vocalizations are supported by brain areas distinct from the language areas. Second, I will describe a set of computational studies using information theory and large language models to quantify the overlap between prosodic and linguistic information in natural spoken language. The findings suggest that prosodic information is largely redundant with linguistic content. The overlap between prosodic and linguistic processing in the brain and in the information they carry suggests that prosodic and linguistic representations are tightly coupled within the cognitive architecture of human communication. I will speculate on why this might be, including the distributional properties of prosody vs. other cues during communication, the role of prosody in linguistic prediction, and the scaffolding that prosody may provide during early language learning.