Zoom webinar link: https://mit.zoom.us/j/97519108549

Carina Kauf

Processing position-degraded linguistic input

The human language processing system is surprisingly robust against position order violations, both at a word and at a sentence level. For example, we are able to recognize and integrate sarcmbled wrods into our overall semantic sentence representation, and we can recover from word order canonical violations. However, there are restrictions to these capabilities, characterizable as features of the noisy channel through which the linguistic input has been passed. Here I talk about such restrictions and describe a study that investigates a. what kind of robustness to positional errors (at the word level) emerge out of different artificial neural network architectures under a specific noisy-channel assumption, and b. how the models’ semantic representations of position-degraded input compare to human processing data. The goal is to help shed light on the cognitive computations underlying robust language processing in the human mind.

Tiwalayo Eisape

Improving neural language models with human next-word predictions

Computational language models trained purely on corpus data have been shown to capture features of human language processing. However, past work has found dissociations between corpus statistics and human linguistic expectations, suggesting corpus statistics alone may be limited in their ability to endow language models with human-like predictive mechanisms.
In this talk, I will present work that evaluates several state-of-the-art language models for their match to human next-word predictions and reading time behavior. I will then present a method for distilling the linguistic information implicit in human next-word predictions into language models along with results showing that this method increases both the psychometric predictive power and language modeling performance of a baseline neural language model.