Martin Schrimpf

Advisor: James DiCarlo

Abstract: Research in the brain and cognitive sciences attempts to uncover the neural mechanisms underlying intelligent behavior in domains such as vision or language. Due to the complexities of brain processing, studies necessarily had to start with a narrow scope of experimental investigation and computational modeling. This thesis argues that it is time for our field to take the next step: build systems models that capture neural mechanisms and supported behaviors within an entire domain of intelligence. To make progress on systems models, we propose integrative benchmarking – integrating experimental results from many laboratories into suites of benchmarks that guide and constrain those models at multiple stages and scales. We show-case this approach by developing Brain-Score benchmark suites for neural and behavioral experiments in the primate visual ventral stream and the human language system, as well as direct neural (causal) perturbation experiments in inferotemporal cortex. By systematically evaluating a wide variety of model candidates, we not only identify models beginning to match a range of brain data (∼50% explained variance), but also discover key relationships: Models’ brain scores are predicted by their object categorization performance in vision (but only up to 70% ImageNet accuracy), and their next-word prediction performance in language. The better models predict internal neural activity, the better they match human behavioral outputs, with architecture substantially contributing to brain-like representations. Using the integrative benchmarks, we develop improved state-of-the-art systems models that more closely match shallow recurrent neuroanatomy and predict primate temporal processing, as well as models that require only a fraction of supervised synaptic updates. Taken together, the integrative benchmarks and systems models presented here are first steps to modeling the complexities of brain processing in entire domains of intelligence.

Please login via Zoom https://mit.zoom.us/j/96705952242?pwd=ZEI4eGd5bDFDV3NicWZrZzZ5MW1XZz09