
Decoding music representational space with high resolution (7T) fMRI
Description
The deep neural codes underlying the perception and imagination of everyday sound and vision are key to advances in brain-computer interfaces. What if a brain-computer interface could capture the image of our mind's eye, and the sound of our mind's ear, and render them for others to see and hear? Whilst this type of mind reading sounds like science fiction, recent work by computer scientists and neuroscientists (Nishimoto et al., 2011; Haxby et al., 2014) has shown that visual features corresponding to subjects' perception of images and movies can be predicted from brain imaging data alone (fMRI). In these studies, brain images are expressed as high-dimensional features, i.e. neural representational space, with dimensions corresponding to voxel locations.
Toward such neural decoding for sound and music, we present our research on learning stimulus encoding models of music, for both perception and imagination of the stimuli (Casey et al., 2012; Hanke et al., 2015). We use between-subject hyper-alignment (Xu et al., 2012) of neural representational spaces so that models trained on one group of subjects can decode neural data from previously unseen subjects. Somewhat remarkably, hyper-aligned models significantly out-perform both within-subject models and models that are aligned by anatomical features only. To encourage further development of such neural decoding methods, the code, stimuli, and high-resolution 7T fMRI data from one of our experiments have been publicly released via the OpenfMRI initiative.
Speaker Bio
Michael Casey is the James Wright Professor of Music and Professor of Computer Science at Dartmouth College. He received his Ph.D. from the MIT Media Laboratory's Machine Listening group in 1998, whereupon he became a Research Scientist at Mitsubishi Electric Research Laboratories (MERL) followed by a Professor of Computer Science at Goldsmiths, University of London, before joining Dartmouth in 2008. His current research combines machine learning methods for audio-visual data with neuroimaging methods for brain-computer applications. His research is funded by the National Science Foundation (NSF), the Mellon Foundation, and the Neukom Institute for Computational Science, with prior research awards from Google Inc., the Engineering and Physical Sciences Research Council (EPSRC, UK), and the National Endowment for the Humanities (NEH). Michael and his family live in Hanover, NH.