Mechanistic studies of complex, ethological animal behaviors are poised to define the next decade of neuroscience. Fully understanding the ontogeny, evolution, and neural basis of these behaviors requires precise 3D measurements of their underlying kinematics. While 2D convolutional networks have allowed for kinematic correlates to be monitored in repetitive behavioral tasks, they are ill-suited to track keypoints in 3D and across multiple behaviors. To address this, we developed a pair of tools: CAPTURE and DANNCE, that enable continuous whole-body 3D kinematic tracking across species, behaviors, and environments (Marshall et al. Neuron 2021; Dunn* Marshall* et al. Nature Methods 2021). CAPTURE uses chronically attached retroreflective markers and motion capture to continuously track the head, trunk, and limbs of rats in 3D. DANNCE generalizes this tracking capacity to animals not bearing markers by leveraging projective geometry to construct inputs to a 3D CNN that learns to perform 3D geometric reasoning and identify body keypoints in 3D. Together these approaches enable new lines of inquiry into computational ethology, the neural basis of behavior, and artificial models of behavioral production using deep imitation learning. I will discuss the technical details of these approaches and demonstrate how to use them to analyze the structure of animal behavior across multiple timescales.

Zoom meeting: https://mit.zoom.us/j/94265988660?pwd=NmpETmlsR01tYzBMTUIrKzVrN3owZz09
Password: 103478