Abstract: Successfully navigating the social world requires reasoning about both high-level strategic goals, such as whether to cooperate or compete, and the lower-level actions needed to achieve those goals. Unlike a repeated prisoner's dilemma, each social interaction is unique. Cooperation unfolds over time and must be coordinated over physical space across an infinite possibility of environments. Furthermore, we cooperate with different social partners, some of whom we know, some we have never met before, and many we will never meet again. We present new formal mathematical models of cooperation that tackle these challenges by building on three key capacities of social cognition: (1) an abstract utility calculus that allows for recursively valuing the welfare of others, (2) the ability to plan with others and form joint intentions that coordinate action on shared goals in any environment and (3) hierarchical Bayesian inference to understand the intentions and values of others from sparse and noisy observations of behavior. To test this approach, we examine human behavior in novel multi-agent spatial games that people play intuitively. The model explains how cooperation (and competition) is generalized across environments as well as how norms and conventions form to make cooperation more efficient. Finally, evolutionary simulations demonstrate that our approach allows for cooperation to emerge in a wider range of settings than have previously been studied and can outperform inflexible behavioral strategies (such as Tit-for-Tat) in the repeated prisoner’s dilemma. By enriching game-theoretic accounts of social behavior with computational cognitive models, we better explain the fine-grained structure and origins of distinctly human cooperation.