Face recognition is a computationally challenging task. To uncover the nature of the representation that enables human remarkable face recognition abilities, most studies have primarily focused on the role of visual perception by studying the representation of unfamiliar faces. However, recent studies have shown that human expertise in face recognition is limited to familiar faces and not well generalized to unfamiliar faces. In my talk, I will consider two accounts for human expertise in face recognition: the perceptual hypothesis, which emphasizes the role of rich perceptual experience in the generation of an invariant representation of face identity and the conceptual hypothesis, which highlights the contribution of conceptual information to face recognition. I will present findings that show that perceptual information alone may not account for human expertise in familiar face recognition. I will further suggest that the representation of face identity is based on our conceptual and perceptual experience with familiar faces (“supervised”), rather than our passive perceptual exposure to unfamiliar faces (“unsupervised”). This framework, which considers both perceptual and conceptual information to understand recognition, is not specific to faces and may account for recognition of any familiar stimuli including familiar voices, tastes or other objects of expertise.