The puzzle of learning and cognitive development -- how humans learn so much (knowledge) from so little (data) so quickly -- is central to cognitive science. This talk will look at an approach to solving this puzzle which models learning as programming and discuss a subtle discrepancy between existing models and more qualitative theories of learning. It will then propose using term rewriting systems, a formalism from theoretical computer science, as a tool for resolving this discrepancy. This proposal will be discussed in the context of recent work modeling how people learn to perform certain kinds of sequence processing tasks. The talk will close by looking at how this approach may also help in building better models of a core problem both in cognitive development and the history of science: the problem of conceptual change.