A unified discussion of evolutionary and learning dynamics is needed for at least three reasons: i) scientific: exploring all possible algorithmically different modes of adaptation, ii) engineering: how to design systems that combine the combinatorial, open-ended novelty-generating power of Darwinian search with near-optimal learning dynamics, iii) scientific-engineering: since the only algorithm we know that is capable of generating complex adaptive architectures from scratch is an evolutionary one, a deeper and more flexible understanding of AI systems can only be achieved by evolving them and relating engineered or emergent selective pressures with evolved computational functions. I will not solve any of these questions, not even partially. Instead, I’ll discuss three directions along these lines that might be useful from both scientific and engineering perspectives. 1. Lessons from evolutionary biology that are crucial for understanding any biological or non-biological Darwinian system, including the unit of selection, evolution of evolvability, and open-endedness. 2. Implementation of Bayesian computations on the substrate of Darwinian replicators. 3. Exploring the possible ways an evolutionary search can be performed within brains, based on the replication of neural firing patterns.