We humans spend almost a third of our lives asleep, and there is mounting evidence that sleep not only maintains, but actually improves many of our cognitive functions. Memory consolidation–the process of crystallizing and integrating memories into knowledge and skills–is particularly benefitted by sleep. We survey the evidence that sleep aids memory consolidation in various declarative and implicit tasks and review the basic neurophysiological structure of sleep with a focus on understanding what neural systems are involved. Drawing on machine learning research, we discuss why it might be useful for humans–and robots, perhaps–to have such an offline period for processing, even though humans are clearly capable of learning incrementally, online. Finally, we propose and simulate two mechanisms for use in computational memory models to accomplish sleep-based consolidation via either or both 1) re-encoding knowledge representations and 2) reactivating and strengthening recent memories.