AI's next big challenge is integrating and automating the essential cognitive abilities of acting, planning, and learning. This comprehensive overview covers a range of models –deterministic, probabilistic (including MDP and reinforcement learning), hierarchical, nondeterministic, temporal, spatial – and applications in robotics.