Dynamic Programming and Reinforcement LearningSchedule: Fall 2017, Monday 1:00-4:00pm Course DescriptionThis course offers an advanced introduction Markov Decision Processes (MDPs)–a formalization of the problem of optimal sequential decision making under uncertainty–and Reinforcement Learning (RL)–a paradigm for learning from data to make near optimal sequential decisions. The first part of the course will cover foundational material on MDPs. We'll then look at the problem of estimating long run value from data, including popular RL algorithms like temporal difference learning and Q-learning. The final part of the course looks at the design and analysis of efficient exploration algorithms, i.e. those that intelligently probe the environment to collect data that improves decision quality. This a doctoral level course. Students should have experience with mathematical proofs, coding for numerical computation, and the basics of statistics, optimization, and stochastic processes. Course RequirementsThere will be some homework problems in the beginning of class covering fundemental material on MDPs. Afterward, the course will run like a doctoral seminar. You will be expected to engage with the material and to read some papers outside of class. The main assignment will be a course project, which could involve literature review, implementation of algorithms, or original research. TextbooksStrongly Reccomended: Dynamic Programming and Optimal Control, Vol I & II, Dimitris Bertsekas Reinforcement Learning: An Introduction, Second Edition, Richard Sutton and Andrew Barto Algorithms for Reinforcement Learning, Csaba Czepesvári Course MeetingsFollowing the business school calendar, there will be no class on October 23 or November 6. Otherwise, we will meet every Monday from September 11 to December 11. Related Courses at ColumbiaThis course complements two others that will be offered this Fall. Depending on your interests, you may wish to also enroll in one of these courses, or even both.
|