CS 499: Introduction to Intelligent Decision Making

Instructor: Sandhya Saisubramanian

Course Credits: 4 units (Note: The 4 credits from this course can be used towards your degree requirement! Check with your advisor on how to do it.)

Meeting Information: Monday & Wednesday 2:00 pm - 3:50 pm at Holland Hall 202

Instructor Office Hours: TBA

Course Description: What do Alpha-Go, self-driving cars, autonomous robots, and Mars rovers have in common? These systems are capable of making intelligent decisions autonomously. How to design such systems? How to process the available information to make intelligent decisions? How to solve real-world problems using these techniques?

This course will cover some of the key concepts in intelligent decision-making, specifically reinforcement learning. Topics include agent representation, Markov decision process, fundamentals of model-free and model-based decision-making under uncertainty, value iteration, policy iteration, Q-learning, SARSA, actor-critic methods, and imitation learning.

Prerequisites: Students should have taken CS 325 or CS 325H, with a C or higher, before registering

Announcements and Discussion Board: We will be using Canvas for the course. All course-related announcements and readings will be posted on Canvas.

Grading: Assignments: 60%, In-class Quizzes: 15%, Final project: 25%

Statement Regarding Students with Disabilities: Accommodations for students with disabilities are determined and approved by Disability Access Services (DAS). If you, as a student, believe you are eligible for accommodations but have not obtained approval please contact DAS immediately at 541-737-4098 or at http://ds.oregonstate.edu. DAS notifies students and faculty members of approved academic accommodations and coordinates implementation of those accommodations.

Schedule (subject to change):

Week

Monday

Wednesday

Assignments

1

April 1:

·  Course overview and logistics

·  Intelligent decision making

·  One-shot decision making

·  Sequential decision making

April 3:

·  Thinking like humans & rationality

·  Sequential decision making

·  Markov decision process (MDP)

·  Policy

 

 

2

April 8:

·      Designing MDP

 

April 10:

·      Computing a policy

Assignment 1 announced on April 8

3

April 15:

·      Value iteration

 

April 17:

·       Policy iteration

·       Quiz 1

 

4

April 22:

·      Project proposal presentation

April 24:

· Approximate methods to solve MDPs

 

Assignment 1 due on April 21

Assignment 2 announced on April 22

5

April 29:

·      Exploration-Exploitation tradeoff

·      Monte Carlo methods

May 1:

·  TD learning

·  Quiz 2

 

6

May 6:

·      TD learning

May 8:

· Function approximation

· Actor-critic

Assignment 2 due on May 5

Assignment 3 announced on May 6

7

May 13:

· Project progress presentation

May 15:

·   Project progress presentation

 

8

May 20:

· Actor-critic

May 22:

·  Imitation learning

·  Quiz 3

Assignment 3 due on May 19

Assignment 4 announced on May 20

9

May 27:

·  Quiz 5

·  Offline RL

May 29:

· Advanced topics: AI safety, Multi-agent decision making

 

10

June 3:

Project presentation

June 5:

Project presentation

Assignment 4 due on June 2

11

Finals Week: no class

[Project report due on June 10]

Finals Week: no class