Real-World Robot Learning
Spring 2025. ESE 6800/CS7000. Tue / Thu 10:15-11:45. AGH 105.
Announcements
Course Overview
Decision-making has been a cornerstone of artificial intelligence since the field’s inception in the 1960s. While the techniques and algorithms have evolved dramatically over time, the fundamental challenge remains: how to make intelligent decisions in the presence of uncertainty. Over the past six decades, this research has led to the development of highly advanced systems, with some achieving superhuman performance in cognitively demanding tasks like Go, Atari, Gran Turismo, Chess, StarCraft, and SOTA.
However, despite these remarkable successes, most of these systems excel only in controlled, simulated, or game-based environments. Why haven’t the same methods translated seamlessly to real-world decision-making, such as controlling a physical robot to perform household tasks? What makes real-world environments so uniquely challenging? And what recent advances are pushing the boundaries of what’s possible in real-world applications?
This course offers a structured framework to explore these questions. We will study techniques for learning-based decision-making, such as imitation learning and reinforcement learning, focusing on their practical challenges when applied in real-world scenarios. Through a combination of lectures, student presentations, hands-on projects, and guest presentations from leading experts in the field, students will gain a deep understanding of the state-of-the-art decision-making systems and their challenges when applied to robotics.
Prerequisites
This is a graduate-level course. Students are expected to have prior knowledge in deep learning and robotics, such as the topics covered in Robot Learning (ESE 650), Principles of Deep Learning (ESE 546), Applied Machine Learning (CIS 5190), and Introduction to Robotics (MEAM-520).
Schedule (Tentative)
The general idea behind this course is the following. We will start by going off the beaten arxiv track and read oldy but goldy papers. Then, we will use such papers to understand the roots of more recent works. Note that this schedule will evolve during the course.
Overture
- Jan. 16
- Intro + Overview
- Syllabus
- Jan. 21
- Supervised Learning (Review)
- Jan. 23
- How to use Pytorch (Review)
- Jan. 28
- Reinforcement Learning Part I (Review)
- Jan. 30
- Reinforcement Learning Part II (Review)
Foundations
- Feb. 4
- Robot Perception I
- Paper Reading
- Feb. 6
- Robot Perception II
- Paper Reading
- Feb. 11
- The Development Perspective on Robot Learning I
- Paper Reading
- Feb. 13
- The Development Perspective on Robot Learning II
- Paper Reading
Data
- Feb. 18
- Learning from real-world robot data I
- Paper Reading
- Feb. 20
- Learning from real-world robot data II
- Project Proposal Due Paper Reading
- Feb. 25
- Learning from others’ data I
- Paper Reading
- Feb. 27
- Learning from others’ data II
- Paper Reading
- Mar. 4
- Learning from Internet Data I
- Paper Reading
- Mar. 6
- Learning from Internet Data II
- Paper Reading
- Mar. 11
- No Class Spring Break 🏝️
- Mar. 13
- No Class Spring Break 🏝️
- Mar. 18
- Learning from Synthetic Data I
- Paper Reading
- Mar. 20
- Learning from Synthetic Data II
- Project Midterm Report Due Paper Reading
- Mar. 25
- Lifelong Learning I
- Paper Reading
- Mar. 27
- Lifelong Learning II
- Paper Reading
Frontiers
- Apr. 1
- Guest Lecture
- Apr. 3
- Challenges for the field I
- Paper Reading
- Apr. 8
- Guest Lecture
- Apr. 10
- Challenges for the field II
- Paper Reading
- Apr. 15
- Challenges for the field III
- Paper Reading
- Apr. 17
- Grand Finale
- Paper Reading
Project Presentations
- Apr. 22
- Project Presentations
- Presenters TBD
- Apr. 24
- Project Presentations
- Slides Due 11:59 pm ET, April 24 Presenters TBD
- Apr. 29
- Best Presentation Awards 🍨
- Project Report Due May 1
Instructors
Teaching Assistants
Related Courses
Robots that learn, UC Berkeley.
Visual Scene Understanding, UC Berkeley.
Embodied AI Safety, CMU. This course is not only very interesting, but also has an awesome webpage.