Real-World Robot Learning
Spring 2025. ESE 6800/CS7000. Tue / Thu 10:15-11:45. AGH 105.
Announcements
Get ready for the hands-on tutorial and first papers!
Please sign up for a weight and biases and huggingface account before our hands-on tutorial on policy learning. Also, we released the first papers!
Course Overview
Decision-making has been a cornerstone of artificial intelligence since the field’s inception in the 1960s. While the techniques and algorithms have evolved dramatically over time, the fundamental challenge remains: how to make intelligent decisions in the presence of uncertainty. Over the past six decades, this research has led to the development of highly advanced systems, with some achieving superhuman performance in cognitively demanding tasks like Go, Atari, Gran Turismo, Chess, StarCraft, and SOTA.
However, despite these remarkable successes, most of these systems excel only in controlled, simulated, or game-based environments. Why haven’t the same methods translated seamlessly to real-world decision-making, such as controlling a physical robot to perform household tasks? What makes real-world environments so uniquely challenging? And what recent advances are pushing the boundaries of what’s possible in real-world applications?
This course offers a structured framework to explore these questions. We will study techniques for learning-based decision-making, such as imitation learning and reinforcement learning, focusing on their practical challenges when applied in real-world scenarios. Through a combination of lectures, student presentations, hands-on projects, and guest presentations from leading experts in the field, students will gain a deep understanding of the state-of-the-art decision-making systems and their challenges when applied to robotics.
Prerequisites
This is a graduate-level course. Students are expected to have prior knowledge in deep learning and robotics, such as the topics covered in Robot Learning (ESE 650), Principles of Deep Learning (ESE 546), Applied Machine Learning (CIS 5190), and Introduction to Robotics (MEAM-520).
Schedule (Tentative)
The general idea behind this course is the following. We will start by going off the beaten arxiv track and read oldy but goldy papers. Then, we will use such papers to understand the roots of more recent works. Note that this schedule will evolve during the course.
Overture
- Jan. 16
- Jan. 21
- Intro To Imitation Learning & Reinforcement Learning Part I
- Slides
- Jan. 23
- Intro To Imitation Learning & Reinforcement Learning Part II
- Slides
- Jan. 28
- Intro To Imitation Learning & Reinforcement Learning Part III
- Slides
- Jan. 30
- Hands-on Tutorial on Policy Learning (Lab)
- Bring your laptop!
- IL Tutorial Colab, RL Tutorial Colab
Foundations
- Feb. 4
- Robot Perception I
- Slides, Paper Reading A critique of pure vision
- Feb. 6
- Robot Perception II
- Slides, Paper Reading Intelligence w/o Representations, Designing Action-Based Sensors
- Feb. 11
- The Development Perspective on Robot Learning I
- Slides, Paper Reading Six Lessons from Babies
- Feb. 13
- The Development Perspective on Robot Learning II
- Paper Reading Sensorimotor Learning, Intrinsic Motivation
Data
- Feb. 18
- Learning from real-world robot data I
- Paper Reading Motor Learning by Imitation
- Feb. 20
- Learning from real-world robot data II
- Project Proposal Due Paper Reading Diffusion Policy, Aloha, Robots in Homes
- Feb. 25
- Learning from others’ data I
- Paper Reading Surprising Effectiveness of Representations, Unbiased Look at Datasets
- Feb. 27
- Learning from others’ data II
- Paper Reading SFV, X-Embodiment
- Mar. 4
- Learning from Internet Data I
- Paper Reading Learning Action Plans, SayCan, OpenVLA
- Mar. 6
- Learning from Internet Data II
- Paper Reading UniSim
- Mar. 11
- No Class Spring Break 🏝️
- Mar. 13
- No Class Spring Break 🏝️
- Mar. 18
- Learning from Synthetic Data I
- Paper Reading
- Mar. 20
- Learning from Synthetic Data II
- Project Midterm Report Due Paper Reading
- Mar. 25
- Lifelong Learning I
- Paper Reading
- Mar. 27
- Lifelong Learning II
- Paper Reading
Frontiers
- Apr. 1
- Guest Lecture
- Apr. 3
- Challenges for the field I
- Paper Reading
- Apr. 8
- Guest Lecture
- Apr. 10
- Challenges for the field II
- Paper Reading
- Apr. 15
- Challenges for the field III
- Paper Reading
- Apr. 17
- Grand Finale
- Paper Reading
Project Presentations
- Apr. 22
- Project Presentations
- Presenters TBD
- Apr. 24
- Project Presentations
- Slides Due 11:59 pm ET, April 24 Presenters TBD
- Apr. 29
- Best Presentation Awards 🍨
- Project Report Due May 1
Instructors
Teaching Assistants
Related Courses
Robots that learn, UC Berkeley.
Visual Scene Understanding, UC Berkeley.
Embodied AI Safety, CMU. This course is not only very interesting, but also has an awesome webpage.