Syllabus

Table of Contents

  1. Overview
  2. Logistics
  3. Prerequisites
  4. Textbooks
  5. Attendance
  6. Academic Integrity
  7. Late Policy
  8. Communication
  9. Grading
  10. Paper Summaries & Presentations
    1. Paper Summaries
    2. Paper Presentations
  11. Health & Wellness

Overview

Decision-making has been a cornerstone of artificial intelligence since the field’s inception in the 1960s. While the techniques and algorithms have evolved dramatically over time, the fundamental challenge remains: how to make intelligent decisions in the presence of uncertainty. Over the past six decades, this research has led to the development of highly advanced systems, with some achieving superhuman performance in cognitively demanding tasks like Go, Atari, Gran Turismo, Chess, StarCraft, and SOTA.

However, despite these remarkable successes, most of these systems excel only in controlled, simulated, or game-based environments. Why haven’t the same methods translated seamlessly to real-world decision-making, such as controlling a physical robot to perform household tasks? What makes real-world environments so uniquely challenging? And what recent advances are pushing the boundaries of what’s possible in real-world applications?

This course offers a structured framework to explore these questions. We will study techniques for learning-based decision-making, such as imitation learning and reinforcement learning, focusing on their practical challenges when applied in real-world scenarios. Through a combination of lectures, student presentations, hands-on projects, and guest presentations from leading experts in the field, students will gain a deep understanding of the state-of-the-art decision-making systems and their challenges when applied to robotics.

Logistics

  • Title: Real-World Robot Learning, Spring 2025
  • Course Number: ESE 6800-004
  • Lecture: 10:15AM–11:45AM EST, Tue & Thu
  • Office Hours: TBD
  • Location: AGH 105

Prerequisites

This is a graduate-level course. Students are expected to have prior knowledge in deep learning and robotics, such as the topics covered in Robot Learning (ESE 650), Principles of Deep Learning (ESE 546), Applied Machine Learning (CIS 5190), and Introduction to Robotics (MEAM-520).

Textbooks

There is no need buy any textbook for this course. We will provide slides or lecture notes in this course. The following are companion textbooks that can provide useful further reading:

  • Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction
  • Christopher Bishop and Hugh Bishop, Deep Learning: Foundations and Concepts

Attendance

Class attendance and participation are key for both your and your peer’s success in this class. You are expected to attend class in person during the scheduled time, including the final presentations. I understand that occasionally you may have challenges attending (e.g., illness, religious observance, etc.). However, if you anticipate having a challenge regularly attending class, please contact us.

Academic Integrity

Honesty and transparency are important features of good scholarship. On the flip side, plagiarism and cheating are serious academic offenses with serious consequences. If you are discovered engaging in either behavior in this course, you will earn a failing grade on the assignment in question, and further disciplinary action may be taken. We encourage you to work together on projects (teams up to 2) to make use of campus resources like Penn GSE Student Success to assist you in your pursuit of academic excellence. However, please note that in accord with the university’s policy you must acknowledge any collaboration or assistance that you receive on work that is to be graded, either from a person, reference, or a tool (including AI-generation tools like ChatGPT).

Late Policy

Paper summaries are due right before the class starts. Reading and summarizing papers is necessary to foster good discussions. However, it might happen that one forgets to submit a summary, or circumstances arises. You can skip up to two summaries without any penalty. All skipped summaries after the second will receive a zero grade. There is no late policy for summaries. If you submit after the class, the summary is considered as “not submitted”.

You can use a maximum of 3 late days for project proposals, midterms, and final report. A day is counted starting from 1 hour after the deadline. For instance, if your report is due on Monday at 10AM and you submit at 11:01AM, you have used one late day. After using three late days, the assignment will be given a grade of zero.

Unexcused late presentations will immediately receive a zero grade, because they will affect our ability to hold class. Re-scheduling presentations will be based on schedule availability and the professors’ discretion.

If you experience an unforseeable emergency and would like us to consider waiving any of the above penalties, please email us as early as possible to discuss this request.

Communication

  • Website: We will use the class website for posting course content (e.g., slides, paper readings, lecture recordings).
  • Canvas: We will use Canvas for uploading all assignments and grades.
  • Email: If you email your instructors, please include the substring “[RWRL Course]” to begin a meaningful subject line and have tried to resolve the issue appropriately otherwise. Please use your UPenn email account.

Grading

This course will have no exams. Instead, grading will be broken down by

  • Attendance: We want students to attend lectures in person consistently. Students are permitted 2 unexcused absences, no questions asked, before being docked.
  • Paper Summaries, Discussion, and Presentations: One goal we have for this course is for you to understand how to consume, explain, and critique research papers. Paper summaries are 1-2 paragraph descriptions (per paper) of the reading assignments that will be submitted to Canvas. Canvas notes are short comments about the work (can be a limitation, an observation, etc.). Paper presentations occur during in-class discussions.
  • Project: Students will engage in a semester-long research project related to the themes of the course before presenting them at the end of the semester.
  • Extra Credits: You will receive extra credits for the following activities:
  • Run experiments with the code of the paper you’re presenting, showing your empirical analysis during the presentation.
  • Attend Talks and Events: There will be some interesting talks and events throughout the semester, which we will advertise. Going to the talk and submitting a summary of the talk on Canvas will give you extra credits.
PercentageActivity
10%Attendance and Discussion
30%Paper Summaries & Comments
20%Paper Presentation
10%Midterm Project Report
30%Final Project (Presentation + Report)

Paper Summaries & Presentations

Paper Summaries

There will be several paper discussion days during which you will be as- signed research papers to read. You are expected to complete all assigned readings before class and come prepared with comments and questions to discuss with the group. You will share 1–2 paragraphs with your takeaways (per paper) on each reading on Canvas, by 9:59am ET of the day the reading will be discussed.

Paper Presentations

During paper discussion days, we will dive into two or three papers. You will be able to decide which discussion topic you want to be assigned to (first come, first serve). You will present one paper from that topic. (Note: this paper presentation structure is subject to change based on class size).

Health & Wellness

Take care of yourself. Do your best to maintain a healthy lifestyle this semester by eating well, exercising, avoiding drugs and alcohol, getting enough sleep and taking some time to relax. This will help you achieve your goals and cope with stress.

All of us benefit from support during times of struggle. There are many helpful resources available on campus and an important part of the college experience is learning how to ask for help. Asking for support sooner rather than later is almost always helpful.

If you or anyone you know experiences any academic stress, difficult life events, or feelings like anxiety or depression, we strongly encourage you to seek support. Counseling and Psychological Services (CaPS) is here to help: call 215-746-WELL (9355) and visit their website at https://wellness.upenn.edu/student-health-and-counseling. Consider reaching out to a friend, faculty or family member you trust for help getting connected to the support that can help.

If you or someone you know is feeling suicidal or in danger of self-harm, call someone immediately, day or night:

CaPS: 215-746-WELL (9355)