Learning Machines Seminar Series

What: LMSS: Xiaolong Wang (UC San Diego)
When: Thursday, Sep 19, 12:00 p.m. to 1:00 p.m.
Where: Bloomberg 81, Bloomberg Center, Cornell Tech (map)

The series is organized by Associate Professor Yoav Artzi and sponsored by Bloomberg.

Pizza will be served at 11:40 a.m.

"Learning Humanoid Robots"

Having a humanoid robot operating like a human has been a long-standing goal in robotics. The humanoid robot provides a generalized purpose platform to conduct diverse tasks we do in our daily lives. In this talk, we study learning-based approaches for both the mobility and manipulation skills of the humanoid robot, with the goal of generalization to diverse tasks, objects, and scenes. I will discuss how to perform whole-body control in humanoids with rich, diverse, and expressive motions. I will also share some lessons we learned from developing three different teleoperation systems to operate the humanoid robots and collect training data. With the collected data, we aim to build the robot foundation model with Test-Time Training (TTT). 

The applications go much beyond moving boxes in the factory. The humanoid robot provides a generalized purpose platform to conduct diverse tasks we do in our daily lives. In this talk, we study learning-based approaches for both the mobility and manipulation skills of the humanoid robot, with the goal of generalization to diverse tasks, objects, and scenes. We will first discuss how to perform whole-body control in humanoids to generate rich, diverse, and expressive motions. We then discuss what will be the foundation model for the robot to perceive the visual world and perform mobile manipulation in the wild. Finally, we will dive into the humanoid’s manipulator, the hands, and study tactile sensing for flexible manipulations.