Learning Machines Seminar Series

What: LMSS: Raymond Mooney
When: Friday, March 8, 12:30 p.m.
Where: Bloomberg 061, Bloomberg Center, Room 061, Cornell Tech (map)

The series is organized by Associate Professor Yoav Artzi and sponsored by Bloomberg.

Pizza will be served at 12 p.m.

“Using Natural Language to Help Train Robots”
 
To effectively deploy robots in many environments, the end-user must teach robots new skills with just a few demonstrations. This talk will focus on two key areas we have investigated to use language to accelerate robot learning in this few-shot setting. First, we show how to improve task learning by using both language instructions and demonstrations. We show that providing natural language instructions along with a single demonstration has large sample efficiency gains when learning novel tasks over previous methods that teach robots with only one of these two modalities. A second application of language for robotics that we explore is improving sim2real transfer. Effectively teaching robots many tasks requires extensive training in simulation followed by rapid transfer to the real world.  We show that pretraining visual representations with language descriptions in both simulated and real environments enables learning algorithms to better leverage plentiful, cheap simulation data when transferring to a few real-world demonstrations, boosting sim2real performance even with a large visual domain gap.

BIO

Raymond J. Mooney is a Professor in the Department of Computer Science at the University of Texas at Austin. He received his Ph.D. in 1988 from the University of Illinois at Urbana/Champaign. He is an author of over 200 published research papers, primarily in the areas of machine learning and natural language processing. He was the President of the International Machine Learning Society from 2008-2011, program co-chair for AAAI 2006, general chair for HLT-EMNLP 2005, and co-chair for ICML 1990. He is a Fellow of AAAI, ACM, and ACL and the recipient of the Classic Paper award from AAAI-19 and best paper awards from AAAI-96, KDD-04, ICML-05 and ACL-07.