Learning Machines Seminar Series

What: LMSS @ Cornell Tech: Tom Kwiatkowski (Google)
When: Friday, February 21st, 12:15 p.m. (lunch served at 12pm)
Where: Room 081, Bloomberg Center, Cornell Tech (map)

"New Challenges in Question Answering: Natural Questions and Going Beyond Word Matching" 

Recently, learned deep models have surpassed human performance on a number of question answering benchmarks such as SQuAD. However, these models resort to simple word matching and answer typing heuristics, and they are easily fooled. In this talk, I will present two different lines of work that aim to take us beyond the current status quo in question answering, and push us towards more robust representations of semantics, pragmatics, and knowledge.

First, I will present Natural Questions (NQ), a new question answering benchmark from Google. NQ contains real user questions, which require an understanding of the questioner's intent. NQ also requires systems to read entire Wikipedia pages to decide whether they fully answer a question. This is much harder than finding an answer given the knowledge that one is present. I will convince you that the question 'when was the last time a hurricane hit massachusetts?' is under-specified with many reasonable answers, and I will tell you how we developed robust evaluation metrics to deal with this ambiguity.

In the second part of the talk I will present a complementary method of challenging today's question answering systems by removing access to evidence documents at inference time. Instead of building joint representations of questions and documents, we perform ahead of inference time reading and retrieve answers via fast maximum inner product search. I will show that this leads to large gains in accuracy and speed when finding answers in very large corpora. I will also show some preliminary results that show how our methods can be used to aggregate information from multiple diverse documents.
 

BIO

Tom Kwiatkowski is a Research Scientist in Google's New York office. His focus is on building representations of the knowledge that is expressed in text, and he has a particular interest in modeling the ways in which different texts agree and disagree with each other. Tom has worked on question answering products at Google, as well as engaging with academia. Before joining Google he did a PhD in Edinburgh with Sharon Goldwater and Mark Steedman, and a post-doc at the University of Washington with Luke Zettlemoyer. Both Tom's PhD and post-doc had a focus on semantic parsing and grammar induction.