Learning Machines Seminar Series

What: LMSS: Andre F. T. Martins (Tecnico Lisboa)
When: Wednesday, March 5, 2:45-3:45 pm
Where: Bloomberg 301, Bloomberg Center, Cornell Tech (map)

The series is organized by Associate Professor Yoav Artzi and sponsored by Bloomberg.

"Quality-Aware Generation: Reranking Laws and Insights from Communication Theory"

In this talk, I provide a unified perspective of quality-aware generation, a methodology for test-time scaling which has been developed in parallel by different communities with different applications in mind (machine translation, language model reasoning, code generation). In the first part, I introduce quality-aware sampling, a simple method for generating samples from a Gibbs distribution induced by a reward model, through the Metropolis-Hastings algorithm. In the second part, I provide a communication-theoretic perspective of generator-reranker systems. Reranking (or best-of-N) is a commonly used strategy for making large language models (LLMs) more accurate and for reducing hallucination rates, but to which extent are they able to do so? We draw a parallel between this strategy and the use of redundancy to decrease the error rate in noisy communication channels. We conceptualize the generator as a sender transmitting multiple descriptions of a message through parallel noisy channels. The receiver decodes the message by ranking the (potentially corrupted) descriptions and selecting the one found to be most reliable. We provide conditions under which this protocol is asymptotically error-free even in scenarios where the reranker is imperfect (governed by Mallows or Zipf-Mandelbrot models) and the channel distributions are statistically dependent. We use this framework to obtain reranking laws validated empirically on real-world tasks using LLMs (text-to-code generation, math and commonsense reasoning, and machine translation). 

BIO

André F. T. Martins is an Associate Professor at Instituto Superior Técnico, University of Lisbon, researcher at Instituto de Telecomunicações, and the VP of AI Research at Unbabel. His research, funded by a ERC Starting Grant (DeepSPIN) and Consolidator Grant (DECOLLAGE), among other grants, include machine translation, quality estimation, structure and interpretability in deep learning systems for NLP. His work has received several paper awards at ACL conferences. He co-founded and co-organizes the Lisbon Machine Learning School (LxMLS), and he is a Fellow of the ELLIS society and co-director of the ELLIS Program in Natural Language Processing.