Data and Information Markets Seminar Series

Language Generation in the Limit

November 22, -
Speaker(s): Jon Kleinberg, Cornell University

Lunch

Lunch will be served at 11:45 AM.

Abstract 

Although current large language models are complex, the most basic specifications of the underlying language generation problem itself are simple to state: given a finite set of training samples from an unknown language, produce valid new strings from the language that don't already appear in the training data. Here we ask what we can conclude about language generation using only this specification, without any further properties or distributional assumptions. In particular, we consider models in which an adversary enumerates the strings of an unknown target language that is known only to come from a possibly infinite list of candidate languages, and we show that it is possible to give certain non-trivial guarantees for language generation in this setting. The resulting guarantees contrast dramatically with negative results due to Gold and Angluin in a well-studied model of language learning where the goal is to identify an unknown language from samples; the difference between these results suggests that identifying a language is a fundamentally different problem than generating from it. (This is joint work with Sendhil Mullainathan.)

Speaker Bio

Jon Kleinberg is the Tisch University Professor in the Departments of Computer Science and Information Science at Cornell University. His research focuses on the interaction of algorithms and networks, the roles they play in large-scale social and information systems, and their broader societal implications. He is a member of the National Academy of Sciences, the National Academy of Engineering, the American Academy of Arts and Sciences, and the American Philosophical Society, and he serves on the US National AI Advisory Committee. He has received MacArthur, Packard, Simons, Sloan, and Vannevar Bush research fellowships, as well as awards including the Harvey Prize, the Nevanlinna Prize, the World Laureates Association Prize, and the ACM Prize in Computing.

Web Link

https://duke.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=27b960e8-c05d-4cbd-aeef-b225014d934d

 

 

Co-Sponsor(s)

Duke Computer Science; Fuqua School of Business