Lunch will be served at 11:45 AM.
The last decade saw a dramatic shift in how NLP systems are developed and deployed: from bespoke models trained for specific tasks to a single Large Language Model (LLM) capable of solving a variety of tasks via in-context learning (ICL). However, smaller fine-tuned models often outperform ICL approaches for applications in specialized or knowledge-intensive domains (e.g., biomedicine). Moreover, one-size-fits-all models are insufficient to address the specific needs of diverse populations and individuals (e.g., LLMs are known to encode harmful social biases with implications for algorithmic fairness). While techniques such as Reinforcement Learning from Human Feedback can help in aligning models to human values and preferences, it is unclear whose values and preferences are captured by the reward models and whether they are representative of the user population. These approaches are thus brittle and fundamentally limited in that, at best, they align models to the average feedback from human annotators.
In this talk, I will discuss the problem of adapting LLMs to out-of-distribution tasks and datasets. I will first present some of our recent work on tailoring LLMs for information extraction tasks in the clinical and biomedical domains, with varying degrees of supervision. Then, I will describe ongoing efforts aimed at understanding how LLMs encode and utilize demographic information, which can inform strategies to align models to the needs of specific populations. I will conclude by outlining a vision for a world where each person can have a personalized LLM tailored to their individual needs and (dis-)abilities and discuss both key challenges and potential strategies to realize this vision.
Silvio Amir is an assistant professor at the Khoury College of Computer Science at Northeastern University. He completed his PhD at the University of Lisbon and was a postdoc at Johns Hopkins University before joining Northeastern. Silvio Amir works on Natural Language Processing and Machine Learning methods to analyze personal and user generated text, such as social media and clinical notes from Electronic Health Records. He is primarily interested in tasks involving subjective, personalized or user-level inferences (e.g. opinion mining and digital phenotyping). In particular, his work aims to improve the reliability, interpretability and fairness of predictive models and analytics derived from these data.
More broadly, Amir's research is part of ongoing efforts to develop Human-centered AI (i.e. to empower rather than replace humans) and AI for Social Good (i.e. to tackle meaningful social, societal and humanitarian challenges). To achieve these goals, he often collaborates with domain experts in multidisciplinary projects to address real-world problems in the social sciences, medicine and epidemiology.
Department of Computer Science