Physics-Guided Machine Learning for Embodied AI

Bio

Dr. Huajie Shao is a tenure-track Assistant Professor of Computer Science at William & Mary. Before that, he obtained his Ph.D. degree in Computer Science from the University of Illinois at Urbana-Champaign (UIUC) in 2021. Dr. Shao’s research focuses on the intersection of machine learning/foundation models, control, and embodied AI systems. To date, he has published more than 60 papers in top-tier conferences and journals, including ICML, ICLR, KDD, CVPR, ACL, TPAMI, WWW, and SIGIR. His work has received multiple best paper awards, including the Best Paper Awards at KDD 2024, ACM/IEEE CHASE 2025, CAHSE 2024, SenSys 2020, and ICCPS 2017, as well as the FUSION 2019 Student Paper Award and the UbiComp 2019 Distinguished Paper Award.

Abstract

Embodied AI, such as autonomous vehicles and robotic systems, has transformed everyday life, reshaping how we live, work, and interact with the world. However, a key challenge remains: data-driven machine learning models often struggle to generalize to unknown and dynamic environments. In this talk, I will introduce a series of physics-informed machine learning models that incorporate physical laws into model design to enhance generalization in unseen environments. I will begin by presenting a generalizable, physics-informed state-space model for embodied AI systems based on partially known physics knowledge. I will then discuss WestWorld, a knowledge-encoded trajectory world model for multi-robot learning and control. The proposed model demonstrates superior performance in zero- and few-shot trajectory prediction, as well as in downstream model-based control across various robotics. Finally, I will conclude by outlining potential future directions for advancing world models in embodied AI.