Generative AI Meets Music Theory

Lunch will be served at 11:45AM.

Bio:

Dr. Stephen Ni-Hahn is a Postdoctoral Associate at Duke University, and holds a PhD in Electrical and Computer Engineering from Duke University. His research investigates generative AI for computational creativity, specifically on incorporating domain knowledge to enable greater interpretability and controllability in AI music systems. His work has been featured and published broadly in many top machine learning and music conferences such as NeurIPS, AAAI, KDD, ISMIR, and SMT.

Abstract:

The intersection of artificial intelligence and music is vast and quickly evolving, with models such as Suno enabling amateur music enthusiasts to generate impressive compositions with a simple text prompt. There is further an urgent need for AI systems that humans can control. Unfortunately, the vast majority of recent systems rely on black box models trained on massive datasets, heavily limiting themselves based on available data. Furthermore, most models are expected to learn complex notions like music-theoretical structure without any guidance, leading to poor consistency and a lack of large-scale structure. My research addresses these limitations by integrating domain knowledge from music theory to enhance interpretability and human controllability for AI music generation, enabling the generation of coherent and enjoyable music that outperforms much larger state-of-the-art deep learning models. In this talk, I will focus on three methods that facilitate this integration: SchenkComposer, a theory-based framework for hierarchical melody generation; AutoSchA, which build on recent developments in Graph Neural Networks, enabling AI models to interpret deeper musical connections in a more human way; and E-Motion Baton, a real-time human-in-the-loop conducting simulator that incorporates human emotion and gesture for responsive music generation.