Title: “Pragmatic Reinforcement Learning in Safety-critical Domains"
Abstract:
In order to develop practical machine learning aided technology for the benefit of human users, it is critical to anchor scientific research and development by the intended real-world use cases. In this talk, I introduce specific modeling decisions that can be made to develop actionable insights from sequentially observed healthcare data, facilitating the avoidance of sub-optimal decisions in patient care. These modeling choices honor underlying data generation as well as the processes by which clinical experts use to formulate their own decisions. Current state-of-the-art reinforcement learning algorithms, when faced with partial information and the inability to proactively experiment or explore within their environment, fail to reliably learn optimal policies. These are common limitations in real-world settings—in conjunction with limited data—it is intractable to learn an optimal policy. However, I present a framework within which recorded negative outcomes can still be useful to identify behaviors that should be avoided. Additionally, I present work that investigates the proper representation learning of healthcare data for use in downstream reinforcement learning algorithms, taking the natural sequential nature of collected observations into account. This allows for rich inductive biases to be learned that supports the formation of both uncertainty estimates over the unobserved features as well as actionable insights regarding the recorded behaviors in the dataset.
Bio:
Taylor completed, in early 2024, his PhD program in Computer Science at University of Toronto, with standing affiliations at Vector Institute and the MIT Institute of Medical Engineering and Sciences. Since then, he has been employed as a postdoctoral Research Scientist within Apple's Special Projects Group. His research broadly investigates novel applications of Reinforcement Learning to assist sequential decision making in safety-critical domains. In particular, he is interested in developing personalized decision support tools that generalize beyond the environment they were trained in, robust to sources of uncertainty such as distribution shift, covariate mismatch, and missing data. Taylor has prior degrees in Computational Science and Engineering (M.Eng, Harvard University) as well as Mathematics (B.S., Brigham Young University) that he has used in prior research positions investigating the RF scheduling algorithms, sensor placement (MIT Lincoln Laboratory), and in the modeling of fluid phenomenon (BYU R.A.).