(360j) Deep Reinforcement Learning As a Tool to Enable Coarse Grained Vaccine Models

Traditional approaches to vaccine development have yet to effectively compete with highly mutable infectious disease pathogens (hm-IDPs) such as HIV and influenza which present with rapidly evolving viral etiologies, which the human immune system cannot successfully manage alone. The limitations of these traditional approaches, paired with hm-IDPs ability to rapidly develop to bypass these efforts compound to one million deaths annually. To address the need for more effective vaccine development, computational agent-based models may be employed to simulate the complex interactions that occur between human immune cells and hm-IDP-like proteins (antigens) during affinity maturation—the process by which antibodies evolve. In contrast to existing experimental approaches, these computational models offer a safe, low-cost, and rapid means to study and assess the human immune response to vaccines, incorporating a wide range of design variables. However, the highly variable nature of affinity maturation and vast sequence space of hm-IDPs render standard agent-based modeling approaches alone insufficient for exploring all pertinent vaccine design variables and the subset of immunization protocols encompassed therein. To address this continuously diversifying sampling challenge, and to garner in tandem a better understanding of factors essential to vaccine development, we employed deep reinforcement learning to drive a recently developed coarse-grained agent-based model of affinity maturation to sample specific immunization protocols with the potential to improve the chosen metrics of protection (e.g., antibody neutralization potency or neutralizing antibody titers). With this approach, we were able to explore the relevant space of a wide range of vaccine design variables, giving rise to novel and testable insights into how vaccines should be developed to maximize protective immune responses to hm-IDPs in a manner that is more apt and flexible than traditional models. Furthermore, elucidating how these vaccine development models might be minimally tailored to account for major sources of heterogeneity in human immune responses.