(4ap) Reinforcement Learning for the Control of Process Systems

Machine learning has seen significant research activity in the process systems engineering field. I am interested in applying machine learning approaches, especially reinforcement learning, to develop advanced control systems and process models that can be used to improve the performance of a diverse set of systems. I am specifically interested in the theoretical properties of these control approaches when applied to process systems.

Research Interests

• Advanced Process Control

I have experience in working with model predictive control (MPC) and am interested in both applied and theoretical developments in advance process control. My past work has focused on application of advanced controls to energy systems where new operating paradigms are creating the need for adaptive controllers that are effective across a large operating range. I am interested in the intersection of machine learning with process control; there are opportunities for synergetic developments with existing control systems and for novel, standalone control systems. In both cases, machine learning is a powerful tool to exploit the wealth of available process data to generate both better control models and model-free controllers.

• Reinforcement Learning

Most of my work has focused on the application of reinforcement learning (RL) to process control problems. RL is, most commonly, a model-free machine learning paradigm under which an agent learns a better control policy by direct interaction with a system. However, the data-hungry (many interactions are necessary to drive learning) and explorative (sometimes poor actions must be taken) nature of RL create roadblocks for wider application to process systems. I am interested in developing data-efficient approaches for RL by leveraging existing plant data to initialize a near-optimal control policy that can then be refined by an RL approach. Similarly, I want to analyze the properties of an RL agent acting on a process system where some explorative moves may not be allowable and develop algorithms that can operate without loss of stability or significant decreases in the rate of learning.

• Process Modeling

For development of new control systems good process models are almost always necessary. I have experience in the development of both high-fidelity models for individual process units as well as the synthesis of large and highly-integrated process flowsheets, both for steady-state dynamic simulation. While process modeling has largely served as a tool for control development in my work, I am also interested in hybridizing machine learning and first-principles approaches to develop process models that have the performance of reduced-order models that have been identified from data and the predictive properties of more traditional approaches. There are significant opportunities for this kind of approach in energy systems, where a wealth of measurements can be coupled with a predictive model to explore operating spaces that are not commonly seen and to optimize operating parameters in a dynamic setting. Of course, these models will also be a powerful tool for control development.

Teaching Interests

I am interested in teaching any core chemical engineering classes and am specifically interested in teaching in the areas of process control, process design, and process modeling. I would be interested in developing an elective course to introduce machine learning concepts as applied to chemical engineering systems.