(196a) Deep Reinforcement Learning for Model Predictive Control

We propose a deep reinforcement learning algorithm for designing process controllers. The classical approach to designing controllers involves extensive data collection, development of a mathematical model and finally, design of an appropriate controller. This approach can be rather time consuming and requires frequent maintenance of models in response to changes in process conditions. In chemical engineering, there are no good solutions to the controller design and model maintenance problem for complex systems that are nonlinear, high-dimensional and stochastic. We propose an algorithm that uses reinforcement learning to train a deep neural network for learning optimal control policies. One of the key challenges with reinforcement learning is that the learning process often requires tremendous amount of data that is, in practice, difficult to obtain from physical processes. To address that problem. our algorithm samples from a distribution of process observations and uses predictions from a model predictive controller (MPC) that is designed offline. The algorithm maps process observations to control policies and learning is guided by the MPC outputs. Guided policy search reduces training data requirements and has been demonstrated successfully for robotic control [arXiv:1504.00702] and autonomous aerial vehicles [arXiv:1509.06791]. There are several advantages to this approach for designing controllers: (1) Reduction in the size of training data. (2) An explicit model is not required. (3) An explicit control law is not required. (4) Model maintenance is handled by the reinforcement learning algorithm. (5) The algorithm works on complex, non-linear processes.