(4k) Active Learning for Surrogate Model Design in Superstructure Optimization

Moving away from fossil energy carriers and searching for alternatives to safely and sustainably provide energy is one of the key challenges our society is facing. In industrial sectors, efficient resource valorization is crucial to realize the energy transition. Industries converting woody biomass hold great potential for providing storage options of volatile energy carriers and thus support the energy transition, while reducing their emissions by exploiting resources efficiently. Modelling biomass conversion processes and integrating them in industrial systems in an optimal way is a complex task that requires systematic approaches such as computer simulation, mathematical modelling and optimization. Efficient solution generation for such systems is affected by high computational cost and noisy function evaluation of simulation-based optimization models as well as solving issues due to complexity and non-convexity of rigorous first-order process.

Attempting to address the issues described above, interest in developing surrogate models has increased over recent years. Integrated in multi-objective optimization frameworks where many function evaluations are required to generate solutions, new calculations or estimations can be performed in less computational time applying surrogate models than with the original model. Furthermore, they can generalize the model results without convergence issues.

We propose an algorithm for replacing non-linear process simulations in multi-level optimization of energy system superstructures with surrogate models. The main objective of our approach is to provide a flexible and efficient design of surrogate models able to replace systems of varying complexity. Furthermore, the algorithm should be reliable in predicting process characteristics, so the quality of the optimization results is not affected. Lastly, self-learning strategies shall be used to continuously improve the algorithm.

For replacing a process simulation model, an initial set of data points in the input domain of the decision variables of the respective model is sampled and labelled using the original simulation. Multiple deep and machine learning algorithms are trained and evaluated on the initial dataset. In addition to providing an alternative calculation route, all surrogate models are capable of estimating the uncertainty of the result.

The initial set of surrogate models is evaluated based on the achieved test metrics. If one surrogate in the algorithm reaches a user-defined threshold in test prediction, the algorithm is enabled for usage in the optimization framework. If the testing metric threshold is not met, new data points are added to the dataset on which the algorithm is trained and tested. New data points to be added are chosen by generating predictions for the remaining initial sample set. Those with the highest achieved prediction uncertainty are labeled, added, and used for improving the algorithm. Applying this strategy, only relevant sample points are labelled, and unnecessary function evaluations are avoided.

After a desired testing metric quality threshold is reached, the algorithm is authorized for usage in the optimization superstructure. For a process evaluation point given by the optimizer, the algorithm is only used if the predicted uncertainty of the result is satisfying. If the predicted uncertainty is too high, the original simulation model is used instead of the surrogate. This approach limits access the computationally expensive simulation to when the quality of the prediction of the surrogate model is not sufficient. Simultaneously, a continuous improvement of the surrogate models in the algorithm is achieved by using the created simulation results to refine the surrogate model in parallel by adding the created data points to the training set and therefore improve the model's validity range.

It is found that the methodology of continuously adding the data points based on the predicted uncertainty improves the quality of the surrogate models rapidly. Depending on the original simulation model’s complexity, a standardized mean squared error for testing below 2% can be achieved with under 50 function evaluations. Furthermore, it is found that for different simulation models, different types of surrogate models are preferable. Gaussian processes provide better performance for models with few labels and features and for small datasets, while Artificial Neural Networks yield better quality results for more complex models. When applied in the optimization framework, results indicate that the suggested method

Research Interests:

Energy systems, Optimization, Process synthesis, Machine learning

Teaching Interests:

Modelling and optimization of energy systems, Energy conversion systems, Process integration