(243a) Elastic Net with Monte Carlo Sampling for Data-Based Modeling in Biopharmaceutical Manufacturing Facilities

Biopharmaceutical manufacturing
involves multiple process steps that can be challenging to model using first
principles. Oftentimes, operating conditions are studied in bench-scale
experiments and then fixed to specific values during full-scale operations.
This procedure limits the opportunity to tune process variables to correct for
the effects of disturbances. Utilizing process models has the potential to
increase the flexibility and controllability of the biomanufacturing processes.
This work proposes a statistical modeling methodology to predict the outputs of
biopharmaceutical operations. This methodology addresses two important
challenging characteristics typical of data collected in the biopharmaceutical
industry: limited data availability and data heterogeneity. Motivated by the
final aim of control, regularization methods, specifically the elastic net, are
combined with sampling techniques similar to the bootstrap to develop
mathematical models that use only a small number of input variables. These techniques
are of particular interest because of their ability to perform model selection
and estimation simultaneously.

Process modeling techniques can be
grouped into two broad categories: first-principles and data-based. This work
focuses on data-based modeling, which is more often applied in biopharmaceutical
manufacturing facilities. Data-based models have been applied to cell culture
characterization [1] [2] [3], quality control [4] [5], process monitoring [3] [6]
[7] [8], and downstream operations [3]. A drawback of current data-based
methods applied in the biopharmaceutical industry is that the models that are
produced are not easily interpretable because they rely on subspaces that do
not have direct physical meaning.

A successful biopharmaceutical model
was defined as achieving three goals: (1) model accuracy, (2) model simplicity,
and (3) model interpretability. These aims have the caveat of using only a small
amount of heterogeneous data, as data for biopharmaceutical manufacturing are
typically both heterogeneous and relatively limited.

One way to achieve these goals is
through the identification of the input variables in the process that exhibit
the largest effects on the output variables. Regularization methods have been
identified as possible approaches for such problems because of their ability to
simultaneously handle input selection and model estimation [9]. A particular
regularization method, the elastic net [10], was identified because of its
ability to handle data with more measurements than observations. Elastic net is an optimization
formulation for parameter estimation that is formulated as:

,                                                                 (1)

where

,                                                                                                             (2)

N is the number of experiments, y_i
is the ith scalar response, x_i is the p-dimensional
data vector at observation i, λ is a nonnegative
regularization parameter, β₀ is a scalar parameter, β
is a p-dimensional vector of model parameters, and α is on
the interval (0,1]. Using this basis, a five-step methodology, referred to as
the elastic net with Monte Carlo sampling (ENwMC) is proposed.

The first step in ENwMC is an
application of the elastic net, using leave-one-out cross validation to choose
the value of α. In leave-one-out cross validation, all but one of
the experimental observations are used to fit the model, then the remaining
experiment is used to calculate the error. This step is repeated for each
possible set and then averaged. The procedure is performed for many possible
combinations of the regularization parameters α and λ,
where α  and λ captures the convex behavior of the
error. Because α is the weighting between the ℓ₂- and ℓ₁-norm penalties and the goal is a
sparse model, a value of α close to 1 is preferable. Therefore α
is chosen based on a tradeoff between model dimensionality and prediction error.
In some cases, this choice is trivial, as a higher value leads to a more
accurate model.

Once the value of α is
fixed, a test for over-fitting is performed using k-fold cross
validation. Using Monte Carlo samples [11], the data are portioned into a validation
set containing 1/k proportion of the data and a calibration set
containing the rest. The elastic net, with a fixed α, is then
performed and the input variables corresponding to the minimum error are
recorded. This step is repeated many times to converge to the distribution of
models over the possible calibration and validation sets. The frequency with
which each variable is selected is then calculated. In further analysis, only
the variables that were selected above a threshold frequency are considered.

The subset of selected variables is
considered for inclusion in a model using best subset selection. The error of
all possible ordinary least squares models of size m , where p is now the dimensions that were
chosen based on the threshold, is calculated. A model from this set is then
selected based the tradeoff between increasing dimensionality and decreasing
error. This tradeoff is easily visualized by plotting the prediction error
against the model dimensions to create a Pareto curve. Plots of this type will
often exhibit an ?elbow.? The elbow corresponds to the model dimensionality
that compromises between model size and prediction error. The result of this
step is the final model.

Figure 1: Simplified
flowsheet of the antibody production process. The bioreactor volume was 2000L
and was operated in the fed-batch mode. The column loadings were typical of an
antibody purification process. [12].

The developed methodology is
evaluated on an antibody manufacturing dataset (see Figure 1) and compared to
well-known multivariate analysis techniques for the (bio)pharmaceutical field:
principal component regression (PCR) and partial least squares (PLS). In a
majority of cases, the elastic net technique out-performed PCR and PLS in terms
of error and variance (see Table 1). Averaged over all of the output variables
that were considered, the sum-of-squared errors decreased 27% and the variance
decreased 48% using a regularized model as compared to the latent variable
models. The regularized models have the added benefit of being easily
interpreted in terms of the process variables.

Table 1: Comparisons of scaled
error and variance for PCR, PLS, and ENwMC modeling techniques. The bold number
marks the model with the best performance for each variable.

References