(342d) Model Parameter Uncertainty Revisited

Emergence of the QbD initiative and design space concepts more than a decade ago led to productive discussions, research and publications about how to incorporate uncertainty when using mathematical models derived from experimental data.

Peterson (1) helped to elucidate how probabilistic tools could be used to define a design space using the concept of reliability. Albrecht (2) showed how Markov Chain Monte Carlo methods produced more realistic kinetic parameter confidence regions than conventional maximum likelihood approaches when measurement errors were biased. Garcia Munoz et al (3) sampled the model parameter covariance matrix to construct a probabilistic design space. The impact of uncertainty on process responses was estimated using asymptotic confidence and prediction bands in Figueroa et al (4). Best practices in this field are still evolving, including interesting techniques presented this year in (5).

There have been relatively few published examples where several techniques for modeling uncertainty have been applied and compared for the complex reaction kinetic models typical of pharma organic synthesis. We have therefore applied a range of techniques for quantifying uncertainty in these cases.

In this presentation we compare parameter estimates obtained with rapid methods such as maximum likelihood and sampling the parameter covariance matrix with more CPU intensive techniques such as parametric bootstrapping to estimate the posterior distribution of the parameters.

We propagate uncertainty with each method into simulations of multiple responses and compare rapidly calculated asymptotic confidence bands with sampling the covariance matrix and parametric bootstrapping.

Finally we calculate and compare design spaces created using each method to estimate the joint probability of meeting specific process goals.

Other than differences in CPU time, the techniques lead to interesting variations in parameter values and predicted responses, some of which are important for setting realistic expectations for future measured data and model accuracy in general. The results from each technique also vary somewhat depending on some relatively subtle choices and assumptions made in their application.

We have applied the same techniques to other common parameter estimation models for applications such as filtration and heat transfer and will include a brief discussion of these results.

References

1. Peterson, J. J. J. Biopharm. Stat. 2008, 18, 959−975.
2. Albrecht, Jacob. (2013). Estimating reaction model parameter uncertainty with Markov Chain Monte Carlo. Computers & Chemical Engineering. 48. 14–28. 10.1016/j.compchemeng.2012.07.011.
3. Garcia-Munoz, Salvador & Luciani, Carla & Vaidyaraman, Shankar & Seibert, Kevin. (2015). Definition of Design Spaces Using Mechanistic Models and Geometric Projections of Probability Maps. Organic Process Research & Development. 19. 15071609505300, 10.1021/acs.oprd.5b00158.
4. Isabel Figueroa, Shankar Vaidyaraman, and Shekhar Viswanath, Model-Based Scale-up and Design Space Determination for a Batch Reactive Distillation with a Dean–Stark Trap, Organic Process Research & Development 2013 17 (10), 1300-1310, DOI: 10.1021/op4001127
5. Laky, Daniel & Xu, Shu & S. Rodriguez, Jose & Vaidyaraman, Shankar & García Muñoz, Salvador & Laird, Carl. (2019). An Optimization-Based Framework to Define the Probabilistic Design Space of Pharmaceutical Processes with Model Uncertainty. Processes. 7. 96. 10.3390/pr7020096.