We consider the problem of chance constrained optimization where it is sought to optimize a function and satisfy constraints, both of which are affected by uncertainties. The real world declinations of this problem are particularly challenging because of their inherent computational cost.
To tackle such problems, we propose a new Bayesian optimization method. It applies to the situation where the uncertainty comes from some of the inputs, so that it becomes possible to define an acquisition criterion in the joint optimized-uncertain input space. The main contribution of this work is an acquisition criterion that accounts for both the average improvement in objective function and the constraint reliability. The criterion is derived following the Stepwise Uncertainty Reduction logic and its maximization provides both optimal design variables and uncertain parameters. Analytical expressions are given to efficiently calculate the criterion. Numerical studies on test functions are presented. It is found through experimental comparisons with alternative sampling criteria that the adequation between the sampling criterion and the problem contributes to the efficiency of the overall optimization.
Reda El Amri 1; Rodolphe Le Riche 2; Céline Helbert 3; Christophette Blanchet-Scalliet 3; Sébastien Da Veiga 4
CC-BY 4.0
@article{SMAI-JCM_2023__9__285_0,
author = {Reda El Amri and Rodolphe Le Riche and C\'eline Helbert and Christophette Blanchet-Scalliet and S\'ebastien Da Veiga},
title = {A {Sampling} {Criterion} for {Constrained} {Bayesian} {Optimization} with {Uncertainties}},
journal = {The SMAI Journal of computational mathematics},
pages = {285--309},
publisher = {Soci\'et\'e de Math\'ematiques Appliqu\'ees et Industrielles},
volume = {9},
year = {2023},
doi = {10.5802/smai-jcm.102},
language = {en},
url = {https://smai-jcm.centre-mersenne.org/articles/10.5802/smai-jcm.102/}
}
TY - JOUR AU - Reda El Amri AU - Rodolphe Le Riche AU - Céline Helbert AU - Christophette Blanchet-Scalliet AU - Sébastien Da Veiga TI - A Sampling Criterion for Constrained Bayesian Optimization with Uncertainties JO - The SMAI Journal of computational mathematics PY - 2023 SP - 285 EP - 309 VL - 9 PB - Société de Mathématiques Appliquées et Industrielles UR - https://smai-jcm.centre-mersenne.org/articles/10.5802/smai-jcm.102/ DO - 10.5802/smai-jcm.102 LA - en ID - SMAI-JCM_2023__9__285_0 ER -
%0 Journal Article %A Reda El Amri %A Rodolphe Le Riche %A Céline Helbert %A Christophette Blanchet-Scalliet %A Sébastien Da Veiga %T A Sampling Criterion for Constrained Bayesian Optimization with Uncertainties %J The SMAI Journal of computational mathematics %D 2023 %P 285-309 %V 9 %I Société de Mathématiques Appliquées et Industrielles %U https://smai-jcm.centre-mersenne.org/articles/10.5802/smai-jcm.102/ %R 10.5802/smai-jcm.102 %G en %F SMAI-JCM_2023__9__285_0
Reda El Amri; Rodolphe Le Riche; Céline Helbert; Christophette Blanchet-Scalliet; Sébastien Da Veiga. A Sampling Criterion for Constrained Bayesian Optimization with Uncertainties. The SMAI Journal of computational mathematics, Volume 9 (2023), pp. 285-309. doi : 10.5802/smai-jcm.102. https://smai-jcm.centre-mersenne.org/articles/10.5802/smai-jcm.102/
[1] Set inversion under functional uncertainties with joint meta-models (2020) preprint available on HAL as document hal-02986558 (https://hal-ifp.archives-ouvertes.fr/hal-02986558 )
[2] Gradient-based simulation optimization under probability constraints, Eur. J. Oper. Res., Volume 212 (2011) no. 2, pp. 345-351 | DOI | MR | Zbl
[3] Studies in linear and non-linear programming, 2, Stanford University Press, 1958
[4] 3rd GECCO Workshop for Real-Parameter Optimization, https://coco.gforge.inria.fr/doku.php?id=bbob-2012-results, 2012
[5] Overview of Problem Formulations and Optimization Algorithms in the Presence of Uncertainty, Aerospace System Analysis and Optimization in Uncertainty, Springer, 2020, pp. 147-183 | DOI
[6] Adaptive modeling strategy for constrained global optimization with application to aerodynamic wing design, Aerosp. Sci. Technol., Volume 90 (2019), pp. 85-102 | DOI
[7] Sequential design of computer experiments for the estimation of a probability of failure, Stat. Comput., Volume 22 (2012) no. 3, pp. 773-793 | DOI | MR | Zbl
[8] Robust optimization, 28, Princeton University Press, 2009 | DOI
[9] Robust optimization–a comprehensive survey, Comput. Methods Appl. Mech. Eng., Volume 196 (2007) no. 33-34, pp. 3190-3218 | DOI | MR | Zbl
[10] Reliability analysis and optimal design under uncertainty-Focus on adaptive surrogate-based approaches, Habilitation à diriger les recherches, Univ. Clermont-Auvergne (2018) (https://hal.archives-ouvertes.fr/tel-01737299)
[11] Mirrored sampling and sequential selection for evolution strategies, PPSN 2010: Parallel Problem Solving from Nature, PPSN XI, Springer (2010), pp. 11-21
[12] Fast uncertainty reduction strategies relying on Gaussian process models, Ph. D. Thesis, University of Bern (2013)
[13] Corrected kriging update formulae for batch-sequential data assimilation, Mathematics of Planet Earth, Springer, 2014, pp. 119-122 | DOI
[14] Reliability-based design optimization using kriging surrogates and subset simulation, Struct. Multidiscip. Optim., Volume 44 (2011) no. 5, pp. 673-690 | DOI
[15] AK-MCS: an active learning reliability method combining Kriging and Monte Carlo simulation, Structural Safety, Volume 33 (2011) no. 2, pp. 145-154 | DOI
[16] EFISUR, a new acquisition function, https://github.com/elamrireda/EFISUR, 2021
[17] Scalable constrained Bayesian optimization, Proceedings of the 24th International Conference on Artificial Intelligence and Statistics (PMLR), Volume 130, 2021, pp. 730-738
[18] AK-SYS: an adaptation of the AK-MCS method for system reliability, Reliability Engineering & System Safety, Volume 123 (2014), pp. 137-144 | DOI
[19] Recent advances in robust optimization: An overview, Eur. J. Oper. Res., Volume 235 (2014) no. 3, pp. 471-483 | DOI | MR | Zbl
[20] Bayesian Optimization, Cambridge University Press, 2022
[21] Predictive Entropy Search for Bayesian Optimization with Unknown Constraints, ICML’15: Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37, JMLR, 2015, pp. 1699-1707
[22] et al. Uncertainty Management for Robust Industrial Design in Aeronautics, Springer, 2019 | DOI
[23] Simultaneous kriging-based estimation and optimization of mean response, J. Glob. Optim. (2012) | DOI
[24] et al. Stochastic estimation of the maximum of a regression function, Ann. Math. Stat., Volume 23 (1952) no. 3, pp. 462-466 | DOI | MR | Zbl
[25] Adam: A Method for Stochastic Optimization, 2014 | arXiv
[26] Optimization under uncertainties: an overview with a focus on Gaussian processes (2019) (lecture at the CNRS French-German University school on Modeling and Numerical Methods for Uncertainty Quantification, cf. https://hal.archives-ouvertes.fr/cel-02285533)
[27] Quality through design: Experimental design, off-line quality control, and Taguchi’s contributions, Oxford University Press, 1989
[28] Self-adaptive surrogate-assisted covariance matrix adaptation evolution strategy, Proceedings of the 14th GECCO, ACM Press (2012), pp. 321-328 | DOI
[29] A new expected-improvement algorithm for continuous minimax optimization, J. Glob. Optim., Volume 64 (2016), pp. 785-802 | DOI | MR | Zbl
[30] Quantile-based optimization under uncertainties using bootstrap polynomial chaos expansions, Proceedings of the 12th International Conference on Structural Safety and Reliability (ICOSSAR 2017), TU Verlag (2017), pp. 1561-1569
[31] Quantile-based optimization under uncertainties using adaptive Kriging surrogate models, Struct. Multidiscip. Optim., Volume 54 (2016) no. 6, pp. 1403-1421 | DOI | MR
[32] On safe tractable approximations of chance constraints, Eur. J. Oper. Res., Volume 219 (2012) no. 3, pp. 707-718 | DOI | MR | Zbl
[33] Introduction to vector quantization and its applications for numerics, ESAIM, Proc. Surv., Volume 48 (2015), pp. 29-79 | DOI | MR | Zbl
[34] Robust Design: An Overview, AIAA J., Volume 44 (2006), pp. 181-191 | DOI
[35] Efficient global optimization of constrained mixed variable problems, J. Glob. Optim., Volume 73 (2019) no. 3, pp. 583-613 | DOI | MR | Zbl
[36] Constrained bayesian optimization with max-value entropy search (2019) | arXiv
[37] A Stepwise uncertainty reduction approach to constrained global optimization, Proceedings of the 17th International Conference on Artificial Intelligence and Statistics (Samuel Kaski; Jukka Corander, eds.) (Proceedings of Machine Learning Research), Volume 33, PMLR (2014), pp. 787-795
[38] A direct search optimization method that models the objective and constraint functions by linear interpolation, Advances in optimization and numerical analysis, Springer, 1994, pp. 51-67 | DOI | Zbl
[39] The BOBYQA algorithm for bound constrained optimization without derivatives, Cambridge NA Report NA2009/06, University of Cambridge, 2009, pp. 26-46
[40] L’incertitude en conception: formalisation, estimation, Optimisation multidisciplinaire en mécanique: Réduction de modèles, robustesse, fiabilité, réalisations logicielles (Rajan Filomeno Coelho; Piotr Breitkopf, eds.), Hermes Science Publications, 2009
[41] Spectral representation of robustness measures for optimization under input uncertainty, ICML2022, the 39th International Conference on Machine Learning (2022), pp. 1-26
[42] Robust optimization: A kriging-based multi-objective optimization approach, Reliability Engineering & System Safety, Volume 200 (2020), 106913 | DOI
[43] Group kernels for Gaussian process metamodels with categorical inputs, SIAM/ASA J. Uncertain. Quantif. (2019)
[44] Exploration of metamodeling sampling criteria for constrained global optimization, Eng. Optim., Volume 34 (2002) no. 3, pp. 263-278 | DOI
[45] Global optimization of problems with disconnected feasible regions via surrogate modeling, 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization (2002), 5573 | DOI
[46] Global versus local search in constrained optimization of computer models, New developments and applications in experimental design (IMS Lecture Notes Monogr. Ser.), Volume 34, Institute of Mathematical Statistics, 1998, pp. 11-25 | DOI | MR
[47] Computational methods in optimization considering uncertainties–an overview, Comput. Methods Appl. Mech. Eng., Volume 198 (2008) no. 1, pp. 2-13 | DOI | Zbl
[48] Introduction to stochastic search and optimization: estimation, simulation, and control, 65, John Wiley & Sons, 2005
[49] X-Armed Bandits: Optimizing Quantiles, CVaR and Other Risks, Asian Conference on Machine Learning, PMLR (2019), pp. 252-267
[50] A review on quantile regression for stochastic computer experiments, Reliability Engineering & System Safety (2020), 106858 | DOI
[51] A survey on approaches for reliability-based optimization, Struct. Multidiscip. Optim., Volume 42 (2010) no. 5, pp. 645-663 | DOI | MR | Zbl
[52] Constrained Two-step Look-Ahead Bayesian Optimization, Adv. Neural Inf. Process. Syst., Volume 34 (2021), pp. 12563-12575
[53] Analysis of adaptive directional stratification for the controlled estimation of rare event probabilities, Stat. Comput., Volume 22 (2012) no. 3, pp. 809-821 | DOI | MR | Zbl
Cited by Sources: