Bayesian optimization: Difference between revisions

Revision as of 06:17, 9 October 2024

Revision as of 06:19, 15 December 2024

Line 9:

| isbn=978-1107163447

| url = https://www.probabilistic-numerics.org/assets/ProbabilisticNumerics.pdf

}}</ref> that does not assume any functional forms. It is usually employed to optimize expensive-to-evaluate functions. With the rise of [[artificial intelligence]] innovation in the 21st century, Bayesian optimizations have found prominent use in [[machine learning]] problems, for optimizing hyperparameter values.<ref>{{cite journal |first=Jasper |last=Snoek |title=Practical Bayesian Optimization of Machine Learning Algorithms |journal=Advances in Neural Information Processing Systems 25 (NIPS 2012) |year=2012 |volume=25 |arxiv=1206.2944 |url=https://proceedings.neurips.cc/paper/2012/hash/05311655a15b75fab86956663e1819cd-Abstract.html}}</ref><ref>{{cite journal |first=Aaron |last=Klein |title=Fast bayesian optimization of machine learning hyperparameters on large datasets |journal=Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, PMLR |year=2017 |pages=528-536528–536 |arxiv=1605.07079 |url=https://proceedings.mlr.press/v54/klein17a.html}}</ref>

==History==

Line 51:

| publisher = ACM

| title = Proceedings of the 9th International Conference on Information Processing in Sensor Networks, IPSN 2010, April 12–16, 2010, Stockholm, Sweden

| year = 2010| isbn = 978-1-60558-988-6

| year = 2010}}</ref> automatic algorithm configuration,<ref>Frank Hutter, Holger Hoos, and Kevin Leyton-Brown (2011). [http://www.cs.ubc.ca/labs/beta/Projects/SMAC/papers/11-LION5-SMAC.pdf Sequential model-based optimization for general algorithm configuration], Learning and Intelligent Optimization</ref><ref>J. Snoek, H. Larochelle, R. P. Adams [https://papers.nips.cc/paper/4522-practical-bayesian-optimization-of-machine-learning-algorithms.pdf Practical Bayesian Optimization of Machine Learning Algorithms]. Advances in Neural Information Processing Systems: 2951-2959 (2012)</ref> [[Automated machine learning|automatic machine learning]] toolboxes,<ref>J. Bergstra, D. Yamins, D. D. Cox (2013).

[http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.704.3494&rep=rep1&type=pdf Hyperopt: A Python Library for Optimizing the Hyperparameters of Machine Learning Algorithms].

Proc. SciPy 2013.</ref><ref>Chris Thornton, Frank Hutter, Holger H. Hoos, Kevin Leyton-Brown: [https://arxiv.org/abs/1208.3719 Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms]. KDD 2013: 847–855</ref><ref>Jasper Snoek, Hugo Larochelle and Ryan Prescott Adams. [https://papers.nips.cc/paper/4522-practical-bayesian-optimization-of-machine-learning-algorithms.pdf Practical Bayesian Optimization of Machine Learning Algorithms]. Advances in Neural Information Processing Systems, 2012</ref> [[reinforcement learning]],<ref>{{Cite thesis |title=Safe Exploration in Reinforcement Learning: Theory and Applications in Robotics |url=https://www.research-collection.ethz.ch/handle/20.500.11850/370833 |publisher=ETH Zurich |date=2019 |degree=Doctoral Thesis |doi=10.3929/ethz-b-000370833 |language=en |first=Felix |last=Berkenkamp|hdl=20.500.11850/370833 }}</ref> planning, visual attention, architecture configuration in [[deep learning]], static program analysis, experimental [[particle physics]],<ref>Philip Ilten, Mike Williams, Yunjie Yang. [https://arxiv.org/abs/1610.08328 Event generator tuning using Bayesian optimization]. 2017 JINST 12 P04028. DOI: 10.1088/1748-0221/12/04/P04028</ref><ref>Evaristo Cisbani et al. [https://iopscience.iop.org/article/10.1088/1748-0221/15/05/P05009 AI-optimized detector design for the future Electron-Ion Collider: the dual-radiator RICH case] 2020 JINST 15 P05009. DOI: 10.1088/1748-0221/15/05/P05009</ref> quality-diversity optimization,<ref>{{Cite arXiv |last1=Kent |first1=Paul |last2=Gaier |first2=Adam |last3=Mouret |first3=Jean-Baptiste |last4=Branke |first4=Juergen |date=2023-07-19 |title=BOP-Elites, a Bayesian Optimisation Approach to Quality Diversity Search with Black-Box descriptor functions |class=math.OC |eprint=2307.09326}} Preprint: Arxiv.</ref><ref>{{Cite book |last1=Kent |first1=Paul |last2=Branke |first2=Juergen |title=Proceedings of the Genetic and Evolutionary Computation Conference |chapter=Bayesian Quality Diversity Search with Interactive Illumination |date=2023-07-12 |chapter-url=https://dl.acm.org/doi/10.1145/3583131.3590486 |series=GECCO '23 |___location=New York, NY, USA |publisher=Association for Computing Machinery |pages=1019–1026 |doi=10.1145/3583131.3590486 |isbn=979-8-4007-0119-1|s2cid=259833672 |url=https://wrap.warwick.ac.uk/175161/7/3583131.3590486.pdf }}</ref><ref>{{Cite journal |last1=Gaier |first1=Adam |last2=Asteroth |first2=Alexander |last3=Mouret |first3=Jean-Baptiste |date=2018-09-01 |title=Data-Efficient Design Exploration through Surrogate-Assisted Illumination |journal=Evolutionary Computation |volume=26 |issue=3 |pages=381–410 |doi=10.1162/evco_a_00231 |pmid=29883202 |s2cid=47003986 |issn=1063-6560|doi-access=free |arxiv=1806.05865 }}</ref> chemistry, material design, and drug development.<ref name=":0" /><ref>Gomez-Bombarelli et al. [https://pubs.acs.org/doi/10.1021/acscentsci.7b00572 Automatic Chemical Design using a Data-Driven Continuous Representation of Molecules]. ACS Central Science, Volume 4, Issue 2, 268-276 (2018)</ref><ref>Griffiths et al. [https://pubs.rsc.org/en/content/articlehtml/2020/sc/c9sc04026a Constrained Bayesian Optimization for Automatic Chemical Design using Variational Autoencoders] Chemical Science: 11, 577-586 (2020)</ref>