Content deleted Content added
the mathematic foundation of Bayesian optimization from 1960s-1980s Tags: Reverted use of deprecated (unreliable) source Visual edit |
the mathematic foundation of Bayesian optimization from 1960s-1980s Tags: Reverted use of deprecated (unreliable) source Visual edit |
||
Line 18:
==== From 1960s to 1980s ====
The earliest idea of Bayesian optimization sprang in 1964, from a paper by American applied mathematician Harold J. Kushner<ref>{{Citation |title=Harold J. Kushner |date=2024-11-26 |work=Wikipedia |url=https://en.wikipedia.org/wiki/Harold_J._Kushner |access-date=2025-02-25 |language=en}}</ref>, [https://asmedigitalcollection.asme.org/fluidsengineering/article/86/1/97/392213/A-New-Method-of-Locating-the-Maximum-Point-of-an “A New Method of Locating the Maximum Point of an Arbitrary Multipeak Curve in the Presence of Noise”]. Although not directly proposing Bayesian optimization, in this paper, he first proposed a new method of locating the maximum point of an arbitrary multipeak curve in a noisy environment. This method provided an important theoretical foundation for subsequent Bayesian optimization.
By the 1980s, the framework we now use for Bayesian optimization was explicitly established. In 1978, the Soviet scientist [[:lt:Jonas_Mockus|Jonas Mockus]], in his paper “The Application of Bayesian Methods for Seeking the Extremum”, discussed how to use Bayesian methods to find the extreme value of a function under various uncertain conditions. In his paper, Mockus first proposed the [https://schneppat.com/expected-improvement_ei.html Expected Improvement principle (EI)]<ref>{{Citation |title=Gaussian elimination |date=2025-01-26 |work=Wikipedia |url=https://en.wikipedia.org/wiki/Gaussian_elimination |access-date=2025-02-27 |language=en}}</ref>, which is one of the core sampling strategies of Bayesian optimization. This criterion balances exploration while optimizing the function efficiently by maximizing the expected improvement. Because of the usefulness and profound impact of this principle, Jonas Mockus is widely regarded as the founder of Bayesian optimization. Although Expected Improvement principle (IE) is one of the earliest proposed core sampling strategies for Bayesian optimization, it is not the only one, with the development of modern society, we also have Probability of Improvement (PI), or Upper Confidence Bound (UCB) and so on.
==Strategy==
| |||