Content deleted Content added
→Bayesian: Added BOCS |
Citation bot (talk | contribs) m Add: class, doi-broken-date, author pars. 1-2. You can use this bot yourself. Report bugs here. |
||
Line 33:
{{main article|Random search}}
Random Search replaces the exhaustive enumeration of all combinations by selecting them randomly. This can be simply applied to the discrete setting described above, but also generalizes to continuous and mixed spaces. It can outperform Grid search, especially when only a small number of hyperparameters affects the final performance of the machine learning algorithm<ref name="bergstra" />. In this case, the optimization problem is said to have a low intrinsic dimensionality<ref>{{Cite journal|last=Ziyu|first=Wang,|last2=Frank|first2=Hutter,|last3=Masrour|first3=Zoghi,|last4=David|first4=Matheson,|last5=Nando|first5=de Feitas,|date=2016|title=Bayesian Optimization in a Billion Dimensions via Random Embeddings|url=http://jair.org/papers/paper4806.html|journal=Journal of Artificial Intelligence Research|language=en|volume=55|doi=10.1613/jair.4806|doi-broken-date=2018-06-26}}</ref>. Random Search is also [[embarrassingly parallel]], and additionally allows to include prior knowledge by specifying the distribution from which to sample.
=== Bayesian optimization ===
Line 148:
* [https://github.com/mlr-org/mlrMBO mlrMBO], also with [https://github.com/mlr-org/mlr mlr], is an [[R (programming language)|R]] package for model-based/Bayesian optimization of black-box functions.
* [https://github.com/PhilippPro/tuneRanger tuneRanger] is an R package for tuning random forests using model-based optimization.
* [https://github.com/baptistar/BOCS BOCS] is a Matlab package which uses [[semidefinite programming]] for minimizing a black-box function over discrete inputs<ref name="arXiv:1806.08838">{{cite arXiv |year=2018 |title=Bayesian Optimization of Combinatorial Structures |eprint=1806.08838|last1=Baptista |first1=Ricardo |last2=Poloczek |first2=Matthias |class=stat.ML }}</ref>
===Gradient based===
| |||