Content deleted Content added
Citation bot (talk | contribs) Alter: title, template type. Add: chapter-url, chapter. Removed or converted URL. Removed parameters. Some additions/deletions were parameter name changes. | Use this bot. Report bugs. | #UCB_CommandLine |
m Added missing commas, missing acronym definition. Tags: Visual edit Mobile edit Mobile web edit |
||
Line 20:
== Bayesian optimization ==
[[Bayesian Optimization]] (BO), which has proven to be an efficient method for hyperparameter optimization, can also be applied to NAS. In this context, the objective function maps an architecture to its validation error after being trained for a number of epochs. At each iteration, BO uses a surrogate to model this objective function based on previously obtained architectures and their validation errors. One then chooses the next architecture to evaluate by maximizing an acquisition function, such as expected improvement, which provides a balance between exploration and exploitation. Acquisition function maximization and objective function evaluation are often computationally expensive for NAS, and make the application of BO challenging in this context. Recently, BANANAS<ref>{{cite arXiv|last1=White|first1=Colin|last2=Neiswanger|first2=Willie|last3=Savani|first3=Yash|date=2020-11-02|title=BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search|class=cs.LG|eprint=1910.11858}}</ref> has achieved promising results in this direction by introducing a high-performing instantiation of BO coupled to a neural predictor.
==Hill-climbing==
| |||