Sharpness aware minimization: Difference between revisions

Revision as of 21:59, 14 June 2025

Revision as of 13:57, 22 June 2025

Line 8:

{{AfC submission|||ts=20250530075726|u=2600:1700:3EC7:2CC0:5B82:5CA9:6050:DFE|ns=118}}

{{AFC submission|d|npov|u=2600:1700:3EC7:2CC0:F69C:2A90:F461:223D|ns=118|decliner=ToadetteEdit|declinets=20250522114510|ts=20250522053247}}

<!-- Do not remove this line! -->

<!-- Important, do not remove anything above this line before article has been created. -->

'''Sharpness Aware Minimization''' ('''SAM''') is an [[optimization algorithm]] used in [[machine learning]] that aims to improve model [[generalization (machine learning)|generalization]]. The method seeks to find model parameters that are located in regions of the loss landscape with uniformly low loss values, rather than parameters that only achieve a minimal loss value at a single point. This approach is described as finding "flat" minima instead of "sharp" ones. The rationale is that models trained this way are less sensitive to variations between training and test [[data set|data]], which can lead to better performance on unseen data.<ref name="Foret2021">{{cite conference |last1=Foret |first1=Pierre |last2=Kleiner |first2=Ariel |last3=Mobahi |first3=Hossein |last4=Neyshabur |first4=Behnam |title=Sharpness-Aware Minimization for Efficiently Improving Generalization |book-title=International Conference on Learning Representations (ICLR) 2021 |year=2021 |arxiv=2010.01412 |url=https://openreview.net/forum?id=6Tm1m_rRrwY}}</ref>

Line 56 ⟶ 53:

Theoretical work continues to analyze the algorithm's behavior, including its implicit bias towards flatter minima and the development of broader frameworks for sharpness-aware optimization that use different measures of sharpness.<ref name="Wen2022SAMLandscape">{{cite arXiv |last1=Wen |first1=Yulei |last2=Zhang |first2=Zhe |last3=Liu |first3=Zhen |last4=Li |first4=Yue |last5=Zhang |first5=Tiantian |title=How Does SAM Influence the Loss Landscape? |eprint=2203.08065 |year=2022 |class=cs.LG}}</ref><ref name="Zhou2023SAMUnified">{{cite arXiv |last1=Zhou |first1=Kaizheng |last2=Zhang |first2=Yulai |last3=Tao |first3=Dacheng |title=Sharpness-Aware Minimization: A Unified View and A New Theory |eprint=2305.10276 |year=2023 |class=cs.LG}}</ref>

==References==

{{reflist}}

== References ==

<!-- Inline citations added to your article will automatically display here. See en.wikipedia.org/wiki/WP:REFB for instructions on how to add citations. -->

{{reflist}}