Activation function: Difference between revisions

Revision as of 23:00, 18 February 2025

Revision as of 00:35, 21 February 2025

Line 220:

|title = [[2008 IEEE International Conference on Acoustics, Speech and Signal Processing]]

|date = 2008

|pages=3265–3268 |doi=10.1109/ICASSP.2008.4518347|isbn=978-1-4244-1483-3 |s2cid=9959057}}</ref> SiL,<ref>{{Cite journal |arxiv = 1702.03118|title = Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning|journal = Neural Networks |volume = 107|pages = 3–11|last1 = Elfwing|first1 = Stefan|last2 = Uchibe|first2 = Eiji|last3 = Doya|first3 = Kenji|year = 2018|pmid = 29395652|doi = 10.1016/j.neunet.2017.12.012|s2cid = 6940861}}</ref> or [[Swish function|Swish]]-{{zwj}}1<ref>{{Cite arXiv |eprint = 1710.05941 |title = Searching for Activation Functions|last1 = Ramachandran|first1 = Prajit|last2 = Zoph|first2 = Barret|last3 = Le|first3 = Quoc V|year = 2017|class = cs.NE}}</ref>)

| [[File:Swish.svg|Swish Activation Function|120px]]

| <math>\frac{x}{1 + e^{-x}}</math>