Content deleted Content added
Citation bot (talk | contribs) Added doi-broken-date. | Use this bot. Report bugs. | #UCB_CommandLine Tag: Reverted |
Citation bot (talk | contribs) Removed parameters. | Use this bot. Report bugs. | Suggested by Headbomb | Category:CS1 maint: DOI inactive as of June 2024 | #UCB_Category 81/305 |
||
Line 28:
== Neural models ==
=== Recurrent neural network ===
Continuous representations or [[Word embedding|embeddings of words]] are produced in [[recurrent neural network]]-based language models (known also as ''continuous space language models'').<ref>{{cite web |last1=Karpathy |first1=Andrej |title=The Unreasonable Effectiveness of Recurrent Neural Networks |url=https://karpathy.github.io/2015/05/21/rnn-effectiveness/ |access-date=27 January 2019 |archive-date=1 November 2020 |archive-url=https://web.archive.org/web/20201101215448/http://karpathy.github.io/2015/05/21/rnn-effectiveness/ |url-status=live }}</ref> Such continuous space embeddings help to alleviate the [[curse of dimensionality]], which is the consequence of the number of possible sequences of words increasing [[Exponential growth|exponentially]] with the size of the vocabulary, furtherly causing a data sparsity problem. Neural networks avoid this problem by representing words as non-linear combinations of weights in a neural net.<ref name="bengio">{{cite encyclopedia|title=Neural net language models|first=Yoshua|last=Bengio|year=2008|encyclopedia=[[Scholarpedia]]|volume=3|issue=1|page=3881|url=http://www.scholarpedia.org/article/Neural_net_language_models|doi=10.4249/scholarpedia.3881
=== Large language models ===
| |||