Language model: Difference between revisions

Revision as of 00:06, 17 April 2025

Revision as of 15:14, 12 May 2025

Line 7:

== History ==

[[Noam Chomsky]] did pioneering work on language models in the 1950s by developing a theory of [[formal grammar]]s, which became fundamental to the field of [[programming language]]s.<ref>{{Cite journal |last=Chomsky |first=N. |date=September 1956 |title=Three models for the description of language |url=https://ieeexplore.ieee.org/document/1056813 |journal=IRE Transactions on Information Theory |volume=2 |issue=3 |pages=113–124 |doi=10.1109/TIT.1956.1056813 |issn=2168-2712}}</ref>

In 1980, statistical approaches were explored and found to be more useful for many purposes than rule-based formal grammars. Discrete representations like [[Word n-gram language model|word ''n''-gram language models]], with probabilities for discrete combinations of words, made significant advances.