Factored language model: Difference between revisions

Revision as of 15:36, 7 April 2007

Latest revision as of 23:02, 24 June 2025

(13 intermediate revisions by 12 users not shown)

Line 1:

The '''factored language model''' ('''FLM''') is an extension of a conventional [[Languagelanguage model]] introduced by Jeff Bilmes and Katrin Kirchoff in 2003. In an FLM, each word is viewed as a vector of ''k'' factors: <math>w_i = \{f_i^1, ..., f_i^k\}.</math>. An FLM provides the probabilistic model <math>P(f|f_if_1, ..., f_N)</math> where the prediction of a factor <math>f</math> is based on <math>N</math> parents <math>\{f_1, ..., f_N\}</math>. For an example, if <math>w</math> represents a word token and <math>t</math> represents a [[Part of speech]] tag for English, the modelexpression <math>P(w_i|w_{i-2}, w_{i-1}, t_{i-1})</math> gives a model for predicting current workword token based on a traditional [[Ngram]] model as well as the [[Part of speech]] tag of the previous word.

{{Orphan|date=August 2006}}

The '''factored language model''' ('''FLM''') is an extension of conventional [[Language model]]. In an FLM, each word is viewed as a vector of ''k'' factors: <math>w_i = \{f_i^1, ..., f_i^k\}</math>. An FLM provides the probabilistic model <math>P(f|f_i, ..., f_N)</math> where the prediction of factor <math>f</math> is based on <math>N</math> parents <math>\{f_1, ..., f_N\}</math>. For an example, if <math>w</math> represents word token and <math>t</math> represents [[Part of speech]] tag for English, the model <math>P(w_i|w_{i-2}, w_{i-1}, t_{i-1})</math> gives a model for predicting current work token based on traditional [[Ngram]] model as well as [[Part of speech]] tag of the previous word.

A mainmajor advantage of factored language models is that they allow users to put inspecify linguistic knowledge such as explicitly model the relationship between word tokens and [[Part of speech]] in English, or morphological information (stems, root, etc.) in Arabic.

Like [[N-gram]] models, smoothing techniques are necessary in parameter estimation. In particular, generalized backingback-off is used in training an FLM.

==References==

*{{cite conference | author=J Bilmes and K Kirchhoff | url=http://ssli.ee.washington.edu/people/bilmes/mypapers/hlt03.pdf | title=Factored Language Models and Generalized Parallel Backoff | booktitlebook-title=Human Language Technology Conference | pagesyear=2003 | yeararchive-url=2003https://web.archive.org/web/20120717075838/http://ssli.ee.washington.edu/people/bilmes/mypapers/hlt03.pdf | archive-date=17 July 2012}}

[[Category:Language modeling]]

*{{cite conference | author=J Bilmes and K Kirchhoff | url=http://ssli.ee.washington.edu/people/bilmes/mypapers/hlt03.pdf | title=Factored Language Models and Generalized Parallel Backoff | booktitle=Human Language Technology Conference | pages= | year=2003}}

[[Category:NaturalStatistical natural language processing]]

[[Category:Probabilistic models]]

[[Category:Computational linguistics]]

[[Category:Natural language processing]]

{{compu-AINLP-stub}}

{{compu-AI-stub}}