Outline of natural language processing: Difference between revisions

Revision as of 22:20, 22 December 2019

Revision as of 12:20, 6 March 2020

Line 187:

* [[Naive Bayes spam filtering|Spam filtering]] –

* [[Sentiment analysis]] – extracts subjective information usually from a set of documents, often using online reviews to determine "polarity" about specific objects. It is especially useful for identifying trends of public opinion in the social media, for the purpose of marketing.

* [[Speech recognition]] – given a sound clip of a person or people speaking, determine the textual representation of the speech. This is the opposite of [[text to speech]] and is one of the extremely difficult problems colloquially termed "[[AI-complete]]" (see above). In [[natural speech]] there are hardly any pauses between successive words, and thus [[speech segmentation]] is a necessary subtask of speech recognition (see below). Note also that inIn most spoken languages, the sounds representing successive letters blend into each other in a process termed [[coarticulation]], so the conversion of the analog signal to discrete characters can be a very difficult process.

* [[Speech synthesis]] (Text-to-speech) –

* [[Text-proofing]] –

Line 216:

* [[Lemmatisation]] – groups together all like terms that share a same lemma such that they are classified as a single item.

* [[Morphology (linguistics)|Morphological segmentation]] – separates words into individual [[morphemes]] and identifies the class of the morphemes. The difficulty of this task depends greatly on the complexity of the [[morphology (linguistics)|morphology]] (i.e. the structure of words) of the language being considered. [[English language|English]] has fairly simple morphology, especially [[inflectional morphology]], and thus it is often possible to ignore this task entirely and simply model all possible forms of a word (e.g. "open, opens, opened, opening") as separate words. In languages such as [[Turkish language|Turkish]], however, such an approach is not possible, as each dictionary entry has thousands of possible word forms.

* [[Named entity recognition]] (NER) – given a stream of text, determines which items in the text map to proper names, such as people or places, and what the type of each such name is (e.g. person, ___location, organization). Note that, althoughAlthough [[capitalization]] can aid in recognizing named entities in languages such as English, this information cannot aid in determining the type of named entity, and in any case is often inaccurate or insufficient. For example, the first word of a sentence is also capitalized, and named entities often span several words, only some of which are capitalized. Furthermore, many other languages in non-Western scripts (e.g. [[Chinese language|Chinese]] or [[Arabic language|Arabic]]) do not have any capitalization at all, and even languages with capitalization may not consistently use it to distinguish names. For example, [[German language|German]] capitalizes all [[noun]]s, regardless of whether they refer to names, and [[French language|French]] and [[Spanish language|Spanish]] do not capitalize names that serve as [[adjective]]s.

* [[Ontology learning]] – automatic or semi-automatic creation of [[Ontology (information science)|ontologies]], including extracting the corresponding ___domain's terms and the relationships between those concepts from a corpus of natural language text, and encoding them with an [[ontology language]] for easy retrieval. Also called "ontology extraction", "ontology generation", and "ontology acquisition".

* [[Parsing]] – determines the [[parse tree]] (grammatical analysis) of a given sentence. The [[grammar]] for [[natural language]]s is [[ambiguous]] and typical sentences have multiple possible analyses. In fact, perhaps surprisingly, for a typical sentence there may be thousands of potential parses (most of which will seem completely nonsensical to a human).

** [[Shallow parsing]] –

* [[Part-of-speech tagging]] – given a sentence, determines the [[part of speech]] for each word. Many words, especially common ones, can serve as multiple [[parts of speech]]. For example, "book" can be a [[noun]] ("the book on the table") or [[verb]] ("to book a flight"); "set" can be a [[noun]], [[verb]] or [[adjective]]; and "out" can be any of at least five different parts of speech. Note that someSome languages have more such ambiguity than others. Languages with little [[inflectional morphology]], such as [[English language|English]] are particularly prone to such ambiguity. [[Chinese language|Chinese]] is prone to such ambiguity because it is a [[tonal language]] during verbalization. Such inflection is not readily conveyed via the entities employed within the orthography to convey intended meaning.

* [[Query expansion]] –

* [[Relationship extraction]] – given a chunk of text, identifies the relationships among named entities (e.g. who is the wife of whom).