Content deleted Content added
NLPresearch (talk | contribs) Added an open article version to one of the introductory citations for an EMNLP paper. |
Citation bot (talk | contribs) Alter: url. URLs might have been anonymized. Add: chapter-url, year, s2cid, arxiv, authors 1-1. Removed or converted URL. Removed parameters. Some additions/deletions were parameter name changes. | Use this bot. Report bugs. | Suggested by AManWithNoPlan | #UCB_CommandLine |
||
Line 1:
{{short description|Automatic generation or recognition of paraphrased text}}
{{about|automated generation and recognition of paraphrases||Paraphrase (disambiguation)}}
'''Paraphrase''' or '''paraphrasing''' in [[computational linguistics]] is the [[natural language processing]] task of detecting and generating [[paraphrase]]s. Applications of paraphrasing are varied including information retrieval, [[question answering]], [[Automatic summarization|text summarization]], and [[plagiarism detection]].<ref name=Socher /> Paraphrasing is also useful in the [[evaluation of machine translation]],<ref name=Callison>{{cite conference |last=Callison-Burch |first=Chris |title=Syntactic Constraints on Paraphrases Extracted from Parallel Corpora |conference=EMNLP '08 Proceedings of the Conference on Empirical Methods in Natural Language Processing |date=October 25–27, 2008 |place=Honolulu, Hawaii |pages=196–205|url=https://dl.acm.org/citation.cfm?id=1613743}}</ref> as well as [[semantic parsing]]<ref>Berant, Jonathan, and Percy Liang. "[http://www.aclweb.org/anthology/P14-1133 Semantic parsing via paraphrasing]." Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Vol. 1. 2014.</ref> and [[natural language generation|generation]]<ref>{{Cite journal |
== Paraphrase generation ==
Line 25:
=== Transformers ===
With the introduction of [[Transformer (machine learning model)|Transformer models]], paraphrase generation approaches improved their ability to generate text by scaling [[neural network]] parameters and heavily parallelizing training through [[Feedforward neural network|feed-forward layers]].<ref>{{Cite journal |
== Paraphrase recognition ==
=== Recursive Autoencoders ===
Paraphrase recognition has been attempted by Socher et al<ref name=Socher>{{Citation |last1=Socher|first1=Richard |last2=Huang |first2=Eric |last3=Pennington |first3=Jeffrey |last4=Ng |first4=Andrew |last5=Manning |first5=Christopher |title=Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection |chapter=Advances in Neural Information Processing Systems 24 |year=2011 |chapter-url=http://www.socher.org/index.php/Main/DynamicPoolingAndUnfoldingRecursiveAutoencodersForParaphraseDetection}}</ref> through the use of recursive [[autoencoder]]s. The main concept is to produce a vector representation of a sentence and its components by recursively using an autoencoder. The vector representations of paraphrases should have similar vector representations; they are processed, then fed as input into a [[artificial neural network|neural network]] for classification.
Given a sentence <math>W</math> with <math>m</math> words, the autoencoder is designed to take 2 <math>n</math>-dimensional [[word embedding]]s as input and produce an <math>n</math>-dimensional vector as output. The same autoencoder is applied to every pair of words in <math>S</math> to produce <math>\lfloor m/2 \rfloor</math> vectors. The autoencoder is then applied recursively with the new vectors as inputs until a single vector is produced. Given an odd number of inputs, the first vector is forwarded as-is to the next level of recursion. The autoencoder is trained to reproduce every vector in the full recursion tree, including the initial word embeddings.
Line 42:
=== Transformers ===
Similar to how [[Transformer (machine learning model)|Transformer models]] influenced paraphrase generation, their application in identifying paraphrases showed great success. Models such as BERT can be adapted with a [[binary classification]] layer and trained end-to-end on identification tasks.<ref>{{Cite journal |
== Evaluation ==
| |||