Content deleted Content added
timeliness Tags: Visual edit Mobile edit Mobile web edit |
Citation bot (talk | contribs) Removed URL that duplicated identifier. Removed parameters. | Use this bot. Report bugs. | Suggested by Headbomb | Linked from Wikipedia:WikiProject_Academic_Journals/Journals_cited_by_Wikipedia/Sandbox | #UCB_webform_linked 20/967 |
||
| (10 intermediate revisions by 7 users not shown) | |||
Line 1:
{{short description|Automatic generation or recognition of paraphrased text}}
{{about|automated generation and recognition of paraphrases||Paraphrase (disambiguation)}}
'''Paraphrase''' or '''paraphrasing''' in [[computational linguistics]] is the [[natural language processing]] task of detecting and generating [[paraphrase]]s. Applications of paraphrasing are varied including information retrieval, [[question answering]], [[Automatic summarization|text summarization]], and [[plagiarism detection]].<ref name=Socher /> Paraphrasing is also useful in the [[evaluation of machine translation]],<ref name=Callison>{{cite conference |last=Callison-Burch |first=Chris |title=Syntactic Constraints on Paraphrases Extracted from Parallel Corpora |conference=EMNLP '08 Proceedings of the Conference on Empirical Methods in Natural Language Processing |date=October 25–27, 2008 |place=Honolulu, Hawaii |pages=196–205|url=https://dl.acm.org/citation.cfm?id=1613743}}</ref> as well as [[semantic parsing]]<ref>Berant, Jonathan, and Percy Liang. "[http://www.aclweb.org/anthology/P14-1133 Semantic parsing via paraphrasing]." Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Vol. 1. 2014.</ref> and [[natural language generation|generation]]<ref>{{Cite
== Paraphrase generation ==
=== Multiple sequence alignment ===
Barzilay and Lee<ref name=Barzilay>{{cite conference|last1=Barzilay|first1=Regina|last2=Lee|first2=Lillian|title=Learning to Paraphrase: An Unsupervised Approach Using Multiple-Sequence Alignment|conference=Proceedings of HLT-NAACL 2003|date=May–June 2003|url=
* finding recurring patterns in each individual corpus, i.e. "{{mvar|X}} (injured/wounded) {{mvar|Y}} people, {{mvar|Z}} seriously" where {{mvar|X, Y, Z}} are variables
* finding pairings between such patterns the represent paraphrases, i.e. "{{mvar|X}} (injured/wounded) {{mvar|Y}} people, {{mvar|Z}} seriously" and "{{mvar|Y}} were (wounded/hurt) by {{mvar|X}}, among them {{mvar|Z}} were in serious condition"
Line 12:
This is achieved by first clustering similar sentences together using [[n-gram]] overlap. Recurring patterns are found within clusters by using multi-sequence alignment. Then the position of argument words is determined by finding areas of high variability within each cluster, aka between words shared by more than 50% of a cluster's sentences. Pairings between patterns are then found by comparing similar variable words between different corpora. Finally, new paraphrases can be generated by choosing a matching cluster for a source sentence, then substituting the source sentence's argument into any number of patterns in the cluster.
=== Phrase-based
Paraphrase can also be generated through the use of [[statistical machine translation#Phrase-based translation|phrase-based translation]] as proposed by Bannard and Callison-Burch.<ref name=Bannard>{{cite conference |last1=Bannard|first1=Colin|last2=Callison-Burch|first2=Chris|title=Paraphrasing Bilingual Parallel Corpora |conference=Proceedings of the 43rd Annual Meeting of the ACL |place=Ann Arbor, Michigan|pages=597–604|year=2005|url=https://dl.acm.org/citation.cfm?id=1219914}}</ref> The chief concept consists of aligning phrases in a [[pivot language]] to produce potential paraphrases in the original language. For example, the phrase "under control" in an English sentence is aligned with the phrase "unter kontrolle" in its German counterpart. The phrase "unter kontrolle" is then found in another German sentence with the aligned English phrase being "in check," a paraphrase of "under control."
Line 25:
=== Transformers ===
With the introduction of [[Transformer (machine learning model)|Transformer models]], paraphrase generation approaches improved their ability to generate text by scaling [[neural network]] parameters and heavily parallelizing training through [[Feedforward neural network|feed-forward layers]].<ref>{{Cite book |last1=Zhou |first1=Jianing |last2=Bhat |first2=Suma |title=Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing |chapter=Paraphrase Generation: A Survey of the State of the Art |date=2021 |chapter-url=https://aclanthology.org/2021.emnlp-main.414 |language=en |___location=Online and Punta Cana, Dominican Republic |publisher=Association for Computational Linguistics |pages=5075–5086 |doi=10.18653/v1/2021.emnlp-main.414|s2cid=243865349 |doi-access=free }}</ref> These models are so fluent in generating text that human experts cannot identify if an example was human-authored or machine-generated.<ref>{{Cite journal |last1=Dou |first1=Yao |last2=Forbes |first2=Maxwell |last3=Koncel-Kedziorski |first3=Rik |last4=Smith |first4=Noah |last5=Choi |first5=Yejin |date=2022 |title=Is GPT-3 Text Indistinguishable from Human Text? Scarecrow: A Framework for Scrutinizing Machine Text |url=https://aclanthology.org/2022.acl-long.501 |journal=Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) |language=en |___location=Dublin, Ireland |publisher=Association for Computational Linguistics |pages=7250–7274 |doi=10.18653/v1/2022.acl-long.501|s2cid=247315430 |doi-access=free |arxiv=2107.01294 }}</ref> Transformer-based paraphrase generation relies on [[Autoencoder|autoencoding]], [[Autoregressive model|autoregressive]], or [[Seq2seq|sequence-to-sequence]] methods. Autoencoder models predict word replacement candidates with a one-hot distribution over the vocabulary, while autoregressive and seq2seq models generate new text based on the source predicting one word at a time.<ref>{{Cite journal |last1=Liu |first1=Xianggen |last2=Mou |first2=Lili |last3=Meng |first3=Fandong |last4=Zhou |first4=Hao |last5=Zhou |first5=Jie |last6=Song |first6=Sen |date=2020 |title=Unsupervised Paraphrasing by Simulated Annealing |url=https://www.aclweb.org/anthology/2020.acl-main.28 |journal=Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics |language=en |___location=Online |publisher=Association for Computational Linguistics |pages=302–312 |doi=10.18653/v1/2020.acl-main.28|s2cid=202537332 |doi-access=free |arxiv=1909.03588 }}</ref><ref>{{Cite book |last1=Wahle |first1=Jan Philip |last2=Ruas |first2=Terry |last3=Meuschke |first3=Norman |last4=Gipp |first4=Bela |title=2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL) |chapter=Are Neural Language Models Good Plagiarists? A Benchmark for Neural Paraphrase Detection
== Paraphrase recognition ==
=== Recursive
Paraphrase recognition has been attempted by Socher et al<ref name=Socher>{{Citation |last1=Socher |first1=Richard |last2=Huang |first2=Eric |last3=Pennington |first3=Jeffrey |last4=Ng |first4=Andrew |last5=Manning |first5=Christopher |title=Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection |chapter=Advances in Neural Information Processing Systems 24 |year=2011 |chapter-url=http://www.socher.org/index.php/Main/DynamicPoolingAndUnfoldingRecursiveAutoencodersForParaphraseDetection |access-date=2017-12-29 |archive-date=2018-01-06 |archive-url=https://web.archive.org/web/20180106173348/http://www.socher.org/index.php/Main/DynamicPoolingAndUnfoldingRecursiveAutoencodersForParaphraseDetection |url-status=dead }}</ref> through the use of recursive [[autoencoder]]s. The main concept is to produce a vector representation of a sentence and its components by recursively using an autoencoder. The vector representations of paraphrases should have similar vector representations; they are processed, then fed as input into a [[artificial neural network|neural network]] for classification.
Given a sentence <math>W</math> with <math>m</math> words, the autoencoder is designed to take 2 <math>n</math>-dimensional [[word embedding]]s as input and produce an <math>n</math>-dimensional vector as output. The same autoencoder is applied to every pair of words in <math>S</math> to produce <math>\lfloor m/2 \rfloor</math> vectors. The autoencoder is then applied recursively with the new vectors as inputs until a single vector is produced. Given an odd number of inputs, the first vector is forwarded as-is to the next level of recursion. The autoencoder is trained to reproduce every vector in the full recursion tree, including the initial word embeddings.
Given two sentences <math>W_1</math> and <math>W_2</math> of length 4 and 3 respectively, the autoencoders would produce 7 and 5 vector representations including the initial word embeddings. The [[euclidean distance]] is then taken between every combination of vectors in <math>W_1</math> and <math>W_2</math> to produce a similarity matrix <math>S \in \mathbb{R}^{7 \times 5}</math>. <math>S</math> is then subject to a dynamic min-[[
=== Skip-thought vectors ===
Line 42:
=== Transformers ===
Similar to how [[Transformer (machine learning model)|Transformer models]] influenced paraphrase generation, their application in identifying paraphrases showed great success. Models such as BERT can be adapted with a [[binary classification]] layer and trained end-to-end on identification tasks.<ref>{{Cite journal |last1=Devlin |first1=Jacob |last2=Chang |first2=Ming-Wei |last3=Lee |first3=Kenton |last4=Toutanova |first4=Kristina |title=Proceedings of the 2019 Conference of the North |date=2019 |url=http://aclweb.org/anthology/N19-1423
== Evaluation ==
Multiple methods can be used to evaluate paraphrases. Since paraphrase recognition can be posed as a classification problem, most standard evaluations metrics such as [[accuracy]], [[f1 score]], or an [[receiver operating characteristic|ROC curve]] do relatively well. However, there is difficulty calculating f1-scores due to trouble producing a complete list of paraphrases for a given phrase and the fact that good paraphrases are dependent upon context. A metric designed to counter these problems is ParaMetric.<ref name=Burch2>{{cite conference |last1=Callison-Burch |first1=Chris |last2=Cohn |first2=Trevor |last3=Lapata |first3=Mirella |title=ParaMetric: An Automatic Evaluation Metric for Paraphrasing |conference=Proceedings of the 22nd International Conference on Computational Linguistics|place=Manchester |year=2008 |pages=97–104 |doi=10.3115/1599081.1599094 |s2cid=837398
The evaluation of paraphrase generation has similar difficulties as the evaluation of [[machine translation]]. The quality of a paraphrase depends on its context, whether it is being used as a summary, and how it is generated, among other factors. Additionally, a good paraphrase usually is lexically dissimilar from its source phrase. The simplest method used to evaluate paraphrase generation would be through the use of human judges. Unfortunately, evaluation through human judges tends to be time-consuming. Automated approaches to evaluation prove to be challenging as it is essentially a problem as difficult as paraphrase recognition. While originally used to evaluate machine translations, bilingual evaluation understudy ([[BLEU]]) has been used successfully to evaluate paraphrase generation models as well. However, paraphrases often have several lexically different but equally valid solutions, hurting BLEU and other similar evaluation metrics.<ref name=Chen>{{cite conference |last1=Chen |first1=David |last2=Dolan |first2=William |title=Collecting Highly Parallel Data for Paraphrase Evaluation |conference=Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies |place=Portland, Oregon |year=2008 |pages=190–200 |url=https://dl.acm.org/citation.cfm?id=2002497}}</ref>
Line 51:
Metrics specifically designed to evaluate paraphrase generation include paraphrase in n-gram change (PINC)<ref name=Chen /> and paraphrase evaluation metric (PEM)<ref name=Liu>{{cite conference|last1=Liu|first1=Chang|last2=Dahlmeier|first2=Daniel|last3=Ng|first3=Hwee Tou|title=PEM: A Paraphrase Evaluation Metric Exploiting Parallel Texts |conference=Proceedings of the 2010 Conference on Empricial Methods in Natural Language Processing |place=MIT, Massachusetts |year=2010 |pages=923–932 |url=http://www.aclweb.org/anthology/D10-1090}}</ref> along with the aforementioned ParaMetric. PINC is designed to be used with BLEU and help cover its inadequacies. Since BLEU has difficulty measuring lexical dissimilarity, PINC is a measurement of the lack of n-gram overlap between a source sentence and a candidate paraphrase. It is essentially the [[Jaccard index|Jaccard distance]] between the sentence, excluding n-grams that appear in the source sentence to maintain some semantic equivalence. PEM, on the other hand, attempts to evaluate the "adequacy, fluency, and lexical dissimilarity" of paraphrases by returning a single value heuristic calculated using [[N-gram]]s overlap in a pivot language. However, a large drawback to PEM is that it must be trained using large, in-___domain parallel corpora and human judges.<ref name=Chen /> It is equivalent to training a paraphrase recognition to evaluate a paraphrase generation system.
The Quora Question Pairs Dataset, which contains hundreds of thousands of duplicate questions, has become a common dataset for the evaluation of paraphrase detectors.<ref>{{cite web |title=Paraphrase Identification on Quora Question Pairs |url=https://paperswithcode.com/sota/paraphrase-identification-on-quora-question|website=Papers with Code}}</ref> Consistently reliable paraphrase detection have all used the Transformer architecture and all have relied on large amounts of pre-training with more general data before fine-tuning
== See also ==
*
*
*
== References ==
| |||