Content deleted Content added
Neonrights (talk | contribs) |
Neonrights (talk | contribs) Created section on skip-thought vectors |
||
Line 32:
=== Skip-thought vectors ===
Skip-thought vectors are an attempt to create a vector representation of the semantic meaning of a sentence in a similar fashion as the [[word2vec|skip gram model]].<ref name=Kiros>{{Citation|last1=Kiros|first1=Ryan|last2=Zhu|first2=Yukun|last3=Salakhutdinov|first3=Ruslan|last4=Zemel|first4=Richard|last5=Torralba|first5=Antonio|last6=Urtasun|first6=Raquel|last7=Fidler|first7=Sanja|title=Skip-Thought Vectors|year=2015|url=https://arxiv.org/abs/1506.06726}}</ref> Skip-thought vectors are produced through the use of a skip-thought model which consists of three key components, an encoder and two decoders. Given a corpus of documents, the skip-thought model is trained to take a sentence as input and encode it into a skip-thought vector. The skip-thought vector is used as input for both decoders, one of which attempts to reproduce the previous sentence and the other the following sentence in its entirety. The encoder and decoder can be implemented through the use of an [[recursive neural network|RNN]] or an [[long short-term memory|LSTM]].
Since paraphrases carry the same semantic meaning between one another, they should have similar skip-thought vectors. Thus a simple [[logistic regression]] can be trained to a good performance with the absolute difference and component-wise product of two skip-thought vectors as input.
== Evaluation ==
| |||