Vector space model: Difference between revisions

Revision as of 05:11, 9 December 2023

Revision as of 05:30, 9 December 2023

Line 1:

{{Short description|Model for representing text documents}}

'''Vector space model''' or '''term vector model''' is an algebraic model for representing text documents (or more generally, items) as [[vector space|vectors]] such that the distance between vectors represents the relevance between the documents. It is used in [[information filtering]], [[information retrieval]], [[index (search engine)|index]]ing and relevancy rankings. Its first use was in the [[SMART Information Retrieval System]]{{cncitation needed|date=December 2023}}.

==Definitions==

Line 12:

The definition of ''term'' depends on the application. Typically terms are single words, [[keyword (linguistics)|keyword]]s, or longer phrases. If words are chosen to be the terms, the dimensionality of the vector is the number of words in the vocabulary (the number of distinct words occurring in the [[text corpus|corpus]]).

Vector operations can be used to compare documents with queries.<ref name=":0">{{Cite book |last=Büttcher |first=Stefan |title=Information retrieval: implementing and evaluating search engines |last2=Clarke |first2=Charles L. A. |last3=Cormack |first3=Gordon V. |date=2016 |publisher=The MIT Press |isbn=978-0-262-52887-0 |edition=First MIT Press paperback edition |___location=Cambridge, Massachusetts London, England}}</ref>

==Applications==

Line 78:

* [[Apache Lucene]]. Apache Lucene is a high-performance, open source, full-featured text search engine library written entirely in Java.

* [[OpenSearch (software)]] and [[Apache Solr|Solr]] : the 2 most famous search engine software (many smaller exist) based on Lucene.

* [[Gensim]] is a Python+[[NumPy]] framework for Vector Space modelling. It contains incremental (memory-efficient) algorithms for [[tf–idf|term frequency-inverse document frequency]], [[Latent Semantic Indexing]], [[Locality_sensitive_hashingLocality sensitive hashing#Random_projectionRandom projection|Random Projections]] and [[Latent Dirichlet Allocation]].

* [[Weka (machine learning)|Weka]]. Weka is a popular data mining package for Java including WordVectors and [[Bag-of-words model|Bag Of Words models]].

* [[Word2vec]]. Word2vec uses vector spaces for word embeddings.