Learning vector quantization: Difference between revisions

Revision as of 01:08, 23 May 2025

Revision as of 01:08, 23 May 2025

Line 20:

* The hyperparameters <math>w</math> and <math>\epsilon</math> are used by LVQ2 and LVQ3. The original paper suggests <math>\epsilon \in [0.1, 0.5]</math> and <math>w \in [0.2, 0.3]</math>.

=== LVQ1 ===

Initialize several code vectors per label. Iterate until convergence criteria is reached.

Line 26:

# If its label <math>y_i</math> is the same as that of <math>w_j</math>, then <math>w_j \leftarrow w_j + \alpha_t(x_i - w_j)</math>, otherwise, <math>w_j \leftarrow w_j - \alpha_t(x_i - w_j)</math>.

=== LVQ3LVQ2 ===

LVQ2 is the same as LVQ3, but with this sentence removed: "If <math>w_j</math> and <math>w_k</math> and <math>x_i</math> have the same class, then <math>w_j \leftarrow w_j - \alpha_t(x_i - w_j)</math> and <math>w_k \leftarrow w_k + \alpha_t(x_i - w_k)</math>.".

=== LVQ1LVQ3 ===

Initialize several code vectors per label. Iterate until convergence criteria is reached.

Line 37 ⟶ 40:

#* If <math>w_k</math> and <math>x_i</math> have different classes, and <math>w_j</math> and <math>x_i</math> have different classes, then the original paper simply does not explain what happens in this case.

# Otherwise, skip.

== LVQ2 ==

LVQ2 is the same as LVQ3, but with this sentence removed: "If <math>w_j</math> and <math>w_k</math> and <math>x_i</math> have the same class, then <math>w_j \leftarrow w_j - \alpha_t(x_i - w_j)</math> and <math>w_k \leftarrow w_k + \alpha_t(x_i - w_k)</math>.".

=== LVQ1 ===

The algorithm consists of three basic steps. The algorithm's input is:

* how many neurons the system will have <math>M</math> (in the simplest case it is equal to the number of classes)

* what weight each neuron has <math>\vec{w_i}</math> for <math>i = 0,1,...,M - 1 </math>

* the corresponding label <math>c_i</math> to each neuron <math> \vec{w_i} </math>

* how fast the neurons are learning <math> \eta </math>

* and an input list <math> L </math> containing all the vectors of which the labels are known already (training set).

The algorithm's flow is:

# For next input <math>\vec{x}</math> (with label <math>y</math>) in <math> L </math> find the closest neuron <math>\vec{w_m}</math>, <br>i.e. <math>d(\vec{x},\vec{w_m}) = \min\limits_i {d(\vec{x},\vec{w_i})} </math>, where <math>\, d</math> is the metric used ( [[Euclidean distance|Euclidean]], etc. )

== References ==