Non-negative matrix factorization

Non-negative matrix factorization (NMF) is a group of algorithms in multivariate analysis and linear algebra where a matrix is factorized into (usually) two matrices

${\displaystyle \mathbf {WH} \leftarrow \operatorname {nmf} (\mathbf {X} )}$

Usually all three matrices must be non-negative, i.e., all elements must be equal to or greater than zero. It might be said to be a sort of non-negative version of singular value decomposition. Usually the numbers of columns of W and the numbers of rows of H is selected so the product WH will become an approximation to X, and it has been suggested that the NMF model should be called nonnegative matrix approximation instead.

It was used by a Finnish group of researchers in the middle of the 1990s under the name positive matrix factorization. It became more widely known after Lee and Seung's investigations of the properties of the algorithm, and after they published a simple useful algorithm.

There are different types of non-negative matrix factorizations and one of these is related to probabilistic latent semantic analysis and the latent class model. They different types arise from using different cost functions (divergence functions) and/or by regularization of the W and/or H matrices^[1].

Uniqueness

The factorization is not unique: A matrix and its inverse can be used to transform the two factorization matrices by, e.g.,

${\displaystyle \mathbf {WH} =\mathbf {WBB} ^{-1}\mathbf {H} }$ 

If the two new matrices ${\displaystyle \mathbf {{\tilde {W}}=WB} }$  and ${\displaystyle \mathbf {\tilde {H}} =\mathbf {B} ^{-1}\mathbf {H} }$  are non-negative they form another parametrization of the factorization.

The non-negativity of ${\displaystyle \mathbf {\tilde {W}} }$  and ${\displaystyle \mathbf {\tilde {H}} }$  applies at least if B is a non-negative monomial matrix. In this simple case it will just correspond to a scaling and a permutation.

Sources and external links

• P. Paatero, U. Tapper, "Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values", Environmetrics, 5:111-126, 1994.
• Pia Anttila, Pentti Paatero, Unto Tapper, Olli Järvinen. "Source identification of bulk wet deposition in Finland by positive matrix factorization", Atmospheric Environment, 29(14):1705-1718, 1995
• Pentti Paatero, "Least squares formulation of robust non-negative factor analysis", Chemometrics and Intelligent Laboratory Systems, 37(1):23-35, 1997 May.
• Daniel D. Lee and H. Sebastian Seung, "Learning the parts of objects by non-negative matrix factorization", Nature, 401(6755):788-791, 1999 October.
• Daniel D. Lee and H. Sebastian Seung, "Algorithms for Non-negative Matrix Factorization", Advances in Neural Information Processing Systems 13: Proceedings of the 2000 Conference, 556-562, MIT Press, 2001.

References

1. Inderjit S. Dhillon, Suvrit Sra, "Generalized Nonnegative Matrix Approximations with Bregman Divergences", NIPS, 2005.