Principal component analysis: Difference between revisions

Revision as of 01:12, 22 April 2025

Revision as of 01:26, 22 April 2025

Line 155:

The truncation of a matrix '''M''' or '''T''' using a truncated singular value decomposition in this way produces a truncated matrix that is the nearest possible matrix of [[Rank (linear algebra)|rank]] ''L'' to the original matrix, in the sense of the difference between the two having the smallest possible [[Frobenius norm]], a result known as the [[Low-rank approximation#Proof of Eckart–Young–Mirsky theorem (for Frobenius norm)|Eckart–Young theorem]] [1936].

<blockquote>

'''Theorem (Optimal k‑dimensional fit).'''

Let P be an n×m data matrix whose columns have been mean‑centered and scaled, and let

<math>P = U \,\Sigma\, V^{T}</math>

be its singular value decomposition. Then the best rank‑k approximation to P in the least‑squares (Frobenius‑norm) sense is

<math>P_{k} = U_{k}\,\Sigma_{k}\,V_{k}^{T}</math>,

where V_k consists of the first k columns of V. Moreover, the relative residual variance is

<math>R(k)=\frac{\sum_{j=k+1}^{m}\sigma_{j}^{2}}{\sum_{j=1}^{m}\sigma_{j}^{2}}</math>.

</blockquote>

<ref>{{cite book

| last = Holmes

| first = M.

| title = Introduction to Scientific Computing and Data Analysis

| edition = 2nd

| year = 2023

| publisher = Springer

| isbn = 978-3-031-22429-4

}}</ref>