Least-squares function approximation: Difference between revisions

Content deleted Content added
merging from linear regression
Line 2:
 
==Functional analysis==
 
{{See also|Fourier series|Generalized Fourier series}}
 
Line 43 ⟶ 42:
 
</ref> Particular examples of such a representation of a function are the [[Fourier series]] and the [[generalized Fourier series]].
 
==Further discussion==
===Using linear algebra===
It follows that one can find a "best" approximation of another function by minimizing the area between two functions, a continuous function <math>f</math> on <math>[a,b]</math> and a function <math>g\in W</math> where <math>W</math> is a subspace of <math>C[a,b]</math>:
:<math>\text{Area} = \int_a^b \left\vert f(x) - g(x)\right\vert \, dx,</math>
all within the subspace <math>W</math>. Due to the frequent difficulty of evaluating integrands involving absolute value, one can instead define
:<math>\int_a^b [ f(x) - g(x) ] ^2\, dx</math>
as an adequate criterion for obtaining the least squares approximation, function <math>g</math>, of <math>f</math> with respect to the inner product space <math>W</math>.
 
As such, <math>\lVert f-g \rVert ^2</math> or, equivalently, <math>\lVert f-g \rVert</math>, can thus be written in vector form:
 
:<math>\int_a^b [ f(x)-g(x) ]^2\, dx = \left\langle f-g , f-g\right\rangle = \lVert f-g\rVert^2.</math>
 
In other words, the least squares approximation of <math>f</math> is the function <math>g\in \text{ subspace } W</math> closest to <math>f</math> in terms of the inner product <math>\left \langle f,g \right \rangle</math>. Furthermore, this can be applied with a theorem:
:Let <math>f</math> be continuous on <math>[ a,b ]</math>, and let <math>W</math> be a finite-dimensional subspace of <math>C[a,b]</math>. The least squares approximating function of <math>f</math> with respect to <math>W</math> is given by
::<math>g = \left \langle f,\vec w_1 \right \rangle \vec w_1 + \left \langle f,\vec w_2 \right \rangle \vec w_2 + \cdots + \left \langle f,\vec w_n \right \rangle \vec w_n,</math>
:where <math>B = \{\vec w_1 , \vec w_2 , \dots , \vec w_n \}</math> is an orthonormal basis for <math>W</math>.
 
==References==