Least-squares function approximation: Difference between revisions

Revision as of 01:06, 16 July 2018

Latest revision as of 01:46, 13 December 2023

(7 intermediate revisions by 7 users not shown)

Line 1:

{{Short description|Mathematical method}}

In [[mathematics]], '''least squares function approximation''' applies the principle of [[least squares]] to [[function approximation]], by means of a weighted sum of other functions. The best approximation can be defined as that which minimisesminimizes the difference between the original function and the approximation; for a least-squares approach the quality of the approximation is measured in terms of the squared differences between the two.

==Functional analysis==

{{See also|Fourier series|Generalized Fourier series}}

Line 14:

with the set of functions {<math>\ \phi _j (x) </math>} an [[Orthonormal_set#Real-valued_functions|orthonormal set]] over the interval of interest, {{nowrap|say [a, b]}}: see also [[Fejér's theorem]]. The coefficients {<math>\ a_j </math>} are selected to make the magnitude of the difference ||{{nowrap|''f'' − ''f''_''n''}}||² as small as possible. For example, the magnitude, or norm, of a function {{nowrap|''g'' (''x'' )}} over the {{nowrap|interval [a, b]}} can be defined by:<ref name=Folland>

{{cite book |title=Fourier analysis and its application |page =69 |chapter=Equation 3.14 |author=Gerald B Folland |chapter-url=https://books.google.com/books?id=ix2iCQ-o9x4C&pg=PA69 |isbn=978-0-8218-4790-29 |publisher=American Mathematical Society Bookstore |year=2009 |edition=Reprint of Wadsworth and Brooks/Cole 1992}}

</ref>

Line 21:

where the ‘*’ denotes [[complex conjugate]] in the case of complex functions. The extension of Pythagoras' theorem in this manner leads to [[function space]]s and the notion of [[Lebesgue measure]], an idea of “space” more general than the original basis of Euclidean geometry. The {{nowrap|{ <math>\phi_j (x)\ </math> } }} satisfy [[Orthogonal#Orthogonal_functions|orthonormality relations]]:<ref name=Folland2>

{{cite book |title=Fourier Analysis and Its Applications|page =69 |first1=Gerald B | last1= Folland|url=https://books.google.com/books?id=ix2iCQ-o9x4C&pg=PA69 |isbn=978-0-8218-4790-29 |year=2009 |publisher=American Mathematical Society}}

</ref>

Line 29:

the ''n''-dimensional [[Pythagorean theorem]]:<ref name=Wood>

{{cite book |title=Statistical methods: the geometric approach |author= David J. Saville, Graham R. Wood |chapter=§2.5 Sum of squares |page=30 |chapter-url=https://books.google.com/books?id=8ummgMVRev0C&pg=PA30 |isbn=0-387-97517-9 |year=1991 |edition=3rd |publisher=Springer}}

</ref>

Line 40:

The generalization of the ''n''-dimensional Pythagorean theorem to ''infinite-dimensional&thinsp;'' [[real number|real]] inner product spaces is known as [[Parseval's identity]] or Parseval's equation.<ref name=Folland3>

{{cite book |title=cited work |page =77 |chapter=Equation 3.22 |author=Gerald B Folland |chapter-url=https://books.google.com/books?id=ix2iCQ-o9x4C&pg=PA77 |isbn=978-0-8218-4790-29 |date=2009-01-13 |publisher =American Mathematical Soc. }}

</ref> Particular examples of such a representation of a function are the [[Fourier series]] and the [[generalized Fourier series]].

==Further discussion==

===Using linear algebra===

It follows that one can find a "best" approximation of another function by minimizing the area between two functions, a continuous function <math>f</math> on <math>[a,b]</math> and a function <math>g\in W</math> where <math>W</math> is a subspace of <math>C[a,b]</math>:

:<math>\text{Area} = \int_a^b \left\vert f(x) - g(x)\right\vert \, dx,</math>

all within the subspace <math>W</math>. Due to the frequent difficulty of evaluating integrands involving absolute value, one can instead define

:<math>\int_a^b [ f(x) - g(x) ] ^2\, dx</math>

as an adequate criterion for obtaining the least squares approximation, function <math>g</math>, of <math>f</math> with respect to the inner product space <math>W</math>.

As such, <math>\lVert f-g \rVert ^2</math> or, equivalently, <math>\lVert f-g \rVert</math>, can thus be written in vector form:

:<math>\int_a^b [ f(x)-g(x) ]^2\, dx = \left\langle f-g , f-g\right\rangle = \lVert f-g\rVert^2.</math>

In other words, the least squares approximation of <math>f</math> is the function <math>g\in \text{ subspace } W</math> closest to <math>f</math> in terms of the inner product <math>\left \langle f,g \right \rangle</math>. Furthermore, this can be applied with a theorem:

:Let <math>f</math> be continuous on <math>[ a,b ]</math>, and let <math>W</math> be a finite-dimensional subspace of <math>C[a,b]</math>. The least squares approximating function of <math>f</math> with respect to <math>W</math> is given by

::<math>g = \left \langle f,\vec w_1 \right \rangle \vec w_1 + \left \langle f,\vec w_2 \right \rangle \vec w_2 + \cdots + \left \langle f,\vec w_n \right \rangle \vec w_n,</math>

:where <math>B = \{\vec w_1 , \vec w_2 , \dots , \vec w_n \}</math> is an orthonormal basis for <math>W</math>.

==References==