Content deleted Content added
m Open access bot: url-access updated in citation with #oabot. |
TokenByToken (talk | contribs) category |
||
| (12 intermediate revisions by 4 users not shown) | |||
Line 3:
Sums of radial basis functions are typically used to [[function approximation|approximate given functions]]. This approximation process can also be interpreted as a simple kind of [[artificial neural network|neural network]]; this was the context in which they were originally applied to [[machine learning]], in work by [[David Broomhead]] and David Lowe in 1988,<ref>[http://www.anc.ed.ac.uk/rbf/intro/node8.html Radial Basis Function networks] {{webarchive|url=https://web.archive.org/web/20140423232029/http://www.anc.ed.ac.uk/rbf/intro/node8.html |date=2014-04-23 }}</ref><ref>{{cite journal |first1 = David H. |last1 = Broomhead |first2 = David |last2 = Lowe |title = Multivariable Functional Interpolation and Adaptive Networks |journal = Complex Systems |volume = 2 |pages = 321–355 |year = 1988 |url = https://www.complex-systems.com/pdf/02-3-5.pdf |archive-url = https://web.archive.org/web/20140714173428/https://www.complex-systems.com/pdf/02-3-5.pdf |archive-date = 2014-07-14}}</ref> which stemmed from [[Michael J. D. Powell]]'s seminal research from 1977.<ref>{{cite journal |title = Restart procedures for the conjugate gradient method |author = Michael J. D. Powell |journal = [[Mathematical Programming]] |volume = 12 |number = 1 |pages = 241–254 |year = 1977 |doi=10.1007/bf01593790|s2cid = 9500591 |author-link = Michael J. D. Powell }}</ref><ref>{{cite thesis |type = M.Sc. |first = Ferat |last = Sahin |title = A Radial Basis Function Approach to a Color Image Classification Problem in a Real Time Industrial Application |publisher = [[Virginia Tech]] |year = 1997 |quote = Radial basis functions were first introduced by Powell to solve the real multivariate interpolation problem. |page = 26 |hdl = 10919/36847 |url = http://hdl.handle.net/10919/36847 }}</ref><ref name="CITEREFBroomheadLowe1988">{{Harvnb|Broomhead|Lowe|1988|p=347}}: "We would like to thank Professor M.J.D. Powell at the Department of Applied Mathematics and Theoretical Physics at Cambridge University for providing the initial stimulus for this work."</ref><!--this doesn't seem to be working, probably a bug with {{sfn}}: <ref>{{sfn|Broomhead|Lowe|1988|p=347}}: "We would like to thank Professor M.J.D. Powell at the Department of Applied Mathematics and Theoretical Physics at Cambridge University for providing the initial stimulus for this work."</ref>-->
RBFs are also used as a [[Radial basis function kernel|kernel]] in [[support vector machine|support vector classification]].<ref>{{cite web |url=https://beta.oreilly.com/learning/intro-to-svm |title=Introduction to Support Vector Machines |last=VanderPlas |first=Jake |publisher=[O'Reilly] |date=6 May 2015 |access-date=14 May 2015 |archive-date=5 September 2015 |archive-url=https://web.archive.org/web/20150905100859/https://beta.oreilly.com/learning/intro-to-svm |url-status=dead }}</ref> The technique has proven effective and flexible enough that radial basis functions are now applied in a variety of engineering applications.<ref>{{Cite book|title=Radial basis functions : theory and implementations|first=Martin Dietrich|last=Buhmann|date=2003|publisher=Cambridge University Press|isbn=978-0511040207|oclc=56352083}}</ref><ref>{{Cite book|title=Fast radial basis functions for engineering applications|last=Biancolini|first=Marco Evangelos|date=2018|isbn=9783319750118|publisher=Springer International Publishing|oclc=1030746230}}</ref>
== Definition ==
Line 32:
{{NumBlk||<math display="block">\varphi(r) = e^{-(\varepsilon r)^2}, </math>|{{EquationRef|2}}}}
[[File:Gaussian function shape parameter.png|thumb|right|[[Gaussian function]] for several choices of <math>\varepsilon</math>]]
[[File:WiKipic1new.svg|thumb|Comparison of RTH, Multiquadric (ε = 2), and Linear RBFs]]
[[File:Bump function shape.png|thumb|Plot of the scaled [[bump function]] with several choices of <math>\varepsilon</math>]]
| [[Inverse quadratic]]:
{{NumBlk||<math display="block">\varphi(r) = \dfrac{1}{1+(\varepsilon r)^2}, </math>|{{EquationRef|
| [[Inverse multiquadric]]:
{{NumBlk||<math display="block">\varphi(r) = \dfrac{1}{\sqrt{1 + (\varepsilon r)^2}}, </math>|{{EquationRef|
}}
|Other Infinitely Smooth RBFs
{{pb}}
These radial basis functions are also from <math>C^\infty(\mathbb{R})</math> and require tuning a shape parameter <math>\varepsilon</math>, but they are not strictly [[Positive-definite function|positive definite]].
{{bulleted list
| [[Multiquadric]]:
{{NumBlk||<math display="block">\varphi(r) = \sqrt{1 + (\varepsilon r)^2}, </math>|{{EquationRef|5}}}}
| RTH:<ref>{{cite journal
| last1 = Heidari
| first1 = Mohammad
| last2 = Mohammadi
| first2 = Maryam
| last3 = De Marchi
| first3 = Stefano
| authorlink3 = Stefano De Marchi
| title = A shape preserving quasi-interpolation operator based on a new transcendental RBF
| journal = Dolomites Research Notes on Approximation
| volume = 14
| issue = 1
| pages = 56–73
| year = 2021
| doi = 10.14658/PUPJ-DRNA-2021-1-6
}}</ref>
{{NumBlk||<math display="block">\varphi(r) = r \tanh(\varepsilon r), </math>|{{EquationRef|6}}}}
}}
| [[Polyharmonic spline]]:
{{NumBlk||<math display="block">\begin{aligned}
Line 43 ⟶ 73:
\\
\varphi(r) &= r^k \ln(r),& k&=2,4,6,\dotsc
\end{aligned} </math>|{{EquationRef|
''*For even-degree polyharmonic splines'' <math>(k = 2,4,6,\dotsc)</math>'', to avoid numerical problems at <math>r = 0</math> where <math>\ln(0) = -\infty</math>, the computational implementation is often written as <math>\varphi(r) = r^{k-1}\ln(r^r)</math>.''{{citation needed|date=May 2021}}
| [[Thin plate spline]] (a special polyharmonic spline):
{{NumBlk||<math display="block">\varphi(r) = r^2 \ln(r), </math>|{{EquationRef|
| Compactly [[Support (mathematics)|Supported]] RBFs
Line 61 ⟶ 91:
0 & \text{ otherwise}
\end{cases},
</math>|{{EquationRef|
}}
}}
Line 70 ⟶ 100:
Radial basis functions are typically used to build up [[function approximation]]s of the form
{{NumBlk||<math display="block">y(\mathbf{x}) = \sum_{i=1}^N w_i \, \varphi(\left\|\mathbf{x} - \mathbf{x}_i\right\|), </math>|{{EquationRef|
where the approximating function <math display="inline">y(\mathbf{x})</math> is represented as a sum of <math>N</math> radial basis functions, each associated with a different center <math display="inline">\mathbf{x}_i</math>, and weighted by an appropriate coefficient <math display="inline">w_i.</math> The weights <math display="inline">w_i</math> can be estimated using the matrix methods of [[Weighted least squares|linear least squares]], because the approximating function is ''linear'' in the weights <math display="inline">w_i</math>.
Line 81 ⟶ 111:
The sum
{{NumBlk||<math display="block">y(\mathbf{x}) = \sum_{i=1}^N w_i \, \varphi(\left\|\mathbf{x} - \mathbf{x}_i\right\|), </math>|{{EquationRef|
can also be interpreted as a rather simple single-layer type of [[artificial neural network]] called a [[radial basis function network]], with the radial basis functions taking on the role of the activation functions of the network. It can be shown that any [[continuous function]] on a [[Compact space|compact]] interval can in principle be interpolated with arbitrary accuracy by a sum of this form, if a sufficiently large number <math display="inline">N</math> of radial basis functions is used.
Line 91 ⟶ 121:
{{main|Kansa method}}
Radial basis functions are used to approximate functions and so can be used to discretize and numerically solve Partial Differential Equations (PDEs). This was first done in 1990 by E. J. Kansa who developed the first RBF based numerical method. It is called the [[Kansa method]] and was used to solve the elliptic [[Poisson's equation|Poisson equation]] and the linear [[advection-diffusion equation]]. The function values at points <math>\mathbf{x}</math> in the ___domain are approximated by the linear combination of RBFs:
{{NumBlk||<math display="block">u(\mathbf{x}) = \sum_{i=1}^N \lambda_i \, \varphi(\left\|\mathbf{x} - \mathbf{x}_i\right\|),\quad \mathbf{x}\in\R^d </math>|{{EquationRef|
The derivatives are approximated as such:
{{NumBlk||<math display="block">\frac{\partial^n u(\textbf{x})}{\partial x^n} = \sum_{i=1}^N \lambda_i \, \frac{\partial^n}{\partial x^n}\varphi(\left\|\mathbf{x} - \mathbf{x}_i\right\|),\quad \mathbf{x}\in\R^d </math>|{{EquationRef|
where <math>N</math> are the number of points in the discretized ___domain, <math>d</math> the dimension of the ___domain and <math>\lambda</math> the scalar coefficients that are unchanged by the differential operator.<ref>{{Cite journal | last=Kansa | first=E. J. | date=1990-01-01|title=Multiquadrics—A scattered data approximation scheme with applications to computational fluid-dynamics—II solutions to parabolic, hyperbolic and elliptic partial differential equations|journal=Computers & Mathematics with Applications|language=en|volume=19 | issue=8 | pages=147–161|doi=10.1016/0898-1221(90)90271-K|issn=0898-1221|doi-access=free}}</ref>
Line 111 ⟶ 141:
*{{cite journal|last1=Hardy|first1=R.L.|year=1971|title=Multiquadric equations of topography and other irregular surfaces|journal=Journal of Geophysical Research|volume=76|issue=8|pages=1905–1915|doi=10.1029/jb076i008p01905|bibcode=1971JGR....76.1905H}}
* {{cite journal | last1 = Hardy | first1 = R.L. | year = 1990 | title = Theory and applications of the multiquadric-biharmonic method, 20 years of Discovery, 1968 1988 | journal = Comp. Math Applic | volume = 19 | issue = 8/9| pages = 163–208 | doi=10.1016/0898-1221(90)90272-l| doi-access = free }}
* {{Citation |last1 = Press |first1 = WH |last2 = Teukolsky |first2 = SA |last3 = Vetterling |first3 = WT |last4 = Flannery |first4 = BP |year = 2007 |title = Numerical Recipes: The Art of Scientific Computing |edition = 3rd |publisher = Cambridge University Press
* Sirayanone, S., 1988, Comparative studies of kriging, multiquadric-biharmonic, and other methods for solving mineral resource problems, PhD. Dissertation, Dept. of Earth Sciences, Iowa State University, Ames, Iowa.
* {{cite journal | last1 = Sirayanone | first1 = S. | last2 = Hardy | first2 = R.L. | year = 1995 | title = The Multiquadric-biharmonic Method as Used for Mineral Resources, Meteorological, and Other Applications | journal = Journal of Applied Sciences and Computations | volume = 1 | pages = 437–475 }}
Line 118 ⟶ 148:
[[Category:Interpolation]]
[[Category:Numerical analysis]]
[[Category:1988 in artificial intelligence]]
| |||