Content deleted Content added
lists; equation numbering/anchoring |
|||
Line 1:
A '''radial basis function''' ('''RBF''') is a [[real-valued function]] <math display="inline">\varphi</math> whose value depends only on the distance between the input and some fixed point, either the [[Origin (mathematics)|origin]], so that <math display="inline">\varphi(\mathbf{x}) = \hat\varphi(\left\|\mathbf{x}\right\|)</math>, or some other fixed point <math display="inline">\mathbf{c}</math>, called a ''center'', so that <math display="inline">\varphi(\mathbf{x}) = \hat\varphi(\left\|\mathbf{x}-\mathbf{c}\right\|)</math>. Any function <math display="inline">\varphi</math> that satisfies the property <math display="inline">\varphi(\mathbf{x}) = \hat\varphi(\left\|\mathbf{x}\right\|)</math> is a [[radial function]]. The distance is usually [[Euclidean distance]], although other [[distance function|metric]]s are sometimes used. They are often used as a collection <math>\{ \varphi_k \}_k</math> which forms a [[Basis (linear algebra)|basis]] for some [[function space]] of interest, hence the name.
Sums of radial basis functions are typically used to [[function approximation|approximate given functions]]. This approximation process can also be interpreted as a simple kind of [[artificial neural network|neural network]]; this was the context in which they were originally applied to machine learning, in work by [[David Broomhead]] and David Lowe in 1988,<ref>[http://www.anc.ed.ac.uk/rbf/intro/node8.html Radial Basis Function networks] {{webarchive|url=https://web.archive.org/web/20140423232029/http://www.anc.ed.ac.uk/rbf/intro/node8.html |date=2014-04-23 }}</ref><ref>{{cite journal |first1 = David H. |last1 = Broomhead |first2 = David |last2 = Lowe |title = Multivariable Functional Interpolation and Adaptive Networks |journal = Complex Systems |volume = 2 |pages = 321–355 |year = 1988 |url = https://www.complex-systems.com/pdf/02-3-5.pdf |archive-url = https://web.archive.org/web/20140714173428/https://www.complex-systems.com/pdf/02-3-5.pdf |archive-date = 2014-07-14}}</ref> which stemmed from [[Michael J. D. Powell]]'s seminal research from 1977.<ref>{{cite journal |title = Restart procedures for the conjugate gradient method |author = Michael J. D. Powell |journal = [[Mathematical Programming]] |volume = 12 |number = 1 |pages = 241–254 |year = 1977 |doi=10.1007/bf01593790|s2cid = 9500591 |author-link = Michael J. D. Powell }}</ref><ref>{{cite thesis |type = M.Sc. |first = Ferat |last = Sahin |title = A Radial Basis Function Approach to a Color Image Classification Problem in a Real Time Industrial Application |publisher = [[Virginia Tech]] |year = 1997 |quote = Radial basis functions were first introduced by Powell to solve the real multivariate interpolation problem. |page = 26 |hdl = 10919/36847 |url = http://hdl.handle.net/10919/36847 }}</ref><ref name="CITEREFBroomheadLowe1988">{{Harvnb|Broomhead|Lowe|1988|p=347}}: "We would like to thank Professor M.J.D. Powell at the Department of Applied Mathematics and Theoretical Physics at Cambridge University for providing the initial stimulus for this work."</ref><!--this doesn't seem to be working, probably a bug with {{sfn}}: <ref>{{sfn|Broomhead|Lowe|1988|p=347}}: "We would like to thank Professor M.J.D. Powell at the Department of Applied Mathematics and Theoretical Physics at Cambridge University for providing the initial stimulus for this work."</ref>-->
Line 6:
== Definition ==
A radial function is a function <math display="inline">\varphi:[0,\infty) \to \mathbb{R}</math>. When paired with a metric on a vector space <math display="inline"> \|\cdot\|:V \to [0,\infty)</math> a function <math display="inline"> \varphi_\mathbf{c} = \varphi(\|\mathbf{x}-\mathbf{c}\|) </math> is said to be a radial kernel centered at <math display="inline"> \mathbf{c} </math>. A Radial function and the associated radial kernels are said to be radial basis functions if, for any set of nodes <math>\{\mathbf{x}_k\}_{k=1}^n</math>
{{bulleted list
{{NumBlk||<math display="block">▼
▲* The kernels <math>\varphi_{\mathbf{x}_1}, \varphi_{\mathbf{x}_2}, \dots, \varphi_{\mathbf{x}_n}</math> form a basis for a [[Haar space|Haar Space]], meaning that the [[radial basis function interpolation|interpolation matrix]]
▲<math display="block">
\begin{bmatrix}
\varphi(\|\mathbf{x}_1 - \mathbf{x}_1\|) & \varphi(\|\mathbf{x}_2 - \mathbf{x}_1\|) & \dots & \varphi(\|\mathbf{x}_n - \mathbf{x}_1\|) \\
Line 16 ⟶ 15:
\vdots & \vdots & \ddots & \vdots \\
\varphi(\|\mathbf{x}_1 - \mathbf{x}_n\|) & \varphi(\|\mathbf{x}_2 - \mathbf{x}_n\|) & \dots & \varphi(\|\mathbf{x}_n - \mathbf{x}_n\|) \\
\end{bmatrix},
</math>|{{EquationRef|1}}}}
is non-singular.<ref>{{cite book |last1=Fasshauer |first1=Gregory E. |title=Meshfree Approximation Methods with MATLAB |date=2007 |publisher=World Scientific Publishing Co. Pte. Ltd. |___location=Singapore |isbn=9789812706331 |pages=17–25}}</ref><ref name="wendland2005">{{cite book |last1=Wendland |first1=Holger |title=Scattered Data Approximation |date=2005 |publisher=Cambridge University Press |___location=Cambridge |isbn=0521843359 |pages=11, 18-23,64-66}}</ref>▼
}}
▲<ref>{{cite book |last1=Fasshauer |first1=Gregory E. |title=Meshfree Approximation Methods with MATLAB |date=2007 |publisher=World Scientific Publishing Co. Pte. Ltd. |___location=Singapore |isbn=9789812706331 |pages=17–25}}</ref>
=== Examples ===
Commonly used types of radial basis functions include (writing <math display="inline">r = \left\|\mathbf{x} - \mathbf{x}_i\right\|</math> and using <math display="inline">\varepsilon </math> to indicate a ''shape parameter'' that can be used to scale the input of the radial kernel<ref>{{cite book |last1=Fasshauer |first1=Gregory E. |title=Meshfree Approximation Methods with MATLAB |date=2007 |publisher=World Scientific Publishing Co. Pte. Ltd. |___location=Singapore |isbn=9789812706331 |page=37}}</ref>):
{{bulleted list
These radial basis functions are from <math>C^\infty(\mathbb{R})</math> and are strictly [[Positive-definite function|positive definite functions]]<ref>{{cite book |last1=Fasshauer |first1=Gregory E. |title=Meshfree Approximation Methods with MATLAB |date=2007 |publisher=World Scientific Publishing Co. Pte. Ltd. |___location=Singapore |isbn=9789812706331 |pages=37–45}}</ref> that require tuning a shape parameter <math>\varepsilon</math>▼
{{pb}}
:* [[Gaussian function|Gaussian]]:<math display="block">\varphi(r) = e^{-(\varepsilon r)^2}, \qquad (2)</math>▼
▲These radial basis functions are from <math>C^\infty(\mathbb{R})</math> and are strictly [[Positive-definite function|positive definite functions]]<ref>{{cite book |last1=Fasshauer |first1=Gregory E. |title=Meshfree Approximation Methods with MATLAB | date = 2007 |publisher=World Scientific Publishing Co. Pte. Ltd. |___location=Singapore |isbn=9789812706331 |pages=37–45}}</ref> that require tuning a shape parameter <math>\varepsilon</math>
{{bulleted list
▲
[[File:Gaussian function shape parameter.png|thumb|right|[[Gaussian function]] for several choices of <math>\varepsilon</math>]]
[[File:Bump function shape.png|thumb|Plot of the scaled [[bump function]] with several choices of <math>\varepsilon</math>]]
| [[Multiquadric]]:
| [[Inverse quadratic]]:
| [[Inverse multiquadric]]:
}}
{{NumBlk||<math display="block">\begin{aligned} \varphi(r) &= r^k,& k&=1,3,5,\dotsc
\\
\varphi(r) &= r^k \ln(r),& k&=2,4,6,\dotsc
\end{aligned} </math>|{{EquationRef|6}}}}
{{NumBlk||<math display="block">\varphi(r) = r^2 \ln(r), {{pb}}
These RBFs are compactly supported and thus are non-zero only within a radius of <math>1 / \varepsilon</math>, and thus have sparse differentiation matrices
{{bulleted list |
▲:* [[Bump function]]:
[[Bump function]]:
{{NumBlk||
<math display="block">\varphi(r) =
\begin{cases}
\exp\left( -\frac{1}{1 - (\varepsilon r)^2}\right) & \
0 & \
\end{cases},
</math>|{{EquationRef|8}}}}
}}
}}
==Approximation==
Line 60 ⟶ 70:
{{main|Radial basis function interpolation}}
Radial basis functions are typically used to build up [[function approximation]]s of the form
{{NumBlk||<math display="block">y(\mathbf{x}) = \sum_{i=1}^N w_i \, \varphi(\left\|\mathbf{x} - \mathbf{x}_i\right\|), where the approximating function <math display="inline">y(\mathbf{x})</math> is represented as a sum of <math>N</math> radial basis functions, each associated with a different center <math display="inline">\mathbf{x}_i</math>, and weighted by an appropriate coefficient <math display="inline">w_i.</math> The weights <math display="inline">w_i</math> can be estimated using the matrix methods of [[Weighted least squares|linear least squares]], because the approximating function is ''linear'' in the weights
▲where the approximating function <math display="inline">y(\mathbf{x})</math> is represented as a sum of <math>N</math> radial basis functions, each associated with a different center <math display="inline">\mathbf{x}_i</math>, and weighted by an appropriate coefficient <math display="inline">w_i.</math> The weights <math display="inline">w_i</math> can be estimated using the matrix methods of [[Weighted least squares|linear least squares]], because the approximating function is ''linear'' in the weights ''<math display="inline">w_i</math>''.
Approximation schemes of this kind have been particularly used{{citation needed|date=July 2013}} in [[time series prediction]] and [[Control theory|control]] of [[nonlinear systems]] exhibiting sufficiently simple [[chaos theory|chaotic]] behaviour and 3D reconstruction in [[computer graphics]] (for example, [[hierarchical RBF]] and [[Pose Space Deformation]]).
Line 71 ⟶ 81:
[[File:Unnormalized radial basis functions.svg|thumb|350px|right|Two unnormalized Gaussian radial basis functions in one input dimension. The basis function centers are located at <math display="inline">x_1 = 0.75</math> and <math display="inline">x_2 = 3.25</math>.]]
The sum
The sum<math display="block">y(\mathbf{x}) = \sum_{i=1}^N w_i \, \varphi(\left\|\mathbf{x} - \mathbf{x}_i\right\|), \qquad (10)</math>can also be interpreted as a rather simple single-layer type of [[artificial neural network]] called a [[radial basis function network]], with the radial basis functions taking on the role of the activation functions of the network. It can be shown that any continuous function on a [[Compact space|compact]] interval can in principle be interpolated with arbitrary accuracy by a sum of this form, if a sufficiently large number ''<math display="inline">N</math>'' of radial basis functions is used. ▼
{{NumBlk||<math display="block">
▲
The approximant <math display="inline">y(\mathbf{x})</math> is differentiable with respect to the weights
Using radial basis functions in this manner yields a reasonable interpolation approach provided that the fitting set has been chosen such that it covers the entire range systematically (equidistant data points are ideal). However, without a polynomial term that is orthogonal to the radial basis functions, estimates outside the fitting set tend to perform poorly.
== RBFs for PDEs ==
{{main|Kansa method}}
Radial basis functions are used to approximate functions and so can be used to discretize and numerically solve Partial Differential Equations (PDEs). This was first done in 1990 by E. J. Kansa who developed the first RBF based numerical method. It is called the [[Kansa method]] and was used to solve the elliptic [[Poisson's equation|Poisson equation]] and the linear [[advection-diffusion equation]]. The function values at points <math>\mathbf{x}</math> in the ___domain are approximated by the linear combination of RBFs:
{{NumBlk||<math display="block">
▲<math display="block">u(\mathbf{x}) = \sum_{i=1}^N \lambda_i \, \varphi(\left\|\mathbf{x} - \mathbf{x}_i\right\|),\quad \mathbf{x}\in\R^d \qquad (11)</math>
The derivatives are approximated as such:
{{NumBlk||<math display="block">\frac{\partial^n u(\textbf{x})}{\partial x^n} = \sum_{i=1}^N \lambda_i \, \frac{\partial^n}{\partial x^n}\varphi(\left\|\mathbf{x} - \mathbf{x}_i\right\|),\quad \mathbf{x}\in\R^d </math>|{{EquationRef|12}}}}
where <math>N</math> are the number of points in the discretized ___domain, <math>d</math> the dimension of the ___domain and <math>\lambda</math> the scalar coefficients that are unchanged by the differential operator.<ref>{{Cite journal | last=Kansa | first=E. J. | date=1990-01-01|title=Multiquadrics—A scattered data approximation scheme with applications to computational fluid-dynamics—II solutions to parabolic, hyperbolic and elliptic partial differential equations|url=https://www.sciencedirect.com/science/article/pii/089812219090271K|journal=Computers & Mathematics with Applications|language=en|volume=19 | issue=8 | pages=147–161|doi=10.1016/0898-1221(90)90271-K|issn=0898-1221|doi-access=free}}</ref>▼
Different numerical methods based on Radial Basis Functions were developed thereafter. Some methods are the RBF-FD method,<ref>{{Cite journal|last=Tolstykh|first=A. I.|last2=Shirobokov|first2=D. A.|date=2003-12-01|title=On using radial basis functions in a “finite difference mode” with applications to elasticity problems|url=https://
▲<math display="block">\frac{\partial^n u(\textbf{x})}{\partial x^n} = \sum_{i=1}^N \lambda_i \, \frac{\partial^n}{\partial x^n}\varphi(\left\|\mathbf{x} - \mathbf{x}_i\right\|),\quad \mathbf{x}\in\R^d \qquad (12)</math>
▲where <math>N</math> are the number of points in the discretized ___domain, <math>d</math> the dimension of the ___domain and <math>\lambda</math> the scalar coefficients that are unchanged by the differential operator.<ref>{{Cite journal|last=Kansa|first=E. J.|date=1990-01-01|title=Multiquadrics—A scattered data approximation scheme with applications to computational fluid-dynamics—II solutions to parabolic, hyperbolic and elliptic partial differential equations|url=https://www.sciencedirect.com/science/article/pii/089812219090271K|journal=Computers & Mathematics with Applications|language=en|volume=19|issue=8|pages=147–161|doi=10.1016/0898-1221(90)90271-K|issn=0898-1221|doi-access=free}}</ref>
▲Different numerical methods based on Radial Basis Functions were developed thereafter. Some methods are the RBF-FD method,<ref>{{Cite journal|last=Tolstykh|first=A. I.|last2=Shirobokov|first2=D. A.|date=2003-12-01|title=On using radial basis functions in a “finite difference mode” with applications to elasticity problems|url=https://doi.org/10.1007/s00466-003-0501-9|journal=Computational Mechanics|language=en|volume=33|issue=1|pages=68–79|doi=10.1007/s00466-003-0501-9|issn=1432-0924}}</ref><ref>{{Cite journal|last=Shu|first=C|last2=Ding|first2=H|last3=Yeo|first3=K. S|date=2003-02-14|title=Local radial basis function-based differential quadrature method and its application to solve two-dimensional incompressible Navier–Stokes equations|url=https://www.sciencedirect.com/science/article/pii/S0045782502006187|journal=Computer Methods in Applied Mechanics and Engineering|language=en|volume=192|issue=7|pages=941–954|doi=10.1016/S0045-7825(02)00618-7|issn=0045-7825}}</ref> the RBF-QR method<ref>{{Cite journal|last=Fornberg|first=Bengt|last2=Larsson|first2=Elisabeth|last3=Flyer|first3=Natasha|date=2011-01-01|title=Stable Computations with Gaussian Radial Basis Functions|url=https://epubs.siam.org/doi/10.1137/09076756X|journal=SIAM Journal on Scientific Computing|volume=33|issue=2|pages=869–892|doi=10.1137/09076756X|issn=1064-8275}}</ref> and the RBF-PUM method.<ref>{{Cite journal|last=Safdari-Vaighani|first=Ali|last2=Heryudono|first2=Alfa|last3=Larsson|first3=Elisabeth|date=2015-08-01|title=A Radial Basis Function Partition of Unity Collocation Method for Convection–Diffusion Equations Arising in Financial Applications|url=https://doi.org/10.1007/s10915-014-9935-9|journal=Journal of Scientific Computing|language=en|volume=64|issue=2|pages=341–367|doi=10.1007/s10915-014-9935-9|issn=1573-7691}}</ref>
==See also==
* [[Matérn covariance function]]
* [[Radial basis function interpolation]]
* [[Kansa method]]
== References ==
| |||