Content deleted Content added
Iridescent (talk | contribs) m Cleanup & typo fixing , typos fixed: the the → the (2), an unique → a unique using AWB |
|||
Line 37:
is known as a "normalized radial basis function".
[[Image:060804 3 normalized basis functions.png|thumb|350px|right|Figure 4: Three normalized radial basis functions in one input dimension. The additional basis function has center at <math> c_3=2.75 </math>
====Theoretical motivation for normalization====
There is theoretical justification for this architecture in the case of stochastic data flow. Assume a [[stochastic kernel]] approximation for the joint probability density
Line 48:
:<math> \int \sigma \big ( \left \vert y - e_i \right \vert \big ) \, dy =1</math>.
[[Image:060804 4 normalized basis functions.png|thumb|350px|right|Figure 5: Four normalized radial basis functions in one input dimension. The fourth basis function has center at <math> c_4=0 </math>. Note that the first basis function (dark blue) has become localized.
The probability densities in the input and output spaces are
Line 174:
where the entries of ''G'' are the values of the radial basis functions evaluated at the points <math>x_i</math>: <math>g_{ji} = \rho(||x_j-c_i||)</math>.
The existence of this linear solution means that unlike [[
====Gradient descent training of the linear weights====
Line 216:
For one basis function, projection operator training reduces to [[Newton's method]].
[[Image:060731 logistic map time series 2.png|thumb|350px|right|Figure 6: Logistic map time series. Repeated iteration of the logistic map generates a chaotic time series. The values lie between zero and one. Displayed here are the 100 training points used to train the examples in this section. The weights c are the first five points from this time series.
==Examples==
Line 227:
where t is a time index. The value of x at time t+1 is a parabolic function of x at time t. This equation represents the underlying geometry of the chaotic time series generated by the logistic map.
Generation of the time series from this equation is the [[forward problem]]. The examples here illustrate the [[inverse problem]]; identification of
:<math> x(t+1) = f \left [ x(t) \right ] \approx \varphi(t) = \varphi \left [ x(t)\right ] </math>
Line 238:
The architecture is
[[Image:060728b unnormalized basis function phi.png|thumb|350px|right|Figure 7: Unnormalized basis functions. The Logistic map (blue) and the approximation to the logistic map (red) after one pass through the training set.
:<math> \varphi ( \mathbf{x} ) \ \stackrel{\mathrm{def}}{=}\ \sum_{i=1}^N a_i \rho \big ( \left \Vert \mathbf{x} - \mathbf{c}_i \right \Vert \big ) </math>
Line 251:
where the learning rate <math> \nu </math> is taken to be 0.3. The training is performed with one pass through the 100 training points. The [[Mean squared error|rms error]] is 0.15.
[[Image:Normalized basis functions.png|thumb|350px|right|Figure 8: Normalized basis functions. The Logistic map (blue) and the approximation to the logistic map (red) after one pass through the training set. Note the improvement over the unnormalized case.
====Normalized radial basis functions====
Line 272:
where the learning rate <math> \nu </math> is again taken to be 0.3. The training is performed with one pass through the 100 training points. The [[Mean squared error|rms error]] on a test set of 100 exemplars is 0.084, smaller than the unnormalized error. Normalization yields accuracy improvement. Typically accuracy with normalized basis functions increases even more over unnormalized functions as input dimensionality increases.
[[Image:060803b chaotic time series prediction.png|thumb|350px|right|Figure 9: Normalized basis functions. The Logistic map (blue) and the approximation to the logistic map (red) as a function of time. Note that the approximation is good for only a few time steps. This is a general characterisitc of chaotic time series.
===Time series prediction===
Line 284:
:<math> {x}(t+1) \approx \varphi(t)=\varphi [\varphi(t-1)]</math>.
A comparison of the actual and estimated time series is displayed in the figure. The estimated times series starts out at time zero with an exact knowledge of x(0). It then uses the estimate of the dynamics to update
Note that the estimate is accurate for only a few time steps. This is a general characteristic of chaotic time series. This is a property of the sensitive dependence on initial conditions common to chaotic time series. A small initial error is amplified with time. A measure of the divergence of time series with nearly identical initial conditions is known as the [[Lyapunov exponent]].
Line 321:
* J. Moody and C. J. Darken, "Fast learning in networks of locally tuned processing units," Neural Computation, 1, 281-294 (1989). Also see [http://www.ki.inf.tu-dresden.de/~fritzke/FuzzyPaper/node5.html Radial basis function networks according to Moody and Darken]
* T. Poggio and F. Girosi, "Networks for approximation and learning," Proc. IEEE 78(9), 1484-1487 (1990).
* [[Roger Jones (physicist and entrepreneur)
* {{cite book | author=Martin D. Buhmann, M. J. Ablowitz | title=Radial Basis Functions : Theory and Implementations | publisher= Cambridge University| year=2003 | id=ISBN 0-521-63338-9}}
* {{cite book | author=Yee, Paul V. and Haykin, Simon | title=Regularized Radial Basis Function Networks: Theory and Applications | publisher= John Wiley| year=2001 | id=ISBN 0-471-35349-3}}
* John R. Davies, Stephen V. Coggeshall, [[Roger Jones (physicist and entrepreneur)
* {{cite book | author=Simon Haykin | title=Neural Networks: A Comprehensive Foundation | edition=2nd edition | ___location=Upper Saddle River, NJ | publisher=Prentice Hall| year=1999 | id=ISBN 0-13-908385-5}}
| |||