Neural modeling fields: Difference between revisions

Revision as of 18:51, 11 January 2009

Revision as of 19:25, 11 January 2009

Line 53:

==Learning in NMF using dynamic logic algorithm==

The learning process consists of estimating model parameters '''S''' and associating signals with concepts by maximizing the similarity L. Note that all possible combinations of signals and models are accounted for in expression (2) for L. This can be seen by expanding a sum and multiplying all the terms resulting in M^N items, a huge number. This is the number of combinations between all signals (N) and all models (M). This is the source of Combinatorial Complexity, which is solved in NMF by utilizing the idea of [[Perlovsky|dynamic logic]]<ref>Perlovsky, L.I. (1996). Mathematical Concepts of Intellect. Proc. World Congress on Neural Networks, San Diego, CA; Lawrence Erlbaum Associates, NJ, pp.1013-16</ref>, <ref>Perlovsky, L.I.(1997). Physical Concepts of Intellect. Proc. Russian Academy of Sciences, 354(3), pp. 320-323.</ref>. An important aspect of dynamic logic is ''matching vagueness or fuzziness of similarity measures to the uncertainty of models''. Initially, parameter values are not known, and uncertainty of models is high; so is the fuzziness of the similarity measures. In the process of learning, models become more accurate, and the similarity measure more crisp, the value of the similarity increases.

The maximization of similarity L is done as follows. First, the unknown parameters {'''S'''_m} are randomly initialized. Then the association variables f(m|n) are computed,

:<bigmath> f(m|n) = \frac{r(m) l(''' \vec X'''(n)|m)) /}{ &sum;_{\sum_{m'=1..}^M} { r(m') l(''' \vec X'''(n)|m').) } } </bigmath>&nbsp;&nbsp;&nbsp; (3).

Equation for f(m|n) looks like the Bayes formula for a posteriori probabilities; if l(n|m) in the result of learning become conditional likelihoods, f(m|n) become Bayesian probabilities for signal n originating from object m. The dynamic logic of the NMF is defined as follows:

:<bigmath> \frac{d'''S'''_m/ \vec S_m }{dt} = &sum;_{\sum_{n=1..}^N} { f(m|n)[&part; \frac{\partial{\ln l(n|m)/&part;'''M'''<sub>m} }{\partial{\vec M_m} } \frac{\partial{\vec M_m}}{\partial{\vec S_m}} } </submath>]&part;'''M'''_m/&part;'''S'''_m,</big> &nbsp;&nbsp;&nbsp; (4).

:<bigmath> \frac{df(m|n)/}{dt} = f(m|n) &sum;_{\sum_{m'=1..}^M} { [&delta;_{\delta_{mm'}} - f(m'|n)][&part; \frac{\partial{\ln l(n|m')/&part;'''M'''_{}}{\partial{\vec M_{m'}]}}} &part;'''M'''_{} \frac{\partial{\vec M_{m'}/&part;'''S'''_{}}}{\partial{\vec S_{m'}}}} \frac{d d'''S'''_{\vec S_{m'}/}}{dt,} </bigmath> &nbsp;&nbsp;&nbsp; (5)

The following theorem has been proved (Perlovsky 2001):

Line 69:

''Theorem''. Equations (3), (4), and (5) define a convergent dynamic NMF system with stationary states defined by max{S_m}L.

It follows that the stationary states of an MF system are the maximum similarity states. When partial similarities are specified as probability density functions (pdf), or likelihoods, the stationary values of parameters {'''S'''_m} are asymptotically unbiased and efficient estimates of these parameters<ref>Cramer, H. (1946). Mathematical Methods of Statistics, Princeton University Press, Princeton NJ.</ref>. The computational complexity of dynamic logic is linear in N.

Practically, when solving the equations through successive iterations, f(m|n) can be recomputed at every iteration using (3), as opposed to incremental formula (5).