Dynamic causal modeling

This sandbox is in the article namespace. Either move this page into your userspace, or remove the {{User sandbox}} template.

Bayesian model reduction

Bayesian model reduction is a method for computing the evidence and parameters of Bayesian models which differ only in the specification of their priors. Typically, a full model is fitted to the available data with standard approaches. Hypotheses are then tested by defining one or more 'reduced' models, which differ only in their priors. The evidence and parameters of the reduced models can be computed from the evidence and parameters of the full model. If the priors and posteriors are Gaussian, this has an analytic solution which can be computed rapidly. Bayesian model reduction has multiple scientific and engineering applications, including rapidly scoring large numbers of models and facilitating the estimation of hierarchical models (Parametric Empirical Bayes).

Theory

Consider some model with parameters ${\displaystyle \theta }$  and a prior probability density on those parameters ${\displaystyle p(\theta )}$ . The posterior belief about ${\displaystyle \theta }$  after seeing the data is given by Bayes rule:

${\displaystyle {\begin{aligned}p(\theta |y)&={\frac {p(y|\theta )p(\theta )}{p(y)}}\\p(y)&=\int p(y|\theta )p(\theta )d\theta \end{aligned}}}$ 

The second line of the equation is the model evidence, which is the probability of observing the data given the model. In practice, the posterior cannot usually be computed analytically due to the integral. Therefore, the posteriors are estimated using approaches such as MCMC sampling or variational Bayes. Having estimated the posteriors and evidence using one of these approaches, a reduced model can be defined with an alternative set of priors ${\displaystyle {\tilde {p}}(\theta )}$ :

${\displaystyle {\begin{aligned}{\tilde {p}}(\theta |y)&={\frac {p(y|\theta ){\tilde {p}}(\theta )}{{\tilde {p}}(y)}}\\{\tilde {p}}(y)&=\int p(y|\theta ){\tilde {p}}(\theta )d\theta \end{aligned}}}$ 

The objective is to compute the reduced posterior ${\displaystyle {\tilde {p}}(\theta |y)}$  and evidence ${\displaystyle {\tilde {p}}(y)}$  from the full posterior ${\displaystyle p(\theta |y)}$  and evidence ${\displaystyle p(y)}$ . We can express this as follows: