Functional regression: Difference between revisions

Revision as of 09:40, 20 May 2017

Revision as of 09:41, 20 May 2017

Line 1:

{{User sandbox}}

<!-- EDIT BELOW THIS LINE -->

'''Functional regression''' is a version of [[Regression analysis|regression analysis]] when [[Dependent and independent variables|responses]] or [[Dependent and independent variables|covariates]] include [[Functional data analysis|functional data]]. Functional regression models can be classified into four types depending on whether the responses or covariates are functional or scalar: (i) scalar responses with functional covariates, (ii) functional responses with scalar covariates, (iii) functional responses with functional covariates, and (iv) scalar or functional responses with functional and scalar covariates. In addition, functional regression models can be [[Linear regression|linear]], partially linear, or [[Nonlinear regression|nonlinear]]. In particular, functional polynomial models, functional [[Semiparametric_regression#Index_models|single and multiple index models]] and functional [[Additive model|additive models]] are three special cases of functional nonlinear models.

Line 15 ⟶ 13:

where <math>\langle \cdot, \cdot \rangle</math> here denotes the inner product in <math>L^2</math>. One approach to estimating <math>\beta_0</math> and <math>\beta(\cdot)</math> is to expand the centered covariate <math>X^c(\cdot)</math> and the coefficient function <math>\beta(\cdot)</math> in the same [[Basis function|functional basis]], for example, [[B-spline|B-spline]] basis or the eigenbasis used in the [[Karhunen&ndash;Lo&egrave;ve theorem|Karhunen&ndash;Lo&egrave;ve expansion]]. Suppose <math>\{\phi_k\}_{k=1}^\infty</math> is an [[Orthonormal basis|orthonormal basis]] of <math>L^2</math>. Expanding <math>X^c</math> and <math>\beta</math> in this basis, <math>X^c(\cdot) = \sum_{k=1}^\infty x_k \phi_k(\cdot)</math>, <math>\beta(\cdot) = \sum_{k=1}^\infty \beta_k \phi_k(\cdot)</math>, model ({{EquationNote|2}}) becomes

<math display="block">Y = \beta_0 + \sum_{k=1}^\infty \beta_k x_k +\epsilon.</math>

For implementation, regularization is needed and can be done through truncation, <math>L^2</math> penalization or <math>L^1</math> penalization.<ref name=morr:15>Morris (2015). "Functional regression". ''Annual Review of Statistics and Its Application''. '''2''':321&ndash;359. [[Digital object identifier|doi]]:[http://doi.org/10.1146/annurev-statistics-010814-020413 10.1146/annurev-statistics-010814-020413].</ref>. In addition, a [[Reproducing kernel Hilbert space|reproducing kernel Hilbert space]] (RKHS) approach can also be used to estimate <math>\beta_0</math> and <math>\beta(\cdot)</math> in model ({{EquationNote|2}})<ref>Yuan and Cai (2010). "A reproducing kernel Hilbert space approach to functional linear regression". ''The Annals of Statistics''. '''38''' (6):3412&ndash;3444. [[Digital object identifier|doi]]:[http://doi.org/10.1214/09-AOS772 10.1214/09-AOS772].</ref>.

<br />

Line 41 ⟶ 39:

Adding multiple functional covariates, model ({{EquationNote|6}}) can also be extended to

<math display="block">Y(t) = \alpha_0(t) + \sum_{j=1}^p\alpha_j(t)X_j(t)+\epsilon(t),\ \text{for}\ t\in\mathcal{T},</math>

where <math>X_1,\cdots,X_p</math> are multiple functional covariates with ___domain <math>\mathcal{T}</math> and <math>\alpha_0,\alpha_1,\cdots,\alpha_p</math> are the coefficient functions with the same ___domain.<ref name=wang:16/>.

== Functional nonlinear models ==

Line 47 ⟶ 45:

Functional polynomial models are an extension of the FLMs with scalar responses, analogous to extending linear regression to [[Polynomial regression|polynomial regression]]. For a scalar response <math>Y</math> and a functional covariate <math>X(\cdot)</math> with ___domain <math>\mathcal{T}</math>, the simplest example of functional polynomial models is functional quadratic regression<ref name=yao:10>Yao and M&uuml;ller (2010). "Functional quadratic regression". ''Biometrika''. '''97''' (1):49&ndash;64. [[Digital object identifier|doi]]:[http://doi.org/10.1093/biomet/asp069 10.1093/biomet/asp069].</ref>

<math display="block">Y = \alpha + \int_\mathcal{T}\beta(t)X^c(t)dt + \int_\mathcal{T} \int_\mathcal{T} \gamma(s,t) X^c(s)X^c(t) dsdt + \epsilon,</math>

where <math>X^c(\cdot) = X(\cdot) - \mathbb{E}(X(\cdot))</math> is the centered functional covariate, <math>\alpha</math> is a scalar coefficient, <math>\beta(\cdot)</math> and <math>\gamma(\cdot,\cdot)</math> are coefficient functions with domains <math>\mathcal{T}</math> and <math>\mathcal{T}\times\mathcal{T}</math>, respectively, and <math>\epsilon</math> is a random error with mean zero and finite variance. By analogy to FLMs with scalar responses, estimation of functional polynomial models can be obtained through expanding both the centered covariate <math>X^c</math> and the coefficient functions <math>\beta</math> and <math>\gamma</math> in an orthonormal basis.<ref name=yao:10/>.

=== Functional single and multiple index models ===

A functional multiple index model is given by

<math display="block">Y = g\left(\int_{\mathcal{T}} X^c(t) \beta_1(t)dt, \cdots, \int_{\mathcal{T}} X^c(t) \beta_p(t)dt \right) + \epsilon.</math>

Taking <math>p=1</math> yields a functional single index model. However, for <math>p>1</math>, this model is problematic due to [[Curse of dimensionality|curse of dimensionality]]. With <math>p>1</math> and relatively small sample sizes, the estimator given by this model often has large variance.<ref name=chen:11>Chen, Hall and M&uuml;ller (2011). "Single and multiple index functional regression models with nonparametric link". ''The Annals of Statistics''. '''39''' (3):1720&ndash;1747. [[Digital object identifier|doi]]:[http://doi.org/10.1214/11-AOS882 10.1214/11-AOS882].</ref>. An alternative <math>p</math>-component functional multiple index model can be expressed as

<math display="block">Y = g_1\left(\int_{\mathcal{T}} X^c(t) \beta_1(t)dt\right)+ \cdots+ g_p\left(\int_{\mathcal{T}} X^c(t) \beta_p(t)dt \right) + \epsilon.</math>

Estimation methods for functional single and multiple index models are available<ref name=chen:11/><ref>Jiang and Wang (2011). "Functional single index models for longitudinal data". '''39''' (1):362&ndash;388. [[Digital object identifier|doi]]:[http://doi.org/10.1214/10-AOS845 10.1214/10-AOS845].</ref>.

Line 63 ⟶ 61:

where <math>f_k</math> satisfies <math>\mathbb{E}(f_k(x_k))=0</math> for <math>k\in\mathbb{N}</math><ref name=wang:16/><ref>M&uuml;ller and Yao (2008). "Functional additive models". ''Journal of the American Statistical Association''. '''103''' (484):1534&ndash;1544. [[Digital object identifier|doi]]:[http://doi.org/10.1198/016214508000000751 10.1198/016214508000000751].</ref>. Another form of FAMs consists of a sequence of time-additive models:

<math display="block">\mathbb{E}(Y|X(t_1),\cdots,X(t_p))=\sum_{j=1}^p f_j(X(t_j)),</math>

where <math>\{t_1\cdots,t_p\}</math> is a dense grid on <math>\mathcal{T}</math> with increasing size <math>p\in\mathbb{N}</math>, and <math>f_j(x) = g(t_j,x)</math> with <math>g</math> a smooth function, for <math>j=1,\cdots,p</math><ref name=wang:16/><ref>Fan, James and Radchenko (2015). "Functional additive regression". ''The Annals of Statistics''. '''43''' (5):2296&ndash;2325. [[Digital object identifier|doi]]:[http://doi.org/10.1214/15-AOS1346 10.1214/15-AOS1346].</ref>.

== Extensions ==

Line 84 ⟶ 82:

== Functional regression ==

{{AFC submission|||ts=20170511005724|u=Ms.chen|ns=2}}