Functional regression: Difference between revisions

Revision as of 05:19, 21 May 2017

Revision as of 10:29, 22 May 2017

Line 1:

'''Functional regression''' is a version of [[Regression analysis|regression analysis]] when [[Dependent and independent variables|responses]] or [[Dependent and independent variables|covariates]] include [[Functional data analysis|functional data]]. Functional regression models can be classified into four types depending on whether the responses or covariates are functional or scalar: (i) scalar responses with functional covariates, (ii) functional responses with scalar covariates, (iii) functional responses with functional covariates, and (iv) scalar or functional responses with functional and scalar covariates. In addition, functional regression models can be [[Linear regression|linear]], partially linear, or [[Nonlinear regression|nonlinear]]. In particular, functional polynomial models, functional [[Semiparametric_regression#Index_models|single and multiple index models]] and functional [[Additive model|additive models]] are three special cases of functional nonlinear models.

__TOC__

Line 6:

Functional linear models (FLMs) are an extension of [[Linear regression|linear models]] (LMs). A linear model with scalar response <math>Y\in\mathbb{R}</math> and scalar covariates <math>X\in\mathbb{R}^p</math> can be written as

{{NumBlk|::|<math display="block">Y = \beta_0 + \langle X,\beta\rangle + \varepsilon,</math>|{{EquationRef|1}}}}

where <math>\langle\cdot,\cdot\rangle</math> denotes the [[Inner product space|inner product]] in [[Euclidean space|Euclidean space]], <math>\beta_0\in\mathbb{R}</math> and <math>\beta\in\mathbb{R}^p</math> denote the regression coefficients, and <math>\varepsilon</math> is a random error with [[Expected value|mean]] zero and finite [[Variance|variance]]. FLMs can be divided into two types based on the responses.

=== Functional linear models with scalar responses ===

Functional linear models with scalar responses can be obtained by replacing the scalar covariates <math>X</math> and the coefficient vector <math>\beta</math> in model ({{EquationNote|1}}) by a centered functional covariate <math>X^c(\cdot) = X(\cdot) - \mathbb{E}(X(\cdot))</math> and a coefficient function <math>\beta = \beta(\cdot)</math> with [[Domain of a function|___domain]] <math>\mathcal{T}</math>, respectively, and replacing the inner product in Euclidean space by that in [[Hilbert space]] [[Lp space|<math>L^2</math>]],

{{NumBlk|::|<math display="block">Y = \beta_0 + \langle X^c, \beta\rangle +\varepsilon = \beta_0 + \int_\mathcal{T} X^c(t)\beta(t)\,dt + \varepsilon,</math>|{{EquationRef|2}}}}

where <math>\langle \cdot, \cdot \rangle</math> here denotes the inner product in <math>L^2</math>. One approach to estimating <math>\beta_0</math> and <math>\beta(\cdot)</math> is to expand the centered covariate <math>X^c(\cdot)</math> and the coefficient function <math>\beta(\cdot)</math> in the same [[Basis function|functional basis]], for example, [[B-spline|B-spline]] basis or the eigenbasis used in the [[Karhunen&ndash;Lo&egrave;ve theorem|Karhunen&ndash;Lo&egrave;ve expansion]]. Suppose <math>\{\phi_k\}_{k=1}^\infty</math> is an [[Orthonormal basis|orthonormal basis]] of <math>L^2</math>. Expanding <math>X^c</math> and <math>\beta</math> in this basis, <math>X^c(\cdot) = \sum_{k=1}^\infty x_k \phi_k(\cdot)</math>, <math>\beta(\cdot) = \sum_{k=1}^\infty \beta_k \phi_k(\cdot)</math>, model ({{EquationNote|2}}) becomes

<math display="block">Y = \beta_0 + \sum_{k=1}^\infty \beta_k x_k +\varepsilon.</math>

For implementation, regularization is needed and can be done through truncation, <math>L^2</math> penalization or <math>L^1</math> penalization.<ref name=morr:15>Morris (2015). "Functional regression". ''Annual Review of Statistics and Its Application''. '''2''':321&ndash;359. [[Digital object identifier|doi]]:[http://doi.org/10.1146/annurev-statistics-010814-020413 10.1146/annurev-statistics-010814-020413].</ref>. In addition, a [[Reproducing kernel Hilbert space|reproducing kernel Hilbert space]] (RKHS) approach can also be used to estimate <math>\beta_0</math> and <math>\beta(\cdot)</math> in model ({{EquationNote|2}})<ref>Yuan and Cai (2010). "A reproducing kernel Hilbert space approach to functional linear regression". ''The Annals of Statistics''. '''38''' (6):3412&ndash;3444. [[Digital object identifier|doi]]:[http://doi.org/10.1214/09-AOS772 10.1214/09-AOS772].</ref>

<br />

Line 23:

For a functional response <math>Y(\cdot)</math> with ___domain <math>\mathcal{T}</math> and a functional covariate <math>X(\cdot)</math> with ___domain <math>\mathcal{S}</math>, two FLMs regressing <math>Y(\cdot)</math> on <math>X(\cdot)</math> have been considered<ref name=wang:16/><ref>Ramsay and [[Bernard Silverman|Silverman]] (2005). ''Functional data analysis'', 2nd ed., New York&#160;: Springer, [[Special:BookSources/038740080X|ISBN 0-387-40080-X]].</ref>. One of these two models is of the form

{{NumBlk|::|<math display="block">Y(t) = \beta_0(t) + \int_{\mathcal{S}} \beta(s,t) X^c(s)\,ds + \varepsilon(t),\ \text{for}\ t\in\mathcal{T},</math>|{{EquationRef|4}}}}

where <math>X^c(\cdot) = X(\cdot) - \mathbb{E}(X(\cdot))</math> is still the centered functional covariate, <math>\beta_0(\cdot)</math> and <math>\beta(\cdot,\cdot)</math> are coefficient functions, and <math>\varepsilon(\cdot)</math> is usually assumed to be a random process with mean zero and finite variance. In this case, at any given time <math>t\in\mathcal{T}</math>, the value of <math>Y</math>, i.e., <math>Y(t)</math>, depends on the entire trajectory of <math>X</math>. Model ({{EquationNote|4}}), for any given time <math>t</math>, is an extension of [[Multivariate linear regression|multivariate linear regression]] with the inner product in Euclidean space replaced by that in <math>L^2</math>. An estimating equation motivated by multivariate linear regression is

<math display="block">r_{XY} = R_{XX}\beta, \text{ for } \beta\in L^2(\mathcal{S}\times\mathcal{S}),</math>

where <math>r_{XY}(s,t) = \text{cov}(X(s),Y(t))</math>, <math>R_{XX}: L^2(\mathcal{S}\times\mathcal{S}) \rightarrow L^2(\mathcal{S}\times\mathcal{T})</math> is defined as <math>(R_{XX}\beta)(s,t) = \int_\mathcal{S} r_{XX}(s,w)\beta(w,t)dw</math> with <math>r_{XX}(s,w) = \text{cov}(X(s),X(w))</math> for <math>s,w\in\mathcal{S}</math><ref name=wang:16/>. Regularization is needed and can be done through truncation, <math>L^2</math> penalization or <math>L^1</math> penalization<ref name=morr:15/>. Various estimation methods for model ({{EquationNote|4}}) are available<ref>Ramsay and Dalzell (1991). "Some tools for functional data analysis". ''Journal of the Royal Statistical Society. Series B (Methodological)''. '''53''' (3):539&ndash;572. http://www.jstor.org/stable/2345586.</ref><ref>Yao, M&uuml;ller and Wang (2005). "Functional linear regression analysis for longitudinal data". ''The Annals of Statistics''. '''33''' (6):2873&ndash;2903. [[Digital object identifier|doi]]:[http://doi.org/10.1214/009053605000000660 10.1214/009053605000000660].</ref>.<br />

Line 43:

== Functional nonlinear models ==

=== Functional polynomial models ===

Functional polynomial models are an extension of the FLMs with scalar responses, analogous to extending linear regression to [[Polynomial regression|polynomial regression]]. For a scalar response <math>Y</math> and a functional covariate <math>X(\cdot)</math> with ___domain <math>\mathcal{T}</math>, the simplest example of functional polynomial models is functional quadratic regression<ref name=yao:10>Yao and M&uuml;ller (2010). "Functional quadratic regression". ''Biometrika''. '''97''' (1):49&ndash;64. [[Digital object identifier|doi]]:[http://doi.org/10.1093/biomet/asp069 10.1093/biomet/asp069].</ref>

<math display="block">Y = \alpha + \int_\mathcal{T}\beta(t)X^c(t)\,dt + \int_\mathcal{T} \int_\mathcal{T} \gamma(s,t) X^c(s)X^c(t) \,ds\,dt + \varepsilon,</math>

where <math>X^c(\cdot) = X(\cdot) - \mathbb{E}(X(\cdot))</math> is the centered functional covariate, <math>\alpha</math> is a scalar coefficient, <math>\beta(\cdot)</math> and <math>\gamma(\cdot,\cdot)</math> are coefficient functions with domains <math>\mathcal{T}</math> and <math>\mathcal{T}\times\mathcal{T}</math>, respectively, and <math>\varepsilon</math> is a random error with mean zero and finite variance. By analogy to FLMs with scalar responses, estimation of functional polynomial models can be obtained through expanding both the centered covariate <math>X^c</math> and the coefficient functions <math>\beta</math> and <math>\gamma</math> in an orthonormal basis.<ref name=yao:10/>

Line 50:

A functional multiple index model is given by

<math display="block">Y = g\left(\int_{\mathcal{T}} X^c(t) \beta_1(t)\,dt, \ldots, \int_{\mathcal{T}} X^c(t) \beta_p(t)\,dt \right) + \varepsilon.</math>

Taking <math>p=1</math> yields a functional single index model. However, for <math>p>1</math>, this model is problematic due to [[Curse of dimensionality|curse of dimensionality]]. With <math>p>1</math> and relatively small sample sizes, the estimator given by this model often has large variance.<ref name=chen:11>Chen, Hall and M&uuml;ller (2011). "Single and multiple index functional regression models with nonparametric link". ''The Annals of Statistics''. '''39''' (3):1720&ndash;1747. [[Digital object identifier|doi]]:[http://doi.org/10.1214/11-AOS882 10.1214/11-AOS882].</ref>. An alternative <math>p</math>-component functional multiple index model can be expressed as

<math display="block">Y = g_1\left(\int_{\mathcal{T}} X^c(t) \beta_1(t)\,dt\right)+ \cdots+ g_p\left(\int_{\mathcal{T}} X^c(t) \beta_p(t)\,dt \right) + \varepsilon.</math>

Estimation methods for functional single and multiple index models are available<ref name=chen:11/><ref>Jiang and Wang (2011). "Functional single index models for longitudinal data". '''39''' (1):362&ndash;388. [[Digital object identifier|doi]]:[http://doi.org/10.1214/10-AOS845 10.1214/10-AOS845].</ref>.

Line 64:

== Extensions ==

A direct extension of FLMs with scalar responses shown in model ({{EquationNote|2}}) is to add a link function to create a [[Generalized functional linear model|generalized functional linear model]] (GFLM) by analogy to extending [[Linear regression|linear regression]] to [[Generalized linear model|generalized linear regression]] (GLM), of which the three components are:

# Linear predictor <math>\eta = \beta_0 + \int_{\mathcal{T}} X^c(t)\beta(t)\,dt</math>;

# [[Variance function]] <math>\text{Var}(Y|X) = V(\mu)</math>, where <math>\mu = \mathbb{E}(Y|X)</math> is the [[Conditional expectation|conditional mean]];

Line 70:

== See also ==

* [[Functional data analysis|]]
* [[Functional dataprincipal component analysis]]

* [[Karhunen&ndash;Lo&egrave;ve_theorem|Karhunen&ndash;Lo&egrave;ve theorem]]

* [[Functional principal component analysis|Functional principal component analysis]]

* [[Generalized functional linear model|Generalized functional linear model]]

* [[Karhunen&ndash;Lo&egrave;ve_theorem|Karhunen&ndash;Lo&egrave;ve theorem]]

* [[Stochastic processes|Stochastic processes]]

* [[Generalized functional linear model|Generalized functional linear model]]

* [[Stochastic processes|Stochastic processes]]

== References ==