Content deleted Content added
m Replace magic links with templates per local RfC and MediaWiki RfC |
m WP:CHECKWIKI error fixes, References after punctuation per WP:CITEFOOT and WP:PAIC, added orphan tag using AWB |
||
Line 1:
{{Orphan|date=June 2017}}
'''Functional regression''' is a version of [[regression analysis]] when [[Dependent and independent variables|responses]] or [[Dependent and independent variables|covariates]] include [[Functional data analysis|functional data]]. Functional regression models can be classified into four types depending on whether the responses or covariates are functional or scalar: (i) scalar responses with functional covariates, (ii) functional responses with scalar covariates, (iii) functional responses with functional covariates, and (iv) scalar or functional responses with functional and scalar covariates. In addition, functional regression models can be [[Linear regression|linear]], partially linear, or [[Nonlinear regression|nonlinear]]. In particular, functional polynomial models, functional [[Semiparametric_regression#Index_models|single and multiple index models]] and functional [[Additive model|additive models]] are three special cases of functional nonlinear models.▼
▲'''Functional regression''' is a version of [[regression analysis]] when [[Dependent and independent variables|responses]] or [[Dependent and independent variables|covariates]] include [[Functional data analysis|functional data]]. Functional regression models can be classified into four types depending on whether the responses or covariates are functional or scalar: (i) scalar responses with functional covariates, (ii) functional responses with scalar covariates, (iii) functional responses with functional covariates, and (iv) scalar or functional responses with functional and scalar covariates. In addition, functional regression models can be [[Linear regression|linear]], partially linear, or [[Nonlinear regression|nonlinear]]. In particular, functional polynomial models, functional [[
__TOC__
Line 13 ⟶ 15:
where <math>\langle \cdot, \cdot \rangle</math> here denotes the inner product in <math>L^2</math>. One approach to estimating <math>\beta_0</math> and <math>\beta(\cdot)</math> is to expand the centered covariate <math>X^c(\cdot)</math> and the coefficient function <math>\beta(\cdot)</math> in the same [[Basis function|functional basis]], for example, [[B-spline]] basis or the eigenbasis used in the [[Karhunen–Loève theorem|Karhunen–Loève expansion]]. Suppose <math>\{\phi_k\}_{k=1}^\infty</math> is an [[orthonormal basis]] of <math>L^2</math>. Expanding <math>X^c</math> and <math>\beta</math> in this basis, <math>X^c(\cdot) = \sum_{k=1}^\infty x_k \phi_k(\cdot)</math>, <math>\beta(\cdot) = \sum_{k=1}^\infty \beta_k \phi_k(\cdot)</math>, model ({{EquationNote|2}}) becomes
<math display="block">Y = \beta_0 + \sum_{k=1}^\infty \beta_k x_k +\varepsilon.</math>
For implementation, regularization is needed and can be done through truncation, <math>L^2</math> penalization or <math>L^1</math> penalization.<ref name=morr:15>Morris (2015). "Functional regression". ''Annual Review of Statistics and Its Application''. '''2''':321–359. [[Digital object identifier|doi]]:[http://doi.org/10.1146/annurev-statistics-010814-020413 10.1146/annurev-statistics-010814-020413].</ref>
Adding multiple functional and scalar covariates, model ({{EquationNote|2}}) can be extended to
{{NumBlk|::|<math display="block">Y = \sum_{k=1}^q Z_k\alpha_k + \sum_{j=1}^p \int_{\mathcal{T}_j} X_j^c(t) \beta_j(t) \,dt + \varepsilon,</math>|{{EquationRef|3}}}}
where <math>Z_1,\ldots,Z_q</math> are scalar covariates with <math>Z_1=1</math>, <math>\alpha_1,\ldots,\alpha_q</math> are regression coefficients for <math>Z_1,\ldots,Z_q</math>, respectively, <math>X^c_j</math> is a centered functional covariate given by <math>X_j^c(\cdot) = X_j(\cdot) - \mathbb{E}(X_j(\cdot))</math>, <math>\beta_j</math> is regression coefficient function for <math>X_j^c(\cdot)</math>, and <math>\mathcal{T}_j</math> is the ___domain of <math>X_j</math> and <math>\beta_j</math>, for <math>j=1,\ldots,p</math>. However, due to the parametric component <math>\alpha</math>, the estimation methods for model ({{EquationNote|2}}) cannot be used in this case<ref name=wang:16>Wang, Chiou and Müller (2016). "Functional data analysis". ''Annual Review of Statistics and Its Application''. '''3''':257–295. [[Digital object identifier|doi]]:[http://doi.org/10.1146/annurev-statistics-041715-033624 10.1146/annurev-statistics-041715-033624].</ref> and alternative estimation methods for model ({{EquationNote|3}}) are available.<ref>Kong, Xue, Yao and Zhang (2016). "Partially functional linear regression in high dimensions". ''Biometrika''. '''103''' (1):147–159. [[Digital object identifier|doi]]:[http://doi.org/10.1093/biomet/asv062 10.1093/biomet/asv062].</ref><ref>Hu, Wang and Carroll (2004). "Profile-kernel versus backfitting in the partially linear models for longitudinal/clustered data". ''Biometrika''. '''91''' (2): 251–262. [[Digital object identifier|doi]]:[http://doi.org/10.1093/biomet/91.2.251 10.1093/biomet/91.2.251].</ref
=== Functional linear models with functional responses ===
For a functional response <math>Y(\cdot)</math> with ___domain <math>\mathcal{T}</math> and a functional covariate <math>X(\cdot)</math> with ___domain <math>\mathcal{S}</math>, two FLMs regressing <math>Y(\cdot)</math> on <math>X(\cdot)</math> have been considered.<ref name=wang:16/><ref>Ramsay and [[Bernard Silverman|Silverman]] (2005). ''Functional data analysis'', 2nd ed., New York : Springer, {{ISBN|0-387-40080-X}}.</ref>
{{NumBlk|::|<math display="block">Y(t) = \beta_0(t) + \int_{\mathcal{S}} \beta(s,t) X^c(s)\,ds + \varepsilon(t),\ \text{for}\ t\in\mathcal{T},</math>|{{EquationRef|4}}}}
where <math>X^c(\cdot) = X(\cdot) - \mathbb{E}(X(\cdot))</math> is still the centered functional covariate, <math>\beta_0(\cdot)</math> and <math>\beta(\cdot,\cdot)</math> are coefficient functions, and <math>\varepsilon(\cdot)</math> is usually assumed to be a random process with mean zero and finite variance. In this case, at any given time <math>t\in\mathcal{T}</math>, the value of <math>Y</math>, i.e., <math>Y(t)</math>, depends on the entire trajectory of <math>X</math>. Model ({{EquationNote|4}}), for any given time <math>t</math>, is an extension of [[multivariate linear regression]] with the inner product in Euclidean space replaced by that in <math>L^2</math>. An estimating equation motivated by multivariate linear regression is
<math display="block">r_{XY} = R_{XX}\beta, \text{ for } \beta\in L^2(\mathcal{S}\times\mathcal{S}),</math>
where <math>r_{XY}(s,t) = \text{cov}(X(s),Y(t))</math>, <math>R_{XX}: L^2(\mathcal{S}\times\mathcal{S}) \rightarrow L^2(\mathcal{S}\times\mathcal{T})</math> is defined as <math>(R_{XX}\beta)(s,t) = \int_\mathcal{S} r_{XX}(s,w)\beta(w,t)dw</math> with <math>r_{XX}(s,w) = \text{cov}(X(s),X(w))</math> for <math>s,w\in\mathcal{S}</math>.<ref name=wang:16/>
When <math>X</math> and <math>Y</math> are concurrently observed, i.e., <math>\mathcal{S}=\mathcal{T}</math>,<ref>Grenander (1950). "Stochastic processes and statistical inference". ''Arkiv Matematik''. '''1''' (3):195–277. [[Digital object identifier|doi]]:[http://doi.org/10.1007/BF02590638 10.1007/BF02590638].</ref>
Adding multiple functional covariates, model ({{EquationNote|4}}) can be extended to
{{NumBlk|::|<math display="block">Y(t) = \beta_0(t) + \sum_{j=1}^p\int_{\mathcal{S}_j} \beta_j(s,t) X^c_j(s)\,ds + \varepsilon(t),\ \text{for}\ t\in\mathcal{T},</math>|{{EquationRef|5}}}}
where for <math>j=1,\ldots,p</math>, <math>X_j^c(\cdot)=X_j(\cdot) - \mathbb{E}(X_j(\cdot))</math> is a centered functional covariate with ___domain <math>\mathcal{S}_j</math>, and <math>\beta_j(\cdot,\cdot)</math> is the corresponding coefficient function with the same ___domain, respectively.<ref name=wang:16/>
<math display="block">Y(t) = \sum_{j=1}^p X_j \beta_j(t) + \varepsilon(t),\ \text{for}\ t\in\mathcal{T},</math>
which is a FLM with functional responses and scalar covariates.
==== Functional concurrent models ====
Assuming that <math>\mathcal{S} = \mathcal{T}</math>, another model, known as the functional concurrent model, sometimes also referred to as the varying-coefficient model, is of the form
{{NumBlk|::|<math display="block">Y(t) = \alpha_0(t) + \alpha(t)X(t)+\varepsilon(t),\ \text{for}\ t\in\mathcal{T},</math>|{{EquationRef|6}}}}
where <math>\alpha_0</math> and <math>\alpha</math> are coefficient functions. Note that model ({{EquationNote|6}}) assumes the value of <math>Y</math> at time <math>t</math>, i.e., <math>Y(t)</math>, only depends on that of <math>X</math> at the same time, i.e., <math>X(t)</math>. Various estimation methods can be applied to model ({{EquationNote|6}}).<ref>Fan and Zhang (1999). "Statistical estimation in varying coefficient models". ''The Annals of Statistics''. '''27''' (5):1491–1518. [[Digital object identifier|doi]]:[http://doi.org/10.1214/aos/1017939139 10.1214/aos/1017939139].</ref><ref>Huang, Wu and Zhou (2004). "Polynomial spline estimation and inference for varying coefficient models with longitudinal data". ''Biometrika''. '''14''' (3):763–788. http://www.jstor.org/stable/24307415.</ref><ref>Şentürk and Müller (2010). "Functional varying coefficient models for longitudinal data". ''Journal of the American Statistical Association''. '''105''' (491):1256–1264. [[Digital object identifier|doi]]:[http://doi.org/10.1198/jasa.2010.tm09228 10.1198/jasa.2010.tm09228].</ref>
Adding multiple functional covariates, model ({{EquationNote|6}}) can also be extended to
<math display="block">Y(t) = \alpha_0(t) + \sum_{j=1}^p\alpha_j(t)X_j(t)+\varepsilon(t),\ \text{for}\ t\in\mathcal{T},</math>
Line 50 ⟶ 51:
A functional multiple index model is given by
<math display="block">Y = g\left(\int_{\mathcal{T}} X^c(t) \beta_1(t)\,dt, \ldots, \int_{\mathcal{T}} X^c(t) \beta_p(t)\,dt \right) + \varepsilon.</math>
Taking <math>p=1</math> yields a functional single index model. However, for <math>p>1</math>, this model is problematic due to [[curse of dimensionality]]. With <math>p>1</math> and relatively small sample sizes, the estimator given by this model often has large variance.<ref name=chen:11>Chen, Hall and Müller (2011). "Single and multiple index functional regression models with nonparametric link". ''The Annals of Statistics''. '''39''' (3):1720–1747. [[Digital object identifier|doi]]:[http://doi.org/10.1214/11-AOS882 10.1214/11-AOS882].</ref>
<math display="block">Y = g_1\left(\int_{\mathcal{T}} X^c(t) \beta_1(t)\,dt\right)+ \cdots+ g_p\left(\int_{\mathcal{T}} X^c(t) \beta_p(t)\,dt \right) + \varepsilon.</math>
Estimation methods for functional single and multiple index models are available.<ref name=chen:11/><ref>Jiang and Wang (2011). "Functional single index models for longitudinal data". '''39''' (1):362–388. [[Digital object identifier|doi]]:[http://doi.org/10.1214/10-AOS845 10.1214/10-AOS845].</ref>
=== Functional additive models (FAMs) ===
Line 59 ⟶ 60:
One form of FAMs is obtained by replacing the linear function of <math>x_k</math>, i.e., <math>\beta_k x_k</math>, by a general smooth function <math>f_k</math>,
<math display="block">\mathbb{E}(Y|X)=\mathbb{E}(Y) + \sum_{k=1}^\infty f_k(x_k),</math>
where <math>f_k</math> satisfies <math>\mathbb{E}(f_k(x_k))=0</math> for <math>k\in\mathbb{N}</math>.<ref name=wang:16/><ref>Müller and Yao (2008). "Functional additive models". ''Journal of the American Statistical Association''. '''103''' (484):1534–1544. [[Digital object identifier|doi]]:[http://doi.org/10.1198/016214508000000751 10.1198/016214508000000751].</ref>
<math display="block">\mathbb{E}(Y|X(t_1),\ldots,X(t_p))=\sum_{j=1}^p f_j(X(t_j)),</math>
where <math>\{t_1,\ldots,t_p\}</math> is a dense grid on <math>\mathcal{T}</math> with increasing size <math>p\in\mathbb{N}</math>, and <math>f_j(x) = g(t_j,x)</math> with <math>g</math> a smooth function, for <math>j=1,\ldots,p</math><ref name=wang:16/><ref>Fan, James and Radchenko (2015). "Functional additive regression". ''The Annals of Statistics''. '''43''' (5):2296–2325. [[Digital object identifier|doi]]:[http://doi.org/10.1214/15-AOS1346 10.1214/15-AOS1346].</ref>
| |||