Content deleted Content added
No edit summary |
No edit summary |
||
Line 5:
== Functional linear models (FLMs) ==
Functional linear models (FLMs) are an extension of [[Linear regression|linear models]] (LMs). A linear model with scalar response <math>Y\in\mathbb{R}</math> and scalar covariates <math>X\in\mathbb{R}^p</math> can be written as
{{NumBlk|::|<math display="block">Y = \beta_0 + \langle X,\beta\rangle + \
where <math>\langle\cdot,\cdot\rangle</math> denotes the [[Inner product space|inner product]] in [[Euclidean space|Euclidean space]], <math>\beta_0\in\mathbb{R}</math> and <math>\beta\in\mathbb{R}^p</math> denote the regression coefficients, and <math>\
=== Functional linear models with scalar responses ===
Functional linear models with scalar responses can be obtained by replacing the scalar covariates <math>X</math> and the coefficient vector <math>\beta</math> in model ({{EquationNote|1}}) by a centered functional covariate <math>X^c(\cdot) = X(\cdot) - \mathbb{E}(X(\cdot))</math> and a coefficient function <math>\beta = \beta(\cdot)</math> with [[Domain of a function|___domain]] <math>\mathcal{T}</math>, respectively, and replacing the inner product in Euclidean space by that in [[Hilbert space]] [[Lp space|<math>L^2</math>]],
{{NumBlk|::|<math display="block">Y = \beta_0 + \langle X^c, \beta\rangle +\
where <math>\langle \cdot, \cdot \rangle</math> here denotes the inner product in <math>L^2</math>. One approach to estimating <math>\beta_0</math> and <math>\beta(\cdot)</math> is to expand the centered covariate <math>X^c(\cdot)</math> and the coefficient function <math>\beta(\cdot)</math> in the same [[Basis function|functional basis]], for example, [[B-spline|B-spline]] basis or the eigenbasis used in the [[Karhunen–Loève theorem|Karhunen–Loève expansion]]. Suppose <math>\{\phi_k\}_{k=1}^\infty</math> is an [[Orthonormal basis|orthonormal basis]] of <math>L^2</math>. Expanding <math>X^c</math> and <math>\beta</math> in this basis, <math>X^c(\cdot) = \sum_{k=1}^\infty x_k \phi_k(\cdot)</math>, <math>\beta(\cdot) = \sum_{k=1}^\infty \beta_k \phi_k(\cdot)</math>, model ({{EquationNote|2}}) becomes
<math display="block">Y = \beta_0 + \sum_{k=1}^\infty \beta_k x_k +\
For implementation, regularization is needed and can be done through truncation, <math>L^2</math> penalization or <math>L^1</math> penalization.<ref name=morr:15>Morris (2015). "Functional regression". ''Annual Review of Statistics and Its Application''. '''2''':321–359. [[Digital object identifier|doi]]:[http://doi.org/10.1146/annurev-statistics-010814-020413 10.1146/annurev-statistics-010814-020413].</ref>. In addition, a [[Reproducing kernel Hilbert space|reproducing kernel Hilbert space]] (RKHS) approach can also be used to estimate <math>\beta_0</math> and <math>\beta(\cdot)</math> in model ({{EquationNote|2}})<ref>Yuan and Cai (2010). "A reproducing kernel Hilbert space approach to functional linear regression". ''The Annals of Statistics''. '''38''' (6):3412–3444. [[Digital object identifier|doi]]:[http://doi.org/10.1214/09-AOS772 10.1214/09-AOS772].</ref>
<br />
Adding multiple functional and scalar covariates, model ({{EquationNote|2}}) can be extended to
{{NumBlk|::|<math display="block">Y = \sum_{k=1}^q Z_k\alpha_k + \sum_{j=1}^p \int_{\mathcal{T}_j} X_j^c(t) \beta_j(t) \,dt + \
where <math>Z_1,\
=== Functional linear models with functional responses ===
For a functional response <math>Y(\cdot)</math> with ___domain <math>\mathcal{T}</math> and a functional covariate <math>X(\cdot)</math> with ___domain <math>\mathcal{S}</math>, two FLMs regressing <math>Y(\cdot)</math> on <math>X(\cdot)</math> have been considered<ref name=wang:16/><ref>Ramsay and [[Bernard Silverman|Silverman]] (2005). ''Functional data analysis'', 2nd ed., New York : Springer, [[Special:BookSources/038740080X|ISBN 0-387-40080-X]].</ref>. One of these two models is of the form
{{NumBlk|::|<math display="block">Y(t) = \beta_0(t) + \int_{\mathcal{S}} \beta(s,t) X^c(s)\,ds + \
where <math>X^c(\cdot) = X(\cdot) - \mathbb{E}(X(\cdot))</math> is still the centered functional covariate, <math>\beta_0(\cdot)</math> and <math>\beta(\cdot,\cdot)</math> are coefficient functions, and <math>\
<math display="block">r_{XY} = R_{XX}\beta, \text{ for } \beta\in L^2(\mathcal{S}\times\mathcal{S}),</math>
where <math>r_{XY}(s,t) = \text{cov}(X(s),Y(t))</math>, <math>R_{XX}: L^2(\mathcal{S}\times\mathcal{S}) \rightarrow L^2(\mathcal{S}\times\mathcal{T})</math> is defined as <math>(R_{XX}\beta)(s,t) = \int_\mathcal{S} r_{XX}(s,w)\beta(w,t)dw</math> with <math>r_{XX}(s,w) = \text{cov}(X(s),X(w))</math> for <math>s,w\in\mathcal{S}</math><ref name=wang:16/>. Regularization is needed and can be done through truncation, <math>L^2</math> penalization or <math>L^1</math> penalization<ref name=morr:15/>. Various estimation methods for model ({{EquationNote|4}}) are available<ref>Ramsay and Dalzell (1991). "Some tools for functional data analysis". ''Journal of the Royal Statistical Society. Series B (Methodological)''. '''53''' (3):539–572. http://www.jstor.org/stable/2345586.</ref><ref>Yao, Müller and Wang (2005). "Functional linear regression analysis for longitudinal data". ''The Annals of Statistics''. '''33''' (6):2873–2903. [[Digital object identifier|doi]]:[http://doi.org/10.1214/009053605000000660 10.1214/009053605000000660].</ref>.<br />
When <math>X</math> and <math>Y</math> are concurrently observed, i.e., <math>\mathcal{S}=\mathcal{T}</math><ref>Grenander (1950). "Stochastic processes and statistical inference". ''Arkiv Matematik''. '''1''' (3):195–277. [[Digital object identifier|doi]]:[http://doi.org/10.1007/BF02590638 10.1007/BF02590638].</ref>, it is reasonable to consider a historical functional linear model, where the current value of <math>Y</math> only depends on the history of <math>X</math>, i.e., <math>\beta(s,t)=0</math> for <math>s>t</math> in model ({{EquationNote|4}})<ref name=wang:16/><ref>Malfait and Ramsay (2003). "The historical functional linear model". ''Canadian Journal of Statistics''. '''31''' (2):115–128. [[Digital object identifier|doi]]:[http://doi.org/10.2307/3316063 10.2307/3316063].</ref>. A simpler version of the historical functional linear model is the functional concurrent model (see below).<br />
Adding multiple functional covariates, model ({{EquationNote|4}}) can be extended to
{{NumBlk|::|<math display="block">Y(t) = \beta_0(t) + \sum_{j=1}^p\int_{\mathcal{S}_j} \beta_j(s,t) X^c_j(s)\,ds + \
where for <math>j=1,\
<math display="block">Y(t) = \sum_{j=1}^p X_j \beta_j(t) + \
which is a FLM with functional responses and scalar covariates.<br />
==== Functional concurrent models ====
Assuming that <math>\mathcal{S} = \mathcal{T}</math>, another model, known as the functional concurrent model, sometimes also referred to as the varying-coefficient model, is of the form
{{NumBlk|::|<math display="block">Y(t) = \alpha_0(t) + \alpha(t)X(t)+\
where <math>\alpha_0</math> and <math>\alpha</math> are coefficient functions. Note that model ({{EquationNote|6}}) assumes the value of <math>Y</math> at time <math>t</math>, i.e., <math>Y(t)</math>, only depends on that of <math>X</math> at the same time, i.e., <math>X(t)</math>. Various estimation methods can be applied to model ({{EquationNote|6}})<ref>Fan and Zhang (1999). "Statistical estimation in varying coefficient models". ''The Annals of Statistics''. '''27''' (5):1491–1518. [[Digital object identifier|doi]]:[http://doi.org/10.1214/aos/1017939139 10.1214/aos/1017939139].</ref><ref>Huang, Wu and Zhou (2004). "Polynomial spline estimation and inference for varying coefficient models with longitudinal data". ''Biometrika''. '''14''' (3):763–788. http://www.jstor.org/stable/24307415.</ref><ref>Şentürk and Müller (2010). "Functional varying coefficient models for longitudinal data". ''Journal of the American Statistical Association''. '''105''' (491):1256–1264. [[Digital object identifier|doi]]:[http://doi.org/10.1198/jasa.2010.tm09228 10.1198/jasa.2010.tm09228].</ref>.<br />
Adding multiple functional covariates, model ({{EquationNote|6}}) can also be extended to
<math display="block">Y(t) = \alpha_0(t) + \sum_{j=1}^p\alpha_j(t)X_j(t)+\
where <math>X_1,\
== Functional nonlinear models ==
=== Functional polynomial models ===
Functional polynomial models are an extension of the FLMs with scalar responses, analogous to extending linear regression to [[Polynomial regression|polynomial regression]]. For a scalar response <math>Y</math> and a functional covariate <math>X(\cdot)</math> with ___domain <math>\mathcal{T}</math>, the simplest example of functional polynomial models is functional quadratic regression<ref name=yao:10>Yao and Müller (2010). "Functional quadratic regression". ''Biometrika''. '''97''' (1):49–64. [[Digital object identifier|doi]]:[http://doi.org/10.1093/biomet/asp069 10.1093/biomet/asp069].</ref>
<math display="block">Y = \alpha + \int_\mathcal{T}\beta(t)X^c(t)\,dt + \int_\mathcal{T} \int_\mathcal{T} \gamma(s,t) X^c(s)X^c(t)
where <math>X^c(\cdot) = X(\cdot) - \mathbb{E}(X(\cdot))</math> is the centered functional covariate, <math>\alpha</math> is a scalar coefficient, <math>\beta(\cdot)</math> and <math>\gamma(\cdot,\cdot)</math> are coefficient functions with domains <math>\mathcal{T}</math> and <math>\mathcal{T}\times\mathcal{T}</math>, respectively, and <math>\
=== Functional single and multiple index models ===
A functional multiple index model is given by
<math display="block">Y = g\left(\int_{\mathcal{T}} X^c(t) \beta_1(t)\,dt, \
Taking <math>p=1</math> yields a functional single index model. However, for <math>p>1</math>, this model is problematic due to [[Curse of dimensionality|curse of dimensionality]]. With <math>p>1</math> and relatively small sample sizes, the estimator given by this model often has large variance.<ref name=chen:11>Chen, Hall and Müller (2011). "Single and multiple index functional regression models with nonparametric link". ''The Annals of Statistics''. '''39''' (3):1720–1747. [[Digital object identifier|doi]]:[http://doi.org/10.1214/11-AOS882 10.1214/11-AOS882].</ref>. An alternative <math>p</math>-component functional multiple index model can be expressed as
<math display="block">Y = g_1\left(\int_{\mathcal{T}} X^c(t) \beta_1(t)\,dt\right)+ \cdots+ g_p\left(\int_{\mathcal{T}} X^c(t) \beta_p(t)\,dt \right) + \
Estimation methods for functional single and multiple index models are available<ref name=chen:11/><ref>Jiang and Wang (2011). "Functional single index models for longitudinal data". '''39''' (1):362–388. [[Digital object identifier|doi]]:[http://doi.org/10.1214/10-AOS845 10.1214/10-AOS845].</ref>.
Line 60:
<math display="block">\mathbb{E}(Y|X)=\mathbb{E}(Y) + \sum_{k=1}^\infty f_k(x_k),</math>
where <math>f_k</math> satisfies <math>\mathbb{E}(f_k(x_k))=0</math> for <math>k\in\mathbb{N}</math><ref name=wang:16/><ref>Müller and Yao (2008). "Functional additive models". ''Journal of the American Statistical Association''. '''103''' (484):1534–1544. [[Digital object identifier|doi]]:[http://doi.org/10.1198/016214508000000751 10.1198/016214508000000751].</ref>. Another form of FAMs consists of a sequence of time-additive models:
<math display="block">\mathbb{E}(Y|X(t_1),\
where <math>\{t_1,\
== Extensions ==
A direct extension of FLMs with scalar responses shown in model ({{EquationNote|2}}) is to add a link function to create a [[Generalized functional linear model|generalized functional linear model]] (GFLM) by analogy to extending [[Linear regression|linear regression]] to [[Generalized linear model|generalized linear regression]] (GLM), of which the three components are:
# Linear predictor <math>\eta = \beta_0 + \int_{\mathcal{T}} X^c(t)\beta(t)\,dt</math>;
# [[Variance function]] <math>\text{Var}(Y|X) = V(\mu)</math>, where <math>\mu = \mathbb{E}(Y|X)</math> is the [[Conditional expectation|conditional mean]];
# Link function <math>g</math> connecting the conditional mean and the linear predictor through <math>\mu=g(\eta)</math>.
| |||