Additive model: Difference between revisions

Revision as of 12:44, 27 August 2013

Latest revision as of 04:09, 31 December 2024

(15 intermediate revisions by 12 users not shown)

Line 1:

{{Short description|Statistical regression model}}

In [[statistics]], an '''additive model''' ('''AM''') is a [[nonparametric regression]] method. It was suggested by Jerome H. Friedman and Werner Stuetzle (1981),<ref>{{cite journal |last=Friedman |first=J. H. |last2=Stuetzle |first2=W. |year=1981 |title=Projection Pursuit Regression |journal=[[Journal of the American Statistical Association|J. Amer. Statist. Assoc.]] |volume=76 |issue=376 |pages=817–823 |doi=10.1080/01621459.1981.10477729 }}</ref> and is an essential part of the [[Alternating conditional expectation model|ACE]] algorithm. The ''AM'' uses a one dimensional [[Smoothing|smoother]] to build a restricted class of nonparametric regression models. Because of this, it is less affected by the [[curse of dimensionality]] than e.g. a ''p''-dimensional smoother. Furthermore, the ''AM'' is more flexible than a [[linear regression|standard linear model]], while being more interpretable than a general regression surface at the cost of approximation errors. Problems with ''AM'' include [[model selection]], [[overfitting]], and [[multicollinearity]].

{{About|the statistical method|additive color models|Additive color}}

In [[statistics]], an '''additive model''' ('''AM''') is a [[nonparametric regression]] method. It was suggested by [[Jerome H. Friedman]] and Werner Stuetzle (1981),<ref>{{cite journal |last=[[Friedman, |first=J. H.]] and |last2=Stuetzle, |first2=W. |year=(1981). |title="Projection Pursuit Regression", |journal=[[''Journal of the American Statistical Association|J.'' Amer76:817&ndash;823. Statist. Assoc.]] {{doi|volume=76 |issue=376 |pages=817–823 |doi=10.1080/01621459.1981.10477729 }}</ref> and is an essential part of the [[Alternating conditional expectation modelexpectations|ACE]] algorithm. The ''AM'' uses a one -dimensional [[Smoothing|smoother]] to build a restricted class of nonparametric regression models. Because of this, it is less affected by the [[curse of dimensionality]] than e.g. a ''p''-dimensional smoother. Furthermore, the ''AM'' is more flexible than a [[linear regression|standard linear model]], while being more interpretable than a general regression surface at the cost of approximation errors. Problems with ''AM'', like many other machine-learning methods, include [[model selection]], [[overfitting]], and [[multicollinearity]].

==Description==

Given a [[data]] set <math>\{y_i,\, x_{i1}, \ldots, x_{ip}\}_{i=1}^n</math> of ''n'' [[statistical unit]]s, where <math>\{x_{i1}, \ldots, x_{ip}\}_{i=1}^n</math> represent predictors and <math>y_i</math> is the outcome, the ''additive model'' takes the form

: <math>\mathrm{E}[y_i|x_{i1}, \ldots, x_{ip}] = \beta_0+\sum_{j=1}^p f_j(x_{ij}) </math>

or

: <math>Y= \beta_0+\sum_{j=1}^p f_j(X_{j})+\varepsilon </math>

Where <math>\mathrm{E}[ \epsilon ] = 0</math>, <math>\mathrm{Var}(\epsilon) = \sigma^2</math> and <math>\mathrm{E}[ f_j(X_{j}) ] = 0</math>. The functions <math>f_j(x_{ij})</math> are unknown [[smooth function]]s fit from the data. Fitting the ''AM'' (i.e. the functions <math>f_j(x_{ij})</math>) can be done using the [[backfitting algorithm]] proposed by Andreas Buja, [[Trevor Hastie]] and [[Robert Tibshirani]] (1989).<ref>{{cite journal |last=Buja, |first=A., |last2=Hastie, |first2=T., and |last3=Tibshirani, |first3=R. |year=(1989). |title="Linear Smoothers and Additive Models", ''The |journal=[[Annals of Statistics|Ann.'' Stat.]] |volume=17 |issue=(2):453&ndash;555. {{JSTOR|pages=453–555 |jstor=2241560 }}</ref>

==See also==

*[[Generalized additive model]]

*[[Backfitting algorithm]]

*[[Alternating conditional expectation model]]

*[[Projection pursuit regression]]

*[[Generalized additive model for ___location, scale, and shape]] (GAMLSS)

*[[Median polish]]

*[[Projection pursuit]]

==References==

Line 20 ⟶ 22:

==Further reading==

*{{cite journal |last=Breiman, |first=L. |last2=and [[Friedman, |first2=J. H.]] |year=(1985). |title="Estimating Optimal Transformations for Multiple Regression and Correlation", |journal=''[[Journal of the American Statistical Association|J. Amer. Statist. Assoc.]]'' |volume=80:580&ndash;598. {{doi|issue=391 |pages=580–598 |doi=10.1080/01621459.1985.10478157 }}

[[Category:Regression analysis]]

[[Category:Nonparametric regression]]

[[Category:Regression analysismodels]]