Alternating conditional expectations: Difference between revisions

Revision as of 06:12, 6 August 2018

Revision as of 05:19, 8 August 2018

Line 5:

----

{{multiple issues|

{{one source|date=August 2018}}

{{technical|date=August 2018}}

{{underlinked|date=August 2018}}

}}

ACE'''Alternating algorithmconditional expectations''' ('''ACE''') is an [[algorithm]] to find the optimal transformations between the [[response variable]] and predictor variables in [[regression analysis]].<ref>Breiman, L. and Friedman, J. H. [http://www.dtic.mil/dtic/tr/fulltext/u2/a123908.pdf Estimating optimal transformations for multiple regression and correlation]. J. Am. Stat. Assoc., 80(391):580–598, September 1985b. {{PD-notice}}</ref>

Stat. Assoc., 80(391):580–598, September 1985b. {{PD-notice}}</ref>

==Introduction==

In statistics, nonlinear transformation of variables is commonly used in practice in regression problems. Alternating conditional expectations (ACE) is one of these method to find those transformations that produce the best fitting additive model. Knowledge of such transformations aids in the interpretation and understanding of the relationship between the response and predictors.

In statistics, nonlinear transformation of variables is commonly used in

practice in regression problems. '''Alternating conditional expectations'''(ACE) is one of these method

to find those transformations that produce the best fitting

additive model. Knowledge of such transformations aids

in the interpretation and understanding of the relationship between

the response and predictors.

ACE transform the response variable <math>Y</math> and its predictor variables, <math>X_i</math> to minimize the fraction of variance not explained. The transformation is nonlinear and is obtained from data in an iterative way.

The transformation is nonlinear and is obtained from data in an iterative way.

== Mathematical Descriptiondescription ==

Let <math>Y,X_1,\dots,X_p</math> be random variables. We use <math>X_1,\dots,X_p</math> to predict <math>Y</math>. Suppose <math>\theta(Y),\varphi_1(X_1),\dots,\varphi_p(X_p)</math> are mean-zero functions and with these transformation functions, the fraction of variance of <math>\theta(Y)</math> not explained is

: <math> e^2(\theta,\varphi_1,\dots,\varphi_p)=\frac{E[\theta(Y)-\sum_{i=1}^p \varphi_i(X_i)]^2}{E\theta^2(Y)}</math>

Line 31 ⟶ 29:

# Iterate the above three steps until <math>e^2</math> is within error tolerance.

==Bivariate Casecase==

The optical transformation <math>\theta^*(Y), \varphi^*(X)</math> for <math>p=1</math> satisfies

: <math> \rho^*(X, Y) = \rho^*(\theta^*, \varphi^*) = \max_{\theta, \varphi} \rho[\theta(Y), \varphi(X)]</math>

Line 38 ⟶ 36:

In the bivariate case, ACE algorithm can also be regarded as a method for estimating the maximal correlation between two variables.

== Software Implementationimplementation ==

The ACE algorithm was developed in the context of known distributions. In practice, data distributions are seldom known and the conditional expectation should be estimated from data. [[R language]] has a package <kbd>acepack</kbd> which implements ACE algorithm. The following example shows its usage:

should be estimated from data. [[R language]] has a package <kbd>acepack</kbd> which implements ACE algorithm. The following example shows its usage:

library(acepack)

TWOPI <- 8*atan(1)

Line 54 ⟶ 51:

The ACE algorithm provides a fully automated method for estimating optimal transformations in multiple regression. It also provides a method for estimating maximal correlation between random variables. Since the process of iteration usually terminates in a limited number of runs, the time complexity of the algorithm is <math>O(np)</math> where <math>n</math> is the number of samples. The algorithm is reasonably computer efficient.

A strong advantage of the ACE procedure is the ability to incorporate variables of quite different type in terms of the set of values they can assume. The transformation functions <math>\theta(y), \varphi_i(x_i)</math> assume values on the real line. Their arguments can, however, assume values on any set. For example, ordered real and unordered categorical variables can be incorporated in the same regression equation. Variables of mixed type are admissible.

A strong advantage of the ACE procedure is the ability to

incorporate variables of quite different type in terms of the set

of values they can assume. The transformation functions <math>\theta(y),

\varphi_i(x_i)</math> assume values on the real line. Their

arguments can, however, assume values on any set. For example,

ordered real and unordered categorical variables can be incorporated in the

same regression equation. Variables of mixed type are admissible.

As a tool for data analysis, the ACE procedure provides graphical output to indicate a need for transformations as well as to guide in their choice. If a particular plot suggests a familiar functional form for a transformation, then the data can be pre-transformed using this functional form and the ACE algorithm can be rerun.

As a tool for data analysis, the ACE procedure provides

graphical output to indicate a need for transformations as well

as to guide in their choice. If a particular plot suggests a familiar

functional form for a transformation, then the data can be pre-transformed

using this functional form and the ACE algorithm

can be rerun.

As with any regression procedure, a high degree of association between predictor variables can sometimes cause the individual transformation estimates to be highly variable, even though the complete model is reasonably stable. When this is suspected, running the algorithm on randomly selected subsets of the data, or on bootstrap samples can assist

As with any regression procedure, a high degree of association

between predictor variables can sometimes cause the individual

transformation estimates to be highly variable, even

though the complete model is reasonably stable. When this is

suspected, running the algorithm on randomly selected subsets

of the data, or on bootstrap samples can assist

in assessing the variability.

== References ==

<!-- Inline citations added to your article will automatically display here. See https://en.wikipedia.org/wiki/WP:REFB for instructions on how to add citations. -->

{{reflist}}

* [[File:PD-icon.svg|15px|link=|alt=]] ''This draft contains quotations from [http://www.dtic.mil/dtic/tr/fulltext/u2/a123908.pdf Estimating Optimal Transformations For Multiple Regression And Correlation By Leo Breiman And Jerome Freidman. Technical Report No. 9 July 1982], which is in the public ___domain.''