Alternating conditional expectations

ACE algorithm is an algorithm to find the optimal transformations between the response variable and predictor variables in regression analysis.^[1]

Introduction

In statistics, Nonlinear transformation of variables is commonly used in practice in regression problems. Alternating conditional expectations is one of these method to find those transformations that produce the best fitting additive model. Knowledge of such transformations aids in the interpretation and understanding of the relationship between the response and predictors.

Mathematical Description

Let ${\displaystyle Y,X_{1},\dots ,X_{p}}$  be random variables. We use ${\displaystyle X_{1},\dots ,X_{p}}$  to predict ${\displaystyle Y}$ . Suppose ${\displaystyle \theta (Y),\varphi _{1}(X_{1}),\dots ,\varphi _{p}(X_{p})}$  are mean-zero functions and with these transformation functions, the fraction of variance of ${\displaystyle \theta (Y)}$  not explained is

${\displaystyle e^{2}(\theta ,\varphi _{1},\dots ,\varphi _{p})={\frac {E[\theta (Y)-\sum _{i=1}^{p}\varphi _{i}(X_{i})]^{2}}{E\theta ^{2}(Y)}}}$ 

Generally, the optimal transformations that minimize the unexplained part are difficult to compute directly. As an alternative, ACE is an iterative method to calculate the optimal transformations. The procedure of ACE has the following steps:

1. Hold ${\displaystyle \phi _{1}(X_{1}),\dots ,\phi _{p}(X_{p})}$  fixed, minimizing ${\displaystyle e^{2}}$ gives ${\displaystyle \theta _{1}(Y)=E[\sum _{i=1}^{p}\varphi _{i}(X_{i})|Y]}$ 
2. Normalize ${\displaystyle \theta _{1}(Y)}$  to unit variance.
3. For each ${\displaystyle k}$ , fix other ${\displaystyle \varphi _{i}(X_{i})}$  and ${\displaystyle \theta (Y)}$ , minimizing ${\displaystyle e^{2}}$  and the solution is:: ${\displaystyle {\tilde {\varphi }}_{k}=E[\theta (Y)-\sum _{i\neq k}\varphi _{i}(X_{i})|X_{k}]}$ 
4. Iterate the above three steps until ${\displaystyle e^{2}}$  is within error tolerance.

References

1. Breiman, L. and Friedman, J. H. Estimating optimal transformations for multiple regression and correlation. J. Am. Stat. Assoc., 80(391):580–598, September 1985b.