Proportionate reduction of error

Proportionate reduction of error (PRE) is the gain in precision of predicting dependent variable ${\displaystyle y}$ from knowing the independent variable ${\displaystyle x}$ (or a collection of multiple variables). It is a goodness of fit measure of statistical models, and forms the mathematical basis for several correlation coefficients.^[1] The summary statistics is particularly useful and popular when used to evaluate models where the dependent variable is binary, taking on values {0,1}.

Example

If both ${\displaystyle x}$  and ${\displaystyle y}$  vectors have cardinal (interval or rational) scale, then without knowing ${\displaystyle x}$ , the best predictor for an unknown ${\displaystyle y}$  would be ${\displaystyle {\bar {y}}}$ , the arithmetic mean of the ${\displaystyle y}$ -data. The total prediction error would be ${\displaystyle E_{1}=\sum _{i=1}^{n}{(y_{i}-{\bar {y}})^{2}}}$  .

If, however, ${\displaystyle x}$  and a function relating ${\displaystyle y}$  to ${\displaystyle x}$  are known, for example a straight line ${\displaystyle {\hat {y}}_{i}=a+bx_{i}}$ , then the prediction error becomes ${\displaystyle E_{2}=\sum _{i=1}^{n}{(y_{i}-{\hat {y}})^{2}}}$ . The coefficient of determination then becomes ${\displaystyle r^{2}={\frac {E_{1}-E_{2}}{E_{1}}}=1-{\frac {E_{2}}{E_{1}}}}$  and is the fraction of variance of ${\displaystyle y}$  that is explained by ${\displaystyle x}$ . Its square root is Pearson's product-moment correlation ${\displaystyle r}$ .

There are several other correlation coefficients that have PRE interpretation and are used for variables of different scales:

predict	from	coefficient	symmetric
nominal, binary	nominal, binary	Guttman's λ[2]	yes
ordinal	nominal	Freeman's θ[3]	yes
cardinal	nominal	η${\displaystyle ^{2}}$ [4]	no
ordinal	binary, ordinal	Wilson's e [5]	yes
cardinal	binary	point biserial correlation	yes

References

1. Freeman, L.C.: Elementary applied statistics, New, York, London, Sidney (John Wiley and Sons) 1965
2. Guttman, L. The quantification of a class of attributes: A theory and method of scale construction. In: The prediction of personal adjustment. Horst, P.; Wallin, P.; Guttman, L. et al. (eds.) New York (Social Science Research Council) 1941, pp. 319–348.
3. Freeman, L.C.: Elementary applied statistics, New, York, London, Sidney (John Wiley and Sons) 1965
4. de:Fehlerreduktionsmaße#.CE.B72^{[circular reference]}, accessed 2017-07-29
5. Freeman, L.C.: Order-based statistics and monotonicity: A family of ordinal measures of association Archived 2020-10-28 at the Wayback Machine. J. Math. Sociol. 1986, vol. 12, no. 1, pp. 49–69.