Content deleted Content added
m →External links: remove duplicate link |
→Choice of Fitting Criterion: adding paragraph on local likelihood. |
||
Line 152:
As described above, local regression uses a locally weighted least squares criterion to estimate the regression parameters. This inherits many of the advantages (ease of implementation and interpretation; good properties when errors are normally distributed) and disadvantages (sensitivity to extreme values and outliers; inefficiency when errors have unequal variance or are not normally distributed) usually associated with least squares regression.
These disadvantages can be addressed by replacing the local least-squares estimation by something else. Two such ideas are presented here: Local likelihood estimation, which applies local estimation to the [[Generalized linear model]], and Robust local regression, which localizes methods from [[robust regression]].
====Local Likelihood Estimation====
In local likelihood estimation, developed in Hastie and Tibshirani (1987)<ref name=htloclike />, the observations <math>Y_i</math> are assumed to come from a parametric family of distributions, with a known probability density function (or mass function, for discrete data),
<math display="block">
Y_i \sim f(y,\theta(x_i)),
</math>
where the parameter function <math>\theta(x)</math> is the unkown quantity to be estimated. To estimate <math>\theta(x)</math> at a particular point <math>x</math>, the local likelihood criterion is
<math display="block">
\sum_{i=1}^n w_i(x) \log \left ( f(Y_i,
\beta_0 + \beta_1(x_i-x) + \ldots + \beta_p (x_i-x)^p \right ).
</math>
Estimates of the regression coefficients (in, particular, <math>\hat\beta_0</math>) are obtained by maximizing the local likelihood criterion, and
the local likelihood estimate is
<math display="block">
\hat\theta(x) = \hat\beta_0.
</math>
When <math>f(y,\theta(x))</math> is the normal distribution and <math>\theta(x)</math> is the mean function, the local likelihood method reduces to the standard local least-squares regression. For other likelihood families, there is (usually) no closed-form solution for the local likelihood estimate, and iterative procedures such as [[iteratively reweighted least squares]] must be used to compute the estimate.
'''Example''' (local logistic regression). All response observations are 0 or 1, and the mean function is the "success" probability, <math>\mu(x_i) = \Pr (Y_i=1 | x_i)</math>. Since <math>\mu(x_i)</math> must be between 0 and 1, a local polynomial model should not be used for <math>\mu(x)</math> directly. Insead, the logistic transformation
<math display="block">
\theta(x) = \log \left ( \frac{\mu(x)}{1-\mu(x)} \right )
</math>
can be used; equivalently,
<math display="block">
\begin{align}
1-\mu(x) &= \frac{1}{1+e^{\theta(x)}} ;\\
\mu(x) &= \frac{e^{\theta(x)}}{1+e^{\theta(x)}}
\end{align}
</math>
and the mass function is
<math display="block">
f(Y_i,\theta(x_i)) = \frac{ e^{Y_i \theta(x_i)}}{1+e^{\theta(x_i)}}.
</math>
An asymptotic theory for local likelihood estimation is developed in J. Fan, [[Nancy E. Heckman]] and M.P.Wand (1995);<ref>{{citeQ|Q132508409}}</ref> the book Loader (1999)<ref>{{citeQ|Q59410587}}</ref> discusses many more applications of local likelihood.
====Robust Local Regression====
To address the sensitivity to outliers, techniques from [[robust regression]] can be employed. In local [[M-estimator|M-estimation]], the local least-squares criterion is replaced by a criterion of the form
| |||