Nonparametric regression: Difference between revisions

Revision as of 04:16, 6 August 2020

Latest revision as of 18:59, 1 August 2025

(15 intermediate revisions by 13 users not shown)

Line 1:

{{Short description|Category of regression analysis}}

{{Regression bar}}

'''Nonparametric regression''' is a categoryform of [[regression analysis]] in whichwhere the predictor does not take a predetermined form but is completely constructed according tousing information derived from the data. NonparametricThat regressionis, requiresno [[parametric equation]] is assumed for the relationship between [[Dependent_and_independent_variables|predictors]] and dependent variable. A larger [[Sampling_(statistics)|sample]] sizessize thanis regressionneeded basedto onbuild a nonparametric model having the same level of [[Prediction_interval|uncertainty]] as a [[parametric model]]s because the data must supply both the model structure as well asand the modelparameter estimates.

== Definition ==

==Gaussian process regression or Kriging==

Nonparametric regression assumes the following relationship, given the random variables <math>X</math> and <math>Y</math>:

:<math>

\mathbb{E}[Y\mid X=x] = m(x),

</math>

where <math>m(x)</math> is some deterministic function. [[Linear regression]] is a restricted case of nonparametric regression where <math>m(x)</math> is assumed to be a linear function of the data.

Sometimes a slightly stronger assumption of additive noise is used:

:<math>

Y = m(X) + U,

</math>

where the random variable <math>U</math> is the `noise term', with mean 0.

Without the assumption that <math>m</math> belongs to a specific parametric family of functions it is impossible to get an unbiased estimate for <math>m</math>, however most estimators are [[Consistency_(statistics)|consistent]] under suitable conditions.

== Common nonparametric regression algorithms ==

This is a non-exhaustive list of non-parametric models for regression.

* [[nearest neighbor smoothing]] (see also [[k-nearest neighbors algorithm]])

* [[regression tree|regression trees]]

* [[kernel regression]]

* [[local regression]]

* [[multivariate adaptive regression splines]]

* [[smoothing spline|smoothing splines]]

* [[Artificial neural network|neural networks]]<ref>{{Cite journal |last=Cherkassky |first=Vladimir |last2=Mulier |first2=Filip |date=1994 |editor-last=Cheeseman |editor-first=P. |editor2-last=Oldford |editor2-first=R. W. |title=Statistical and neural network techniques for nonparametric regression |url=https://link.springer.com/chapter/10.1007/978-1-4612-2660-4_39 |journal=Selecting Models from Data |series=Lecture Notes in Statistics |language=en |___location=New York, NY |publisher=Springer |pages=383–392 |doi=10.1007/978-1-4612-2660-4_39 |isbn=978-1-4612-2660-4|url-access=subscription }}</ref>

== Examples ==

=== Gaussian process regression or Kriging ===

{{Main|Gaussian process regression}}

In Gaussian process regression, also known as Kriging, a Gaussian prior is assumed for the regression curve. The errors are assumed to have a [[multivariate normal distribution]] and the regression curve is estimated by its [[posterior mode]]. The Gaussian prior may depend on unknown hyperparameters, which are usually estimated via [[empirical Bayes]].

Line 9 ⟶ 35:

[[Smoothing splines]] have an interpretation as the posterior mode of a Gaussian process regression.

=== Kernel regression ===

{{Main|Kernel regression}}

[[File:NonparRegrGaussianKernel.png|thumb| Example of a curve (red line) fit to a small data set (black points) with nonparametric regression using a Gaussian kernel smoother. The pink shaded area illustrates the kernel function applied to obtain an estimate of y for a given value of x. The kernel function defines the weight given to each data point in producing the estimate for a target point.]]{{Unreferenced section|date=August 2020}}

Kernel regression estimates the continuous dependent variable from a limited set of data points by [[Convolution|convolving]] the data points' locations with a [[kernel function]]—approximately speaking, the kernel function specifies how to "blur" the influence of the data points so that their values can be used to predict the value for nearby locations.

=== Regression trees ===

{{Main|Decision tree learning}}

Line 56 ⟶ 82:

* {{cite book |last=Henderson |first=D. J. |first2=C. F. |last2=Parmeter |title=Applied Nonparametric Econometrics |___location=New York |publisher=Cambridge University Press |year=2015 |isbn=978-1-107-01025-3 |url=https://books.google.com/books?id=hD3WBQAAQBAJ }}

* {{cite book |last=Li |first=Q. |first2=J. |last2=Racine |year=2007 |title=Nonparametric Econometrics: Theory and Practice |___location=Princeton |publisher=Princeton University Press |isbn=978-0-691-12161-1 |url=https://books.google.com/books?id=BI_PiWazY0YC }}

* {{cite book |last=Pagan |first=A. |authorlinkauthor-link=Adrian Pagan |first2=A. |last2=Ullah |year=1999 |title=Nonparametric Econometrics |___location=New York |publisher=Cambridge University Press |isbn=0-521-35564-8 |url=https://archive.org/details/nonparametriceco00paga |url-access=registration }}

==External links==