Multivariate adaptive regression spline: Difference between revisions

Revision as of 19:23, 1 January 2023

Revision as of 17:19, 14 October 2023

Line 211:

*MARS models are simple to understand and interpret.<ref name=":0">{{Cite book|title=Applied Predictive Modeling|last1=Kuhn|first1=Max|last2=Johnson|first2=Kjell|date=2013|publisher=Springer New York|isbn=9781461468486|___location=New York, NY|language=en|doi=10.1007/978-1-4614-6849-3}}</ref> Compare the equation for ozone concentration above to, say, the innards of a trained [[Artificial neural network|neural network]] or a [[random forest]].

*MARS can handle both continuous and [[categorical data]].<ref>{{cite book | last=Friedman | first=Jerome H. | chapter=Estimating Functions of Mixed Ordinal and Categorical Variables Using Adaptive Splines | author-link=Friedman, J. H.|year=1993|title=New Directions in Statistical Data Analysis and Robustness |editor=Stephan Morgenthaler |editor2=Elvezio Ronchetti |editor3=Werner Stahel|publisher=Birkhauser}}</ref><ref name="Friedman 1991">{{cite journal | last=Friedman | first=Jerome H. | title=Estimating Functions of Mixed Ordinal and Categorical Variables Using Adaptive Splines | website=DTIC | date=1991-06-01 | url=https://apps.dtic.mil/sti/citations/ADA590939 | archive-url=https://web.archive.org/web/20220411085148/https://apps.dtic.mil/sti/citations/ADA590939 | url-status=live | archive-date=April 11, 2022 | access-date=2022-04-11}}</ref> MARS tends to be better than recursive partitioning for numeric data because hinges are more appropriate for numeric variables than the piecewise constant segmentation used by recursive partitioning.

*Building MARS models often requires little or no data preparation.<ref name=":0" /> The hinge functions automatically partition the input data, so the effect of outliers is contained. In this respect MARS is similar to [[recursive partitioning]] which also partitions the data into disjoint regions, although using a different method. (Nevertheless, as with most statistical modeling techniques, known outliers should be considered for removal before training a MARS model.{{Citation needed|date=March 2019}})

*MARS (like recursive partitioning) does automatic [[Feature selection|variable selection]] (meaning it includes important variables in the model and excludes unimportant ones). However, there can be some arbitrariness in the selection, especially when there are correlated predictors, and this can affect interpretability.<ref name=":0" />

*MARS models tend to have a good bias-variance trade-off. The models are flexible enough to model non-linearity and variable interactions (thus MARS models have fairly low bias), yet the constrained form of MARS basis functions prevents too much flexibility (thus MARS models have fairly low variance).

*MARS is suitable for handling large datasets, and implementations run very quickly. However, recursive partitioning can be faster than MARS{{Citation needed|date=March 2019}}.

*MARS is suitable for handling fairly large datasets. It is a routine matter to build a MARS model from an input matrix with, say, 100 predictors and 10⁵ observations. Such a model can be built in about a minute on a 1&nbsp;GHz machine, assuming the maximum degree of interaction of MARS terms is limited to one (i.e. additive terms only). A degree two model with the same data on the same 1&nbsp;GHz machine takes longer—about 12 minutes. Be aware that these times are highly data dependent. Recursive partitioning is much faster than MARS.{{Citation needed|date=March 2019}}

*With MARS models, as with any non-parametric regression, parameter confidence intervals and other checks on the model cannot be calculated directly (unlike [[linear regression]] models). [[Cross-validation (statistics)|Cross-validation]] and related techniques must be used for validating the model instead.

*MARS models do not give as good fits as [[Boosting (meta-algorithm)|boosted]] trees, but can be built much more quickly and are more interpretable. (An 'interpretable' model is in a form that makes it clear what the effect of each predictor is.)

*The <code>earth</code>, <code>mda</code>, and <code>polspline</code> implementations do not allow missing values in predictors, but free implementations of regression trees (such as <code>rpart</code> and <code>party</code>) do allow missing values using a technique called surrogate splits.

*MARS models can make predictions very quickly, as they only require evaluating a linear function of the predictors.

*MARS models can make predictions quickly. The prediction function simply has to evaluate the MARS model formula. Compare that to making a prediction with say a [[Support Vector Machine]], where every variable has to be multiplied by the corresponding element of every support vector. That can be a slow process if there are many variables and many support vectors.

*The resulting fitted function is notcontinuous, smoothunlike recursive partitioning, which can give a more realistic model in some situations. (However, the model is not differentiablesmooth alongor hingesdifferentiable).

== Extensions and related concepts ==

* [[Generalized linear model]]s (GLMs) can be incorporated into MARS models by applying a link function after the MARS model is built. Thus, for example, MARS models can incorporate [[logistic regression]] to predict probabilities.

* [[Nonlinear regression|Non-linear regression]] is used when the underlying form of the function is known and regression is used only to estimate the parameters of that function. MARS, on the other hand, estimates the functions themselves, albeit with severe constraints on the nature of the functions. (These constraints are necessary because discovering a model from the data is an [[inverse problem]] that is not [[Well-posed problem|well-posed]] without constraints on the model.)

* [[Recursive partitioning]] (commonly called CART). MARS can be seen as a generalization of recursive partitioning that allows thefor modelcontinuous tomodels, betterwhich handlecan numericalprovide (i.e.a non-categorical)better fit for numerical data.

* [[Generalized additive model]]s. FromUnlike the user's perspectiveMARS, GAMs are similar to MARS but (a) fit smooth [[Local regression|loess]] or polynomial [[Spline (mathematics)|splines]] insteadrather ofthan MARS basishinge functions, and (b)they do not automatically model variable interactions. The fittingsmoother methodfit usedand internallylack byof GAMsregression isterms veryreduces differentvariance fromwhen thatcompared ofto MARS., but For models that do not require automatic discovery ofignoring variable interactions GAMscan oftenworsen competethe favorably with MARSbias.

* [[TSMARS]]. Time Series Mars is the term used when MARS models are applied in a time series context. Typically in this set up the predictors are the lagged time series values resulting in autoregressive spline models. These models and extensions to include moving average spline models are described in "Univariate Time Series Modelling and Forecasting using TSMARS: A study of threshold time series autoregressive, seasonal and moving average models using TSMARS".

* [[Bayesian MARS]] (BMARS) uses the same model form, but builds the model using a Bayesian approach. It may arrive at different optimal MARS models because the model building approach is different. The result of BMARS is typically an ensemble of posterior samples of MARS models, which allows for probabilistic prediction.<ref>{{cite journal |last1=Denison |first1=D. G. T. |last2=Mallick |first2=B. K. |last3=Smith |first3=A. F. M. |title=Bayesian MARS |journal=Statistics and Computing |date=1 December 1998 |volume=8 |issue=4 |pages=337–346 |doi=10.1023/A:1008824606259 |s2cid=12570055 |url=https://link.springer.com/content/pdf/10.1023/A:1008824606259.pdf |language=en |issn=1573-1375}}</ref>