Content deleted Content added
m →Software implementations: HTTP to HTTPS for SourceForge |
|||
| (7 intermediate revisions by 5 users not shown) | |||
Line 2:
{{redirect|ARMA model||ARMA (disambiguation)}}
In the [[statistics|statistical]] analysis of [[time series]], '''autoregressive–moving-average''' ('''ARMA''') '''models''' are a way to describe
The general ARMA model was described in the 1951 thesis of [[Peter Whittle (mathematician)|Peter Whittle]], ''Hypothesis testing in time series analysis'', and it was popularized in the 1970 book by [[George E. P. Box]] and [[Gwilym Jenkins]].
Line 8:
ARMA models can be estimated by using the [[Box–Jenkins method]].
==
=== Autoregressive model ===
{{Main|Autoregressive model}}
The notation AR(''p'') refers to the autoregressive model of order ''p''. The AR(''p'') model is written as
Line 14 ⟶ 16:
:<math> X_t = \sum_{i=1}^p \varphi_i X_{t-i}+ \varepsilon_t</math>
where <math>\varphi_1, \ldots, \varphi_p</math> are [[parameter]]s and the random variable <math>\varepsilon_t</math> is [[white noise]], usually [[Independent and identically distributed random variables|independent and identically distributed]] (i.i.d.) [[Normal distribution|normal random variables]].<ref>{{Cite book |last=Box |first=George E. P.
In order for the model to remain [[Stationary process|stationary]], the roots of its [[Autoregressive model#Characteristic polynomial|characteristic polynomial]] must lie outside the unit circle. For example, processes in the AR(1) model with <math>|\varphi_1| \ge 1</math> are not stationary because the root of <math>1 - \varphi_1B = 0</math> lies within the unit circle.<ref>{{Cite book |last1=Box |first1=George E. P.
The [[augmented Dickey–Fuller test]] can assesses the stability of
=== Moving
{{Main|Moving-average model}}
The notation MA(''q'') refers to the moving average model of order ''q'':
Line 26 ⟶ 28:
:<math> X_t = \mu + \varepsilon_t + \sum_{i=1}^q \theta_i \varepsilon_{t-i}\,</math>
where the <math>\theta_1,...,\theta_q</math> are the parameters of the model, <math>\mu</math> is the expectation of <math>X_t</math> (often assumed to equal 0), and <math>\varepsilon_1</math>, ..., <math>\varepsilon_t</math> are i.i.d. white noise error terms that are commonly normal random variables.<ref>{{Cite book |last1=Box |first1=George E. P.
=== ARMA model ===
The notation ARMA(''p'', ''q'') refers to the model with ''p'' autoregressive terms and ''q'' moving-average terms. This model contains the AR(''p'') and MA(''q'') models,<ref>{{Cite book |last=Shumway |first=Robert H.
▲The notation ARMA(''p'', ''q'') refers to the model with ''p'' autoregressive terms and ''q'' moving-average terms. This model contains the AR(''p'') and MA(''q'') models,<ref>{{Cite book |last=Shumway |first=Robert H. |url=https://www.worldcat.org/oclc/42392178 |title=Time series analysis and its applications |date=2000 |publisher=Springer |others=David S. Stoffer |isbn=0-387-98950-1 |___location=New York |pages=98 |language=en |oclc=42392178}}</ref>
:<math> X_t = \varepsilon_t + \sum_{i=1}^p \varphi_i X_{t-i} + \sum_{i=1}^q \theta_i \varepsilon_{t-i}.\,</math>
The general ARMA model was described in the 1951 thesis of [[Peter Whittle (mathematician)|Peter Whittle]], who used mathematical analysis ([[Laurent series]] and [[Fourier analysis]]) and statistical inference.<ref>{{cite book|last=Hannan|first=Edward James|author-link=Edward James Hannan|title=Multiple time series|series=Wiley series in probability and mathematical statistics|year=1970|___location=New York|publisher=John Wiley and Sons}}</ref><ref>{{cite book|title=Hypothesis Testing in Time Series Analysis|author=Whittle, P.|publisher=Almquist and Wicksell|year=1951}}▼
{{cite book|title=Prediction and Regulation|author=Whittle, P.|publisher=English Universities Press|year=1963|isbn=0-8166-1147-5}}▼
:Republished as: {{cite book|title=Prediction and Regulation by Linear Least-Square Methods|author=Whittle, P.|publisher=University of Minnesota Press|year=1983|isbn=0-8166-1148-3}}</ref> ARMA models were popularized by a 1970 book by [[George E. P. Box]] and Jenkins, who expounded an iterative ([[Box–Jenkins]]) method for choosing and estimating them. This method was useful for low-order polynomials (of degree three or less).<ref>{{harvtxt|Hannan|Deistler|1988|loc=p. 227}}: {{cite book|last1=Hannan|first1=E. J.|author-link=Edward James Hannan|last2=Deistler|first2=Manfred|title=Statistical theory of linear systems|series=Wiley series in probability and mathematical statistics|year=1988|___location=New York|publisher=John Wiley and Sons}}</ref>▼
The ARMA model is essentially an [[infinite impulse response]] filter applied to white noise, with some additional interpretation placed on it.▼
▲== Specification in terms of lag operator ==
In some texts, the models is specified using the [[lag operator]] ''L''. In these terms, the AR(''p'') model is given by
Line 75 ⟶ 67:
Moreover, starting summations from <math> i=0 </math> and setting <math> \phi_0 = -1 </math> and <math> \theta_0 = 1 </math>, then we get an even more elegant formulation: <math> -\sum_{i=0}^p \phi_i L^i \; X_t = \sum_{i=0}^q \theta_i L^i \; \varepsilon_t \, .</math>
== Spectrum ==▼
The [[spectral density]] of an ARMA process is<math display="block">S(f) = \frac{\sigma^2}{2\pi} \left\vert \frac{\theta(e^{-if})}{\phi(e^{-if})} \right\vert^2</math>where <math>\sigma^2</math> is the [[variance]] of the white noise, <math>\theta</math> is the characteristic polynomial of the moving average part of the ARMA model, and <math>\phi</math> is the characteristic polynomial of the autoregressive part of the ARMA model.<ref>{{Cite book |last=Rosenblatt |first=Murray
In [[digital signal processing]], the ARMA model is represented as a digital filter with white noise at the input and the ARMA process at the output.▼
== Fitting models ==
Line 92 ⟶ 84:
ARMA outputs are used primarily to forecast (predict), and not to infer causation as in other areas of econometrics and regression methods such as OLS and 2SLS.
=== Software implementations ===
* In [[R (programming language)|R]], standard package <code>stats</code> has function <code>arima</code>, documented in [http://search.r-project.org/R/library/stats/html/arima.html ARIMA Modelling of Time Series]. Package [https://cran.r-project.org/web/packages/astsa/index.html <code>astsa</code>] has an improved script called <code>sarima</code> for fitting ARMA models (seasonal and nonseasonal) and <code>sarima.sim</code> to simulate data from these models. Extension packages contain related and extended functionality: package <code>tseries</code> includes the function <code>arma()</code>, documented in [http://finzi.psych.upenn.edu/R/library/tseries/html/arma.html "Fit ARMA Models to Time Series"]; package[https://cran.r-project.org/web/packages/fracdiff <code>fracdiff</code>] contains <code>fracdiff()</code> for fractionally integrated ARMA processes; and package [https://cran.r-project.org/web/packages/forecast/index.html <code>forecast</code>] includes <code>auto.arima</code> for selecting a parsimonious set of ''p, q''. The CRAN task view on [https://cran.r-project.org/web/views/TimeSeries.html Time Series] contains links to most of these.
* [[Mathematica]] has a complete library of time series functions including ARMA.<ref>[http://www.wolfram.com/products/applications/timeseries/features.html Time series features in Mathematica] {{webarchive |url=https://web.archive.org/web/20111124032002/http://www.wolfram.com/products/applications/timeseries/features.html |date=November 24, 2011 }}</ref>
* [[MATLAB]] includes functions such as [http://www.mathworks.com/help/econ/arma-model.html <code>arma</code>], [http://www.mathworks.com/help/ident/ref/ar.html <code>ar</code>] and [http://www.mathworks.com/help/ident/ref/arx.html <code>arx</code>] to estimate autoregressive, exogenous autoregressive and ARMAX models. See [http://www.mathworks.com/help/ident/ug/estimating-ar-and-arma-models.html System Identification Toolbox] and [http://www.mathworks.com/help/econ/arima.estimate.html Econometrics Toolbox] for details.
* [[Julia_(programming_language) | Julia]] has community-driven packages that implement fitting with an ARMA model such as [https://github.com/joefowler/ARMA.jl <code>arma.jl</code>].
* Python has the <code>statsmodels</code>[
* [[PyFlux]] has a Python-based implementation of ARIMAX models, including Bayesian ARIMAX models.
* [[IMSL Numerical Libraries]] are libraries of numerical analysis functionality including ARMA and ARIMA procedures implemented in standard programming languages like C, Java, C# .NET, and Fortran.
* [[gretl]] can estimate ARMA models, as mentioned [http://constantdream.wordpress.com/2008/03/16/gnu-regression-econometrics-and-time-series-library-gretl/ here]
* [[GNU Octave]] extra package [
* [[Stata]] includes the function <code>arima</code>. for ARMA and [[Autoregressive integrated moving average|ARIMA]] models.
* [[SuanShu]] is a Java library of numerical methods that implements univariate/multivariate ARMA, ARIMA, ARMAX, etc models, documented in [http://www.numericalmethod.com/javadoc/suanshu/ "SuanShu, a Java numerical and statistical library"].
* [[SAS (software)|SAS]] has an econometric package, ETS, that estimates ARIMA models. [https://web.archive.org/web/20110930032431/http://support.sas.com/rnd/app/ets/proc/ets_arima.html See details].
== History and interpretations ==
▲== Spectrum ==
▲The general ARMA model was described in the 1951 thesis of [[Peter Whittle (mathematician)|Peter Whittle]], who used mathematical analysis ([[Laurent series]] and [[Fourier analysis]]) and statistical inference.<ref>{{cite book |last=Hannan |first=Edward James |author-link=Edward James Hannan |title=Multiple time series |publisher=John Wiley and Sons |year=1970 |series=Wiley series in probability and mathematical statistics
▲The [[spectral density]] of an ARMA process is<math display="block">S(f) = \frac{\sigma^2}{2\pi} \left\vert \frac{\theta(e^{-if})}{\phi(e^{-if})} \right\vert^2</math>where <math>\sigma^2</math> is the [[variance]] of the white noise, <math>\theta</math> is the characteristic polynomial of the moving average part of the ARMA model, and <math>\phi</math> is the characteristic polynomial of the autoregressive part of the ARMA model.<ref>{{Cite book |last=Rosenblatt |first=Murray |url=https://www.worldcat.org/oclc/42061096 |title=Gaussian and non-Gaussian linear time series and random fields |date=2000 |publisher=Springer |isbn=0-387-98917-X |___location=New York |pages=10 |language=en |oclc=42061096}}</ref><ref>{{Cite book |last=Wei |first=William W. S. |url=https://www.worldcat.org/oclc/18166355 |title=Time series analysis : univariate and multivariate methods |date=1990 |publisher=Addison-Wesley Pub |isbn=0-201-15911-2 |___location=Redwood City, Calif. |pages=242–243 |language=en |oclc=18166355}}</ref>
▲{{cite book
▲: Republished as: {{cite book |author=Whittle, P. |title=Prediction and Regulation by Linear Least-Square Methods
▲
▲In [[digital signal processing]],
== Applications ==
Line 114:
== Generalizations ==
There are various generalizations of ARMA. '''Nonlinear'''
=== {{anchor|ARMAX}}Autoregressive–moving-average model with exogenous inputs
The notation ARMAX(''p'', ''q'', ''b'') refers to
:<math> X_t = \varepsilon_t + \sum_{i=1}^p \varphi_i X_{t-i} + \sum_{i=1}^q \theta_i \varepsilon_{t-i} + \sum_{i=1}^b \eta_i d_{t-i}.\,</math>
Line 125:
Some nonlinear variants of models with exogenous variables have been defined: see for example [[Nonlinear autoregressive exogenous model]].
Statistical packages implement the ARMAX model through the use of "exogenous" (that is, independent
: <math> X_t - m_t = \varepsilon_t + \sum_{i=1}^p \varphi_i (X_{t-i} - m_{t-i}) + \sum_{i=1}^q \theta_i \varepsilon_{t-i}.\,</math>
where <math>m_t</math> incorporates all exogenous (or independent) variables:
Line 142:
== References ==
{{Reflist}}
== Further reading ==
* {{cite book |last=Mills |first=Terence C. |title=Time Series Techniques for Economists |publisher=Cambridge University Press |year=1990 |isbn=0521343399 |url-access=registration |url=https://archive.org/details/timeseriestechni0000mill }}
* {{cite book |last1=Percival |first1=Donald B. |first2=Andrew T. |last2=Walden |title=Spectral Analysis for Physical Applications |url=https://archive.org/details/spectralanalysis0000perc |publisher=Cambridge University Press |year=1993 |isbn=052135532X }}
* {{citation| first1= C. | last1= Francq | first2= J.-M. | last2= Zakoïan | chapter= Recent results for linear time series models with non independent innovations | pages= 241–265 | title= Statistical Modeling and Analysis for Complex Data Problems | editor1-first= P. |editor1-last= Duchesne | editor2-first=B. | editor2-last= Remillard | publisher= Springer | year= 2005| citeseerx= 10.1.1.721.1754 }}.
* [https://link.springer.com/book/10.1007/978-3-319-52452-8 Shumway, R.H. and Stoffer, D.S. (2017). ''Time Series Analysis and Its Applications with R Examples''. Springer. DOI: 10.1007/978-3-319-52452-8]
| |||