Linear model: Difference between revisions

Revision as of 05:39, 19 October 2020

Revision as of 05:42, 19 October 2020

Line 17:

:<math>S = \sum_{i = 1}^n \left(Y_i - \beta_0 - \beta_1 \phi_1(X_{i1}) - \cdots - \beta_p \phi_p(X_{ip})\right)^2 .</math>

From this, it can readily be seen that the "linear" aspect of the model means the following:

:*the function to be minimised is a quadratic function of the ''β<submath>j\beta_j</submath>'' for which minimisation is a relatively simple problem;

:*the derivatives of the function are linear functions of the ''β<submath>j\beta_j</submath>'' making it easy to find the minimising values;

:*the minimising values ''β<submath>j\beta_j</submath>'' are linear functions of the observations ''Y<submath>iY_i</submath>'';

:*the minimising values ''β<submath>j\beta_j</submath>'' are linear functions of the random errors ''ε<submath>i\epsilon_i</submath>'' which makes it relatively easy to determine the statistical properties of the estimated values of ''β<submath>j\beta_j</submath>''.

==Time series models==

An example of a linear time series model is an [[autoregressive moving average model]]. Here the model for values {''X<submath>tX_t</submath>''} in a time series can be written in the form

:<math> X_t = c + \varepsilon_t + \sum_{i=1}^p \phi_i X_{t-i} + \sum_{i=1}^q \theta_i \varepsilon_{t-i}.\,</math>

where again the quantities ''ε_t'' are random variables representing [[Innovation (signal processing)|innovations]] which are new random effects that appear at a certain time but also affect values of ''<math>X</math>'' at later times. In this instance the use of the term "linear model" refers to the structure of the above relationship in representing ''X_t'' as a linear function of past values of the same time series and of current and past values of the innovations.<ref>Priestley, M.B. (1988) ''Non-linear and Non-stationary time series analysis'', Academic Press. {{ISBN|0-12-564911-8}}</ref> This particular aspect of the structure means that it is relatively simple to derive relations for the mean and [[covariance]] properties of the time series. Note that here the "linear" part of the term "linear model" is not referring to the coefficients ''φ<submath>i\phi_i</submath>'' and ''θ<submath>i\theta_i</submath>'', as it would be in the case of a regression model, which looks structurally similar.

==Other uses in statistics==