Linear model: Difference between revisions

Revision as of 13:33, 7 November 2019

Latest revision as of 23:29, 17 November 2024

(12 intermediate revisions by 8 users not shown)

Line 1:

{{Short description|Type of statistical model}}

{{distinguishDistinguish|linear model of innovation}}

In [[statistics]], the term '''linear model''' isrefers usedto inany differentmodel wayswhich accordingassumes to[[linearity]] in the contextsystem. The most common occurrence is in connection with regression models and the term is often taken as synonymous with [[linear regression]] model. However, the term is also used in [[time series analysis]] with a different meaning. In each case, the designation "linear" is used to identify a subclass of models for which substantial reduction in the complexity of the related [[statistical theory]] is possible.

==Linear regression models==

{{main|Linear regression}}

For the regression case, the [[statistical model]] is as follows. Given a (random) sample <math> (Y_i, X_{i1}, \ldots, X_{ip}), \, i = 1, \ldots, n </math> the relation between the observations ''Y<submath>iY_i</submath>'' and the [[independent variables]] ''X<submath>X_{ij}</submath>'' is formulated as

:<math>Y_i = \beta_0 + \beta_1 \phi_1(X_{i1}) + \cdots + \beta_p \phi_p(X_{ip}) + \varepsilon_i \qquad i = 1, \ldots, n </math>

where <math> \phi_1, \ldots, \phi_p </math> may be [[Nonlinear system|nonlinear]] functions. In the above, the quantities ''ε<submath>i\varepsilon_i</submath>'' are [[random variablesvariable]]s representing errors in the relationship. The "linear" part of the designation relates to the appearance of the [[regression coefficient]]s, ''β<submath>j\beta_j</submath>'' in a linear way in the above relationship. Alternatively, one may say that the predicted values corresponding to the above model, namely

:<math>\hat{Y}_i = \beta_0 + \beta_1 \phi_1(X_{i1}) + \cdots + \beta_p \phi_p(X_{ip}) \qquad (i = 1, \ldots, n), </math>

are linear functions of the ''β<submath>j\beta_j</submath>''.

Given that estimation is undertaken on the basis of a [[least squares]] analysis, estimates of the unknown parameters ''β<submath>j\beta_j</submath>'' are determined by minimising a sum of squares function

:<math>S = \sum_{i = 1}^n \varepsilon_i^2 = \sum_{i = 1}^n \left(Y_i - \beta_0 - \beta_1 \phi_1(X_{i1}) - \cdots - \beta_p \phi_p(X_{ip})\right)^2 .</math>

From this, it can readily be seen that the "linear" aspect of the model means the following:

:*the function to be minimised is a quadratic function of the ''β<submath>j\beta_j</submath>'' for which minimisation is a relatively simple problem;

:*the derivatives of the function are linear functions of the ''β<submath>j\beta_j</submath>'' making it easy to find the minimising values;

:*the minimising values ''β<submath>j\beta_j</submath>'' are linear functions of the observations ''Y<submath>iY_i</submath>'';

:*the minimising values ''β<submath>j\beta_j</submath>'' are linear functions of the random errors ''ε<submath>i\varepsilon_i</submath>'' which makes it relatively easy to determine the statistical properties of the estimated values of ''β<submath>j\beta_j</submath>''.

==Time series models==

An example of a linear time series model is an [[autoregressive moving average model]]. Here the model for values {''X<submath>tX_t</submath>''} in a time series can be written in the form

:<math> X_t = c + \varepsilon_t + \sum_{i=1}^p \phi_i X_{t-i} + \sum_{i=1}^q \theta_i \varepsilon_{t-i}.\,</math>

where again the quantities ''ε<submath>t\varepsilon_i</submath>'' are random variables representing [[Innovation (signal processing)|innovations]] which are new random effects that appear at a certain time but also affect values of ''<math>X''</math> at later times. In this instance the use of the term "linear model" refers to the structure of the above relationship in representing ''X<submath>tX_t</submath>'' as a linear function of past values of the same time series and of current and past values of the innovations.<ref>Priestley, M.B. (1988) ''Non-linear and Non-stationary time series analysis'', Academic Press. {{ISBN|0-12-564911-8}}</ref> This particular aspect of the structure means that it is relatively simple to derive relations for the mean and [[covariance]] properties of the time series. Note that here the "linear" part of the term "linear model" is not referring to the coefficients ''φ<submath>i\phi_i</submath>'' and ''θ<submath>i\theta_i</submath>'', as it would be in the case of a regression model, which looks structurally similar.

==Other uses in statistics==

Line 39 ⟶ 40:

* [[Linear predictor function]]

* [[Linear system]]

* [[Linear regression]]

* [[Statistical model]]

==References==

{{Reflist}}

<references/>

{{Statistics}}

{{Authority control}}

[[Category:Curve fitting]]

[[Category:Regression models]]