Simple linear regression: Difference between revisions

Revision as of 01:56, 2 December 2023

Revision as of 02:09, 2 December 2023

Line 33:

: <math>Q(\alpha, \beta) = \sum_{i=1}^n\widehat\varepsilon_i^{\,2} = \sum_{i=1}^n (y_i -\alpha - \beta x_i)^2\ .</math>

By expanding to get a quadratic expression in <math>\alpha</math> and <math>\beta,</math> we can derive minimizing values of the function arguments, denoted <math>\widehat{\alpha}</math> and <math>\widehat{\beta}</math>):<ref>Kenney, J. F. and Keeping, E. S. (1962) "Linear Regression and Correlation." Ch. 15 in ''Mathematics of Statistics'', Pt. 1, 3rd ed. Princeton, NJ: Van Nostrand, pp. 252–285</ref>

: <math display="inlineblock">\begin{align}

\widehat\alpha & = \bar{y} - ( \widehat\beta\,\bar{x}), \\[5pt]

\widehat\beta &= \frac{ \sum_{i=1}^n (x_i - \bar{x})(y_i - \bar{y}) }{ \sum_{i=1}^n (x_i - \bar{x})^2 } \\[6pt]

&= \frac{ s_\sum_{x, yi=1}^n \Delta x_i \Delta y_i }{ s^2_\sum_{xi=1}^n \Delta x_i^2 } \\[5pt]

&= r_{xy} \frac{s_y}{s_x}. \\[6pt]

\end{align}</math>

Line 45 ⟶ 44:

{{unordered list

|<math>\bar x</math> and <math>\bar y</math> as the average of the {{math|''x''_''i''}} and {{math|''y''_''i''}}, respectively

|<math>\Delta x_i</math> and <math>\Delta y_i</math> as the [[deviation (statistics)|deviations]] in {{math|''x''_''i''}} and {{math|''y''_''i''}} with respect to their respective means.

|{{math|''r''_''xy''}} as the [[Correlation#Sample correlation coefficient|sample correlation coefficient]] between {{mvar|x}} and {{mvar|y}}

|{{math|''s''_''x''}} and {{math|''s_y''}} as the [[standard deviation#Uncorrected sample standard deviation|uncorrected sample standard deviations]] of {{mvar|x}} and {{mvar|y}}

|<math>s^2_x</math> and <math>s_{x, y}</math> as the [[Variance#Sample variance|sample variance]] and [[Sample mean and covariance#Sample covariance|sample covariance]], respectively

}}

===Relationship to sample covariance matrix===

Substituting the above expressions for <math>\widehat{\alpha}</math> and <math>\widehat{\beta}</math> into

The solution can be reformulated using elements of the [[covariance matrix]]:

<math display="block">

\widehat\beta = \frac{ s_{x, y} }{ s^2_{x} } &= r_{xy} \frac{s_y}{s_x}. \\[6pt]

</math>

where

: <math>f = \widehat{\alpha} + \widehat{\beta} x,</math>

{{unordered list

|{{math|''r''_''xy''}} asis the [[Correlation#Sample correlation coefficient|sample correlation coefficient]] between {{mvar|x}} and {{mvar|y}}

|{{math|''s''_''x''}} and {{math|''s_y''}} asare the [[standard deviation#Uncorrected sample standard deviation|uncorrected sample standard deviations]] of {{mvar|x}} and {{mvar|y}}

|<math>s^2_x</math> and <math>s_{x, y}</math> asare the [[Variance#Sample variance|sample variance]] and [[Sample mean and covariance#Sample covariance|sample covariance]], respectively

}}

Substituting the above expressions for <math>\widehat{\alpha}</math> and <math>\widehat{\beta}</math> into the original solution yields

yields

: <math>\frac{ f - \bar{y}}{s_y} = r_{xy} \frac{ x - \bar{x}}{s_x} .</math>