Cross-correlation matrix: Difference between revisions

Revision as of 14:27, 5 May 2006

Latest revision as of 21:03, 14 April 2025

(97 intermediate revisions by 63 users not shown)

Line 1:

{{other uses|Correlation function (disambiguation)}}

{{cleanup-date|February 2006}}

{{Multiple issues|

{{disputed|date=December 2018}}

{{More citations needed|date=December 2009}}

}}

{{Correlation and covariance}}

The '''cross-correlation matrix''' of two [[random vector]]s is a matrix containing as elements the cross-correlations of all pairs of elements of the random vectors. The cross-correlation matrix is used in various digital signal processing algorithms.

For [[stochastic process]]es, including those that arise in [[statistical mechanics]] and Euclidean [[quantum field theory]], a '''correlation function''' is the [[correlation]] between [[random variable]]s at two different points in space or time. If one considers the correlation function between random variables at the same point but at two different times then one refers to this as the '''autocorrelation function'''. If there are multiple random variables in the problem then correlation functions of the ''same'' random variable are also sometimes called autocorrelation. Correlation functions of different random variables are sometimes called '''cross correlations'''.

Correlation functions used in [[correlation function (astronomy)|astronomy]], [[financial analysis]], [[quantum field theory]] and [[statistical mechanics]] differ only in the particular stochastic processes they are applied to with the caveat that we are dealing with "quantum distributions" in QFT.

==Definition==

For two [[random vector]]s <math>\mathbf{X} = (X_1,\ldots,X_m)^{\rm T}</math> and <math>\mathbf{Y} = (Y_1,\ldots,Y_n)^{\rm T}</math>, each containing [[random element]]s whose [[expected value]] and [[variance]] exist, the '''cross-correlation matrix''' of <math>\mathbf{X}</math> and <math>\mathbf{Y}</math> is defined by<ref name=Gubner>{{cite book |first=John A. |last=Gubner |year=2006 |title=Probability and Random Processes for Electrical and Computer Engineers |publisher=Cambridge University Press |isbn=978-0-521-86470-1}}</ref>{{rp|p.337}}

{{Equation box 1

For random variables ''X''(''s'') and ''X''(''t'') at different points ''s'' and ''t'' of some space, the correlation function is

|indent =

|title=

|equation = <math>\operatorname{R}_{\mathbf{X}\mathbf{Y}} \triangleq\ \operatorname{E}[\mathbf{X} \mathbf{Y}^{\rm T}]</math>

|cellpadding= 6

|border

|border colour = #0073CF

|background colour=#F5FFFA}}

and has dimensions <math>m \times n</math>. Written component-wise:

:<math>C(s,t) = \operatorname{corr}( X(s), X(t) ).</math>

:<math>\operatorname{R}_{\mathbf{X}\mathbf{Y}} =

In this definition, it has been assumed that the stochastic variable is scalar-valued. If it is not, then one can define more complicated correlation functions. For example, if one has a vector ''X''_''i''(''s''), then one can define the matrix of correlation functions

\begin{bmatrix}

\operatorname{E}[X_1 Y_1] & \operatorname{E}[X_1 Y_2] & \cdots & \operatorname{E}[X_1 Y_n] \\ \\

\operatorname{E}[X_2 Y_1] & \operatorname{E}[X_2 Y_2] & \cdots & \operatorname{E}[X_2 Y_n] \\ \\

\vdots & \vdots & \ddots & \vdots \\ \\

\operatorname{E}[X_m Y_1] & \operatorname{E}[X_m Y_2] & \cdots & \operatorname{E}[X_m Y_n] \\ \\

\end{bmatrix}

</math>

The random vectors <math>\mathbf{X}</math> and <math>\mathbf{Y}</math> need not have the same dimension, and either might be a scalar value.

:<math>C_{ij}(s,s') = \operatorname{corr}( X_i(s), X_j(s') )</math>

==Example==

or a scalar, which is the trace of this matrix. If the [[probability distribution]] has any target space symmetries, i.e. symmetries in the space of the stochastic variable (also called '''internal symmetries'''), then the correlation matrix will have induced symmetries. If there are symmetries of the space (or time) in which the random variables exist (also called '''spacetime symmetries''') then the correlation matrix will have special properties. Examples of important spacetime symmetries are &mdash;

For example, if <math>\mathbf{X} = \left( X_1,X_2,X_3 \right)^{\rm T}</math> and <math>\mathbf{Y} = \left( Y_1,Y_2 \right)^{\rm T}</math> are random vectors, then

*'''translational symmetry''' yields ''C''(''s'',''s''<nowiki>'</nowiki>) = ''C''(''s''&nbsp;&minus;&nbsp;''s''<nowiki>'</nowiki>) where ''s'' and ''s''<nowiki>'</nowiki> are to be interpreted as vectors giving coordinates of the points

<math>\operatorname{R}_{\mathbf{X}\mathbf{Y}}</math> is a <math>3 \times 2</math> matrix whose <math>(i,j)</math>-th entry is <math>\operatorname{E}[X_i Y_j]</math>.

*'''rotational symmetry''' in addition to the above gives ''C''(''s'', ''s''<nowiki>'</nowiki>) = ''C''(|''s''&nbsp;&minus;&nbsp;''s''<nowiki>'</nowiki>|) where |''x''| denotes the norm of the vector ''x'' (for actual rotations this is the Euclidean or 2-norm).

''n'' is

==Complex random vectors==

:<math>C_{i_1i_2\cdots i_n}(s_1,s_2,\cdots,s_n) = \langle X_{i_1}(s_1) X_{i_2}(s_2) \cdots X_{i_n}(s_n)\rangle.</math>

If <math>\mathbf{Z} = (Z_1,\ldots,Z_m)^{\rm T}</math> and <math>\mathbf{W} = (W_1,\ldots,W_n)^{\rm T}</math> are [[complex random vector]]s, each containing random variables whose expected value and variance exist, the cross-correlation matrix of <math>\mathbf{Z}</math> and <math>\mathbf{W}</math> is defined by

:<math>\operatorname{R}_{\mathbf{Z}\mathbf{W}} \triangleq\ \operatorname{E}[\mathbf{Z} \mathbf{W}^{\rm H}]</math>

If the random variable has only one component, then the indices <math>i_j</math> are redundant. If there are symmetries, then the correlation function can be broken up into [[irreducible representation]]s of the symmetries &mdash; both internal and spacetime.

where <math>{}^{\rm H}</math> denotes [[Hermitian transpose|Hermitian transposition]].

The case of correlations of a single random variable can be thought of as a special case of autocorrelation of a stochastic process on a space which contains a single point.

==Uncorrelatedness==

==Properties of probability distributions==

Two random vectors <math>\mathbf{X}=(X_1,\ldots,X_m)^{\rm T} </math> and <math>\mathbf{Y}=(Y_1,\ldots,Y_n)^{\rm T} </math> are called '''uncorrelated''' if

:<math>\operatorname{E}[\mathbf{X} \mathbf{Y}^{\rm T}] = \operatorname{E}[\mathbf{X}]\operatorname{E}[\mathbf{Y}]^{\rm T}.</math>

They are uncorrelated if and only if their cross-covariance matrix <math>\operatorname{K}_{\mathbf{X}\mathbf{Y}}</math> matrix is zero.

With these definitions, the study of correlation functions is equivalent to the study of probability distributions. Probability distributions defined on a finite number of points can always be normalized, but when these are defined over continuous spaces, then extra care is called for. The study of such distributions started with the study of [[random walk]]s and led to the notion of the [[Ito calculus]].

In the case of two [[complex random vector]]s <math>\mathbf{Z}</math> and <math>\mathbf{W}</math> they are called uncorrelated if

The Feynman [[path integral]] in Euclidean space generalizes this to other problems of interest to [[statistical mechanics]]. Any probability distribution which obeys a condition on correlation functions called [[reflection positivity]] lead to a local [[quantum field theory]] after [[Wick rotation]] to [[Minkowski spacetime]]. The operation of [[renormalization]] is a specified set of mappings from the space of probability distributions to itself. A [[quantum field theory]] is called renormalizable if this mapping has a fixed point which gives a quantum field theory.

:<math>\operatorname{E}[\mathbf{Z} \mathbf{W}^{\rm H}] = \operatorname{E}[\mathbf{Z}]\operatorname{E}[\mathbf{W}]^{\rm H}</math>

and

:<math>\operatorname{E}[\mathbf{Z} \mathbf{W}^{\rm T}] = \operatorname{E}[\mathbf{Z}]\operatorname{E}[\mathbf{W}]^{\rm T}.</math>

==Properties==

===Relation to the cross-covariance matrix===

The cross-correlation is related to the ''cross-covariance matrix'' as follows:

:<math>\operatorname{K}_{\mathbf{X}\mathbf{Y}} = \operatorname{E}[(\mathbf{X} - \operatorname{E}[\mathbf{X}])(\mathbf{Y} - \operatorname{E}[\mathbf{Y}])^{\rm T}] = \operatorname{R}_{\mathbf{X}\mathbf{Y}} - \operatorname{E}[\mathbf{X}] \operatorname{E}[\mathbf{Y}]^{\rm T}</math>

: Respectively for complex random vectors:

:<math>\operatorname{K}_{\mathbf{Z}\mathbf{W}} = \operatorname{E}[(\mathbf{Z} - \operatorname{E}[\mathbf{Z}])(\mathbf{W} - \operatorname{E}[\mathbf{W}])^{\rm H}] = \operatorname{R}_{\mathbf{Z}\mathbf{W}} - \operatorname{E}[\mathbf{Z}] \operatorname{E}[\mathbf{W}]^{\rm H}</math>

==See also==

*[[CorrelationAutocorrelation]]

*[[Correlation does not imply causation]]

*[[Spearman's rank correlation coefficient]]

*[[Covariance function]]

*[[Pearson product-moment correlation coefficient]]

*[[Correlation function (astronomy)]]

*[[Correlation function (statistical mechanics)]]

*[[Correlation function (quantum field theory)]]

*[[Rate distortion theory#Rate-Distortion_Functions |Mutual information]]

*[[Rate distortion theory#Rate–distortion functions|Rate distortion theory]]

*[[Radial distribution function]]

==References==

{{reflist}}

==Further reading==

* Hayes, Monson H., ''Statistical Digital Signal Processing and Modeling'', John Wiley & Sons, Inc., 1996. {{ISBN|0-471-59431-8}}.

* Solomon W. Golomb, and [[Guang Gong]]. [http://www.cambridge.org/us/academic/subjects/computer-science/cryptography-cryptology-and-coding/signal-design-good-correlation-wireless-communication-cryptography-and-radar Signal design for good correlation: for wireless communication, cryptography, and radar]. Cambridge University Press, 2005.

* M. Soltanalian. [http://theses.eurasip.org/theses/573/signal-design-for-active-sensing-and/download/ Signal Design for Active Sensing and Communications]. Uppsala Dissertations from the Faculty of Science and Technology (printed by Elanders Sverige AB), 2014.

{{DEFAULTSORT:Correlation Function}}

[[Category:Statistics]]

[[Category:Covariance and correlation]]

{{Statistics-stub}}

[[Category:StatisticsTime series]]

[[Category:Spatial analysis]]

[[Category:Matrices (mathematics)]]

[[Category:Signal processing]]