Analogía de la correlación de Pearson para 3 variables


17

Me interesa saber si una "correlación" de tres variables es algo o no, y si es así, ¿cuál sería?

Coeficiente de correlación de momento del producto de Pearson

E { ( X - μ X ) ( Y - μ Y ) }V a r ( X ) V a r ( Y )

E{(XμX)(YμY)}Var(X)Var(Y)

Ahora la pregunta para 3 variables: es

E { ( X - μ X ) ( Y - μ Y ) ( Z - μ Z ) }V a r ( X ) V a r ( Y ) V a r ( Z )

E{(XμX)(YμY)(ZμZ)}Var(X)Var(Y)Var(Z)

anything?

En R parece algo interpretable:

> a <- rnorm(100); b <- rnorm(100); c <- rnorm(100)
> mean((a-mean(a)) * (b-mean(b)) * (c-mean(c))) / (sd(a) * sd(b) * sd(c))
[1] -0.3476942

Normalmente observamos la correlación entre 2 variables dado el valor de una tercera variable fija. ¿Alguien podría aclarar?


2
1) In your bivariate Pearson formula, if "E" (mean in your code) implies division by n then st. deviations must also be based on n (not n-1). 2) Let all three variables be the same variable. In this case, we expect correlation to be 1 (as in bivariate case), but alas...
ttnphns

For a trivariate normal distribution it's zero, regardless of what the correlations are.
Ray Koopman

1
I really think the title would benefit from being changed to "Analogy of Pearson correlation for 3 variables" or similar - it would make links here rather more informative
Silverfish

1
@Silverfish I agree! I have updated the title, thanks.
PascalVKooten

Respuestas:


11

It is indeed something. To find out, we need to examine what we know about correlation itself.

  1. The correlation matrix of a vector-valued random variable X=(X1,X2,,Xp)X=(X1,X2,,Xp) is the variance-covariance matrix, or simply "variance," of the standardized version of XX. That is, each XiXi is replaced by its recentered, rescaled version.

  2. The covariance of XiXi and XjXj is the expectation of the product of their centered versions. That is, writing Xi=XiE[Xi]Xi=XiE[Xi] and Xj=XjE[Xj]Xj=XjE[Xj], we have

    Cov(Xi,Xj)=E[XiXj].

    Cov(Xi,Xj)=E[XiXj].
  3. The variance of XX, which I will write Var(X)Var(X), is not a single number. It is the array of values Var(X)ij=Cov(Xi,Xj).

    Var(X)ij=Cov(Xi,Xj).
  4. The way to think of the covariance for the intended generalization is to consider it a tensor. That means it's an entire collection of quantities vijvij, indexed by ii and jj ranging from 11 through pp, whose values change in a particularly simple predictable way when XX undergoes a linear transformation. Specifically, let Y=(Y1,Y2,,Yq)Y=(Y1,Y2,,Yq) be another vector-valued random variable defined by

    Yi=pj=1ajiXj.

    Yi=j=1pajiXj.

    The constants ajiaji (ii and jj are indexes--jj is not a power) form a q×pq×p array A=(aji)A=(aji), j=1,,pj=1,,p and i=1,,qi=1,,q. The linearity of expectation implies

    Var(Y)ij=akialjVar(X)kl.

    Var(Y)ij=akialjVar(X)kl.

    In matrix notation,

    Var(Y)=AVar(X)A.

    Var(Y)=AVar(X)A.
  5. All the components of Var(X)Var(X) actually are univariate variances, due to the Polarization Identity

    4Cov(Xi,Xj)=Var(Xi+Xj)Var(XiXj).

    4Cov(Xi,Xj)=Var(Xi+Xj)Var(XiXj).

    This tells us that if you understand variances of univariate random variables, you already understand covariances of bivariate variables: they are "just" linear combinations of variances.


The expression in the question is perfectly analogous: the variables XiXi have been standardized as in (1)(1). We can understand what it represents by considering what it means for any variable, standardized or not. We would replaced each XiXi by its centered version, as in (2)(2), and form quantities having three indexes,

μ3(X)ijk=E[XiXjXk].

μ3(X)ijk=E[XiXjXk].

These are the central (multivariate) moments of degree 33. As in (4)(4), they form a tensor: when Y=AXY=AX, then

μ3(Y)ijk=l,m,naliamjankμ3(X)lmn.

μ3(Y)ijk=l,m,naliamjankμ3(X)lmn.

The indexes in this triple sum range over all combinations of integers from 11 through pp.

The analog of the Polarization Identity is

24μ3(X)ijk=μ3(Xi+Xj+Xk)μ3(XiXj+Xk)μ3(Xi+XjXk)+μ3(XiXjXk).

24μ3(X)ijk=μ3(Xi+Xj+Xk)μ3(XiXj+Xk)μ3(Xi+XjXk)+μ3(XiXjXk).

On the right hand side, μ3μ3 refers to the (univariate) central third moment: the expected value of the cube of the centered variable. When the variables are standardized, this moment is usually called the skewness. Accordingly, we may think of μ3(X)μ3(X) as being the multivariate skewness of XX. It is a tensor of rank three (that is, with three indices) whose values are linear combinations of the skewnesses of various sums and differences of the XiXi. If we were to seek interpretations, then, we would think of these components as measuring in pp dimensions whatever the skewness is measuring in one dimension. In many cases,

  • The first moments measure the location of a distribution;

  • The second moments (the variance-covariance matrix) measure its spread;

  • The standardized second moments (the correlations) indicate how the spread varies in pp-dimensional space; and

  • The standardized third and fourth moments are taken to measure the shape of a distribution relative to its spread.

To elaborate on what a multidimensional "shape" might mean, observed that we can understand PCA as a mechanism to reduce any multivariate distribution to a standard version located at the origin and equal spreads in all directions. After PCA is performed, then, μ3μ3 would provide the simplest indicators of the multidimensional shape of the distribution. These ideas apply equally well to data as to random variables, because data can always be analyzed in terms of their empirical distribution.


Reference

Alan Stuart & J. Keith Ord, Kendall's Advanced Theory of Statistics Fifth Edition, Volume 1: Distribution Theory; Chapter 3, Moments and Cumulants. Oxford University Press (1987).


Appendix: Proof of the Polarization Identity

Let x1,,xnx1,,xn be algebraic variables. There are 2n2n ways to add and subtract all nn of them. When we raise each of these sums-and-differences to the nthnth power, pick a suitable sign for each of those results, and add them up, we will get a multiple of x1x2xnx1x2xn.

More formally, let S={1,1}nS={1,1}n be the set of all nn-tuples of ±1±1, so that any element sSsS is a vector s=(s1,s2,,sn)s=(s1,s2,,sn) whose coefficients are all ±1±1. The claim is

2nn!x1x2xn=sSs1s2sn(s1x1+s2x2++snxn)n.

2nn!x1x2xn=sSs1s2sn(s1x1+s2x2++snxn)n.(1)

Indeed, the Multinomial Theorem states that the coefficient of the monomial xi11xi22xinnxi11xi22xinn (where the ij are nonnegative integers summing to n) in the expansion of any term on the right hand side is

(ni1,i2,,in)si11si22sinn.

In the sum (1), the coefficients involving xi11 appear in pairs where one of each pair involves the case s1=1, with coefficient proportional to s1 times si11, equal to 1, and the other of each pair involves the case s1=1, with coefficient proportional to 1 times (1)i1, equal to (1)i1+1. They cancel in the sum whenever i1+1 is odd. The same argument applies to i2,,in. Consequently, the only monomials that occur with nonzero coefficients must have odd powers of all the xi. The only such monomial is x1x2xn. It appears with coefficient (n1,1,,1)=n! in all 2n terms of the sum. Consequently its coefficient is 2nn!, QED.

We need take only half of each pair associated with x1: that is, we can restrict the right hand side of (1) to the terms with s1=1 and halve the coefficient on the left hand side to 2n1n! . That gives precisely the two versions of the Polarization Identity quoted in this answer for the cases n=2 and n=3: 2212!=4 and 2313!=24.

Of course the Polarization Identity for algebraic variables immediately implies it for random variables: let each xi be a random variable Xi. Take expectations of both sides. The result follows by linearity of expectation.


Well done on explaining so far! Multivariate skewness kind of makes sense. Could you perhaps add an example that would show the importance of this multivariate skewness? Either as an issue in a statistical models, or perhaps more interesting, what real life case would be subject to multivariate skewness :)?
PascalVKooten

3

Hmmm. If we run...

a <- rnorm(100);
b <- rnorm(100);
c <- rnorm(100)
mean((a-mean(a))*(b-mean(b))*(c-mean(c)))/
  (sd(a) * sd(b) * sd(c))

it does seem to center on 0 (I haven't done a real simulation), but as @ttnphns alludes, running this (all variables the same)

a <- rnorm(100)
mean((a-mean(a))*(a-mean(a))*(a-mean(a)))/
  (sd(a) * sd(a) * sd(a))

also seems to center on 0, which certainly makes me wonder what use this could be.


2
The nonsense apparently comes from the fact that sd or variance is a function of squaring, as is covariance. But with 3 variables, cubing occurs in the numerator while denominator remains based on originally squared terms
ttnphns

2
Is that the root of it (pun intended)? Numerator and denominator have the same dimensions and units, which cancel, so that alone doesn't make the measure poorly formed.
Nick Cox

3
@Nick That's right. This is simply one of the multivariate central third moments. It is one component of a rank-three tensor giving the full set of third moments (which is closely related to the order-3 component of the multivariate cumulant generating function). In conjunction with the other components it could be of some use in describing asymmetries (higher-dimensional "skewness") in the distribution. It's not what anyone would call a "correlation," though: almost by definition, a correlation is a second-order property of the standardized variable.
whuber

1

If You need to calculate "correlation" between three or more variables, you could not use Pearson, as in this case it will be different for different order of variables have a look here. If you are interesting in linear dependency, or how well they are fitted by 3D line, you may use PCA, obtain explained variance for first PC, permute your data and find probability, that this value may be to to random reasons. I've discuss something similar here (see Technical details below).

Matlab code

% Simulate our experimental data
x=normrnd(0,1,100,1);
y=2*x.*normrnd(1,0.1,100,1);
z=(-3*x+1.5*y).*normrnd(1,2,100,1);
% perform pca
[loadings, scores,variance]=pca([x,y,z]);
% Observed Explained Variance for first principal component
OEV1=variance(1)/sum(variance)
% perform permutations
permOEV1=[];
for iPermutation=1:1000
    permX=datasample(x,numel(x),'replace',false);
    permY=datasample(y,numel(y),'replace',false);
    permZ=datasample(z,numel(z),'replace',false);
    [loadings, scores,variance]=pca([permX,permY,permZ]);
    permOEV1(end+1)=variance(1)/sum(variance);
end

% Calculate p-value
p_value=sum(permOEV1>=OEV1)/(numel(permOEV1)+1)
Al usar nuestro sitio, usted reconoce que ha leído y comprende nuestra Política de Cookies y Política de Privacidad.
Licensed under cc by-sa 3.0 with attribution required.