Multivariate Data Analysis (STAT 755) Spring 2007
Homework 5 Solution
Written by: Fares Qeadan, Revised by: Anna Panorska
Department of Mathematics and Statistics, University of Nevada, Reno
Problem 4.1.
Let X1 , X2 have the bivariate normal distribution with µ1 = 1, µ2 = 3, σ11 = 2, σ22 = 1, and ρ12 = −0.8.
(a) The bivariate normal density
f (x1 , x2 ) =
=
"
#!
2 2
(x1 − µ1 )(x2 − µ2 )
1
x1 − µ 1
x2 − µ 2
p
+
− 2ρ12
exp −
√
√
√ √
2(1 − ρ212 )
σ11
σ22
σ11 σ22
2π σ11 σ22 (1 − ρ212 )
i
√
1
25 h
√ exp −
(x1 − 1)2 + 2(x2 − 3)2 + 1.6 2(x1 − 1)(x2 − 3) .
36
1.2π 2
1
(b) Note that the bivariate normal density could be written in terms of the squared statistical distance
f (x1 , x2 ) =
0
1
1
−1
p
(x
−
µ)
Σ
(x
−
µ)
exp
−
2
2π σ11 σ22 (1 − ρ212 )
thus,
0
−1
(x − µ) Σ
(x − µ)
i
√
25 h
2
2
= −2 −
(x1 − 1) + 2(x2 − 3) + 1.6 2(x1 − 1)(x2 − 3)
36
i
√
25 h
=
(x1 − 1)2 + 2(x2 − 3)2 + 1.6 2(x1 − 1)(x2 − 3) .
18
Two dimensional Normal Distribution
µ1 = 1, µ2 = 3, σ11 = 2, σ22 = 1, ρ12 = − 0.8
0.15
0.10
z
0.05
0.00
−4
6
4
2
−2
0
0
x1
f (x) =
1
2 π σ11 σ22 (1 − ρ2)
2
4
x2
−2
6
−4

(x1 − µ1)2
1
x1 − µ1 x2 − µ2 (x2 − µ2)2

exp −
,
− 2ρ
+
σ11
σ22
σ22 
2(1 − ρ2)  σ11


1
2
Problem 4.2.
Let X1 , X2 have the bivariate normal distribution with µ1 = 0, µ2 = 2, σ11 = 2, σ22 = 1, and ρ12 = 0.5.
(a) The bivariate normal density
f (x1 , x2 ) =
=
"
#!
2 2
x2 − µ 2
(x1 − µ1 )(x2 − µ2 )
1
x1 − µ 1
p
+
− 2ρ12
exp −
√
√
√ √
2(1 − ρ212 )
σ11
σ22
σ11 σ22
2π σ11 σ22 (1 − ρ212 )
2
1
1 x1
x1 (x2 − 2)
√
√
+ (x2 − 2)2 −
exp −
.
1.5 2
2π 0.75
2
1
(b) Note that the bivariate normal density could be written in terms of the squared statistical distance
0
1
−1
f (x1 , x2 ) = p
exp − (x − µ) Σ (x − µ)
2
2π σ11 σ22 (1 − ρ212 )
1
thus,
0
(x − µ) Σ−1 (x − µ)
1 x21
x1 (x2 − 2)
√
+ (x2 − 2)2 −
= −2 −
1.5 2
2
2
1
x1 (x2 − 2)
x1
√
+ (x2 − 2)2 −
=
.
0.75 2
2
(c) We will construct the constant-density cantour that contains 50% of the probability
0
P (x − µ) Σ−1 (x − µ) ≤ c2 = P χ2(2) ≤ c2 = 0.50.
Using the SPLUS command ”qchisq” we obtain
> c=sqrt(qchisq(0.5,2))
> c
[1] 1.17741
√
√
contour
find the
Note that σ12 = σ21 = ρ12 σ11 σ22 = 0.5 2 and to sketch the constant-density
 we have to


√ 
2
0.5 2
σ11 σ12
=
.
eigenvalues and normalized egenvectors of the covariance matrix Σ = 
√
0.5 2
σ21 σ22
1
Using SPLUS we get
> sigma<-matrix(c(2,0.5*sqrt(2),0.5*sqrt(2),1),2,2,byrow=T)
> sigma
[,1]
[,2]
[1,] 2.0000000 0.7071068
[2,] 0.7071068 1.0000000
> eigen(sigma)
$values
[1] 2.3660254 0.6339746
$vectors
[,1]
[,2]
[1,] 0.8880738 -0.4597008
[2,] 0.4597008 0.8880738
3
The length of
1
2
The length of
1
2
√
√
the major axis is c λ1 = 1.17741 2.3660254 = 1.811079.
√
√
the minor axis is c λ2 = 1.17741 0.6339746 = 0.9374835.
Two dimensional Normal Distribution
µ1 = 0, µ2 = 2, σ11 = 2, σ22 = 1, ρ12 = 0.5
4
3
x2
0.10
2
z
0.05
1
6
4
0.00
−6
2
0
−4
−2
−2
0
x1
2
x2
−4
4
–2
−6
6
–1
1
x1
–1
f (x) =

(x1 − µ1)2
1
x1 − µ1 x2 − µ2 (x2 − µ2)2

exp −
,
− 2ρ
+
σ11
σ22
σ22 
2(1 − ρ2)  σ11


1
2 π σ11 σ22 (1 − ρ2)
Problem 4.3.
0
Let X be N3 (µ, Σ) with µ =
h
i
−3 1 4

1
−2 0


and Σ =  −2

0
5
0



0 .

2
(a) X1 and X2 are not independent since Cov(X1 , X2 ) = σ12 = −2 6= 0.
(b) X2 and X3 
are independent
since Cov(X2 , X3 ) = σ23 = 0.

X1
 and X (2) = X3 . Since Σ12 = 0, then X (1) and X (2) are independent.
(c) Let X (1) = 
e
X2
(d) Let Y1 =
X1 +X2
2
and Y2 = X3 then



and 
Y1
Y2
Y1
Y2


=
1
2
1
2
0
0







 X2  = 


1
X3
0

X1
X1 +X2
2
X3



 ∼ N2 Aµ, AΣAT where

ΣY = AΣA = 
T
1
2
1
2
0
0


1


 −2
1 
0
0
−2 0

1
2


0   12

0
2
5
0




  0.5 0 
0 =

0 2
1
0
Since σY1 Y2 = 0 then Y1 is independent from Y2 .
(e) Let Y1 = X2 and Y2 = X2 − 52 X1 − X3 then









X2


  X2  = 

=
,

−2.5 1 −1 
Y2
X2 − 2.5X1 − X3
X3
Y1

0
1
0
X1
2
4

and 
Y1
Y2

 ∼ N2 Aµ, AΣAT where



1
−2 0


ΣY = AΣAT = 
 −2
−2.5 1 −1 
0
0
1
0

0 −2.5


0  1

2
0
5
0
1
−1




5
10
 
.
=

10 23.25
Since σY1 Y2 = 10 6= 0 then Y1 is not indepemdent from Y2 .
Problem 4.4.

0 
Note that Y = a 

1 1 1


and Σ =  1 3 2

1 2 2



X1
X
 h
i 1 
0



X2  = 3 −2 1  X2  = a X.



X3
X3
0
Let X be N3 (µ, Σ) with µ =


h
i
2 −3 1



 where Y = 3X1 − 2X2 + X3 .

Thus,
0
0
0
E(Y ) = E(a X) = a E(X) = a µ =
ΣY =
h

3 −2 1
h


2

i



3 −2 1
−3  = 13,


1
1 1 1
i

 1 3 2

1 2 2


3




  −2  = 9.


1
Hence Y ∼ N (13, 9).
Problem 4.7.
0
Let X be distributed as N3 (µ, Σ) where µ =
h
1 −1 2
i

4
0 −1


and Σ =  0 5

−1 0



0 .

2
(a) Note that the joint distribution of (X1 , X3 ) is:


(X1 , X3 ) ∼ N2 η = 
µ1
µ3


=
1
2


,Ω = 
w11
w12
w21
w22


=
4
−1
−1
2

 .
Thus, the conditional distribution f (X1 |X3 = x3 ) is:
X1 |X3 = x3 ∼ N
1
1
w12
w2
1
= N − x3 + 2, 3.5 .
µ1 +
(x3 − µ3 ), w11 − 12 = N 1 − (x3 − 2), 4 −
w22
w22
2
2
2
(b) Since σ12 = 0 then the conditional distribution of X1 , given X2 = x2 and X3 = x3 is the same as the
conditional distribution in part-a (i.e. X1 does not depend on X2 ). However, for illustration we will show this
result explicitly:
5

X
 1

 ···
Let X = 

 X2

X3


(1)

  X
 
 =  ···
 


X (2)


µ
 1


 ···

 with µ = 


 µ2

µ3


1
 
 
  ···
=
 
  −1
 
2






Σ
 and Σ =  11


Σ21

Σ12
Σ22
4
 

 ···
=

 0


−1
..
.
···
..
.
..
.
Note that |Σ22 | > 0, thus the conditional distribution of (X1 |X2 = x2 , X3 = x3 ) is normal and has
X (2) − µ(2)
−1 
0
X
  2
2
X3
 
0.2 0
 
0 0.5


h
i X2 + 1

1 + 0 −0.5 
X3 − 2
M ean = µ(1) + Σ12 Σ22−1


0
5

= 1+
−1
0

h
i
= 1 + 0 −1 
=
=


−
X2 + 1

X3 − 2

2

1
2 − X3 .
2
and
Cov
Σ11 − Σ12 Σ−1
22 Σ21

−1 

h
i 5 0
0

 
= 4 − 0 −1 
−1
0 2



h
i 0.2 0
0


= 4 − 0 −1 
0 0.5
−1
=
=
3.5.
Hence
X1 |(X2 = x2 , X3 = x3 ) ∼ N
1
− x3 + 2, 3.5 .
2
Problem 4.16.
Let X1 , X2 , X3 and X4 be independent Np (µ, Σ) random vectors where
V1 =
1
1
1
1
X1 − X2 + X3 − X4
4
4
4
4
V2 =
1
1
1
1
X1 + X2 − X3 − X4 .
4
4
4
4
and
(a) V1 ∼ Np
P
P4
4
2
j=1 cj µj , (
j=1 cj )Σ

−1
where cj=1,2,3,4 = { 41 , − 14 , 14 , − 41 } and
0
···
5
0

−1 

··· 

.
0 


2
6
µ V1 =
4
X
cj µj = (
j=1
Cov(V1 ) = (
4
X
4
X
1 1 1 1
cj )µj = ( − + − )µ = 0,
4 4 4 4
j=1
c2j )Σ = (
j=1
Hence V1 ∼ N (0,
V 2 ∼ Np
1
1
1
1
1
+
+
+ )Σ = Σ.
16 16 16 16
4
1
4 Σ).
P
4
j=1 bj µj , (
P4
2
j=1 bj )Σ
µ V2 =
4
X
where bj=1,2,3,4 = { 14 , 14 , − 41 , − 14 } and
bj µj = (
j=1
Cov(V2 ) = (
4
X
4
X
1 1 1 1
bj )µj = ( + − − )µ = 0,
4 4 4 4
j=1
b2j )Σ = (
j=1
1
1
1
1
1
+
+
+ )Σ = Σ.
16 16 16 16
4
Hence V2 ∼ N (0, 14 Σ).
(b) The joint distribution of the random vectors V1 and V2 is


V1
V2


 ∼ N2p 
0
0


 , Σ∗  .
where
 P
4
( j=1 c2j )Σ
Σ∗ = 
bT cΣ

 
1
bT cΣ
4Σ


=
P4
0
( j=1 b2j )Σ
1
4
0
1
4Σ



1 
i


4 
Note, that bT c = 14 − 14 14 − 41 
 1  = 0.
 −4 


− 41
Hence V1 and V2 are independent and



 
1
V1
0
Σ

 ∼ N2p 
, 4
0
0
V2
h
0
1
4Σ

 .

.