Department of Statistics & OR
College of Science - King Saud University
STAT- 627: Generalized Linear Models
Final Examination - First Semester 1427/1428
-------------------------------------------------------------------------------------------------------Time allowed is 3 hours.
SECTION (A): Attempt one question.
Question A-1:
(i) Suppose that f() is a function of p variables, where =(1, …, p)t. Show that the
value of  optimizing f() can be numerically obtained by the iterative formula:
 1



θ m  θ m1  H (θ m1 ) f (θ m1 ) ; m  1, 2, , where H (θ m 1 ) and f (θ m 1 ) are the Hessian
matrix and the gradient vector of f(), respectively, evaluated at θ m 1 , and θ 0 is an
initial value of .
(ii) Suppose that U 1 and U 2 are two independent random variables such that
U 1 ~  2 (n1 , 1 ) and U 2 ~  2 (n2 , 2 ) . Show that U1  U 2 ~  2 (n1  n2 , 1  2 ) .
Question A-2:
Consider the general linear model Y=X+ where ~Nn(0,2In) ),  is a vector of p
unknown parameters, and X is an np matrix of full rank; i.e., r(X)=p (p<n). Suppose
that βˆ  X  Y is the least squares estimate of , X  is the generalized inverse of X,
ˆ is the vector of residuals.
and r  Y  Y
1. Show that Cov(r ) , the variance-covariance matrix of r , is non-negative
definite.
2. Show that Cov(r ) is not positive definite.
3. Let

n
i 1 i
r be the sum of the residuals. Show that

n
i 1 i
r  0 if and only if
j  C (X) , where j  (1,1, ,1)' and C (X) is the column space of X (i.e.,
C (X) = the vector space spanned (or generated) by the columns of X ).
SECTION (B): Attempt 3 questions.
Question B-1:
Suppose that Y1, Y2, …, YN are independent random variables satisfying the properties
of a generalized linear model with:
1. Yi ~ f ( yi ; i )  e y i b( i )  c( i )  d ( y i )
(i=1, 2, …, N).
2. E(Yi) = i , Var(Yi)=  i2 (i=1, 2, …, N).
3. g(i)= xit = i (i=1, 2, …, N); g is monotone and differentiable.
 l (β)
Let U j 
be the score statistic corresponding to  j (j=1, …, p), where l() is
j
the log likelihood function.
N
(Y   )

(i)
Show that Uj =  i 2 i xij i .
 i
i
i 1
N x x
 i 2
ij
ik
(
) .
(ii)
Show that  jk  Cov(U j ,U k )  
 i
 i2
i 1
Question B-2:
Let l() be the log likelihood function where =(1,…,p)t is the vector of parameters
-1-
STAT- 627
Final Examination - First Semester 1427/1428
------------------------------------------------------------------------------------------------------- l (β)
of interest. Let U(β) 
be the vector of score statistics. Suppose that U(b) is
β
the vector of scores evaluated at β  b , and (b)  Cov(U(β)) evaluated at β  b .
1
(i)
Show that l (β)  l (b)  (β  b) t U(b)  (β  b) t (b) (β  b) .
2
(ii)
Show that U(β)  U(b)  (b) (β  b) .
(iii)
Show that W  (β  b) t (b) (β  b) ~  2 ( p) asymptotically.
Hint: You may use that fact that U(b) ~ N p (0, (b)) asymptotically.
Question B-3:
Suppose that Y1, Y2, …, YN are independent random variables satisfying the properties
of a generalized linear model where g(i)= xit (i=1, 2, …, N) and E(Yi)= i. Derive
the asymptotic sampling distribution of the deviance D  2 ln(  ) under H O : the
model of interest is adequate, where   L(b m ) / L(b) , b m is the MLE of β m for the
saturated model, b is the MLE of β for the model of interest, and L is the likelihood
function. Suppose that m is the maximum number of parameters that can be
estimated by the saturated model and p is the number of parameters of the model of
interest ( p  m) .
Hint: You may use the fact that (β  b) t (b) (β  b) ~  2 ( p) asymptotically.
Question B-4:
Suppose that Y1 , Y2 , , YN are independent random variables such that
Yi~Binomial(ni,i), i=1, 2, …, N. Consider modeling the probabilities i by the
logistic model: logit ( i )  1   2 xi ; i  1, 2,, N
where x i is the i-th covariate pattern of the explanatory variables and  is the vector
of parameters. Suppose that ̂ i is the MLE of  i based on this model and ŷi is the
fitted value of yi .
(i)
Show that the deviance of this model can be given by:
N
y
n  yi
D  2[ yi ln( i )  (ni  yi ) ln( i
)]
yˆ i
ni  yˆ i
i 1
(ii)
Show that the deviance, D , is asymptotically equivalent to
N
( y  ni ˆ i ) 2
.
Xˆ 2   i
i 1 ni ˆ i (1  ˆ i )
(iii)
Suppose that the MLEs of 1 and  2 based on the data given in the table
below are b1  2.9 and b2  0.19 .
i ni y i x i
1 10 2 5
2 10 3 15
3 10 8 20
4 10 9 25
(a) Calculate fitted values ŷi ’s.
(b) Calculate the value of the deviance, D .
(c) Is the model, logit ( i )  1   2 xi , adequate to described the data? Use
=0.1.
-2-
STAT- 627
Final Examination - First Semester 1427/1428
-------------------------------------------------------------------------------------------------------SECTION (C): Attempt 2 questions.
Question C-1:
Suppose that Y1 , Y2 , , Y10 are independent random variables such that
Yi~Binomial(ni,i), i=1, 2, …, 10. Consider modeling the probabilities i by the
following 3 logistic models:
Model (1): logit ( i )   0  1 xi   2 xi2
Model (2): logit ( i )   0  1 xi
Model (3): logit ( i )   0
Suppose that there are only 5 different values (patterns) of the explanatory variable X.
Based on the following information:
Model maximum of log-likelihood function
Saturated
30.55
1
31.95
2
32.05
3
104.55
(i)
Is Model (1) adequate to describe the data? Use =0.05.
(ii)
Test H O :  2  0 against H1 :  2  0 . Use =0.05.
Test H O :  1   2  0 against H1 : 1  0 or  2  0 . Use =0.05.
Does the explanatory variable, X, has an effect on the proportions
(response)? Use =0.05.
(v)
Is Model (2) adequate to describe the data? Use =0.05.
(vi)
Which model do you prefer among the three models? Justify your answer
using =0.05.
Question C-2:
Suppose that Y1 , Y2 , , Y10 are independent random variables such that
Yi~Binomial(ni,i), i=1, 2, …, 10. Consider modeling the probabilities i by the
following 4 logistic models:
Model (1): logit ( i )   0  1 xi   2 zi   3 xi zi
Model (2): logit ( i )   0  1 xi   2 zi
Model (3): logit ( i )   0  1 xi
Model (4): logit ( i )   0
Suppose that all patterns of the explanatory variables are different and both of the
explanatory variables X and Z are continuous. Based on the following information:
Model maximum of log-likelihood function
Saturated
47.023
1
48.723
2
249.078
3
265.035
4
270.311
(i)
Is Model (1) adequate to describe the data? Use =0.05.
(ii)
Test H O : there is no interaction between X and Z against H1 : X and Z
interact. Use =0.05.
(iii)
Does the explanatory variable Z have an effect on the proportions
(iii)
(iv)
-3-
STAT- 627
Final Examination - First Semester 1427/1428
-------------------------------------------------------------------------------------------------------(response)? Use =0.05.
(iv)
Test H O : both explanatory variables have no effect on the response against
H1 : at least one explanatory variable affects the response. Use =0.05.
(v)
Which model do you prefer among the four models? Justify your answer
using =0.05.
Question C-3:
Suppose that we are interested in studying the effect of the gender (j=1 for male, j=2
for female) on the employment opportunity within a year after graduation for the
graduates of the college of science. Another explanatory variable that might affect the
proportions (response) of graduates who employed during the first year of graduation
is the grand point average (X) with 7 different levels ( x1 , x2 , x3 , x4 , x5 , x6 , x7 ). Define
the following:
n jk = no. of graduates with gender j and GPA x k .
y jk = no. of graduates employed during the first year of graduation among the n jk
graduates.
Suppose that y jk ’s are independent and y jk ~ Binomial (n jk ,  jk ) . Consider modeling
the probabilities  jk (j=1, 2; and k=1, 2, …, 7) by the following 4 logistic models:
Model (1): logit ( jk )   j   j xk
Model (2): logit ( jk )   j   xk
Model (3): logit ( jk )     xk
Model (4): logit ( jk )  
Based on the following information:
Model Deviance
1
4.05
2
5.15
3
7.10
4
40.50
(i)
Determine the maximum number of parameters that can be estimated by
the saturated model?
(ii)
Is Model (1) adequate to describe the data? Use =0.05.
(iii)
Test H O : 1   2 against H1 : 1   2 . Use =0.05.
(iv)
Test H O : 1   2 and 1   2 against H1 : 1   2 or 1  2 . Use
=0.05.
(v)
Does the gender affect the response (proportions of graduates employed
during the first year of graduation). Use =0.05.
(vi)
Which model do you prefer among the four models? Justify your answer
using =0.05.
-4-