Multivariate Statistical Analysis, Second Edition

Mu1
tivaria
t
e
Statistical
Analysis
Second Edition, Revised
and
Expanded
Narayan
C.
Giri
University
of
Montreal
Montreal, Quebec, Canada
MARCEL
MARCEL DEKKER,
INC.
DEKKER
NEW
YORK
BASEL
Although great care has been taken to provide accurate and current infor mation, neither the
author(s) nor the publisher, nor anyone else associated with this publication, shall be liable
for any loss, damage, or liability directly or indirectly caused or alleged to be caused by this
book. The material contained herein is not intended to provide specific advice or
recommendations for any specific situation.
Trademark notice: Product or cor porate names may be trademarks or registered trademarks
and are used only for identification and explanation without intent to infringe.

Library of Congress Cataloging-in-Publication Data
A catalog record for this book is available from the Library of Congress.
ISBN: 0-8247-4713-5
This book is printed on acid-free paper.
Headquarters
Marcel Dekker, Inc., 270 Madison Avenue, New York, NY 10016, U.S.A.
tel: 212-696-9000; fax: 212-685-4540
Distribution and Customer Service
Marcel Dekker, Inc., Cimarron Road, Monticello, New York 12701, U.S.A.
tel: 800-228-1160; fax: 845-796-1772
Eastern Hemisphere Distribution
Marcel Dekker AG, Hutgasse 4, Postfach 812, CH-4001 Basel, Switzerland
tel: 41-61-260-6300; fax: 41-61-260-6333
World Wide Web

The publisher offers discounts on this book when ordered in bulk quantities. For more
information, write to Special Sales/Professional Marketing at the headquarters address
above.
Copyright # 2004 by Marcel Dekker, Inc. All Rights Reserved.
Neither this book nor any part may be reproduced or transmitted in any form or by any
means, electronic or mechanical, including photocopying, microfilming, and recording, or
by any information storage and retrieval system, without permission in writing from the
publisher.
Current printing (last digit):
10987654321
PRINTED IN THE UNITED STATES OF AMERICA
STATISTICS: Textbooks
and
Monographs
D.

B. Owen
Founding Editor,
1972-1991
Associate Editors
Statistical Computing/ Multivariate Analysis
Professor Anant M. Kshirsagar
University
of
Michigan
Nonparametric Statistics
Professor William
R.
Schucany
Southern Methodist Universig
Probability Quality ControllReliability
Professor Edward G. Schilling
Rochester Institute
of
Technology
Professor Marcel
F.
Neuts
University
of
Arizona
Editorial Board
Applied Probability
Dr. Paul
R.
Garvey

The
MITRE
Corporation
Statistical Distributions
Professor N. Balakrishnan
McMaster University
Economic Statistics Statistical Process Improvement
Professor David
E.
A.
Giles
University
of
Victoria
Professor G. Geoffrey Vining
Virginia Polytechnic Institute
Experimental Designs Stochastic Processes
Mr. Thomas B. Barker
Rochester Institute
of
Technology
Professor
V.
Lakshrmkantham
Florida Institute
of
Technology
Multivariate Analysis Survey Sampling
Professor Subir Ghosh
University

of
Calgornia-Riverside
Professor Lynne Stokes
Southern Methodist University
Time Series
Sastry G. Pantula
North Carolina State University
1. The Generalized Jackknife Statistic,
H.
L.
Gray and
W.
R. Schucany
2. Multivariate Analysis,
Anant M. Kshirsagar
3. Statistics and Society,
Walter T. Federer
4. Multivariate Analysis: A Selected and Abstracted Bibliography, 1957-1 972,
Kocher-
lakota Subrahmaniam and Kathleen Subrahmaniam
5.
Design of Experiments: A Realistic Approach,
Vigil L. Anderson and Robert A.
McLean
6. Statistical and Mathematical Aspects of Pollution Problems,
John
W.
Pratt
7.
Introduction to Probability and Statistics (in

two
parts), Part
I:
Probability; Part
II:
Statistics,
Narayan
C.
Gin'
8. Statistical Theory of the Analysis of Experimental Designs,
J. Ogawa
9.
Statistical Techniques in Simulation (in
two
parts),
Jack
P.
C.
Kleijnen
10. Data Quality Control and Editing,
Joseph
1.
Naus
11. Cost of Living Index Numbers: Practice, Precision, and Theory,
Kali
S.
Banejee
12. Weighing Designs: For Chemistry, Medicine, Economics, Operations Research,
Statistics,
Kali

S.
Banejee
13.
The Search for Oil: Some Statistical Methods and Techniques,
edited by
D.
B.
Owen
14. Sample Size Choice: Charts for Experiments with Linear Models,
Robert
E.
Odeh and
Martin
Fox
15.
Statistical Methods for Engineers and Scientists,
Robert
M.
Bethea, Benjamin
S.
Duran, and Thomas
L.
Boullion
16. Statistical Quality Control Methods,
Irving
W.
Bun
17.
On the History of Statistics and Probability,
edited by

D.
B.
Owen
18. Econometrics,
Peter Schmidt
19. Sufficient Statistics: Selected Contributions,
Vasant
S.
Huzurbazar (edited by Anant M.
Kshirsagar)
20. Handbook of Statistical Distributions,
Jagdish K. Patel,
C.
H.
Kapadia, and
D.
8.
Owen
21. Case Studies in Sample Design,
A.
C.
Rosander
22.
Pocket Book of Statistical Tables,
compiled by R.
E.
Odeh,
D.
B.
Owen,

Z.
W.
Birnbaum, and L. Fisher
23. The Information in Contingency Tables,
D.
V. Gokhale and Solomon Kullback
24. Statistical Analysis of Reliability and Life-Testing
Models:
Theory and Methods,
Lee J.
Bain
25. Elementary Statistical Quality Control,
Irving
W.
Burr
26. An Introduction to Probability and Statistics Using BASIC,
Richard A. Gmeneveld
27. Basic Applied Statistics,
8.
L. Raktoe and J. J. Hubert
28. A Primer in Probability,
Kathleen Subrahmaniarn
29. Random Processes: A First Look,
R.
Syski
30. Regression Methods: A Tool for Data Analysis,
Rudolf J. Freund and Paul
D.
Minton
31. Randomization Tests,

Eugene
S.
Edgington
32. Tables for Normal Tolerance Limits, Sampling Plans and Screening,
Robert
E.
Odeh
and
D.
B.
Owen
33. Statistical Computing,
William J. Kennedy, Jr., and James
E.
Gentle
34. Regression Analysis and Its Application: A Data-Oriented Approach,
Richard
F.
Gunst
and Robert
L.
Mason
35.
Scientific Strategies to Save Your Life,
1.
D.
J.
Bross
36. Statistics in the Pharmaceutical Industry,
edited by C. Ralph Buncher and Jia-Yeong

Tsay
37. Sampling from a Finite Population,
J. Hajek
38. Statistical Modeling Techniques,
S.
S.
Shapiro and A. J.
Gross
39. Statistical Theory and Inference in Research,
T. A. Bancroff and C P. Han
40. Handbook of the Normal Distribution,
Jagdish
K.
Patel and Campbell
B.
Read
41. Recent Advances in Regression Methods,
Hrishikesh D. Vinod and Aman Ullah
42. Acceptance Sampling in Quality Control,
Edward
G.
Schilling
43. The Randomized Clinical Trial and Therapeutic Decisions,
edited by Niels Tygstrup,
John M Lachin, and Erik Juhl
44. Regression Analysis of Survival Data in Cancer Chemotherapy,
Walter
H.
Carter, Jr.,
Galen

L.
Wampler, and Donald
M.
Stablein
45.
A Course in Linear Models,
Anant
M.
Kshirsagar
46.
Clinical Trials: Issues and Approaches,
edited by Stanley H. Shapiro and Thomas
H.
Louis
47. Statistical Analysis
of
DNA Sequence Data,
edited by
B.
S.
Weir
48. Nonlinear Regression Modeling: A Unified Practical Approach,
David A. Ratkowsky
49. Attribute Sampling Plans, Tables of Tests and Confidence Limits for Proportions,
Rob-
ert
€.
Odeh and D.
B.
Owen

50.
Experimental Design, Statistical Models, and Genetic Statistics,
edited by Klaus
Hinkelmann
51.
Statistical Methods for Cancer Studies,
edited by Richard G. Comell
52.
Practical Statistical Sampling for Auditors,
Arthur J.
Wilbum
53.
Statistical Methods for Cancer Studies,
edited by Edward J. Wegman and James G.
Smith
54.
Self-organizing Methods in Modeling: GMDH Type Algorithms,
edited by Stanley J.
Fadow
55.
Applied Factorial and Fractional Designs,
Ro6ert A. McLean and
Virgil
L.
Anderson
56.
Design of Experiments: Ranking and Selection,
edited by Thomas J. Santner and Ajit
C. Tamhane
57. Statistical Methods for Engineers and Scientists: Second Edition, Revised and Ex-

panded,
Robert
M.
Bethea, Benjamin
S.
Duran, and Thomas
L.
Boullion
58. Ensemble Modeling: Inference from Small-Scale Properties to Large-Scale Systems,
Alan
€.
Gelfand and Crayton C. Walker
59. Computer Modeling for Business and Industry,
Bruce
L.
Boweman and Richard
T.
0
'Connell
60. Bayesian Analysis of Linear Models,
Lyle D. Broemeling
61. Methodological Issues for Health Care Surveys,
Brenda Cox and Steven Cohen
62. Applied Regression Analysis and Experimental Design,
Richard J.
Brook
and Gregory
C. Arnold
63. Statpal: A Statistical Package for Microcomputers-PC-DOS Version for the IBM PC
and Compatibles,

Bruce
J.
Chalmer and David G. Whitmore
64.
Statpal:
A Statistical Package for Microcomputers-Apple Version for the
II,
II+,
and
I
le,
David G. Whitmore and Bruce J. Chalmer
65. Nonparametric Statistical Inference: Second Edition, Revised and Expanded,
Jean
Dickinson Gibbons
66. Design and Analysis of Experiments,
Roger G. Petersen
67. Statistical Methods for Pharmaceutical Research Planning,
Sten
W.
Bergman and
John C. Giffins
68. Goodness-of-Fit Techniques,
edited by Ralph
B.
D'Agostino and Michael A. Stephens
69. Statistical Methods in Discrimination Litigation,
edited by D.
H.
Kaye and Mike/ Aickin

70. Truncated and Censored Samples from Normal Populations,
Helmut Schneider
71. Robust Inference, M.
L.
Tiku,
W.
Y.
Tan, andN. Balakrishnan
72. Statistical Image Processing and Graphics,
edited by Edward J. Wegman and Douglas
J.
DePriest
73. Assignment Methods in Combinatorial Data Analysis,
Lawrence
J.
Hubert
74. Econometrics and Structural Change,
Lyle
D.
Broemeling and
Hiroki
Tsurumi
75. Multivariate Interpretation of Clinical Laboratory Data,
Adelin Albert and Eugene
K.
Hanis
76. Statistical
Tools
for Simulation Practitioners,
Jack P. C. Kleijnen

77. Randomization Tests: Second Edition,
Eugene
S.
Edgington
78. A Folio of Distributions: A Collection
of
Theoretical Quantile-Quantile Plots,
Edward
5.
Fowlkes
79. Applied Categorical Data Analysis,
Daniel
H.
Freeman, Jr.
80. Seemingly Unrelated Regression Equations Models: Estimation and Inference,
Viren-
dra
K.
Snvastava and David
E.
A. Giles
81. Response Surfaces: Designs and Analyses,
Andre
1.
Khun' and John A. Cornell
82. Nonlinear Parameter Estimation: An Integrated System in BASIC,
John C.
Nash
and
Mary Walker-Smith

83. Cancer Modeling,
edited by James R. Thompson and
Bany
W.
Brown
84.
Mixture Models: Inference and Applications to Clustering,
Geoffrey J. McLachlan and
Kaye
E.
Basford
85. Randomized Response: Theory and Techniques,
Anjit Chaudhun' and Rahul Mukedee
86. Biopharmaceutical Statistics for Drug Development,
edited by
Karl
f.
Peace
87. Parts per Million Values for Estimating Quality Levels,
Robert
E.
Odeh and
D.
B.
Owen
88. Lognormal Distributions: Theory and Applications,
edited by
Edwin
L. Crow and
Kunio

Shimizu
89. Properties of Estimators for the Gamma Distribution,
K.
0.
Bowman and L. R. Shenton
90. Spline Smoothing and Nonparametric Regression,
Randall L. Eubank
91. Linear Least Squares Computations,
R.
W
Farebrother
92. Exploring Statistics,
Damaraju Raghavarao
93. Applied Time Series Analysis for Business and Economic Forecasting,
Sufi
M. Nazem
94.
Bayesian Analysis of Time Series and Dynamic Models,
edited by James C. Spall
95. The Inverse Gaussian Distribution: Theory, Methodology, and Applications,
Raj S.
Chhikara and J. Leroy Folks
96. Parameter Estimation in Reliability and Life Span Models,
A. Clifford Cohen and Betty
Jones Whitten
97. Pooled Cross-Sectional and Time Series Data Analysis,
Teny
E.
Dielman
98. Random Processes: A First Look, Second Edition, Revised and Expanded,

R. Syski
99. Generalized Poisson Distributions: Properties and Applications,
P. C. Consul
100. Nonlinear L,-Norm Estimation,
Rene Gonin and Arthur
H.
Money
101. Model Discrimination for Nonlinear Regression Models,
Dale
S.
Bomwiak
102. Applied Regression Analysis in Econometrics,
Howard
E.
Doran
103. Continued Fractions in Statistical Applications,
K.
0.
Bowman and
L.
R. Shenton
104. Statistical Methodology in the Pharmaceutical Sciences,
Donald A. Beny
105. Experimental Design in Biotechnology,
Peny
D.
Haaland
106. Statistical Issues in Drug Research and Development,
edited by Karl
15.

Peace
107. Handbook of Nonlinear Regression Models,
David A. Ratkowsky
108.
Robust Regression: Analysis and Applications,
edited by Kenneth
D.
Lawrence and
Jeffrey L. Arthur
109. Statistical Design and Analysis of Industrial Experiments,
edited by Subir
Ghosh
11
0.
U-Statistics: Theory and Practice,
A. J. Lee
111. A Primer in Probability: Second Edition, Revised and Expanded,
Kathleen Subrah-
maniam
112. Data Quality Control: Theory and Pragmatics,
edited by Gunar
E.
Liepins and
V.
R. R.
11 3. Engineering Quality by Design: Interpreting the Taguchi Approach,
Thomas
B.
Barker
114. Survivorship Analysis for Clinical Studies,

Eugene
K.
Hanis
and Adelin Albert
115.
Statistical Analysis of Reliability and Life-Testing Models: Second Edition,
Lee J. Bain
and Max Engelhardt
116.
Stochastic Models of Carcinogenesis,
Wai-Yuan
Tan
117. Statistics and Society: Data Collection and Interpretation, Second Edition, Revised and
Expanded,
Walter T. Federer
11 8. Handbook
of
Sequential Analysis,
B.
K.
Ghosh
and P.
K.
Sen
11
9.
Truncated and Censored Samples: Theory and Applications,
A. Clifford Cohen
120. Survey Sampling Principles,
E.

K.
Foreman
121. Applied Engineering Statistics,
Robert
M.
Bethea and
R.
Russell Rhinehart
122. Sample Size Choice: Charts for Experiments with Linear Models: Second Edition,
Robert
E.
Odeh and Martin
Fox
123. Handbook
of
the Logistic Distribution,
edited by
N.
Balakrishnan
124. Fundamentals
of
Biostatistical Inference,
Chap
T.
Le
125. Correspondence Analysis Handbook,
J P. Benzecri
Uppuluri
126. Quadratic Forms in Random Variables: Theory and Applications,
A.

M. Mathai and
Serge B. Provost
127. Confidence Intervals on Variance Components,
Richard
K.
Burdick and Franklin A.
Graybill
128. Biopharmaceutical Sequential Statistical Applications,
edited by Karl
E.
Peace
129. Item Response Theory: Parameter Estimation Techniques,
Frank B. Baker
130. Survey Sampling: Theory and Methods,
Anjit Chaudhuri and Horst Stenger
131. Nonparametric Statistical Inference: Third Edition, Revised and Expanded,
Jean Dick-
inson Gibbons and Subhabrata Chakraborti
132. Bivariate Discrete Distribution,
Subrahmaniam Kocherlakota and Kathleen Kocher-
lakota
133. Design and Analysis of Bioavailability and Bioequivalence Studies,
Shein-Chung Chow
and Jen-pei
Liu
'
134. Multiple Comparisons, Selection, and Applications in Biometry,
edited by Fred M.
135. Cross-Over Experiments: Design, Analysis, and Application,
David

A.
Ratkowsky,
Marc A. Evans, and J. Richard Alldredge
136. Introduction
to
Probability and Statistics: Second Edition, Revised and Expanded,
Narayan C.
Giri
137. Applied Analysis of Variance in Behavioral Science,
edited by Lynne K. Edwards
138. Drug Safety Assessment in Clinical Trials,
edited by Gene
S.
Gilbert
139. Design of Experiments: A No-Name Approach,
Thomas
J.
Lorenzen and Virgil
L.
An-
derson
140. Statistics in the Pharmaceutical Industry: Second Edition, Revised and Expanded,
edited by
C.
Ralph Buncher and Jia-Yeong Tsay
141. Advanced Linear Models: Theory and Applications,
Song-Gui Wang and Shein-Chung
Chow
142. Multistage Selection and Ranking Procedures: Second-Order Asymptotics,
Nitis Muk-

hopadhyay and Tumulesh K.
S.
Solanky
143. Statistical Design and Analysis in Pharmaceutical Science: Validation, Process Con-
trols, and Stability,
Shein-Chung
Chow
and Jen-pei
Liu
144. Statistical Methods for Engineers and Scientists: Third Edition, Revised and Expanded,
Robert M. Bethea, Benjamin
S.
Duran, and Thomas L. Boullion
145. Growth Curves,
Anant M. Kshirsagar and William Boyce Smith
146. Statistical Bases
of
Reference Values in Laboratory Medicine,
Eugene K. Hanis and
James C. Boyd
147. Randomization Tests: Third Edition, Revised and Expanded,
Eugene
S.
Edgington
148. Practical Sampling Techniques: Second Edition, Revised and Expanded,
Ranjan K.
Som
149. Multivariate Statistical Analysis,
Narayan C.
Giri

150. Handbook of the Normal Distribution: Second Edition, Revised and Expanded,
Jagdish
K.
Patel and Campbell B. Read
151. Bayesian Biostatistics,
edited by Donald A. Berry and Dalene
K.
Stangl
152. Response Surfaces: Designs and Analyses, Second Edition, Revised and Expanded,
Andre
1.
Khuri and John A. Cornell
153. Statistics of Quality,
edited by Subir Ghosh, William R. Schucany, and William B. Smith
154. Linear and Nonlinear Models for the Analysis
of
Repeated Measurements,
Edward
f.
Vonesh and Vernon
M.
Chinchilli
155. Handbook
of
Applied Economic Statistics,
Aman Ullah and David
E.
A.
Giles
156. Improving Efficiency by Shrinkage: The James-Stein and Ridge Regression Estima-

tors,
Marvin
H.
J.
Gruber
157. Nonparametric Regression and Spline Smoothing: Second Edition,
Randall
L.
Eu-
bank
158. Asymptotics, Nonparametrics, and Time Series,
edited by Subir Ghosh
159. Multivariate Analysis, Design
of
Experiments, and Survey Sampling,
edited by Subir
Ghosh
HoPPe
160.
161.
162.
163.
164.
165.
166.
167.
168.
169.
170.
171.

172.
Statistical Process Monitoring and Control,
edited by Sung
H.
Park and G. Geoffrey
Vining
Statistics for the 21st Century: Methodologies for Applications of the Future,
edited
by C. R. Rao and Gabor
J.
Szekely
Probability and Statistical Inference,
Nitis
Mukhopadhyay
Handbook of Stochastic Analysis and Applications,
edited by D. Kannan and
V.
Lak-
shmikantham
Testing
for
Normality,
Henry
C.
Thode,
Jr.
Handbook of Applied Econometrics and Statistical Inference,
edited by Aman Ullah,
Alan
T.

K. Wan, and Anoop Chaturvedi
Visualizing Statistical Models and Concepts,
R. W. Farebrother
Financial and Actuarial Statistics:
An
Introduction,
Dale
S.
Borowiak
Nonparametric Statistical Inference: Fourth Edition, Revised and Expanded,
Jean
Dickinson Gibbons and Subhabrata Chakraborti
Computer-Aided Econometrics,
edited by David
E.
A. Giles
The EM Algorithm and Related Statistical Models,
edited by Michiko Watanabe and
Kazunon Yamaguchi
Multivariate Statistical Analysis: Second Edition, Revised and Expanded,
Narayan C.
Gin
Computational Methods in Statistics and Econometrics,
Hisashi Tanizaki
Additional
Volumes
in Preparation
To Nilima, Nabanita, and Nandan

Preface to the Second Edition

As in the first edition the aim has been to provide an up-to-date presentation of
both the theoretical and applied aspects of multivariate analysis using the
invariance approach for readers with a basic knowledge of mathematics and
statistics at the undergraduate level. This new edition updates the original book
by adding new results, examples, problems, and references. The following new
subsections are added. Section 4.3 deals with the symmetric distributions: its
properties and characterization. Section 4.3.6 treats elliptically symmetric
distributions (multivariate) and Section 4.3.7 considers the singular symmetrical
distribution. Regression and correlations in symmetrical distributions are
discussed in Section 4.5.1. The redundancy index is included in Section 4.7. In
Section 5.3.7 we treat the problem of estimation of covariance matrices and the
equivariant estimation under curved model of mean, and covariance matrix is
treated in Section 5.4. Basic distributions in symmetrical distributions are given
in Section 6.12. Tests of mean against one-sided alternatives are given in Section
7.3.1. Section 8.5.2 treats multipl e correlation with partial information and
Section 8.1 deals with tests with missing data. In Section 9.5 we discuss the
relationship between discriminant analysis and cluster analysis.
A new Appendix A deal ing with tables of chi-square adjustments to the Wilks’
criterion U (Schatkoff, M. (1966), Biometrika, pp. 347– 358, and Pillai, K.C.S.
and Gupta, A.K. (1969), Biometrika, pp. 109 – 118) is added. Appendix B lists the
publications of the author.
In preparing this volume I have tried to incorporate various comments of
reviewers of the first edition and colleagues who have used it. The comments of
v
my own students and my long experience in teaching the subject have also been
utilized in preparing the Second Edition.
Narayan C. Giri
vi Preface to the Second Edition
Preface to the First Edition
This book is an up-to-date pres entation of both theoretical and applied aspects of

multivariate analysis using the invariance approach. It is written for readers with
knowledge of mathematics and statistics at the undergraduate level. Various
concepts are explained with live data from applied areas. In conformity with the
general nature of introductory textbooks, we have tried to include many examples
and motivations relevant to specific topics. The material presented here is
developed from the subjects included in my earlier books on multivariate
statistical inference. My long experience teaching multivariate statistical analysis
courses in several universities and the comments of my students have also been
utilized in writing this volume.
Invariance is the mathematical term for symmetry with respect to a certain
group of transformations. As in other branches of mathematics the notion of
invariance in statistical inference is an old one. The unpublished work of Hunt
and Stein toward the end of World War II has given very strong support to the
applicability and meaningfulness of this notion in the framework of the general
class of statistical tests. It is now established as a very powerful tool for proving
the optimality of many statistical test procedures. It is a generally accepted
principle that if a problem with a unique solution is invariant under a certain
transformation, then the solution should be invariant under that transformation.
Another compelling reason for discussing multivariate analysis through
invariance is that most of the commonly used test procedures are likelihood
ratio tests. Under a mild restriction on the parametric space and the probability
vii
density functions under consideration, the likelihood ratio tests are almost
invariant.
Invariant tests depend on the observations only through maximal invariant. To
find optimal invariant tests we need to find the explicit form of the maximal
invariant statistic and its distribut ion. In many testing problems it is not always
convenient to find the explicit form of the maximal invariant. Stein (1956) gave a
representation of the ratio of probability densities of a maximal invariant by
integrating with respect to a invariant measure on the group of transformations

leaving the problem invariant. Stein did not give explicitly the conditions under
which his representation is valid. Subsequently many workers gave sufficient
conditions for the validity of his representation. Spherically and elliptically
symmetric distributions form an important family of nonnormal symmetric
distributions of which the multivariate normal distribution is a member. This
family is becoming increasingly important in robustness studies where the aim is
to determine how sensitive the commonly used multivariate methods are to the
multivariate normality assumption. Chapter 1 contains some special results
regarding characteristic roots and vectors, and partitioned submatrices of real and
complex matrices. It also contains some special results on determinants and
matrix derivatives and some special theorems on real and complex matrices.
Chapter 2 deals with the theory of groups and related results that are useful for
the development of invariant statistical test procedures. It also contains results on
Jacobians of some important transformations that are used in multivariate
sampling distributions.
Chapter 3 is devoted to basic notions of multivariate distributions and the
principle of invariance in statistical inference. The interrelationship between
invariance and sufficiency, invariance and unbiasedness, invariance and optimal
tests, and invariance and most stringent tests are examined. This chapter also
includes the Stein representation theorem, Hunt and Stein theorem, and
robustness studies of statistical tests.
Chapter 4 deals with multivariate normal distributions by means of the
probability density function and a simple characterization. The second approach
simplifies multivariate theory and allows suitable generalization from univariate
theory without further analysis. This chapter also contains some characterizations
of the real multivariate normal distribution, concentration ellipsoid and axes,
regression, multiple and partial correlation, and cumulants and kurtosis. It also
deals with analogous results for the complex multivariate normal distribution,
and elliptically and spherically symmetric distributions. Results on vec operator
and tensor product are also included here.

Maximum likelihood estimators of the parameters of the multivariate normal,
the multivariate complex normal, the elliptically and spherically symmetric
distributions and their optimal properties are the main subject matter of Chapter
5. The James – Stein estimator, the positive part of the James – Stein estimator,
viii Preface to the First Edition
unbiased estimation of risk, smoother shrinkage estimation of mean with known
and unknown covariance matrix are considered here.
Chapter 6 contains a systematic derivation of basic multivariate sampling
distributions for the multivariate normal case, the complex multivariate normal
case, and the case of symmetric distributions.
Chapter 7 deals with tests and confiden ce regions of mean vectors of
multivariate normal populations with known and unknown covariance matrices
and their optimal properties, tests of hypotheses concerning the subvectors of
m
in
multivariate normal, tests of mean in multivariate complex normal and
symmetric distributions, and the robustness of the T
2
-test in the family of
elliptically symmetric distributions.
Chapter 8 is devoted to a systematic derivation of tests concerning covariance
matrices and mean vectors, the sphericity test, tests of independence, the R
2
-test,
a special problem in a test of independence, MANOVA, GMANOVA, extended
GMANOVA, equality of covaria nce matrice in multivariate normal populations
and their extensions to complex multivariate normal, and the study of robustness
in the family of elliptically symmetric distributions.
Chapter 9 contains a modern treatment of discriminant analysis. A brief
history of discriminant analysis is also included here.

Chapter 10 deals with several aspects of principal component analysis in
multivariate normal populations.
Factor analysis is treated in Chapter 11 and various aspects of canonical
correlation analysis are treated in Chapter 12.
I believe that it would be appropriate to spread the materials over two three-
hour one-semester basic courses on multivariate analysis for statistics graduate
students or one three-hour one-semester course for graduate students in
nonstatistic majors by proper selection of materials according to need.
Narayan C. Giri
Preface to the First Edition ix

Contents
Preface to the Second Edition v
Preface to the First Edition vii
1 VECTOR AND MATRIX ALGEBRA 1
1.0 Introduction 1
1.1 Vectors 1
1.2 Matrices 4
1.3 Rank and Trace of a Matrix 7
1.4 Quadratic Forms and Positive Definite Matrix 7
1.5 Characteristic Roots and Vectors 8
1.6 Partitioned Matrix 16
1.7 Some Special Theorems on Matrix Derivatives 21
1.8 Complex Matrices 24
Exercises 25
References 27
2 GROUPS, JACOBIAN OF SOME TRANSFORMATIONS,
FUNCTIONS AND SPACES 29
2.0 Introduction 29
2.1 Groups 29

2.2 Some Examples of Groups 30
2.3 Quotient Group, Homomorphism, Isomorphism 31
2.4 Jacobian of Some Transformations 33
2.5 Functions and Spaces 38
References 39
xi
3 MULTIVARIATE DISTRIBUTIONS AND INVARIANCE 41
3.0 Introduction 41
3.1 Multivariate Distributions 41
3.2 Invariance in Statistical Testing of Hypotheses 44
3.3 Almost Invariance and Invariance 49
3.4 Sufficiency and Invariance 55
3.5 Unbiasedness and Invariance 56
3.6 Invariance and Optimum Tests 57
3.7 Most Stringent Tests and Invariance 58
3.8 Locally Best and Uniformly Most Powerful Invariant Tests 58
3.9 Ratio of Distributions of Maximal Invariant, Stein’s Theorem 59
3.10 Derivation of Locally Best Invariant Tests (LBI) 61
Exercises 63
References 65
4 PROPERTIES OF MULTIVARIATE DISTRIBUTIONS 69
4.0 Introduction 69
4.1 Multivariate Normal Distribution (Classical Approach) 70
4.2 Complex Multivariate Normal Distribution 84
4.3 Symmetric Distribution: Its Properties and Characterizations 91
4.4 Concentration Ellipsoid and Axes (Multivariate Normal) 110
4.5 Regression, Multiple and Partial Correlation 112
4.6 Cumulants and Kurtosis 118
4.7 The Redundancy Index 120
Exercises 120

References 127
5 ESTIMATORS OF PARAMETERS AND THEIR FUNCTIONS 131
5.0 Introduction 131
5.1 Maximum Likelihood Estimators of
m
, S in N
p
ð
m
; SÞ 132
5.2 Classical Properties of Maximum Likelihood Estimators 141
5.3 Bayes, Minimax, and Admissible Characters 151
5.4 Equivariant Estimation Under Curved Models 184
Exercises 202
References 206
6 BASIC MULTIVARIATE SAMPLING DISTRIBUTIONS 211
6.0 Introduction 211
6.1 Noncentral Chi-Square, Student’s t-, F-Distributions 211
6.2 Distribution of Quadratic Forms 213
6.3 The Wishart Distribution 218
6.4 Properties of the Wishart Distribution 224
6.5 The Noncentral Wishart Distr ibution 231
xii Contents
6.6 Generalized Variance 232
6.7 Distribution of the Bartlett Decomposition (Rectangular
Coordinates) 233
6.8 Distribution of Hotelling’s T
2
234
6.9 Multiple and Partial Correlation Coefficients 241

6.10 Distribution of Multiple Partial Correlation Coefficients 245
6.11 Basic Distributions in Multivariate Complex Normal 248
6.12 Basic Distributions in Symmetrical Distributions 250
Exercises 258
References 264
7 TESTS OF HYPOTHESES OF MEAN VECTORS 269
7.0 Introduction 269
7.1 Tests: Known Covariances 270
7.2 Tests: Unknown Covariances 272
7.3 Tests of Subvectors of
m
in Multivariate Normal 299
7.4 Tests of Mean Vector in Complex Normal 307
7.5 Tests of Means in Symmetric Distributions 309
Exercises 317
References 320
8 TESTS CONCERNING COVARIANCE MATRICES AND
MEAN VECTORS 325
8.0 Introduction 325
8.1 Hypothesis: A Covariance Matrix Is Unknown 326
8.2 The Sphericity Test 337
8.3 Tests of Independence and the R
2
-Test 342
8.4 Admissibility of the Test of Independence and the R
2
-Te st 349
8.5 Minimax Character of the R
2
-Test 353

8.6 Multivariate General Linear Hypothesis 369
8.7 Equality of Several Covariance Matrices 389
8.8 Complex Analog of R
2
-Test 406
8.9 Tests of Scale Matrices in E
p
ð
m
; SÞ 407
8.10 Tests with Missing Data 412
Exercises 423
References 427
9 DISCRIMINANT ANALYSIS 435
9.0 Introduction 435
9.1 Examples 437
9.2 Formulation of the Problem of Discriminant Analysis 438
9.3 Classification into One of Two Multivariate Normals 444
9.4 Classification into More than Two Multivariate Normals 468
Contents xiii
9.5 Concluding Remarks 473
9.6 Discriminant Analysis and Cluster Analysis 473
Exercises 474
References 477
10 PRINCIPAL COMPONENTS 483
10.0 Introduction 483
10.1 Principal Components 483
10.2 Population Principal Components 485
10.3 Sample Principal Components 490
10.4 Example 492

10.5 Distribution of Characteristic Roots 495
10.6 Testing in Principal Components 498
Exercises 501
References 502
11 CANONICAL CORRELATIONS 505
11.0 Introduction 505
11.1 Population Canonical Correlations 506
11.2 Sample Canonical Correlations 510
11.3 Tests of Hypotheses 511
Exercises 514
References 515
12 FACTOR ANALYSIS 517
12.0 Introduction 517
12.1 Orthogonal Factor Model 518
12.2 Oblique Factor Model 519
12.3 Estimation of Factor Loadings 519
12.4 Tests of Hypothesis in Factor Models 524
12.5 Time Series 525
Exercises 526
References 526
13 BIBLIOGRAPHY OF RELATED RECENT PUBLICATIONS 529
Appendix A TABLES FOR THE CHI-SQUARE ADJUSTMENT
FACTOR 531
Appendix B PUBLICATIONS OF THE AUTHOR 543
Author Index 551
Subject Index 555
xiv Contents
1
Vector and Matrix Algebra
1.0. INTRODUCTION

The study of multivariate analysis requires knowledge of vector and matrix
algebra, some basic results of which are considered in this chapter. Some of these
results are stated herein without proof; proofs can be obtained from Besilevsky
(1983), Giri (1993), Graybill (1969), Maclane and Birkoff (1967), Markus and
Mine (1967), Perlis (1952), Rao (1973), or any textbook on matrix algebra.
1.1. VECTORS
A vector is an ordered p-tuple x
1
; ; x
p
and is written as
x ¼
x
1
.
.
.
x
p
0
@
1
A
:
Actually it is called a p-dimensional column vector. For brevity we shall simply
call it a p-vector or a vector. The transpose of x is given by x
0
¼ðx
1
; ; x

p
Þ.Ifall
components of a vector are zero, it is called the null vector 0. Geometrically a
p-vector represents a point A ¼ðx
1
; ; x
p
Þ or the directed line segment 0A
!
with
1
the point A in the p-dimensional Euclidean space E
p
. The set of all p-vectors is
denoted by V
p
. Obviously V
p
¼ E
p
if all components of the vecto rs are real
numbers. Fo r any two vectors x ¼ðx
1
; ; x
p
Þ
0
and y ¼ðy
1
; ; y

p
Þ
0
we define
the vector sum x þ y ¼ðx
1
þ y
1
; ; x
p
þ y
p
Þ
0
and scalar multiplication by a
constant a by
ax ¼ðax
1
; ; ax
p
Þ
0
:
Obviously vector addition is an associative and commutative operation, i.e.,
x þ y ¼ y þ x; ðx þ yÞþz ¼ x þðy þ zÞ where z ¼ðz
1
; ; z
p
Þ
0

,andscalar
multiplication is a distributive operation, i.e., for constants a; b; ða þ bÞx ¼
ax þ bx. For x; y [ V
p
; x þ y and ax also belong to V
p
. Furthermore, for scalar
constants a; b; aðx þ yÞ¼ax þ ay and aðbxÞ¼bðaxÞ¼abx:
The quantity x
0
y ¼ y
0
x ¼
P
p
1
x
i
y
i
is called the dot product of two vectors x; y
in V
p
. The dot product of a vector x ¼ðx
1
; ; x
p
Þ
0
with itself is denoted by

kxk
2
¼ x
0
x, where kxk is called the norm of x. Some geometrical significances of
the norm are
1. kxk
2
is the square of the distance of the point x from the origin in E
p
,
2. the square of the distance between two points ðx
1
; ; x
p
Þ; ðy
1
; ; y
p
Þ is
given by kx  yk
2
,
3. the angle
u
between two vectors x; y is given by cos
u
¼ðx=kxkÞ
0
ðy=kykÞ.

Definition 1.1.1. Orthogonal vectors. Two vectors x; y in V
p
are said to be
orthogonal to each other if and only if x
0
y ¼ y
0
x ¼ 0. A set of vectors in V
p
is
orthogonal if the vectors are pairwise orthogonal.
Geometrically two vectors x; y are orthogonal if and only if the angle between
them is 908. An orthogonal vector x is called an orthonormal vector if kxk
2
¼ 1.
Definition 1.1.2. Projection of a vector. The projection of a vector x on yð= 0Þ,
both belonging to V
p
, is given by kyk
2
ðx
0
yÞy. (See Fig. 1.1.)
If 0A
!
¼ x; 0B
!
¼ y, and P is the foot of the perpendicular from the point A on
0B, then 0P
!

¼kyk
2
ðx
0
yÞy where 0 is the origin of E
p
. For two orthogonal
vectors x; y the projection of x on y is zero.
Definition 1.1.3. A set of vectors
a
1
; ;
a
k
in V
p
is said to be linearly
independent if none of the vectors can be expressed as a linear combination of the
others.
Thus if
a
1
; ;
a
k
are linearly independent, then there does not exist a set of
scalar constants c
1
; ; c
k

not all zero such that c
1
a
1
þþc
k
a
k
¼ 0. It may be
verified that a set of orthogonal vectors in V
p
is linearly independent.
2 Chapter 1
Definition 1.1.4. Vector space spanned by a set of vectors. Let
a
1
; ;
a
k
be a
set of k vectors in V
P
. Then the vector space V spanned by
a
1
; ;
a
k
is the set of
all vectors which can be expressed as linear combinations of

a
1
; ;
a
k
and the
null vector 0.
Thus if
a
;
b
[ V, then for scalar constants a; b; a
a
þ b
b
and a
a
also belong
to V. Furthermore, since
a
1
; ;
a
k
belong to V
p
, any linear combination of
a
1
; ;

a
k
also belongs to V
p
and hence V , V
p
.SoV is a linear subspace of V
p
.
Definition 1.1.5. Basis of a vector space. A basis of a vector space V is a set
of linearly independent vectors which span V.
In V
p
the unit vectors
e
1
¼ð1; 0; ; 0Þ
0
;
e
2
¼ð0; 1; 0; ; 0Þ
0
; ;
e
p
¼
ð0; ; 0; 1Þ
0
form a basis of V

p
.IfA and B are two disjoint linear subspaces of V
p
such that A < B ¼ V
p
then A and B are complementary subspaces.
Theorem 1.1.1. Every vector space V has a basis and two bases of V have the
same number of elements.
Theorem 1.1.2. Let the vector space V be spanned by the vectors
a
1
; ;
a
k
.
Any element
a
[ V can be uniquely expressed as
a
¼
P
k
1
c
i
a
i
for scalar
constants c
1

; ; c
k
, not all zero, if and only if
a
1
; ;
a
k
is a basis of V.
Definition 1.1.6. Coordinates of a vector.If
a
1
; ;
a
k
is a basis of a vector
space V and if
a
[ V is uniquely expressed as
a
¼
P
k
1
c
i
a
i
for scalar constants
c

1
; ; c
k
, then the coefficient c
i
of the vector
a
i
is called the ith coordinate of
a
with respect to the basis
a
1
; ;
a
k
.
Figure 1.1. Projection of x on y
Vector and Matrix Algebra 3
Definition 1.1.7. Rank of a vector space. The number of vectors in a basis of a
vector space V is called the rank or the dimension of V.
1.2. MATRICES
Definition 1.2.1. Matrix. A real matrix A is an ordered rectangular array of
elements a
ij
(reals)
A ¼
a
11
 a

1q
.
.
.
.
.
.
a
p1
 a
pq
0
B
@
1
C
A
ð1:1Þ
and is written as A
pq
¼ða
ij
Þ.
A matrix with p rows and q columns is called a matrix of dimension p  q(p by
q), the number of rows always being listed first. If p ¼ q, we call it a square
matrix of dimension p.
A p-dimensional column vector is a matrix of dimension p  1. Two matrice s
of the same dimension A
pq
; B

pq
are said to be equal (written as A ¼ B)if
a
ij
¼ b
ij
for i ¼ 1; ; p; j ¼ 1; ; q.Ifalla
ij
¼ 0, then A is called a null matrix
and is denoted 0. The transpose of a p  q matrix A is a q  p matrix A
0
:
A
0
¼
a
11
 a
p1
.
.
.
.
.
.
a
1q
 a
pq
0

B
@
1
C
A
ð1:2Þ
and is obtained by interchanging the row s and columns of A. Obviously ðA
0
Þ
0
¼ A.
A square matrix A is said to be symmetric if A ¼ A
0
and is skew symmetric if
A ¼A
0
. The diagonal elements of a skew symmetric matrix are zero. In what
follows we shall use the notation “A of dimension p  q” instead of A
pq
.
For any two matrices A ¼ða
ij
Þ and B ¼ðb
ij
Þ of the same dimension p  q we
define the matrix sum A þ B as a matrix ða
ij
þ b
ij
Þ of dimension p  q. The

matrix A  B is to be understood in the same sense as A þ B where the plus (þ )is
replaced by the minus (2 ) sign. Clearly ðA þ BÞ
0
¼ A
0
þ B
0
; A þ B ¼ B þ A, and
for any three matrices A; B; C; ðA þ BÞþC ¼ A þðB þ CÞ. Thus the operation
matrix sum is commutative and associative.
For any matrix A ¼ða
ij
Þ and a scalar constant c, the scalar product cA is
defined by cA ¼ Ac ¼ðca
ij
Þ. Obviously ðcAÞ
0
¼ cA
0
, so scalar product is a
distributive operation.
4 Chapter 1