Matrix mathematics theory, facts, and formulas, second edition

Matrix Mathematics


www.pdfgrip.com

Skew Reflectors

itar

y

itian

Re

fle

cto

rs

Herm

Un

w
Ske

O

N

o
ilp

te

nt

Projectors
Positiv

Positive
Definite

e Semidefinite
Hermitian

Norm

al

er
ge H
Ran

mit

ian

ve

p In
Grou

rtib

le


www.pdfgrip.com

Matrix Mathematics

Theory, Facts, and Formulas

Dennis S. Bernstein

PRINCETON UNIVERSITY PRESS
PRINCETON AND OXFORD


www.pdfgrip.com

Copyright c 2009 by Princeton University Press
Published by Princeton University Press,
41 William Street, Princeton, New Jersey 08540
In the United Kingdom: Princeton University Press,
6 Oxford Street, Woodstock, Oxfordshire, 0X20 1TW
All Rights Reserved

Library of Congress Cataloging-in-Publication Data

Bernstein, Dennis S., 1954–
Matrix mathematics: theory, facts, and formulas / Dennis S. Bernstein. – 2nd ed.
p. cm.
Includes bibliographical references and index.
ISBN 978-0-691-13287-7 (hardcover : alk. paper)
ISBN 978-0-691-14039-1 (pbk. : alk. paper)
1. Matrices. 2. Linear systems. I. Title.
QA188.B475 2008
512.9’434—dc22
2008036257
British Library Cataloging-in-Publication Data is available
This book has been composed in Computer Modern and Helvetica.
The publisher would like to acknowledge the author of this volume for providing the
camera-ready copy from which this book was printed.
Printed on acid-free paper. ∞
www.press.princeton.edu
Printed in the United States of America
10

9

8

7

6

5

4

3

2

1


www.pdfgrip.com

To the memory of my parents


www.pdfgrip.com


www.pdfgrip.com

. . . vessels, unable to contain the great light flowing into them, shatter and
break. . . . the remains of the broken vessels fall . . . into the lowest world, where
they remain scattered and hidden
— D. W. Menzi and Z. Padeh,
The Tree of Life, Chayyim Vital’s
Introduction to the Kabbalah of
Isaac Luria, Jason Aaronson,
Northvale, 1999

Thor . . . placed the horn to his lips . . . He drank with all his might and kept
drinking as long as ever he was able; when he paused to look, he could see that the
level had sunk a little, . . . for the other end lay out in the ocean itself.

— P. A. Munch, Norse Mythology,
AMS Press, New York, 1970


www.pdfgrip.com


www.pdfgrip.com

Contents

Preface to the Second Edition

xv

Preface to the First Edition

xvii

Special Symbols

xxi

Conventions, Notation, and Terminology

xxxiii

1. Preliminaries
1.1
Logic and Sets

1.2
Functions
1.3
Relations
1.4
Graphs
1.5
Facts on Logic, Sets, Functions, and Relations
1.6
Facts on Graphs
1.7
Facts on Binomial Identities and Sums
1.8
Facts on Convex Functions
1.9
Facts on Scalar Identities and Inequalities in One Variable
1.10 Facts on Scalar Identities and Inequalities in Two Variables
1.11 Facts on Scalar Identities and Inequalities in Three Variables
1.12 Facts on Scalar Identities and Inequalities in Four Variables
1.13 Facts on Scalar Identities and Inequalities in Six Variables
1.14 Facts on Scalar Identities and Inequalities in Eight Variables
1.15 Facts on Scalar Identities and Inequalities in n Variables
1.16 Facts on Scalar Identities and Inequalities in 2n Variables
1.17 Facts on Scalar Identities and Inequalities in 3n Variables
1.18 Facts on Scalar Identities and Inequalities in Complex Variables
1.19 Facts on Trigonometric and Hyperbolic Identities
1.20 Notes

1
1

3
5
8
10
13
14
21
22
30
39
46
47
47
48
60
67
68
74
76

2. Basic Matrix Properties
2.1
Matrix Algebra
2.2
Transpose and Inner Product
2.3
Convex Sets, Cones, and Subspaces
2.4
Range and Null Space
2.5

Rank and Defect
2.6
Invertibility

77
77
84
89
93
95
98


www.pdfgrip.com

x

CONTENTS

2.7
2.8
2.9
2.10
2.11
2.12
2.13
2.14
2.15
2.16
2.17

2.18
2.19
2.20
2.21
2.22

The Determinant
Partitioned Matrices
Facts on Polars, Cones, Dual Cones, Convex Hulls, and Subspaces
Facts on Range, Null Space, Rank, and Defect
Facts on the Range, Rank, Null Space, and Defect of
Partitioned Matrices
Facts on the Inner Product, Outer Product, Trace, and Matrix
Powers
Facts on the Determinant
Facts on the Determinant of Partitioned Matrices
Facts on Left and Right Inverses
Facts on the Adjugate and Inverses
Facts on the Inverse of Partitioned Matrices
Facts on Commutators
Facts on Complex Matrices
Facts on Geometry
Facts on Majorization
Notes

3. Matrix Classes and Transformations
3.1
3.2
3.3
3.4

3.5
3.6
3.7
3.8
3.9
3.10
3.11
3.12
3.13
3.14
3.15
3.16
3.17
3.18
3.19
3.20
3.21
3.22
3.23

Matrix Classes
Matrices Based on Graphs
Lie Algebras and Groups
Matrix Transformations
Projectors, Idempotent Matrices, and Subspaces
Facts on Group-Invertible and Range-Hermitian Matrices
Facts on Normal, Hermitian, and Skew-Hermitian Matrices
Facts on Commutators
Facts on Linear Interpolation
Facts on the Cross Product

Facts on Unitary and Shifted-Unitary Matrices
Facts on Idempotent Matrices
Facts on Projectors
Facts on Reflectors
Facts on Involutory Matrices
Facts on Tripotent Matrices
Facts on Nilpotent Matrices
Facts on Hankel and Toeplitz Matrices
Facts on Hamiltonian and Symplectic Matrices
Facts on Miscellaneous Types of Matrices
Facts on Groups
Facts on Quaternions
Notes

4. Polynomial Matrices and Rational Transfer Functions
4.1
4.2
4.3

Polynomials
Polynomial Matrices
The Smith Decomposition and Similarity Invariants

102
106
110
115
120
126
128

132
140
141
146
149
151
154
162
164
165
165
170
171
173
175
177
178
184
185
186
189
198
206
211
212
212
213
215
216
217

221
225
229
231
231
234
236


www.pdfgrip.com

CONTENTS

4.4
4.5
4.6
4.7
4.8
4.9
4.10
4.11
4.12

Eigenvalues
Eigenvectors
The Minimal Polynomial
Rational Transfer Functions and the Smith-McMillan
Decomposition
Facts on Polynomials and Rational Functions
Facts on the Characteristic and Minimal Polynomials

Facts on the Spectrum
Facts on Graphs and Nonnegative Matrices
Notes

xi
239
245
247
249
253
260
265
272
281

5. Matrix Decompositions
5.1
Smith Form
5.2
Multicompanion Form
5.3
Hypercompanion Form and Jordan Form
5.4
Schur Decomposition
5.5
Eigenstructure Properties
5.6
Singular Value Decomposition
5.7
Pencils and the Kronecker Canonical Form

5.8
Facts on the Inertia
5.9
Facts on Matrix Transformations for One Matrix
5.10 Facts on Matrix Transformations for Two or More Matrices
5.11 Facts on Eigenvalues and Singular Values for One Matrix
5.12 Facts on Eigenvalues and Singular Values for Two or More
Matrices
5.13 Facts on Matrix Pencils
5.14 Facts on Matrix Eigenstructure
5.15 Facts on Matrix Factorizations
5.16 Facts on Companion, Vandermonde, and Circulant Matrices
5.17 Facts on Simultaneous Transformations
5.18 Facts on the Polar Decomposition
5.19 Facts on Additive Decompositions
5.20 Notes

283
283
283
287
292
295
301
304
307
311
316
321

6. Generalized Inverses

363

6.1
6.2
6.3
6.4
6.5
6.6
6.7

Moore-Penrose Generalized Inverse
Drazin Generalized Inverse
Facts on the Moore-Penrose Generalized Inverse for One
Matrix
Facts on the Moore-Penrose Generalized Inverse for Two or
More Matrices
Facts on the Moore-Penrose Generalized Inverse for
Partitioned Matrices
Facts on the Drazin and Group Generalized Inverses
Notes

7. Kronecker and Schur Algebra
7.1
7.2

Kronecker Product
Kronecker Sum and Linear Matrix Equations

333
338
338
345
352
358
359
360
361

363
367
369
377
385
393
398
399
399
402


www.pdfgrip.com

xii

CONTENTS

7.3
7.4

7.5
7.6
7.7

Schur Product
Facts on the Kronecker Product
Facts on the Kronecker Sum
Facts on the Schur Product
Notes

404
405
409
413
416

8. Positive-Semidefinite Matrices
8.1
Positive-Semidefinite and Positive-Definite Orderings
8.2
Submatrices
8.3
Simultaneous Diagonalization
8.4
Eigenvalue Inequalities
8.5
Exponential, Square Root, and Logarithm of Hermitian Matrices
8.6
Matrix Inequalities
8.7

Facts on Range and Rank
8.8
Facts on Structured Positive-Semidefinite Matrices
8.9
Facts on Identities and Inequalities for One Matrix
8.10 Facts on Identities and Inequalities for Two or More Matrices
8.11 Facts on Identities and Inequalities for Partitioned Matrices
8.12 Facts on the Trace
8.13 Facts on the Determinant
8.14 Facts on Convex Sets and Convex Functions
8.15 Facts on Quadratic Forms
8.16 Facts on Simultaneous Diagonalization
8.17 Facts on Eigenvalues and Singular Values for One Matrix
8.18 Facts on Eigenvalues and Singular Values for Two or More
Matrices
8.19 Facts on Alternative Partial Orderings
8.20 Facts on Generalized Inverses
8.21 Facts on the Kronecker and Schur Products
8.22 Notes

417
417
419
422
424
430
431
443
444
450

456
467
475
485
494
500
507
508

9. Norms
9.1
Vector Norms
9.2
Matrix Norms
9.3
Compatible Norms
9.4
Induced Norms
9.5
Induced Lower Bound
9.6
Singular Value Inequalities
9.7
Facts on Vector Norms
9.8
Facts on Matrix Norms for One Matrix
9.9
Facts on Matrix Norms for Two or More Matrices
9.10 Facts on Matrix Norms for Partitioned Matrices
9.11 Facts on Matrix Norms and Eigenvalues Involving One Matrix

9.12 Facts on Matrix Norms and Eigenvalues Involving Two or More
Matrices
9.13 Facts on Matrix Norms and Singular Values for One Matrix
9.14 Facts on Matrix Norms and Singular Values for Two or More
Matrices
9.15 Facts on Least Squares

543
543
546
549
553
558
560
563
571
580
593
596

512
522
525
531
541

599
602
607
618

www.pdfgrip.com

CONTENTS

xiii

9.16

619

Notes

10. Functions of Matrices and Their Derivatives
10.1
10.2
10.3
10.4
10.5
10.6
10.7
10.8
10.9
10.10
10.11
10.12

Open Sets and Closed Sets
Limits

Continuity
Derivatives
Functions of a Matrix
Matrix Square Root and Matrix Sign Functions
Matrix Derivatives
Facts Involving One Set
Facts Involving Two or More Sets
Facts on Matrix Functions
Facts on Functions and Derivatives
Notes

11. The Matrix Exponential and Stability Theory
11.1
11.2
11.3
11.4
11.5
11.6
11.7
11.8
11.9
11.10
11.11
11.12
11.13
11.14
11.15
11.16
11.17
11.18

11.19
11.20
11.21
11.22
11.23
11.24

Definition of the Matrix Exponential
Structure of the Matrix Exponential
Explicit Expressions
Matrix Logarithms
The Logarithm Function
Lie Groups
Lyapunov Stability Theory
Linear Stability Theory
The Lyapunov Equation
Discrete-Time Stability Theory
Facts on Matrix Exponential Formulas
Facts on the Matrix Sine and Cosine
Facts on the Matrix Exponential for One Matrix
Facts on the Matrix Exponential for Two or More Matrices
Facts on the Matrix Exponential and Eigenvalues,
Singular Values, and Norms for One Matrix
Facts on the Matrix Exponential and Eigenvalues,
Singular Values, and Norms for Two or More Matrices
Facts on Stable Polynomials
Facts on Stable Matrices
Facts on Almost Nonnegative Matrices
Facts on Discrete-Time-Stable Polynomials
Facts on Discrete-Time-Stable Matrices

Facts on Lie Groups
Facts on Subspace Decomposition
Notes

12. Linear Systems and Control Theory
12.1
12.2
12.3
12.4

State Space and Transfer Function Models
Laplace Transform Analysis
The Unobservable Subspace and Observability
Observable Asymptotic Stability

621
621
622
623
625
628
629
630
632
634
637
638
642
643
643

646
651
654
656
658
660
662
666
669
671
677
677
681
689
692
695
698
706
708
712
715
716
722
723
723
726
727
732

www.pdfgrip.com

xiv
12.5
12.6
12.7
12.8
12.9
12.10
12.11
12.12
12.13
12.14
12.15
12.16
12.17
12.18
12.19
12.20
12.21
12.22
12.23
12.24

CONTENTS

Detectability
The Controllable Subspace and Controllability
Controllable Asymptotic Stability
Stabilizability

Realization Theory
Zeros
H2 System Norm
Harmonic Steady-State Response
System Interconnections
Standard Control Problem
Linear-Quadratic Control
Solutions of the Riccati Equation
The Stabilizing Solution of the Riccati Equation
The Maximal Solution of the Riccati Equation
Positive-Semidefinite and Positive-Definite Solutions of the
Riccati Equation
Facts on Stability, Observability, and Controllability
Facts on the Lyapunov Equation and Inertia
Facts on Realizations and the H2 System Norm
Facts on the Riccati Equation
Notes

734
735
743
747
749
757
765
768
770
772
775
778

782
787
789
790
793
798
802
805

Bibliography

807

Author Index

891

Index

903


www.pdfgrip.com

Preface to the Second Edition

This second edition of Matrix Mathematics represents a major expansion of the
original work. While the total number of pages is increased 46% from 752 to 1100,
the increase is actually greater since this edition is typeset in a smaller font to
facilitate a manageable physical size.

The second edition expands on the first edition in several ways. For example,
the new version includes material on graphs (developed within the framework of
relations and partially ordered sets), as well as alternative partial orderings of
matrices, such as rank subtractivity, star, and generalized Lăowner. This edition also
includes additional material on the Kronecker canonical form and matrix pencils;
realizations of finite groups; zeros of multi-input, multi-output transfer functions;
identities and inequalities for real and complex numbers; bounds on the roots of
polynomials; convex functions; and vector and matrix norms.
The additional material as well as works published subsequent to the first edition
increased the number of cited works from 820 to 1503, an increase of 83%. To
increase the utility of the bibliography, this edition uses the “back reference” feature
of LATEX, which indicates where each reference is cited in the text. As in the first
edition, the second edition includes an author index. The expansion of the first
edition resulted in an increase in the size of the index from 108 pages to 156 pages.
The first edition included 57 problems, while the current edition has 73. These
problems represent various extensions or generalizations of known results, sometimes motivated by gaps in the literature.
In this edition, I have attempted to correct all errors that appeared in the first
edition. As with the first edition, readers are encouraged to contact me about errors
or omissions in the current edition, which I will periodically update on my home
page.

Acknowledgments
I am grateful to many individuals who graciously provided useful advice and material for this edition. Some readers alerted me to errors, while others suggested
additional material. In other cases I sought out researchers to help me understand
the precise nature of interesting results. At the risk of omitting those who were helpful, I am pleased to acknowledge the following: Mark Balas, Jason Bernstein, Vijay
Chellaboina, Sever Dragomir, Harry Dym, Masatoshi Fujii, Rishi Graham, Wassim Haddad, Nicholas Higham, Diederich Hinrichsen, Iman Izadi, Pierre Kabamba,


www.pdfgrip.com

xvi

PREFACE TO THE SECOND EDITION

Marthe Kassouf, Christopher King, Michael Margliot, Roy Mathias, Peter Mercer,
Paul Otanez, Bela Palancz, Harish Palanthandalam-Madapusi, Fotios Paliogiannis,
Wei Ren, Mario Santillo, Christoph Schmoeger, Wasin So, Robert Sullivan, Yongge
Tian, Panagiotis Tsiotras, Gă
otz Trenkler, Chenwei Zhang, and Fuzhen Zhang.
As with the first edition, I am especially indebted to my family, who endured three
more years of my consistent absence to make this revision a reality. It is clear that
any attempt to fully embrace the enormous body of mathematics known as matrix
theory is a neverending task. After committing almost two decades to the project, I
remain, like Thor, barely able to perceive a dent in the vast knowledge that resides
in the hundreds of thousands of pages devoted to this fascinating and incredibly
useful subject. Yet, it my hope, that this book will prove to be valuable to all of
those who use matrices, and will inspire interest in a mathematical construction
whose secrets and mysteries know no bounds.
Dennis S. Bernstein
Ann Arbor, Michigan

October 2008


www.pdfgrip.com

Preface to the First Edition

The idea for this book began with the realization that at the heart of the solution
to many problems in science, mathematics, and engineering often lies a “matrix

fact,” that is, an identity, inequality, or property of matrices that is crucial to the
solution of the problem. Although there are numerous excellent books on linear
algebra and matrix theory, no one book contains all or even most of the vast number
of matrix facts that appear throughout the scientific, mathematical, and engineering
literature. This book is an attempt to organize many of these facts into a reference
source for users of matrix theory in diverse applications areas.
Viewed as an extension of scalar mathematics, matrix mathematics provides the
means to manipulate and analyze multidimensional quantities. Matrix mathematics
thus provides powerful tools for a broad range of problems in science and engineering. For example, the matrix-based analysis of systems of ordinary differential equations accounts for interaction among all of the state variables. The discretization of
partial differential equations by means of finite differences and finite elements yields
linear algebraic or differential equations whose matrix structure reflects the nature
of physical solutions [1238]. Multivariate probability theory and statistical analysis
use matrix methods to represent probability distributions, to compute moments,
and to perform linear regression for data analysis [504, 606, 654, 702, 947, 1181].
The study of linear differential equations [691, 692, 727] depends heavily on matrix
analysis, while linear systems and control theory are matrix-intensive areas of engineering [3, 65, 142, 146, 311, 313, 348, 371, 373, 444, 502, 616, 743, 852, 865, 935,
1094, 1145, 1153, 1197, 1201, 1212, 1336, 1368, 1455, 1498]. In addition, matrices
are widely used in rigid body dynamics [26, 726, 733, 789, 806, 850, 970, 1026,
1068, 1069, 1185, 1200, 1222, 1351], structural mechanics [863, 990, 1100], computational fluid dynamics [305, 479, 1426], circuit theory [30], queuing and stochastic
systems [642, 919, 1034], econometrics [403, 948, 1119], geodesy [1241], game theory
[225, 898, 1233], computer graphics [62, 498], computer vision [941], optimization
[255, 374, 953], signal processing [702, 1163, 1361], classical and quantum information theory [353, 702, 1042, 1086], communications systems [778, 779], statistics
[580, 654, 948, 1119, 1177], statistical mechanics [16, 159, 160, 1372], demography
[297, 805], combinatorics, networks, and graph theory [165, 128, 179, 223, 235, 266,
269, 302, 303, 335, 363, 405, 428, 481, 501, 557, 602, 702, 844, 920, 931, 1143, 1387],
optics [549, 659, 798], dimensional analysis [641, 1252], and number theory [841].
In all applications involving matrices, computational techniques are essential for
obtaining numerical solutions. The development of efficient and reliable algorithms
for matrix computations is therefore an important area of research that has been

www.pdfgrip.com

xviii

PREFACE TO THE FIRST EDITION

extensively developed [95, 304, 396, 569, 681, 683, 721, 752, 1224, 1225, 1227, 1229,
1315, 1369, 1427, 1431, 1433, 1478]. To facilitate the solution of matrix problems,
entire computer packages have been developed using the language of matrices. However, this book is concerned with the analytical properties of matrices rather than
their computational aspects.
This book encompasses a broad range of fundamental questions in matrix theory, which, in many cases can be viewed as extensions of related questions in scalar
mathematics. A few such questions follow.
What are the basic properties of matrices?
characterized, classified, and quantified?

How can matrices be

How can a matrix be decomposed into simpler matrices? A matrix
decomposition may involve addition, multiplication, and partition.
Decomposing a matrix into its fundamental components provides
insight into its algebraic and geometric properties. For example, the
polar decomposition states that every square matrix can be written
as the product of a rotation and a dilation analogous to the polar
representation of a complex number.
Given a pair of matrices having certain properties, what can be
inferred about the sum, product, and concatenation of these matrices?
In particular, if a matrix has a given property, to what extent does that
property change or remain unchanged if the matrix is perturbed by
another matrix of a certain type by means of addition, multiplication,

or concatenation? For example, if a matrix is nonsingular, how large
can an additive perturbation to that matrix be without the sum
becoming singular?
How can properties of a matrix be determined by means of simple
operations? For example, how can the location of the eigenvalues of a
matrix be estimated directly in terms of the entries of the matrix?
To what extent do matrices satisfy the formal properties of the real
numbers? For example, while 0 ≤ a ≤ b implies that ar ≤ br for real
numbers a, b and a positive integer r, when does 0 ≤ A ≤ B imply
Ar ≤ B r for positive-semidefinite matrices A and B and with the
positive-semidefinite ordering?
Questions of these types have occupied matrix theorists for at least a century,
with motivation from diverse applications. The existing scope and depth of knowledge are enormous. Taken together, this body of knowledge provides a powerful
framework for developing and analyzing models for scientific and engineering applications.
This book is intended to be useful to at least four groups of readers. Since
linear algebra is a standard course in the mathematical sciences and engineering,
graduate students in these fields can use this book to expand the scope of their


www.pdfgrip.com

PREFACE TO THE FIRST EDITION

xix

linear algebra text. For instructors, many of the facts can be used as exercises to
augment standard material in matrix courses. For researchers in the mathematical
sciences, including statistics, physics, and engineering, this book can be used as
a general reference on matrix theory. Finally, for users of matrices in the applied
sciences, this book will provide access to a large body of results in matrix theory.

By collecting these results in a single source, it is my hope that this book will prove
to be convenient and useful for a broad range of applications. The material in this
book is thus intended to complement the large number of classical and modern texts
and reference works on linear algebra and matrix theory [10, 376, 503, 540, 541,
558, 586, 701, 790, 872, 939, 956, 963, 1008, 1045, 1051, 1098, 1143, 1194, 1238].
After a review of mathematical preliminaries in Chapter 1, fundamental properties of matrices are described in Chapter 2. Chapter 3 summarizes the major classes
of matrices and various matrix transformations. In Chapter 4 we turn to polynomial and rational matrices whose basic properties are essential for understanding
the structure of constant matrices. Chapter 5 is concerned with various decompositions of matrices including the Jordan, Schur, and singular value decompositions.
Chapter 6 provides a brief treatment of generalized inverses, while Chapter 7 describes the Kronecker and Schur product operations. Chapter 8 is concerned with
the properties of positive-semidefinite matrices. A detailed treatment of vector and
matrix norms is given in Chapter 9, while formulas for matrix derivatives are given
in Chapter 10. Next, Chapter 11 focuses on the matrix exponential and stability
theory, which are central to the study of linear differential equations. In Chapter
12 we apply matrix theory to the analysis of linear systems, their state space realizations, and their transfer function representation. This chapter also includes a
discussion of the matrix Riccati equation of control theory.
Each chapter provides a core of results with, in many cases, complete proofs.
Sections at the end of each chapter provide a collection of Facts organized to correspond to the order of topics in the chapter. These Facts include corollaries and
special cases of results presented in the chapter, as well as related results that go
beyond the results of the chapter. In some cases the Facts include open problems,
illuminating remarks, and hints regarding proofs. The Facts are intended to provide
the reader with a useful reference collection of matrix results as well as a gateway
to the matrix theory literature.

Acknowledgments
The writing of this book spanned more than a decade and a half, during which
time numerous individuals contributed both directly and indirectly. I am grateful for the helpful comments of many people who contributed technical material
and insightful suggestions, all of which greatly improved the presentation and content of the book. In addition, numerous individuals generously agreed to read
sections or chapters of the book for clarity and accuracy. I wish to thank Jasim
Ahmed, Suhail Akhtar, David Bayard, Sanjay Bhat, Tony Bloch, Peter Bullen,
Steve Campbell, Agostino Capponi, Ramu Chandra, Jaganath Chandrasekhar,

Nalin Chaturvedi, Vijay Chellaboina, Jie Chen, David Clements, Dan Davison,
Dimitris Dimogianopoulos, Jiu Ding, D. Z. Djokovic, R. Scott Erwin, R. W. Farebrother, Danny Georgiev, Joseph Grcar, Wassim Haddad, Yoram Halevi, Jesse
Hoagg, Roger Horn, David Hyland, Iman Izadi, Pierre Kabamba, Vikram Kapila,


www.pdfgrip.com

xx

PREFACE TO THE FIRST EDITION

Fuad Kittaneh, Seth Lacy, Thomas Laffey, Cedric Langbort, Alan Laub, Alexander Leonessa, Kai-Yew Lum, Pertti Makila, Roy Mathias, N. Harris McClamroch,
Boris Mordukhovich, Sergei Nersesov, JinHyoung Oh, Concetta Pilotto, Harish
Palanthandalum-Madapusi, Michael Piovoso, Leiba Rodman, Phil Roe, Carsten
Scherer, Wasin So, Andy Sparks, Edward Tate, Yongge Tian, Panagiotis Tsiotras,
Feng Tyan, Ravi Venugopal, Jan Willems, Hong Wong, Vera Zeidan, Xingzhi Zhan,
and Fuzhen Zhang for their assistance. Nevertheless, I take full responsibility for
any remaining errors, and I encourage readers to alert me to any mistakes, corrections of which will be posted on the web. Solutions to the open problems are also
welcome.
Portions of the manuscript were typed by Jill Straehla and Linda Smith at Harris
Corporation, and by Debbie Laird, Kathy Stolaruk, and Suzanne Smith at the
University of Michigan. John Rogosich of Techsetters, Inc., provided invaluable
assistance with LATEX issues, and Jennifer Slater carefully copyedited the entire
manuscript. I also thank JinHyoung Oh and Joshua Kang for writing C code to
refine the index.
I especially thank Vickie Kearn of Princeton University Press for her wise guidance and constant encouragement. Vickie managed to address all of my concerns
and anxieties, and helped me improve the manuscript in many ways.
Finally, I extend my greatest appreciation for the (uncountably) infinite patience
of my family, who endured the days, weeks, months, and years that this project
consumed. The writing of this book began with toddlers and ended with a teenager

and a twenty-year old. We can all be thankful it is finally finished.
Dennis S. Bernstein
Ann Arbor, Michigan

January 2005


www.pdfgrip.com

Special Symbols

General Notation
π

3.14159 . . .

e

2.71828 . . .

=

equals by definition

limε↓0

limit from the right

α
m

α(α−1)···(α−m+1)
m!

n
m

n!
m!(n−m)!

a

largest integer less than or equal to a

δij

1 if i = j, 0 if i = j (Kronecker delta)

log

logarithm with base e

sign α

1 if α > 0, −1 if α < 0, 0 if α = 0

Chapter 1
{}

set (p. 2)

∈

is an element of (p. 2)

∈

is not an element of (p. 2)

∅

empty set (p. 2)

{ }ms

multiset (p. 2)

card

cardinality (p. 2)

∩

intersection (p. 2)

∪

union (p. 2)

Y\X

X

∼

complement of X relative to Y (p. 2)
complement of X (p. 2)


www.pdfgrip.com

xxii

SPECIAL SYMBOLS

⊆

is a subset of (p. 2)

⊂

is a proper subset of (p. 3)

(x1, . . . , xn )

tuple or n-tuple (p. 3)

Graph(f )

{(x, f(x)): x ∈ X} (p. 3)

f: X → Y

f is a function with domain X and codomain
Y (p. 3)

f •g

composition of functions f and g (p. 3)

−1

f (S)

inverse image of S (p. 4)

rev(R)

reversal of the relation R (p. 5)

∼

R

complement of the relation R (p. 5)

ref(R)

reflexive hull of the relation R (p. 5)

sym(R)

symmetric hull of the relation R (p. 5)

trans(R)

transitive hull of the relation R (p. 5)

equiv(R)

equivalence hull of the relation R (p. 5)

R

x=y

(x, y) is an element of the equivalence relation
R (p. 6)

glb(S)

greatest lower bound of S (p. 7, Definition
1.3.9)

lub(S)

least upper bound of S (p. 7, Definition 1.3.9)

inf(S)

infimum of S (p. 7, Definition 1.3.9)

sup(S)

supremum of S (p. 7, Definition 1.3.9)

rev(G)

reversal of the graph G (p. 8)

∼

G

complement of the graph G (p. 8)

ref(G)

reflexive hull of the graph G (p. 8)

sym(G)

symmetric hull of the graph G (p. 8)

trans(G)

transitive hull of the graph G (p. 8)

equiv(G)

equivalence hull of the graph G (p. 8)

indeg(x)

indegree of the node x (p. 9)

outdeg(x)

outdegree of the node x (p. 9)

deg(x)

degree of the node x (p. 9)

Chapter 2
Z

integers (p. 77)

N

nonnegative integers (p. 77)


www.pdfgrip.com

xxiii

SPECIAL SYMBOLS

P

positive integers (p. 77)

R

real numbers (p. 77)

C

complex numbers (p. 77)

F
j

R or C (p. 77)
√
−1 (p. 77)

z

complex conjugate of z ∈ C (p. 77)

Re z

real part of z ∈ C (p. 77)

Im z

imaginary part of z ∈ C (p. 77)

|z|

absolute value of z ∈ C (p. 77)

OLHP

open left half plane in C (p. 77)

CLHP

closed left half plane in C (p. 77)

ORHP

open right half plane in C (p. 77)

CRHP

closed right half plane in C (p. 77)

jR

imaginary numbers (p. 77)

Rn

Rn×1 (real column vectors) (p. 78)

Cn

Cn×1 (complex column vectors) (p. 78)

Fn

Rn or Cn (p. 78)

x(i)

ith component of x ∈ Fn (p. 78)

x ≥≥ y

x(i) ≥ y(i) for all i (x − y is nonnegative)
(p. 79)

x >> y

x(i) > y(i) for all i (x − y is positive) (p. 79)

Rn×m

n × m real matrices (p. 79)

C

n × m complex matrices (p. 79)

n×m

Fn×m

Rn×m or Cn×m (p. 79)

rowi(A)

ith row of A (p. 79)

coli(A)

ith column of A (p. 79)

A(i,j)

(i, j) entry of A (p. 79)

i

A←b

matrix obtained from A ∈ Fn×m by replacing
coli(A) with b ∈ Fn or rowi(A) with b ∈ F1×m
(p. 80)

dmax(A) = d1(A)

largest diagonal entry of A ∈ Fn×n having real
diagonal entries (p. 80)

di(A)

ith largest diagonal entry of A ∈ Fn×n having
real diagonal entries (p. 80)


www.pdfgrip.com

xxiv

SPECIAL SYMBOLS

dmin(A) = dn(A)

smallest diagonal entry of A ∈ Fn×n having
real diagonal entries (p. 80)

A(S1,S2 )

submatrix of A formed by retaining the rows
of A listed in S1 and the columns of A listed
in S2 (p. 80)

A(S)

A(S,S) (p. 80)

A ≥≥ B

A(i,j) ≥ B(i,j) for all i, j (A − B is
nonnegative) (p. 81)

A >> B

A(i,j) > B(i,j) for all i, j (A − B is positive)
(p. 81)

[A, B]

commutator AB − BA (p. 82)

adA(X)

adjoint operator [A, X] (p. 82)

x×y

cross product of vectors x, y ∈ R3 (p. 82)

K(x)

cross-product matrix for x ∈ R3 (p. 82)

0n×m , 0

n × m zero matrix (p. 83)

In , I

n × n identity matrix (p. 83)

ei,n , ei

coli(In ) (p. 84)

Iˆn , Iˆ

n × n reverse identity matrix
(p. 84)

Ei,j,n×m , Ei,j

ei,n eT
j,m (p. 84)

1n×m , 1

n × m ones matrix (p. 84)

T

0

1
.
..

1

0

A

transpose of A (p. 86)

tr A

trace of A (p. 86)

C

complex conjugate of C ∈ Cn×m (p. 87)

A∗

A conjugate transpose of A (p. 87)

Re A

real part of A ∈ Fn×m (p. 87)

Im A

imaginary part of A ∈ Fn×m (p. 87)

S

{Z : Z ∈ S} or {Z : Z ∈ S}ms (p. 87)
ˆ
T

T

A

ˆ TIˆ reverse transpose of A (p. 88)
IA

A∗ˆ

ˆ ∗Iˆ reverse complex conjugate transpose of A
IA
(p. 88)

|x|

absolute value of x ∈ Fn (p. 88)

|A|

absolute value of A ∈ Fn×n (p. 88)

sign x

sign of x ∈ Rn (p. 89)

sign A

sign of A ∈ Rn×n (p. 89)


www.pdfgrip.com

xxv

SPECIAL SYMBOLS

co S

convex hull of S (p. 89)

cone S

conical hull of S (p. 89)

coco S

convex conical hull of S (p. 89)

span S

span of S (p. 90)

aff S

affine hull of S (p. 90)

dim S

dimension of S (p. 90)

⊥

S

orthogonal complement of S (p. 91)

polar S

polar of S (p. 91)

dcone S

dual cone of S (p. 91)

R(A)

range of A (p. 93)

N(A)

null space of A (p. 94)

rank A

rank of A (p. 95)

def A

defect of A (p. 96)

AL

left inverse of A (p. 98)

AR

right inverse of A (p. 98)

−1

A

A−T
−∗

inverse of A (p. 101)
AT

−1

∗ −1

(p. 102)

A

(A )

det A

determinant of A (p. 103)

A[i;j]

submatrix A({i}∼ ,{j}∼ ) of A obtained by
deleting rowi(A) and colj (A) (p. 105)

AA

adjugate of A (p. 105)

(p. 102)

rs

A≤B
∗

A≤B

rank subtractivity partial ordering (p. 119, Fact
2.10.32)
star partial ordering (p. 120, Fact 2.10.35)

Chapter 3
Nn , N

n × n standard nilpotent matrix (p. 166)
a1

0

..

diag(a1, . . . , an )

(p. 167)

.

0

an

0

a1
.
..

revdiag(a1, . . . , an )
an

(p. 167)
0