Otto bretscher linear algebra with applications 4th edition

Contents
Text Features ix
Preface xi

1

Linear Equations
1.1
1.2
1.3

2

Linear Transformations
2.1
2.2
2.3
2.4

3

Introduction to Linear Systems
Matrices, Vectors, and Gauss-Jordan Elimination
On the Solutions of Linear Systems; Matrix Algebra

Introduction to Linear Transformations and
Their Inverses
Linear Transformations in Geometry
Matrix Products

The Inverse of a Linear Transformation

Subspaces of Mn and Their Dimensions
3.1 Image and Kernel of a Linear Transformation
3.2 Subspaces of R"; Bases and Linear Independence
3.3 The Dimension of a Subspace of R"
3.4 Coordinates

4

Linear Spaces
4.1
4.2
4.3

5

Introduction to Linear Spaces
Linear Transformations and Isomorphisms
The Matrix o f a Linear Transformation

Orthogonality and Least Squares
5.1 Orthogonal Projections and Orthonormal Bases
5.2 Gram-Schmidt Process and QR Factorization
5.3 Orthogonal Transformations and Orthogonal Matrices
5.4 Least Squares and Data Fitting
5.5 Inner Product Spaces

1
1

8
25

40
40
54
69
79

101
101
113
123
137

153
153
165
112

187
187
203
210
220
233

vii

viii

Contents

6 Determinants
6.1
6.2
6.3

7

Introduction to Determinants
Properties of the Determinant
Geometrical Interpretations of the Determinant;
Cramer’s Rule

Eigenvalues and Eigenvectors
7.1

249
249
261
277

294

Dynamical Systems and Eigenvectors:
An Introductory Example
Finding the Eigenvalues of a Matrix
Finding the Eigenvectors of a Matrix

Diagonalization
Complex Eigenvalues
Stability

294
308
319
332
343
357

8 Symmetric Matrices and Quadratic Forms

367

7.2
7.3
7.4
7.5
7.6

8.1
8.2
8.3

9

Symmetric Matrices
Quadratic Forms
Singular Values

Linear Differential Equations
9.1
9.2
9.3

An Introduction to Continuous Dynamical Systems
The Complex Case: Euler’s Formula
Linear Differential Operators and
Linear Differential Equations

Appendix A Vectors 437
Answers to Odd-Numbered Exercises 446
Subject Index 471
Name Index 478

367
376
385

397
397
410
423


Text Features
Continuing Text Features
• Linear transformations are introduced early on in the text to make the discus­
sion of matrix operations more meaningful and easier to visualize.

• Visualization and geometrical interpretation are emphasized extensively
throughout.
• The reader will find an abundance of thought-provoking (and occasionally
delightful) problems and exercises.
• Abstract concepts are introduced gradually throughout the text. The major
ideas are carefully developed at various levels of generality before the student
is introduced to abstract vector spaces.
• Discrete and continuous dynamical systems are used as a motivation for eigen­
vectors, and as a unifying theme thereafter.

New Features in the Fourth Edition
Students and instructors generally found the third edition to be accurate and well
structured. While preserving the overall organization and character of the text, some
changes seemed in order.
• A large number of exercises have been added to the problem sets, from the
elementary to the challenging. For example, two dozen new exercises on conic
and cubic curve fitting lead up to a discussion of the Cramer-Euler Paradox
on fitting a cubic through nine points.
• The section on matrix products now precedes the discussion of the inverse
of a matrix, making the presentation more sensible from an algebraic point
of view.
• Striking a balance between the earlier editions, the determinant is defined
in terms o f “patterns”, a transparent way to deal with permutations. Laplace
expansion and Gaussian elimination are presented as alternative approaches.
• There have been hundreds of small editorial improvements—offering a hint
in a difficult problem for example— or choosing a more sensible notation in a
theorem.

Preface (with David Steinsaltz)
police officer on patrol at midnight, so runs an old joke, notices a man
crawling about on his hands and knees under a streetlamp. He walks over
to investigate, whereupon the man explains in a tired and somewhat slurred
voice that he has lost his housekeys. The policeman offers to help, and for the next
five minutes he too is searching on his hands and knees. At last he exclaims, “Are
you absolutely certain that this is where you dropped the keys?”

A

“Here? Absolutely not. I dropped them a block down, in the middle of the
street.”
“Then why the devil have you got me hunting around this lamppost?”
“Because this is where the light is.”
It is mathematics, and not just (as Bismarck claimed) politics, that consists in “the
art of the possible.” Rather than search in the darkness for solutions to problems of
pressing interest, we contrive a realm of problems whose interest lies above all in
the fact that solutions can conceivably be found.
Perhaps the largest patch of light surrounds the techniques of matrix arithmetic
and algebra, and in particular matrix multiplication and row reduction. Here we
might begin with Descartes, since it was he who discovered the conceptual meetingpoint of geometry and algebra in the identification of Euclidean space with R 3; the
techniques and applications proliferated since his day. To organize and clarify those
is the role of a modem linear algebra course.

Computers and Computation
An essential issue that needs to be addressed in establishing a mathematical m ethod'
ology is the role of computation and of computing technology. Are the proper subjects
of mathematics algorithms and calculations, or are they grand theories and abstractions that evade the need for computation? If the former, is it important that the
students learn to carry out the computations with pencil and paper, or should the al­
gorithm “press the calculator’s x ~ 1 button” be allowed to substitute for the traditional

method of finding an inverse? If the latter, should the abstractions be taught through
elaborate notational mechanisms or through computational examples and graphs?
We seek to take a consistent approach to these questions: Algorithms and com­
putations are primary, and precisely for this reason computers are not. Again and
again we examine the nitty-gritty of row reduction or matrix multiplication in or­
der to derive new insights. Most of the proofs, whether of rank-nullity theorem,
the volume-change formula for determinants, or the spectral theorem for symmetric
matrices, are in this way tied to hands-on procedures.
The aim is not just to know how to compute the solution to a problem, but to
imagine the computations. The student needs to perform enough row reductions by
hand to be equipped to follow a line of argument of the form: “If we calculate the
reduced row echelon form of such a matrix . . . , ” and to appreciate in advance the
possible outcomes of a particular computation.
In applications the solution to a problem is hardly more important than recog­
nizing its range o f validity and appreciating how sensitive it is to perturbations o f the
input. We emphasize the geometric and qualitative nature of the solutions, notions of
approximation, stability, and “typical” matrices. The discussion of Cram er’s rule, for
instance, underscores the value of closed-form solutions for visualizing a system’s
behavior and understanding its dependence from initial conditions.


xii

Preface

The availability of computers is, however, neither to be ignored nor regretted.
Each student and instructor will have to decide how much practice is needed to be
sufficiently familiar with the inner workings of the algorithm. As the explicit compu­
tations are being replaced gradually by a theoretical overview of how the algorithm
works, the burden of calculation will be taken up by technology, particularly for

those wishing to carry out the more numerical and applied exercises.

Examples, Exercises, Applications, and History
The exercises and examples are the heart of this book. Our objective is not just to
show our readers a “patch of light” where questions may be posed and solved, but
to convince them that there is indeed a great deal of useful, interesting material
to be found in this area if they take the time to look around. Consequently, we
have included genuine applications of the ideas and methods under discussion to
a broad range of sciences: physics, chemistry, biology, economics, and, of course,
mathematics itself. Often we have simplified them to sharpen the point, but they use
the methods and models of contemporary scientists.
With such a large and varied set of exercises in each section, instructors should
have little difficulty in designing a course that is suited to their aims and to the needs
of their students. Quite a few straightforward computation problems are offered,
of course. Simple (and, in a few cases, not so simple) proofs and derivations are
required in some exercises. In many cases, theoretical principles that are discussed
at length in more abstract linear algebra courses are here found broken up in bite-size
exercises.
The examples make up a significant portion of the text; we have kept abstract
exposition to a minimum. It is a matter of taste whether general theories should
give rise to specific examples or be pasted together from them. In a text such as this
one, attempting to keep an eye on applications, the latter is clearly preferable: The
examples always precede the theorems in this book.
Scattered throughout the mathematical exposition are quite a few names and
dates, some historical accounts, and anecdotes. Students of mathematics are too
rarely shown that the seemingly strange and arbitrary concepts they study are the
results of long and hard struggles. It will encourage the readers to know that a mere
two centuries ago some of the most brilliant mathematicians were wrestling with
problems such as the meaning of dimension or the interpretation of el\ and to realize
that the advance of time and understanding actually enables them, with some effort

of their own, to see farther than those great minds.

Outline of the Text
C hapter I This chapter provides a careful introduction to the solution of systems
of linear equations by Gauss-Jordan elimination. Once the concrete problem is
solved, we restate it in terms of matrix formalism and discuss the geometric properties
of the solutions.
C hapter 2 Here we raise the abstraction a notch and reinterpret matrices as linear
transformations. The reader is introduced to the modem notion of a function, as
an arbitrary association between an input and an output, which leads into a dis­
cussion of inverses. The traditional method for finding the inverse of a matrix is
explained: It fits in naturally as a sort of automated algorithm for Gauss-Jordan
elimination.


Preface

xiii

We define linear transformations primarily in terms of matrices, since that is how
they are used; the abstract concept of linearity is presented as an auxiliary notion.
Rotations, reflections, and orthogonal projections in R 2 are emphasized, both as
archetypal, easily visualized examples, and as preparation for future applications.
C hapter 3 We introduce the central concepts of linear algebra: subspaces, image
and kernel, linear independence, bases, coordinates, and dimension, still firmly fixed
in R".
C hapter 4 Generalizing the ideas of the preceding chapter and using an abun­
dance of examples, we introduce abstract vector spaces (which are called linear
spaces here, to prevent the confusion some students experience with the term
“vector”).

C hapter 5 This chapter includes some of the most basic applications of linear
algebra to geometry and statistics. We introduce orthonormal bases and the G ram Schmidt process, along with the QR factorization. The calculation of correlation
coefficients is discussed, and the important technique of least-squares approxima­
tions is explained, in a number of different contexts.
C hapter 6 Our discussion of determinants is algorithmic, based on the counting
of patterns (a transparent way to deal with permutations). We derive the properties of
the determinant from careful analysis of this procedure, tieing it together with G aussJordan elimination. The goal is to prepare for the main application of determinants:
the computation of characteristic polynomials.
C hapter 7 This chapter introduces the central application of the latter half of
the text: linear dynamical systems. We begin with discrete systems and are nat­
urally led to seek eigenvectors, which characterize the long-term behavior of the
system. Qualitative behavior is emphasized, particularly stability conditions. Com­
plex eigenvalues are explained, without apology, and tied into earlier discussions of
two-dimensional rotation matrices.


xiv

Preface

C hapter 8 The ideas and methods of Chapter 7 are applied to geometry. We discuss
the spectral theorem for symmetric matrices and its applications to quadratic forms,
conic sections, and singular values.
C hapter 9 Here we apply the methods developed for discrete dynamical systems
to continuous ones, that is, to systems of first-order linear differential equations.
Again, the cases of real and complex eigenvalues are discussed.

Solutions Manuals
• Student's Solutions Manual, with carefully worked solutions to all oddnumbered problems in the text (ISBN 0-13-600927-1)
• Instructor's Solutions Manual, with solutions to all the problems in the text

(ISBN 0-13-600928-X)

Acknowledgments
I first thank my students and colleagues at Harvard University, Colby College, and
Koq University (Istanbul) for the key role they have played in developing this text
out of a series of rough lecture notes. The following colleagues, who have taught the
course with me, have made invaluable contributions:
Attila A§kar
Persi Diaconis
Jordan Ellenberg
Matthew Emerton
Edward Frenkel
Alexandru Ghitza

Fernando Gouvea
Jan Holly
Varga Kalantarov
David Kazhdan
Oliver Knill
Leo Livshits

Barry Mazur
David Mumford
David Steinsaltz
Shlomo Sternberg
Richard Taylor
George Welch

I am grateful to those who have taught me algebra: Peter Gabriel, Volker Strassen,
and Bartel van der Waerden at the University of Zurich; John Tate at Harvard

University; Maurice Auslander at Brandeis University; and many more.
I owe special thanks to John Boiler, William Calder, Devon Ducharme, and
Robert Kaplan for their thoughtful review of the manuscript.
I wish to thank Sylvie Bessette, Dennis Kletzing, and Laura Lawrie for the
careful preparation of the manuscript and Paul Nguyen for his well-drawn figures.
Special thanks go to Kyle Burke, who has been my assistant at Colby College for
three years, having taken linear algebra with me as a first-year student. Paying close
attention to the way his students understood or failed to understand the material, Kyle
came up with new ways to explain many of the more troubling concepts (such as the
kernel of a matrix or the notion of an isomorphism). Many of his ideas have found
their way into this text. Kyle took the time to carefully review various iterations
of the manuscript, and his suggestions and contributions have made this a much
better text.
I am grateful to those who have contributed to the book in many ways: Marilyn
Baptiste, Menoo Cung, Srdjan Divac, Devon Ducharme, Robin Gottlieb, Luke
Hunsberger, George Mani, Bridget Neale, Alec van Oot, Akilesh Palanisamy, Rita
Pang, Esther Silberstein, Radhika de Silva, Jonathan Tannenhauser, Ken WadaShiozaki, Larry Wilson, and Jingjing Zhou.


Preface

xv

I have received valuable feedback from the book’s reviewers for the various
editions:
Hirotachi Abo, University o f Idaho
Loren Argabright, Drexel University
Stephen Bean, Cornell College
Frank Beatrous, University o f Pittsburgh
Tracy Bibelnieks, University o f Minnesota

Jeff D. Farmer, University o f Northern Colorado
Michael Goldberg, Johns Hopkins University
Herman Gollwitzer, Drexel University
Fernando Gouvea, Colby College
David G. Handron, Jr., Carnegie Mellon University
Christopher Heil, Georgia Institute o f Technology
Willy Hereman, Colorado School o f Mines
Konrad J. Heuvers, Michigan Technological University
Charles Holmes, Miami University
Matthew Hudelson, Washington State University
Thomas Hunter, Swarthmore College
Michael Kallaher, Washington State University
Daniel King, Oberlin College
Richard Kubelka, San Jose State University
Michael G. Neubauer, California State University-Northridge
Peter C. Patton, University o f Pennsylvania
V. Rao Potluri, Reed College
Jeffrey M. Rabin, University o f California, San Diego
Daniel B. Shapiro, Ohio State University
David Steinsaltz, Technische Universitat Berlin
James A. Wilson, Iowa State University
Darryl Yong, Harvey Mudd College
Jay Zimmerman, Towson University
I
thank my editors, Caroline Celano and Bill Hoffman, for their unfailing
encouragement and thoughtful advice.
The development of this text has been supported by a grant from the Instructional
Innovation Fund of Harvard College.
I
love to hear from the users of this text. Feel free to write to

with any comments or concerns.
Otto Bretscher
www.colby.edu/^obretsch



Linear Equations

IH lntroduction to Linear Systems
Traditionally, algebra was the art of solving equations and systems of equations.
The word algebra comes from the Arabic al-jabr
), which means restoration
(of broken parts ) . 1 The term was first used in a mathematical sense by Mohammed
al-Khowarizmi (c. 780-850), who worked at the House o f Wisdom, an academy
established by Caliph al-M a’mun in Baghdad. Linear algebra, then, is the art of
solving systems of linear equations.
The need to solve systems of linear equations frequently arises in mathematics,
statistics, physics, astronomy, engineering, computer science, and economics.
Solving systems of linear equations is not conceptually difficult. For small
systems, ad hoc methods certainly suffice. Larger systems, however, require more
systematic methods. The approach generally used today was beautifully explained
2,000 years ago in a Chinese text, the Nine Chapters on the Mathematical Art
(Jiuzhang Suanshu, A, # # #f ).2 Chapter 8 o f that text, called Method o f Rectan­
gular Arrays (Fang Cheng,
31), contains the following problem:
The yield of one bundle of inferior rice, two bundles of medium grade rice,
and three bundles of superior rice is 39 dou of grain .3 The yield of one
bundle of inferior rice, three bundles of medium grade rice, and two bundles
of superior rice is 34 dou. The yield of three bundles of inferior rice, two
bundles of medium grade rice, and one bundle of superior rice is 26 dou.

What is the yield of one bundle of each grade of rice?
In this problem the unknown quantities are the yields of one bundle of inferior,
one bundle of medium grade, and one bundle of superior rice. Let us denote these
quantities by jc, v, and c, respectively. The problem can then be represented by the

1At one time, it was not unusual to see the sign Algehrista y Sangrador (bone setter and blood letter) at
the entrance of a Spanish barber’s shop.
2Shen Kangshen et al. (ed.). The Nine Chapters on the Mathematical Art, Companion and
Commentary, Oxford University Press, 1999.
3The dou is a measure of volume, corresponding to about 2 liters at that time.


2

CHAPTER I

Linear Equations

following system of linear equations:
x + 2 y + 3z = 39
jc + 3y + 2z = 34
3x + 2y + z = 26
To solve for jc, y, and z, we need to transform this system from the form
x

x + 2y + 3z = 39
x + 3y + 2z = 34
3a* + 2y + z = 26

into the form

v

=

In other words, we need to eliminate the terms that are off the diagonal, those circled
in the following equations, and make the coefficients of the variables along the
diagonal equal to 1 :
*

+ (2v) + (3z) =

39

(^x)

+

34

@

+ @

3y + (2^) =
+

z =

26.

We can accomplish these goals step bystep, one variable at a time. In the past,
you may have simplified systems of equations by adding equations to one another
or subtracting them. In this system, we can eliminate the variable jc from the second
equation by subtracting the first equation from the second:
—►
—1st equation

■r + 2 v + 3z = 39
■r + 3v + 2 z = 34
3jc + 2 v 4- c = 26

x + 2x + 3c = 39
v - z = -S
3.v + 2y + z = 26

To eliminate the variable jc from the third equation, we subtract the first equation
from the third equation three times. We multiply the first equation by 3 to get
3jc + 6 y + 9z = 117

(3 x 1st equation)

and then subtract this result from the third equation:
* + 2v + 3z = 39
v - z = -5
3jc + 2v + z = 26

x + 2y + 3z = 39
v - z = -5
— 4v — 8 z = —91

—3 x 1 st equation

Similarly, we eliminate the variable y above and below the diagonal:
jc

+ 2y + 3z = 39
v - z = -5
— 4y — 8 z = -9 1

—2 x 2 nd equation

*

-1-4 x 2nd equation

+ 5z =
49
v z=
-5
- 12 z = —111

Before we eliminate the variable z above the diagonal, we make the coefficient of z
on the diagonal equal to 1 , by dividing the last equation by —1 2 :
x

+ 5 z=
49
>• c=
-5

- \2z = - 1 1 1

.r
—►
-M —1 2 )

+ 5 ; = 49
v - c = -5
z = 9.25

Finally, we eliminate the variable z above the diagonal:
+ 5c = 49
v - z = —5
c = 9.25

—5 x third equation
+ third equation
—►

*
v

= 2.75
= 4.25
c = 9.25

The yields of inferior, medium grade, and superior rice are 2.75,4.25, and 9.25 dou
per bundle, respectively.

I . I Introduction to Linear System s

3

By substituting these values, we can check that x = 2.75, y = 4.25, z = 9.25
is indeed the solution of the system:
2.75 + 2 x 4.25 + 3 x 9.25 =
2.75 + 3 x 4.25 + 2 x 9.25 =
3 x 2.75 + 2 x 4.25 +
9.25 =

39
34
26.

Happily, in linear algebra, youare almost always able to check your solutions.
It will help you if you get into the habit of checking now.
G e o m e tr ic I n te r p r e ta tio n
How can we interpret this result geometrically? Each of the three equations of the
system defines a plane in jc-y-z-space. The solution set of the system consists of
those points ( jc , y, z) that lie in all three planes (i.e., the intersection of the three
planes). Algebraically speaking, the solution set consists of those ordered triples of
numbers ( jc , y, z) that satisfy all three equations simultaneously. Our computations
show that the system has only one solution, ( j c , v , z ) = (2.75,4.25,9.25). This
means that the planes defined by the three equations intersect at the point ( jc , y , z) =
(2.75,4.25, 9.25), as shown in Figure 1.

While three different planes in space usually intersect at a point, they may have
a line in common (see Figure 2a) or may not have a common intersection at all, as
shown in Figure 2b. Therefore, a system of three equations with three unknowns

may have a unique solution, infinitely many solutions, or no solutions at all.

Figure 2(a) Three planes having a line in
common.

Figure 2(b) Three planes with no common
intersection.


4

CHAPTER I

Linear Equations

A S y s t e m wi th I nfi ni te ly M a n y S o l u t i o n s
Next, let’s consider a system of linear equations that has infinitely many solutions:
2x + Ay + 6z = 0
4x + 5_y + 6 z = 3
I x + 8 v + 9z = 6
We can solve this system using elimination as previously discussed. For sim­
plicity, we label the equations with Roman numerals.
2x + 4y -(- 6 z = 0
4x + 5y + 6z = 3
7x + Sy + 9z = 6
x + 2 y + 3z — 0
— 3 v — 6z = 3
— 6 y — 12 z = 6
x

— z = 2
v + 2z, = —1

0=

0

x + 2 j + 3z = 0
4x + 5>» + 6z = 3 - 4 ( 1 )
I x + 8 y + 9z = 6 - 7 (I)

-e-2

-H - 3 )

x

^ + 2y + 3z = 0 -2(11)
y + 2z = - 1
— 6 v — 12 z = 6 + 6 (II)
— z = 2
v + 2z = - \

After omitting the trivial equation 0 = 0, we are left with only two equations
with three unknowns. The solution set is the intersection of two nonparallel planes
in space (i.e., a line). This system has infinitely many solutions.
The two foregoing equations can be written as follows:
x =
z + 2
y = -2 z-\

We see that both x and y are determined by z. We can freely choose a value of z, an
arbitrary real number; then the two preceding equations give us the values of x and
y for this choice of z. For example,
• Choose z = 1. Then x = z + 2 = 3 and v = —2z — 1 = —3. The solution is
(jr,y ,z ) = (3, - 3 , 1).
• Choose z = 7. Then x = z + 2 = 9 and y = —2z — 1 = —15. The solution
is (x, y ,z ) = (9, - 1 5 , 7).
More generally, if we choose z = t, an arbitrary real number, we get x = t + 2
and y = —21 - 1. Therefore, the general solution is
( x , y \ z ) = ( f + 2 , —2 r — 1 , 0 = (2 , - 1 , 0 ) + f ( l,
This equation represents a line in space, as shown in Figure 3.

- 2, 1).


I . I Introduction to Linear System s

5

A S ystem without Solutions
In the following system, perform the eliminations yourself to obtain the result shown:
x + 2y + 3z = 0
4x + 5y + 6 z = 3
I x + 8 y + 9z = 0

x
—►

- z = 2
y + 2z = - \

0 = -6

Whatever values we choose for jc , >’, and z, the equation 0 = —6 cannot be
satisfied. This system is inconsistent; that is, it has no solutions.

EXERCISES 1.1
GOAL Set up and solve systems with as many as three
linear equations with three unknowns, and interpret the
equations and their solutions geometrically:
In Exercises 1 through 10, find all solutions of the linear
systems using elimination as discussed in this section. Then
check your solutions.
x + 2y = 1
2jc + 3y = 1

3.

2x 4 4y = 3
3jc + 6y = 2

2x 4 4y = 2
3jc 4 6y = 3

2x + 3y = 0
4jc 4 5y = 0

a + 2y 4 3z = 8
jc 4 3y 4 3z = 10
a + 2 y + 4z = 9

5.

4- 2y + 3z = 1
7. x + 3y 4 4z = 3
x + 4y + 5z = 4
+ 2y + 3z = 1
3x 4 2y + z = 1
7jc 4 2y — 3z = 1

8.

+ 3z = 1
2jc + 4 y 4 I z = 2
3x 4 7y 4 1 lz = 8

jc

jc

10 .

+

2y

/n Exercises 11 through 13, find all solutions o f the linear
systems. Represent your solutions graphically, as intersec­
tions o f lines in the x-y-plane.

11.

JC - 2y =
2
3jc + 5y = 17

13.

Jc -2 y = 3
2x - 4y = 8

.

12

x
2x

2y = 3
4v = 6

In Exercises 14 through 16, find all solutions o f the linear
systems. Describe your solutions in terms o f intersecting
planes. You need not sketch these planes.
x 4

4y +

z= 0
lz = 0
I x + 22y 4 13z = 1

14. 4x 4 13y +
x 4

z= 0
lz = 0
I x 4 22y 4 13z = 0

x-b y-

where a and b are arbitrary constants.

18. Find all solutions of the linear system

where a, b, and c are arbitrary constants.

x 4 2 v 4 3z = 0
4x 4 5 y 4 6z — 0
7jc 4 8y + lOz = 0

jc

x 4 2y = a
3jc 4 5y = b

x 4 2v 4 3z = a
x 4 3v 4 8z = b
x 4 2y 4 2z = c

4jc + 3y = 2

I x 4 5y = 3

1.

2.

17. Find all solutions of the linear system

19. Consider a two-commodity market. When the unit prices
of the products are P\ and Pi , the quantities demanded,
D\ and Dj, and the quantities supplied, S] and S2, are
given by
D 1 = 70 - 2P{ 4 Pi,
D2 = 105 4 P\ - P2,

5, = —14 4 3 P j,
4 2P2.
S2 = - 7

a. What is the relationship between the two commodi­
ties? Do they compete, as do Volvos and BMWs, or
do they complement one another, as do shirts and
ties?
b. Find the equilibrium prices (i.e., the prices for which
supply equals demand), for both products.
20. The Russian-born U.S. economist and Nobel laureate
Wassily Leontief (1906-1999) was interested in the fol­
lowing question: What output should each of the indus­
tries in an economy produce to satisfy the total demand
for all products? Here, we consider a very simple exam­

ple of input-output analysis, an economy with only two
industries, A and B. Assume that the consumer demand
for their products is, respectively, 1,000 and 780, in mil­
lions of dollars per year.

z= 0

15. 4jc - y 4 5z = 0
6a: 4 y 4 4z = 0

4y 4

16. 4x 4 13y 4

What outputs a and b (in millions of dollars per year)
should the two industries generate to satisfy the demand ?


6

CHAPTER I

Linear Equations

You may be tempted to say 1,000 and 780, respectively,
but things are not quite as simple as that. We have to take
into account the interindustry demand as well. Let us
say that industry A produces electricity. Of course, pro­
ducing almost any product will require electric power.
Suppose that industry B needs 10^worth of electricity

for each $1 of output B produces and that industry A
needs 20^worth of B ’s products for each $1 of output
A produces. Find the outputs a and b needed to satisfy
both consumer and interindustry demand.

25. Consider the linear system
x + y - z= - 2
—5y + 1 3 z =
jc —2y + 5z
=

3 jc

18 ,

k

where k is an arbitrary number.
a. For which value(s) of k does this system have one or
infinitely many solutions?
b. For each value of k you found in part a, how many
solutions does the system have?
c. Find all solutions for each value of k.

26. Consider the linear system
jf + y x + 2y +
x + y + (k2

21. Find the outputs a and b needed to satisfy the consumer
and interindustry demands given in the following figure

(see Exercise 20):

Z = 2
z=3 ,
- 5)z = k

where k is an arbitrary constant. For which value(s) of
k does this system have a unique solution? For which
value(s) of k does the system have infinitely many solu­
tions? For which value(s) of k is the system inconsistent?

27. Emile and Gertrude are brother and sister. Emile has
twice as many sisters as brothers, and Gertrude has just
as many brothers as sisters. How many children are there
in this family?

28. In a grid of wires, the temperature at exterior mesh points
is maintained at constant values (in °C) as shown in the
accompanying figure. When the grid is in thermal equi­
librium, the temperature T at each interior mesh point
is the average of the temperatures at the four adjacent
points. For example,

22. Consider the differential equation
d 2x
dx
■ jy — j ------ x = cos(f).
d t2
dt
This equation could describe a forced damped oscilla­

tor, as we will see in Chapter 9. We are told that the
differential equation has a solution of the form

+ T\ + 200 + 0
72 = ~ -------- 4---------- '
Find the temperatures T \ , 72, and 73 when the grid is
in thermal equilibrium.

jt(/) = a sin(r) + fccos(r).
Find a and b, and graph the solution.

23. Find all solutions of the system
I x — y = kx
—6x + 8y = ky ’
a. A. = 5

b. k = 10,

and

r
°r
c. k = 15.

24. On your next trip to Switzerland, you should take the
scenic boat ride from Rheinfall to Rheinau and back.
The trip downstream from Rheinfall to Rheinau takes
20 minutes, and the return trip takes 40 minutes; the
distance between Rheinfall and Rheinau along the river
is 8 kilometers. How fast does the boat travel (relative

to the water), and how fast does the river Rhein flow
in this area? You may assume both speeds to be constant
throughout the journey.

29. Find the polynomial of degree 2 [a polynomial of the
form f i t ) = a + bt + ct2] whose graph goes through
the points (1, - 1 ) , (2, 3), and (3, 13). Sketch the graph
of this polynomial.

30. Find a polynomial of degree < 2 [a polynomial of the
form f ( t ) = a + bt + ct2] whose graph goes through


I . I Introduction to Linear System s

the points (1, p), (2, q ), (3, r), where p , q , r are ar­
bitrary constants. Does such a polynomial exist for all
values of p> q , r?

7

41. Consider the linear system
* 4-

V= 1

31* Find all the polynomials / ( f ) of degree < 2 whose
graphs run through the points (1,3) and (2, 6), such that
/ '( l ) = 1 [where / '( f ) denotes the derivative].

32. Find all the polynomials / ( f ) of degree < 2 whose
graphs run through the points (1,1) and (2,0), such that

f? f i t ) dt = -1 .
33. Find all the polynomials / ( f ) of degree < 2 whose
graphs run through the points (1,1) and (3, 3), such that
/'( 2 ) = 1.

where f is a nonzero constant.
a. Determine the *- and ^-intercepts of the lines *4-y =
1 and * 4- (t/2 )y = f ; sketch these lines. For which
values of the constant f do these lines intersect? For
these values of f, the point of intersection (*, y) de­
pends on the choice of the constant f; that is, we
can consider * and y as functions of f. Draw rough
sketches of these functions.

34. Find all the polynomials / ( f ) of degree < 2 whose
graphs run through the points (1,1) and (3,3), such that
/'( 2 ) = 3.

1-

35. Find the function / (f) of the form / ( / ) = ae3t 4- be2t
such that /( 0 ) = 1 and /'( 0 ) = 4.

36. Find the function / (f) of the form / ( f ) = a cos(2f) 4fcsin(2f) such that / " ( f ) + 2 / '( f ) + 3 /( f ) = 17cos(2f).
(This is the kind of differential equation you might have
to solve when dealing with forced damped oscillators,
in physics or engineering.)

37. Find the circle that runs through the points (5,5), (4, 6),
and (6, 2). Write your equation in the form a 4- bx +
cy 4- x 2 + y 2 = 0. Find the center and radius of this
circle.

1 --

38. Find the ellipse centered at the origin that runs through
the points (1, 2), (2, 2), and (3, 1). Write your equation
in the form a x 2 + bxy + cy2 = 1.

39. Find all points (a , b, c) in space for which the system
x + 2 v + 3z = a
4x + 5y 4- 6z = b
I x + 8>> -I- 9z = c
has at least one solution.

40. Linear systems are particularly easy to solve when they
are in triangular form (i.e., all entries above or below the
diagonal are zero).
a. Solve the lower triangular system
x\
-3jci +
*1 +
—x\ +

= -3
X2
=14
2X2 + *3

=
9
8*2 ~ 5*3 + *4 = 33

by forward substitution, finding x\ first, then *2, then
*3, and finally *4.
b. Solve the upper triangular system
*1 + 2*2 — *3 + 4x4 = - 3
X2 + 3*3 -I- 7*4 = 5
*3 4- 2*4 = 2
*4 = 0

Explain briefly how you found these graphs.
Argue geometrically, without solving the system
algebraically.
b. Now solve the system algebraically. Verify that the
graphs you sketched in part (a) are compatible with
your algebraic solution.

42. Find a

system oflinear equations withthreeunknowns
whose solutions are the points on the line through
(1,1, l)a n d (3 ,5 ,0 ).

43. Find a

system oflinear equations
*, y, z whose solutions are
* = 6 4- 5f,

y = 4 4- 3f,

and

withthreeunknowns
z=

2 + f,

where f is an arbitrary constant.

44. Boris and Marina are shopping for chocolate bars. Boris
observes, “If I add half my money to yours, it will be
enough to buy two chocolate bars.” Marina naively asks,
“If I add half my money to yours, how many can we
buy?” Boris replies, “One chocolate bar.” How much
money did Boris have? (From Yuri Chernyak and Robert
Rose, The Chicken from Minsk, Basic Books, 1995.)

45. Here is another method to solve a system of linear equa­
tions: Solve one of the equations for one of the variables,
and substitute the result into the other equations. Repeat


CHAPTER I

Linear Equations

this process until you run out of variables or equations.

Consider the example discussed on page 2:
x -f 2 v + 3c = 39
x + 3>’ + 2c = 34 .
3.v + 2y + c = 26
We can solve the first equation for x:
x = 3 9 - 2y - 3c.
Then we substitute this equation into the other equations:
(39 —2v —3c) + 3v + 2c = 34
3(39 - 2y - 3z) + 2y + c = 26
We can simplify:
v - c = -5
—4y - 8c = -9 1
Now, v = c —5, so that —4 (c - 5) —8c = —91, or
-1 2 c = -1 1 1 .
We find that c = - - = 9.25. Then
12

v = c - 5 = 4.25.
and

Explain why this method is essentially the same as the
method discussed in this section; only the bookkeeping
is different.

46. A hermit eats only two kinds of food: brown rice and yo­
gurt. The rice contains 3 grams of protein and 30 grams
of carbohydrates per serving, while the yogurt contains
12 grams of protein and 20 grams of carbohydrates.
a. If the hermit wants to take in 60 grams of protein
and 300 grams of carbohydrates per day, how many

servings of each item should he consume?
b. If the hermit wants to take in P grams of protein
and C grams of carbohydrates per day, how many
servings of each item should he consume?

47. I have 32 bills in my wallet, in the denominations of
US$ 1, 5. and 10, worth $100 in total. How many do I
have of each denomination?

48. Some parking meters in Milan, Italy, accept coins in the
denominations of 20tf, 50tf, and €2. As an incentive pro­
gram, the city administrators offer a big reward (a brand
new Ferrari Testarossa) to any meter maid who brings
back exactly 1,000 coins worth exactly € 1,000 from the
daily rounds. What are the odds of this reward being
claimed anytime soon?

x = 39 - 2v - 3c = 2.75.

Matrices, Vectors, and Gauss-Jordan Elimination
When mathematicians in ancient China had to solve a system of simultaneous linear
equations such as 4

—6x —

21 v - 3z =
2v— z =

0
62

2x -

3 v + 8c =

32

3.v +

they took all the numbers involved inthis system and arranged them in a rectangular
pattern (Fang Cheng in Chinese), as follows :5
21
-6

-2

2 -3

0
1-1

, 62

8 , 32

All the information about this system is conveniently stored in this array of numbers.
The entries were represented by bamboo rods, as shown below; red and black
rods stand for positive and negative numbers, respectively. (Can you detect how this
4This example is taken from Chapter 8 of the Nine Chapters on the Mathematical Art: sec page 1. Our
source is George Gheverghese Joseph, The Crest o f the Peacock, Non-European Roots o f Mathematics,

2nd ed., Princeton University Press, 2000.

5Actually, the roles of rows and columns were reversed in the Chinese representation.


1.2 Matrices, Vectors, and Gauss-Jordan Elim ination

9

number system works?) The equations were then solved in a hands-on fashion, by
manipulating the rods. We leave it to the reader to find the solution.

|
II
III

Ill
T
II

III
1
m

—

111
Ell

Today, such a rectangular array of numbers,

3
-6

2

21

-3

-2

-1

-3

8

0
62
32

is called a matrix .6 Since this particular matrix has three rows and four columns, it
is called a 3 x 4 matrix (“three by four”).
The four columns of the matrix

/ /\ \
[3
The three rows of the matrix

21-3

- 2 -1
2 -3
8

—6

O'
62
32

Note that the first column of this matrix corresponds to the first variable of the
system, while the first row corresponds to the first equation.
It is customary to label the entries of a 3 x 4 matrix A with double subscripts
as follows:
aK
a 12
A =
U2\ 022
_fl3l
CIJ2

a 13 a u
Ql 3 #24
a .13 ^34

The first subscript refers to the row, and the second to the column: The entry atj is
located in the / th row and the yth column.
Two matrices A and B are equal if they are the same size and if corresponding
entries are equal:
= btj.

If the number of rows of a matrix A equals the number of columns (A is n x n),
then A is called a square matrix, and the entries a i j , a 22, .. •, ann form the (main)
diagonal of A. A square matrix A is called diagonal if all its entries above and below
the main diagonal are zero; that is,
= 0 whenever / / j . A square matrix A is
called upper triangular if all its entries below the main diagonal are zero: that is,
aij = 0 whenever i exceeds j . Lower triangular matrices are defined analogously.
A matrix whose entries are all zero is called a zero matrix and is denoted by 0
(regardless of its size). Consider the matrices
"2

0
0 3
0 0

C=
■5

K=
3

0

O'

4 0

0

2

1

O'
0
0

Alt appears that the term matrix was first used in this sense by the English mathematician
J. J. Sylvester, in 1850.


10

CHAPTER I

Linear Equations

The matrices # , C, D, and E are square, C is diagonal, C and D are upper triangular,
and C and £ are lower triangular.
Matrices with only one column or row are of particular interest.

Vectors and vector spaces
A matrix with only one column is called a column vector, or simply a vector. The
entries of a vector are called its components. The set of all column vectors with
n components is denoted by R n; we will refer to R n as a vector space.
A matrix with only one row is called a row vector.
In this text, the term vector refers to column vectors, unless otherwise stated.
The reason for our preference for column vectors will become apparent in the
next section.
Examples of vectors are

“1“

2
9

1

.1 .
a (column) vector in R 4, and
[1

5

5

3

7 ],

a row vector with five components. Note that the m columns of an n x m matrix are
vectors in R " .
In previous courses in mathematics or physics, you may have thought about
vectors from a more geometric point of view. (See the Appendix for a summary of
basic facts on vectors.) Let’s establish some conventions regarding the geometric
representation of vectors.

(*
Standard representation of vectors
The standard representation of a vector

X
v=

y

JC
V

in the Cartesian coordinate plane is as an arrow (a directed line segment) from
the origin to the point ( jc , y ), as shown in Figure 1.
The standard representation of a vector in R 3 is defined analogously.
In this text, we will consider the standard representation of vectors, unless
stated otherwise.

Figure I

(a +x, b +y)

Occasionally, it is helpful to translate (or shift) the vector in the plane (preserv­
ing its direction and length), so that it will connect some point (a , b ) to the point
{a + jc , b + y ), as shown in Figure 2.
When considering an infinite set of vectors, the arrow representation becomes
impractical. In this case, it is sensible to represent the vector u =

JC

simply by the

,y.

point

>t), the head of the standard arrow representation of v.
x
For example, the set of all vectors v =
(where jc is arbitrary) can be
x - \- \
represented as the line y = jc 4-1. For a few special values of jc we may still use the
arrow representation, as illustrated in Figure 3.
( jc ,


1.2 Matrices, Vectors, and Gauss-Jordan Elimination

11

In this course, it will often be helpful to think about a vector numerically, as a
list of numbers, which we will usually write in a column.
In our digital age, information is often transmitted and stored as a string of
numbers (i.e., as a vector). A section of 10 seconds of music on a CD is stored as
a vector with 440,000 components. A weather photograph taken by a satellite is
transmitted to Earth as a string of numbers.
Consider the system
2x +
2x +
Ax +

8 v + 4z = 2
5v+ z = 5 .

lOy — z = 1

Sometimes we are interested in the matrix
'2

8

4'

2

5

4

10

1
- 1.

which contains the coefficients of the system, called its coefficient matrix.
By contrast, the matrix
'2

8

4

2

2

5

1

5

4

10

-1

1

which displays all the numerical information contained in the system, is called its
augmented matrix. For the sake of clarity, we will often indicate the position of the
equal signs in the equations by a dotted line:
'2

8

4

2'

2

5

1

5

4

10

-1

1.

To solve the system, it is more efficient to perform the elimination on the aug­
mented matrix rather than on the equations themselves. Conceptually, the two ap­
proaches are equivalent, but working with the augmented matrix requires less writing


CHAPTER I

Linear Equations

yet is easier to read, with some practice. Instead of dividing an equation by a scalar,7
you can divide a row by a scalar. Instead of adding a multiple of an equation to
another equation, you can add a multiple of a row to another row.
As you perform elimination on the augmented matrix, you should always re­
member the linear system lurking behind the matrix. To illustrate this method, we
perform the elimination both on the augmented matrix and on the linear system it
represents:
'2

2
4

8
5
10

4
1
-1

2'
5
1.

2
1
-1

r
5
1.

+2

+2

2x +
2x +
4x +

Sy +
5y +
10? -

4z =
z
z =

2
5
1

x +
2x +
4x +

4y +
5y +
10? -

2z =
z =
z =

1
5

* +

4y +
-3 y -6 y -

1
2z =
3z = 3
9z = —3

4y +
y +
-6 y -

2z =
1
z = -1
9z = - 3

+ 6 (II)

— 2z = 5
V+
z = —1
-3 z = - 9

-H-3)

I
'1
2
4

4
5
10

1

0
0

4
-3
-6

r
2 i
3
-3
-9 : -3

1
0
0

4
1
-6

2
1

-9

"1

0

0
0

1

0

- 2 i: 5"
i i —1
-3 i - 9 .

"1

0

—2 |

r
-l
-3 .

5'

0

1

0

0

1 -1
i i 3.

'1
0
0

0
1
0

0
0
1

-2 (1 )
- 4 (I)

-(-3 )

- 4 (II)

x+

+ 6 (II)

x

-H-3)
+ 2 (III)
- (HI)

x

ir
-4
3.

v+

X

y

1

2z= 5
z = - I
z=
3

-2 (1 )
- 4 (I)

-H —3)

- 4 (II)

+ 2 (III)
- (HD

= n
= -4
z=
3

The solution is often represented as a vector:
x'
y =
_z_

' 11 '
-4
3.

Thus far we have been focusing on systems of 3 linear equations with 3 un­
knowns. Next we will develop a technique for solving systems of linear equations
of arbitrary size.
7In vector and matrix algebra, the term scalar is synonymous with (real) number.


1.2 Matrices, Vectors, and Gauss-Jordan Elimination

13

Here is an example of a system of three linear equations with five unknowns:
X2

X\ -

*3

+ 4* 5 = 2
- *5=2
*4
*5 = 3

We can proceed as in the example on page 4. We solve each equation for the leading
variable:
X[

= 2+

*3 =

X2

— 4*5

2

+

*5

*4 = 3

+

*5

.

Now we can freely choose values for thenonleading variables, *2 = / and *5 = r,
for example. The leading variables are then determined by these choices:
*1 = 2 + f —4r,

*3 = 2 + r,

*4 = 3 + r.

This system has infinitely many solutions; we can write the solutions in vector form as
"2

’■*1 "
*2

=

*3
*4

*5 _

2
3

+t
t

- 4 r~
+r
+r
r

Again, you can check this answer by substituting the solutions into the original
equations, for example, *3 —*5 = (2 + r) — r = 2 .
What makes this system so easy to solve? The following three properties are
responsible for the simplicity of the solution, with the second property playing a key
role:
• PI : The leading coefficient in each equation is 1. (The leading coefficient is
the coefficient of the leading variable.)
• P2: The leading variable in each equation does not appear in any of the other
equations. (For example, the leading variable *3 of the second equation appears
neither in the first nor in the third equation.)
• P3: The leading variables appear in the “natural order,” with increasing indices
as we go down the system (* 1, * 3 , *4 as opposed to * 3 , * 1, * 4 , for example).
Whenever we encounter a linear system with these three properties, we can solve
for the leading variables and then choose arbitrary values for the other, nonleading
variables, as we did above and on page 4.
Now we are ready to tackle the case of an arbitrary system of linear equations.
We will illustrate our approach by means of an example:
2 *i +

4*2 —2*3 + 2*4 + 4*5 = 2

*1 +

2*2 — *3 + 2*4
= 4
6*2 —2*3 + *4 + 9*5 = 1

3*i +
5*i +

10*2 —4*3 + 5*4 + 9*5 = 9

We wish to reduce this system to a system satisfying the three properties (PI, P2,
and P3); this reduced system will then be easy to solve.
We will proceed from equation to equation, from top to bottom. The leading
variable in the first equation is * 1, with leading coefficient 2. To satisfy property P I,
we will divide this equation by 2. To satisfy property P2 for the variable * 1, we will
then subtract suitable multiples of the first equation from the other three equations