Tom M. Apostol - Calculus, Volume II_ Multi-Variable Calculus and Linear Algebra, with Applications to Differential Equations and Probability. Vol. 2-John Wiley & Sons (1969)

Tom IN. Apostol

CALCULUS
VOLUME II

Mlul ti Variable Calculus and Linear
Algebra, with Applications to
DifFeren tial Equations and Probability
SECOND EDITION

John Wiley & Sons
New York London Sydney Toronto


CONSULTING

EDITOR

George Springer, Indiana University

COPYRIGHT 0 1969 BY XEROX CORPORATION .
All rights reserved. No part of the material covered by this copyright
may be produced in any form, or by any means of reproduction.
Previous edition copyright 0 1962 by Xerox Corporation.
Librar of Congress Catalog Card Number: 67-14605
ISBN 0 471 00007 8 Printed in the United States of America.

1098765432

www.pdfgrip.com

To
Jane and Stephen

www.pdfgrip.com


www.pdfgrip.com


PREFACE

This book is a continuation of the author’s Calculus, Volume I, Second Edition. The
present volume has been written with the same underlying philosophy that prevailed in the
first. Sound training in technique is combined with a strong theoretical development.
Every effort has been made to convey the spirit of modern mathematics without undue
emphasis on formalization. As in Volume I, historical remarks are included to give the
student a sense of participation in the evolution of ideas.
The second volume is divided into three parts, entitled Linear Analysis, Nonlinear
Ana!ysis, and Special Topics. The last two chapters of Volume I have been repeated as the
first two chapters of Volume II so that all the material on linear algebra will be complete
in one volume.
Part 1 contains an introduction to linear algebra, including linear transformations,
matrices, determinants, eigenvalues, and quadratic forms. Applications are given to
analysis, in particular to the study of linear differential equations. Systems of differential
equations are treated with the help of matrix calculus. Existence and uniqueness theorems
are proved by Picard’s method of successive approximations, which is also cast in the
language of contraction operators.
Part 2 discusses the calculus of functions of several variables. Differential calculus is
unified and simplified with the aid of linear algebra. It includes chain rules for scalar and

vector fields, and applications to partial differential equations and extremum problems.
Integral calculus includes line integrals, multiple integrals, and surface integrals, with
applications to vector analysis. Here the treatment is along more or less classical lines and
does not include a formal development of differential forms.
The special topics treated in Part 3 are Probability and Numerical Analysis. The material
on probability is divided into two chapters, one dealing with finite or countably infinite
sample spaces; the other with uncountable sample spaces, random variables, and distribution functions. The use of the calculus is illustrated in the study of both one- and
two-dimensional random variables.
The last chapter contains an introduction to numerical analysis, the chief emphasis
being on different kinds of polynomial approximation. Here again the ideas are unified
by the notation and terminology of linear algebra. The book concludes with a treatment of
approximate integration formulas, such as Simpson’s rule, and a discussion of Euler’s
summation formula.

www.pdfgrip.com


\‘I11

Preface

There is ample material in this volume for a full year’s course meeting three or four times
per week. It presupposes a knowledge of one-variable calculus as covered in most first-year
calculus courses. The author has taught this material in a course with two lectures and two
recitation periods per week, allowing about ten weeks for each part and omitting the
starred sections.
This second volume has been planned so that many chapters can be omitted for a variety
of shorter courses. For example, the last chapter of each part can be skipped without
disrupting the continuity of the presentation. Part 1 by itself provides material for a combined course in linear algebra and ordinary differential equations. The individual instructor
can choose topics to suit his needs and preferences by consulting the diagram on the next

page which shows the logical interdependence of the chapters.
Once again I acknowledge with pleasure the assistance of many friends and colleagues.
In preparing the second edition I received valuable help from Professors Herbert S.
Zuckerman of the University of Washington, and Basil Gordon of the University of
California, Los Angeles, each of whom suggested a number of improvements. Thanks are
also due to the staff of Blaisdell Publishing Company for their assistance and cooperation.
As before, it gives me special pleasure to express my gratitude to my wife for the many
ways in which she has contributed. In grateful acknowledgement I happily dedicate this
book to her.
T. M. A.
Pasadena, California
September 16, 1968

www.pdfgrip.com


ix

Logical Interdependence of the Chapters

1
LINEAR
SPACES

INTRODUCTION
TO NUMERICAL
ANALYSIS

TRANSFORMATIONS
AND MATRICES

DETERMINANTS

I

6
LINEAR
I
\
DIFFERENTIAL
EQUATIONS

,

I

DIFFERENTIAL
CALCULUS OF
SCALAR AND
VECTOR FIELDS

4
I

7

SYSTEMS OF
DIFFERENTIAL

EIGENVALUES

H

10

I I

1

1

13

LINE

SET FUNCTIONS

INTEGRALS

AND ELEMENTARY
PROBABILITY

AND
EIGENVECTORS

I-

EQUATIONS
OPERATORS ACTING
ON EUCLIDEAN

SPACES

9
’ APPLICATIONS OF
DIFFERENTIAL

1 P R OBABILITI E S 1

1

INTEGRALS

CALCULUS

www.pdfgrip.com


www.pdfgrip.com


CONTENTS

PART 1. LINEAR ANALYSIS
1. LINEAR SPACES
1.1 Introduction
1.2 The definition of a linear space
1.3 Examples of linear spaces
1.4
1.5
1.6

1.7
1.8
1.9

Elementary consequences of the axioms
Exercises
Subspaces of a linear space
Dependent and independent sets in a linear space
Bases and dimension
Components

1.10 Exercises
1.11 Inner products, Euclidean spaces. Norms
1.12
1.13
1.14
1.15
1.16

Orthogonality in a Euclidean space
Exercises
Construction of orthogonal sets. The Gram-Schmidt process
Orthogonal complements. Projections
Best approximation of elements in a Euclidean space by elements in a finitedimensional subspace
1.17 Exercises

3
3
4
6

7
8
9
12
13
13
14
18
20
22
26
28
30

2. LINEAR TRANSFORMATIONS AND MATRICES
31
32
34

2.1 Linear transformations
2.2 Null space and range
2.3 Nullity and rank

xi

www.pdfgrip.com


Contents

xii

2.4 Exercises
2.5 Algebraic operations on linear transformations
2.6 Inverses
2.7 One-to-one linear transformations
2.8 Exercises
2.9 Linear transformations with prescribed values
2.10 Matrix representations of linear transformations
2.11 Construction of a matrix representation in diagonal form
2.12 Exercises
2.13 Linear spaces of matrices
2.14 Tsomorphism between linear transformations and matrices
2.15
2.16
2.17
2.18
2.19

Multiplication of matrices
Exercises
Systems of linear equations
Computation techniques
Inverses of square matrices

2.20 Exercises
2.21 Miscellaneous exercises on matrices

35
36

38
41
42
44
45
48
50
51
52
54
57
58
61
65
67
68

3. DETERMINANTS
3.1
3.2
3.3
3.4
3.5

Introduction
Motivation for the choice of axioms for a determinant function
A set of axioms for a determinant function
Computation of determinants
The uniqueness theorem

3.6 Exercises
3.7 The product formula for determinants
3.8 The determinant of the inverse of a nonsingular matrix
3.9 Determinants and independence of vectors
3.10 The determinant of a block-diagonal matrix
3.11 Exercises
3.12 Expansion formulas for determinants. Minors and cofactors
3.13 Existence of the determinant function
3.14 The determinant of a transpose
3.15 The cofactor matrix
3.16 Cramer’s rule
3.17 Exercises

www.pdfgrip.com

71
72
73
76
79
79
81
83
83
84
85
86
90
91
92

93
94


Contents

...

XIII

4. EIGENVALUES AND EIGENVECTORS
4.1 Linear transformations with diagonal matrix representations
4.2 Eigenvectors and eigenvalues of a linear transformation
4.3 Linear independence of eigenvectors corresponding to distinct eigenvalues
4.4 Exercises *
4.5 The finite-dimensional case. Characteristic polynomials
4.6 Calculation of eigenvalues and eigenvectors in the finite-dimensional case
4.7 Trace of a matrix
4.8 Exercises
4.9 Matrices representing the same linear transformation. Similar matrices
4.10 Exercises

96
97
100
101
102
103
106
107

108
112

5. EIGENVALUES OF OPERATORS ACTING ON
EUCLIDEAN SPACES
5.1 Eigenvalues and inner products
5.2 Hermitian and skew-Hermitian transformations
5.3 Eigenvalues and eigenvectors of Hermitian and skew-Hermitian operators
5.4 Orthogonality of eigenvectors corresponding to distinct eigenvalues
5.5 Exercises
5.6 Existence of an orthonormal set of eigenvectors for Hermitian and
skew-Hermitian operators acting on finite-dimensional spaces
5.7 Matrix representations for Hermitian and skew-Hermitian operators
5.8 Hermitian and skew-Hermitian matrices. The adjoint of a matrix
5.9 Diagonalization of a Hermitian or skew-Hermitian matrix
5.10 Unitary matrices. Orthogonal matrices
5.11 Exercises
5.12 Quadratic forms
5.13 Reduction of a real quadratic form to a diagonal form
5.14 Applications to analytic geometry
5.15 Exercises
A5.16 Eigenvalues of a symmetric transformation obtained as values of its
quadratic form
k5.17 Extremal properties of eigenvalues of a symmetric transformation
k5.18 The finite-dimensional case
5.19 Unitary transformations
5.20 Exercises

www.pdfgrip.com

114
115
117
117
118
120
121
122
122
123
124
126
128
130
134
135
136
137
138
141


Contents

xiv

6. LINEAR DIFFERENTIAL EQUATIONS
6.1 Historical introduction
6.2 Review of results concerning linear equations of first and second orders
6.3 Exercises

6.4 Linear differential equations of order n
6.5 The existence-uniqueness theorem
6.6 The dimension of the solution space of a homogeneous linear equation

142
143
144
145
147
147
148

6.7 The algebra of constant-coefficient operators
6.8 Determination of a basis of solutions for linear equations with constant
coefficients by factorization of operators
6.9 Exercises
6.10 The relation between the homogeneous and nonhomogeneous equations

150
154
156

6.11 Determination of a particular solution of the nonhomogeneous equation.
The method of variation of parameters

157

6.12 Nonsingularity of the Wronskian matrix of n independent solutions of a
homogeneous linear equation
6.13 Special methods for determining a particular solution of the nonhomogeneous

equation. Reduction to a system of first-order linear equations
6.14 The annihilator method for determining a particular solution of the
nonhomogeneous equation
6.15 Exercises
6.16 Miscellaneous exercises on linear differential equations
6.17 Linear equations of second order with analytic coefficients
6.18
6.19
6.20
6.21
6.22
6.23
6.24

The Legendre equation
The Legendre polynomials
Rodrigues’ formula for the Legendre polynomials
Exercises
The method of Frobenius
The Bessel equation
Exercises

161
163
163
166
167
169
171
174

176
177
180
182
188

7. SYSTEMS OF DIFFERENTIAL EQUATIONS
7.1 Introduction
7.2 Calculus of matrix functions
7.3 Infinite series of matrices. Norms of matrices
7.4 Exercises
7.5 The exponential matrix

www.pdfgrip.com

191
193
194
195
197


Contents

7.6
7.7
7.8
7.9
7.10
7.11

7.12
7.13
7.14
7.15
7.16
7.17
7.18
7.19
7.20
7.21
7.22
7.23
7.24
*7.25
*7.26
k7.27
k7.28
A7.29

The differential equation satisfied by etA
Uniqueness theorem for the matrix differential equation F’(t) = AF(t)
The law of exponents for exponential matrices
Existence and uniqueness theorems for homogeneous linear systems
with constant coefficients
The problem of calculating etA4
The Cayley-Hamilton theorem
Exercises
Putzer’s method for calculating eta
Alternate methods for calculating et-’ in special cases
Exercises

Nonhomogeneous linear systems with constant coefficients
Exercises
The general linear system Y’(t) = P(t) Y(t) + Q(t)
A power-series method for solving homogeneous linear systems
Exercises
Proof of the existence theorem by the method of successive approximations
The method of successive approximations applied to first-order nonlinear systems
Proof of an existence-uniqueness theorem for first-order nonlinear systems
Exercises
Successive approximations and fixed points of operators
Normed linear spaces
Contraction operators
Fixed-point theorem for contraction operators
Applications of the fixed-point theorem

xv

197
198
199
200
201
203
205
205
208
211
213
215
217

220
221
222
227
229
230
232
233
234
235
237

PART 2. NONLINEAR ANALYSIS
8. DIFFERENTIAL CALCULUS OF SCALAR AND
VECTOR FIELDS
8.1
8.2
8.3
8.4
8.5
8.6
8.7
8.8
8.9

Functions from R” to R”. Scalar and vector fields
Open balls and open sets
Exercises
Limits and continuity
Exercises

The derivative of a scalar field with respect to a vector
Directional derivatives and partial derivatives
Partial derivatives of higher order
Exercises

www.pdfgrip.com

243
244
245
247
251
252
254
255
255


Contents

xvi
8.10
8.11
8.12
8.13
8.14
8.15

Directional derivatives and continuity
The total derivative

The gradient of a scalar field
A sufficient condition for differentiability
Exercises
A chain rule for derivatives of scalar fields

8.16 Applications to geometry. Level sets. Tangent planes
8.17 Exercises
8.18 Derivatives of vector fields
8. I9 Differentiability implies continuity
8.20 The chain rule for derivatives of vector fields
8.21 Matrix form of the chain rule
8.22 Exercises
A8.23 Sufficient conditions for the equality of mixed partial derivatives
8.24 Miscellaneous exercises

251
258
259
261
262
263
266
268
269
271
272
273
275
277
281

9. APPLICATIONS OF THE DIFFERENTIAL CALCULUS
9.1 Partial differential equations
9.2 A first-order partial differential equation with constant coefficients
9.3 Exercises
9.4 The one-dimensional wave equation
9.5 Exercises
9.6 Derivatives of functions defined implicitly
9.7 Worked examples
9.8 Exercises
9.9 Maxima, minima, and saddle points
9. IO Second-order Taylor formula for scalar fields

283
284
286
288
292
294
298
302
303
308

9.11 The nature of a stationary point determined by the eigenvalues of the Hessian
310
matrix
312
9.12 Second-derivative test for extrema of functions of two variables
313

9.13 Exercises
314
9.14 Extrema with constraints. Lagrange’s multipliers
318
9. I5 Exercises
319
9.16 The extreme-value theorem for continuous scalar fields
321
9.17 The small-span theorem for continuous scalar fields (uniform continuity)

10. LINE INTEGRALS
323
323

10.1 Introduction
10.2 Paths and line integrals

www.pdfgrip.com


Contents

xvii

10.3

Other notations for line integrals

324

10.4
10.5

Basic properties of line integrals
Exercises

10.6
10.7
10.8

The concept of work as a line integral
Line integrals with respect to arc length
Further applications of line integrals

10.9
10.10
10.11
10.12
10.13

Exercises
Open connected sets. Independence of the path
The second fundamental theorem of calculus for line integrals
Applications to mechanics
Exercises

326
328
328
329

330
331
332
333
335
336
331
339
340
342
345
346
349
350

10.14 The first fundamental theorem of calculus for line integrals
10.15 Necessary and sufficient conditions for a vector field to be a gradient
10.16 Necessary conditions for a vector field to be a gradient
10.17 Special methods for constructing potential functions
10.18 Exercises
10.19 Applications to exact differential equations of first order
10.20 Exercises
10.21 Potential functions on convex sets

11. MULTIPLE INTEGRALS
353
353
355

11 .l

11.2

Introduction
Partitions of rectangles. Step functions

11.3
11.4

The double integral of a step function
The definition of the double integral of a function defined and bounded on a
357
rectangle
357
Upper and lower double integrals
358
Evaluation of a double integral by repeated one-dimensional integration
359
Geometric interpretation of the double integral as a volume
360
Worked examples
362
Exercises
363
Integrability of continuous functions

11.5

11.6
11.7
11.8

11.9
11.10
11 .I 1 Integrability of bounded functions with discontinuities
11.12 Double integrals extended over more general regions
11.13 Applications to area and volume
11.14 Worked examples
11. I5 Exercises
11.16 Further applications of double integrals
11.17 Two theorems of Pappus
11.18 Exercises

www.pdfgrip.com

364
365
368
369
371
373
376
377


.. .

Contents

xv111

11.19 Green’s theorem in the plane

11.20 Some applications of Green’s theorem
11.21 A necessary and sufficient condition for a two-dimensional vector field to be a
gradient
11.22 Exercises
kll.23 Green’s theorem for multiply connected regions
*11.24 The winding number
*I 1.25 Exercises
11.26 Change of variables in a double integral
11.27 Special cases of the transformation formula
11.28 Exercises
11.29 Proof of the transformation formula in a special case
11.30
11.31
11.32
11.33
11.34

Proof of the transformation formula in the general case
Extensions to higher dimensions
Change of variables in an n-fold integral
Worked examples
Exercises

378
382
383
385
387
389
391

392
396
399
401
403
405
407
409
413

12. SURFACE INTEGRALS
12.1
12.2
12.3
12.4
12.5
12.6

Parametric representation of a surface
The fundamental vector product
The fundamental vector product as a normal to the surface
Exercises
Area of a parametric surface
Exercises

12.7 Surface integrals
12.8 Change of parametric representation
12.9 Other notations for surface integrals
12.10 Exercises
12.11 The theorem of Stokes

12.12 The curl and divergence of a vector field
12.13 Exercises
12.14 Further properties of the curl and divergence
12.15 Exercises
*12.16
*12.17
12.18
12.19
12.20
12.21

Reconstruction of a vector field from its curl
Exercises
Extensions of Stokes’ theorem
The divergence theorem (Gauss’ theorem:)
Applications of the divergence theorem
Exercises

www.pdfgrip.com

417
420
423
424
424
429
430
432
434
436

438
440
442
443
447
448
452
453
457
460
462


Contents

xix

PART 3. SPECIAL TOPICS
13. SET FUNCTIONS AND ELEMENTARY PROBABILITY
13.1
13.2
13.3
13.4

Historical introduction
Finitely additive set functions
Finitely additive measures
Exercises
13.5 The definition of probability for finite sample spaces
13.6 Special terminology peculiar to probability theory

13.7 Exercises
13.8 Worked examples
13.9 Exercises
13.10 Some basic principles of combinatorial analysis
13.11 Exercises
13.12 Conditional probability
13.13 Independence
13.14 Exercises

479
481
485
486
488
490
492

13.15 Compound experiments
13.16 Bernoulli trials
13.17
13.18
13.19
13.20
13.21

469
470
471
472
473

475
477
477

The most probable number of successes in n Bernoulli trials
Exercises
Countable and uncountable sets
Exercises
The definition of probability for countably infinite sample spaces

13.22 Exercises
13.23 Miscellaneous exercises on probability

495
497
499
501
504
506
507
507

14. CALCULUS OF PROBABILITIES
14.1
14.2
14.3
14.4
14.5
14.6
14.7

14.8
14.9

The definition of probability for uncountable sample spaces
Countability of the set of points with positive probability
Random variables
Exercises
Distribution functions
Discontinuities of distribution functions
Discrete distributions. Probability mass functions
Exercises
Continuous distributions. Density functions

www.pdfgrip.com

510
511
512
513
514
517
520
523
525


Contents

xx

14.10 Uniform distribution over an interval
14.11 Cauchy’s distribution
14.12 Exercises
14.13 Exponential distributions
14.14 Normal distributions

526
530

14.15 Remarks on more general distributions
14.16 Exercises
14.17 Distributions of functions of random variables

539
540
541
542
543
545
546
548
550
553
556
559
560

532
533
535

14.18 Exercises
14.19 Distributions of two-dimensional random variables
14.20 Two-dimensional discrete distributions
14.21

Two-dimensional

continuous

distributions.

Density

functions

14.22 Exercises
14.23 Distributions of functions of two random variables
14.24 Exercises
14.25 Expectation and variance
14.26 Expectation of a function of a random variable
14.27 Exercises
14.28 Chebyshev’s inequality
14.29 Laws of large numbers
14.30 The central limit theorem of the calculus of probabilities
14.3 1 Exercises
Suggested References

562
564

566
568
569

15. INTRODUCTION TO NUMERICAL ANALYSIS
15.1
15.2
15.3
15.4
15.5
15.6

Historical introduction
Approximations by polynomials
Polynomial approximation and normed linear spaces
Fundamental problems in polynomial approximation
Exercises
Interpolating polynomials

15.7 Equally spaced interpolation points
15.8 Error analysis in polynomial interpolation
15.9 Exercises
15.10 Newton’s interpolation formula
15.11 Equally spaced interpolation points. The forward difference operator
15.12 Factorial polynomials
15.13 Exercises
15.14 A minimum problem relative to the max norm

www.pdfgrip.com

571
572
574
575
577
579
582
583
585
588
590
592
593
595


Contents
15.15 Chebyshev polynomials
15.16 A minimal property of Chebyshev polynomials
15.17 Application to the error formula for interpolation
15.18 Exercises
15.19 Approximate integration. The trapezoidal rule
15.20
15.21
15.22
15.23

Simpson’s rule
Exercises
The Euler summation formula

Exercises
Suggested References
Answers to exercises

xxi
596
598
599
600
602
605
610
613
618
621
622
665

Index

www.pdfgrip.com


www.pdfgrip.com


Calculus

www.pdfgrip.com

www.pdfgrip.com


PART 1
LINEAR ANALYSIS

www.pdfgrip.com


www.pdfgrip.com


1
LINEAR SPACES

1.1

Introduction

Throughout mathematics we encounter many examples of mathematical objects that
can be added to each other and multiplied by real numbers. First of all, the real numbers
themselves are such objects. Other examples are real-valued functions, the complex
numbers, infinite series, vectors in n-space, and vector-valued functions. In this chapter we
discuss a general mathematical concept, called a linear space, which includes all these
examples and many others as special cases.
Briefly, a linear space is a set of elements of any kind on which certain operations (called
addition and multiplication by numbers) can be performed. In defining a linear space, we
do not specify the nature of the elements nor do we tell how the operations are to be
performed on them. Instead, we require that the operations have certain properties which

we take as axioms for a linear space. We turn now to a detailed description of these axioms.
1.2 The definition of a linear space
Let V denote a nonempty set of objects, called elements. The set V is called a linear
space if it satisfies the following ten axioms which we list in three groups.
Closure axioms
AXIOM 1.
CLOSURE
U N DER
ADDITION .
For every pair of elements x and y in V there
corresponds a unique element in V called the sum of x and y, denoted by x + y .
AXIOM 2. CLOSURE UNDER MULTIPLICATION BY REAL NUMBERS. For every x in V and
every real number a there corresponds an element in V called the product of a and x, denoted
by ax.

Axioms for addition
AXIOM
AXIOM

3.

COMMUTATIVE

LAW.

4. ASSOCIATIVELAW.

For all x and y in V, we have x + y = y + x.
Forallx,y,andzinV,wehave(x+y)

www.pdfgrip.com

+ z =x +(y+z).
?