Solving ordinary differential equations i nonstiff problems ( pdfdrive com )

Springer Series in
Computational
Mathematics
Editorial Board
R. Bank
R.L. Graham
J. Stoer
R. Varga
H. Yserentant

8


E. Hairer
S. P. Nørsett
G. Wanner

Solving Ordinary
Differential Equations I
Nonstiff Problems

Second Revised Edition
With 135 Figures

123


Ernst Hairer
Gerhard Wanner
Université de Genève
Section de Mathématiques

2–4 rue du Lièvre
1211 Genève 4
Switzerland



Syvert P. Nørsett
Norwegian University of Science
and Technology (NTNU)
Department of Mathematical Sciences
7491 Trondheim
Norway


Corrected 3rd printing 2008
ISBN 978-3-540-56670-0

e-ISBN 978-3-540-78862-1

DOI 10.1007/978-3-540-78862-1
Springer Series in Computational Mathematics ISSN 0179-3632
Library of Congress Control Number: 93007847
Mathematics Subject Classification (2000): 65Lxx, 34A50
© 1993, 1987 Springer-Verlag Berlin Heidelberg
This work is subject to copyright. All rights are reserved, whether the whole or part of the
material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data
banks. Duplication of this publication or parts thereof is permitted only under the provisions
of the German Copyright Law of September 9, 1965, in its current version, and permission
for use must always be obtained from Springer. Violations are liable to prosecution under the

German Copyright Law.
The use of general descriptive names, registered names, trademarks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from
the relevant protective laws and regulations and therefore free for general use.
Cover design: WMX Design GmbH, Heidelberg
Typesetting: by the authors
Production: LE-TEX Jelonek, Schmidt & Vöckler GbR, Leipzig
Printed on acid-free paper
98765 4321
springer.com


This edition is dedicated to
Professor John Butcher
on the occasion of his 60th birthday

His unforgettable lectures on Runge-Kutta methods, given in June
1970 at the University of Innsbruck, introduced us to this subject
which, since then, we have never ceased to love and to develop with all
our humble abilities.


From the Preface to the First Edition
So far as I remember, I have never seen an Author’s Preface
which had any purpose but one — to furnish reasons for the
publication of the Book.
(Mark Twain)
Gauss’ dictum, “when a building is completed no one should
be able to see any trace of the scaffolding,” is often used by
mathematicians as an excuse for neglecting the motivation

behind their own work and the history of their field. Fortunately, the opposite sentiment is gaining strength, and numerous asides in this Essay show to which side go my sympathies.
(B.B. Mandelbrot 1982)
This gives us a good occasion to work out most of the book
until the next year.
(the
Authors in a letter, dated Oct. 29, 1980, to Springer-Verlag)

There are two volumes, one on non-stiff equations, . . ., the second
on stiff equations, . . . . The first volume has three chapters, one on
classical mathematical theory, one on Runge-Kutta and extrapolation
methods, and one on multistep methods. There is an Appendix containing some Fortran codes which we have written for our numerical
examples.
Each chapter is divided into sections. Numbers of formulas, theorems, tables and figures are consecutive in each section and indicate,
in addition, the section number, but not the chapter number. Cross references to other chapters are rare and are stated explicitly. . . . References to the Bibliography are by “Author” plus “year” in parentheses.
The Bibliography makes no attempt at being complete; we have listed
mainly the papers which are discussed in the text.
Finally, we want to thank all those who have helped and encouraged us to prepare this book. The marvellous “Minisymposium”
which G. Dahlquist organized in Stockholm in 1979 gave us the first
impulse for writing this book. J. Steinig and Chr. Lubich have read the
whole manuscript very carefully and have made extremely valuable
mathematical and linguistical suggestions. We also thank J.P. Eckmann for his troff software with the help of which the whole manuscript has been printed. For preliminary versions we had used textprocessing programs written by R. Menk. Thanks also to the staff of the
Geneva computing center for their help. All computer plots have been
done on their beautiful HP plotter. Last but not least, we would like
to acknowledge the agreable collaboration with the planning and production group of Springer-Verlag.
October 29, 1986

The Authors


VIII

Preface

Preface to the Second Edition
The preparation of the second edition has presented a welcome opportunity to improve the first edition by rewriting many sections and by
eliminating errors and misprints. In particular we have included new
material on
– Hamiltonian systems (I.14) and symplectic Runge-Kutta methods
(II.16);
– dense output for Runge-Kutta (II.6) and extrapolation methods
(II.9);
– a new Dormand & Prince method of order 8 with dense output
(II.5);
– parallel Runge-Kutta methods (II.11);
– numerical tests for first- and second order systems (II.10 and III.7).
Our sincere thanks go to many persons who have helped us with our
work:
– all readers who kindly drew our attention to several errors and misprints in the first edition;
– those who read preliminary versions of the new parts of this edition for their invaluable suggestions: D.J. Higham, L. Jay, P. Kaps,
Chr. Lubich, B. Moesli, A. Ostermann, D. Pfenniger, P.J. Prince,
and J.M. Sanz-Serna.
– our colleague J. Steinig, who read the entire manuscript, for his numerous mathematical suggestions and corrections of English (and
Latin!) grammar;
– our colleague J.P. Eckmann for his great skill in manipulating
Apollo workstations, font tables, and the like;
– the staff of the Geneva computing center and of the mathematics
library for their constant help;
– the planning and production group of Springer-Verlag for numerous suggestions on presentation and style.
This second edition now also benefits, as did Volume II, from the marvels of TEXnology. All figures have been recomputed and printed,
together with the text, in Postscript. Nearly all computations and

text processings were done on the Apollo DN4000 workstation of the
Mathematics Department of the University of Geneva; for some longtime and high-precision runs we used a VAX 8700 computer and a
Sun IPX workstation.
November 29, 1992

The Authors


Contents

Chapter I. Classical Mathematical Theory
I.1
I.2

I.3

I.4

I.5

I.6

I.7

I.8

Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The Oldest Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . .
Newton . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Leibniz and the Bernoulli Brothers . . . . . . . . . . . . . . . . . . . . . . . . .

Variational Calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Clairaut . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Elementary Integration Methods . . . . . . . . . . . . . . . . . . . . . . . . . .
First Order Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Second Order Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Linear Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Equations with Constant Coefficients . . . . . . . . . . . . . . . . . . . . . . .
Variation of Constants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Equations with Weak Singularities . . . . . . . . . . . . . . . . . . . . . . . .
Linear Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Nonlinear Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Systems of Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The Vibrating String and Propagation of Sound . . . . . . . . . . . . . .
Fourier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Lagrangian Mechanics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Hamiltonian Mechanics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
A General Existence Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Convergence of Euler’s Method . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Existence Theorem of Peano . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Existence Theory using Iteration Methods and Taylor Series
Picard-Lindelăof Iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Taylor Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Recursive Computation of Taylor Coefficients . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2
4
4
6
7
9
10
12
12
13
14
16
16
18
19
20
20
23
24
26
26
29
30
32
34
35
35
41
43

44
45
46
47
49


X

Contents

I.9

Existence Theory for Systems of Equations . . . . . . . . . . . . . . . .
Vector Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Subordinate Matrix Norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

51
52
53
55

I.10

Differential Inequalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The Fundamental Theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Estimates Using One-Sided Lipschitz Conditions . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

56
56
57
60
62

I.11

Systems of Linear Differential Equations . . . . . . . . . . . . . . . . . .
Resolvent and Wronskian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Inhomogeneous Linear Equations . . . . . . . . . . . . . . . . . . . . . . . . . .
The Abel-Liouville-Jacobi-Ostrogradskii Identity . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

64
65
66
66
67

I.12

Systems with Constant Coefficients . . . . . . . . . . . . . . . . . . . . . . . .
Linearization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Diagonalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The Schur Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Numerical Computations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The Jordan Canonical Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Geometric Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

69
69
69
70
72
73
77
78

I.13

Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The Routh-Hurwitz Criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Computational Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Liapunov Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Stability of Nonlinear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Stability of Non-Autonomous Systems . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

80
80
81
85
86
87
88
89

I.14

Derivatives with Respect to Parameters and Initial Values . . . 92
The Derivative with Respect to a Parameter . . . . . . . . . . . . . . . . . . 93
Derivatives with Respect to Initial Values . . . . . . . . . . . . . . . . . . . . 95
The Nonlinear Variation-of-Constants Formula . . . . . . . . . . . . . . . 96
Flows and Volume-Preserving Flows . . . . . . . . . . . . . . . . . . . . . . . . 97
Canonical Equations and Symplectic Mappings . . . . . . . . . . . . . . 100
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

I.15

Boundary Value and Eigenvalue Problems . . . . . . . . . . . . . . . . .
Boundary Value Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Sturm-Liouville Eigenvalue Problems . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

105
105
107
110

I.16

Periodic Solutions, Limit Cycles, Strange Attractors . . . . . . . .
Van der Pol’s Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Chemical Reactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Limit Cycles in Higher Dimensions, Hopf Bifurcation . . . . . . . .
Strange Attractors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

The Ups and Downs of the Lorenz Model . . . . . . . . . . . . . . . . . . .
Feigenbaum Cascades . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

111
111
115
117
120
123
124
126


Contents

XI

Chapter II. Runge-Kutta and Extrapolation Methods
II.1

II.2

II.3

II.4

II.5

II.6

II.7

The First Runge-Kutta Methods . . . . . . . . . . . . . . . . . . . . . . . . . .
General Formulation of Runge-Kutta Methods . . . . . . . . . . . . . . .
Discussion of Methods of Order 4 . . . . . . . . . . . . . . . . . . . . . . . . . .
“Optimal” Formulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Numerical Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Order Conditions for Runge-Kutta Methods . . . . . . . . . . . . . . .
The Derivatives of the True Solution . . . . . . . . . . . . . . . . . . . . . . . .
Conditions for Order 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Trees and Elementary Differentials . . . . . . . . . . . . . . . . . . . . . . . . .
The Taylor Expansion of the True Solution . . . . . . . . . . . . . . . . . .
Fa`a di Bruno’s Formula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The Derivatives of the Numerical Solution . . . . . . . . . . . . . . . . . . .
The Order Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Error Estimation and Convergence for RK Methods . . . . . . . .
Rigorous Error Bounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The Principal Error Term . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Estimation of the Global Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Practical Error Estimation and Step Size Selection . . . . . . . . .
Richardson Extrapolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Embedded Runge-Kutta Formulas . . . . . . . . . . . . . . . . . . . . . . . . . .
Automatic Step Size Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Starting Step Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Numerical Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Explicit Runge-Kutta Methods of Higher Order . . . . . . . . . . . .
The Butcher Barriers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6-Stage, 5 th Order Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Embedded Formulas of Order 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Higher Order Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Embedded Formulas of High Order . . . . . . . . . . . . . . . . . . . . . . . . .
An 8 th Order Embedded Method . . . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Dense Output, Discontinuities, Derivatives . . . . . . . . . . . . . . . . .
Dense Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Continuous Dormand & Prince Pairs . . . . . . . . . . . . . . . . . . . . . . . .
Dense Output for DOP853 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Event Location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Discontinuous Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Numerical Computation of Derivatives with Respect
to Initial Values and Parameters . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Implicit Runge-Kutta Methods . . . . . . . . . . . . . . . . . . . . . . . . . . .
Existence of a Numerical Solution . . . . . . . . . . . . . . . . . . . . . . . . . .
The Methods of Kuntzmann and Butcher of Order 2s . . . . . . . . .
IRK Methods Based on Lobatto Quadrature . . . . . . . . . . . . . . . . .

132
134
135
139
140
141
143
145

145
145
148
149
151
153
154
156
156
158
159
163
164
164
165
167
169
170
172
173
173
175
176
179
180
181
185
188
188
191

194
195
196
200
202
204
206
208
210


XII

II.8

II.9

II.10

II.11

II.12

II.13

II.14

Contents
Collocation Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Asymptotic Expansion of the Global Error. . . . . . . . . . . . . . . . .
The Global Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Variable h . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Negative h . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Properties of the Adjoint Method . . . . . . . . . . . . . . . . . . . . . . . . . . .
Symmetric Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Extrapolation Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Definition of the Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The Aitken - Neville Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The Gragg or GBS Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Asymptotic Expansion for Odd Indices . . . . . . . . . . . . . . . . . . . . .
Existence of Explicit RK Methods of Arbitrary Order . . . . . . . . .
Order and Step Size Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Dense Output for the GBS Method . . . . . . . . . . . . . . . . . . . . . . . . .
Control of the Interpolation Error . . . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Numerical Comparisons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Performance of the Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
A “Stretched” Error Estimator for DOP853 . . . . . . . . . . . . . . . . . .
Effect of Step-Number Sequence in ODEX . . . . . . . . . . . . . . . . . .
Parallel Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Parallel Runge-Kutta Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Parallel Iterated Runge-Kutta Methods . . . . . . . . . . . . . . . . . . . . . .
Extrapolation Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Increasing Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Composition of B-Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Composition of Runge-Kutta Methods . . . . . . . . . . . . . . . . . . . . . .

B-Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Order Conditions for Runge-Kutta Methods . . . . . . . . . . . . . . . . .
Butcher’s “Effective Order” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Higher Derivative Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Collocation Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Hermite-Obreschkoff Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Fehlberg Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
General Theory of Order Conditions . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Numerical Methods for Second Order Differential Equations
Nystrăom Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The Derivatives of the Exact Solution . . . . . . . . . . . . . . . . . . . . . . .
The Derivatives of the Numerical Solution . . . . . . . . . . . . . . . . . . .
The Order Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
On the Construction of Nystrăom Methods . . . . . . . . . . . . . . . . . . .
An Extrapolation Method for y  = f (x, y) . . . . . . . . . . . . . . . . . .
Problems for Numerical Comparisons . . . . . . . . . . . . . . . . . . . . . . .

211
214
216
216
218
219
220
221
223
224
224

226
228
231
232
233
237
240
241
244
244
249
254
256
257
258
259
261
261
263
264
264
266
269
270
272
274
275
277
278
280

281
283
284
286
288
290
291
294
296


Contents

XIII

Performance of the Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
II.15 P-Series for Partitioned Differential Equations . . . . . . . . . . . . .
Derivatives of the Exact Solution, P-Trees . . . . . . . . . . . . . . . . . . .
P-Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Order Conditions for Partitioned Runge-Kutta Methods . . . . . . .
Further Applications of P-Series . . . . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

302
303
306
307
308
311

II.16 Symplectic Integration Methods . . . . . . . . . . . . . . . . . . . . . . . . . .
Symplectic Runge-Kutta Methods . . . . . . . . . . . . . . . . . . . . . . . . . .
An Example from Galactic Dynamics . . . . . . . . . . . . . . . . . . . . . . .
Partitioned Runge-Kutta Methods . . . . . . . . . . . . . . . . . . . . . . . . . .
Symplectic Nystrăom Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Conservation of the Hamiltonian; Backward Analysis . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

312
315
319
326
330
333
337

II.17 Delay Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Existence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Constant Step Size Methods for Constant Delay . . . . . . . . . . . . . .
Variable Step Size Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
An Example from Population Dynamics . . . . . . . . . . . . . . . . . . . . .
Infectious Disease Modelling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
An Example from Enzyme Kinetics . . . . . . . . . . . . . . . . . . . . . . . .
A Mathematical Model in Immunology . . . . . . . . . . . . . . . . . . . . .
Integro-Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

339

339
341
342
343
345
347
248
349
351
352

Chapter III. Multistep Methods
and General Linear Methods
III.1 Classical Linear Multistep Formulas . . . . . . . . . . . . . . . . . . . . . .
Explicit Adams Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Implicit Adams Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Numerical Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Explicit Nystrăom Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Milne–Simpson Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Methods Based on Differentiation (BDF) . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

356
357
359
361
362
363
364
366

III.2 Local Error and Order Conditions . . . . . . . . . . . . . . . . . . . . . . . .
Local Error of a Multistep Method . . . . . . . . . . . . . . . . . . . . . . . . .
Order of a Multistep Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Error Constant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Irreducible Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The Peano Kernel of a Multistep Method . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

368
368
370
372
374
375
377

III.3 Stability and the First Dahlquist Barrier . . . . . . . . . . . . . . . . . . .
Stability of the BDF-Formulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Highest Attainable Order of Stable Multistep Methods . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

378
380
383
387


XIV

Contents

III.4 Convergence of Multistep Methods . . . . . . . . . . . . . . . . . . . . . . . .
Formulation as One-Step Method . . . . . . . . . . . . . . . . . . . . . . . . . .
Proof of Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
III.5 Variable Step Size Multistep Methods . . . . . . . . . . . . . . . . . . . . .
Variable Step Size Adams Methods . . . . . . . . . . . . . . . . . . . . . . . . .
Recurrence Relations for gj (n) , Φj (n) and Φ∗j (n) . . . . . . . . . .
Variable Step Size BDF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
General Variable Step Size Methods and Their Orders . . . . . . . . .
Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
III.6 Nordsieck Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Equivalence with Multistep Methods . . . . . . . . . . . . . . . . . . . . . . .
Implicit Adams Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
BDF-Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
III.7 Implementation and Numerical Comparisons . . . . . . . . . . . . . .
Step Size and Order Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Some Available Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Numerical Comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
III.8 General Linear Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
A General Integration Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . .
Stability and Order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Order Conditions for General Linear Methods . . . . . . . . . . . . . . .
Construction of General Linear Methods . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

III.9 Asymptotic Expansion of the Global Error. . . . . . . . . . . . . . . . .
An Instructive Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Asymptotic Expansion for Strictly Stable Methods (8.4) . . . . . . .
Weakly Stable Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The Adjoint Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Symmetric Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
III.10 Multistep Methods for Second Order Differential Equations
Explicit Stăormer Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Implicit Stăormer Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Numerical Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
General Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Asymptotic Formula for the Global Error . . . . . . . . . . . . . . . . . . . .
Rounding Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

391
393
395
396
397
397
399
400
401
402
407
409
410

412
417
419
420
421
421
423
427
430
431
436
438
441
443
445
448
448
450
454
457
459
460
461
462
464
465
467
468
471
472

473

Appendix. Fortran Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Driver for the Code DOPRI5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Subroutine DOPRI5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Subroutine DOP853 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Subroutine ODEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

475
475
477
481
482


Contents

XV

Subroutine ODEX2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484
Driver for the Code RETARD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 486
Subroutine RETARD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 488
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

491

Symbol Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 521
Subject Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523

Chapter I. Classical Mathematical Theory
. . . halte ich es immer făur besser, nicht mit dem Anfang anzufangen, der immer das Schwerste ist.
(B. Riemann copied this from F. Schiller into his notebook)

This first chapter contains the classical theory of differential equations, which we
judge useful and important for a profound understanding of numerical processes
and phenomena. It will also be the occasion of presenting interesting examples of
differential equations and their properties.
We first retrace in Sections I.2-I.6 the historical development of classical integration methods by series expansions, quadrature and elementary functions, from
the beginning (Newton and Leibniz) to the era of Euler, Lagrange and Hamilton. The next part (Sections I.7-I.14) deals with theoretical properties of the solutions (existence, uniqueness, stability and differentiability with respect to initial
values and parameters) and the corresponding flow (increase of volume, preservation of symplectic structure). This theory was initiated by Cauchy in 1824 and
then brought to perfection mainly during the next 100 years. We close with a brief
account of boundary value problems, periodic solutions, limit cycles and strange
attractors (Sections I.15 and I.16).


I.1 Terminology

A differential equation of first order is an equation of the form
y  = f (x, y)

(1.1)

with a given function f (x, y) . A function y(x) is called a solution of this equation
if for all x ,


(1.2)
y  (x) = f x, y(x) .
It was observed very early by Newton, Leibniz and Euler that the solution usually

contains a free parameter, so that it is uniquely determined only when an initial
value
y(x0 ) = y0
(1.3)
is prescribed. Cauchy’s existence and uniqueness proof of this fact will be discussed in Section I.7. Differential equations arise in many applications. We shall
see the first examples of such equations in Section I.2, and in Section I.3 how some
of them can be solved explicitly.
A differential equation of second order for y is of the form
y  = f (x, y, y ).

(1.4)

Here, the solution usually contains two parameters and is only uniquely determined
by two initial values
y(x0 ) = y0 ,

y  (x0 ) = y0 .

(1.5)

Equations of second order can rarely be solved explicitly (see I.3). For their numerical solution, as well as for theoretical investigations, one usually sets y1 (x) :=
y(x) , y2 (x) := y  (x) , so that equation (1.4) becomes
y1 = y2

y1 (x0 ) = y0

y2

y2 (x0 ) = y0 .

= f (x, y1 , y2 )

(1.4’)

This is an example of a first order system of differential equations, of dimension n
(see Sections I.6 and I.9),
y1 = f1 (x, y1 , . . . , yn )
...
yn

= fn (x, y1 , . . . , yn )

y1 (x0 ) = y10
...
yn (x0 ) = yn0 .

(1.6)


I.1 Terminology

3

Most of the theory of this book is devoted to the solution of the initial value problem for the system (1.6). At the end of the 19th century (Peano 1890) it became
customary to introduce the vector notation
y = (y1 , . . . , yn )T ,

f = (f1 , . . . , fn )T

so that (1.6) becomes y  = f (x, y) , which is again the same as (1.1), but now with

y and f interpreted as vectors.
Another possibility for the second order equation (1.4), instead of transforming
it into a system (1.4’), is to develop methods specially adapted to second order
equations (Nystrăom methods). This will be done in special sections of this book
(Sections II.13 and III.10). Nothing prevents us, of course, from considering (1.4)
as a second order system of dimension n .
If, however, the initial conditions (1.5) are replaced by something like y(x0 ) =
a , y(x1 ) = b , i.e., if the conditions determining the particular solution are not all
specified at the same point x0 , we speak of a boundary value problem. The theory
of the existence of a solution and of its numerical computation is here much more
complicated. We give some examples in Section I.15.
Finally, a problem of the type

∂u ∂ 2 u 
∂u
= f t, u,
,
(1.7)
∂t
∂x ∂x2
for an unknown function u(t, x) of two independent variables will be called a partial differential equation. We can also deal with partial differential equations of
higher order, with problems in three or four independent variables, or with systems of partial differential equations. Very often, initial value problems for partial
differential equations can conveniently be transformed into a system of ordinary
differential equations, for example with finite difference or finite element approximations in the variable x . In this way, the equation
∂u
∂ 2u
= a2 2
∂t
∂x
would become


a2 
dui
u
−
2u
+
u
=
i
i−1 ,
dt
Δx2 i+1

where ui (t) ≈ u(t, xi ) . This procedure is called the “method of lines” or “method
of discretization in space” (Berezin & Zhidkov 1965). We shall see in Section I.6
that this connection, the other way round, was historically the origin of partial differential equations (d’Alembert, Lagrange, Fourier). A similar idea is the “method
of discretization in time” (Rothe 1930).


I.2 The Oldest Differential Equations
. . . So zum Beispiel die Aufgabe der umgekehrten Tangentenmethode, von welcher auch Descartes eingestand, dass er sie nicht in
seiner Gewalt habe.
(Leibniz, 27. Aug 1676)
. . . et on sait que les seconds Inventeurs n’ont pas de droit a` l’Invention.
(Newton, 29 mai 1716)
Il ne paroist point que M. Newton ait eu avant moy la characteristique & l’algorithme infinitesimal . . .
(Leibniz)
And by these words he acknowledged that he had not yet found the

reduction of problems to differential equations.
(Newton)

Newton
Differential equations are as old as differential calculus. Newton considered them
in his treatise on differential calculus (Newton 1671) and discussed their solution
by series expansion. One of the first examples of a first order equation treated by
Newton (see Newton (1671), Problema II, Solutio Casus II, Ex. I) was
y  = 1 − 3x + y + x2 + xy.

(2.1)

For each value x and y , such an equation prescribes the derivative y  of the solutions. We thus obtain a vector field, which, for this particular equation, is sketched
in Fig. 2.1a. (So, contrary to the belief of many people, vector fields existed long
before Van Gogh). The solutions are the curves which respect these prescribed
directions everywhere (Fig. 2.1b).
Newton discusses the solution of this equation by means of infinite series,
whose terms he obtains recursively (“. . . & ils se jettent sur les series, o´u M. Newton m’a preced´e sans difficult´e; mais . . .”, Leibniz). The first term
y = 0+...
is the initial value for x = 0 . Inserting this into the differential equation (2.1) he
obtains
y = 1 + . . .
which, integrated, gives
y = x+....
Again, from (2.1), we now have
y  = 1 − 3x + x + . . . = 1 − 2x + . . .
and by integration

y = x − x2 + . . . .

I.2 The Oldest Differential Equations

a:

b:




m 
m 

c:





m 





5


correct
m 




Fig. 2.1. a) vector field, b) various solution curves of equation (2.1),
c) Correct solution vs. approximate solution

The next round gives
y  = 1 − 2x + x2 + . . . ,

y = x − x2 +

x3
+....
3

Continuing this process, he finally arrives at
1
1
1
1
y = x − xx + x3 − x4 + x5 − x6 ; &c.
(2.2)
3
6
30
45
These approximations, term after term, are plotted in Fig. 2.1c together with the
correct solution. It can be seen that these approximations are closer and closer
to the true solution for small values of x . For more examples see Exercises 1-3.

Convergence will be discussed in Section I.8.


6

I. Classical Mathematical Theory

Leibniz and the Bernoulli Brothers
A second access to differential equations is the consideration of geometrical problems such as inverse tangent problems (Debeaune 1638 in a letter to Descartes). A
particular example describes the path of a silver pocket watch (“horologio portabili suae thecae argentae”) and was proposed around 1674 by “Claudius Perraltus
Medicus Parisinus” to Leibniz: a curve y(x) is required whose tangent AB is
given, say everywhere of constant length a (Fig. 2.2). This leads to
y
y = − 
,
(2.3)
2
a − y2
a first order differential equation. Despite the efforts of the “plus c´el`ebres math´ematiciens de Paris et de Toulouse” (from a letter of Descartes 1645, “Toulouse”
means “Fermat”) the solution of these problems had to wait until Leibniz (1684)
and above all until the famous paper of Jacob Bernoulli (1690). Bernoulli’s idea
applied to equation
 (2.3) is as follows: let the curve BM in Fig. 2.3 be such that
LM is equal to a2 − y 2 /y . Then (2.3), written as

a2 − y 2
dy,
(2.3’)
dx = −
y

shows that for all y the areas S1 and S2 (Fig. 2.3) are the same. Thus (“Ergo &
horum integralia aequantur”) the areas BM LB and A1 A2 C2 C1 must be equal
too. Hence (2.3’) becomes (Leibniz 1693)

 a 2

a − y2
a − a2 − y 2
dy = − a2 − y 2 − a · log
.
(2.3”)
x=
y
y
y

Ba

solution

dy
M
S

L
y
A

x

A a
dx

S
C
Fig. 2.2. Illustration from
Leibniz (1693)

C

Fig. 2.3. Jac. Bernoulli’s
Solution of (2.3)


I.2 The Oldest Differential Equations

7

Variational Calculus
In 1696 Johann Bernoulli invited the brightest mathematicians of the world (“Profundioris in primis Mathesos cultori, Salutem!”) to solve the brachystochrone
(shortest time) problem, mainly in order to fault his brother Jacob, from whom
he expected a wrong solution. The problem is to find a curve y(x) connecting two
points P0 , P1 , such that a point gliding on this curve under gravitation reaches P1
in the shortest time possible. In order to solve his problem, Joh. Bernoulli (1697b)
imagined thin layers of homogeneous media and knew from optics (Fermat’s principle) that a light ray with speed v obeying the law of Snellius
sin α = Kv
passes through in the shortest time. Since the speed is known to be proportional to
the square root of the fallen height, he obtains, by passing to thinner and thinner
layers,


1
sin α = 
=
K
2g(y − h),
(2.4)
1 + y 2
a differential equation of the first order.

Fig. 2.4. Solutions of the variational problem (Joh. Bernoulli,
Jac. Bernoulli, Euler)

The solutions of (2.4) can be shown to be cycloids (see Exercise 6 of Section I.3). Jacob, in his reply, also furnished a solution, much less elegant but unfortunately correct. Jacob’s method (see Fig. 2.4) was something like today’s (inverse)


8

I. Classical Mathematical Theory

“finite element” method and more general than Johann’s and led to the famous work
of Euler (1744), which gives the general solution of the problem
 x1
F (x, y, y ) dx = min
(2.5)
x0

with the help of the differential equation of the second order

d 
Fy (x, y, y ) = Fy − Fy y y  − Fy y y  − Fy x = 0, (2.6)

Fy (x, y, y ) −
dx
and treated 100 variational problems in detail. Equation (2.6), in the special case
where F does not depend on x , can be integrated to give
F − Fy y  = K.

(2.6’)

Euler’s original proof used polygons in order to establish equation (2.6). Only the
ideas of Lagrange, in 1755 at the age of 19, led to the proof which is today the usual
one (letter of Aug. 12, 1755; Oeuvres vol. 14, p. 138): add an arbitrary “variation”
δy(x) to y(x) and linearize (2.5).
 x1


(2.7)
F x, y + δy, y  + (δy) dx
x0
 x1 
 x1



F x, y, y  dx +
=
Fy (x, y, y ) δy + Fy (x, y, y )(δy) dx + . . .
x0

x0

The last integral in (2.7) represents the “derivative” of (2.5) with respect to δy .
Therefore, if y(x) is the solution of (2.5), we must have
 x1 

Fy (x, y, y ) δy + Fy (x, y, y )(δy) dx = 0
(2.8)
x0

or, after partial integration,
 x1 

d
Fy (x, y, y ) · δy(x) dx = 0.
Fy (x, y, y ) −
dx
x0

(2.8’)

Since (2.8’) must be fulfilled by all δy , Lagrange “sees” that
d
(2.9)
F  (x, y, y ) = 0
dx y
is necessary for (2.5). Euler, in his reply (Sept. 6, 1755) urged a more precise proof
of this fact (which is now called the “fundamental Lemma of variational Calculus”).
For several unknown functions

(2.10)
F (x, y1 , y1 , . . . , yn , yn ) dx = min

Fy (x, y, y ) −

the same proof leads to the equations
Fyi (x, y1 , y1 , . . . , yn , yn ) −

d
F  (x, y1 , y1 , . . . , yn , yn ) = 0
dx yi

(2.11)


I.2 The Oldest Differential Equations

9

for i = 1, . . . , n. Euler (1756) then gave, in honour of Lagrange, the name “Variational calculus” to the whole subject (“. . . tamen gloria primae inventionis acutissimo Geometrae Taurinensi La Grange erat reservata”).

Clairaut
A class of equations with interesting properties was found by Clairaut (see Clairaut
(1734), Probl`eme III). He was motivated by the movement of a rectangular wedge
(see Fig. 2.5), which led him to differential equations of the form
y − xy  + f (y  ) = 0.

(2.12)

This was the first implicit differential equation and possesses the particularity that
not only the lines y = Cx − f (C) are solutions, but also their enveloping curves
(see Exercise 5). An example is shown in Fig. 2.6 with f (C) = 5(C 3 − C)/2 .

Fig. 2.5. Illustration from Clairaut (1734)

Since the equation is of the third degree in y  , a given initial value may allow
up to three different solution lines. Furthermore, where a line touches an enveloping curve, the solution may be continued either along the line or along the curve.
There is thus a huge variety of different possible solution curves. This phenomenon
attracted much interest in the classical literature (see e.g., Exercises 4 and 6). Today we explain this curiosity by the fact that at these points no Lipschitz condition
is satisfied (see also Ince (1944), p. 538–539).


10

I. Classical Mathematical Theory











Fig. 2.6. Solutions of a Clairaut differential equation

Exercises
1. (Newton). Solve equation (2.1) with another initial value y(0) = 1 .
Newton’s result: y = 1 + 2x + x3 + 14 x4 + 14 x5 , &c.
2. (Newton 1671, “Problema II, Solutio particulare”). Solve the total differential
equation

3x2 − 2ax + ay − 3y 2 y  + axy  = 0.
Solution given by Newton: x3 − ax2 + axy − y 3 = 0 . Observe that he missed
the arbitrary integration constant C .
3. (Newton 1671). Solve the equations
a)
b)

y xy x2 y x3 y
+
+ 3 + 4 , &c.
a a2
a
a

y = −3x + 3xy + y 2 − xy 2 + y 3 − xy 3 + y 4 − xy 4 + 6x2 y

y = 1 +

− 6x2 + 8x3 y − 8x3 + 10x4 y − 10x4 , &c.
Results given by Newton:
a)
b)

x2
x4
x5
x6
x3
+ 2 + 3 + 4 + 5 , &c.
2a 2a

2a
2a
2a
25 4 91 5 111 6 367 7
3 2
3
x −
x , &c.
y = − x − 2x − x − x −
2
8
20
16
35
y =x+


I.2 The Oldest Differential Equations

4. Show that the differential equation
x + yy  = y 



11

x2 + y 2 − 1

possesses the solutions 2ay = a2 + 1 − x2 for all a . Sketch these curves and
find yet another solution of the equation (from Lagrange (1774), p. 7, which

was written to explain the “Clairaut phenomenon”).
5. Verify that the envelope of the solutions y = Cx − f (C) of the Clairaut equation (2.12) is given in parametric representation by
x(p) = f  (p)
y(p) = pf  (p) − f (p) .
Show that this envelope is also a solution of (2.12) and calculate it for f (C) =
5(C 3 − C)/2 (cf. Fig. 2.6).
6. (Cauchy 1824). Show that the family y = C(x + C)2 satisfies the differential
equation (y  )3 = 8y 2 − 4xyy  . Find yet another solution which is not included
in this family (see Fig. 2.7).
4 3
Answer: y = − 27
x .












Fig. 2.7. Solution family of Cauchy’s example in Exercise 6


I.3 Elementary Integration Methods

We now discuss some of the simplest types of equations, which can be solved by

the computation of integrals.

First Order Equations
The equation with separable variables.
y  = f (x)g(y).

(3.1)

Extending the idea of Jacob Bernoulli (see (2.3’)), we divide by g(y) , integrate and
obtain the solution (Leibniz 1691, in a letter to Huygens)


dy
= f (x) dx + C.
g(y)
A special example of this is the linear equation y  = f (x)y , which possesses the
solution


y(x) = CR(x),
R(x) = exp
f (x) dx .
The inhomogeneous linear equation.
y  = f (x)y + g(x).

(3.2)

Here, the substitution y(x) = c(x)R(x) leads to c (x) = g(x)/R(x) (Joh. Bernoulli
1697). One thus obtains the solution


  x g(s)
ds + C .
(3.3)
y(x) = R(x)
x0 R(s)
Total differential equations. An equation of the form
P (x, y) + Q(x, y)y  = 0

(3.4)

is found to be immediately solvable if
∂P
∂Q
=
.
∂y
∂x

(3.5)