Introduction to Numerical Methods for Diferentia

Lecture 3
Introduction to Numerical Methods for Differential
and Differential Algebraic Equations
Dr. Abebe Geletu
Ilmenau University of Technology
Department of Simulation and Optimal Processes (SOP)

Winter Semester 2011/12

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

TU Ilmenau


Topics
Newton’s Law:
m¨
x = − Fl x
m¨
y = mg Fl y
Conservation of mechanical
energy: x2 + y 2 = l2

(DAE)

x˙ 1 = x3
x˙ 2 = x4
F
x1
ml
F

x˙ 4 = g x2
l
0 = x2 + y 2 − l 2 .
x˙ 3 = −

1
2

Numerical Methods of Ordinary Differential Equations
Initial Value Problems (IVPs)

Lecture 3 Introduction1 to Numerical Methods for Differential and Differential Algebraic Equations
TU Ilmenau

Single Step Methods
Multi-step Methods


Initial Value Problems
General form of an initial value problem (IVP) is
x(t)
˙
= f (t, x(t)), t0 ≤ t ≤ tf ,

(1)

x(t0 ) = x0

(2)

(given),

where x and f can be vector functions:




f1 (t, x(t))
x1 (t)
 x2 (t) 
 f2 (t, x(t)) 




x(t) =  .  , f (t, x(t)) = 
.
..
.
 . 


.
xn (t)
fn (t, x(t))
If f is (explicitly) time independent; i.e. f (t, x(t)) = f (x(t)), then
the system (1) - (2) is said to be autonomous.
Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

TU Ilmenau

x(t0 ) = x0 is the initial condition (start time of the motion)
The expression x(t)
˙
represents the first order derivative of x(t)
d
with respect to time; i.e. x(t)
˙
= dt
x(t). Thus, the IVP (1) - (2)
represents a system of first order differential equations.
If f (t, x(t)) = A(t)x(t), then we have
x(t)
˙
= A(t)x(t), x(t0 ) = x0 is a first order linear IVP. In the
autonomous case we have A(t) = A.
Example: Equation of motion of a simple pendulum:
θ˙ = ω

g
ω˙ = − sin(θ)
L
⇒
θ˙
ω˙

=

ω

− Lg sin(θ)

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

TU Ilmenau

,

θ(0) = π4
ω(0) = 0


Commonly, numerical methods are developed for systems of first
order differential equations.
⇒ any higher order differential equation should be written as a
system of first order differential equations.
NB: The Matlab ODE Toolbox works only with systems of first order
differential equations.
Strategy: Convert any n-th order initial value problem into a system
of first-order IVP by introduction new variables.
dx d2 x
dn−1 x
, 2 , . . . , n−1
dt dt
dt
dx(t)
dn−1 x(t)
x(t0 ) = a1 , x(t
˙ 0 ) = a2 ,
= a3 , . . . ,

dt t=t0
dtn−1
dn x(t)
=f
dtn

t,

Introduce new variables:
2
x1 = x, x2 = x,
˙ x3 = d dtx(t)
2 , . . . , xn =

dn−1 x(t)
.
dtn−1

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

TU Ilmenau

= an .
t=t0


Then the n-th order IVP reduces to
x˙ 1 = x2 ,

x1 (t0 ) = a1

x˙ 2 = x3 ,
..
.

x2 (t0 ) = a2

x˙ n = f (t, x1 , x2 , . . . , xn ), xn (t0 ) = an .
Example: The equation of motion of a simple pendulum (neglecting
air resistance) with mass m is a 2nd-order equation
mLθ¨ + mg sin(θ(t)) = 0,
(3)
˙
θ(t0 ) = θ0 , θ = ω0 .
(4)
˙ Then
Introduce ω = θ.
θ˙ = ω,

θ(t0 ) = θ0
g
ω˙ = − sin(θ), ω(t0 ) = ω0 .
L

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

TU Ilmenau


Objective:

To determine a continuous function x∗ (t) that satisfies the dynamic
equation (1); i.e. x∗ (t) = f (t, x∗ (t)) and the initial condition (2); i.e.
x∗ (t0 ) = x0 .
!!! In general, the solution function x∗ (t) of the IVP is not easy to be
analytically determined.

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

TU Ilmenau


Basic idea of numerical solution of differential equations:
Select a set of equally spaced time points t1 , t2 , . . . , tN from the
interval [t0 , tf ] (tf freely chosen for IVPs). That is, t1 = t0 + h ,
t2 = t1 + h, and so on; so that h = tk+1 − tk , k = 0, 1, . . . , N − 1.
Approximate the continuous solution x∗ (t) of (1) & (2) by values
at the discrete time points t1 , t2 , . . . , tN ; i.e.
x(t1 ) = x1 , x(t2 ) = x2 , . . . , x(tN ) = xN .
determine the values x1 , x2 , . . . , xN through the IVP (1) - (2).
Note:
more discretization points from [t0 , tf ] implies
better accuracy of the approximation; but
demands more computation effort,
depending on the algorithm used, it may be necessary to solve a
system of linear or nonlinear equations.
Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

TU Ilmenau

Euler’s Method
Motivation: for a single IVP x(t)
˙
= f (t, x(t)), x(t0 ) = x0 ; i.e.
x(t) ∈ R, 0 ≤ t ≤ tf .
Given the point (t0 , x0 ) the value x(t
˙ 0) =

dx(t)
dt t=t
0

is the slope of

the tangent line to the graph of x(t) at the point (t0 , x0 ).

Figure: Euler’s Method
Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

TU Ilmenau


Since the function x(t) is not yet known, the derivative (slope) can be
approximated by
x(t0 + h) − x(t0 )
x(t
˙ 0) ≈
h
Thus, using x(t
˙ 0 ) = f (t0 , x0 ), we have

x(t0 + h) − x(t0 )
= f (t0 , x(t0 ))
h
This implies
x(t0 + h) = x(t0 ) + hf (t0 , x(t0 ))
⇒
x1 = x(t1 ) = x(t0 + h) = x(t0 ) + hf (t0 , x(t0 ))
From this we obtain the value of x1 = x(t1 ). By the same procedure
x2 can be also determined.
In general, once xk is known, xk+1 can be found from
xk+1 = xk + hf (tk , xk ), k = 0, 1, . . . , m − 1.
Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

TU Ilmenau


Euler’s Algorithm
Select N .
(t −t )
Set h = f N 0 ; x0 = x(t0 )
FOR k = 0 : (n − 1)
xk+1 = xk + hf (tk , xk );
tk+1 = tk + h;
ENDFOR.
Matlab:
function [t,x]=myEuler(@f,x0,t0,tf,N)
h=(tf-tn)/N;
t=[t0:h:tf]; %discrete time points between, and including,
x(1)=x0;
for k=2:N

x(k)=x(k-1) + h*feval(f,t(k-1),x(k-1));
end;
Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

TU Ilmenau


Simple Pendulum with Euler’s Method
function time=MyPenEuler(N,tf)
t=cputime;
m=1;
L=9.81;
g=9.81;
beta=0;
t0=0;
theta0=(1/10)*pi;
omega0=0;
h=(tf-t0)/N;
tt=[t0:h:tf];
ttheta=zeros(size(tt));
oomega=zeros(size(tt));
ttheta(1)=theta0;
oomega(1)=omega0;
Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

TU Ilmenau


Simple Pendulum with Euler’s Method...

for k=1:N
oomega(k+1)=oomega(k)+h*...
(-beta/m/L*oomega(k)-g/L*sin(ttheta(k)));
ttheta(k+1)=ttheta(k)+h*oomega(k);
end
time=cputime-t;
plot(tt,oomega,tt,ttheta)
title(’Simple pendulum using Euler’’s method’)

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

TU Ilmenau


Advantages/Disadvantages of Euler’s Method
Advantages:
Euler’s Method is simple and direct
Can be used for nonlinear IVPs
Disadvantages:
it is less accurate and numerically unstable.
Approximation error is proportional to the step size h. Hence,
good approximation is obtained with a very small h. This
requires a large number of time discretization leading to a larger
computation time.
Usually applicable to explicit differential equations.

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

TU Ilmenau

Improvements on Euler’s Method
(A) The Mid-Point Method
Let k1 = hf (ti , xi ).
Note that: k1 = hf (ti , xi ) = xi+1 − xi . But to compute k1 , it is not
necessary to know the value of xi+1 .

Figure: Use also the mid point
Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

TU Ilmenau


Improvements on Euler’s Method...
• The point (ti + 21 ti , xi + 12 k1 ) is the mid-point of the segment that
connects the points (ti , xi ) and (ti+1 , xi+1 ).
• Now, instead of the single slope at (ti , xi ), we use the slopes at
(ti , xi ) and at (ti + 12 ti , xi + 12 k1 ) to approximate xi+1 .
⇒ This leads to the Mid-Point Runge-Kutta Algorithm:
The mid-point Runge-Kutta Method
k1 = hf (ti , xi )
1
1
k2 = hf (ti + h, xi + k1 )
2
2
xi+1 = xi + k2 .

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

TU Ilmenau


Improvements on Euler’s Method...
(B) Methods that use Intermediate Points
Question: Can we improve the mid-point RK method by freely
choosing any point from the segment connecting (ti , xi ) and
(ti+1 , xi+1 )?

Figure: Choose freely any intermediate point
Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

TU Ilmenau


• A weighted sum of f (ti , xi ) and f (ti + ch, xi + mk1 ) provideds
more slope information than f (ti , xi ) alone.
Scheme:
(RK2)

x(t + h) = x(t) + hf (t, x)

where f (t, x) = [w1 f (t, x) + w2 f (t + ch, x + mk1 )] .
How to determine w1 , w2 , c and h?
We use:
(i) Taylor approximation of f (t + ch, x + mk1 ):
f (t + ch, x + mk1 ) = f (t, x) + chft (t, x) + mk1 fx (t, x)x

(∗)

Put this into (*) to obtain
x(t+h) = x(t)+h [w1 f (t, x) + w2 {f (t, x) + chft (t, x) + mk1 fx (t, x)})]
⇒ (with k1 = hf (t, x))

x(t+h) = x(t)+h (w1 + w2 ) f (t, x)+w2 ch2 ft (t, x)+mw2 h2 fx (t, x)f (t, x)
Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

TU Ilmenau


Improvements on Euler’s Method...
(ii) 2nd- order Taylor approximation of x(t + h):
x(t + h) = x(t) + hx (t) +

h2
x (t)
2

(∗∗)

h2
x (t) (∗∗)
2
Recall x (t) = x(t)
˙
= f (t, x) ⇒ x (t) = ft (t, x) + fx (t, x)f (t, x).
Putting these into (**) we obtain
x(t + h) = x(t) + hx (t) +

x(t + h) = x(t) + hf (x, t) +

h2
h2
ft (t, x) + fx (t, x)f (t, x))
2
2

Comparing (*) and (**) we find

(∗∗)

w1 + w2 = 1, cw2 = 12 , mw2 = 12 .

A system of three equations with four unknowns ⇒ This system has
infinitely many solutions.
Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

TU Ilmenau


Improvements on Euler’s Method...
• By fixing one of the unknowns w1 , w2 , c and m, we can solve the
system of equations for the other three.
⇒ Hence, we can construct several RK2 methods.
Example:
• If we fix w1 = 0, then w2 = 1, c = 12 and m = 21 . Hence, (RK2)
reduces to the mid-point RK2 method.
w1
0

w2
1

c

m

1
2
1
3

1
2
2
3

1
2

1
2

1

1

3
4

3
4

Method
Modified Euler Method
Heun’s Method
Ralston’s Method

Heun’s Method:
k1 = f (ti , xi )
xi+1 = xi + h 12 f (ti , xi ) + 12 f (ti + h, xi + k1 )
Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

TU Ilmenau


Runge-Kutta Methods
The above schemes are known as second-order Runge-Kutta
methods, since x(t + h) has been approximated by a second degree
Taylor polynomial.
For an accurate approximation use more intermediate points
between (ti , xi ) and (ti + h, xi + k1 ) and include more slope
information.
Requires higher degree Taylor approximations of x(t + h).

Figure: Use several intermediate points
Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

TU Ilmenau

Runge-Kutta Methods...
General Scheme:
[RK(m)]

xi+1 = xi + hf (ti , xi );

where
f (ti , xi ) = w1 f (ti , xi ) + w2 f (ti + α1 h, xi + β1 k1 ) + . . . + wm f (ti + αm h, xi + βm km ).

• The values k1 , k2 , . . . , km are found at the intermediate points.
• w1 + w2 + . . . + wm = 1.
• The relation between ω1 , . . . , ωm , α1 , . . . , αm , β1 , . . . , βm is
determined by comparing the m-th order Talyor approximation of
x(t + h) and the scheme RK(m)

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

TU Ilmenau


Hence, the resulting method is called an m-th order Runge-Kutta
method.
There are third-order Runge-Kutta (RK3), fourth-order
Runge-Kutta methods (RK4), etc.
The classical 4-th order Runge-Kutta Method has the form:
(RK4s) k1 = f (ti , xi )
1
1
k2 = hf (ti + h + xi + k1 )

2
2
1
1
k3 = hf (ti + h + xi + k2 )
2
2
k4 = hf (ti + h, xi + k3 )
1
1
1
1
xi+1 = xi + k1 + k2 + k3 + k4
6
3
3
6

Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

TU Ilmenau


The classical 4th order Runge-Kutta Method
function myRK4(N,tf)
m=5; %Mass
L=9.81; %length of the string
g=9.81; %gravitaional acceleration
beta=0; %
t0=0;

theta0=(1/10)*pi;
omega0=0;
h=(tf-t0)/N
tt=[t0:h:tf];
ttheta=zeros(size(tt));
oomega=zeros(size(tt));
ttheta(1)=theta0;
oomega(1)=omega0;
Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

TU Ilmenau


%The classic 4th order Runge Kutta method
for k=1:N
k1=h*(-beta/m/L*oomega(k)-g/L*sin(ttheta(k)));
m1=h*oomega(k);
k2=h*(-beta/m/L*(oomega(k)+k1/2)-g/L*(sin(ttheta(k)+m1/2
m2=h*(oomega(k)+m1/2);
k3=h*(-beta/m/L*(oomega(k)+k2/2)-g/L*(sin(ttheta(k)+m2/2
m3=h*(oomega(k)+k2/2);
k4=h*(-beta/m/L*(oomega(k)+k3/2)-g/L*(sin(ttheta(k)+m3/2
m4=h*(oomega(k)+k3);
oomega(k+1)=oomega(k)+1/6*(k1+2*k2+2*k3+k4);
ttheta(k+1)=ttheta(k)+1/6*(m1+2*m2+2*m3+m4);
end
% Plot the solutions.
plot(tt,oomega,tt,ttheta)
title(’Simple pendulum solved with 4th order Runge Kutta’)
Lecture 3 Introduction to Numerical Methods for Differential and Differential Algebraic Equations

TU Ilmenau