Tài liệu Thuật toán Algorithms (Phần 9) ppt

CURVE FITTING
73
the same steps of determining the coefficients for each of the spline pieces by
solving the system of linear equations derived from imposing constraints on
how they are joined.
Method of Least Squares
A very common experimental situation is that, while the data values that we
have are not exact, we do have some idea of the form of the function which
is to fit the data. The function might depend on some parameters
and the curve fitting procedure is to find the choice of parameters that “best”
matches the observed values at the given points. If the function were a poly-
nomial (with the parameters being the coefficients) and the values were exact,
then this would be interpolation. But now we are considering more general
functions and inaccurate data. To simplify the discussion, we’ll concentrate
on fitting to functions which are expressed as a linear combination of simpler
functions, with the unknown parameters being the coefficients:
f(x)
= +

This includes most of the functions that we’ll be interested in. After studying
this case, we’ll consider more general functions.
A common way of measuring how well a function fits is the least-squares
criterion: the error is calculated by adding up the squares of the errors at
each of the observation points:
E=
--
This is a very natural measure: the squaring is done to stop cancellations
among errors with different signs. Obviously, it is most desirable to find the
choice of parameters that minimizes E. It turns out that this choice can be
computed efficiently: this is the so-called method of least squares.
The method follows quite directly from the definition. To simplify the

derivation, we’ll do the case = 2, N = but the general method will follow
directly. Suppose that we have three points xi, and corresponding values
which are to be fitted to a function of the form f(x) = +
Our job is to find the choice of the coefficients which minimizes
the least-squares error
+
+

74
6
To find the choices of and which minimize this error, we simply need to
set the derivatives and to zero. For we have:
+
1
+
Setting the derivative equal to zero leaves an equation which the variables
and must satisfy etc. are all “constants” with known values):
+
+
=
We get a similar equation when we set the derivative to zero.
These rather formidable-looking equations can be greatly simplified using
vector notation and the “dot product” operation that we encountered briefly
in Chapter 2. If we define the vectors x = and y = and
then the dot product of x and y is the real number defined by
=
Now, if we define the vectors = and =

en
our equations for the coefficients and can be very

simply expressed:
. + = y .

These can be solved with Gaussian elimination to find the desired coefficients.
For example, suppose that we know that the data points

should be fit by a function of the form + (These data points are
slightly perturbed from the exact values for 1 + In this case, we have
= 1.0) and = so we have
to solve the system of equations
CURVE FITTING
75
with the result = 0.998 and = 1.054 (both close to 1, as expected).
The method outlined above easily generalizes to find more than two
coefficients. To find the constants . . in
= + +
which minimize the least squares error for the point and observation vectors
first compute the function component vectors
fl = . . . ,
= .
Then make up an M-by-M linear system of equations with

= . y.
The solution to this system of simultaneous equations yields the required
coefficients.
This method is easily implemented by maintaining a two dimensional
array for the f vectors, considering y as the (M + vector. Then an array
can be filled as follows:
for to Mdo
for to do

begin
0.0;
for to N do k];

end;
and then solved using the Gaussian elimination procedure from Chapter 5.
The method of least squares can be extended to handle nonlinear func-
tions (for example a function such as = and it is often
76
CHAPTER 6
used for this type of application. The idea is fundamentally the same; the
problem is that the derivatives may not be easy to compute. What is used
is an iterative method: use some estimate for the coefficients, then use these
within the method of least squares to compute the derivatives, thus producing
a better estimate for the coefficients. This basic method, which is widely used
today, was outlined by Gauss in the 1820s.
CURVE FITTING 77
Exercises
1.
Approximate the function lgx with a degree 4 interpolating polynomial
at the points and 5. the quality of the fit by computing
the sum of the squares of the errors at 1.5, 2.5, 3.5, and 4.5.
2.
Solve the previous problem for the function sinx. Plot the function and
the approximation, if that’s possible on your computer system.
3.
Solve the previous problems using a cubic spline instead of an interpolat-
ing polynomial.
4.
Approximate the function lgx with a cubic spline with knots at for

N between 1 and 10. Experiment with different placements of knots in
the same range to try to obtain a better fit.
5.
What would happen in least squares data fitting if one of the functions
was the function = 0 for some
6.
What would happen in least squares data-fitting if all the observed values
were O?
7.
What values of a, c minimize the least-squares error in using the function
f(x) = log x + bx c to approximate the observations = 0, f(4) =
13, f(8) =
8.
Excluding the Gaussian elimination phase, how many multiplications are
involved in using the method of least squares to find M coefficients based
on N observations?
9.
Under what circumstances would the matrix which arises in least-squares
curve fitting be singular?
10.
Does the least-squares method work if two different observations are in-
cluded for the same point?