Mô Hình Hóa Nhận Dạng và Mô Phỏng - Closed lôp ident

Closed-lo op Identication Revisited {
Up dated Version
Urban Forssell and Lennart Ljung
Department of Electrical Engineering
Linkping University, S-581 83 Linkping, Sweden
WWW:
Email: ,
1 April 1998
R
E
G
L
E
R
T
E
K
N
I
K
A
U
T
O
M
A
T
I
C

C

O
N
T
R
O
L
LINKÖPING
Rep ort no.: LiTH-ISY-R-2021
Submitted to Automatica
Technical rep orts from the Automatic Control group in Linkping are available
by anonymous ftp at the address ftp.control.isy.liu.se. This rep ort is
contained in the compressed p ostscript le 2021.ps.Z.
Closed-lo op Identication Revisited {
Up dated Version
?
Urban Forssell and Lennart Ljung
Division of Automatic Control, Department of Electrical Engineering, Linkoping
University, S-581 83 Linkoping, Sweden. URL: />Abstract
Identication of systems op erating in closed lo op has long b een of prime interest
in industrial applications. The problem oers many p ossibilities, and also some
fallacies, and a wide variety of approaches have b een suggested, many quite recently.
The purp ose of the current contribution is to place most of these approaches in a
coherent framework, therebyshowing their connections and display similarities and
dierences in the asymptotic prop erties of the resulting estimates. The common
framework is created by the basic prediction error metho d, and it is shown that most
of the common metho ds corresp ond to dierent parameterizations of the dynamics
and noise mo dels. The so called indirect methods, e.g., are indeed "direct" metho ds
employing noise mo dels that contain the regulator. The asymptotic prop erties of
the estimates then follow from the general theory and take dierent forms as they
are translated to the particular parameterizations. In the course of the analysis we

also suggest a pro jection approach to closed-lo op identication with the advantage
of allowing approximation of the op en lo op dynamics in a given, and user-chosen
frequency domain norm, even in the case of an unknown, non-linear regulator.
Key words: System identication Closed-lo op identication Prediction error
metho ds
?
This pap er was not presented at any IFAC meeting. Corresp onding author U.
Forssell. Tel. +46-13-282226. Fax +46-13-282622. E-mail
Preprint submitted to Elsevier Preprint 1 April 1998
-
Extra input
+
+
f -
Input
Plant
- f
+
+
?
Noise
-
Output
?
f
-
+

Set point


Controller
6
Fig. 1. A closed-lo op system
1 Intro duction
1.1 Motivation and Previous Work
System identication is a well established eld with a number of approaches,
that can broadly be classied into the prediction error family, e.g,, 22], the
subspace approaches, e.g., 31], and the non-parametric correlation and spectral
analysis methods, e.g., 5]. Of sp ecial interest is the situation when the data
to b e used has b een collected under closed-lo op op eration, as in Fig. 1.
The fundamental problem with closed-lo op data is the correlation between
the unmeasurable noise and the input. It is clear that whenever the feedback
controller is not identically zero, the input and the noise will be correlated.
This is the reason why several metho ds that work in op en lo op fail when
applied to closed-lo op data. This is for example true for the subspace approach
and the non-parametric metho ds, unless sp ecial measures are taken. Despite
these problems, p erforming identication exp eriments under output feedback
(i.e. in closed lo op) maybenecessary due to safety or economic reasons, or if
the system contains inherent feedback mechanisms. Closed-lo op exp eriments
may also b e advantageous in certain situations:
 In 13] the problem of optimal exp eriment design is studied. It is shown
that if the mo del is to be used for minimum variance control design the
identication exp eriment should b e p erformed in closed-lo op with the opti-
mal minimum variance controller in the lo op. In general it can b e seen that
optimal exp eriment design with variance constraints on the output leads to
closed-lo op solutions.
 In \identication for control" the ob jective is to achieve a mo del that is
suited for robust control design (see, e.g., 7, 19, 33]). Thus one has to tailor
the exp eriment and prepro cessing of data so that the mo del is reliable in
regions where the design pro cess do es not tolerate signicant uncertainties.

The use of closed-lo op exp eriments has b een a prominent feature in these
approaches.
3
Historically, there has b een a substantial interest in b oth sp ecial identication
techniques for closed-lo op data, and for analysis of existing metho ds when
applied to such data. One of the earliest results was given by Akaike 1] who
analyzed the eect of feedback lo ops in the system on correlation and sp ectral
analysis. In the seventies there was a very activeinterest in questions concern-
ing closed-lo op identication, as summarized in the survey pap er 15]. See also
3]. Up to this p ointmuch of the attention had b een directed towards identi-
ability and accuracy problems. With the increasing interest "identication for
control", the fo cus has shifted to the abilityto shap e the bias distribution so
that control-relevant mo del approximations of the system are obtained. The
surveys 12] and 29] cover most of the results along this line of research.
1.2 Scope and Outline
It is the purp ose of the present pap er to \revisit" the area of closed-lo op
identication, to put some of the new results and metho ds into p ersp ective,
andtogive a status rep ort of what can b e done and what cannot. In the course
of this exp ose, some new results will also b e generated.
We will exclusively deal with metho ds derived in the prediction error frame-
work and most of the results will be given for the multi-input multi-output
(MIMO) case. The leading idea in the pap er will be to provide a unied
framework for many closed-lo op metho ds by treating them as dierent pa-
rameterizations of the prediction error metho d:
There is only one method. The dierent approaches are
obtained by dierent parameterizations of the dynamics
and noise models.
Despite this we will often use the terminology \metho d" to distinguish b etween
the dierent approaches and parameterizations. This has also been standard
in the literature.

The organization of the pap er is as follows. Next, in Section 2 wecharacterize
the kinds of assumptions that can b e made ab out the nature of the feedback.
This leads to a classication of closed-lo op identication metho ds into, so
called, direct, indirect, and joint input-output metho ds. As we will show, these
approaches can b e viewed as variants of the prediction error metho d with the
mo dels parameterized in dierentways. A consequence of this is that wemay
use all results for the statistical prop erties of the prediction error estimates
known from the literature. In Section 3 the assumptions we will make regarding
the data generating mechanism are formalized. This section also intro duces
the some of the notation that will b e used in the pap er.
4
Section 4 contains a brief review of the standard prediction error metho d as
well as the basic statements on the asymptotic statistical prop erties of this
metho d:
 Convergence and bias distribution of the limit transfer function estimate.
 Asymptotic variance of the transfer function estimates (as the mo del orders
increase).
 Asymptotic variance and distribution of the parameter estimates.
The application of these basic results to the direct, indirect, and joint input-
output approaches will b e presented in some detail in Sections 5{8. All pro ofs
will b e given in the App endix. The pap er ends with a summarizing discussion
in Section 9.
2 Approaches to Closed-lo op Identication
2.1 A Classication of Approaches
In the literature several dierent typ es of closed-lo op identication metho ds
have been suggested. In general one may distinguish b etween metho ds that
(a) Assume no knowledge ab out the nature of the feedback mechanism, and
do not use the reference signal r (t)even if known.
(b) Assume the feedback to b e known and typically of the form
u(t)=r (t) ; K (q )y (t) (1)

where u(t) is the input, y (t) the output, r (t) an external reference signal,
and K (q ) a linear time-invariant regulator. The symbol q denotes the
usual shift op erator, q
;1
y (t)=y (t ; 1), etc.
(c) Assume the regulator to b e unknown, but of a certain structure (like (1)).
If the regulator indeed has the form (1), there is no ma jor dierence between
(a), (b) and (c): The noise-free relation (1) can b e exactly determined based on
a fairly short data record, and then r (t) carries no further information ab out
the system, if u(t) is measured. The problem in industrial practice is rather
that no regulator has this simple, linear form: Various delimiters, anti-windup
functions and other non-linearities will have the input deviate from (1), even
if the regulator parameters (e.g. PID-co ecients) are known. This strongly
disfavors the second approach.
In this pap er we will use a classication of the dierent metho ds that is similar
to the one in 15]. See also 26]. The basis for the classication is the dierent
5
kinds of p ossible assumptions on the feedback listed ab ove. The closed-lo op
identication metho ds corresp ondingly fall into the following main groups:
(1) The Direct Approach : Ignore the feedback and identify the op en-lo op
system using measurements of the input u(t) and the output y (t).
(2) The Indirect Approach : Identify some closed-lo op transfer function and
determine the op en-lo op parameters using the knowledge of the con-
troller.
(3) The Joint Input-Output Approach : Regard the input u(t) and the output
y (t) jointly as the output from a system driven by the reference signal
r (t) and noise. Use some metho d to determine the op en-lo op parameters
from an estimate of this system.
These categories are basically the same as those in 15], the only dierence
is that in the joint input-output approach we allow the joint system to have

a measurable input r (t) in addition to the unmeasurable noise e(t). For the
indirect approach it can b e noted that most metho ds studied in the literature
assume a linear regulator but the same ideas can also b e applied if non-linear
and/or time-varying controllers are used. The price is, of course, that the
estimation problems then become much more involved.
In the closed-lo op identication literature it has b een common to classify the
metho ds primarily based on how the nal estimates are computed (e.g. di-
rectly or indirectly using multi-step estimation schemes), and then the main
groupings have b een into \direct" and \indirect" metho ds. This should not,
however, be confused with the classication (1)-(3) which is based on the
assumptions made on the feedback.
3 Technical Assumptions and Notation
The basis of all identication is the data set
Z
N
= fu(1)y(1):::u(N )y(N )g (2)
consisting of measured input-output signals u(t) and y (t), t = 1:::N. We
will make the following assumptions regarding how this data set was generated.
Assumption 1 The true system S is linear with p outputs and m inputs and
given by
y (t)=G
0
(q )u(t)+v (t)
v (t)=H
0
(q )e(t)
(3)
where fe(t)g (p  1) is a zero-mean white noise process with covariance matrix
6


0
, and bounded moments of order 4+ , some >0,andH
0
(q ) is an inversely
stable, monic lter.
For some of the analytic treatment we shall assume that the input fu(t)g is
generated as
u(t)=r (t) ; K (q )y (t) (4)
where K (q ) is a linear regulator of appropriate dimensions and where the
reference signal fr (t)g is indep endent of fv (t)g.
This assumption of a linear feedbacklaw is rather restrictive and in general we
shall only assume that the input u(t) satises the following milder condition
(cf. 20], condition S3):
Assumption 2 The input u(t) is given by
u(t)=k (t y
t
u
t;1
r(t)) (5)
where y
t
=y (1):::y(t)], etc., and where where the reference signal fr (t)g is
a given quasi-stationary signal, independent of fv (t)g and k is a given deter-
ministic function such that the closed-loop system (3) and (5) is exponential ly
stable, which we dene as fol lows: For each t s t  s there exist random
variables y
s
(t) u
s
(t), independent of r

s
and v
s
but not independent of r
t
and
v
t
, such that

E ky (t) ; y
s
(t)k
4
<C
t;s
(6)

E ku(t) ; u
s
(t)k
4
<C
t;s
(7)
for some C < 1 <1. In addition, k is such that either G
0
(q ) or k contains
a delay.
Here we have used the notation


Ef (t)= lim
N !1
1
N
N
X
t=1
Ef (t) (8)
The concept of quasi-stationarity is dened in, e.g., 22].
If the feedback is indeed linear and given by (4) then Assumption 2 means
that the closed-lo op system is asymptotically stable.
Let us now intro duce some further notation for the linear feedback case. By
combining the equations (3) and (4) we have that the closed-lo op system is
y (t)=S
0
(q )G
0
(q )r (t)+S
0
(q )v (t) (9)
7
where S
0
(q ) is the sensitivity function,
S
0
(q )=(I + G
0
(q )K (q ))

;1
(10)
This is also called the output sensitivity function. With
G
c0
(q )=S
0
(q )G
0
(q )andH
c0
(q )=S
0
(q )H
0
(q ) (11)
we can rewrite (9) as
y (t)=G
c0
(q )r (t)+v
c
(t)
v
c
(t)=H
c0
(q )e(t)
(12)
In closed lo op the input can be written as
u(t)=S

i
0
(q )r (t) ; S
i
0
(q )K (q )v (t) (13)
= S
i
0
(q )r (t) ; K (q )S
0
(q )v (t) (14)
The input sensitivity function S
i
0
(q ) is dened as
S
i
0
(q )=(I + K (q )G
0
(q ))
;1
(15)
The sp ectrum of the input is (cf. (14))

u
= S
i
0


r
(S
i
0
)

+ KS
0

v
S

0
K

(16)
where 
r
is the sp ectrum of the reference signal and 
v
= H
0

0
H

0
the noise
sp ectrum. Sup erscript  denotes complex conjugate transp ose. Here we have

suppressed the arguments ! and e
i!
which also will be done in the sequel
whenever there is no risk of confusion. Similarly,we will also frequently sup-
press the arguments t and q for notational convenience. We shall denote the
two terms in (16)

r
u
= S
i
0

r
(S
i
0
)

(17)
and

e
u
= KS
0

v
S


0
K

= S
i
0
K 
v
K

(S
i
0
)

(18)
The cross sp ectrum between u and e is

ue
= ;KS
0
H
0

0
= ;S
i
0
KH
0


0
(19)
The cross sp ectrum between e and u will b e denoted 
eu
, 
eu
=

ue
.
Occasionally we shall also consider the case where the regulator is linear as in
(4) but contains an unknown additive disturbance d:
u(t)=r (t) ; K (q )y (t)+d(t) (20)
8
The disturbance d could for instance be due to imp erfect knowledge of the
true regulator: Supp ose that the true regulator is given by
K
true
(q )=K (q )+
K
(q ) (21)
for some (unknown) function 
K
. In this case the signal d = ;
K
y . Let 
rd
(
dr

) denote the cross sp ectrum b etween r and d (d and r ), whenever it exists.
4 Prediction Error Identication
In this section we shall review some basic results on prediction error metho ds,
that will b e used in the sequel. See App endix A and 22] for more details.
4.1 The Method
We will work with a mo del structure M of the form
y (t)=G(q )u(t)+H (q )e(t) (22)
G will b e called the dynamics mo del and H the noise mo del. We will assume
that either G (and the true system G
0
) or the regulator k contains a delay
and that H is monic. The parameter vector  ranges over a set D
M
which is
assumed compact and connected. The one-step-ahead predictor for the mo del
structure (22) is 22]
^y (tj )=H
;1
(q )G(q )u(t)+(I ; H
;1
(q ))y (t) (23)
The prediction errors are
"(t  )=y (t) ; ^y (tj )=H
;1
(q )(y (t) ; G(q )u(t)) (24)
Given the mo del (23) and measured data Z
N
we determine the prediction
error estimate through
^


N
= arg min
2D
M
V
N
( Z
N
) (25)
V
N
( Z
N
)=
1
N
N
X
t=1
"
T
F
(t  )
;1
"
F
(t  ) (26)
"
F

(t  )=L(q )"(t  ) (27)
Here  is a symmetric, p ositive denite weighting matrix and L a (p ossibly
parameter-dep endent) monic prelter that can be used to enhance certain
9
frequency regions. It is easy to see that
"
F
(t  )=L(q )H
;1
(q )(y (t) ; G(q )u(t)) (28)
Thus the eect of the prelter L can be included in the noise mo del and
L(q ) = 1 can b e assumed without loss of generality. This will b e done in the
sequel.
Wesay that the true system is contained in the mo del set if, for some 
0
2 D
M
,
G(q 
0
)=G
0
(q ) H (q 
0
)=H
0
(q ) (29)
This will also be written S 2 M. The case when the true noise prop erties
cannot be correctly describ ed within the mo del set but where there exists a


0
2 D
M
such that
G(q 
0
)=G
0
(q ) (30)
will b e denoted G
0
2G.
4.2 Convergence
Dene the average criterion

V ( ) as

V ( )=

E"
T
(t  )
;1
"(t  ) (31)
Then we have the following result (see, e.g., 20, 22]):
^

N
! D
c

= arg min
2D
M

V ( ) with probability (w. p.) 1asN !1 (32)
In case the input-output data can be describ ed by (3) we have the following
characterization of D
c
(G

is short for G(q ), etc.):
D
c
= arg min
2D
M
Z

;
tr
h

;1
H
;1

2
6
4
(G

0
; G

)

(H
0
; H

)

3
7
5

2
6
4

u

ue

eu

0
3
7
5
2

6
4
(G
0
; G

)

(H
0
; H

)

3
7
5
H
;

i
d!
(33)
This is shown in App endix A.1. Note that the result holds regardless of the
nature of the regulator, as long as Assumptions 1 and 2 hold and the signals
involved are quasistationary.
From (33) several conclusions regarding the consistency of the metho d can b e
drawn. First of all, supp ose that the parameterization of G and H is suciently
exible so that S 2 M. If this holds then the metho d will in general give
10

consistent estimates of G
0
and H
0
if the exp eriment is informative 22], which
means that the matrix


0
=
2
6
4

u

ue

eu

0
3
7
5
(34)
is positive denite for all frequencies. (Note that it will always be p ositive
semi-denite since it is a sp ectral matrix.) Supp ose for the moment that the
regulator is linear and given by (4). Then we can factorize the matrix in (34)
as



0
=
2
6
4
I 
ue

;1
0
0 I
3
7
5
2
6
4

r
u
0
0 
0
3
7
5
2
6
4

I 0

;1
0

eu
I
3
7
5
(35)
The left and right factors in (35) always have full rank, hence the condition
b ecomes that 
r
u
is p ositive denite for all frequencies (
0
is assumed p ositive
denite). This is true if and only if 
r
is p ositive denite for all frequencies
(which is the same as to say that the reference signal is p ersistently excit-
ing 22]). In the last step we used the fact that the analytical function S
i
0
(cf.
(17)) can be zero at at most nitely many points. The conclusion is that for
linear feedbackwe should use a p ersistently exciting, external reference signal,
otherwise the exp eriment maynotb e informative.
The general condition is that there should not b e a linear, time-invariant, and

noise-free relationship b etween u and y . With an external reference signal this
is automatically satised but it should also be clear that informative closed-
lo op exp eriments can also b e guaranteed if weswitch between dierent linear
regulators or use a non-linear regulator. For a more detailed discussion on this
see, e.g., 15] and 22].
4.3 Asymptotic Variance of Black Box Transfer Function Estimates
Consider the mo del (23). Intro duce
T (q )=vec
h
G(q ) H (q )
i
(36)
(The vec-op erator stacks the columns of its argument on top of each other in
a vector. A more formal denition is given in App endix A.2.) Supp ose that
the vector  can be decomp osed so that
 =
h

1
 
2
 ::: 
n
i
T
dim 
k
= s dim = n  s (37)
11
We shall call n the order of the mo del (23) and we allow n to tend to innity

as N tends to innity. Supp ose also that T in (36) has the following shift
structure:
@
@
k
T (q )=q
;k+1
@
@
1
T (q ) (38)
It should b e noted that most p olynomial-typ e mo del structures, including the
ones studied in this pap er, satisfy this shift structure. Thus (38) is a rather
weak assumption.
More background material including further technical assumptions and addi-
tional notation can b e found in App endix A.2. For brevity reasons weherego
directly to the main result ( denotes the Kronecker pro duct):
Covvec
h
^
T
N
(e
i!
)
i

n
N
2

6
4

u
(! ) 
ue
(! )

eu
(! ) 
0
3
7
5
;T
 
v
(! ) (39)
The covariance matrix is thus prop ortional to the mo del order divided n by
the number of data N . This holds asymptotically as both n and N tend to
innity.Inop en lo op we have 
ue
=0 and
Covvec
h
^
G
N
(e
i!

)
i

n
N

u
(! )
;T
 
v
(! ) (40)
Covvec
h
^
H
N
(e
i!
)
i

n
N

;1
0
 
v
(! ) (41)

Notice that (40) for the dynamics mo del holds also in case the noise mo del is
xed (e.g. H (q )=I ).
4.4 Asymptotic Distribution of the Parameter Vector Estimates
If S 2Mthen
^

N
! 
0
as N !1under reasonable conditions (e.g., 

0
> 0,
see 22]). Then, if  = 
0
,
p
N (
^

N
; 

) 2 AsN (0P

) (42a)
P

=
h


E(t 
0
)
;1
0

T
(t 
0
)
i
;1
(42b)
where  is the negative gradientoftheprediction errors " with resp ect to  .
In this pap er we will restrict to the SISO case when discussing the asymptotic
distribution of the parameter vector estimates for notational convenience. For
ease of reference wehave in App endix A.3 stated a variant of (42) as a theorem.
12
5 Closed-lo op Identication in the Prediction Error Framework
5.1 The Direct Approach
The direct approach amounts to applying a prediction error metho d directly
to input-output data, ignoring p ossible feedback. In general one works with
mo dels of the form (cf. (23))
^y (tj )=H
;1
(q )G(q )u(t)+(I ; H
;1
(q ))y (t) (43)
The direct metho d can thus b e formulated as in (25)-(27). This coincides with

the standard (op en-lo op) prediction error metho d 22, 26]. Since this metho d
is well known we will not go into any further details here. Instead we turn to
the indirect approach.
5.2 The Indirect Approach
5.2.1 General
Consider the linear feedback set-up (4). If the regulator K is known and r is
measurable, we can use the indirect identication approach. It consists of two
steps:
(1) Identify the closed-lo op system from the reference signal r to the output
y .
(2) Determine the op en-lo op system parameters from the closed-lo op mo del
obtained in step 1, using the knowledge of the regulator.
Instead of identifying the closed-lo op system in the rst step one can identify
any closed-lo op transfer function, for instance the sensitivity function. Here
we will concentrate on metho ds in which the closed-lo op system is identied.
The mo del structure is
y (t)=G
c
(q )r(t)+ H

(q )e(t) (44)
Here G
c
(q ) is a mo del of the closed-lo op system. We have also included a
xed noise mo del H

which is standard in the indirect metho d. Often H

(q )=
1 is used, but we can also use H


as a xed prelter to emphasize certain
frequency ranges. The corresp onding one-step-ahead predictor is
^y (tj )=H
;1

(q )G
c
(q )r(t)+ (I ; H
;1

(q ))y (t) (45)
Note that estimating  in (45) is an \op en-lo op" problem since the noise
and the reference signal are uncorrelated. This implies that we may use any
13
identication metho d that works in op en lo op to nd this estimate of the
closed-lo op system. For instance, we can use output error mo dels with xed
noise mo dels (prelters) and still guarantee consistency (cf. Corollary 4 b elow).
Consider the closed-lo op system (cf. (12))
y (t)=G
c0
(q )r (t)+v
c
(t) (46)
Supp ose that we in the rst step have obtained an estimate
^
G
cN
(q ) =
G

c
(q
^

N
)ofG
c0
(q ). In the second step we then have to solve the equation
^
G
cN
(q )=(I +
^
G
N
(q )K (q ))
;1
^
G
N
(q ) (47)
using the knowledge of the regulator. The exact solution is
^
G
N
(q )=
^
G
cN
(q )(I ;

^
G
cN
(q )K (q ))
;1
(48)
Unfortunately this gives a high-order estimate
^
G
N
in general { typically the
order of
^
G
N
will b e equal to the sum of the orders of
^
G
cN
and K .Ifwe attempt
to solve (47) with the additional constraint that
^
G
N
should be of a certain
(low) order we end up with an over-determined system of equations whichcan
be solved in many ways, for instance in a weighted least-squares sense. For
metho ds, like the prediction error metho d, that allow arbitrary parameteriza-
tions G
c

(q ) it is natural to let the parameters  relate to prop erties of the
op en-lo op system G,sothat in the rst step we should parameterize G
c
(q )
as
G
c
(q )=(I + G(q )K (q ))
;1
G(q ) (49)
This was apparently rst suggested as an exercise in 22]. This parameteriza-
tion has also b een analyzed in 8].
The choice (49) will of course have the eect that the second step in the
indirect metho d b ecomes sup eruous, since we directly estimate the op en-
lo op parameters. The choice of parameterization may thus be imp ortant for
numerical and algebraic issues, but it do es not aect the statistical prop erties
of the estimated transfer function:
As long as the parameterization describes the same set
of G, the resulting transfer function
^
G wil l be the same,
regard less of the parameterizations.
5.2.2 The Dual-Youla Parameterization
A nice and interesting idea is to use the so called dual-Youla parameterization
that parameterizes all systems that are stabilized by a certain regulator K
14
(see, e.g., 32]). To present the idea, the concept of coprime factorizations of
transfer functions is required: A pair of stable transfer functions N D 2 R H
1
is a right coprime factorization (rcf ) of G if G = ND

;1
and there exist stable
transfer functions X Y 2 R H
1
such that XN + YD = I . The dual-Youla
parameterization nowworks as follows. Let G
nom
with rcf (N D)beany system
that is stabilized by K with rcf (X Y ). Then, as R ranges over all stable
transfer functions, the set
n
G : G(q )=(N (q )+Y (q )R(q ))(D(q ) ; X (q )R(q ))
;1
o
(50)
describ es all systems that are stabilized by K . The unique value of R that
corresp onds to the true plant G
0
is given by
R
0
(q )=Y
;1
(q )(I + G
0
(q )K (q ))
;1
(G
0
(q ) ; G

nom
(q ))D (q ) (51)
This idea can now b e used for identication (see, e.g., 16], 17], and 6]): Given
an estimate
^
R
N
of R
0
we can compute an estimate of G
0
as
^
G
N
(q )=(N (q )+Y (q )
^
R
N
(q ))(D (q ) ; X (q )
^
R
N
(q ))
;1
(52)
Using the dual-Youla parameterization we can write
G
c
(q )=(N (q )+Y (q )R(q ))(D(q )+X (q )Y

;1
(q )N (q ))
;1
(53)
, (N (q )+Y (q )R(q ))M (q ) (54)
With this parameterization the identication problem
y (t)=G
c
(q )r(t)+ v
c
(t) (55)
b ecomes
z (t)=R(q )x(t)+v
c
(t) (56)
where
z (t)=Y
;1
(q )(y (t) ; N (q )M (q )r (t)) (57)
x(t)=M (q )r (t) (58)
v
c
(t)=Y
;1
(q )v
c
(t) (59)
Thus the dual-Youla metho d is a sp ecial parameterization of the general indi-
rect metho d. This means, esp ecially, that the statistical prop erties of the re-
sulting estimates for the indirect metho d remain unaected for the dual-Youla

metho d. The main advantage of this metho d is of course that the obtained
estimate
^
G
N
is guaranteed to be stabilized by K , which clearly is a nice fea-
ture. A drawback is that this metho d typically will give high-order estimates
|typically the order will b e equal to the sum of the orders of G
nom
and
^
R
N
.
15
In this pap er we will use (49) as the generic indirect metho d. Before turning to
the joint input-output approach, let us pause and study an interesting variant
of the parameterization idea used in (49) which will provide useful insights
into the connection between the direct and indirect metho ds.
5.3 A Formal Connection Between Direct and Indirect Methods
The noise mo del H in a linear dynamics mo del structure has often turned
out to be a key to interpretation of dierent \metho ds". The distinction be-
tween the mo dels/\metho ds" ARX, ARMAX, output error, Box-Jenkins, etc.,
is entirely explained by the choice of the noise mo del. Also the practically im-
p ortant feature of preltering is equivalenttochanging the noise mo del. Even
the choice b etween minimizing one- or k -step prediction errors can b e seen as
a noise mo del issue. See, e.g., 22] for all this.
Therefore it should not come as a surprise that also the distinction between
the fundamental approaches of direct and indirect identication can be seen
as achoice of noise mo del.

The idea is to parameterize G as G(q ) and H as
H (q )=(I + G(q )K (q ))H
1
(q ) (60)
Wethus link the noise mo del to the dynamics mo del. There is nothing strange
with that: So do ARX and ARMAX mo dels. Although this parameterization
is p erfectly valid, it must still b e pointed out that the choice (60) is a highly
sp ecialized one using the knowledge of K . Also note that this particular pa-
rameterization scales H
1
with the inverse mo del sensitivity function. (Similar
parameterizations have b een suggested in, e.g., 4, 9, 18].)
Now, the predictor for
y (t)=G(q )u(t)+H (q )e(t) (61)
is
^y (tj )=H
;1
(q )G(q )u(t)+(I ; H
;1
(q ))y (t) (62)
Using u = r ; Ky and inserting (60) we get
^y (tj )=H
;1
1
(q )(I + G(q )K (q ))
;1
G(q )r(t)+ (I ; H
;1
1
(q ))y (t) (63)

But this is exactly the predictor also for the closed-lo op mo del structure
y (t)=(I + G(q )K (q ))
;1
G(q )r(t)+H
1
(q )e(t) (64)
16
and hence the two approaches are equivalent. We formulate this result as a
lemma:
Lemma 3 Suppose that the input is generated as in (4) and that both u and
r are measurable and that the linear regulator K is known. Then, applying a
prediction error method to (61) with H parameterized as in (60), or to (64)
gives identical estimates
^

N
. This holds regardless of the parameterization of
G and H
1
.
Among other things, this shows that we can use any theory develop ed for the
direct approach (allowing for feedback) to evaluate prop erties of the indirect
approach, and vice versa. It can also be noted that the particular choice of
noise mo del (60) is the answer to the question how H should b e parameterized
in the direct metho d in order to avoid the bias in the G-estimate in the case of
closed-lo op data, even if the true noise characteristics is not correctly mo deled.
This is shown in 23].
5.4 The Joint Input-output Approach
The third main approach to closed-lo op identication is the so called joint
input-output approach. The basic assumption in this approach is that the in-

put is generated using a regulator of a certain form, e.g., (4). Exact knowledge
of the regulator parameters is not required |anadvantage over the indirect
metho d where this is a necessity.
Supp ose that the regulator is linear and of the form (20). The output y and
input u then ob ey
2
6
4
y (t)
u(t)
3
7
5
=
2
6
4
G
c0
(q )
S
i
0
(q )
3
7
5
r (t)+
2
6

4
S
0
(q )H
0
(q ) G
0
(q )S
i
0
(q )
;K (q )S
0
(q )H
0
(q ) S
i
0
(q )
3
7
5
2
6
4
e(t)
d(t)
3
7
5

(65)
, G
0
(q )r (t)+H
0
(q )
2
6
4
e(t)
d(t)
3
7
5
(66)
Consider the parameterized mo del structure
2
6
4
y (t)
u(t)
3
7
5
= G(q )r(t)+ H(q )
2
6
4
e(t)
d(t)

3
7
5
(67)
=
2
6
4
G
yr
(q )
G
ur
(q )
3
7
5
r (t)+
2
6
4
H
ye
(q ) H
yd
(q )
H
ue
(q ) H
ud

(q )
3
7
5
2
6
4
e(t)
d(t)
3
7
5
(68)
17
where the parameterizations of the indicated transfer functions, for the time
b eing, are not further sp ecied. Dierent parameterization will lead to dierent
metho ds, as we shall see. Previously wehave used a slightly dierent notation,
e.g., G
yr
(q ) = G
c
(q ). This will also be done in the sequel but for the
momentwe will use the \generic" mo del structure (68) in order not to obscure
the presentation with to o many parameterization details.
The basic idea in the joint input-output approach is to compute estimates
of the op en-lo op system using estimates of the dierent transfer functions in
(68). We can for instance use
^
G
N

yu
(q )=
^
G
N
yr
(q )(
^
G
N
ur
(q ))
;1
(69)
The rationale b ehind this choice is the relation G
0
= G
c0
(S
i
0
)
;1
(cf. (65)).
We may also include a prelter, F
r
, for r in the mo del (68), so that instead
of using r directly, x = F
r
r is used. The op en-lo op estimate would then be

computed as
^
G
N
yu
(q )=
^
G
N
yx
(q )(
^
G
N
ux
(q ))
;1
(70)
where
^
G
N
yx
(q )and
^
G
N
ux
(q ) are the estimated transfer functions from x to y and
u, resp ectively. In the ideal case the use of the prelter F

r
will not aect the
results since G
0
= G
c0
F
;1
r
(S
i
0
F
;1
r
)
;1
regardless of F
r
, but in practice, with
noisy data, the lter F
r
can be used to improve the quality of the estimates.
This idea really go es back to Akaike 1] who showed that sp ectral analysis
of closed-lo op data should be p erformed as follows: Compute the sp ectral
estimates (SISO)
^
G
N
yx

=
^

N
yx
^

N
x
and
^
G
N
ux
=
^

N
ux
^

N
x
(71)
where the signal x is correlated with y and u but uncorrelated with the noise
e | a standard choice is x = r . The op en-lo op system maynow b e estimated
as
^
G
N

yu
=
^
G
N
yx
^
G
N
ux
=
^

N
yx
^

N
ux
(72)
In 30] a parametric variant of this idea was presented. This will be briey
discussed in Section 5.4.1 b elow. A problem when using parametric metho ds
is that the resulting op en-lo op estimate will typically be of high-order: from
(70) it follows that the order of
^
G
N
will generically b e equal to the sum of the
orders of the factors
^

G
yx
(q ) and
^
G
ux
(q ). This problem is similar to the one
we are faced with in the indirect metho d, where we noted that solving for the
op en-lo op estimate in (47) typically gives high-order estimates. However in
the joint input-output metho d (70) this can be circumvented, at least in the
18
SISO case, by parameterizing the factors G
yx
(q )andG
ux
(q ) in a common-
denominator form. The nal estimate will then b e the ratio of the numerator
p olynomials in the original mo dels.
Another way of avoiding this problem is to consider parameterizations of the
form
G
yx
(q )=G
uy
(q )G
ux
(q ) (73a)
G
ux
(q )=G

ux
(q ) (73b)
This way we will have control over the order of the nal estimate through
the factor G
uy
(q ). If we disregard the correlation b etween the noise sources
aecting y and u we may rst estimate the  -parameters using u and r and
then estimate the -parameters using y and r , keeping  -parameters xed to
their estimated values. Such ideas will b e studied in Sections 5.4.2-5.4.3 b elow.
We note that also H
0
(q )contains all the necessary information ab out the op en-
lo op system so that we can compute consistent estimates of G
0
even when no
reference signal is used (r = 0). As an example wehavethat
^
G
N
yu
=
^
H
N
yd
(
^
H
N
ud

)
;1
is a consistent estimate of G
0
.Such metho ds were studied in 15]. See also 3]
and 26].
5.4.1 The Coprime Factor Identication Scheme
Consider the metho d (70). Recall that this metho d gives consistent estimates
of G
0
regardless of the prelter F
r
(F
r
is assumed stable). Can this freedom
in the choice of prelter F
r
b e utilized to give a b etter nite sample b ehavior?
In 30] it is suggested to cho ose F
r
so as to make
^
G
yx
(q ) and
^
G
ux
(q ) normalized
coprime. The main advantage with normalized coprime factors is that they

form a decomp osition of the op en-lo op estimate
^
G
N
in minimal order, stable
factors. There is a problem, though, and that is that the prop er prelter F
r
that would make
^
G
yx
(q )and
^
G
ux
(q ) normalized coprime is not known a priori.
To cop e with this problem, an iterative pro cedure is prop osed in 30] in which
the prelter F
(i)
r
at step i is up dated using the current mo dels
^
G
(i)
yx
(q ) and
^
G
(i)
ux

(q ) giving F
(i+1)
r
, and so on. The hop e is, of course, that the iterations
lead to normalized coprime factors
^
G
yx
(q )and
^
G
ux
(q ).
5.4.2 The Two-stage Method
The next joint input-output metho d we will study is the two-stage metho d 28].
It is usually presented using the following two steps (cf. (73)):
19
(1) Identify the sensitivity function S
i
0
using, e.g., an output error mo del
^u(tj)=S
i
(q )r(t) (74)
(2) Construct the signal ^u =
^
S
i
N
r and identify the op en-lo op system using

the output error mo del
^y (tj )=G(q )^u(t)=G(q )
^
S
i
N
(q )r (t) (75)
p ossibly using a xed prelter
Note that in the rst step a high-order mo del of S
0
can be used since we in
the second step can control the op en-lo op mo del order indep endently. Hence
it should be p ossible to obtain very go o d estimates of the true sensitivity
function in the rst step, esp ecially if the noise level is low. Ideally
^
S
i
N
! S
i
0
as N ! 1 and ^u will be the noise free part of the input signal. Thus in
the ideal case, the second step will be an \op en-lo op" problem so that an
output error mo del with xed noise mo del (prelter) can be used without
lo osing consistency. See, e.g., Corollary 4 b elow. This result requires that the
disturbance term d in (66) is uncorrelated with r .
5.4.3 The Projection Method
We will now present another metho d for closed-lo op identication that is in-
spired byAkaike's idea (71)-(72) whichmaybeinterpreted as a way to corre-
late out the noise using the reference signal as instrumental variable. In form

it will b e similar to the two-stage metho d but the motivation for the metho ds
will b e quite dierent. Moreover, as we shall see the feedback need not b e lin-
ear for this metho d to give consistent estimates. The metho d will b e referred
to as the pro jection metho d 10, 11].
This metho d uses the same two steps as the two-stage metho d. The only
dierence to the two-stage metho d is that in the rst step one should use a
doubly innite, non-causal FIR lter instead. The mo del can be written
^u(tj)=S
i
(q )r(t)=
M
X
k=;M
s
i
k
r (t ; k ) M !1 M = o(N ) (76)
This may b e viewed as a \pro jection" of the input u onto the reference signal
r and will result in a partitioning of the input u into two asymptotically
uncorrelated parts:
u(t)= ^u(t)+ ~u(t) (77)
20
where
^u(t)=
^
S
i
N
(q )r (t) (78)
~u(t)=u(t) ; ^u(t) (79)

We say asymptotical ly uncorrelated b ecause ^u will always dep end on e since
u do es and S
i
(q ) is estimated using u. However, as this is a second order
eect it will b e neglected.
The advantage over the two-stage metho d is that the pro jection metho d gives
consistent estimates of the op en-lo op system regardless of the feedback, even
with a xed prelter (cf. Corollary 4 b elow). A consequence of this is that
with the pro jection metho d we can use a xed prelter to shap e the bias
distribution of the G-estimate at will, just as in the op en-lo op case with output
error mo dels.
Further comments on the pro jection metho d:
 Here we chose to p erform the pro jection using a non-causal FIR lter but
this step may also be p erformed non-parametrically as in Akaike's cross-
sp ectral metho d (71)-(72).
 In practice M can be chosen rather small. Go o d results are often obtained
even with very mo dest values of M . This is clearly illustrated in Example
5 b elow.
 Finally,itwould also b e p ossible to pro ject b oth the input u and the output
y onto r in the rst step. This is in fact what is done in (71)-(72).
5.5 Unifying Framework for All Joint Input-Output Methods
Consider the joint system (66) and assume, for the moment, that d is white
noise with covariance matrix 
d
indep endent of e. The maximum likeliho o d
estimates of G
0
and H
0
are computed as

min
2D
M
1
N
N
X
t=1
2
6
4
y (t) ; ^y (tj )
u(t) ; ^u(tj )
3
7
5
T
2
6
4

0
0
0 
d
3
7
5
;1
2

6
4
y (t) ; ^y (tj )
u(t) ; ^u(tj)
3
7
5
(80)
where
2
6
4
^y (tj )
^u(tj )
3
7
5
= H
;1
(q )G(q )r(t)+ (I ; H
;1
(q ))
2
6
4
y (t)
u(t)
3
7
5

(81)
The parameterizations of G and H can b e arbitrary. Consider the system (66).
This system was obtained using the assumption that the noise e aect the
21
op en-lo op system only and the disturbance d aect the regulator only. The
natural way to parameterize H in order to reect these assumptions in the
mo del is
H(q )=
2
6
4
(I + G(q )K (q ))
;1
H (q ) G(q )(I + K (q )G(q ))
;1
;K (q )(I + G(q )K (q ))
;1
H (q ) (I + K (q )G(q ))
;1
3
7
5
(82)
The inverse of H is
H
;1
(q )=
2
6
4

H
;1
(q ) ;H
;1
(q )G(q )
K (q ) I
3
7
5
(83)
Thus, with G parameterized as
G(q )=
2
6
4
G(q )(I + K (q )G(q ))
;1
(I + K (q )G(q ))
;1
3
7
5
(84)
we get
2
6
4
^y (tj )
^u(tj)
3

7
5
=
2
6
4
0
I
3
7
5
r (t)+
2
6
4
I ; H
;1
(q ) H
;1
(q )G(q )
;K (q ) 0
3
7
5
2
6
4
y (t)
u(t)
3

7
5
(85)
or
^y (tj )=(I ; H
;1
(q ))y (t)+H
;1
(q )G(q )u(t) (86)
^u(tj )=r (t) ; K (q )y (t) (87)
The predictor (86) is the same as for the direct metho d (cf. (23)), while (87) is
the natural predictor for estimating the regulator K . The maximum likeliho o d
estimate b ecomes
min
2D
M
1
N
(
N
X
t=1
(y (t) ; ^y (tj ))
T

;1
0
(y (t) ; ^y (tj ))
+
N

X
t=1
(u(t) ; ^u(tj ))
T

;1
d
(u(t) ; ^u(tj ))
)
(88)
We may thus view the joint input-output metho d as a combination of di-
rect identication of the op en-lo op system and a direct identication of the
regulator. Note that this holds even in the case where r (t) = 0. If the pa-
rameterization of the regulator K is indep endent of the one of the system
the two terms in (88) can be minimized separately which decouples the two
identication problems.
22
Let us return to (66). The natural output-error predictor for the joint system
is
2
6
4
^y (tj )
^u(tj )
3
7
5
=
2
6

4
G(q )
I
3
7
5
(I + K (q )G(q ))
;1
r (t) (89)
According to standard op en-lo op prediction error theory this will give con-
sistent estimates of G
0
and K indep endently of 
0
and 
d
, as long as the
parameterization of G and K is suciently exible. See Corollary 4 below.
With S
i
(q )=(I + K (q )G(q ))
;1
the mo del (89) can b e written
2
6
4
^y (tj )
^u(tj )
3
7

5
=
2
6
4
G(q )
I
3
7
5
S
i
(q )r(t) (90)
Consistency can b e guaranteed if the parameterization of S
i
(q )contains the
true input sensitivity function S
i
0
(q ) (and similarly that G(q ) = G
0
(q ) for
some  2 D
M
). See, e.g., Corollary 4 b elow. If
G(q )=G(q ) S
i
(q )=S
i
(q )  =

2
6
4


3
7
5
(91)
the maximum likeliho o d estimate b ecomes
min
2D
M
1
N
(
N
X
t=1
(y (t) ; G(q )S
i
(q )r (t))
T

;1
0
(y (t) ; G(q )S
i
(q )r (t))
+

N
X
t=1
(u(t) ; S
i
(q )r (t))
T

;1
d
(u(t) ; S
i
(q )r (t))
)
(92)
Now, if 
0
= 
0
 I and 
d
= 
d
 I , 
d
! 0 then the maximum likeliho o d
estimate will b e identical to the one obtained with the two-stage or pro jection
metho ds. This is true because for small 
d
the  -parameters will minimize

1
N
N
X
t=1
(u(t) ; S
i
(q )r (t))
T
(u(t) ; S
i
(q )r (t)) (93)
regardless of the -parameters, which then will minimize
1
N
N
X
t=1
(y (t) ; G(q )S
i
(q )r (t))
T
(y (t) ; G(q )S
i
(q )r (t)) (94)
The weighting matrices 
0
= 
0
 I and 

d
= 
d
 I may be included in
the noise mo dels (prelters). Thus the two-stage metho d (and the pro jection
metho d) may be viewed as sp ecial cases of the general joint input-output
approach corresp onding to sp ecial choices of the noise mo dels. In particular,
23
this means that any result that holds for the joint input-output approach
without constraints on the noise mo dels, holds for the two-stage and pro jection
metho ds as well. We will for instance use this fact in Corollary 6 below.
6 Convergence Results for the Closed-lo op Identication Metho ds
Let us now apply the result of Theorem A.1 to the sp ecial case of closed-lo op
identication. In the following we will suppress the arguments ! e
i!
, and  .
Thus we write G
0
as short for G
0
(e
i!
) and G

as short for G(e
i!
), etc. The
subscript  is included to emphasize the parameter dep endence.
Corollary 4 Consider the situation in Theorem A.1. Then, for
(1) the direct approach, with a model structure

y (t)=G(q )u(t)+H (q )e(t) (95)
where G(q ) is such that either G
0
(q ) and G(q ) or the control ler k in
(5) contains a delay, we have that
^

N
! D
c
= arg min
2D
M
Z

;
tr
h

;1
n
H
;1

(G
0
+ B

; G


)
u
(G
0
+ B

; G

)

+(H
0
; H

)(
0
; 
eu

;1
u

ue
)(H
0
; H

)

]H

;

oi
d! w. p. 1 as N !1
(96)
where
B

=(H
0
; H

)
eu

;1
u
(97)
(
eu
is the cross spectrum between e and u.)
(2) the indirect approach, if the model structure is
y (t)=G
c
(q )r(t)+ H

(q )e(t) (98)
and the input is given by (20), we have that
^


N
! D
c
= arg min
2D
M
Z

;
tr
h

;1
H
;1

(G
c0
 D ; G
c
)
r

 (G
c0
 D ; G
c
)

(H


)
;
i
d! w. p. 1 as N !1 (99)
where
D =(I +
dr

;1
r
) (100)
(
dr
is the cross spectrum between d and r .)
24
(3) the joint input-output approach,
(a) if the model structure is
2
6
4
y (t)
u(t)
3
7
5
=
2
6
4

G
c
(q )
S
i
(q )
3
7
5
r (t)+
2
6
4
H
1
(q ) 0
0 H
2
(q )
3
7
5
2
6
4
e(t)
d(t)
3
7
5

(101)
and the input is given by (20), we have that
^

N
! D
c
= arg min
2D
M
Z

;
tr
h

;1
n
H
;1
1
(G
c0
 D ; G
c
)
r

 (G
c0

 D ; G
c
)

H
;
1
+ H
;1
2
(S
i
0
 D ; S
i

)
r
(S
i
0
 D ; S
i

)

H
;
2
oi

d!
w. p. 1 as N !1 (102)
where D is given by (100).
(b) if the model structure is
y (t)=G(q )^u(t)+H

(q )e(t)=G(q )
^
S
i
N
(q )r (t)+H

(q )e(t)
(103)
and the input is given by (20), we have that
^

N
! D
c
= arg min
2D
M
Z

;
tr
h


;1
H
;1

(G
c0
 D ; G

^

S )
r

 (G
c0
 D ; G

^

S )

(H

)
;
i
d! w. p. 1 as N !1 (104)
where D is given by (100).
The pro of is given in App endix B.1.
Remarks:

(1) Let us rst discuss the result for the direct approach, expression (96). If
the parameterization of the mo del G and the noise mo del H is exible
enough so that for some 
0
2 D
M
, G(q 
0
)=G
0
(q ) and H (q 
0
)=H
0
(q )
(i.e. S 2 M) then

V (
0
) = 
0
so D
c
= f
0
g (provided this is a unique
minimum) under reasonable conditions. See, e. g., the discussion following
Eq. (33) and Theorem 8.3 in 22].
(2) If the system op erates in op en lo op, so that 
ue

=0, then the bias term
B

=0 regardless of the noise mo del H and the limit mo del will b e the
best p ossible approximation of G
0
(in a frequency weighting norm that
dep ends on H and 
u
). A consequence of this is that in op en lo op we
can use xed noise mo dels and still get consistent estimates of G
0
pro-
vided that G(q 
0
)=G
0
(q ) for some  2 D
M
and certain identiability
conditions hold. See, e.g., Theorem 8.4 in 22].
25