Tài liệu 16 Validation, Testing, and Noise Modeling pdf

Tugnait, J.K. “Validation, Testing, and Noise Modeling”
Digital Signal Processing Handbook
Ed. Vijay K. Madisetti and Douglas B. Williams
Boca Raton: CRC Press LLC, 1999
c

1999byCRCPressLLC
16
Validation, Testing, and Noise
Modeling
Jitendra K. Tugnait
Auburn University
16.1 Introduction
16.2 Gaussianity, Linearity, and Stationarity Tests
Gaussianity Tests
•
Linearity Tests
•
Stationarity Tests
16.3 Order Selection, Model Validation, and Confidence Intervals
Order Selection
•
Model Validation
•
Confidence Intervals
16.4 Noise Modeling
Generalized Gaussian Noise
•
MiddletonClassANoise
•
Stable

Noise Distribution
16.5 Concluding Remarks
References
16.1 Introduction
Linear parametric models of stationary random processes, whether signal or noise, have been found
to be useful in a wide variety of signal processing tasks such as signal detection, estimation, filtering,
and classification, and in a wide variety of applications such as digital communications, automatic
control, radar and sonar, and other engineering disciplines and sciences. A general representation of
a linear discrete-time stationary signal x(t) is given by
x(t) =
∞

i=0
h(i)(t − i)
(16.1)
where {(t)} is a zero-mean, i.i.d. (independent and identically distributed) random sequence
with finite variance, and {h(i), i ≥ 0} is the impulse response of the linear system such that

∞
i=−∞
h
2
(i) < ∞. Much effort has been expended on developing approaches to linear model
fitting given a single measurement record of the signal (or noisy signal). Parsimonious parametric
models such as AR (autoregressive), MA (moving average), ARMA or state-space, as opposed to
impulse response modeling, have been popular together with the assumption of Gaussianity of the
data.
Define
H(q)=
∞


i=0
h(i)q
−i
(16.2)
where q
−1
is the backward shift operator (i.e., q
−1
x(t) = x(t − 1), etc.). If q is replaced with the
complex variable z, then H(z) is the Z-transform of {h(i)}, i.e., it is the system transfer function.
c

1999 by CRC Press LLC
Using (16.2), (16.1) may be rewritten as
x(t) = H(q)(t).
(16.3)
Fittinglinear models tothemeasurementrecordrequiresestimationof H(q), orequivalentlyof{h(i)}
(without observing {(t)} ). Typically H(q)is parameterized by a finite number of parameters, say
by the parameter vector θ
(M)
of dimension M. For instance, an AR model representation of order
M means that
H
AR
(q; θ
(M)
) =
1
1 +


M
i=1
a
i
q
−i
,θ
(M)
= (a
1
,a
2
,···,a
M
)
T
.
(16.4)
This reduces the number of estimated parameters from a “large” number to M.
In this section several aspects of fitting models such as (16.1)to(16.3) to the given measurement
record are considered. These aspects are (see also Fig. 16.1):
• Is the model of the type (16.1) appropriate to the given record? This requires testing for
linearity and stationarity of the data.
• Linear Gaussian models have long been dominant both for signals as well as for noise pro-
cesses. Assumption of Gaussianity allows implementation of statistically efficient param-
eter estimators such as maximum likelihoodestimators. A Gaussian processis completely
characterized by its second-order statistics (autocorrelation function or, equivalently, its
power spectral density). Since the power spectrum of {x(t)} of (16.1)isgivenby
S

xx
(ω) = σ
2

|H(e
jω
)|
2
,σ
2

= E{
2
(t)},
(16.5)
one cannot determine the phase of H(e
jω
) independent of |H(e
jω
)|. Determination
of the true phase characteristic is crucial in several applications such as blind equaliza-
tion of digital communications channels. Use of higher-order statistics allows one to
uniquely identify nonminimum-phase parametric models. Higher-order cumulants of
Gaussian processes vanish, hence, if the data are stationary Gaussian, a minimum-phase
(ormaximum-phase) model isthe“best” that onecan estimate. Therefore, another aspect
considered in this section is testing for non-Gaussianity of the given record.
• If the data are Gaussian, one may fit models based solely upon the second-order statistics
of the data —else useof higher-orderstatistics in addition toor in lieu of the second-order
statistics is indicated, particularly if the phase of the linear system is crucial. In either case,
one typically fits a model H(q; θ

(M)
) by estimating the M unknown parameters through
optimization of some cost function. In practice, (the model order) M is unknown and its
choice has a significant impact on the quality of the fitted model. In this section another
aspect of the model-fitting problem considered is that of order selection.
• Having fittedamodel H(q; θ
(M)
), onewouldalsoliketoknowhowgood arethe estimated
parameters? Typically this is expressed in terms of error bounds or confidence intervals
on the fitted parameters and on the corresponding model transfer function.
• Having fitted a model, a final step is that of model falsification. Is the fitted model an
appropriate representation of the underlying system? This is referred to variously as
model validation, model verification, or model diagnostics.
• Finally, variousmodelsof univariate noise pdf(probability density function)arediscussed
to complete the discussion of model fitting.
c

1999 by CRC Press LLC
FIGURE 16.1: Section outline (SOS — second-order statistics; HOS — higher-order statistics).
16.2 Gaussianity, Linearity, and Stationarity Tests
Given a zero-mean, stationary random sequence{x(t)}, its third-order cumulant function C
xxx
(i, k)
isgivenby[12]
C
xxx
(i, k) := E{x(t + i)x(t + k)x(t)}.
(16.6)
Its bispectrum B
xxx

(ω
1
,ω
2
) is defined as [12]
B
xxx
(ω
1
,ω
2
) =
∞

i=−∞
∞

k=−∞
C
xxx
(i, k)e
−j(ω
1
i+ω
2
k)
.
(16.7)
Similarly, its fourth-order cumulant function C
xxxx

(i,k,l)isgivenby[12]
C
xxxx
(i,k,l) := E{x(t)x(t + i)x(t + k)x(t + l)}
− E{x(t)x(t + i)}E{x(t + k)x(t + l)}
− E{x(t)x(t + k)}E{x(t + l)x(t + i)}
− E{x(t)x(t + l)}E{x(t + k)x(t + i)}.
(16.8)
Its trispectrum is defined as [12]
T
xxxx
(ω
1
,ω
2
,ω
3
) :=
∞

i=−∞
∞

k=−∞
∞

l=−∞
C
xxxx
(i,k,l)e

−j(ω
1
i+ω
2
k+ω
3
l)
.
(16.9)
c

1999 by CRC Press LLC
If {x(t)} obeys (16.1), then [12]
B
xxx
(ω
1
,ω
2
) = γ
3
H(e
jω
1
)H (e
jω
2
)H
∗
(e

j(ω
1
+ω
2
)
)
(16.10)
and
T
xxxx
(ω
1
,ω
2
,ω
3
) = γ
4
H(e
jω
1
)H (e
jω
2
)H (e
jω
3
)H
∗
(e

j(ω
1
+ω
2
+ω
3
)
)
(16.11)
where
γ
3
= C

(0, 0, 0) and γ
4
= C

(0, 0, 0, 0).
(16.12)
For Gaussian processes, B
xxx
(ω
1
,ω
2
) ≡ 0 and T
xxxx
(ω
1

,ω
2
,ω
3
) ≡ 0; equivalently, C
xxx
(i, k) ≡
0 and C
xxxx
(i,k,l)≡ 0. This forms a basis for testing Gaussianity of a given measurement record.
When {x(t)} is linear (i.e., it obeys (16.1)), then using (16.5) and (16.10),
|B
xxx
(ω
1
,ω
2
)|
2
S
xx
(ω
1
)S
xx
(ω
1
)S
xx
(ω

1
+ ω
2
)
=
γ
3
σ
6

= constant ∀ ω
1
,ω
2
,
(16.13)
and using (16.5) and (16.11),
|T
xxxx
(ω
1
,ω
2
,ω
3
)|
2
S
xx
(ω

1
)S
xx
(ω
1
)S
xx
(ω
3
)S
xx
(ω
1
+ ω
2
+ ω
3
)
=
γ
4
σ
8

= constant ∀ ω
1
,ω
2
,ω
3

.
(16.14)
The above two relations form a basis for testing linearity of a given measurement record. How
the tests are implemented depends upon the statistics of the estimators of the higher-order cumulant
spectra as well as that of the power spectra of the given record.
16.2.1 Gaussianity Tests
Suppose that the given zero-mean measurement record is of length N denoted by {x(t), t =
1, 2,···,N}. Suppose that the given sample sequence of length N is divided into K nonover-
lapping segments each of size N
B
samples so that N = KN
B
.LetX
(i)
(ω) denote the discrete Fourier
transform (DFT) of the ith block {x(t + (i − 1)N
B
), 1 ≤ t ≤ N
B
} (i = 1, 2,···,K)given by
X
(i)
(ω
m
) =
N
B
−1

l=0

x(l + 1 + (i − 1)N
B
)exp(−jω
m
l)
(16.15)
where
ω
m
=
2π
N
B
m, m = 0, 1,···,N
B
− 1.
(16.16)
Denote the estimate of the bispectrum B
xxx
(ω
m
,ω
n
) at bifrequency (ω
m
=
2π
N
B
m, ω

n
=
2π
N
B
n) as

B
xxx
(m, n), given by averaging over K blocks

B
xxx
(m, n) =
1
K
K

i=1

1
N
B
X
(i)
(ω
m
)X
(i)
(ω

n
)

X
(i)
(ω
m
+ ω
n
)

∗

,
(16.17)
where X
∗
denotes the complex conjugate of X. A principal domain of

B
xxx
(m, n) is the triangular
grid
D =

(m, n)|0 ≤ m ≤
N
B
2
, 0 ≤ n ≤ m, 2m + n ≤ N

B

.
(16.18)
Values of

B
xxx
(m, n) outside D can be inferred from that in D.
c

1999 by CRC Press LLC