advanced engineering mathematics – mathematics
Bạn đang xem bản rút gọn của tài liệu. Xem và tải ngay bản đầy đủ của tài liệu tại đây (87.22 KB, 35 trang )
<span class='text_page_counter'>(1)</span><div class='page_container' data-page=1>
<b>Introduction to Time Series Analysis. Lecture 8.</b>
1. Review: Linear prediction, projection in Hilbert space.
2. Forecasting and backcasting.
3. Prediction operator.
</div>
<span class='text_page_counter'>(2)</span><div class='page_container' data-page=2>
<b>Linear prediction</b>
Given X<sub>1</sub>, X<sub>2</sub>, . . . , Xn, the best linear predictor
X<sub>n+m</sub>n = α<sub>0</sub> +
n
X
i=1
αiXi
of Xn+m <b>satisfies the prediction equations</b>
E Xn+m − X<sub>n</sub>n<sub>+</sub><sub>m</sub>
= 0
E X<sub>n+m</sub> − X<sub>n+m</sub>n Xi
</div>
<span class='text_page_counter'>(3)</span><div class='page_container' data-page=3>
<b>Projection theorem</b>
If H is a Hilbert space,
M is a closed subspace of H,
and y ∈ H,
then there is a point P y ∈ M
<b>(the projection of</b> y <b>on</b> M)
satisfying
1. kP y − yk ≤ kw − yk
2. hy − P y, wi = 0
for w ∈ M.
<i>y</i>
<i>y−Py</i>
<i>Py</i>
</div>
<span class='text_page_counter'>(4)</span><div class='page_container' data-page=4>
<b>Projection theorem for linear forecasting</b>
Given 1, X<sub>1</sub>, X<sub>2</sub>, . . . , Xn ∈
r.v.s X : EX2 < ∞ ,
choose α<sub>0</sub>, α<sub>1</sub>, . . . , αn ∈ R
so that Z = α<sub>0</sub> + Pn<sub>i</sub><sub>=1</sub> αiXi minimizes E(Xn+m − Z)2.
Here, hX, Y i = E(XY ),
M = {Z = α<sub>0</sub> + Pn<sub>i</sub><sub>=1</sub> αiXi : αi ∈ R} = ¯sp{1, X1, . . . , Xn}, and
</div>
<span class='text_page_counter'>(5)</span><div class='page_container' data-page=5>
<b>Projection theorem: Linear prediction</b>
Let X<sub>n</sub>n<sub>+</sub><sub>m</sub> denote the best linear predictor:
kX<sub>n+m</sub>n − Xn+mk2 ≤ kZ − Xn+mk2 for all Z ∈ M.
The projection theorem implies the orthogonality
hX<sub>n</sub>n<sub>+</sub><sub>m</sub> − Xn+m, Zi = 0 for all Z ∈ M
⇔ hX<sub>n</sub>n<sub>+</sub><sub>m</sub> − Xn+m, Zi = 0 for all Z ∈ {1, X1, . . . , Xn}
⇔ E X
n
n+m − Xn+m
= 0
E X<sub>n</sub>n<sub>+</sub><sub>m</sub> − Xn+m
Xi
</div>
<span class='text_page_counter'>(6)</span><div class='page_container' data-page=6>
<b>Linear prediction</b>
<i>That is, the prediction errors (</i>X<sub>n</sub>n<sub>+</sub><sub>m</sub> − Xn+m<i>) are orthogonal to the</i>
<i>prediction variables (</i>1, X<sub>1</sub>, . . . , Xn<i>).</i>
</div>
<span class='text_page_counter'>(7)</span><div class='page_container' data-page=7>
<b>One-step-ahead linear prediction</b>
Write X<sub>n</sub>n<sub>+1</sub> = φn1Xn + φn2Xn−1+ · · · + φnnX1
Prediction equations: E (X<sub>n+1</sub>n − X<sub>n+1</sub>)Xi
= 0, for i = 1, . . . , n
⇔
n
X
j=1
φnjE (Xn+1−jXi) = E(Xn+1Xi)
⇔
n
X
j=1
φnjγ(i − j) = γ(i)
</div>
<span class='text_page_counter'>(8)</span><div class='page_container' data-page=8>
<b>One-step-ahead linear prediction</b>
Prediction equations: Γnφn = γn.
Γn =
γ(0) γ(1) · · · γ(n − 1)
γ(1) γ(0) γ(n − 2)
..
. . .. ...
γ(n − 1) γ(n − 2) · · · γ(0)
,
φn = (φn1, φn2, . . . , φnn)
′
, γn = (γ(1), γ(2), . . . , γ(n))
′
</div>
<span class='text_page_counter'>(9)</span><div class='page_container' data-page=9>
<b>Mean squared error of one-step-ahead linear prediction</b>
P<sub>n</sub>n<sub>+1</sub> = E Xn+1 − Xnn+1
2
= E X<sub>n+1</sub> − X<sub>n+1</sub>n X<sub>n+1</sub> − X<sub>n+1</sub>n
= E Xn+1 Xn+1 − Xnn+1
= γ(0) − E(φ′<sub>n</sub>XXn+1)
= γ(0) − γ<sub>n</sub>′ Γ−1
n γn,
where X = (Xn, Xn−1, . . . , X1)
′
</div>
<span class='text_page_counter'>(10)</span><div class='page_container' data-page=10>
<b>Mean squared error of one-step-ahead linear prediction</b>
Variance is reduced:
P<sub>n</sub>n<sub>+1</sub> = E Xn+1 − Xnn+1
2
= γ(0) − γ<sub>n</sub>′ Γ−1
n γn
= Var(Xn+1) − Cov(Xn+1, X)Cov(X, X)
−1
Cov(X, Xn+1)
= E (Xn+1 − 0)2 − Cov(Xn+1, X)Cov(X, X)
−1
Cov(X, Xn+1),
where X = (Xn, Xn−1, . . . , X1)
′
</div>
<span class='text_page_counter'>(11)</span><div class='page_container' data-page=11>
<b>Introduction to Time Series Analysis. Lecture 8.</b>
1. Review: Linear prediction, projection in Hilbert space.
2. Forecasting and backcasting.
3. Prediction operator.
</div>
<span class='text_page_counter'>(12)</span><div class='page_container' data-page=12>
<b>Backcasting: Predicting</b>
m
<b>steps in the past</b>
Given X<sub>1</sub>, . . . , Xn, we wish to predict X1−m for m > 0.
That is, we choose Z ∈ M = ¯sp{X<sub>1</sub>, . . . , Xn} to minimize kZ −X1−mk2.
The prediction equations are
hX<sub>1</sub>n−m − X1−m, Zi = 0 for all Z ∈ M
⇔ E X<sub>1</sub>n−m − X1−m
Xi
</div>
<span class='text_page_counter'>(13)</span><div class='page_container' data-page=13>
<b>One-step backcasting</b>
Write the least squares prediction of X<sub>0</sub> given X<sub>1</sub>, . . . , Xn as
X<sub>0</sub>n = φn1X1 + φn2X2 + · · · + φnnXn = φ
′
nX,
where the predictor vector is reversed: now X = (X<sub>1</sub>, . . . , Xn)
′
.
The prediction equations are
E((X<sub>0</sub>n − X<sub>0</sub>) Xi) = 0 for i = 1, . . . , n
⇔ E
n
X
j=1
φnjXj − X0
X<sub>i</sub>
= 0
⇔
n
X
j=1
φnjγ(j − i) = γ(i)
</div>
<span class='text_page_counter'>(14)</span><div class='page_container' data-page=14>
<b>One-step backcasting</b>
The prediction equations are
Γnφn = γn,
which is exactly the same as for forecasting, but with the indices of the
predictor vector reversed: X = (X<sub>1</sub>, . . . , Xn)
′
</div>
<span class='text_page_counter'>(15)</span><div class='page_container' data-page=15>
<b>Example: Forecasting AR(1)</b>
AR(1) model: Xt = φ1Xt−1 + Wt
linear prediction of X<sub>2</sub>: X<sub>2</sub>1 = φ<sub>11</sub>X<sub>1</sub>
Prediction equation: γ(0)φ<sub>11</sub> = γ(1)
= Cov(X<sub>0</sub>, X<sub>1</sub>)
= φ<sub>1</sub>γ(0)
</div>
<span class='text_page_counter'>(16)</span><div class='page_container' data-page=16>
<b>Example: Backcasting AR(1)</b>
AR(1) model: Xt = φ1Xt−1 + Wt
linear prediction of X<sub>0</sub>: X<sub>0</sub>1 = φ<sub>11</sub>X<sub>1</sub>
Prediction equation: γ(0)φ<sub>11</sub> = γ(1)
= Cov(X<sub>0</sub>, X<sub>1</sub>)
= φ<sub>1</sub>γ(0)
</div>
<span class='text_page_counter'>(17)</span><div class='page_container' data-page=17>
<b>Introduction to Time Series Analysis. Lecture 8.</b>
1. Review: Linear prediction, projection in Hilbert space.
2. Forecasting and backcasting.
3. Prediction operator.
</div>
<span class='text_page_counter'>(18)</span><div class='page_container' data-page=18>
<b>The prediction operator</b>
For random variables Y, Z<sub>1</sub>, . . . , Zn, define the
<b>best linear prediction of</b> Y <b>given</b> Z = (Z<sub>1</sub>, . . . , Zn)
′
as the operator P(·|Z) applied to Y :
P(Y |Z) = µY + φ
′
(Z − µZ)
with Γφ = γ,
where γ = Cov(Y, Z)
</div>
<span class='text_page_counter'>(19)</span><div class='page_container' data-page=19>
<b>Properties of the prediction operator</b>
<b>1. E</b>(Y − P(Y |Z)) = 0, E((Y − P(Y |Z))Z) = 0.
<b>2. E</b>((Y − P(Y |Z))2) = Var(Y ) − φ′
γ.
<b>3.</b> P(α<sub>1</sub>Y<sub>1</sub> + α<sub>2</sub>Y<sub>2</sub> + α<sub>0</sub>|Z) = α<sub>0</sub> + α<sub>1</sub>P(Y<sub>1</sub>|Z) + α<sub>2</sub>P(Y<sub>2</sub>|Z).
<b>4.</b> P(Zi|Z) = Zi.
</div>
<span class='text_page_counter'>(20)</span><div class='page_container' data-page=20>
<b>Example: predicting</b>
m
<b>steps ahead</b>
Write X<sub>n+m</sub>n = φ(<sub>n</sub>m<sub>1</sub>)Xn + φ(
m)
n2 Xn−1 + · · · + φ(
m)
nn X1
Γnφ(<sub>n</sub>m) = γ<sub>n</sub>(m),
with Γn = Cov(X, X),
γ<sub>n</sub>(m) = Cov(Xn+m, X)
= (γ(m), γ(m + 1), . . . , γ(m + n − 1))′
.
Also, E((Xn+m − X<sub>n</sub>n<sub>+</sub><sub>m</sub>)2) = γ(0) − φ(m)
′
</div>
<span class='text_page_counter'>(21)</span><div class='page_container' data-page=21>
<b>Introduction to Time Series Analysis. Lecture 8.</b>
1. Review: Linear prediction, projection in Hilbert space.
2. Forecasting and backcasting.
3. Prediction operator.
</div>
<span class='text_page_counter'>(22)</span><div class='page_container' data-page=22>
<b>Partial autocovariance function</b>
AR(1) model: Xt = φ1Xt−1 + Wt
γ(1) = Cov(X<sub>0</sub>, X<sub>1</sub>) = φ<sub>1</sub>γ(0)
γ(2) = Cov(X<sub>0</sub>, X<sub>2</sub>)
= Cov(X<sub>0</sub>, φ<sub>1</sub>X<sub>1</sub> + W<sub>2</sub>)
= Cov(X<sub>0</sub>, φ2<sub>1</sub>X<sub>0</sub> + φ<sub>1</sub>W<sub>1</sub> + W<sub>2</sub>)
= φ2<sub>1</sub>γ(0).
</div>
<span class='text_page_counter'>(23)</span><div class='page_container' data-page=23>
<b>Partial autocovariance function</b>
For AR(1) model: X<sub>2</sub>1 = φ<sub>1</sub>X<sub>1</sub>,
X<sub>0</sub>1 = φ<sub>1</sub>X<sub>1</sub>,
so Cov(X<sub>2</sub>1 − X<sub>2</sub>, X<sub>0</sub>1 − X<sub>0</sub>) = Cov(φ<sub>1</sub>X<sub>1</sub> − X<sub>2</sub>, φ<sub>1</sub>X<sub>1</sub> − X<sub>0</sub>)
= Cov(W<sub>2</sub>, φ<sub>1</sub>X<sub>1</sub> − X<sub>0</sub>)
</div>
<span class='text_page_counter'>(24)</span><div class='page_container' data-page=24>
<b>Partial autocorrelation function</b>
The Partial AutoCorrelation Function (PACF) of a stationary
time series {Xt} is
φ<sub>11</sub> = Corr(X<sub>1</sub>, X<sub>0</sub>) = ρ(1)
φhh = Corr(Xh − Xh
−1
h , X0 − X
h−1
0 ) for h = 2,3, . . .
This removes the linear effects of X<sub>1</sub>, . . . , Xh−1:
. . . , X−1, X0, X1, X2, . . . , Xh−1
| {z }
</div>
<span class='text_page_counter'>(25)</span><div class='page_container' data-page=25>
<b>Partial autocorrelation function</b>
The PACF φhh is also the last coefficient in the best linear prediction of
Xh+1 given X1, . . . , Xh:
Γhφh = γh X<sub>h</sub>h<sub>+1</sub> = φ
′
hX
</div>
<span class='text_page_counter'>(26)</span><div class='page_container' data-page=26>
<b>Example: Forecasting an AR(p)</b>
For Xt =
p
X
i=1
φiXt−i + Wt,
X<sub>n</sub>n<sub>+1</sub> = P(Xn+1|X1, . . . , Xn)
= P
p
X
i=1
φiXn+1−i + Wn+1|X1, . . . , Xn
!
=
p
X
i=1
φiP (Xn+1−i|X1, . . . , Xn)
p
</div>
<span class='text_page_counter'>(27)</span><div class='page_container' data-page=27>
<b>Example: PACF of an AR(p)</b>
For Xt =
p
X
i=1
φiXt−i + Wt,
X<sub>n+1</sub>n =
p
X
i=1
φiXn+1−i.
Thus, φhh =
φh if 1 ≤ h ≤ p
</div>
<span class='text_page_counter'>(28)</span><div class='page_container' data-page=28>
<b>Example: PACF of an invertible MA(q)</b>
For Xt =
q
X
i=1
θiWt−i + Wt, Xt = −
∞
X
i=1
πiXt−i + Wt,
X<sub>n+1</sub>n = P(X<sub>n+1</sub>|X1, . . . , Xn)
= P −
∞
X
i=1
πiXn+1−i + Wn+1|X1, . . . , Xn
!
= −
∞
X
i=1
πiP (Xn+1−<sub>i</sub>|X1, . . . , X<sub>n</sub>)
= −
n
X
πiXn+1−i −
∞
X
</div>
<span class='text_page_counter'>(29)</span><div class='page_container' data-page=29>
<b>ACF of the MA(1) process</b>
−100 −8 −6 −4 −2 0 2 4 6 8 10
0.2
0.4
0.6
0.8
1
θ/(1+θ2)
MA(1): X
</div>
<span class='text_page_counter'>(30)</span><div class='page_container' data-page=30>
<b>ACF of the AR(1) process</b>
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
φ|h|
AR(1): X
</div>
<span class='text_page_counter'>(31)</span><div class='page_container' data-page=31>
<b>PACF of the MA(1) process</b>
0 1 2 3 4 5 6 7 8 9 10
−0.2
0
0.2
0.4
0.6
0.8
1
MA(1): X
</div>
<span class='text_page_counter'>(32)</span><div class='page_container' data-page=32>
<b>PACF of the AR(1) process</b>
0.2
0.4
0.6
0.8
1
AR(1): X
</div>
<span class='text_page_counter'>(33)</span><div class='page_container' data-page=33>
<b>PACF and ACF</b>
<b>Model:</b> <b>ACF:</b> <b>PACF:</b>
AR(p) decays zero for h > p
MA(q) zero for h > q decays
</div>
<span class='text_page_counter'>(34)</span><div class='page_container' data-page=34>
<b>Sample PACF</b>
For a realization x<sub>1</sub>, . . . , xn of a time series,
<b>the sample PACF is defined by</b>
ˆ
φ<sub>00</sub> = 1
ˆ
φhh = last component of φˆh,
</div>
<span class='text_page_counter'>(35)</span><div class='page_container' data-page=35>
<b>Introduction to Time Series Analysis. Lecture 8.</b>
1. Review: Linear prediction, projection in Hilbert space.
2. Forecasting and backcasting.
3. Prediction operator.
</div>
<!--links-->
