Handbook of Economic Forecasting part 54 ppt

504 H. White
Table 8
Artificial data: Modified nonlinear least squares – Logistic
Summary goodness of fit
Hidden
units
Estimation
MSE
CV
MSE
Hold-out
MSE
Estimation
R-squared
CV
R-squared
Hold-out
R-squared
01.30098 1.58077 0.99298 0.23196 0.06679 0.06664
11.30013 1.49201 0.99851 0.23247 0.11919 0.06144
21.30000 1.50625 1.00046 0.23255 0.11079 0.05961
30.91397 1.10375 0.84768 0.46044 0.34840 0.20321
40.86988 1.05591 0.80838 0.48647 0.37665 0.24016
50.85581 1.03175 0.80328 0.49478 0.39091 0.24495
60.85010 1.01461 0.80021 0.49815 0.40102 0.24783
70.84517 1.00845 0.79558 0.50105 0.
40466 0.25219
80.83541 1.00419
∗
0.75910 0.50681 0.40718 0.28648
∗

90.80738 1.07768 0.75882 0.52336 0.36379 0.28674
10 0.79669 1.03882 0.73159 0.52967 0.38673 0.31233
11 0.79664 1.04495 0.73181 0.52971 0.38312 0.31213
12 0.79629 1.05454 0.72912 0.52991 0.37745 0.31466
13 0.79465 1.06053 0.72675 0.53088 0.37392 0.31688
14 0.78551 1.04599 0.71959 0.53628 0.38250 0.32361
15 0.78360 1.07676 0.72182 0.53740 0.36433 0.32152
16 0.76828 1.09929 0.70041 0.54645 0.
35103 0.34165
17 0.76311 1.08872 0.70466 0.54950 0.35727 0.33765
18 0.76169 1.11237 0.70764 0.55034 0.34332 0.33484
19 0.76160 1.13083 0.70768 0.55039 0.33242 0.33481
20 0.76135 1.13034 0.70736 0.55054 0.33271 0.33511
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
41 0.68366 1.14326 0.65124 0.59640 0.32508 0.38786
∧
results as good as those seen in Table 9. Nevertheless, we observe quite good perfor-
mance. The best CV MSE performance occurs with 50 hidden units, corresponding to a
respectable hold-out R
2
of 0.471. Moreover, CV MSE appears to be trending downward,
suggesting that additional terms could further improve performance.
Table 11 shows analogous results for the polynomial version of QuickNet. Again we
see that additional polynomial terms do not improve in-sample fit as rapidly as do the
ANN terms. We also again see the extremely erratic behavior of CV MSE, arising from
precisely the same source as before, rendering CV MSE useless for polynomial model
selection purposes. Interestingly, however, the hold-out R
2
of the better-performing
models isn’t bad, with a maximum value of 0.390. The challenge is that this model
could never be identified using CV MSE.
We summarize these experiments with the following remarks. Compared to the fa-
miliar benchmark of algebraic polynomials, the use of ANNs appears to offer the
ability to more quickly capture nonlinearities; and the alarmingly erratic behavior of
Ch. 9: Approximate Nonlinear Forecasting Methods 505
Table 9
Artificial data: QuickNet – Logistic
Summary goodness of fit
Hidden
units

Estimation
MSE
CV
MSE
Hold-out
MSE
Estimation
R-squared
CV
R-squared
Hold-out
R-squared
01.30098 1.58077 0.99298 0.23196 0.06679 0.06664
11.21467 1.44012 0.93839 0.28292 0.14983 0.11795
21.00622 1.16190 0.86194 0.40598 0.31407 0.18982
30.87534 1.02132 0.81237 0.48324 0.39706 0.23641
40.82996 0.94456 0.71615 0.51004 0.44238 0.32685
50.79297 0.91595 0.67986 0.53187 0.45927 0.36096
60.76903 0.89458 0.67679 0.54600 0.47188 0.36384
70.72552 0.84374 0.62678 0.57169 0.
50190 0.41085
80.68977 0.81835 0.58523 0.59280 0.51689 0.44991
90.66635 0.80670 0.55821 0.60662 0.52376 0.47530
10 0.63501 0.79596 0.55889 0.62512 0.53010 0.47466
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
29 0.49063 0.62450 0.49194 0.71036 0.63133 0.53759
30 0.47994 0.61135 0.49207 0.71667 0.63909 0.53747
31 0.47663 0.61293 0.48731 0.71862 0.63816 0.
54195
32 0.47217 0.60931 0.48532 0.72125 0.64029 0.54382
33 0.46507 0.59559
∗
0.48624 0.72545 0.64840 0.54295
∗
34 0.46105 0.59797 0.48943 0.72782 0.64699 0.53995
35 0.45784 0.60633 0.48603 0.72971 0.64206 0.54315
36 0.45480 0.60412 0.48765 0.73151 0.64336 0.54163
37 0.45401 0.60424 0.48977 0.73198 0.64329 0.53964
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
49 0.43136 0.64107 0.47770 0.74535 0.62154 0.55098
∧
CV MSE for polynomials definitely serves as a cautionary note. In our controlled en-
vironment, QuickNet, either with logistic cdf or ridgelet activation function, performs
well in rapidly extracting a reliable nonlinear predictive relationship. Naïve NLS is bet-
ter than a simple linear forecast, as is modified NLS. The lackluster performance of
the latter method does little to recommend it, however. Nor do the computational com-
plexity, modest performance, and somewhat erratic behavior of naïve NLS support its
routine use. The relatively good performance of QuickNet seen here suggests it is well
worth application, further study, and refinement.
7.2. Explaining forecast outcomes

In this section we illustrate application of the explanatory taxonomy provided in Sec-
tion 6.2. For conciseness, we restrict attention to examining the out-of-sample pre-
dictions made with the CV MSE-best nonlinear forecasting model corresponding to
506 H. White
Table 10
Artificial data: QuickNet – Ridgelet
Summary goodness of fit
Hidden
units
Estimation
MSE
CV
MSE
Hold-out
MSE
Estimation
R-squared
CV
R-squared
Hold-out
R-squared
01.30098 1.58077 0.99298 0.23196 0.06679 0.06664
11.22724 1.43273 0.87504 0.27550 0.15419 0.17750
21.17665 1.39998 0.83579 0.30537 0.17352 0.21439
31.09149 1.30517 0.75993 0.35564 0.22949 0.28570
40.98380 1.22154 0.75393 0.41922 0.27887 0.29134
50.88845 1.13625 0.73192 0.47550 0.32922 0.31203
60.85571 1.03044 0.71145 0.49483 0.39168 0.33126
70.83444 1.02006 0.69144 0.50739 0.
39781 0.35008

80.81150 0.98440 0.64753 0.52093 0.41886 0.39135
90.78824 0.99417 0.67279 0.53467 0.41309 0.36761
10 0.77323 0.96053 0.70196 0.54352 0.43295 0.34018
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
27 0.56099 0.82982 0.55838 0.66882 0.51012 0.47515
28 0.55073 0.80588 0.53706 0.67488 0.52425 0.49518
29 0.54414 0.82178 0.51536 0.67877 0.51487 0.
51559
∧
30 0.54103 0.81704 0.53229 0.68060 0.51766 0.49967

31 0.53545 0.80240 0.53970 0.68390 0.52630 0.49271
32 0.53222 0.80171 0.55080 0.68581 0.52671 0.48227
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
47 0.47173 0.75552 0.56503 0.72152 0.55398 0.46890
48 0.46773 0.74575 0.55972 0.72388 0.55975 0.47389
49 0.46531 0.73767 0.55892 0.72530 0.56452 0.47464
50 0.
46239 0.73640
∗
0.56272 0.72703 0.56527 0.47107

∗
Table 9. This is an ANN with logistic cdf activation and 33 hidden units, achieving a
hold-out R2 of 0.5493.
The first step in applying the taxonomy is to check whether the forecast function
ˆ
f is
monotone or not. A simple way to check this is to examine the first partial derivatives
of
ˆ
f with respect to the predictors, x, which we write D
ˆ
f = (D
1
ˆ
f, ,D
9
ˆ
f),D
j
ˆ
f ≡
∂
ˆ
f/∂x
j
. If any of these derivatives change sign over the estimation or hold-out sam-
ples, then
ˆ
f is not monotone. Note that this is a necessary and not sufficient condition
for monotonicity. In particular, if

ˆ
f is nonmonotone over regions not covered by the
data, then this simple check will not signal nonmonotonicity. In such cases, further ex-
ploration of the forecast function may be required. In Table 12 we display summary
statistics including the minimum and maximum values of the elements of D
ˆ
f over the
hold-out sample. The nonmonotonicity is obvious from the differing signs of the max-
ima and minima. We are thus in Case II of the taxonomy.
Ch. 9: Approximate Nonlinear Forecasting Methods 507
Table 11
Artificial data: QuickNet – Polynomial
Summary goodness of fit
Hidden
units
Estimation
MSE
CV
MSE
Hold-out
MSE
Estimation
R-squared
CV
R-squared
Hold-out
R-squared
01.30098 1.58077 0.99298 0.23196 0.06679 0.06664
11.20939 1.42354
∗

0.96230 0.28604 0.15962 0.09547
∗
21.13967 1.54695 0.93570 0.32720 0.08676 0.12048
31.09208 2.26962 0.93592 0.35529 −0.33987 0.12027
41.03733 2.14800 0.89861 0.38761 −0.26807 0.15534
51.00583 4.26301 0.87986 0.40621 −1.51666 0.17297
60.98113 4.01405 0.86677 0.42079 −1.36969 0.18527
70.95294 3.34959 0.85683 0.43743 −0.97743 0.19461
80.93024 3.88817 0.86203 0.45083 −1.29538 0
.18972
90.90701 4.35370 0.84558 0.46455 −1.57020 0.20519
10 0.89332 3.45478 0.84267 0.47263 −1.03953 0.20792
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
41 0.61881 15.22200 0.67752 0.63468 −7.98627 0.36316
42 0.61305 14.85660 0.67194 0.63809 −7.77057 0.36841
43 0.60894 15.82990 0.67470 0.64051 −8.34518 0.36581
44 0.60399 15
.23310 0.67954 0.64344 −7.99283 0.36126
45 0.60117 13.93220 0.67664 0.64510 −7.22489 0.36399
46 0.59572 15.58510 0.66968 0.64832 −8.20064 0.37053
47 0.59303 15.63730 0.66592 0.64990 −8.23149 0.37407
48 0.58907 16.39490 0.65814 0.65224 −8.67874 0.38137
49 0.58607 15.33290 0.65483 0.65402 −8.05178 0.38448
50 0.58171 16.08150 0.64922 0.65659 −8.49372 0.
38976
∧
Table 12
Hold-out sample: Summary statistics
Summary statistics for derivatives of prediction function
x
1
x
2
x
3
x
4
x
5
x

6
x
7
x
8
x
9
mean −8.484 7.638 3.411 −7.371 −9.980 −8.375 0.538 −5.512 −12.267
sd 17.353 19.064 6.313 13.248 18.843 10.144 8.918 7.941 17.853
min −155.752 −5.672 −6.355 −115.062 −168.269 −93.124 −9.563 −68.698 −156.821
max 3.785 166.042 51.985 2.084 4.331 0.219 70.
775 3.177 2.722
Summary statistics for predictions and predictors
Prediction x
1
x
2
x
3
x
4
x
5
x
6
x
7
x
8
x

9
mean −0.111 0.046 0.048 0.043 0.580 0.582 0.586 1.009 1.010 1.013
sd 0.775 0.736 0.738 0.743 0.455 0.456 0.457 0.406 0.406 0.408
min −2.658 −1.910 −1.910 −1.910 0.000 0.000 0.000 0.000 0.000 0.000
max 3.087 2.234 2.234 2.234 2.234 2.234 2.234 2.182 2.182 2.182
508 H. White
Table 13
Hold-out sample: Actual and standardized values of predictors
Order stat. Prediction x
1
x
2
x
3
x
4
x
5
x
6
x
7
x
8
x
9
253 3.087 −1.463 −0.577 −0.835 1.463 0.577 0.835 1.686 0.896 1.132
−2.051 −0.847 −1.183 1.944 −0.010 0.545 1.668 −0.281 0.290
252 1.862 0.014 1.240 0.169 0.014 1.240 0.169 0.303 1.089 0.339
−0.043 1.615 0.169 −1.243 1.444 −0.913 −1.738 0.193 −

1.654
251 1.750 −0.815 −0.315 −1.043 0.815 0.315 1.043 1.093 0.523 1.583
−1.170 −0.492 −1.463 0.517 −0.584 1.001 0.208 −1.198 1.397
2 −2.429 −0.077 0.167 0.766 0.077 0.167 0.766 1.008 1.965 0.786
−0.167 0.161 0.973 −1.107 −0.909 0.394
−0.003 2.349 −0.559
1 −2.658 −0.762 −0.014 1.146 0.762 0.014 1.146 1.483 0.634 1.194
−1.097 −0.084 1.484 0.400 −1.244 1.225 1.167 −0.925 0.444
Note: Actual values in first row, standardized values in second row.
Thenextstepistoexamine
ˆ
δ =
ˆ
f −Y for remarkable values, that is, values that are
either unusual or extreme. When one is considering a single out-of-sample prediction,
the comparison must be done relative to the estimation data set. Here, however, we have
a hold-out sample containing a relatively large number of observations, so we can con-
duct our examination relative to the hold-out data. For this, it is convenient to sort the
hold-out observations in order of
ˆ
δ (equivalently
ˆ
f ) and examine the distances between
the order statistics. Large values for these distances identify potentially remarkable val-
ues. In this case we have that the largest values between order statistics occur only in
the tail, so the only remarkable values are the extreme values. We are thus dealing with
cases II.C.2, II.D.3, or II.D.4.
The taxonomy resolves the explanation once we determine whether the predictors are
remarkable or not, and if remarkable in what way (unusual or extreme). The comparison
data must be the estimation sample if there are only a few predictions, but given the rel-

atively large hold-out sample here, we can assess the behavior of the predictors relative
to the hold-out data. As mentioned in Section 6.2, a quick and dirty way to check for
remarkable values is to consider each predictor separately. A check of the order statis-
tic spacings for the individual predictors does not reveal unusual values in the hold-out
data, so in Table 13 we present information bearing on whether or not the values of
the predictors associated with the five most extreme
ˆ
f ’s are extreme. We provide both
actual values and standardized values, in terms of (hold-out) standard deviations from
the (hold-out) mean.
The largest and most extreme prediction (
ˆ
f = 3.0871) has associated predictor val-
ues that are plausibly extreme: x
1
and x
4
are approximately two standard deviations
from their hold-out sample means, and x
7
is at 1.67 standard deviations. This first
example therefore is plausibly case II.D.4: an extreme forecast explained by extreme
predictors. This classification is also plausible for examples 2 and 4, as predictors x
2
,
Ch. 9: Approximate Nonlinear Forecasting Methods 509
x
7
, and x
9

are moderately extreme for example 2 and predictor x
8
is extreme for ex-
ample 4. On the other hand, the predictors for examples 3 and 5 do not appear to be
particularly extreme. As we earlier found no evidence of unusual nonextreme predic-
tors, these examples are plausibly classified as case II.C.2: extreme forecasts explained
by nonmonotonicities.
It is worth emphasizing that the discussion of this section is not definitive, as we
have illustrated our explanatory taxonomy using only the most easily applied tools.
This is certainly relevant, as these tools are those most accessible to practitioners, and
they afford a simple first cut at understanding particular outcomes. They are also help-
ful in identifying cases for which further analysis, and in particular application of more
sophisticated tools, such as those involving multivariate density estimation, may be war-
ranted.
8. Summary and concluding remarks
In this chapter, we have reviewed key aspects of forecasting using nonlinear models.
In economics, any model, whether linear or nonlinear, is typically misspecified. Con-
sequently, the resulting forecasts provide only an approximation to the best possible
forecast. As we have seen, it is possible, at least in principle, to obtain superior approx-
imations to the optimal forecast using a nonlinear approach. Against this possibility lie
some potentially serious practical challenges. Primary among these are computational
difficulties, the dangers of overfit, and potential difficulties of interpretation.
As we have seen, by focusing on models linear in the parameters and nonlinear in
the predictors, it is possible to avoid the main computational difficulties and retain the
benefits of the additional flexibility afforded by predictor nonlinearity. Further, use of
nonlinear approximation, that is, using only the more important terms of a nonlinear se-
ries, can afford further advantages. There is a vast range of possible methods of this sort.
Choice among these methods can be guided to only a modest degree by a priori knowl-
edge. The remaining guidance must come from the data. Specifically, careful applica-
tion of methods for controlling model complexity, such as Geisser’s (1975) delete-d

cross-validation for cross-section data or Racine’s (2000) hv-block cross-validation for
time-series data, is required in order to properly address the danger of overfit. A care-
ful consideration of the interpretational issues shows that the difficulties there lie not
so much with nonlinear models as with their relative unfamiliarity; as we have seen,
the interpretational issues are either identical or highly parallel for linear and nonlinear
approaches.
In our discussion here, we have paid particular attention to nonlinear models con-
structed using artificial neural networks (ANNs), using these to illustrate both the
challenges to the use of nonlinear methods and effective solutions to these challenges.
In particular, we propose QuickNet, an appealing family of algorithms for constructing
nonlinear forecasts that retains the benefits of using a model nonlinear in the predictors
while avoiding or mitigating the other challenges to the use of nonlinear forecasting
510 H. White
models. In our limited example with artificial data, we saw some encouraging perfor-
mance from QuickNet, both in terms of computational speed relative to more standard
ANN methods and in terms of resulting forecasting performance relative to more fa-
miliar polynomial approximations. In our real-world data example, we also saw that
building useful forecasting models can be quite challenging. There is no substitute for
a thorough understanding of the strengths and weaknesses of the methods applied; nor
can the importance of a thorough understanding of the domain being modeled be over-
emphasized.
Acknowledgements
The author is grateful for the comments and suggestions of the editors and three anony-
mous referees, which have led to substantial improvements over the initial draft. Any
errors remain the author’s responsibility.
References
Akaike, H. (1970). “Statistical predictor identification”. Annals of the Institute of Statistical Mathematics 22,
203–217.
Akaike, H. (1973). “Information theory and an extension of the likelihood principle”. In: Petrov, B.N., Csaki,
F. (Eds.), Proceedings of the Second International Symposium of Information Theory. Akademiai Kiado,

Budapest.
Allen, D. (1974). “The relationship between variable selection and data augmentation and a method for pre-
diction”. Technometrics 16, 125–127.
Benjamini, Y., Hochberg, Y. (1995). “Controlling the false discovery rate: A practical and powerful approach
to multiple testing”. Journal of the Royal Statistical Society, Series B 57, 289–300.
Burman, P., Chow, E., Nolan, D. (1994). “A cross validatory method for dependent data”. Biometrika 81,
351–358.
Bierens, H. (1990). “A consistent conditional moment test of functional form”. Econometrica 58, 1443–1458.
Candes, E. (1998). “Ridgelets: Theory and applications”. Ph.D. Dissertation, Department of Statistics, Stan-
ford University.
Candes, E. (1999a). “Harmonic analysis of neural networks”. Applied and Computational Harmonic Analy-
sis 6, 197–218.
Candes, E. (1999b). “On the representation of mutilated Sobolev functions”. SIAM Journal of Mathematical
Analysis 33, 2495–2509.
Candes, E. (2003). “Ridgelets: Estimating with ridge functions”. Annals of Statistics 33, 1561–1599.
Chen, X. (2005). “Large sample sieve estimation of semi-nonparametric models”. C.V. Starr Center Working
Paper, New York University.
Coifman, R., Wickhauser, M. (1992). “Entropy based algorithms for best basis selection”. IEEE Transactions
on Information Theory 32, 712–718.
Craven, P., Wahba, G. (1979). “Smoothing noisy data with spline functions: Estimating the correct degree of
smoothing by the method of generalized cross-validation”. Numerical Mathematics 31, 377–403.
Daubechies, I. (1988). “Orthonormal bases of compactly supported wavelets”. Communications in Pure and
Applied Mathematics 41, 909–996.
Daubechies, I. (1992). Ten Lectures on Wavelets. SIAM, Philadelphia, PA.
Ch. 9: Approximate Nonlinear Forecasting Methods 511
Dekel, S., Leviatan, D. (2003). “Adaptive multivariate piecewise polynomial approximation”. SPIE Proceed-
ings 5207, 125–133.
DeVore, R. (1998). “Nonlinear approximation”. Acta Numerica 7, 51–150.
DeVore, R., Temlyakov, V.(1996). “Some remarks on greedy algorithms”. Advances in Computational Math-
ematics 5, 173–187.

Gallant, A.R. (1981). “On the bias in flexible functional forms and an essentially unbiased form: The Fourier
flexible form”. Journal of Econometrics 15, 211–245.
Geisser, S. (1975). “The predictive sample reuse method with applications”. Journal of the American Statis-
tical Association 70, 320–328.
Gencay, R., Selchuk, F., Whitcher, B. (2001). An Introduction to Wavelets and other Filtering Methods in
Finance and Econometrics. Academic Press, New York.
Gonçalves, S., White, H. (2005). “Bootstrap standard error estimation for linear regressions”. Journal of the
American Statistical Association 100, 970–979.
Hahn, J. (1998). “On the role of the propensity score in efficient semiparametric estimation of average treat-
ment effects”. Econometrica 66, 315–331.
Hannan, E., Quinn, B. (1979). “The determination of the order of an autoregression”. Journal of the Royal
Statistical Society, Series B 41, 190–195.
Hendry, D.F., Krolzig, H M. (2001). Automatic Econometric Model Selection with PcGets. Timberlake Con-
sultants Press, London.
Hirano, K., Imbens, G. (2001). “Estimation of causal effects using propensity score weighting: An application
to right heart catheterization”. Health Services & Outcomes Research 2, 259–278.
Jones, L.K. (1992). “A simple lemma on greedy approximation in Hilbert space and convergence rates for
projection pursuit regression and neural network training”. Annals of Statistics 20, 608–613.
Jones, L.K. (1997). “The computational intractability of training sigmoid neural networks”. IEEE Transac-
tions on Information Theory 43, 167–173.
Kim, T., White, H. (2003). “Estimation, inference, and specification testing for possibly misspecified quantile
regressions”. In: Fomby, T., Hill, R.C. (Eds.), Maximum Likelihood Estimation of Misspecified Models:
Twenty Years Later. Elsevier, New York, pp. 107–132.
Koenker, R., Basset, G. (1978). “Regression quantiles”. Econometrica 46, 33–50.
Kuan, C M., White, H. (1994). “Artificial neural networks: An econometric perspective”. Econometric Re-
views 13, 1–92.
Lehmann, E.L., Romano, J.P. (2005). “Generalizations of the familywise error rate”. Annals of Statistics 33,
1138–1154.
Lendasse, A., Lee, J., de Bodt, E., Wertz, V., Verleysen, M. (2003). “Approximation by radial basis function
networks: Application to option pricing”. In: Lesage, C., Cottrell, M. (Eds.), Connectionist Approaches

in Economics and Management Sciences. Kluwer, Amsterdam, pp. 203–214.
Li, Q., Racine, J. (2003). “Nonparametric estimation of distributions with categorical and continuous data”.
Journal of Multivariate Analysis 86, 266–292.
Mallows, C. (1973). “Some comments on C
p
”. Technometrics 15, 661–675.
Pérez-Amaral, T., Gallo, G.M., White, H. (2003). “A flexible tool for model building: The RElevant Trans-
formation of the Inputs Network Approach (RETINA)”. Oxford Bulletin of Economics and Statistics 65,
821–838.
Pérez-Amaral, T., Gallo, G.M., White, H. (2005). “A comparison of complementary automatic modeling
methods: RETINA and PcGets”. Econometric Theory 21, 262–277.
Pisier G. (1980). “Remarques sur un resultat non publie de B. Maurey”. Seminaire d’Analyse Fonctionelle
1980–81, Ecole Polytechnique, Centre de Mathematiques, Palaiseau.
Powell, M. (1987). “Radial basis functions for multivariate interpolation: A review”. In: Mason, J.C., Cox,
M.G. (Eds.), Algorithms for Approximation. Oxford University Press, Oxford, pp. 143–167.
Racine, J. (1997). “Feasible cross-validatory model selection for general stationary processes”. Journal of
Applied Econometrics 12, 169–179.
Racine, J. (2000). “A consistent cross-validatory method for dependent data: hv-block cross-validation”. Jour-
nal of Econometrics 99, 39–61.
512 H. White
Rissanen, J. (1978). “Modeling by shortest data description”. Automatica 14, 465–471.
Schwarz, G. (1978). “Estimating the dimension of a model”. Annals of Statistics 6, 461–464.
Shao, J. (1993). “Linear model selection by cross-validation”. Journal of the American Statistical Associa-
tion 88, 486–495.
Shao, J. (1997). “An asymptotic theory for linear model selection”. Statistica Sinica 7, 221–264.
Stinchcombe, M., White, H. (1998). “Consistent specification testing with nuisance parameters present only
under the alternative”. Econometric Theory 14, 295–325.
Stone, M. (1974). “Cross-validatory choice and assessment of statistical predictions”. Journal of the Royal
Statistical Society, Series B 36, 111–147.
Stone, M. (1976). “An asymptotic equivalence of choice of model by cross-validation and Akaike’s criterion”.

Journal of the Royal Statistical Society, Series B 39, 44–47.
Sullivan, R., Timmermann, A., White, H. (1999). “Data snooping, technical trading rule performance, and the
bootstrap”. Journal of Finance 54, 1647–1692.
Swanson, N.,White, H. (1995). “A model selection approach to assessing the information in the term structure
using linear models and artificial neural networks”. Journal of Business and Economic Statistics 13, 265–
276.
Teräsvirta, T. (2006). “Forecasting economic variables with nonlinear models”. In: Elliott, G., Granger,
C.W.J., Timmermann, A. (Eds.), Handbook of Economic Forecasting. Elsevier, Amsterdam. Chapter 8
in this volume.
Timmermann, A., Granger, C.W.J. (2004). “Efficient market hypothesis and forecasting”. International Jour-
nal of Forecasting 20, 15–27.
Trippi, R., Turban, E. (1992). Neural Networks in Finance and Investing: Using Artificial Intelligence to
Improve Real World Performance. McGraw-Hill, New York.
Vu, V.H. (1998). “On the infeasibility of training neural networks with small mean-squared error”. IEEE
Transactions on Information Theory 44, 2892–2900.
Wahba, G. (1990). Spline Models for Observational Data. SIAM, Philadelphia, PA.
Wahba, G., Wold, S. (1975). “A completely automatic French curve: Fitting spline functions by cross-
validation”. Communications in Statistics 4, 1–17.
Westfall, P., Young, S. (1993). Resampling-Based Multiple Testing: Examples and Methods for P -Value
Adjustment. Wiley, New York.
White, H. (1980). “Using least squares to approximate unknown regression functions”. International Eco-
nomic Review 21, 149–170.
White, H. (1981). “Consequences and detection of misspecified nonlinear regression models”. Journal of the
American Statistical Association 76, 419–433.
White, H. (2001). Asymptotic Theory for Econometricians. Academic Press, San Diego, CA.
Williams, E. (2003). “Essays in multiple comparison testing”. Ph.D. Dissertation, Department of Economics,
University of California, San Diego, CA.
PART 3
FORECASTING WITH PARTICULAR
DATA STRUCTURES