Statistics and data analysis for financial engineering

Springer Texts in Statistics
Series Editors
G. Casella
S. Fienberg
I. Olkin

For other titles published in this series, go to
www.springer.com/series/417



David Ruppert

Statistics and Data Analysis
for Financial Engineering


David Ruppert
School of Operations Research
and Information Engineering
Cornell University
Comstock Hall 1170
14853-3801 Ithaca New York
USA

Series Editors:
George Casella
Department of Statistics
University of Florida
Gainesville, FL 32611-8545

USA

Stephen Fienberg
Department of Statistics
Carnegie Mellon University
Pittsburgh, PA 15213-3890
USA

Ingram Olkin
Department of Statistics
Stanford University
Stanford, CA 94305
USA

ISSN 1431-875X
ISBN 978-1-4419-7786-1
e-ISBN 978-1-4419-7787-8
DOI 10.1007/978-1-4419-7787-8
Springer New York Dordrecht Heidelberg London
© Springer Science+Business Media, LLC 2011
All rights reserved. This work may not be translated or copied in whole or in part without the written
permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York,
NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in
connection with any form of information storage and retrieval, electronic adaptation, computer
software, or by similar or dissimilar methodology now known or hereafter developed is forbidden.
The use in this publication of trade names, trademarks, service marks, and similar terms, even if they
are not identified as such, is not to be taken as an expression of opinion as to whether or not they are
subject to proprietary rights.
Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.springer.com)


To the memory of my grandparents



Preface

I developed this textbook while teaching the course Statistics for Financial
Engineering to master’s students in the financial engineering program at Cornell University. These students have already taken courses in portfolio management, fixed income securities, options, and stochastic calculus, so I concentrate on teaching statistics, data analysis, and the use of R, and I cover
most sections of Chapters 4–9 and 17–20. These chapters alone are more than
enough to fill a one semester course. I do not cover regression (Chapters 12–14
and 21) or the more advanced time series topics in Chapter 10, since these
topics are covered in other courses. In the past, I have not covered cointegration (Chapter 15), but I will in the future. The master’s students spend much
of the third semester working on projects with investment banks or hedge
funds. As a faculty adviser for several projects, I have seen the importance of
cointegration.
A number of different courses might be based on this book. A two-semester
sequence could cover most of the material. A one-semester course with more
emphasis on finance would include Chapters 11 and 16 on portfolios and the
CAPM and omit some of the chapters on statistics, for instance, Chapters 8,
18, and 20 on copulas, GARCH models, and Bayesian statistics. The book
could be used for courses at both the master’s and Ph.D. levels.
Readers familiar with my textbook Statistics and Finance: An Introduction may wonder how that volume differs from this book. This book is at a
somewhat more advanced level and has much broader coverage of topics in
statistics compared to the earlier book. As the title of this volume suggests,
there is more emphasis on data analysis and this book is intended to be more
than just “an introduction.” Chapters 8, 15, and 20 on copulas, cointegration,
and Bayesian statistics are new. Except for some figures borrowed from Statistics and Finance, in this book R is used exclusively for computations, data

analysis, and graphing, whereas the earlier book used SAS and MATLAB.
Nearly all of the examples in this book use data sets that are available in R,
so readers can reproduce the results. In Chapter 20 on Bayesian statistics,
WinBUGS is used for Markov chain Monte Carlo and is called from R using


viii

Preface

the R2WinBUGS package. There is some overlap between the two books, and,
in particular, a substantial amount of the material in Chapters 2, 3, 9, 11–13,
and 16, has been taken from the earlier book. Unlike Statistics and Finance,
this volume does not cover options pricing and behavioral finance.
The prerequisites for reading this book are knowledge of calculus, vectors
and matrices; probability including stochastic processes; and statistics typical
of third- or fourth-year undergraduates in engineering, mathematics, statistics, and related disciplines. There is an appendix that reviews probability and
statistics, but it is intended for reference and is certainly not an introduction
for readers with little or no prior exposure to these topics. Also, the reader
should have some knowledge of computer programming. Some familiarity with
the basic ideas of finance is helpful.
This book does not teach R programming, but each chapter has an “R lab”
with data analysis and simulations. Students can learn R from these labs and
by using R’s help or the manual An Introduction to R (available at the CRAN
website and R’s online help) to learn more about the functions used in the
labs. Also, the text does indicate which R functions are used in the examples.
Occasionally, R code is given to illustrate some process, for example, in Chapter 11 finding the tangency portfolio by quadratic programming. For readers
wishing to use R, the bibliographical notes at the end of each chapter mention
books that cover R programming and the book’s website contains examples
of the R and WinBUGS code used to produce this book. Students enter my

course Statistics for Financial Engineering with quite disparate knowledge of
R. Some are very accomplished R programmers, while others have no experience with R, although all have experience with some programming language.
Students with no previous experience with R generally need assistance from
the instructor to get started on the R labs. Readers using this book for selfstudy should learn R first before attempting the R labs.

Ithaca, New York
July 2010

David Ruppert


Contents

Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi
1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1
3
4

2

Returns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.1.1 Net Returns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.1.2 Gross Returns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2.1.3 Log Returns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.1.4 Adjustment for Dividends . . . . . . . . . . . . . . . . . . . . . . . . .
2.2 The Random Walk Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2.1 Random Walks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2.2 Geometric Random Walks . . . . . . . . . . . . . . . . . . . . . . . . .
2.2.3 Are Log Prices a Lognormal Geometric Random Walk?
2.3 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.4 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.5 R Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.5.1 Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.5.2 Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5
5
5
6
6
7
8
8
8
9
10
10
11
11
12
14

3

Fixed Income Securities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 Zero-Coupon Bonds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2.1 Price and Returns Fluctuate with the Interest Rate . . .
3.3 Coupon Bonds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3.1 A General Formula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4 Yield to Maturity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4.1 General Method for Yield to Maturity . . . . . . . . . . . . . . .

17
17
18
18
19
20
21
22


x

Contents

3.4.2 Spot Rates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Term Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.5.1 Introduction: Interest Rates Depend Upon Maturity . .
3.5.2 Describing the Term Structure . . . . . . . . . . . . . . . . . . . . .
3.6 Continuous Compounding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3.7 Continuous Forward Rates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.8 Sensitivity of Price to Yield . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.8.1 Duration of a Coupon Bond . . . . . . . . . . . . . . . . . . . . . . .
3.9 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.10 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.11 R Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.11.1 Computing Yield to Maturity . . . . . . . . . . . . . . . . . . . . . .
3.11.2 Graphing Yield Curves . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.12 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

23
24
24
24
29
30
32
32
33
34
34
34
36
36

4

Exploratory Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2 Histograms and Kernel Density Estimation . . . . . . . . . . . . . . . . .

4.3 Order Statistics, the Sample CDF, and Sample Quantiles . . . . .
4.3.1 The Central Limit Theorem for Sample Quantiles . . . . .
4.3.2 Normal Probability Plots . . . . . . . . . . . . . . . . . . . . . . . . . .
4.3.3 Half-Normal Plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.3.4 Quantile–Quantile Plots . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.4 Tests of Normality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.5 Boxplots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.6 Data Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.7 The Geometry of Transformations . . . . . . . . . . . . . . . . . . . . . . . . .
4.8 Transformation Kernel Density Estimation . . . . . . . . . . . . . . . . .
4.9 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.10 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.11 R Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.11.1 European Stock Indices . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.12 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

41
41
43
48
49
50
54
57
59
61
62
66
70
73

73
74
74
77

5

Modeling Univariate Distributions . . . . . . . . . . . . . . . . . . . . . . . . . .
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.2 Parametric Models and Parsimony . . . . . . . . . . . . . . . . . . . . . . . . .
5.3 Location, Scale, and Shape Parameters . . . . . . . . . . . . . . . . . . . . .
5.4 Skewness, Kurtosis, and Moments . . . . . . . . . . . . . . . . . . . . . . . . .
5.4.1 The Jarque–Bera test . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.4.2 Moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.5 Heavy-Tailed Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.5.1 Exponential and Polynomial Tails . . . . . . . . . . . . . . . . . .
5.5.2 t-Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.5.3 Mixture Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

79
79
79
80
81
86
86
87
87
88
90

3.5


Contents

5.6
5.7
5.8
5.9
5.10
5.11
5.12
5.13
5.14
5.15
5.16
5.17
5.18
5.19
5.20

5.21

xi

Generalized Error Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
Creating Skewed from Symmetric Distributions . . . . . . . . . . . . . 95
Quantile-Based Location, Scale, and Shape Parameters . . . . . . . 97
Maximum Likelihood Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . 98

Fisher Information and the Central Limit Theorem for the
MLE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
Likelihood Ratio Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
AIC and BIC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
Validation Data and Cross-Validation . . . . . . . . . . . . . . . . . . . . . . 103
Fitting Distributions by Maximum Likelihood . . . . . . . . . . . . . . . 106
Profile Likelihood . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
Robust Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
Transformation Kernel Density Estimation with a Parametric
Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
R Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
5.20.1 Earnings Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
5.20.2 DAX Returns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

6

Resampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
6.2 Bootstrap Estimates of Bias, Standard Deviation, and MSE . . 132
6.2.1 Bootstrapping the MLE of the t-Distribution . . . . . . . . . 133
6.3 Bootstrap Confidence Intervals . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
6.3.1 Normal Approximation Interval . . . . . . . . . . . . . . . . . . . . 136
6.3.2 Bootstrap-t Intervals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
6.3.3 Basic Bootstrap Interval . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
6.3.4 Percentile Confidence Intervals . . . . . . . . . . . . . . . . . . . . . 140
6.4 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
6.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

6.6 R Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
6.6.1 BMW Returns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
6.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

7

Multivariate Statistical Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
7.2 Covariance and Correlation Matrices . . . . . . . . . . . . . . . . . . . . . . . 149
7.3 Linear Functions of Random Variables . . . . . . . . . . . . . . . . . . . . . 151
7.3.1 Two or More Linear Combinations of Random Variables153
7.3.2 Independence and Variances of Sums . . . . . . . . . . . . . . . . 154
7.4 Scatterplot Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
7.5 The Multivariate Normal Distribution . . . . . . . . . . . . . . . . . . . . . . 156
7.6 The Multivariate t-Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . 157


xii

Contents

7.6.1 Using the t-Distribution in Portfolio Analysis . . . . . . . . 160
Fitting the Multivariate t-Distribution by Maximum Likelihood160
Elliptically Contoured Densities . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
The Multivariate Skewed t-Distributions . . . . . . . . . . . . . . . . . . . 164
The Fisher Information Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
Bootstrapping Multivariate Data . . . . . . . . . . . . . . . . . . . . . . . . . . 167
Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
R Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

7.14.1 Equity Returns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
7.14.2 Simulating Multivariate t-Distributions . . . . . . . . . . . . . . 171
7.14.3 Fitting a Bivariate t-Distribution . . . . . . . . . . . . . . . . . . . 172
7.15 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
7.7
7.8
7.9
7.10
7.11
7.12
7.13
7.14

8

Copulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
8.2 Special Copulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
8.3 Gaussian and t-Copulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
8.4 Archimedean Copulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
8.4.1 Frank Copula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
8.4.2 Clayton Copula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
8.4.3 Gumbel Copula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
8.5 Rank Correlation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
8.5.1 Kendall’s Tau . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
8.5.2 Spearman’s Correlation Coefficient . . . . . . . . . . . . . . . . . . 184
8.6 Tail Dependence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
8.7 Calibrating Copulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
8.7.1 Maximum Likelihood . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
8.7.2 Pseudo-Maximum Likelihood . . . . . . . . . . . . . . . . . . . . . . . 188

8.7.3 Calibrating Meta-Gaussian and Meta-t-Distributions . . 189
8.8 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
8.9 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
8.10 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
8.11 R Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
8.11.1 Simulating Copulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
8.11.2 Fitting Copulas to Returns Data . . . . . . . . . . . . . . . . . . . 197
8.12 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

9

Time Series Models: Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
9.1 Time Series Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
9.2 Stationary Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
9.2.1 White Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
9.2.2 Predicting White Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
9.3 Estimating Parameters of a Stationary Process . . . . . . . . . . . . . . 206
9.3.1 ACF Plots and the Ljung–Box Test . . . . . . . . . . . . . . . . . 206


Contents

9.4

9.5

9.6
9.7

9.8

9.9
9.10
9.11
9.12

9.13
9.14
9.15
9.16

9.17

xiii

AR(1) Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
9.4.1 Properties of a stationary AR(1) Process . . . . . . . . . . . . 209
9.4.2 Convergence to the Stationary Distribution . . . . . . . . . . 211
9.4.3 Nonstationary AR(1) Processes . . . . . . . . . . . . . . . . . . . . . 211
Estimation of AR(1) Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
9.5.1 Residuals and Model Checking . . . . . . . . . . . . . . . . . . . . . 213
9.5.2 Maximum Likelihood and Conditional Least-Squares . . 217
AR(p) Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
Moving Average (MA) Processes . . . . . . . . . . . . . . . . . . . . . . . . . . 222
9.7.1 MA(1) Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
9.7.2 General MA Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
ARMA Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
9.8.1 The Backwards Operator . . . . . . . . . . . . . . . . . . . . . . . . . . 225
9.8.2 The ARMA Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
9.8.3 ARMA(1,1) Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226

9.8.4 Estimation of ARMA Parameters . . . . . . . . . . . . . . . . . . . 227
9.8.5 The Differencing Operator . . . . . . . . . . . . . . . . . . . . . . . . . 227
ARIMA Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
9.9.1 Drifts in ARIMA Processes . . . . . . . . . . . . . . . . . . . . . . . . 232
Unit Root Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
9.10.1 How Do Unit Root Tests Work? . . . . . . . . . . . . . . . . . . . . 235
Automatic Selection of an ARIMA Model . . . . . . . . . . . . . . . . . . 236
Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
9.12.1 Forecast Errors and Prediction Intervals . . . . . . . . . . . . . 239
9.12.2 Computing Forecast Limits by Simulation . . . . . . . . . . . 241
Partial Autocorrelation Coefficients . . . . . . . . . . . . . . . . . . . . . . . . 245
Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248
R Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248
9.16.1 T-bill Rates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248
9.16.2 Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

10 Time Series Models: Further Topics . . . . . . . . . . . . . . . . . . . . . . . . 257
10.1 Seasonal ARIMA Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
10.1.1 Seasonal and nonseasonal differencing . . . . . . . . . . . . . . . 258
10.1.2 Multiplicative ARIMA Models . . . . . . . . . . . . . . . . . . . . . 259
10.2 Box–Cox Transformation for Time Series . . . . . . . . . . . . . . . . . . . 262
10.3 Multivariate Time Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
10.3.1 The cross-correlation function . . . . . . . . . . . . . . . . . . . . . . 264
10.3.2 Multivariate White Noise . . . . . . . . . . . . . . . . . . . . . . . . . . 265
10.3.3 Multivariate ARMA processes . . . . . . . . . . . . . . . . . . . . . . 266
10.3.4 Prediction Using Multivariate AR Models . . . . . . . . . . . 268
10.4 Long-Memory Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
10.4.1 The Need for Long-Memory Stationary Models . . . . . . . 270

xiv

Contents

10.5
10.6
10.7
10.8

10.9

10.4.2 Fractional Differencing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
10.4.3 FARIMA Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272
Bootstrapping Time Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
R Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
10.8.1 Seasonal ARIMA Models . . . . . . . . . . . . . . . . . . . . . . . . . . 277
10.8.2 VAR Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278
10.8.3 Long-Memory Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
10.8.4 Model-Based Bootstrapping of an ARIMA Process . . . . 280
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282

11 Portfolio Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285
11.1 Trading Off Expected Return and Risk . . . . . . . . . . . . . . . . . . . . . 285
11.2 One Risky Asset and One Risk-Free Asset . . . . . . . . . . . . . . . . . . 285
11.2.1 Estimating E(R) and σR . . . . . . . . . . . . . . . . . . . . . . . . . . 287
11.3 Two Risky Assets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287

11.3.1 Risk Versus Expected Return . . . . . . . . . . . . . . . . . . . . . . 287
11.4 Combining Two Risky Assets with a Risk-Free Asset . . . . . . . . . 289
11.4.1 Tangency Portfolio with Two Risky Assets . . . . . . . . . . . 289
11.4.2 Combining the Tangency Portfolio with the Risk-Free
Asset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
11.4.3 Effect of ρ12 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
11.5 Selling Short . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
11.6 Risk-Efficient Portfolios with N Risky Assets . . . . . . . . . . . . . . . 294
11.7 Resampling and Efficient Portfolios . . . . . . . . . . . . . . . . . . . . . . . . 299
11.8 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
11.9 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
11.10 R Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306
11.10.1 Efficient Equity Portfolios . . . . . . . . . . . . . . . . . . . . . . . . . 306
11.11 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307
12 Regression: Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
12.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
12.2 Straight-Line Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310
12.2.1 Least-Squares Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . 310
12.2.2 Variance of β1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314
12.3 Multiple Linear Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
12.3.1 Standard Errors, t-Values, and p-Values . . . . . . . . . . . . . 317
12.4 Analysis of Variance, Sums of Squares, and R2 . . . . . . . . . . . . . . 318
12.4.1 AOV Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318
12.4.2 Degrees of Freedom (DF) . . . . . . . . . . . . . . . . . . . . . . . . . . 320
12.4.3 Mean Sums of Squares (MS) and F -Tests . . . . . . . . . . . . 321
12.4.4 Adjusted R2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323
12.5 Model Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323


Contents

xv

12.6
12.7
12.8
12.9
12.10
12.11
12.12

Collinearity and Variance Inflation . . . . . . . . . . . . . . . . . . . . . . . . 325
Partial Residual Plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332
Centering the Predictors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334
Orthogonal Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334
Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
R Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
12.12.1 U.S. Macroeconomic Variables . . . . . . . . . . . . . . . . . . . . . 335
12.13 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338
13 Regression: Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341
13.1 Regression Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341
13.1.1 Leverages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343
13.1.2 Residuals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344
13.1.3 Cook’s D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346
13.2 Checking Model Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348
13.2.1 Nonnormality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349
13.2.2 Nonconstant Variance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351
13.2.3 Nonlinearity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351
13.2.4 Residual Correlation and Spurious Regressions . . . . . . . 354

13.3 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
13.4 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
13.5 R Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
13.5.1 Current Population Survey Data . . . . . . . . . . . . . . . . . . . 361
13.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364
14 Regression: Advanced Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369
14.1 Linear Regression with ARMA Errors . . . . . . . . . . . . . . . . . . . . . . 369
14.2 The Theory Behind Linear Regression . . . . . . . . . . . . . . . . . . . . . 373
14.2.1 The Effect of Correlated Noise and Heteroskedasticity . 374
14.2.2 Maximum Likelihood Estimation for Regression . . . . . . 374
14.3 Nonlinear Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376
14.4 Estimating Forward Rates from Zero-Coupon Bond Prices . . . . 381
14.5 Transform-Both-Sides Regression . . . . . . . . . . . . . . . . . . . . . . . . . . 386
14.5.1 How TBS Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388
14.6 Transforming Only the Response . . . . . . . . . . . . . . . . . . . . . . . . . . 389
14.7 Binary Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390
14.8 Linearizing a Nonlinear Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396
14.9 Robust Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397
14.10 Regression and Best Linear Prediction . . . . . . . . . . . . . . . . . . . . . 401
14.10.1 Best Linear Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
14.10.2 Prediction Error in Best Linear Prediction . . . . . . . . . . . 402
14.10.3 Regression Is Empirical Best Linear Prediction . . . . . . . 402
14.10.4 Multivariate Linear Prediction . . . . . . . . . . . . . . . . . . . . . 403
14.11 Regression Hedging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403


xvi

Contents

14.12 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405
14.13 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405
14.14 R Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 406
14.14.1 Regression with ARMA Noise . . . . . . . . . . . . . . . . . . . . . . 406
14.14.2 Nonlinear Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 406
14.14.3 Response Transformations . . . . . . . . . . . . . . . . . . . . . . . . . 409
14.14.4 Binary Regression: Who Owns an Air Conditioner? . . . 410
14.15 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410
15 Cointegration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413
15.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413
15.2 Vector Error Correction Models . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
15.3 Trading Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419
15.4 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419
15.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419
15.6 R Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 420
15.6.1 Cointegration Analysis of Midcap Prices . . . . . . . . . . . . . 420
15.6.2 Cointegration Analysis of Yields . . . . . . . . . . . . . . . . . . . . 421
15.6.3 Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421
15.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422
16 The
16.1
16.2
16.3

Capital Asset Pricing Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 423
Introduction to the CAPM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423
The Capital Market Line (CML) . . . . . . . . . . . . . . . . . . . . . . . . . . 424
Betas and the Security Market Line . . . . . . . . . . . . . . . . . . . . . . . 426
16.3.1 Examples of Betas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428
16.3.2 Comparison of the CML with the SML . . . . . . . . . . . . . . 428

16.4 The Security Characteristic Line . . . . . . . . . . . . . . . . . . . . . . . . . . 429
16.4.1 Reducing Unique Risk by Diversification . . . . . . . . . . . . . 430
16.4.2 Are the Assumptions Sensible? . . . . . . . . . . . . . . . . . . . . . 432
16.5 Some More Portfolio Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432
16.5.1 Contributions to the Market Portfolio’s Risk . . . . . . . . . 432
16.5.2 Derivation of the SML . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433
16.6 Estimation of Beta and Testing the CAPM . . . . . . . . . . . . . . . . . 434
16.6.1 Estimation Using Regression . . . . . . . . . . . . . . . . . . . . . . . 434
16.6.2 Testing the CAPM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 436
16.6.3 Interpretation of Alpha . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437
16.7 Using the CAPM in Portfolio Analysis . . . . . . . . . . . . . . . . . . . . . 437
16.8 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437
16.9 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438
16.10 R Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438
16.11 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440


Contents

xvii

17 Factor Models and Principal Components . . . . . . . . . . . . . . . . . . 443
17.1 Dimension Reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443
17.2 Principal Components Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 443
17.3 Factor Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453
17.4 Fitting Factor Models by Time Series Regression . . . . . . . . . . . . 454
17.4.1 Fama and French Three-Factor Model . . . . . . . . . . . . . . . 455
17.4.2 Estimating Expectations and Covariances of Asset
Returns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 460
17.5 Cross-Sectional Factor Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463

17.6 Statistical Factor Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466
17.6.1 Varimax Rotation of the Factors . . . . . . . . . . . . . . . . . . . . 469
17.7 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 470
17.8 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 470
17.9 R Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 471
17.9.1 PCA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 471
17.9.2 Fitting Factor Models by Time Series Regression . . . . . 473
17.9.3 Statistical Factor Models . . . . . . . . . . . . . . . . . . . . . . . . . . 475
17.10 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475
18 GARCH Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 477
18.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 477
18.2 Estimating Conditional Means and Variances . . . . . . . . . . . . . . . 478
18.3 ARCH(1) Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479
18.4 The AR(1)/ARCH(1) Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481
18.5 ARCH(p) Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482
18.6 ARIMA(pA , d, qA )/GARCH(pG , qG ) Models . . . . . . . . . . . . . . . . . 483
18.6.1 Residuals for ARIMA(pA , d, qA )/GARCH(pG , qG )
Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484
18.7 GARCH Processes Have Heavy Tails . . . . . . . . . . . . . . . . . . . . . . . 484
18.8 Fitting ARMA/GARCH Models . . . . . . . . . . . . . . . . . . . . . . . . . . . 484
18.9 GARCH Models as ARMA Models . . . . . . . . . . . . . . . . . . . . . . . . 488
18.10 GARCH(1,1) Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 489
18.11 APARCH Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 491
18.12 Regression with ARMA/GARCH Errors . . . . . . . . . . . . . . . . . . . 494
18.13 Forecasting ARMA/GARCH Processes . . . . . . . . . . . . . . . . . . . . . 497
18.14 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 498
18.15 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 499
18.16 R Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 500
18.16.1 Fitting GARCH Models . . . . . . . . . . . . . . . . . . . . . . . . . . . 500
18.17 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 501

19 Risk Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 505
19.1 The Need for Risk Management . . . . . . . . . . . . . . . . . . . . . . . . . . . 505
19.2 Estimating VaR and ES with One Asset . . . . . . . . . . . . . . . . . . . . 506
19.2.1 Nonparametric Estimation of VaR and ES . . . . . . . . . . . 507


xviii

Contents

19.3
19.4
19.5
19.6
19.7
19.8
19.9
19.10
19.11
19.12
19.13

19.2.2 Parametric Estimation of VaR and ES . . . . . . . . . . . . . . 508
Confidence Intervals for VaR and ES Using the Bootstrap . . . . 511
Estimating VaR and ES Using ARMA/GARCH Models . . . . . . 512
Estimating VaR and ES for a Portfolio of Assets . . . . . . . . . . . . 514
Estimation of VaR Assuming Polynomial Tails . . . . . . . . . . . . . . 516
19.6.1 Estimating the Tail Index . . . . . . . . . . . . . . . . . . . . . . . . . 518
Pareto Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 522
Choosing the Horizon and Confidence Level . . . . . . . . . . . . . . . . . 523

VaR and Diversification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524
Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 526
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 526
R Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 527
19.12.1 VaR Using a Multivariate-t Model . . . . . . . . . . . . . . . . . . 527
Exercies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 528

20 Bayesian Data Analysis and MCMC . . . . . . . . . . . . . . . . . . . . . . . . 531
20.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 531
20.2 Bayes’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 532
20.3 Prior and Posterior Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . 534
20.4 Conjugate Priors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 536
20.5 Central Limit Theorem for the Posterior . . . . . . . . . . . . . . . . . . . 543
20.6 Posterior Intervals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 543
20.7 Markov Chain Monte Carlo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545
20.7.1 Gibbs Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 546
20.7.2 Other Monte Carlo Samplers . . . . . . . . . . . . . . . . . . . . . . . 547
20.7.3 Analysis of MCMC Output . . . . . . . . . . . . . . . . . . . . . . . . 548
20.7.4 WinBUGS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 549
20.7.5 Monitoring MCMC Convergence and Mixing . . . . . . . . . 551
20.7.6 DIC and pD for Model Comparisons . . . . . . . . . . . . . . . . 556
20.8 Hierarchical Priors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 558
20.9 Bayesian Estimation of a Covariance Matrix . . . . . . . . . . . . . . . . 562
20.9.1 Estimating a Multivariate Gaussian Covariance Matrix 562
20.9.2 Estimating a multivariate-t Scale Matrix . . . . . . . . . . . . 564
20.9.3 Non-conjugate Priors for the Covariate Matrix . . . . . . . 566
20.10 Sampling a Stationary Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . 566
20.11 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 567
20.12 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 569
20.13 R Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 570

20.13.1 Fitting a t-Distribution by MCMC . . . . . . . . . . . . . . . . . . 570
20.13.2 AR Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 574
20.13.3 MA Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 575
20.13.4 ARMA Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 577
20.14 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 577


Contents

xix

21 Nonparametric Regression and Splines . . . . . . . . . . . . . . . . . . . . . 579
21.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 579
21.2 Local Polynomial Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 581
21.2.1 Lowess and Loess . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 584
21.3 Linear Smoothers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 584
21.3.1 The Smoother Matrix and the Effective Degrees of
Freedom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 585
21.3.2 AIC and GCV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 585
21.4 Polynomial Splines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 586
21.4.1 Linear Splines with One Knot . . . . . . . . . . . . . . . . . . . . . . 586
21.4.2 Linear Splines with Many Knots . . . . . . . . . . . . . . . . . . . . 587
21.4.3 Quadratic Splines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 588
21.4.4 pth Degree Splines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 589
21.4.5 Other Spline Bases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 589
21.5 Penalized Splines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 589
21.5.1 Selecting the Amount of Penalization . . . . . . . . . . . . . . . 591
21.6 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 593
21.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 593
21.8 R Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 594

21.8.1 Additive Model for Wages, Education, and Experience 594
21.8.2 An Extended CKLS model for the Short Rate . . . . . . . . 595
21.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 596
A Facts from Probability, Statistics, and Algebra . . . . . . . . . . . . . 597
A.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 597
A.2 Probability Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 597
A.2.1 Cumulative Distribution Functions . . . . . . . . . . . . . . . . . . 597
A.2.2 Quantiles and Percentiles . . . . . . . . . . . . . . . . . . . . . . . . . . 597
A.2.3 Symmetry and Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 598
A.2.4 Support of a Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . 598
A.3 When Do Expected Values and Variances Exist? . . . . . . . . . . . . 598
A.4 Monotonic Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 599
A.5 The Minimum, Maximum, Infinum, and Supremum of a Set . . 599
A.6 Functions of Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . 600
A.7 Random Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 601
A.8 The Binomial Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 601
A.9 Some Common Continuous Distributions . . . . . . . . . . . . . . . . . . . 602
A.9.1 Uniform Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 602
A.9.2 Transformation by the CDF and Inverse CDF . . . . . . . . 602
A.9.3 Normal Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 603
A.9.4 The Lognormal Distribution . . . . . . . . . . . . . . . . . . . . . . . 603
A.9.5 Exponential and Double-Exponential Distributions . . . . 604
A.9.6 Gamma and Inverse-Gamma Distributions . . . . . . . . . . . 605
A.9.7 Beta Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 606
A.9.8 Pareto Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 606


xx

Contents

A.10 Sampling a Normal Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . 607
A.10.1 Chi-Squared Distributions . . . . . . . . . . . . . . . . . . . . . . . . . 607
A.10.2 F -distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 607
A.11 Law of Large Numbers and the Central Limit Theorem for
the Sample Mean . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 608
A.12 Bivariate Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 608
A.13 Correlation and Covariance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 609
A.13.1 Normal Distributions: Conditional Expectations and
Variance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 612
A.14 Multivariate Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 613
A.14.1 Conditional Densities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 613
A.15 Stochastic Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 614
A.16 Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 614
A.16.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 614
A.16.2 Standard Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 615
A.17 Confidence Intervals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 615
A.17.1 Confidence Interval for the Mean . . . . . . . . . . . . . . . . . . . 615
A.17.2 Confidence Intervals for the Variance and Standard
Deviation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 616
A.17.3 Confidence Intervals Based on Standard Errors . . . . . . . 617
A.18 Hypothesis Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 617
A.18.1 Hypotheses, Types of Errors, and Rejection Regions . . 617
A.18.2 p-Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 618
A.18.3 Two-Sample t-Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 618
A.18.4 Statistical Versus Practical Significance . . . . . . . . . . . . . . 620
A.19 Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 620
A.20 Facts About Vectors and Matrices . . . . . . . . . . . . . . . . . . . . . . . . . 621
A.21 Roots of Polynomials and Complex Numbers . . . . . . . . . . . . . . . 621
A.22 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 622

A.23 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 622
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 623


Notation

The following conventions are observed as much as possible:
• Lowercase letters, e.g., a and b, are used for nonrandom scalars.
• Lower-case boldface letters, e.g., a, b, and θ, are used for nonrandom
vectors.
• Upper-case letters, e.g., X and Y , are used for random variables.
• Uppercase bold letters either early in the Roman alphabet or in Greek
without a “hat,” e.g., A, B, and Ω, are used for nonrandom matrices.
• A hat over a parameter or parameter vector, e.g., θ and θ, denotes an
estimator of the corresponding parameter or parameter vector.
• I denotes the identity matrix with dimension appropriate for the context.
• diag(d1 , . . . , dp ) is a diagonal matrix with diagonal elements d1 , . . . , dp .
• Greek alphabet with a “hat” or uppercase bold letters either later in the
Roman alphabet, e.g., X, Y , and θ, will be used for random vectors.
• log(x) is the natural logarithm of x and log10 (x) is the base-10 logarithm.
• E(X) is the expected value of a random variable X.
2
• Var(X) and σX
are used to denote the variance of a random variable X.
• Cov(X, Y ) and σXY are used to denote the covariance between the random
variables X and Y .
• Corr(X, Y ) and ρXY are used to denote the correlation between the random variables X and Y .
• COV(X) is the covariance matrix of a random vector X.
• CORR(X) is the correlation matrix of a random vector X.
• A Greek letter denotes a parameter, e.g., θ.

• A boldface Greek letter, e.g., θ, denotes a vector of parameters.
•
is the set of real numbers and p is the p-dimensional Euclidean space,
the set of all real p-dimensional vectors.
• A ∩ B and A ∪ B are, respectively, the intersection and union of the sets
A and B.
• ∅ is the empty set.


xxii

Contents

• If A is some statement, then I{A} is called the indicator function of A
and is equal to 1 if A is true and equal to 0 if A is false.
• If f1 and f2 are two functions of a variable x, then
f1 (x) ∼ f2 (x) as x → x0
means that
lim

x→x0

f1 (x)
= 1.
f2 (x)

Similarly,
an ∼ bn
means that the sequences {an } and {bn } are such that
an

→ 1 as n → ∞.
bn
• Vectors are column vectors and transposed vectors are rows, e.g.,



x1
.
x =  .. 
xn
and

x T = ( x1

···

xn ) .

• |A| is the determinant of a square matrix A.
• tr(A) is the trace (sum of the diagonal elements) of a square matrix A.
• f (x) ∝ g(x) means that f (x) is proportional to g(x), that is, f (x) = ag(x)
for some nonzero constant a.
• A word appearing in italic font is being defined or introduced in the text.



1
Introduction

This book is about the analysis of financial markets data. After this brief introductory chapter, we turn immediately in Chapters 2 and 3 to the sources

of the data, returns on equities and prices and yields on bonds. Chapter 4
develops methods for informal, often graphical, analysis of data. More formal
methods based on statistical inference, that is, estimation and testing, are introduced in Chapter 5. The chapters that follow Chapter 5 cover a variety of
more advanced statistical techniques: ARIMA models, regression, multivariate models, copulas, GARCH models, factor models, cointegration, Bayesian
statistics, and nonparametric regression.
Much of finance is concerned with financial risk. The return on an investment is its revenue expressed as a fraction of the initial investment. If one
invests at time t1 in an asset with price Pt1 and the price later at time t2 is
Pt2 , then the net return for the holding period from t1 to t2 is (Pt2 − Pt1 )/Pt1 .
For most assets, future returns cannot be known exactly and therefore are
random variables. Risk means uncertainty in future returns from an investment, in particular, that the investment could earn less than the expected
return and even result in a loss, that is, a negative return. Risk is often measured by the standard deviation of the return, which we also call the volatility.
Recently there has been a trend toward measuring risk by value-at-risk (VaR)
and expected shortfall (ES). These focus on large losses and are more direct
indications of financial risk than the standard deviation of the return. Because risk depends upon the probability distribution of a return, probability
and statistics are fundamental tools for finance. Probability is needed for risk
calculations, and statistics is needed to estimate parameters such as the standard deviation of a return or to test hypotheses such as the so-called random
walk hypothesis which states that future returns are independent of the past.
In financial engineering there are two kinds of probability distributions
that can be estimated. Objective probabilities are the true probabilities of
events. Risk-neutral or pricing probabilities give model outputs that agree
with market prices and reflect the market’s beliefs about the probabilities
of future events. The statistical techniques in this book can be used to estiD. Ruppert, Statistics and Data Analysis for Financial Engineering, Springer Texts in Statistics,
DOI 10.1007/978-1-4419-7787-8_1, © Springer Science+Business Media, LLC 2011

1