Download Statistics for Financial Engineering: Some R

Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
Introduction
Nonlinear
Regression
Statistics for Financial Engineering: Some R
Examples
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
David Ruppert
Cornell University
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
April 25, 2009
Outline
Statistics for
Financial
Engineering:
Some R
Examples
1
Introduction
2
Nonlinear Regression
Default probabilities
Data Transformations: some theory
3
Estimating a dynamic model
Interest rate data
Checking the model: residual analysis
GARCH models
4
Bayesian estimation of expected returns
David Ruppert
Introduction
Nonlinear
Regression
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
A little about myself
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
BA and MA in mathematics
Introduction
Nonlinear
Regression
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
PhD in statistics in 1977
taught in the statistics department at North Carolina for
10 years
have been in Operations Research and Information
(formerly Industrial) Engineering at Cornell since 1987
A little about myself
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
starting teaching Statistics and Finance to undergraduates
Introduction
in 2001
Nonlinear
Regression
textbook published in 2004
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
starting teaching Statistics for Financial Engineering to
master’s students in 2008
working on revised and expanded textbook
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
now programming exclusively in R
Undergraduate Textbook
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
Introduction
Nonlinear
Regression
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
A little about my research
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
have done research in
asymptotic theory of splines
semiparametric modeling
Introduction
Nonlinear
Regression
measurement error in regression
smoothing (nonparametric regression and density
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
estimation)
transformation and weighting
stochastic approximation
biostatistics
environmental engineering
modeling of term structure
executive compensation and accounting fraud
Three types of regression
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
Introduction
Nonlinear
Regression
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
Linear regression
Yi = β0 + β1 Xi,1 + · · · + βp Xi,p + i , i = 1, . . . , n
Nonlinear regression
Yi = m(Xi,1 , . . . , Xi,p ; β1 , . . . , βq ) + i , i = 1, . . . , n
where m is a known function depending on unknown
parameters
Nonparametric regression
GARCH models
Bayesian
estimation of
expected
returns
Yi = m(Xi,1 , . . . , Xi,p ) + i , i = 1, . . . , n
where m is an unknown “smooth” function
Usual assumptions on the noise
Statistics for
Financial
Engineering:
Some R
Examples
Usually 1 , . . . , n are assumed to be:
David Ruppert
mutually independent (or at least uncorrelated)
Introduction
homoscedastic (constant variance)
Nonlinear
Regression
Default probabilities
normally distributed
Data
Transformations:
some theory
Estimating a
dynamic
model
Much research over the last 50+ years has looked into ways of
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
1
checking these assumptions
2
statistical methods that require less assumptions
Transform-both-sides model
Statistics for
Financial
Engineering:
Some R
Examples
Ideal model (no errors):
Yi = f (X i , β)
David Ruppert
Introduction
Statistical model (first attempt):
Nonlinear
Regression
Yi = f (X i , β) + i
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
where 1 , . . . , n are iid Gaussian
TBS model:
h{Yi } = h{f (X i , β)} + i
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
where
1 , . . . , n are iid Gaussian
h is an “appropriate” transformation
Estimation of Default Probabilities
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
Introduction
Nonlinear
Regression
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
Data:
ratings: 1=Aaa (best),...,16=B3 (worse)
default frequency (estimate of default probability)
Some statistical models
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
nonlinear model:
Pr(default|rating) = exp{β0 + β1 rating}
Introduction
Nonlinear
Regression
linear/transformation model (in recent textbook):
Default probabilities
Data
Transformations:
some theory
log{Pr(default|rating)} = β0 + β1 rating
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
Problem: cannot take logs of default frequencies that
are 0
(Sub-optimal) solution in textbook: throw out these
observations
A better statistical model
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
Introduction
Nonlinear
Regression
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
Transform-both-sides (TBS) model – see Carroll and
Ruppert (1984, 1988):
using a power transformation:
Pr(default|rating) + κ
λ
=
exp(β0 + β1 rating) + κ
λ chosen by residual plots (or maximum likelihood)
λ = 1/2 works well for this example
log transformations are also commonly used
κ > 0 will shift data away from 0
λ
The Box-Cox family
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
the most common transformation family is due to Box and
Cox (1964):
Introduction
Nonlinear
Regression
yλ − 1
if λ 6= 0
λ
= log(y) if λ = 0
h(y, λ) =
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
derivative has simple form:
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
hy (y, λ) =
d
h(y, λ) = y λ−1 for all λ
dy
TBS fit compared to others
Statistics for
Financial
Engineering:
Some R
Examples
●
Data
Transformations:
some theory
Estimating a
dynamic
model
●
●
●
●
●
−5
Default probabilities
o
BOW
nonlinear
tbs
data ●
●
−10
Nonlinear
Regression
log(default probability)
Introduction
0
David Ruppert
●
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
●
●
●
●
●
●
5
10
rating
15
●
Nonlinear regression residuals
Statistics for
Financial
Engineering:
Some R
Examples
Normal QQ plot
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
●
●
●
●
●
1
●
●
●
●
●
●
●
●
●
●
0
Theoretical Quantiles
●
●
−1
Estimating a
dynamic
model
●
●
●
●
●
●● ●
●
● ●
●
−2
Data
Transformations:
some theory
absolute residual
Default probabilities
0.002
Nonlinear
Regression
0.000
Introduction
0.004
0.006
2
David Ruppert
0.00 0.02 0.04 0.06 0.08 0.10 0.12
fitted values
●
−0.006
−0.002
0.002
Sample Quantiles
0.006
TBS residuals
Statistics for
Financial
Engineering:
Some R
Examples
Normal QQ plot
●
●
●
●
●
●
●
Bayesian
estimation of
expected
returns
1
●
●
●
●
●
−2
●
GARCH models
●
●
●
●
●
●
●
Interest rate data
Checking the model:
residual analysis
●
●
●
●
0
●
●
●
Theoretical Quantiles
Estimating a
dynamic
model
●
●
−1
Data
Transformations:
some theory
0.010
Default probabilities
absolute residual
Nonlinear
Regression
●
●
0.005
Introduction
0.015
0.020
2
David Ruppert
−0.02
0.02
0.06
fitted values
0.10
●
−0.02
−0.01
0.00
0.01
Sample Quantiles
0.02
Estimated default probabilities
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
Introduction
Nonlinear
Regression
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
method
TEXTBOOK
nonlinear
TBS
c
Pr{default|Aaa}
0.005%
0.002%
0.0008%
as % of TEXTBOOK est
100%
40%
16%
A Similar Problem: Challenger Data
2.0
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
*
Data
Transformations:
some theory
Estimating a
dynamic
model
1.5
1.0
Default probabilities
**
*
*
0.5
Nonlinear
Regression
# o−rings distressed
Introduction
Interest rate data
GARCH models
Bayesian
estimation of
expected
returns
0.0
Checking the model:
residual analysis
55
60
***** ** **
** ** *
65
70
temp
75
80
Challenger Data: Extrapolation to 31o
Statistics for
Financial
Engineering:
Some R
Examples
6
Logistic regression
Estimating a
dynamic
model
Interest rate data
4
3
*
2
Data
Transformations:
some theory
1
Default probabilities
# o−rings distressed
Introduction
Nonlinear
Regression
all data
only data with failures
5
David Ruppert
GARCH models
Bayesian
estimation of
expected
returns
0
Checking the model:
residual analysis
30
40
50
** * *
***********
**
60
temp
70
80
90
Variance stabilizing transformation: how it works
David Ruppert
3.0
Introduction
2.5
Statistics for
Financial
Engineering:
Some R
Examples
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
●
●●●
●●
●
● ●
●
1.0
Interest rate data
log(x)
Pop 1
0.5
Estimating a
dynamic
model
y
Data
Transformations:
some theory
transformed
0.0
Default probabilities
●
Pop 2
transformed
1.5
2.0
Nonlinear
Regression
●
●
●
●
●
●
●
●
●
●
Pop 1
0
Pop 2
5
10
Strength of Box-Cox family
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
Introduction
Nonlinear
Regression
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
Take a < b
Then
hy (b, λ)
=
hy (a, λ)
λ−1
b
a
which is increasing in λ and equals 1 when λ = 1
λ = 1 is the dividing point between concave and convex
transformations
h(y, λ) becomes a stronger concave transformation as λ
decreases from 1
also, h(y, λ) becomes a stronger convex transformation as
λ increases from 1
Strength of Box-Cox family, cont.
Statistics for
Financial
Engineering:
Some R
Examples
Example: b/a = 2
2.0
David Ruppert
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
1.5
convex
1.0
Default probabilities
2α−1
0.5
Nonlinear
Regression
Derivative ratio
Introduction
concave
Checking the model:
residual analysis
GARCH models
−1.0
Bayesian
estimation of
expected
returns
−0.5
0.0
0.5
α
1.0
1.5
2.0
Maximum likelihood
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
L(β, λ, σ)
= n log(σ) −
+
n
X
Default probabilities
i=1
Data
Transformations:
some theory
|
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
2σ 2
i=1
Introduction
Nonlinear
Regression
h
i2
n
h(Yi + κ, λ) − h f (X i , β) + κ, λ
X
(λ − 1) log(Yi )
{z
from Jacobian
}
can maximize over σ analytically:
i2
Pn h
σ
b2 = n −1 i=1 h(Yi + κ, λ) − h f (X i , β) + κ, λ
they maximize over (β, λ) with optim, for example
κ is fixed in advance
Reference for TBS
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
Introduction
Nonlinear
Regression
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
Transformation and Weighting in Regression by Carroll and
Ruppert (1988)
Lots of examples
But none in finance _
¨
1-Year Treasury Constant Maturity Rate, daily data
Statistics for
Financial
Engineering:
Some R
Examples
15
David Ruppert
Default probabilities
Data
Transformations:
some theory
10
5
Nonlinear
Regression
rate
Introduction
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
1970
1980
1990
2000
GARCH models
Bayesian
estimation of
expected
returns
year
Source: Board of Governors of the Federal Reserve System
http://research.stlouisfed.org/fred2/
∆Rt versus year
Statistics for
Financial
Engineering:
Some R
Examples
1.0
David Ruppert
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
0.5
0.0
Data
Transformations:
some theory
−0.5
Default probabilities
−1.0
Nonlinear
Regression
change in rate
Introduction
GARCH models
Bayesian
estimation of
expected
returns
1970
1980
year
1990
2000
∆Rt versus Rt−1
Statistics for
Financial
Engineering:
Some R
Examples
1.0
David Ruppert
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
0.5
0.0
Data
Transformations:
some theory
−0.5
Default probabilities
−1.0
Nonlinear
Regression
change in rate
Introduction
5
10
lagged rate
15
∆Rt2 versus Rt−1
Statistics for
Financial
Engineering:
Some R
Examples
1.2
David Ruppert
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
0.8
0.4
Default probabilities
0.0
Nonlinear
Regression
change in rate2
Introduction
5
10
lagged rate
15
Drift function
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
Discretized diffusion model:
Introduction
Nonlinear
Regression
∆Rt = µ(Rt−1 ) + σ(Rt−1 )t
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
µ(x) is the drift function
σ(x) is the volatility function (as before)
Estimating Volatility
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
Introduction
Parametric model:
β1
Var{(∆Rt )} = β0 Rt−1
(Common in practice)
Nonlinear
Regression
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
Nonparametric model:
Var{(∆Rt )} = σ 2 (Rt−1 )
where σ(·) is a smooth function
will be modeled as a spline
In these models: no dependence on t
Spline Software
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
Introduction
Nonlinear
Regression
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
The penalized spline fits shown here were obtained using the
function spm
in R’s SemiPar package
author is Matt Wand
Comparing parametric and nonparametric volatility
fits
1.2
Statistics for
Financial
Engineering:
Some R
Examples
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
●
●
0.08
●
●
● ●
●
●
●
●
●
●● ●
●
●●●
● ●●
●●●
●● ● ● ●
● ●●●
●●●●
●
●
●● ●
●
●
●●●
●
●●
●
●
●
●● ●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
● ● ●●
●●
●
●
●
●
● ● ●●
●
●
●
●
●
●
● ●●
●●
●
●
●
●
●
●
●● ● ●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
5
10
lag_rate
15
0.06
0.04
●
●
nonpar
par
0.02
●
0.6
0.4
Estimating a
dynamic
model
●
●
●
0.2
Data
Transformations:
some theory
0.0
Default probabilities
●
●
0.00
1.0
Nonlinear
Regression
diff_rate^2
Introduction
0.8
David Ruppert
5
10
lag_rate
15
Comparing parametric and nonparametric volatility
fits: zooming in near 0
Statistics for
Financial
Engineering:
Some R
Examples
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
0.002
Default probabilities
−0.002
Nonlinear
Regression
nonpar
par
−0.006
Introduction
0.006
David Ruppert
0
1
2
3
lag_rate
4
5
6
Spline fitting – Estimation of drift function
Nonlinear
Regression
5
Default probabilities
10
0.00
● ●
●
●
●
●
●
●●
●
●●
●
●●
●●●●●
●
●●●
●●
●
●
●
●●
●●
●
● ●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●●
●●●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●●
●●●●●●●
●●
●
●
●
●
●●● ●
● ●● ● ● ●●
●
●
−0.06
−1.0
Introduction
0.0
diff_rate
David Ruppert
1.0
Statistics for
Financial
Engineering:
Some R
Examples
15
5
lag_rate
10
15
lag_rate
GARCH models
Bayesian
estimation of
expected
returns
10
lag_rate
15
0.005
2nd der
5
−0.010
Checking the model:
residual analysis
1st der
Interest rate data
−0.03
Estimating a
dynamic
model
0.00
Data
Transformations:
some theory
5
10
lag_rate
15
Residuals for diffusion model
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
Introduction
Nonlinear
Regression
residualt
E(residualt )
:= ∆Rt − µ
b(Rt−1 )
=
0
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
std residualt
:=
E(std residual2t )
residualt
σ
b(Rt−1 )
=
1
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
Question
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
Introduction
Nonlinear
Regression
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
Are the drift and volatility functions constant in time?
Residual plots: ordinary residuals
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
0.000
Data
Transformations:
some theory
−0.010
Default probabilities
residual
Nonlinear
Regression
0.010
Introduction
GARCH models
Bayesian
estimation of
expected
returns
1970
1980
year
1990
2000
Residual plots: standardized residuals
Statistics for
Financial
Engineering:
Some R
Examples
autocorrelation function
Normal Q−Q Plot
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
0
10
20
Lag
30
40
4
2
0
−2
Data
Transformations:
some theory
0.0
Default probabilities
ACF
Nonlinear
Regression
0.4
0.8
Introduction
−4
Theoretical Quantiles
David Ruppert
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
−10 −5
0
5
Sample Quantiles
●
Residual plots: Squared standardized residuals
Statistics for
Financial
Engineering:
Some R
Examples
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
1.5
1.0
0.5
Nonlinear
Regression
0.0
Introduction
Squared Std residual
David Ruppert
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
1970
1980
year
1990
2000
Residual plots: Squared standardized residuals
Statistics for
Financial
Engineering:
Some R
Examples
1.0
autocorrelation function
David Ruppert
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
0.4
0.2
Data
Transformations:
some theory
0.0
Default probabilities
ACF
Nonlinear
Regression
0.6
0.8
Introduction
GARCH models
Bayesian
estimation of
expected
returns
0
10
20
Lag
30
40
GARCH(p, q) model
Statistics for
Financial
Engineering:
Some R
Examples
The GARCH(p, q) model is
at = t σt ,
David Ruppert
Introduction
where
Nonlinear
Regression
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
v
u
q
p
X
X
u
2 +
2 .
σt = tα0 +
αi at−i
βi σt−i
i=1
i=1
and
t is an iid (strong) white noise process
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
at is weak white noise
uncorrelated but with volatility clustering
GARCH(1,1) fit using garch in tseries
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
Introduction
Nonlinear
Regression
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
Call:
garch(x = std_drift_resid^2, order = c(1, 1))
Model:
GARCH(1,1)
Coefficient(s):
Estimate Std. Error
a0
0.27291
0.00148
a1
0.44690
0.00252
b1
0.80490
0.00075
t value Pr(>|t|)
184
<2e-16 ***
177
<2e-16 ***
1073
<2e-16 ***
Box-Ljung test
data: Squared.Residuals
X-squared = 0.13, df = 1, p-value = 0.7186
GARCH: estimated conditional standard deviations
Statistics for
Financial
Engineering:
Some R
Examples
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
3.5
2.5
Default probabilities
1.5
Nonlinear
Regression
0.5
Introduction
GARCH conditional std
David Ruppert
GARCH models
Bayesian
estimation of
expected
returns
0
2000
4000
6000
year
8000
10000
GARCH: squared residuals with lowess smooth
Statistics for
Financial
Engineering:
Some R
Examples
●
●●
●
●
●
●●
●
●
●
●
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
6
5
4
3
2
Data
Transformations:
some theory
1
Default probabilities
0
Nonlinear
Regression
GARCH squared residuals
Introduction
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●●
● ●● ●
● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●●
●
●
●
● ●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
●
●
● ● ● ●
●●
●
● ●
●
●
●
●●
●
●
●
●
●
● ● ●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
● ●
●
● ●
● ●
●
●
●
● ● ●●
●●
●
●
●
●●
●
●● ●
●
●
●
●
● ●
● ●
●
●
● ● ●●● ●
●
●
●●
● ●●●
● ● ●●
●
●
●
●
●
●
●
● ● ●
●
●●
● ●
●●
● ●●
●● ●
●
●
●
●
●
●
●
●
●
● ●●
●● ●●●
●
●
●
●
● ● ●●
● ● ●●
●●
●
●
●
●
●
●
●
●
●●
● ●
●●
● ●●
● ●
●
● ●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●●●
●
●● ●
●
●
●
●
●
●
● ●●
●●
●
●
●●●
●●●
●
●●
● ●● ●● ●
●
●
● ●
●
●● ●
● ●
●●● ●
● ●●●● ●
●●
●
● ●● ●●●
●
●●
●
●
●● ●
●●
● ●●
● ●● ●
●● ●
●● ●
●●
● ●●
●
● ●● ●
●
●
●
● ●
●
●
●
●
●
●
●
●
● ● ●● ● ●
●
● ●●
●
●●
● ●
●
●●
●● ●●
● ●
●
●●
●●
●
●
● ●
● ●●
●
●
● ●
●
● ●●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
● ●
●
●
●
●●
● ● ●●
●
●
●●●●●
●●●
●
● ●●
● ●● ● ●●
●
●
●
●● ● ● ● ●● ● ●
● ●● ● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
● ●
●●
●●
●
●
●●●
● ●
● ●●● ●
●
●●●
● ● ●●
● ●● ●
●
●●
● ● ●●● ●
●●●
●
●
●
●
●●
●
● ● ●
●●●
●
●●
●
● ● ● ●
●●
● ●
● ● ● ●●
●
● ● ●● ●
●●
●● ●
●
●●
● ● ●●●●●
●●
● ● ● ●● ● ●
● ●● ● ● ●●
● ●●
●●● ●
●
●
●
● ●
●
● ●●
● ●●● ● ●●
● ●● ● ●
●●
● ●●● ●●
●●
●
●●
●
● ●●
●
●● ● ●●●
●
● ●
●
●
● ● ●●●●
●● ● ●
●●● ●
●● ● ● ●●● ● ●
● ●●●● ●●
●
●● ● ●
●●
● ●
●●
●
●
●● ● ●●●
●●
● ● ● ●
●
●● ●
●
●
●●
●● ●
●
●
●● ●●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
● ●●
●
● ●● ● ●
●●●● ●● ● ● ●
●● ●
●
●●
●
●●● ●● ● ●●
●● ●
●
●●● ●
●
●●
●●●
● ●●
●●●
●●
●
● ●●
● ●
●
●●
● ●● ●●
●● ●
● ●● ●●
●● ●
●●
●●
●
● ●●● ●
●
●
●● ●● ●● ●●● ●●
●● ●
● ●●●●●
● ● ● ●●
●●
●● ●
● ●
●●● ●
●●● ●
●●
●● ●
● ●●
● ●●●
●
●●
●
●
● ●● ●● ●
●
●●● ●
●
●●●●●●●
● ●●
●
●
●
● ● ●
●●
●
●● ●
●
●
●
●
●● ●
●● ●
● ●●●
● ●●
●
●● ●
● ● ● ●●● ● ●
●
●
● ●●●●
●● ●●
●
●●●
●●●
●
●
● ●
●●●
● ●
●
●
●●
●● ●●
● ●●
●●
●
● ●●
●● ●● ●●
●●● ●
●
● ●●● ●● ●
●● ●
●
●
●
● ●●●
●
●
●
●●●●●● ●●
●●
●●● ● ● ●●
●●
●●●
● ● ● ●●●●
●
●●●
● ●●●
●● ● ●
●●
●
●●●
●
●
●●
●●●●●
● ●●●
● ●● ●
●●●
●● ●●
●●
●●
●● ● ●●
●●
●
●●
● ●
●
●●
●● ●
●
●
●● ● ● ● ●●●●
●
●●
●●
●
●●
● ●●
●
● ●
●
●●
●
●
●
●
●●
●●
●●●
●●
●●
●●
●
●●● ●●●
●
●
●●●●●
●
● ●● ●
●●
●●
●● ● ● ●●
●
●● ●
●
●
● ●●● ●
●●●●
●●
● ●
●●
●●
●
●●
●●●
● ●
●●
●● ● ● ● ●
●
●
●
●● ●
●
●●
●●●
●●
●●
●
●●
●
●
●●●
●
● ●●
●●
●
●●
●● ●
●
●
●
● ●
●
●
●●●●●●
●●
●●●
●● ● ● ●
●●● ●
●
●
●●
●●
●
●●●
● ●●●
●●
● ●●
●●●
●●
●●
● ● ● ●●●
●● ●
●●●
●●
●● ● ●
●●
● ● ●●●●
●●
●●
●● ●
●
●
●●●●
● ●●
●
●
●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
● ●● ●●
●
●●●
●●●● ●●●
● ● ●●● ●●●● ● ●
●●● ●
● ●●●
●
● ●● ● ●● ●●
●●
●●
●●
● ●●
●
● ●
●●
●●
●●
●●●● ●●
●●
●
●
●
●
●
●
●
●
●
●●●● ●
●
●
● ●●
●
●●●●●●
● ●●
●
●●
●●
●
●
●
●●
●●●●
●
●
●●
●
●●
●●
●●
●●
●●
●
●● ●
●
●
● ●●
●
●●●
●
●●
●● ●●●
●●
●
●
●
●●●
●●● ●●
●●●●●
●●●
● ●
● ●
●
●
●●
●
●
●●
●
●●
●● ●
●
●
●
●
●● ●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●● ●
●●
●
● ●● ● ●● ●●●●
●
●
● ●● ●
● ●
●● ●●
●
●
●● ●
●
●
●●
●
●●
●
●
●
●●●
● ●●
●● ●●●●
●●
●
●●
●
●●● ●●●●●
●●●●●●●●
●●
● ●
●
●
●
●
●
●●
●●● ●● ●
●
●
●●
●
●
●●●●
●
●●●
●●
●
●●
●
●●● ●●
●
●
●
●
●●● ●
●
●
●●
●
●●
●
● ●
●
● ●●
● ●●●
●
●
●●
●●
●
●●●
●●
●●●
●
●●●●
●
● ●●●
●●●●
●
●
●
●●
●
●●
●
●
●
●●
● ●●
●
●●●
●
●● ●●
●●
●
●
●●
●●●
●
●●●
●●
●
●
●
●
●●
●●
●
●● ●
●
● ●
●
●
●
●●●●
●
● ●
● ●●●●●
●
●●
●●
●●
●●
●
●●●
●
●●●●
●
●●
●●●
● ●
●
●● ●
●●
●●
●●●
● ●●●●
●●●
●●
●
●●●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●●
●●
●
●
●●●
●
●●●
●
●●
●●
●●
●
●●
●●
●
●
●
●
●
●●
●
●●
●●
●
●●
●
●
●●●●●
●●
●
●
●●●●
●●
●●
●
●●●●
●●
●●
● ●●●
●
●●
●
●
●
●●●
●
●
●●
●
●
●
●●
●
●●
●●
●●
●
●
●
●
●
●●
●● ●
●
●●
●
●●
●●
●
●●●●
●
●
●
●
●●
●
●
●●●
●
●●●
●
●
●
●
●
●
●
●
●
●●
● ●●
●
●
●●
●●
●●●● ●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●●●●
●●
●
●
●
●●●●●●●●●
●
●●
●
●
●
●
●
●
●●
●●
●
●●●
●
●●
●●
●
●●●
●●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●● ●●
●●
●
●
●
●
●
●
●● ●
●
●
●
●
●●
●
●●
●●
●
●●
●
●●
●●
●
●
●●●
●
●
●
●
●●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●●●
●
●
●
●
●
●
●
●●
●
●●
●●
●●●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●●
●
●●●
●
●
●
●
●●
●●
●
●
●●●●
●● ●●
●
●
●●●
●●●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●●●●●●●
●●
●
●
●●
●
●
●
●
●
●
●●●●●
●●●
●●●
●
●
●●
●●
●
●
●●●
●●●
●
●
●
●
●
●
●●
●
●●
●●●
●
●
●●
●
●
●
●
●●●
●
●●
●●●
●
●●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●●
●●●
●
●●●●
●
●● ●
●●
●●
●
●
●●
●
●
●
●
●
●●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●●
●
●
●
●●
●●
●
●●
●
●●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●●
●●
●●
●●
●
●
●
●
●●
●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●● ● ● ●
●
●
●
●●
●●
●
●
●
●●
●●●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●●●
●●●●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●●●
●
●●
●●●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
● ●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●●
●●
●●●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●●●●●
●●
●●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
David Ruppert
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
GARCH models
Bayesian
estimation of
expected
returns
0
2000
4000
6000
year
8000
10000
GARCH residuals
Statistics for
Financial
Engineering:
Some R
Examples
GARCH residuals
1.0
David Ruppert
Estimating a
dynamic
model
0.4
Data
Transformations:
some theory
0.2
Default probabilities
ACF
Nonlinear
Regression
0.6
0.8
Introduction
Checking the model:
residual analysis
0.0
Interest rate data
GARCH models
Bayesian
estimation of
expected
returns
0
10
20
Lag
30
40
AR(1)/GARCH(1,1)
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
Introduction
Nonlinear
Regression
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
Call:
garchFit(formula = ~arma(1, 0) + garch(1, 1), data = std_drift_resid)
Mean and Variance Equation:
data ~ arma(1, 0) + garch(1, 1)
[data = std_drift_resid]
Conditional Distribution:
norm
Std. Errors:
based on Hessian
Error Analysis:
Estimate
mu
0.001099
ar1
0.138691
omega
0.008443
alpha1 0.073603
beta1
0.923098
Std. Error
0.007476
0.010468
0.001163
0.005483
0.005457
t value Pr(>|t|)
0.147
0.883
13.248 < 2e-16 ***
7.257 3.96e-13 ***
13.424 < 2e-16 ***
169.158 < 2e-16 ***
AR(1)/GARCH(1,1) residuals
Statistics for
Financial
Engineering:
Some R
Examples
AR(1)/GARCH(1,1) residuals
1.0
David Ruppert
Estimating a
dynamic
model
0.4
Data
Transformations:
some theory
0.2
Default probabilities
ACF
Nonlinear
Regression
0.6
0.8
Introduction
Checking the model:
residual analysis
0.0
Interest rate data
GARCH models
Bayesian
estimation of
expected
returns
0
10
20
Lag
30
40
AR(1)/GARCH(1,1) residuals - QQ plot
Statistics for
Financial
Engineering:
Some R
Examples
Normal Q−Q Plot
Data
Transformations:
some theory
Estimating a
dynamic
model
2
0
Default probabilities
−2
Nonlinear
Regression
Theoretical Quantiles
Introduction
4
David Ruppert
Checking the model:
residual analysis
−4
Interest rate data
●
●
●●
●
●●
●●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
GARCH models
Bayesian
estimation of
expected
returns
−10
−5
0
Sample Quantiles
5
●
AR(1)/GARCH(1,1)
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
Introduction
Nonlinear
Regression
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
Call:
garchFit(formula = ~arma(1, 0) + garch(1, 1), data = std_drift_resid,
cond.dist = "std")
Mean and Variance Equation:
data ~ arma(1, 0) + garch(1, 1)
[data = std_drift_resid]
Conditional Distribution:
std
Error Analysis:
Estimate
mu
0.0001087
ar1
0.0969621
omega 0.0016722
alpha1 0.0664390
beta1 0.9413495
shape 3.9169920
Std. Error
0.0059401
0.0090996
0.0005955
0.0065895
0.0052248
0.1600835
t value Pr(>|t|)
0.018 0.98540
10.656 < 2e-16 ***
2.808 0.00498 **
10.083 < 2e-16 ***
180.169 < 2e-16 ***
24.468 < 2e-16 ***
AR(1)/GARCH(1,1) residuals - QQ plot
Statistics for
Financial
Engineering:
Some R
Examples
QQ−plot using t(3.91)
David Ruppert
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
10
5
0
Data
Transformations:
some theory
−5
Default probabilities
−10
Nonlinear
Regression
Theoretical quantiles
Introduction
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
GARCH models
Bayesian
estimation of
expected
returns
−10
−5
0
Sample quantiles
5
Final model for the interest rate dynamics
Statistics for
Financial
Engineering:
Some R
Examples
∆Rt = µ(Rt−1 ) + σ(Rt−1 )at
David Ruppert
Introduction
Nonlinear
Regression
1
Model was fit in two steps:
Default probabilities
1
Data
Transformations:
some theory
spm in SemiPar
Estimating a
dynamic
model
2
Interest rate data
model at as AR(1)/GARCH(1,1)
garchFit in fGarch
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
estimate µ() and σ()
2
Could the two step be combined?
3
Would combining them change the results?
Reference for spline modeling
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
Introduction
Nonlinear
Regression
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
Semiparametric Regression by Ruppert, Wand, and Carroll
(2003)
Lots of examples.
But most from biostatistics and epidemiology
Bayesian statistics
Statistics for
Financial
Engineering:
Some R
Examples
Bayesian analysis allows the use of prior information
David Ruppert
hierarchical priors can:
Introduction
Nonlinear
Regression
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
specify knowledge that a group of parameters are similar to
each other
estimate their common distribution
WinBUGS can be run from inside R using the R2WinBUGS
package
there is a similar BRugs package that runs OpenBugs
BRugs is no longer on CRAN
Data
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
Introduction
Nonlinear
Regression
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
midcapD.ts in fEcofin package
500 daily returns on:
20 stocks
market
Goal
Statistics for
Financial
Engineering:
Some R
Examples
The goal is to use the first 100 days to estimate the mean
David Ruppert
returns for the next 400 days
Introduction
Nonlinear
Regression
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
Four possible estimators:
sample means
Bayes estimation (shrinkage)
mean of means (total shrinkage)
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
CAPM
(expected return) = beta × (expected market return)
Who won?
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
Introduction
Nonlinear
Regression
Estimate
Sum of squared errors
sample means
1.9
Bayes
0.17
mean of means
0.12
Data
Transformations:
some theory
CAPM 1
0.66
Estimating a
dynamic
model
CAPM 2
0.45
Default probabilities
Interest rate data
Checking the model:
residual analysis
Squared estimation errors are summed over the 20 stocks
GARCH models
Bayesian
estimation of
expected
returns
CAPM 1: use mean of first 100 market returns
CAPM 2: use mean of last 400 market returns
Why does shrinkage help?
Statistics for
Financial
Engineering:
Some R
Examples
sample means
David Ruppert
Bayes
Nonlinear
Regression
●
●
0.2
0.2
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
−0.2
Checking the model:
residual analysis
●
●
●
●
●
●
●
●
●
●
●
●
●
−0.2
Interest rate data
mean
Estimating a
dynamic
model
●
●
●
●
●
mean
●
●
●
Default probabilities
Data
Transformations:
some theory
●
0.6
0.6
Introduction
●
●
●
GARCH models
Bayesian
estimation of
expected
returns
1.0
estimate
1.4
1.8
target
1.0
estimate
1.4
1.8
target
Likelihood and prior
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
ri,t = tth return on i stock
Likelihood:
ri,t = µi + i,t
Introduction
Nonlinear
Regression
i,t ∼ IN (0, σ2 )
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
GARCH models
Bayesian
estimation of
expected
returns
IN = “Independent Normal”
Hierarchical Prior:
µi ∼ IN (α, σµ2 )
Diffuse (non-informative) priors on α, σ2 , σµ2
Auto and cross-sectional correlations are ignored (treated as 0)
Data-driven shrinkage
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
Hierarchical Prior:
µi ∼ IN (α, σµ2 )
Introduction
Nonlinear
Regression
the µi are shrunk towards α
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
Interest rate data
Checking the model:
residual analysis
α should be (approximately) the mean of the means
σµ2 /σ2 controls the amount of shrinkage
large σµ2 /σ2 ⇒ less shrinkage
data-driven shrinkage
GARCH models
Bayesian
estimation of
expected
returns
because σµ2 and σ2 are estimated
WinBUGS output
Statistics for
Financial
Engineering:
Some R
Examples
David Ruppert
Introduction
Nonlinear
Regression
Default probabilities
Data
Transformations:
some theory
Estimating a
dynamic
model
> print(means.sim,digits=3)
Inference for Bugs model at "midCap.bug", fit using WinBUGS,
3 chains, each with 5100 iterations (first 100 discarded)
n.sims = 15000 iterations saved
mean
sd
2.5%
25%
50%
75%
97.5% Rhat n.eff
mu[1]
1.1e-01 0.169
-0.22 -1.0e-03 1.1e-01 2.1e-01 4.5e-01
1 4000
mu[2]
1.2e-01 0.170
-0.20 1.5e-02 1.2e-01 2.3e-01 4.7e-01
1 6500
mu[3]
7.7e-02 0.168
-0.27 -2.7e-02 7.9e-02 1.8e-01 4.1e-01
1 3300
mu[4]
4.5e-02 0.176
-0.33 -6.1e-02 5.2e-02 1.6e-01 3.8e-01
1 1300
mu[18]
mu[19]
mu[20]
sigma_mu
sigma_eps
alpha
deviance
8.3e-02
5.1e-02
4.8e-02
1.5e-01
4.3e+00
8.8e-02
1.2e+04
0.170
-0.27 -2.4e-02 8.7e-02 1.9e-01 4.1e-01
0.171
-0.32 -5.1e-02 5.7e-02 1.6e-01 3.7e-01
0.175
-0.33 -5.8e-02 5.5e-02 1.6e-01 3.7e-01
0.065
0.06 9.9e-02 1.3e-01 1.8e-01 3.1e-01
0.068
4.18 4.3e+00 4.3e+00 4.4e+00 4.4e+00
0.102
-0.11 1.7e-02 8.8e-02 1.6e-01 2.8e-01
3.989 11510.00 1.2e+04 1.2e+04 1.2e+04 1.2e+04
1 3000
1 1700
1 1800
1
520
1 15000
1
710
1 5300
Interest rate data
Checking the model:
residual analysis
For each parameter, n.eff is a crude measure of effective sample size,
and Rhat is the potential scale reduction factor (at convergence, Rhat=1).
GARCH models
Bayesian
estimation of
expected
returns
DIC info (using the rule, pD = Dbar-Dhat)
pD = 4.1 and DIC = 11521.6
DIC is an estimate of expected predictive error (lower deviance is better).