Download Monte Carlo Analysis

Monte Carlo Analysis
A Technique for Combining
Distributions
Purpose of lecture



Introduce Monte Carlo Analysis as a tool
for managing uncertainty
To demonstrate how it can be used in
the policy setting
To discuss its uses and shortcomings,
and how they are relevant to policy
making processes.
FIN 591: Financial Modeling, Spring 2004
2
What is Monte Carlo Analysis?



It is a tool for combining distributions,
and thereby propagating more than just
summary statistics
It uses a random number generation,
rather than analytic calculations
It is increasingly popular due to high
speed personal computers.
FIN 591: Financial Modeling, Spring 2004
3
Background/History




“Monte Carlo” from the gambling town
of the same name (no surprise)
Limited use because time consuming
Much more common since late 80’s
Too easy now?
FIN 591: Financial Modeling, Spring 2004
4
Why do Monte Carlo Analysis?



Combining distributions
With more than two distributions,
solving analytically is very difficult
Simple calculations lose information



Mean  mean = mean
95% %ile  95%ile  95%ile!
Gets “worse” with 3 or more distributions.
FIN 591: Financial Modeling, Spring 2004
5
Monte Carlo Analysis

Takes an equation





Example: Risk = probability  consequence
Draws randomly from defined
distributions
Multiplies, stores
Repeats this over and over and over…
Results displayed as a new, combined
distribution.
FIN 591: Financial Modeling, Spring 2004
6
Simple Example


Skin cream additive is an irritant
Many samples of cream provide information
on concentration:



mean 0.02 mg chemical/application
standard dev. 0.005 mg chemical/application
Two tests show probability of irritation given
application


low p(effect per mg exposure)=0.05 / mg
high p(effect per mg exposure)=0.10 / mg.
FIN 591: Financial Modeling, Spring 2004
7
Skin cream additive data
Potency
Exposure
Information
type
{low, high}
Mean,
deviation
Data
{0.05, 0.10}
Distribution?
Uniform?
Triangular?
0.02 mg,
0.005 mg
Normal?
Lognormal?
FIN 591: Financial Modeling, Spring 2004
8
Analytical Results

Risk = Exposure  potency

Mean risk = 0.02 mg  0.075 / mg
= 0.0015
or 0.15% probability that someone using the cream
will be irritated.
FIN 591: Financial Modeling, Spring 2004
9
Analytical results

“Conservative estimate”

Use upper 95th %ile
Risk = 0.03 mg  0.0975 / mg
= 0.0029
or p(irritation|application) = 0.29%.
FIN 591: Financial Modeling, Spring 2004
10
Monte Carlo: Visual example
0.01
0.02
0.03
Exposure (mg
chemical)
0.05
0.10
Potency (probability of
irritation per mg chemical)
Exposure = normal (mean 0.02 mg, s.d. = 0.005 mg)
Potency = uniform (range 0.05 / mg to 0.10 / mg)
FIN 591: Financial Modeling, Spring 2004
11
Random Draw One
0.0165
0.063
0.01
0.02
0.03
Exposure (mg
chemical)
0.05
0.10
Potency (probability of
irritation per mg chemical)
p(irritate) = 0.0165 mg × 0.063 / mg = 0.0010
FIN 591: Financial Modeling, Spring 2004
12
Random Draw Two
0.0175
0.01
0.089
0.02
0.03
Exposure (mg
chemical)
0.05
0.10
Potency (probability of
irritation per mg chemical)
p(irritate) = 0.0175 mg × 0.089 / mg = 0.0016
Summary: {0.0010, 0.0016}
FIN 591: Financial Modeling, Spring 2004
13
Random Draw Three
0.057
0.0152
0.01
0.02
0.03
Exposure (mg
chemical)
0.05
0.10
Potency (probability of
irritation per mg chemical)
p(irritate) = 0.152 mg × 0.057 / mg = 0.0087
Summary: {0.0010, 0.0016, 0.00087}
FIN 591: Financial Modeling, Spring 2004
14
Random Draw Four
0.0238
0.01
0.02
0.03
Exposure (mg
chemical)
0.085
0.05
0.10
Potency (probability of
irritation per mg chemical)
p(irritate) = 0.0238 mg × 0.085 / mg = 0.0020
Summary: {0.0010, 0.0016, 0.00087, 0.0020}
FIN 591: Financial Modeling, Spring 2004
15
After Ten Random Draws
Summary
{0.0010, 0.0016, 0.00087, 0.0020,
0.0011, 0.0018, 0.0024, 0.0016,
0.0015, 0.00062}
Mean = 0.0014
Standard deviation = (0.00055).
FIN 591: Financial Modeling, Spring 2004
16
Using software



Could write this program using a random
number generator
But, several software packages exist
I use @Risk



User friendly
Customizable
RNG good up to about 10,000 iterations.
FIN 591: Financial Modeling, Spring 2004
17
100 iterations (less than
two seconds)

Monte Carlo results



Mean
Standard Deviation
0.00161
0.00048
Compare to analytical results


Mean
standard deviation
FIN 591: Financial Modeling, Spring 2004
0.0015
n/a.
18
Summary chart - 100 trials
Forecast: P(Irritation)
100 Trials
Frequency Chart
1 Outlier
.050
5
.038
3.75
.025
2.5
.013
1.25
.000
0
0.00
0.00103
0.00
FIN 591: Financial Modeling, Spring 2004
0.00
0.00161
0.00
0.00
0.00311
19
Summary - 10,000 Trials

Monte Carlo results



Mean
Standard Deviation
0.00150
0.000472
Compare to analytical results


Mean
standard deviation
FIN 591: Financial Modeling, Spring 2004
0.00150
n/a.
20
Summary chart - 10,000 trials
Forecast: P(Irritation)
10,000 Trials
Frequency Chart
88 Outliers
.023
226
.017
169.5
.011
113
.006
56.5
.000
0
0.00
0.00069
0.00
FIN 591: Financial Modeling, Spring 2004
0.00
0.00150
0.00
0.00
0.00331
21
Issues: Sensitivity Analysis



Which input distributions have the
greatest effect on the eventual distribution
Which parameters can both be influenced
by policy and reduce risks
When better data can be most valuable
(information isn’t free…nor even cheap).
FIN 591: Financial Modeling, Spring 2004
22
Issues: Correlation



Two distributions are correlated when a
change in one is associated with a change
in another
Example: People who eat lots of peas may
eat less broccoli (or may eat more…)
Usually doesn’t have much effect unless
significant correlation (||>0.75).
FIN 591: Financial Modeling, Spring 2004
23
Generating Distributions


Invalid distributions create invalid results,
which leads to inappropriate policies
Two options


Empirical
Theoretical.
FIN 591: Financial Modeling, Spring 2004
24
Empirical Distributions


Most appropriate when developed for the
issue at hand.
Example: local fish consumption



Survey individuals or otherwise estimate
Data from individuals elsewhere may be very
misleading
A number of very large data sets have
been developed and published.
FIN 591: Financial Modeling, Spring 2004
25
Empirical Distributions


Challenge: when there’s very little data
Example of two data points





Uniform distribution?
Triangular distribution?
Not a hypothetical issue…is an ongoing
debate in the literature
Key is to state clearly your assumptions
Better yet…do it both ways!
FIN 591: Financial Modeling, Spring 2004
26
Which Distribution?
0.05
0.10
Potency (probability of
irritation per mg chemical)
0.05
0.10
Potency (probability of
irritation per mg chemical)
FIN 591: Financial Modeling, Spring 2004
0.05
0.10
Potency (probability of
irritation per mg chemical)
0.05
0.10
Potency (probability of
irritation per mg chemical)
27
Random number generation


Shouldn’t be an issue…@Risk is good to
at least 10,000 iterations
10,000 iterations is typically enough, even
with many input distributions.
FIN 591: Financial Modeling, Spring 2004
28
Theoretical Distributions



Appropriate when there’s some
mechanistic or probabilistic basis
Example: small sample (say 50 test
animals) establishes a binomial
distribution
Lognormal distributions show up often in
nature, particular economics/business.
FIN 591: Financial Modeling, Spring 2004
29
Some Caveats

Beware believing that you’ve really
“understood” uncertainty



Central tendencies are NOT “real risk”
Distributions are only PART of uncertainty
Beware misapplication


Ignorance at best
Fraudulent at worst.
FIN 591: Financial Modeling, Spring 2004
30
Example (after Finkel 1995)
Alar “versus” aflatoxin
Exposure has two elements
Peanut butter consumption
aflatoxin residue
Juice consumption
Alar/UDMH residue
Potency has one element
aflatoxin potency
UDMH potency
Risk =
(consumption  residue  potency)/body weight
FIN 591: Financial Modeling, Spring 2004
31
Inputs for Alar & aflatoxin
Variable
Units
Mean
5th %ile 95th %ile Percentile location
of the mean.
Peanut butter
g/day
11.38
2.00
31.86
66
g/day
136.84
16.02
430.02
69
aflatoxin residue
g/g
2.82
1.00
6.50
61
UDMH residue
g/g
13.75
0.5
42.00
67
aflatoxin
kg-
17.5
4.02
28.23
61
potency
day/mg
UDMH potency
kg-
0.49
0.00
0.85
43
consumption
Apple juice
consumption
day/mg
FIN 591: Financial Modeling, Spring 2004
32
Alar and Aflatoxin Point
Estimates

Aflatoxin estimates:

Mean  11.38g  2.82g  17.5kg  day  mg
day
g
mg
1000g
20kg
= 0.028

Alar (UDMH) estimates:

Mean = 0.046.
FIN 591: Financial Modeling, Spring 2004
33
Alar and Aflatoxin Monte Carlo


10,000 runs
Generate distributions


(don’t allow 0)
Don’t expect correlation.
FIN 591: Financial Modeling, Spring 2004
34
Aflatoxin and Alar Monte
Carlo Results (Point Values)
Aflatoxin
Mean
Analytical
0.028
Monte Carlo
0.028
Mean
Analytical
0.046
Monte Carlo
0.046
Alar
FIN 591: Financial Modeling, Spring 2004
35
Forecast: peanut butter risk
10,000 Trials
Frequency Chart
192 Outliers
.016
163
.012
122.2
.008
81.5
.004
40.75
.000
0
0
0.0375
0.075
0.1125
0.15
Certainty is 98.05% from -Infinity to 0.1495
FIN 591: Financial Modeling, Spring 2004
36
Forecast: apple juice risk
10,000 Trials
Frequency Chart
125 Outliers
.102
1020
.077
765
.051
510
.026
255
.000
0
0
0.1125
0.225
0.3375
0.45
Certainty is 93.93% from -Infinity to 0.15
FIN 591: Financial Modeling, Spring 2004
37
Forecast: peanut butter risk
10,000 Trials
Cumulativ e Chart
192 Outliers
1.000
10000
.750
.500
.250
.000
0
0
0.0375
0.075
0.1125
0.15
Certainty is 98.04% from -Infinity to 0.1495
FIN 591: Financial Modeling, Spring 2004
38
Forecast: apple juice risk
10,000 Trials
Cumulativ e Chart
125 Outliers
1.000
10000
.750
.500
.250
.000
0
0
0.1125
0.225
0.3375
0.45
Certainty is 93.93% from -Infinity to 0.15
FIN 591: Financial Modeling, Spring 2004
39
End
FIN 591: Financial Modeling, Spring 2004
40