Download Slayt 1 - SIGs of the System Dynamics Society

1
SIGMA Workshop
Part 3: Statistical Screening
Gönenç Yücel
SESDYN Research Group
Boğaziçi University, Istanbul
2
A brief introduction
• What is stat screening?
• What is it good for?
• What does it rely on?
• Altering uncertain (exogenous) model parameters
• Relating the value of an outcome of interest to changes in
parameter values
• Uses correlation coefficient to quantify the degree and direction of
relationship between a parameter and the value of the outcome at
a certain time point
3
Background literature
• Ford, A., & Flynn, H. (2005). Statistical screening of system dynamics
models. System Dynamics Review, 21(4), 273–303.
• Taylor, T., Ford, D., & Ford, A. (2007). Model Analysis Using Statistical
Screening: Extensions and Example Applications . 25th International
Conference of the System Dynamics Society. Boston: System
Dynamics Society.
• Taylor, T. R. B., David N. Ford, & Ford, A. (2010). Improving model
understanding using statistical screening. System Dynamics Review,
26(1), 73–87.
4
Key Concept: Correlation Coefficients
• A measure of the linear correlation (dependence) between
two variables X and Y, giving a value between +1 and −1
inclusive, where
• 1 is total positive correlation,
• 0 is no correlation, and
• −1 is total negative correlation.
5
Demo Model
• Bass diffusion model
• See Business Dynamics by Sterman (2000) for specifications of the
model
• Vensim version of the model is available in the Stat Screening folder
on your computers
6
Selected Variables
1,000
70
1,000
500
35
500
0
0
0
0
10
20
Adopters : Base
adoption rate : Base
Potential Adopters : Base
30
40
50
60
Time (Month)
70
80
90
100
7
Our demo task
• Perform a Statistical Screening Analysis on the demo
model (i.e. Bass diffusion model) to evaluate the relative
influence of
• Two of the exogenous variables (i.e. contact rate and adoption
fraction) and
• The one exogenous initial condition (i.e. initial potential adopters).
• Using the adoption rate (sales) as the main outcome of
interest (performance variable).
8
Procedure
A.
Perform statistical screening to calculate correlation coefficients
and to plot these over time
B.
Select a time period for analysis
C.
Identify high-leverage parameters.
• High-leverage parameters are the parameters with the highest absolute
correlation coefficient values during the selected time
D.
Create a list of high leverage parameters and their related model
structures
E.
Use additional structure-behavior analysis methods (e.g. verbal
reasoning, scenario analysis, behavioral analysis) to explain how
each parameter the structures they influence drive the behavior of
the system.
9
A. Calculating Correlation Coefficients
1.
Select uncertain model input parameters and a single
performance variable for analysis
2.
Specify a distribution for each uncertain model parameter
3.
Simulate the model using a combination of values from the
specified distributions (e.g. Using Vensim’s Sensitivity
Analysis feature)
4.
Export the results of the simulation set
5.
Pick up the Excel template that best fits the simulation set
6.
Import the results from the simulation set to the Excel
template, and observe the plot of the correlation coefficients
10
Steps A.1 & A.2
• Selected parameters, and distributions
• Model parameters to be analyzed
Parameter
Reference
Value
Range to be
Distribution
Tested
Contact rate
0.5
[0.25, 0.75]
Uniform
Adoption
fraction
0.5
[0.25, 0.75]
Uniform
Initial
Adopters
10
[5, 10]
Uniform
• Model output to be analyzed
• Adoption rate
11
A. Calculating Correlation Coefficients
1.
Select uncertain model input parameters and a single
performance variable for analysis
2.
Specify a distribution for each uncertain model parameter
3.
Simulate the model using a combination of values from the
specified distributions (e.g. Using Vensim’s Sensitivity
Analysis feature)
4.
Export the results of the simulation set
5.
Pick up the Excel template that best fits the simulation set
6.
Import the results from the simulation set to the Excel
template, and observe the plot of the correlation coefficients
12
Conducting a set of simulations on
Vensim
• Monte Carlo option in Vensim
• Lets us to specify ranges for the parameters as well as their
distribution
• We will need to specify 2 things
• An input control file (.vsc file)
• An output savelist file (.lst file)
13
14
Step A.3 Simulation with combinations of
parameter values
15
16
17
18
A. Calculating Correlation Coefficients
1.
Select uncertain model input parameters and a single
performance variable for analysis
2.
Specify a distribution for each uncertain model parameter
3.
Simulate the model using a combination of values from the
specified distributions (e.g. Using Vensim’s Sensitivity
Analysis feature)
4.
Export the results of the simulation set
5.
Pick up the Excel template that best fits the simulation set
6.
Import the results from the simulation set to the Excel
template, and observe the plot of the correlation coefficients
19
A.4 Exporting the simulation results
20
21
22
A. Calculating Correlation Coefficients
1.
Select uncertain model input parameters and a single
performance variable for analysis
2.
Specify a distribution for each uncertain model parameter
3.
Simulate the model using a combination of values from the
specified distributions (e.g. Using Vensim’s Sensitivity
Analysis feature)
4.
Export the results of the simulation set
5.
Pick up the Excel template that best fits the simulation set
6.
Import the results from the simulation set to the Excel
template, and observe the plot of the correlation coefficients
23
A.5 Choosing an Excel template
• Choosing the right template!
• Number of parameters
• Number of simulations
• Number of data points in a single run
• In our example, we have
• 3 parameters
• 200 simulations
• 100 data points for each simulation
• The right template would be StatScreenTemplate3inputs200runs100saveperiods.xls
24
A. Calculating Correlation Coefficients
1.
Select uncertain model input parameters and a single
performance variable for analysis
2.
Specify a distribution for each uncertain model parameter
3.
Simulate the model using a combination of values from the
specified distributions (e.g. Using Vensim’s Sensitivity
Analysis feature)
4.
Export the results of the simulation set
5.
Pick up the Excel template that best fits the simulation set
6.
Import the results from the simulation set to the Excel
template, and observe the plot of the correlation coefficients
25
A.6 Importing the simulations results to
the Excel template
26
27
28
29
30
31
Self-study Practice
• Repeat the statistical screening on a modified version of
the simple Bass diffusion model
• Modification:
• Add a quitting flow that flows from the adopters to the potential
adopters stock
• The amount of the flow is defined as
• Adopters * Quitting Fraction
• Reference value of the quitting fraction is set to be 0.1
32