Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Teaching (Monte-Carlo) Spreadsheet Simulation Roger Grinde, [email protected] University of New Hampshire Files: http://pubpages.unh.edu/~rbg/TMS/TMS_Support_Files.html 5/25/2017 1 Simulation in Spreadsheets Do you teach simulation? In which courses? With spreadsheets? Add-Ins? Monte Carlo? Discrete Event? Do you use simulation to help teach other topics? Do other courses (not taught by you) use simulation? 5/25/2017 2 Session Overview Learning Goals Motivations Examples that Work Well Examples Posing Difficulties Foundations of Simulation Concept Coverage Through Examples Learning Goals (Revisited) Issues to Consider 5/25/2017 3 Learning Goals What are your learning goals when teaching simulation? 5/25/2017 4 Concept Coverage Through Examples Philosophy: Expose students to a number of application areas, “sneaking” in the concepts along the way. Counter to the way many of us were taught. Key: We need to clearly understand which concepts we’re trying to convey with each example. 5/25/2017 5 5/25/2017 X X X X X X X X X X X X X X X X X X X X X X X X X X X X Games/Tournaments X X Personal Financial Planning Multiple Project Selection Stock Price Modeling, Option Pricing Inventory (multiperiod) X X X X Queuing X Capital Project NPV Extension of other analaysis tools Is simulation needed? Variety of probability distributions Model-building issues (where a simulation model would be different than a deterministic model) Output distribution as function of input distributions Historical/empirical data Summary statistics Alternate decision criteria & risk measures Sources of error Correlation and/or relationships among input variables Optimization concepts in simulation Portfolio Allocation Learning Goal/Objective Inventory (singleperiod) Mapping: Goals to Examples X X X X X X X X X X X X X X X X X X X X X X 6 Motivations Two investment alternatives A: Invest $10,000. B: Invest $10,000 Probability of a $100,000 gain is 0.10 Probability of a $10,000 loss is 0.90 Probability of a $500 gain is 1.0 Which would you choose? Why? 5/25/2017 7 Motivations (continued) On Average, “A” is twice as good as “B”! Do we ever actually receive the average? Decisions made based only on the average can be very poor. Other examples 5/25/2017 8 Motivations: Simulation and Risk Analysis Simulation allows us to evaluate the risk of a particular situation. Risk: Typically defined as the uncertainty associated with an undesirable outcome (such as financial loss). Risk is not the same as just being uncertain about something, and is not just the possibility of a bad outcome. Risk considers the likelihood of an undesirable outcome (e.g., the probability) as well as the magnitude of that outcome. 5/25/2017 9 Simulation Model Schmatic Fixed (Known) Inputs Random (Uncertain) Inputs Simulation Model Outputs & Performance Measures Decision Variables Same basic schematic throughout course Concept of an output “distribution.” 5/25/2017 10 Examples that Work Well Fundamentals: Dice Roller, Interactive Simulation Tool Personal Decisions: Car Repair/Purchase Decision, Portfolio (single period, based on CB Model), College Funding (based on Winston & Albright) Capital Project Evaluation: Truck Rental Company (based on Lawrence & Weatherford), Project Selection/Diversification (CB Model), Product Development & Launch (CB Model) Finance: Stock Price Models, Option Pricing, Random Walks, Mean Reverting Processes 5/25/2017 11 Examples (continued) Inventory: DG Winter Coats (NewsVendor), Antarctica (multi-period, based on Lapin & Whisler) Queuing: QueueSimon (Armonn Ingolfsson) Games/Tournaments: NCAA Tourney (based on Winston & Albright) Simulation in Teaching Other Topics: Revenue Management Illustration Crystal Ball Features: CB Macros, CB Functions 5/25/2017 12 Examples Posing Difficulties for Spreadsheets Multi-server queues and queue networks Most production systems Business process redesign However, some add-ins do exist for simple discrete-event models (e.g., SimQuick by David Hartvigsen) 5/25/2017 13 Foundations of Simulation Randomness, Uncertainty Probability Distributions Tools Dice Roller (John Walkenbach: http://www.jwalk.com/ss) Die Roller (modified) Die Roller (modified for investment “game”) Interactive Simulation Tool 5/25/2017 14 From “What If” to “Wow” Simulation as an Extension of Other Methodologies Spreadsheet Engineering, Base Case What-If Analysis Sensitivity Analysis Scenario Analysis Simulation Comparison of Analysis Methodologies 5/25/2017 15 Extending Other Methodologies Familiar Example/Case Students provided with some probability distribution information Develop comfort with mechanics of simulation See the “value added” of simulation Provides entry point for discussion of important questions 5/25/2017 16 Example: Watson Truck Adapted from Lawrence & Weatherford (2001) Students have built base-case model, and have done sensitivity analysis Examples Base Case Sensitivity Analysis Simulation 5/25/2017 17 Watson Truck: Inputs A 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 B C D E F G Watson Truck Rental Parameters/Uncontrollable Inputs Purchase Price Property Tax Var Cost/Truck # Trucks Prop Tax Growth Truck Cost Growth Base Truck Rental Rate % Trucks Rented @ $1000 Rental Rate Slope Rental Rate Inflation Business Sale Multiplier Discount Rate $1,000,000 $35,000 $4,800 50 4% 7% $1,000 60% 7% 9% 3 10.0% Decision Variable Rental Rate (decision variable) $1,000 Intermediate Calculations Rental Rate Slope % Trucks Rented -0.07% 60.0% 5/25/2017 per year per year per month per $100 reduction in rental rate Sales price assumed to be 3*(year 3 revenues) per $1 increase in rental rate 18 Watson Truck: Base Case Model 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 A B Primary Output Net Present Value (@ discount rate) C D E F G H C25: =NPV(C15,D40:F40)+C40 $209,769 Cash Flow Model 0 Cash Inflows Truck Rental Income Business Sale Total Inflows Cash Outflows Purchase Price Property Tax Truck Var Cost Total Outflows 1 2 3 $360,000 $392,400 $360,000 $392,400 $427,716 $1,283,148 $1,710,864 $1,000,000 $35,000 $240,000 $275,000 $36,400 $256,800 $293,200 $37,856 $274,776 $312,632 ($1,000,000) $85,000 $99,200 $1,398,232 F30: =E30*(1+$C13) $0 $1,000,000 F31: =C14*F30 F32: =SUM(F30:F31) F36: =E36*(1+$C8) F37: =E37*(1+$C9) F38: =SUM(F35:F37) F40: =F32-F38 Net Cash Flow 5/25/2017 19 Watson Truck: Sensitivity Analysis Tornado Sensitivity Chart Output Measure $0 $50,000 $100,000 $150,000 $200,000 $250,000 $300,000 $350,000 $400,000 $450,000 $500,000 Base Truck Rental Rate % Trucks Rented @ $1000 # Trucks Parameter Purchase Price Business Sale Multiplier Var Cost/Truck Discount Rate Rental Rate Inflation Rental Rate Slope Property Tax Truck Cost Growth Prop Tax Growth 5/25/2017 -10 Pct +10 Pct 20 Watson: Simulation Forecast: Net Present Value (@ discount rate) 1,000 Trials Frequency Chart 1,000 Displayed .035 35 .026 26.25 .018 17.5 .009 8.75 Mean = $232,119 .000 0 ($525,250) ($139,570) $246,111 $631,792 $1,017,472 Certainty is 82.00% from $0 to +Infinity Dollars 5/25/2017 21 Learning Goals Addressed (at least partially) Linkage with other course/functional area What inputs should we simulate? Useful probability distributions. Choice of parameters. Concept of an output distribution Simulation in context with other tools What results are important? Sources of error in simulation Simulation mechanics 5/25/2017 22 Sources of Error in Simulation What are some of the sources of error in a spreadsheet simulation model/analysis? 5/25/2017 23 Example: Single-Period Portfolio Simple example, but helps address a number of learning goals Do we need to simulate? Precision of estimates from simulation Confidence vs. Prediction (certainty) intervals Effect of correlation among input quantities Quantification of risk, multiple decision criteria Optimization concepts within simulation context 5/25/2017 24 Spreadsheet B 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 C D E Portfolio Allocation Model Investments Money Market fund Income fund Growth and Income fund Aggressive Growth fund Total amount available Decision variables Money Market fund Income fund Growth and Income fund Aggressive Growth fund Total expected return 5/25/2017 Annual return 3.0% 5.0% 7.0% 11.0% $100,000 Amount invested $25,000 $25,000 $25,000 $25,000 $6,500 Lower bound $0 $10,000 $0 $10,000 Upper bound $50,000 $25,000 $80,000 $100,000 25 Do we need simulation? Assuming we know the distributions for the returns, do we need simulation to compute the expected return of the portfolio? variance of the portfolio? tail probabilities? What if the returns of the securities are correlated? What is the effect of correlation? 5/25/2017 26 Correlation of Returns Large Stocks Large Growth Stocks Large Value Stocks Small Stocks Small Growth Stocks Small Value Stocks Foreign Stocks Bonds Large Stocks Large Growth Stocks Large Value Stocks 1 0.958411 0.901159 0.720036 0.755265 0.507037 0.391551 0.366054 1 0.74063 0.606356 0.720758 0.322101 0.294704 0.267404 1 0.776748 1 0.678815 0.921266 1 0.715396 0.875155 0.618755 1 0.454882 0.275857 0.325624 0.140232 1 0.465424 0.28663 0.164181 0.369522 0.112362 Small Stocks Small Growth Stocks Small Value Stocks Foreign Stocks Bonds 1 Based on Standard & Poor Micropal, via Franklin/Templeton Investor Topics Update, Winter 2001 (Asset Returns from 1980-2000) 5/25/2017 27 Results (n=1000) No Correlation Mean = $6842 Standard Deviation = $5449 5% VaR = ($2165) Positive Correlation Mean = $6409 Standard Deviation = $7386 5% VaR = ($5655) 5/25/2017 28 Decision Criteria What criteria are important for making decision as to where to invest? Measures of risk. Simulation gives us the entire output distribution. Entry point for optimization within simulation context Alternate scenarios, efficient frontier, OptQuest, RiskOptimizer, etc. 5/25/2017 29 Crystal Ball Functions and Simple VBA Control Crystal Ball provides built-in functions Control through VBA Distribution Functions (e.g., CB.Normal) Functions for Accessing Simulation Results (e.g., CB.GetForeStatFN) For some students, can be a “hook.” Allows one to prepare a simulation-based model for someone else who doesn’t know Crystal Ball. Example 5/25/2017 30 Precision of Results: Confidence Intervals Students can calculate a confidence interval for the mean? Do they know what it means? 5/25/2017 31 Sample Results (Portfolio Problem) Statistics: Trials Mean Median Mode Standard Deviation 90% Confidence Interval Standard Error $233.56 Z 1.645 Lower Limit $6,025 Upper Limit $6,794 What does that confidence interval mean? Value 1000 $6,409 $6,531 --$7,386 Common (student) error What does this imply about an individual outcome? For example, from any single year? 5/25/2017 32 Sample Results (cont) Percentile 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% 5/25/2017 dollars ($16,088) ($5,655) ($3,052) ($1,008) $271 $1,277 $2,498 $3,484 $4,307 $5,365 $6,531 $7,695 $8,349 $9,310 $10,386 $11,419 $12,431 $13,896 $15,689 $18,659 $30,330 What do these results mean? What is the 90% “prediction” (or “certainty”) interval? 33 Confidence and Prediction Intervals 90% Confidence Interval for the Mean 90% Prediction Interval (centered around median) ($6025, $6794) (-$5655, $18,659) Note: Crystal Ball uses the term “certainty”) Students: Understand the difference? Understand when one is more appropriate than the other? 5/25/2017 34 Precision of Simulation Results Since we know the true value of the mean (for the portfolio problem), this can be a good example to look at precision and sample size issues. Crystal Ball: Precision control for mean, standard deviation, and percentiles. Simulation stops when precision reached Confidence interval for proportion or for a given percentile sometimes makes more sense. 5/25/2017 35 Crystal Ball: Precision Control Nice way to illustrate effect of sample size. Precision Control stops simulation based on user-specified precision on the mean, standard deviation, and/or a percentile. Example (Portfolio Allocation) Example (Option Pricing) 5/25/2017 36 Learning Objectives (Revisited) General Probability Distributions Statistics Relationships Among Variables Decision Making 5/25/2017 37 Possible Learning Goals General Use simulation as an extension of other analysis tools Apply simulation to a variety of business problems Identify when simulation is and is not needed to analyze a situation Probablilty Distributions Understand and use probability distributions to model phenomena Describe the output distribution, understanding this to be a function of the input distributions Use historical/empirical data and subjective assessments appropriately in choosing distributions and parameters 5/25/2017 38 Possible Learning Goals (cont) Statistics Correctly interpret summary statistics, including percentiles/histograms Correctly interpret confidence and prediction (certainty) intervals Identify sources of error in simulation, apply to specific situations Relationships Among Variables Include appropriate correlation and/or other relationships when model building Describe the effect of correlation and/or other relationship on simulation results 5/25/2017 39 Possible Learning Goals (cont) Decision Making Identify and correctly use different risk measures Use appropriate criteria in making recommendations Use optimization concepts in a simulation application 5/25/2017 40 Difficult Issue (for me) Decide which learning goals are the most important, and structure coverage so those goals are attained. Student backgrounds Time constraints Overall course objectives Mapping of learning goals to examples that you will use. 5/25/2017 41 Mapping: Learning Goals to Examples Simulation, Statistidcal, Spreadsheet Modeling, Decision Making Concepts Examples/Application Areas 5/25/2017 42 5/25/2017 X X X X X X X X X X X X X X X X X X X X X X X X X X X X Games/Tournaments X X Personal Financial Planning Multiple Project Selection Stock Price Modeling, Option Pricing Inventory (multiperiod) X X X X Queuing X Capital Project NPV Extension of other analaysis tools Is simulation needed? Variety of probability distributions Model-building issues (where a simulation model would be different than a deterministic model) Output distribution as function of input distributions Historical/empirical data Summary statistics Alternate decision criteria & risk measures Sources of error Correlation and/or relationships among input variables Optimization concepts in simulation Portfolio Allocation Learning Goal/Objective Inventory (singleperiod) Mapping: Goals to Examples X X X X X X X X X X X X X X X X X X X X X X 43 Common Student Errors Thinking of simulation as the method of first choice. Simulating too many quantities. Too much focus on distribution/parameter selection or on the numerical results, not enough on insights/decision. Misinterpretation of results, especially confidence intervals Modeling: Using same return, lead time, etc. for every time period/order, etc. (difference between deterministic and simulation models) Choosing the assumptions, distributions, parameters, etc. that give the “best” numerical results. 5/25/2017 44 Issues to Consider Teaching environment (lab setting or not?) Role of course in curriculum Use add-ins for Monte-Carlo simulation? Teach Discrete-Event simulation? How much of the “quant” course should be devoted to simulation? 5/25/2017 45 Conclusions, Discussion Files available at http://pubpages.unh.edu/~rbg/TMS/TMS_ Support_Files.html 5/25/2017 46 Student Project Example (MBA) PPT File Excel File 5/25/2017 47