Download X - Cox Associates

Decision Analysis
Lecture 1
Tony Cox
My e-mail: [email protected]
Course web site: http://cox-associates.com/DA/
Agenda
• Learning goals for course
• Course grading, readings, schedule,
homework
• Course overview
• The basics
– Expected utility theory
– Decision tables
– Decision trees
2
Learning Goal 1:
Problem solving
• Formulate, solve simple decision problems
– Nothing tricky, just core concepts, methods
– Decision trees, expected utility calculations,
Bayes’ Rule, applied probability and statistics
http://cse-wiki.unl.edu/wiki/index.php/More_depth_on_influence_diagrams:_Decision_trees,_influence_diagrams,_dynamic_networks
3
Learning Goal 2: Understanding
• Understand main ideas of advanced
methods
– Key concepts, results, algorithms, software
• Where to go for technical details
– Be able to use correctly in research or
applications (open-book environment)
Influence diagram:
• Rectangle = choice/decision
• Ellipse = chance node
• Hexagon = value node
http://cse-wiki.unl.edu/wiki/index.php/More_depth_on_influence_diagrams:_Decision_trees,_influence_diagrams,_dynamic_networks
4
Learning Goal 3: Practical insights
• Behavioral decision theory: Help real people and
organizations make better decisions
• Understanding and overcoming decision traps
– For individuals and organizations
http://slideplayer.com/slide/5699971/
http://http-server.carleton.ca/~aramirez/4406/Reviews/TPham.pdf
5
www.behavioraleconomics.com/introduction-to-be/
Example
• Setting: Your company has invested $8M so far
to develop a new product. The project is not yet
finished.
• If they can finish it successfully, they can sell the
resulting IP for $15M; otherwise, they get $0.
• Decision Question: What is the most additional
money that the company should be willing to
spend to finish the project successfully with
certainty? (Assume good credit.)
6
Example: Sunk costs
• Setting: Your company has invested $8M so far
to develop a new product. The project is not yet
finished.
• If they can finish it successfully, they can sell the
resulting IP for $15M; otherwise, they get $0.
• Decision Question: What is the most additional
money that the company should be willing to
spend to finish the project successfully with
certainty? (Assume good credit.)
• Answer: Up to $15M. (Ignore sunk costs.)
http://lesswrong.com/lw/gu1/decision_theory_faq/#what-is-decision-theory
7
Which elicits greater
willingness-to-pay?
• A: “Purchase new equipment at airport
that will save 150 lives if there is an
accident”
• B: “Purchase new equipment at airport
that will save at least 85% of 150 lives if
there is an accident”
8
Which leads to more releases?
• A: "20 out of every 100" similar patients
will commit an act of violence after release
• B: "20 percent" of similar patients will
commit an act of violence after release
http://onlinelibrary.wiley.com/doi/10.1111/risa.12105/abstract
9
Which leads to more releases?
• A: "20 out of every 100" similar patients
will commit an act of violence after release
• B: "20 percent" of similar patients will
commit an act of violence after release
• (Answer: Psychiatrists are about twice as
likely to keep a patient confined if A is
used instead of B)
http://onlinelibrary.wiley.com/doi/10.1111/risa.12105/abstract
10
Framing affects choice
• A: Surgery described as giving a "68%
chance of being alive” a year after surgery
[44% prefer to radiation treatment]
• B: Same surgery described as giving a
"32% chance of dying" within a year after
surgery [18% prefer to radiation treatment]
http://onlinelibrary.wiley.com/doi/10.1111/risa.12105/abstract
11
So… decision psychology matters!
• Must understand how people and
organizations do make decisions to
understand how they can make them better
• Decision psychology alerts us to traps to
avoid and obstacles to be overcome
– Based on suggestions from previous class, will
incorporate it throughout this one
– Please help to improve this class for future
students!
12
Learning Goal 4: Tools and Skills
• Knowledge
– Apply probability, statistics, and decision
analysis techniques to improve decisions
• Software
– Use R for simple applied probability and
statistics calculations, simulations
– Use Netica for Bayesian Networks and
Influence Diagram calculations
– Use Causal Analytics Toolkit (CAT) for
Excel/R interface and machine learning
13
Some free software tools
Software used in this course:
• R
– Probability and statistics models and calculations
– Simulation scripts
– https://cloud.r-project.org/
• Netica Influence diagram software,
– https://norsys.com/download.html
• Causal Analytics Toolkit (CAT) (Excel/R)
– http://cox-associates.com/downloads/
No previous knowledge of R or Netica is assumed.
14
Learning Goal 5: Research
• Projects (papers/presentations)
– Apply decision analysis to a problem
• Final paper: “Improving decisions in …”
– Write or use decision analysis software
– May survey an advanced area
• Clearly summarize main ideas and methods
– Will discuss research topics further as we go
• Be able to read more about the field
15
Learning Goal 5: Research
Example topics
–
–
–
–
–
–
Oil drilling decisions
Competitive bidding/pricing
Health care decision support
Investment optimization software
Inventory control with supply chain risks
Read and report on Predictable Irrationality,
Priceless, Thinking Fast and Slow, Misbehaving,
Superforecasting, or other books
16
Summary of Learning Goals
• Help teams and organizations make
better decisions under uncertainty
– Predict and evaluate the probable
consequences of alternative choices
– Formulate and solve decision problems using
R and Netica software
– Understand main ideas of advanced methods
– Insights for making better decisions
• Read (and contribute to) current
literature applying decision analysis
• Software skills
• Go change the world for the better!
http://http-server.carleton.ca/~aramirez/4406/Reviews/TPham.pdf
17
Course administration
18
Assignments
• Homework problems are handed out in
each class (except at end)
– They are given in the lecture notes
• Due by 4:00 PM on day of next class to
get credit
• Solutions will be discussed in class
• Communicate with me by e-mail at:
[email protected]
19
Turning in assignments
• E-mail your answers to me by 4:00 PM on
day of next class (next Tuesday)
• Show work (may get partial credit)
• Circle (or box or bold) final answers
(clearly identify)
– I will enter credit for each problem in a
spreadsheet, need to be able to find answer
easily
20
Feedback
• I will usually not return papers (all are electronic,
so you will have your own copy)
• I will maintain a running score of achievement
on homework problems and midterm and final
exams
• I will e-mail those who appear to be in danger of
failing (if any)
• Can e-mail me for cumulative score/grade so far
at any time
• I may send you e-mails related to progress
21
Grading
•
•
•
•
•
Class participation 5%
Homework 25%
Midterm 20% (take-home, open book)
Final 20%
Term project 30%
– Weights may be adjusted for each student to
emphasize strengths
• Grading rubric in syllabus (A’s for >90%,
B’s for 80%-90%, etc.)
22
Grading philosophy
• An “A” in my course is my certification that you
are competent to recognize, formulate, and solve
certain kinds of useful decision problems
– Being very solid on the basics, able to solve problems
correctly
– You and I will partner to get you there
• Being great at research, software development or
use, insightful consulting, etc. also earns credit
– Multiple paths to an “A”
23
Course schedule
• March 14: No class: Take-home midterm
(20%)
• March 21: No class (Spring break)
• March 28: Project/paper proposals due
• April 18: Draft of project/term paper due
• May 9: Final Exam (20%)
• May 16: Project/term paper due (30%)
24
Readings for course
Lecture notes are mostly self-contained, but…
To make best use of scarce time,
we will use
• Selected on-line readings
• Modern surveys of main ideas
• Many great book are available
• Will also provide web links
25
Reading Assignment #1
(Due before next class)
• Required
– Tversky and Kahneman (1981), The framing of
decisions and the psychology of choice.
www.stat.columbia.edu/~gelman/surveys.course/TverskyKahneman1981.pdf
– Hulett, Decision Tree Analysis for the Risk Averse
Organization, pages 1-3 (EMV)
www.projectrisk.com/white_papers/Decision_Tree_Analysis_for_the_Risk_Averse_Organization.pdf
• Optional
–
–
–
Decision trees: http://petrowiki.org/Decision_tree_analysis
Decision analysis, Ron Howard (founding father of influence diagrams, decision analysis)
http://web.stanford.edu/class/cee115/wiki/uploads/Main/Schedule/DAPracticeAndPromise.pdf
Influence diagrams and decision trees
• http://ch.lumina.com/technology/influence-diagrams/
•
http://csewiki.unl.edu/wiki/index.php/More_depth_on_influence_diagrams:_Decision_trees,_influence_diagrams,_dynamic_networks
26
Homework problems
• Essential for mastering basic tools and
skills in the course
• Due by 4:00 PM on day of next class
• Emphasis is on assuring competence and
comfort with
– key problem formulations
– solution methods
• Homework problems may appear on
midterm
27
Class 1, Problem #1: EV calculations
• Which is the better choice, A or B:
– A gives probabilities (0.1, 0.2, 0.6, 0.1) of
values (20, 10, 0, -10)
– B gives probabilities (0.7, 0.3) of values (5, -1)
– Which has the greater expected value (EV)?
• Assume risk neutrality: goal is to maximize EV
• Submit answer by giving two numbers:
EV(A) = ? and EV(B) = ?
28
Class 1, Problem #2: Risk attitude
and insurance decisions
• House is worth $1,200,000
• Probability of loss in any year = 0.05
• Can buy full insurance against loss for a cost of
$100,000 per year. Initial wealth is $1.3M
• Should owner buy the insurance?
• Submit answer by giving the EMV or EU for each
decision (Buy or Do Not Buy)
(a) Solve for risk-neutral owner, u(x) = x
(b) Solve for risk-averse owner if utility function is
u(x) = log(x), x = final wealth = $1.3M + change in wealth
29
Why Problem 1?
• Practice expected-utility (EU) calculations
– EU = sum over all outcomes, x, of p(x)u(x)
– p(x) = probability of x, u(x) = utility of x
• For risk-neutral decision-maker, u(x) = x
– p(x) may depend on what decision is chosen
• Draw and solve simple decision trees
• Understand role of utility function,
uncertain state, and evaluation of
alternative choices in decision analysis
30
Class 1, Problem # 3: Decision
tree analysis
a. Which decision (install scrubbers, order new cleaner
coal, or install new transmission line to hydroplant)
maximizes EMV (expected monetary value)?
b. For what range of scrubber prices (shown here as $3M)
is “Install scrubbers” the optimal (EMV-maximizing)
decision?
http://cse-wiki.unl.edu/wiki/index.php/More_depth_on_influence_diagrams:_Decision_trees,_influence_diagrams,_dynamic_networks
31
Skill 1: Expected value and
expected utility (EU) calculations
32
Core skills: Calculating
expected values in R
• Expected value of a numerical random
variable (r.v.), X
• Expected monetary value, EMV(X)
• Expected utility, EU(X)
• Certainty equivalent, CE(X)
• Why bother? Because a rational d.m.
should always pick the act with the
greatest value of EU(X) and CE(X).
– Justification to be given soon. (N-M axioms)
33
To solve problems, you must know how
to calculate expected values
• Formula: E(X) = Sum over all possible
consequence values, x, of Pr(x)*x
• Math notation: EMV(X) = ∑xp(x)*x
– EMV = expected monetary value
– X is a random variable (r.v., meaning an
uncertain quantity with many possible values)
representing monetary value
– x = a possible value (a number) for X
– p(x) denotes the probability of x (that X = x)
– ∑x means “sum over all x values”
34
Calculating expected values
• A ticket has a 45% chance of winning $10,
else it wins nothing. Calculate its EMV
35
Calculating expected values
• A ticket has a 45% chance of winning $10,
else it wins nothing. Calculate its EMV
• Answer: (0.45)*($10) + (0.55)*($0) = $4.5
– This is just EMV(X) = ∑xp(x)*x
– Using R:
> x <- c(10, 0); p <- c(0.45, 0.55);
EMV <- sum(x*p); EMV
[1] 4.5
>
36
Calculating an EMV in CAT
This R script can be selected, then
runfrom CAT’s Excel ribbon:
Input script (in one column)
R:
R:
R:
R:
Output (in the next column)
x <- c(10, 0)
p <- c(0.45, 0.55)
EMV <- sum(x*p)
EMV
37
Expected utility calculations in R
• EU(X) = ∑xp(x)*u(x)
– u(x) is the von Neumann-Morgenstern (N-M) utility of
outcome x.
• u(x) is a number, often (but not always) scaled to lie between
0 (worst, least-preferred) and 1 (best, most-preferred).
• x can be any outcome (not necessarily a number)
• Example: Calculate the expected utility of a
prospect that yields $100 with probability 0.6,
else $10, for a decision-maker whose utility
function is u(x) = log(x)
– Note: In R, “log(x)” means ln(x) (natural logarithm)
38
Defining a function in R
• Define the utility function, as follows:
u <- function(x){
value <- log(x)
return(value)
}
R: u <- function(x){
value <- log(x)
return(value)
}
• Application: Calculate expected utility
R: x <- c(10, 100)
R: p <- c(0.4, 0.6)
R: EU <- sum(p*u(x))
• Result: EU(X) = 3.68
R: x <- c(10, 100)
R: p <- c(0.4, 0.6)
x <- c(10, 100)
R: EU <- sum(p*u(x))
R: EU
p <- c(0.4, 0.6)
EU <- sum(p*u(x))
EU
[1] 3.684136
39
EU calculations in R and CAT
• In CAT software, paste function definition (from
word processor, .ppt, etc.) into a cell, then insert
“R:” before it and hit Return (or click on empty
cell). Now the function is ready for use.
• Can use “Run R script” on selected rows of
commands in an Excel column to run multiple R
commands (each prefixed with “R:” or with “G:”
for graphics to be embedded in Excel)
– Keep and reuse scripts
40
Example: An expected utility
function, EU(x, p)
• The following function computes EU(X) for
any discrete random variable (r.v.) X that
has values x = (x1, x2, …, xn) with
respective probabilities p = (p1, p2, …, pn),
if utility function u(x) has been defined:
EU <- function(x,p){
value <- sum(p*u(x))
return(value)}
Example usage in CAT:
R: EU <- function(x,p){
value <- sum(p*u(x))
return(value)}
R: EU(x, p)
[1] 3.684136
41
Course overview
42
Goals
• Learn to make better decisions
– Learn to evaluate decision performance
– What defines a “better” decision?
• Help others to make better decisions
• Take advantage of current software
• Apply what you learn to important realworld problems
43
Course outline
Unit
1
2
3
4
5
Topics
Readings
Introduction
 How do we make decisions?
 How should we?
 How can we make them better?
Making decisions with known
values and beliefs.
Making decisions with uncertain
probabilities and preferences
Tversky & Kahneman,1981
Pillay, 2014; Schoemaker,
1982, pp 529-32; Hulett, pp 1-3

Kirkwood, 2002, Chapter 1
Kranton, 2005 a, b, pp 30-32
Schoemaker, 1982, pp 533-538
Slovic et al. 2002, Thaler, 1981
Camerer, 2012, Russo &
Schoemaker 1989, Chapter 5



Kirkegaard, 2016; Niu, 2005
CAT User’s Guide; Gigerenzer
1996; Schoemaker, 1982, 54244;550-552 Tversky and
Kahneman, 1974
Charniak, 1991

Calculating and estimating
probabilities
Causality, Bayesian Networks, and
Influence Diagrams
Techniques/Skills




MIDTERM EXAM (March 14, no class)
Formulate and solve decision
problems
Identify and avoid key
heuristics and biases
Expected utility (EU)
Decision tables and trees
SEU calculations; Sensitivity
analysis; Eliciting and
calibrating probabilities
Bayes’ Rule using Netica
Regression and probability in
R; Overcoming common
heuristics & biases
Monte Carlo simulation and
simulation-optimization
Formulating and solving
Bayesian Networks and
influence diagrams with
Netica
44
Course outline
Unit
Topics
Readings
Techniques/Skills
6
Learning causal models from data
Elwert, 2013

Milkman, Chugh, Bazerman, 
2008
7
Obtaining utilities: Single- and
multi-attribute utility theory
Kranton 2005b
Kirkwood, 2002, Chapter 2
Bell 1988
Abbas, 2010, pp 62-67 and
74-77
Powell and Frazier, 2008, pp
213, 216-219, 223-4, 230-40
Kirkwood, 2002, Chapter 3

Coolen-Schrijner 2004
Klein, Klinger 1987
Lee, 2013
Bradberry, 2015
Russo and Schoemaker,
1990

8
9
10
Applied optimal statistical
decisions and optimal learning
Learning, evaluating, and evolving
effective decision rules
Improving decisions in groups,
teams, and organizations







Testing causal hypotheses
Causal analytics in CAT
Assessing and comparing utility
functions; FSD, SSD, TSD
Measures of risk and risk
aversion; single- and multiattribute utility
Solving statistical decision and
classification problems
Simulation-optimization
VoI calculations
Formulating and solving Markov
Decision Processes (MDPs)
EVOP and DOE
Simulation-optimization
Designing effective group
decision rules
FINAL EXAM (Week of May 9), RESEARCH PAPERS/ PROJECTS (due May16)
45
Key themes for course
• Decision analysis
– Tools for rational decisions
• Challenges to rational decision-making
– People are not (and should not be) purely
rational
• Toward decision analysis for real people
– Individual decisions (main emphasis)
– Group decisions, organizational decisions,
social and collective decisions
46
Decision analysis
• How to move beyond “Here’s what I think
(or feel) I should do?”
www.fierceinc.com/blog/fierce-conversations/emotional-first-rationally-second
47
Decision analysis: Theory
Inputs:
•
•
•
•
•
Alternative possible acts/choices/decisions
Possible consequences
Preferences/values for consequences
Risk attitudes
Beliefs, assumptions, or causal models about how choices
affect probabilities of consequences
– Information/observation/evidence
Outputs:
• What to do next
• When to change current policy (decision rules)
48
How decision analysis works
• Ingredients:
– Possible acts, A
– Possible consequences, C
– Causal model, Pr(c | a) = ∑sPr(c | a, s)Pr(s)
– Preferences for consequences, v: C  R
• Preferences represented by value function, v(c)
– Preferences for acts, u: A  R
• Preferences represented by utility function, u(c)
• Normative theory: Represent coherent
preferences for acts by expected utility (EU)
49
Defining “coherent” preferences
• Ordering: Provide a weak order (complete,
transitive, reflexive) for consequences and
probability distributions over consequences
• Substitution: Only final probabilities for
consequences matter.
– No joy or dread of gambling, no loss aversion
– No reference points, aspiration levels, etc.
• Continuity: Unique probability mix of worst
and best, u(c), indifferent to consequence c
50
A different route to coherence:
Dynamic consistency
• Dynamic consistency: Preferring to do
what you originally planned when the
opportunity arises
• Dynamic consistency, with other
conditions (such as Bayesian updating of
beliefs), implies EU
51
Golden rule of rationality:
Maximize expected utility (EU)
• Choose an act a from A to maximize the
expected utility of consequences
• Notation and calculations:
– EU(a) = ∑cPr(c | a)u(c) = ∑s∑cu(c)Pr(c | a, s)Pr(s)
– Monte Carlo evaluation of EU(a)
– Simulation-optimization of a  A
– Bayesian networks/influence diagrams
• Represent Pr(c | a, s), u(c), and Pr(s); solve for a
52
www.prioritysystem.com/reasons4c.html
Decision models to maximize EU
•
•
•
•
Trees
Influence diagrams
Tables c(a, s) or Pr(c | a, s)
Response functions
– Design of experiments, optimization via adaptive
learning, simulation-optimization
– Optimization models maxaAEU(a)
• Dynamic models
– Markov decision processes
– Optimal control and learning
• Stochastic, robust, adaptive; low-regret
53
Skill 2: Decision tree analysis
(“Extensive form” analysis)
54
Now that we can quantify
EU(X), what else can we do?
• EU(x, p) = sum(p*u(x))
• Knowing that EU(X) = 3.684136
tells us nothing useful!
• However, combined with the
Golden Rule of Rationality, choose
to maximize expected utility, it lets
us solve some decision problems
• Decision trees provide a method.
55
Decision tree model of a
decision problem
1. Rectangles represent choices or
decisions
2. Ellipses represent chance events
(random variables)
3. Hexagons (or sometimes triangles)
represent value or utility nodes (tips or
leaves of the tree)
4. Tree shows possible time sequences
of choices and uncertainty resolutions
5. Each choice (or decision) is made
based on (i.e., conditioned on) the
information that leads up to it.
6. Decision trees are solved backward
via “averaging out” (taking expected
values) at chance nodes and “folding
back” (optimizing) at choice nodes.
http://cse-wiki.unl.edu/wiki/index.php/More_depth_on_influence_diagrams:_Decision_trees,_influence_diagrams,_dynamic_networks
56
Decision tree solution algorithm
• Decision tree solution algorithm works from right
(value nodes at leaves or tips of tree) to left
– Averaging out: At each chance node, calculate
expected value of branches leaving that node using
EMV(X) = ∑xp(x)*x
– Folding back: At each choice/decision node, including
the first, choose the choice (branch) with highest EMV
• Non-EMV decision-makers: Choose branch with highest
expected utility (EU) instead of highest EMV
• “Averaging out and folding back” (Raiffa, 1968)
– Stochastic dynamic programming
57
Example: Solving a simple
decision tree
•
•
•
•
•
Calculate EMV of risky investment:
R: 0.10*500 + 0.65*100 - 0.25*600
Or, R: EMV_RI = sum(c(0.10, 0.65, 0.25)*c(500,100,-600))
Either will return the same answer: [1] -35
Since 50 > -35, optimal decision is to choose the CD.
http://cse-wiki.unl.edu/wiki/index.php/More_depth_on_influence_diagrams:_Decision_trees,_influence_diagrams,_dynamic_networks
58
Skill 3: Sensitivity analysis
calculations
59
Example: Sensitivity analysis
• Q: How great would the “Large increase” payoff have
to be to make choosing “Risky investment” optimal?
http://cse-wiki.unl.edu/wiki/index.php/More_depth_on_influence_diagrams:_Decision_trees,_influence_diagrams,_dynamic_networks
60
Example: Sensitivity analysis
• Q: How great would the “Large increase” payoff have
to be to make choosing “Risky investment” optimal?
• By computation:
R: x <- c(0:2000)
R: y <- 0.10*x + 0.65*100 - 0.25*600
G: plot(x, y)
R: x[which(y > 50)[1]]
• A: The payoff would have to exceed $1350.
http://cse-wiki.unl.edu/wiki/index.php/More_depth_on_influence_diagrams:_Decision_trees,_influence_diagrams,_dynamic_networks
61
Example: Sensitivity analysis
• Q: How great would the “Large increase” payoff have
to be to make choosing “Risky investment” optimal?
• By algebra:
0.10*x + 0.65*100 - 0.25*600 > 50
x > 10*(50 + 0.25*600 - 0.65*100)
x > 1350
• A: The payoff would have to exceed $1350.
http://cse-wiki.unl.edu/wiki/index.php/More_depth_on_influence_diagrams:_Decision_trees,_influence_diagrams,_dynamic_networks
62
Example: Sensitivity analysis
• Q: How great would the “Large increase” payoff have
to be to make choosing “Risky investment” optimal?
• By solver:
R: f <- function (x) 0.10*x + 0.65*100 - 0.25*600 - 50
R: str(xmin <- uniroot(f, c(0, 2000), tol = 0.0001))
List of 5 $ root: num 1350 $ f.root: num 0 $ iter: int 1 $ init.it: int NA $ estim.prec: num 1350
A: The payoff would have to exceed $1350.
http://cse-wiki.unl.edu/wiki/index.php/More_depth_on_influence_diagrams:_Decision_trees,_influence_diagrams,_dynamic_networks
63
Why does the decision tree
algorithm work?
• Maximize EU is implied by normative coherence/
rationality axioms (von Neumann-Morgenstern)
– Completeness, transitivity, continuity, independence
• Averaging out and folding back algorithm
maximizes EU
– Stochastic dynamic programming (SDP)
– Solution satisfies Bellman’s Principle of Optimality:
An optimal path (sequence of decisions) continues
optimally starting from each point (decision) on it.
https://en.wikipedia.org/wiki/Bellman_equation#Bellman.27s_Principle_of_Optimality
64