Download Introduction to Decision Theory

Agent Technology for e-Commerce
Appendix A: Introduction to Decision
Theory
Maria Fasli
http://cswww.essex.ac.uk/staff/mfasli/ATe-Commerce.htm
Agent Technology for e-Commerce
Decision theory
Decision theory is the study of making decisions that have a
significant impact
Decision-making is distinguished into:
 Decision-making under certainty
 Decision-making under noncertainty
 Decision-making under risk
 Decision-making under uncertainty
Appendix A
2
Agent Technology for e-Commerce
Probability theory



Most decisions have to be taken in the presence of uncertainty
Probability theory quantifies uncertainty regarding the occurrence
of events or states of the world
Basic elements of probability theory:
 Random variables describe aspects of the world whose state is
initially unknown
 Each random variable has a domain of values that it can take
on (discrete, boolean, continuous)
 An atomic event is a complete specification of the state of the
world, i.e. an assignment of values to variables of which the
world is composed
Appendix A
3
Agent Technology for e-Commerce
Probability space
 The sample space S={e1,e2,…,en} which is a set of atomic events
 The probability measure P which assigns a real number between
0 and 1 to the members of the sample space
Axioms
 All probabilities are between 0 and 1
 The sum of probabilities for the atomic events of a probability
space must sum up to 1
 The certain event S (the sample space itself) has probability 1,
and the impossible event which never occurs, probability 0
Appendix A
4
Agent Technology for e-Commerce
Prior probability




In the absence of any other information, a random variable is
assigned a degree of belief called unconditional or prior
probability
P(X) denotes the vector consisting of the probabilities of all
possible values that a random variable can take
If more than one variable is considered, then we have joint
probability distributions
Lottery: a probability distribution over a set of outcomes
L=[p1,o1;p2,o2;…;pn,on]
Appendix A
5
Agent Technology for e-Commerce
Conditional probability

When we have information concerning previously unknown
random variables then we use posterior or conditional
probabilities: P(a|b) the probability of a given that we know b

Alternatively this can be written (the product rule):
P(ab)=P(a|b)P(b)

Independence
P(a|b)=P(a) and P(b|a)=P(b) or P(ab)=P(a)P(b)
Appendix A
6
Agent Technology for e-Commerce
Bayes’ rule
The product rule can be written as:
P(ab)=P(a|b)P(b)
P(ab)=P(b|a)P(a)
By equating the right-hand sides:
This is known as Bayes’ rule
Appendix A
7
Agent Technology for e-Commerce
Making decisions
Simple example: to take or not my umbrella on my way out
The consequences of decisions can be expressed in terms of payoffs
Payoff table
Loss table
Appendix A
8
Agent Technology for e-Commerce
An alternative representation of payoffs – tree diagram
Appendix A
9
Agent Technology for e-Commerce
Admissibility


An action is said to dominate another, if for each possible state of
the world the first action leads to at least as high a payoff (or at
least as small a loss) as the second one, and there is at least one
state of the world in which the first action leads to a higher payoff
(or smaller loss) than the second one
If one action dominates another, then the latter should never be
selected and it is called inadmissible
Payoff table
Appendix A
10
Agent Technology for e-Commerce
Non-probabilistic decision-making under uncertainty



The maximin rule
The maximax rule
The minimax loss
Payoff table
Loss table
Appendix A
11
Agent Technology for e-Commerce
Probabilistic decision-making under uncertainty


The Expected Payoff (ER) rule dictates that the action with the
highest expected payoff should be chosen
The Expected Loss (EL) rule dictates that the action with the
smallest expected loss should be chosen
If P(rain)=0.7 and P(not rain)=0.3 then:
ER(carry umbrella) = 0.7(-£1)+0.3(-£1)=-£1
ER(not carry umbrella) = 0.7(-£50)+0.3(-£0)=-£35
EL(carry umbrella) = 0.7(£0)+0.3(£1)=£0.3
EL(not carry umbrella) = 0.7(£49)+0.3(£0)=£34.3
Appendix A
12
Agent Technology for e-Commerce
Utilities




Usually the consequences of decisions are expressed in monetary
terms
Additional factors such are reputation, time, etc. are also usually
translated into money
Issue with the use of money to describe the consequences of
actions:
 If a fair coin comes up heads you win £1, otherwise you loose
£0.75, would you take this bet?
 If a fair coin comes up heads you win £1000, otherwise you
loose £750, would you take this bet?
The value of a currency, differs from person to person
Appendix A
13
Agent Technology for e-Commerce
Preferences









The concept of preference is used to indicate that we would
like/desire/prefer one thing over another
o o’ indicates that o is (strictly) preferred to o’
o ~ o’ indicates that an agent is indifferent between o and o’
o o’ indicates that o is (weakly) preferred to o’
Given any o and o’, then o o’, or o’ o, or o ~ o’
Given any o, o’ and o’’, then if o o’ and o’ o’’, then o’ o’’
If o o’ o’’, then there is a p such that [p,o;1-p,o’’] ~ o’
If o ~ o’, then [p,o; 1-p,o’’] ~ [p,o’; 1-p, o’’]
If o o’, then (pq [p,o;1-p,o’] [q,o;1-q,o’] )
Appendix A
14
Agent Technology for e-Commerce
Utility functions






A utility function provides a convenient way of conveying
information about preferences
If o o’, then u(o)>u(o’) and if o ~ o’ then u(o)=u(o’)
If an agent is indifferent between:
(a) outcome o for certain and
(b) taking a bet or lottery in which it receives o’ with probability
p and o’’ with probability 1-p
then u(o)=(p)u(o’)+(1-p)u(o’’)
Ordinal utilities
Cardinal utilities
Monotonic transformation
Appendix A
15
Agent Technology for e-Commerce
Assessing a utility function
How can an agent assess a utility function?

Suppose most and least preferable payoffs are R+ and R- and
u(R+)=1 and u(R-)=0

For any other payoff R, it should be:
u(R+)  u(R)  u(R-) or 1 u(R)  0
Appendix A
16
Agent Technology for e-Commerce

To determine the value of u(R) consider:
 L : Receive R for certain
1
+
 L : Receive R with probability p and R with probability 1-p
2
-

Expected utilities:
 EU(L )=u(R)
1
+
 EU(L )=(p)u(R )+(1-p)u(R )=(p)(1)+(1-p)(0)=p
2
-

If u(R)>p, L1 should be selected, whereas if u(R)<p, L2 should be
selected, and if u(R)=p then the agent is indifferent between the
two lotteries
Appendix A
17
Agent Technology for e-Commerce
Utility and money


The value, i.e. utility, of money may differ from person to person
Consider the lottery
 L : receive £0 for certain
1
 L : receive £100 with probability p and -£100 with (1-p)
2

Suppose an agent decides that for p=0.75 is indifferent between
the two lotteries, i.e. p>0.75 prefers lottery L2

The agent also assess u(-£50)=0.4 and u(£50)=0.9
Appendix A
18
Agent Technology for e-Commerce
Appendix A
19
Agent Technology for e-Commerce

If p is fixed, the amount of money that an agent would need to
receive for certain in L1 to make it indifferent between two
lotteries can be determined. Consider:
 L : receive £x for certain
1
 L : receive £100 with probability 0.5 and -£100 with 0.5
2

Suppose x=-£30, then u(-£30)=0.5 and -£30 is considered to be
the cash equivalent of the gamble involved in L2

The amount of £30 is called the risk premium – the basis of
insurance industry
Appendix A
20
Agent Technology for e-Commerce
Utility function of risk-averse agent
Appendix A
21
Agent Technology for e-Commerce
Utility function of a risk-prone agent
Appendix A
22
Agent Technology for e-Commerce
Utility function of a risk-neutral agent
Appendix A
23
Agent Technology for e-Commerce
Multi-attribute utility functions
The utility of an action may depend on a number of factors
 Multi-dimensional or multi-attribute utility theory deals with
expressing such utilities
 Example: you are made a set of job offers, how do you decide?
u(job-offer) = u(salary) + u(location) +
u(pension package) + u(career opportunities)
u(job-offer) = 0.4u(salary) + 0.1u(location) +
0.3u(pension package) + 0.2u(career opportunities)
But if there are interdependencies between attributes, then additive
utility functions do not suffice. Multi-linear expressions:
u(x,y)=wxu(x)+wyu(y)+(1-wx-wy)u(x)u(y)

Appendix A
24