Download randomness

Finite probability space
set 
(sample space)
function P:  R+ (probability distribution)
 P(x) = 1
x
Finite probability space
set 
(sample space)
function P:  R+ (probability distribution)
 P(x) = 1
x
elements of  are called atomic events
subsets of  are called events
probability of an event A is
P(A)=
 P(x)
xA
Examples
1. Roll a (6 sided) dice. What is the probability
that the number on the dice is even?
2. Flip two coins, what is the probability that
they show the same symbol?
3. Flip five coins, what is the probability that
they show the same symbol?
4. Mix a pack of 52 cards. What is the
probability that all red cards come before
all black cards?
Union bound
P(A  B)  P(A) + P(B)
P(A1 A2 …  An)  P(A1) + P(A2)+…+P(An)
Union bound
P(A1 A2 …  An)  P(A1) + P(A2)+…+P(An)
Suppose that the probability of winning in
a lottery is 10-6. What is the probability that
somebody out of 100 people wins?
Ai = i-th person wins
somebody wins = ?
Union bound
P(A1 A2 …  An)  P(A1) + P(A2)+…+P(An)
Suppose that the probability of winning in
a lottery is 10-6. What is the probability that
somebody out of 100 people wins?
Ai = i-th person wins
somebody wins = A1A2…A100
Union bound
P(A1 A2 …  An)  P(A1) + P(A2)+…+P(An)
Suppose that the probability of winning in
a lottery is 10-6. What is the probability that
somebody out of 100 people wins?
P(A1A2…A100)  100*10-6 = 10-4
Union bound
P(A1 A2 …  An)  P(A1) + P(A2)+…+P(An)
Suppose that the probability of winning in
a lottery is 10-6. What is the probability that
somebody out of 100 people wins?
P(A1A2…A100)  100*10-6 = 10-4
P(A1A2…A100) =
1–P(AC1 AC2… AC100) =
1-P(AC1)P(AC2)…P(AC100)=
1-(1-10-6)100 0.99*10-4
Independence
Events A,B are independent if
P(A  B) = P(A) * P(B)
Independence
Events A,B are independent if
P(A  B) = P(A) * P(B)
“observing whether B happened gives no
information on A”
B
A
Independence
Events A,B are independent if
P(A  B) = P(A) * P(B)
“observing whether B happened gives no
information on A”
B
A
P(A|B) = P(AB)/P(B)
conditional probability
of A, given B
Independence
Events A,B are independent if
P(A  B) = P(A) * P(B)
P(A|B) = P(A)
Examples
Roll two (6 sided) dice. Let S be their sum.
1) What is that probability that S=7 ?
2) What is the probability that S=7,
conditioned on S being odd ?
3) Let A be the event that S is even and
B the event that S is odd. Are A,B
independent?
4) Let C be the event that S is divisible by 4.
Are A,C independent?
5) Let D be the event that S is divisible by 3.
Are A,D independent?
Examples
A
C
B
Are A,B independent ?
Are A,C independent ?
Are B,C independent ?
Is it true that P(ABC)=P(A)P(B)P(C)?
Examples
C
Events A,B,C are
pairwise independent
A but not
(fully) independent
B
Are A,B independent ?
Are A,C independent ?
Are B,C independent ?
Is it true that P(ABC)=P(A)P(B)P(C)?
Full independence
Events A1,…,An are (fully) independent
If for every subset S[n]:={1,2,…,n}
P(
 A ) =  P(A )
iS
i
iS
i
Testing equality of strings
n-bits
Alice: A = 0001110100010101000111
slow network
Bob : B = 0001110100010101000111
n-bits
QUESTION: Is A=B?
Testing equality of strings
n-bits
n-bits
Alice: A = 0001110100010101000111
Bob : B = 0001110100010101000111
slow network
QUESTION: Is A=B?
Protocol:
1. Alice picks a random prime p  n2.
2. Alice computes a:=(A mod p), and sends
p and a to Bob.
3. Bob computes b:=(B mod p), and checks
whether a=b.
Testing equality of strings
How many bits are communicated?
Protocol:
1. Alice picks a random prime p  n2.
2. Alice computes a:=(A mod p), and sends
p and a to Bob.
3. Bob computes b:=(B mod p), and checks
whether a=b.
Testing equality of strings
What is the probabilty of failure?
Protocol:
1. Alice picks a random prime p  n2.
2. Alice computes a:=(A mod p), and sends
p and a to Bob.
3. Bob computes b:=(B mod p), and checks
whether a=b.
Testing equality of strings
What is the probabilty of failure?
BAD EVENT = p divides A-B
Protocol:
1. Alice picks a random prime p  n2.
2. Alice computes a:=(A mod p), and sends
p and a to Bob.
3. Bob computes b:=(B mod p), and checks
whether a=b.
Testing equality of strings
What is the probabilty of failure?
BAD EVENT = p divides A-B
How many (different) primes can divide an
n-bit number?
How many primes  n2 are there?
Testing equality of strings
What is the probabilty of failure?
BAD EVENT = p divides A-B
How many (different) primes can divide an
n-bit number?
n
k
2  M=p1p2…pk  2
kn
How many primes  n2 are there?
Prime Number Theorem
 (m)  m/ln m
number of primes  m
Testing equality of strings
If A=B then
the algorithm always answers YES
If AB then
the algorithms answers NO with
probability  1- (ln n)/n
Monte Carlo algorithm with 1-sided error
Random variable
set 
(sample space)
function P:  R+ (probability distribution)
 P(x) = 1
x
A random variable is a function
Y:R
The expected value of Y is

E[X] :=
P(x)* Y(x)
x
Examples
Roll two dice. Let S be their sum.
If S=7 then player A gives player B $6
otherwise player B gives player A $1
2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12
Examples
Roll two dice. Let S be their sum.
If S=7 then player A gives player B $6
otherwise player B gives player A $1
2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12
Y: -1 , -1,-1 ,-1, -1, 6 ,-1 ,-1 , -1 , -1 , -1
Expected income for B
E[Y] = 6*(1/6)-1*(5/6)= 1/6
Linearity of expectation
E[X + Y] = E[X] + E[Y]
E[X1+ X2+ … + Xn] = E[X1] + E[X2]+…+E[Xn]
Linearity of expectation
Everybody pays me $1 and writes their
name on a card. I mix the cards and
give everybody one card. If you get back
the card with your name – I pay you $10.
Let n be the number of people in the class.
For what n is the game advantageous for me?
Linearity of expectation
Everybody pays me $1 and writes their
name on a card. I mix the cards and
give everybody one card. If you get back
the card with your name – I pay you $10.
X1 = -9 if player 1 gets his card back
1 otherwise
E[X1] = ?
Linearity of expectation
Everybody pays me $1 and writes their
name on a card. I mix the cards and
give everybody one card. If you get back
the card with your name – I pay you $10.
X1 = -9 if player 1 gets his card back
1 otherwise
E[X1] = -9/n + 1*(n-1)/n
Linearity of expectation
Everybody pays me $1 and writes their
name on a card. I mix the cards and
give everybody one card. If you get back
the card with your name – I pay you $10.
X1 = -9
1
X2 = -9
1
if player 1 gets his card back
otherwise
if player 2 gets his card back
otherwise
E[X1+…+Xn] = E[X1]+…+E[Xn] =
n ( -9/n + 1*(n-1)/n ) = n – 10.
Expected number of coin-tosses
until HEADS?
Expected number of coin-tosses
until HEADS?
1/2
1/4
1/8
1/16
….

1
2
3
4
 n.2-n = 2
n=1
Expected number of coin-tosses
until HEADS?
S
S= 1 + ½*S
S=2
Expected number of dice-throws
until you get “6”
S
Expected number of dice-throws
until you get “6”
S
S= 1 + (5/6)*S
S=6
Coupon collector problem
n coupons to collect
What is the expected number of
cereal boxes that you need to buy?
Expected number of coin-tosses
until 3 consecutive HEADS?
Markov’s inequality
A group of 10 people have average income $20,000.
At most how many people in the group can have
average income at least $40,000?
A group of 10 people have average income $20000.
At most how many people in the group can have
average income at least $100,000?
Markov’s inequality
A group of 10 people have average income $20,000.
At most how many people in the group can have
average income at least $40,000?
Let X be a random variable such that X  0.
Then
P(X  a*E[X])  1/a
Example
Alice has an algorithm A which runs in
expected running time T(n).
Bob uses Alice’s algorithm to construct
his own algorithm B.
1. Run algorithm A for 2T(n) steps.
2. If A terminates then B outputs the same,
otherwise goto step 1.
What is the expected running time of B?
What is the probability that A terminates after 100T(n) steps?
What is the probability that B terminates after 100T(n) steps?