Download Slide

Administrative
Sep. 25 (today) – HW3 (=QUIZ #1) due
Sep. 27 – HW4 due
Sep. 28 8am – problem session
Oct. 2
Oct. 4 – QUIZ #2
(pages 45-79 of DPV)
Administrative
Homework rules:
You may work with one other person on homeworks, but you must each write up your
(without any written aid)
solutions separately
. If you work with
another person, indicate who you worked with on your solution (thus you should each
indicate each other).
University Academic Honesty Policy:
All cases of suspected dishonesty must be reported to the Board,
either through a shortform resolution or by forwarding a case to the
Board for a hearing. Faculty may not come to an understanding with a
student on their own in a case of suspected dishonesty, but must use
the short form resolution or submit a case.
Recurrences
T(n) = a T(n/b) + f(n)
c=logb a
• If f(n) = O(nc-) then T(n) =(nc)
• If f(n) = (nc) then T(n) =(nc.log n)
• If f(n) = (nc+) then T(n)=(f(n))
if a.f(n/b)  d.f(n) for some d<1 and n>n0
T(n) = 3 T(n/2) + (n)
T(n) = (nlog 3)
2
T(n) = 2T(n/2) + (n)
T(n) = (n.log n)
Finding the k-th smallest element
k = n/2
= MEDIAN
Split(A[1..n],x)
x
runs in time O(n)
>x
Finding the k-th smallest element
k = n/2
= MEDIAN
Split(A[1..n],x)
>x
x
j
j k  k-th smallest on left
j<k  (k-j)-th smallest on right
Finding the k-th smallest element
6 3 1 7 8 2 6 1 5 8 8 9 1 2 3
6
3
1
7
8
2
6
1
5
8
8
9
1
2
3
Finding the k-th smallest element
6 3 1 7 8 2 6 1 5 8 8 9 1 2 3
6
3
1
7
8
2
6
1
5
8
8
9
1
2
3
1) sort each 5-tuple
Finding the k-th smallest element
6
3
1
7
8
2
6
1
5
8
8
9
1
2
3
1) sort each 5-tuple
Finding the k-th smallest element
1
3
6
7
8
2
6
1
5
8
8
9
1
2
3
1) sort each 5-tuple
Finding the k-th smallest element
1
3
6
7
8
2
6
1
5
8
8
9
1
2
3
1) sort each 5-tuple
Finding the k-th smallest element
1
3
6
7
8
1
2
5
6
8
8
9
1
2
3
1) sort each 5-tuple
Finding the k-th smallest element
TIME = ?
1
3
6
7
8
1
2
5
6
8
1
2
3
7
9
1) sort each 5-tuple
Finding the k-th smallest element
TIME = (n)
1
3
6
7
8
1
2
5
6
8
1
2
3
7
9
1) sort each 5-tuple
Finding the k-th smallest element
2) find median of the middle n/5 elements
TIME = ?
1
3
6
7
8
1
2
5
6
8
1
2
3
7
9
Finding the k-th smallest element
2) find median of the middle n/5 elements
TIME = T(n/5)
1
3
6
7
8
1
2
5
6
8
1
2
3
7
9
We will use this element as the pivot
Finding the k-th smallest element
At least ? Many elements in
the array are  X
1
3
6
7
8
1
2
5
6
8
1
2
3
7
9
Finding the k-th smallest element
At least ? Many elements in
the array are  X
1
2
3
7
9
1
2
5
6
8
1
3
6
7
8
Finding the k-th smallest element
At least 3n/10 elements in
the array are  X
1
2
3
7
9
1
2
5
6
8
1
3
6
7
8
Finding the k-th smallest element
6 3 1 7 8 2 6 1 5 8 8 9 1 2 3
1
2
3
7
9
1
2
5
6
8
1
3
6
7
8
At least 3n/10 elements in the array are  X
Finding the k-th smallest element
6 3 1 7 8 2 6 1 5 8 8 9 1 2 3
X
>X
3 1 2 3 2 1 1 5 8 8 9 6 7 8 6
At least 3n/10 elements in the array are  X
Finding the k-th smallest element
6 3 1 7 8 2 6 1 5 8 8 9 1 2 3
X
>X
3 1 2 3 2 1 1 5 8 8 9 6 7 8 6
Recurse, time ?
At least 3n/10 elements in the array are  X
Finding the k-th smallest element
6 3 1 7 8 2 6 1 5 8 8 9 1 2 3
X
>X
3 1 2 3 2 1 1 5 8 8 9 6 7 8 6
Recurse, time  T(7n/10)
At least 3n/10 elements in the array are  X
Finding the k-th smallest element
631782615889123
6
3
1
7
8
2
6
1
5
8
8
9
1
2
3
1
3
6
7
8
1
2
5
6
8
1
2
3
7
9
1
3
6
7
8
1
2
5
6
8
1
2
3
7
9
6 3 1 7 8 2 6 1 5 8 8 9 1 2 3
Split >X
X
3 1 2 3 2 1 1 5 8 8 9 6 7 8 6
recurse
Finding the k-th smallest element
631782615889123
6
3
1
7
8
2
6
1
5
8
8
9
1
2
3
1
3
6
7
8
1
2
5
6
8
1
2
3
7
9
1
3
6
7
8
1
2
5
6
8
1
2
3
7
9
6 3 1 7 8 2 6 1 5 8 8 9 1 2 3
(n)
T(n/5)
(n)
X
>X
3 1 2 3 2 1 1 5 8 8 9 6 7 8 6
recurse
T(7n/10)
Finding the k-th smallest element
T(n)  T(n/5) + T(7n/10) + O(n)
Finding the k-th smallest element
T(n)  T(n/5) + T(7n/10) + O(n)
T(n)  d.n
Induction step:
T(n)  T(n/5) + T(7n/10) + O(n) 
d.(n/5) + d.(7n/10) + O(n) 
d.n + (O(n) – dn/10)  d.n
Why 5-tuples?
317615912
6 3 1 7 8 2 6 1 5 8 8 9 1 2 3
3 6 9
1 1 1
7 5 2
(n)
1 1 1
3 5 2
7 6 9
1 1 1
3 5 2
7 6 9
(n)
X
>X
3 1 2 3 2 1 1 5 8 8 9 6 7 8 6
recurse
Why 5-tuples?
317615912
6 3 1 7 8 2 6 1 5 8 8 9 1 2 3
3 6 9
1 1 1
7 5 2
(n)
1 1 1
3 5 2
7 6 9
T(n/3)
1 1 1
3 5 2
7 6 9
(n)
X
>X
3 1 2 3 2 1 1 5 8 8 9 6 7 8 6
recurse
T(2n/3)
Why 5-tuples?
T(n) = T(n/3) + T(2n/3) + (n)
Why 5-tuples?
T(n) = T(n/3) + T(2n/3) + (n)
T(n)  c.n.ln n
Induction step:
T(n) = T(n/3) + T(2n/3) + (n) 
c.(n/3).ln (n/3) + c.(2n/3).ln (2n/3) + (n) 
c.n.ln n - c.n.((1/3)ln 3+(2/3)ln 3/2)+(n)
 c.n.ln n
Quicksort(A[b..c])
Split(A[b..c],x)
=x
<x
b
i
>x
j
Quicksort(A[b..i]);
Quicksort(A[j..c]);
c
Quicksort(A[b..c])
Worst-case running time?
How to make
the worst-case running time
O(n.log n) ?
Quicksort(A[b..c])
if pivot = median
then the worst-case running
time satisfies
T(n) = 2T(n/2) + O(n)
Quicksort(A[b..c])
Split(A[b..c],x)
=x
<x
b
i
>x
j
Quicksort(A[b..i]);
Quicksort(A[j..c]);
x = random element of A[b..c]
c
Finding the k-th smallest element
k = n/2
= MEDIAN
Split(A[1..n],x)
>x
x
j
j k  k-th smallest on left
j<k  (k-j)-th smallest on right
Finding the k-th smallest element
Select(k,A[c..d])
x=random element from A[c..d]
Split(A[c..d],x)
>x
x
j
j k  k-th smallest on left
j<k  (k-j)-th smallest on right
Finite probability space
set 
(sample space)
function P:  R+ (probability distribution)
 P(x) = 1
x
Finite probability space
set 
(sample space)
function P:  R+ (probability distribution)
 P(x) = 1
x
elements of  are called atomic events
subsets of  are called events
probability of an event A is
P(A)=
 P(x)
xA
Examples
1. Roll a (6 sided) dice. What is the probability
that the number on the dice is even?
2. Flip two coins, what is the probability that
they show the same symbol?
3. Flip five coins, what is the probability that
they show the same symbol?
4. Mix a pack of 52 cards. What is the
probability that all red cards come before
all black cards?
Union bound
LEMMA:
P(A  B)  P(A) + P(B)
More generally:
P(A1 A2 …  An)  P(A1) + P(A2)+…+P(An)
Union bound
P(A1 A2 …  An)  P(A1) + P(A2)+…+P(An)
Suppose that the probability of winning in
a lottery is 10-6. What is the probability that
somebody out of 100 people wins?
Ai = i-th person wins
somebody wins = ?
Union bound
P(A1 A2 …  An)  P(A1) + P(A2)+…+P(An)
Suppose that the probability of winning in
a lottery is 10-6. What is the probability that
somebody out of 100 people wins?
Ai = i-th person wins
somebody wins = A1A2…A100
Union bound
P(A1 A2 …  An)  P(A1) + P(A2)+…+P(An)
Suppose that the probability of winning in
a lottery is 10-6. What is the probability that
somebody out of 100 people wins?
P(A1A2…A100)  100*10-6 = 10-4
Union bound
P(A1 A2 …  An)  P(A1) + P(A2)+…+P(An)
Suppose that the probability of winning in
a lottery is 10-6. What is the probability that
somebody out of 100 people wins?
P(A1A2…A100)  100*10-6 = 10-4
P(A1A2…A100) =
1–P(AC1 AC2… AC100) =
1-P(AC1)P(AC2)…P(AC100) =
1-(1-10-6)100 0.99*10-4
Independence
Events A,B are independent if
P(A  B) = P(A) * P(B)
Independence
Events A,B are independent if
P(A  B) = P(A) * P(B)
“observing whether B happened gives no
information on A”
B
A
Independence
Events A,B are independent if
P(A  B) = P(A) * P(B)
“observing whether B happened gives no
information on A”
B
A
P(A|B) = P(AB)/P(B)
conditional probability
of A, given B
Independence
Events A,B are independent if
P(A  B) = P(A) * P(B)
P(A|B) = P(A)
Examples
Roll two (6 sided) dice. Let S be their sum.
1) What is that probability that S=7 ?
2) What is the probability that S=7,
conditioned on S being odd ?
3) Let A be the event that S is even and
B the event that S is odd. Are A,B
independent?
4) Let C be the event that S is divisible by 4.
Are A,C independent?
5) Let D be the event that S is divisible by 3.
Are A,D independent?
Examples
A
C
B
Are A,B independent ?
Are A,C independent ?
Are B,C independent ?
Is it true that P(ABC)=P(A)P(B)P(C)?
Examples
C
Events A,B,C are
pairwise independent
A but not
(fully) independent
B
Are A,B independent ?
Are A,C independent ?
Are B,C independent ?
Is it true that P(ABC)=P(A)P(B)P(C)?
Full independence
Events A1,…,An are (fully) independent
If for every subset S[n]:={1,2,…,n}
P(
 A ) =  P(A )
iS
i
iS
i
Random variable
set 
(sample space)
function P:  R+ (probability distribution)
 P(x) = 1
x
A random variable is a function
Y:R
The expected value of Y is

E[X] :=
P(x)* Y(x)
x
Examples
Roll two dice. Let S be their sum.
If S=7 then player A gives player B $6
otherwise player B gives player A $1
2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12
Examples
Roll two dice. Let S be their sum.
If S=7 then player A gives player B $6
otherwise player B gives player A $1
2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12
Y: -1 , -1,-1 ,-1, -1, 6 ,-1 ,-1 , -1 , -1 , -1
Expected income for B
E[Y] = 6*(1/6)-1*(5/6)= 1/6
Linearity of expectation
LEMMA:
E[X + Y] = E[X] + E[Y]
More generally:
E[X1+ X2+ … + Xn] = E[X1] + E[X2]+…+E[Xn]
Linearity of expectation
Everybody pays me $1 and writes their
name on a card. I mix the cards and
give everybody one card. If you get back
the card with your name – I pay you $10.
Let n be the number of people in the class.
For what n is the game advantageous for me?
Linearity of expectation
Everybody pays me $1 and writes their
name on a card. I mix the cards and
give everybody one card. If you get back
the card with your name – I pay you $10.
X1 = -9 if player 1 gets his card back
1 otherwise
E[X1] = ?
Linearity of expectation
Everybody pays me $1 and writes their
name on a card. I mix the cards and
give everybody one card. If you get back
the card with your name – I pay you $10.
X1 = -9 if player 1 gets his card back
1 otherwise
E[X1] = -9/n + 1*(n-1)/n
Linearity of expectation
Everybody pays me $1 and writes their
name on a card. I mix the cards and
give everybody one card. If you get back
the card with your name – I pay you $10.
X1 = -9
1
X2 = -9
1
if player 1 gets his card back
otherwise
if player 2 gets his card back
otherwise
E[X1+…+Xn] = E[X1]+…+E[Xn] =
n ( -9/n + 1*(n-1)/n ) = n – 10.
Expected number of coin-tosses
until HEADS?
Expected number of coin-tosses
until HEADS?
1/2
1/4
1/8
1/16
….

1
2
3
4
 n.2-n = 2
n=1
Expected number of dice-throws
until you get “6” ?