Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Randomized Algorithms CS648 Lecture 7 Two applications of Union Theorem β’ Balls into Bin experiment : Maximum load β’ Randomized Quick Sort: Concentration of the running time 1 Union theorem Theorem: Suppose there is an event π defined over a probability space (π,P) such that π= π ππ , then P(π ) β€ π π(ππ ) Furthermore, if π(ππ ) is same for each π, then P(π ) β€ π π(ππ ) 2 Union theorem When to use Union theorem: Suppose we wish to get an upper bound on P(π ) but it turns out to be difficult to calculate P(π ) directly. How to use Union theorem: Try to express π as union of π events ππ (usually identical) such that it is easy to calculate P(ππ ). Then we can get an upper bound on P(π ) as P(π ) β€ π π(ππ ) 3 APPLICATION 1 OF THE UNION THEOREM BALLS INTO BINS: MAXIMUM LOAD 4 Balls into Bins 1 1 2 3 4 2 5 3 β¦ β¦ i m-1 m β¦ n Ball-bin Experiment: There are π balls and π bins. Each ball selects its bin randomly uniformly and independent of other balls and falls into it. Used in: β’ β’ Hashing Load balancing in distributed environment 5 Balls into Bins 1 1 2 3 4 2 5 3 β¦ β¦ j m-1 m β¦ n Ball-bin Experiment: There are π balls and π bins. Each ball selects its bin randomly uniformly and independent of other balls and falls into it. Theorem: For the case when π = π, prove that with very high probability, every bin has O(log π) balls. 6 Balls into Bins The main difficulty and the way out 1 1 Event π: 2 3 4 2 5 3 β¦ β¦ j m-1 m β¦ n There is some bin having at least c log π balls. Observation: It is too difficult to calculate P(π ) directly. Question: What is the way out? 7 Balls into Bins From perspective of πth bin 1 1 2 3 4 2 5 3 β¦ β¦ j m-1 m β¦ π : There is some bin having at least c log π balls. ο¨ P(π ) β€ Event ππ : πth bin has at least c log π balls. Question: What is the relation π and ππ ? Answer: π = π ππ n Event π π π( π ) 8 Balls into Bins From perspective of πth bin 1 1 2 3 4 2 5 3 β¦ β¦ j m-1 m β¦ n π : There is some bin having at least c log π balls. ο¨ P(π ) β€ π π(ππ ) Event ππ : πth bin has at least c log π balls. Observation: In order to show P(π ) < πβ4 , it suffice to show P(ππ ) < ??πβ5 P(ππ ) < πβ5 Event 9 AIM: TO SHOW P(ππ ) < πβ5 P(πTH BIN HAS AT LEAST π π₯π¨π π BALLS) < πβ5 10 Calculating P(ππ ) P[ππ ] = π=π log π = π=π log π β€ π=π log π πβ πβ1 πβ2 β¦(πβπ+1) 1 π (π ) π=π log π π! 1 π=π log π π! = β€ β€ β€ β€ P(πth bin has π balls) π β (π1 )π β (1 β π1 )πβπ π π 1 β (π )π π 1 2 1 2 1 2 π=π log π (ππ )π π=π log π π π (π log ) π π=2π log π Using Stirlingβs formula π! β (ππ )π 2ππ Choosing π = 2π π π (2π log ) π β€ (12 )2π log π β€ πβ2π β€ πβ5 11 Balls into Bins Theorem: If π balls are thrown randomly uniformly and independently into bins π, then with probability 1 β πβ4 , maximum load of any bin will be O(log π) balls. Note: With slightly more careful calculation, it can be shown that the maximum load will be O((log π)/log log π). 12 APPLICATION 2 OF THE UNION THEOREM RANDOMIZED QUICK SORT: THE SECRET OF ITS POPULARITY 13 Concentration of Randomized Quick Sort π π β¦ A π : random variable for the no. of comparisons during Randomized Quick Sort We know: E[π]= 2π lπππ π β πΆ(π) Our aim: P(π > π π lπππ π) < πβπ For any constant π, we can find constants π and π such that the above inequality holds. We shall show that P(π > 8π lππ4/3 π) < πβ7 14 Concentration of Randomized Quick Sort Tools needed 1. Slightly generalized Union theorem: Suppose there is an event π defined over a probability space (π,P) such that π β= π ππ , then P(π ) β€ π π(ππ ) 2. Probability that we get less than π‘ HEADS during 8π‘ tosses of a fair coin is 3 less than ( )8π‘ . 4 15 Randomized QuickSort The main difficulty and the way out Elements of A arranged in Increasing order of values ππ ππ Question: What is the main difficulty in showing P(π > 8π lππ4/3 π) < πβ7 Answer: No direct way to bound P(π > 8π lππ4/3 π) because β’ sample space is too huge β’ Sample space is non-uniform Question: How could we bound E[π] ? Answer: (by taking microscopic view of Randomized Quick sort) 16 Randomized QuickSort The main difficulty and the way out Elements of A arranged in Increasing order of values ππ Question: What is the main difficulty in showing P(π > 8π lππ4/3 π) < πβ7 Answer: No direct way to bound P(π > 8π lππ4/3 π) because β’ sample space is too huge β’ Sample space is non-uniform Question: How could we bound E[π] ? Answer: (by taking microscopic view of Randomized Quick sort) 17 Randomized QuickSort from perspective of ππ Elements of A arranged in Increasing order of values ππ ππ leaves the algorithm 18 Randomized QuickSort from perspective of ππ ππ : no. of recursive calls in which ππ participates before being selected as a pivot. Question: Is there any relation between π and ππ ? Answer: π = ππ=1 ππ 19 Randomized QuickSort A new way to count the comparisons Elements of A arranged in Increasing order of values Key idea: Assign each comparison during a recursive calls to the non-pivot element. Question: Is there any relation between π and ππ ? Answer: π = ππ=1 ππ Observation: If π > 8π π₯π¨π π/π π, there must be at least one π such that ππ > 8 π₯π¨π π/π π 20 Randomized QuickSort Applying Union theorem Observation: If π > 8π π₯π¨π π/π π, there must be at least one π such that ππ > 8 π₯π¨π π/π π Event π : π > 8π π₯π¨π π/π π Event ππ : ππ > 8 π₯π¨π π/π π Question: What is the relation π and ππ ? Answer: π β ο¨ P(π ) β€ ππ π π(ππ ) π Observation: In order to show P(π ) < πβ7 , it suffice to show P(ππ ) < ??πβ8 P(ππ > 8 π₯π¨π π/π π) < πβ8 21 AIM: TO SHOW P(ππ > 8 π₯π¨π π/π π) < πβ8 22 Randomized Quick Sort Increasing order of values β¦ middle-half Definition: a recursive call is good if the pivot is selected from the middle half, and bad otherwise. 1 P(a recursive call is good) = ?? 2 Notation: The size of a recursive call is the size of the subarray it sorts. 23 Randomized Quick Sort Increasing order of values β¦ middle-half Observation: If a recursive call is good, size of each of its child-recursive calls reduces by a factor of π π. 24 Randomized Quick Sort ππ Increasing order of values β¦ middle-half Question: What is the maximum no. of good recursive calls can ππ have ? Answer: π₯π¨π π/π π. 25 Randomized Quick Sort Summary from the perspective of ππ During Randomized Quick Sort element ππ β’ Participates in a sequence of recursive calls each of which is good independently with probability 1/2. β’ ππ leaves the algorithm on or before participating in π₯π¨π π/π π good recursive calls. ππ can be re-stated as: ππ participated in more than 8 π₯π¨π π/π π recursive calls but fewer than π₯π¨π π/π π turned out to be good. 3 8 π₯π¨π π/π π ) < ( ) π 4 = πβ8 P(π Probability we get less than π‘ HEADS 3 during 8π‘ tosses of a fair coin < (4)8π‘ . 26 Randomized Quick Sort Final result Theorem: Let π be the random variable for the no. of comparisons during Randomized Quick Sort on input of size π P(π > 8π lππ4/3 π) < πβ7 Homework: Rework the calculation to find the smallest possible π such that P(π > ππ lπππ π) < πβ2 27 SOME WELL KNOWN AND WELL STUDIED RANDOM VARIABLES 28 Bernoulli Random Variable Bernoulli Random Variable: A random variable X is said to be a Bernoulli random variable with parameter π if it takes value 1 with probability π and takes value 0 with probability 1 β π. The corresponding random experiment is usually called a Bernoulli trial. Example: Tossing a coin (of HEADS probability= π) once, HEADS corresponds to 1 and TAILS corresponds to 0. E[X] = π 29 Binomial Random Variable Binomial Random Variable: Let π1 ,β¦, ππ be π independent Bernoulli random variables with parameter π, then random variable X= π1 + β― +ππ is said to be a Binomial random variable with parameters π and π. Example: number of HEADS when we toss a coin (of HEADs probability= π) π times. Homework: Prove, without any knowledge of binomial coefficients, that E[X] = ππ. 30 Geometric Random Variable Geometric Random Variable: Consider an infinite sequence of independent and identical Bernoulli trials with parameter π. Let X denote the number of these trials upto and including the trial which gives the first 1 is called a Geometric random variable with parameter π. Example: Number of tosses of a coin (of HEADs probability= π) to get the first HEADS. Homework: β’ Find the probability P(X= π). β’ Prove, that E[X] = 1/π 31 Negative Binomial Random Variable Negative Binomial Random Variable: Let π1 ,β¦, ππ be π independent Geometric random variables with parameter π, then random variable X= π1 + β― +ππ is said to be a negative-Binomial random variable with parameters π and π. Example: number of tosses of a coin (of HEADs probability= π) to get π HEADS. Homework: β’ Guess why it is called βnegativeβ Binomial random variable. β’ Find the probability P(X= π). β’ Prove, without any knowledge of binomial coefficients, that E[X] = π/π 32