Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Selection and Adversary arguments Sung Yong Shin CS Dept., KAIST Contents 1. Introduction 2. Finding max. and min. 3. Finding the 2th largest key 4. The Selection Problem 5. A lower bound for finding the median 1. Introduction SP : (Selection Problem) Given a set of n real numbers, find the kth smallest one, 1 k n. How can you solve it? well, … (1) Sort the numbers. (2) Pick the kth smallest one. O(nlogn) Any better way? What is a trivial lower bound in time complexity for solving SP? TL(n) = (n) Why? What if only considering comparisons? well, ... P : Given a set S of n real numbers, find the largest one. n=3 S = {x1, x2, x3} 1:2 L = {11, 12, 13, 14} < > 1:3 2:3 < > < 11 12 x3 (x1, x2, x3) x2 (?, ?, x2) > 13 x3 (x2, x1, x3) |W| |L| W = {(?, ?, x1), (?, ?, x2), (?, ?, x3)} |W| = n TL(n) = log2|W| = log2n However, this is not tight !!! Why? 14 x1 (?, ?, x1) Adversary Arguments Z1000 = {0, 1, …… , 999} Guess what number in Z1000 I have in mind? Guessing Game !!! Maximize the number of leaves in a decision tree. You can change your mind as long as your answers(responses) are consistent !!! 2. Finding Max. and Min. MM : Given a set of n real numbers, find max and min. max = the largest number min = the smallest number How can you solve MM? x1 x2 x3 x4 …… x2n-1 x2m W {x11, x21, x31,……, xm1} max L {x12, x22, x32,……, xm2} min How many comparisons? m …… dividing m-1 …… finding max m-1 …… finding min 3 n2 2 Any better way? 3m 2 n = 2m What information is needed for finding max and min? Finding max : All numbers except max itself must lose at least once in some comparisons. (n-1 losses) Similarly, Finding min : All numbers except min itself must win at least once in some comparisons. (n-1 wins) 1 unit of information (1 win) or (1 loss) (2n - 2) units of information is needed !!! Is that right? Yes !!! Why? x1 L W x2 L W x3 L W max x4 W x5 L W min x6 L Status of a number (xi, si) Status W at least one win, no lost L at least one lost, no win WL wins and losts N no comparisons Status of keys x and y compared by an algorithm N,N W,N or WL,N L,N W,W L,L W,L or WL,L or W,WL WL,WL Adversary response New Status Units of new information x>y x>y x<y x>y x>y x>y Consistent with assigned values W,L W,L or WL,L L,W W,WL WL,L No change No change 2 1 1 1 1 0 0 Example Theorem : Any algorithm to find max and min of n numbers must do at least 3n/2-2 comparisons in the worst case [proof] n 2m (for simplicity) (N, N) n 2 …… 2m information ? ? …… 2m-2, since 2n-2(4m-2) needed n n 3n (2m 2) (n 2) 2 2 2 2 What if n = 2m + 1 ? Exercise. 3. Finding the 2nd largest key 2L : Given a set of n real numbers find max and max2 max2 the 2nd largest one 19 19 7* 19 19 1* 15* 2 15 7 (n-1) + (log2 n - 1) = n + log2 n - 2 comparisons 9 6 15 3 6 x1 x2 x3 xn W L L L W L L Do we need all those “L” ? Any better algorithm ? max --- n - 1 comparisons max2 --- ? How many numbers are compared directly with max. w(xi) = 1, i = 1, 2, ……, n Comparing xi and xj w(xi) > w(xj) xi > xj w(xi) = w(xj) > 0 w(xi) := w(xi) + w(xj) w(xj) := 0 `` `` w(xi) < w(xj) xi < xj w(xi) = w(xj) = 0 consistent w(xj) := w(xj) + w(xi) w(xi) := 0 no change w(x2) w(x3) w(x1) (x1,x2) 1 1 2 0 1 2 (x3,x5) 3 0 1 w(x5) 1 x1 > x2 (x3,x4) (x1,x3) w(x4) 5 0 x3 > x4 0 x3 > x5 x1 < x3 Lemma : # of direct loser to max = log2n [Proof] max xi for some i w(xi) = n wk(xi) w(xi) after the kth win against a previously undefeated key wk(xi) 2 wk-1(xi), w0(xi) = 1 Why ? n = wt(xi) 2tw0(xi) n = 2t Why? t = log2n Theorem : Any algorithm to find the 2nd largest number in a set of n real numbers must do at least n + log2n - 2 comparisons. 4. Selection Problem SP : Given a set S of n real numbers, find the kth smallest one. n - k numbers > Nk Nk k - 1 numbers < Nk a b b is less than a (b < a) In order to know the kth smallest number Nk, the relation to each number in S to Nk must be known !!! Why ? y Nk y x x An adversary could change the value of y which is not related to Nk !!! n - 1 Comparisons !!! Why ? Theorem : Finding the kth smallest element in S requires at least |S| - 1 comparisons. How to find the kth smallest one A straightforward approach (1) Sort S (2) Pick the kth smallest one O(nlogn) Far from optimality !!! Any better idea ? well, …. Try “Divide and Conquer !!!” S = { 21, 15, 13, 8, 7, 29, 22, 2, 5, 10, 3, 26, 4, 19, 12, 20, 18, 24, 16, 23, 11, 1, 25, 14, 27, 6, 17, 9, 28 } 21 15 13 8 7 29 22 2 5 10 3 26 4 19 12 20 18 24 16 23 11 1 25 14 27 6 17 9 28 21 15 13 8 7 29 22 10 5 2 26 19 12 4 3 24 23 20 18 16 27 25 14 11 1 6 17 9 28 Divide S into |S|/5 sequences of S element each with up to 4 leftover elements Sort each 5-element sequence A 29 22 10 5 2 C 26 19 12 4 3 21 15 13 8 7 B 27 25 14 11 1 D 24 23 20 18 16 6 17 9 28 M= m = the median of M S1 = {s | s < m and s S} S2 = {s | s = m and s S} S3 = {s | s > m and s S} |S1| 3 |S| 4 Why? 3 |S3| |S| 4 . . . . . C A . . . . . . . . . . . . . m. . . . . . . B . . . . . D . . . . . . . . . . S1 = {s | s < m and s S} S2 = {s | s = m and s S} S3 = {s | s > m and s S} 3 3 |S1| |S| and |S3| |S| 4 4 m if |S1| k then select (S1, k) else if |S1| + |S2| k then m is the kth smallest one else select (S3, k - |S1| - |S2|) end 3n T (n) T ( ) c n T (n / 5) 4 Why? Algorithm ( finding the kth smallest element in S ) procedure SELECT(k,S) begin if |S| < 50 then sort S; SELECT := the kth smallest one end {if} else divide S into |S|/5 sequences of 5 elements each with up to 4 leftover elements; sort each 5-element sequences; Let M be the sequence of medians of 5-elements sets (sequence); |M | , M ); m := SELECT( 2 S1 = {s | s < m and s S}; S2 = {s | s = m and s S}; S3 = {s | s > m and s S}; if |S1| k then SELECT ( k, S1 ) else if |S1| + |S2| k then SELECT := m else SELECT ( k - |S1| - |S2|, S3 ) end end end c1 n Why? c2 n n T 5 c3 n 3n T 4 if n 50 c n T ( n) T (n / 5) T (3n / 4) c n if n 50 Show that T(n) 20cn . How? By induction!!! n = 50 150 T (50) T (10) T ( ) c 50 4 T (10) T (38) c 50 c 10 c 38 c 50 c 98 20 c 50 50 < m k T(m) 20cm n = k+1 T(k+1) T((k+1) / 5) + T(3(k+1) / 4) + c·(k+1) 20·c·(k+1) / 5 + 20·c·3(k+1) / 4 + c(k+1) 4c(k+1) + 15·c·(k+1) + c(k+1) 20·c·(k+1) Finding the Median # of comparisons 16n Blum 5.4n Hyafile & Schonhage Paterson & Pippenger The 3n + o(n) best known little “o” algorithm 5. A Lower Bound for Finding the Median k = n/2 n-1 comparisons Can you find any tighter lower bound? well, … Why not using an adversary argument? Observation median Crucial Comparisons non-crucial comparisons Def’n : A comparison involving an element x is said to be a crucial comparison for x if it is the first comparison with y satisfying one of the following conditions : (1) x > y for some y median. (2) x < y for some y median. Note : (i) A crucial comparison for x establishes the relation of x to median. (ii) The relation of y to median is not necessarily known at the time the crucial comparison for x is done. Adversary Strategy “Force an algorithm to perform as many non-crucial comparisons as possible.” How ? Assigning values to variables. (xi, si) Status L: assigned a value larger than median S: assigned a value smaller than median N: not yet in comparison Comparing xi and xj, i j {N, N} -- xi > median > xj {L, N} -- make the unassigned one smaller than median {S, N} -- reverse the above {L, L} consistent with previous responses {S, S} Unless there are already (n - 1) / 2 elements with status S (or L), keep the strategy previously stated !!! Otherwise, make the balance for the number of both status. (n - 1) / 2 non-crucial comparisons possible !!! Why ? (n 1) / 2 (n 1) 3 (n 1) 2 Comparisons crucial non-crucial Theorem : Any algorithm to find the median of n numbers must do at least 3 (n 1) comparisons. 2 3 ( n 1) 2 1.75n log n 1.8n 2n Best lower bound currently known !!! (n - 1) comparisons are tight lower bound only for k = 1 and n !!! Project 2 • “Graph-related algorithms” – Both directed and undirected graphs – Menu-driven 1) Initialize Graphs 2) Min-Cost spanning tree 3) Dijkstra’s shortest path 4) Depth-first Search 5) Breadth-first Search 6) Biconnected components 7) Strongly connected components Due: Nov. 29 (Friday)