Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
THE PROBLEM SOLVING PROCESS ABSTRACT DATA TYPES MATHEMATICAL MODEL INFORMAL ALGORITHM PSEUDO – LANGUAGE PROGRAM OR OTHER FORMAL DESCRIPTION 1 DATA STRUCTURES PROGRAM (Pascal, C, C++, ete.) DATA TYPE VERSUS ABSTRUCT DATA TYPE DATA TYPE: Set of values (or objects) ABSTRUCT DATA TYPE (ADT): Set of objects + a mathematical model with a collection of operations defined on the model. 2 DATA TYPE: Set of values (or objects). Fortran 77: LOGICAL INTEGER, REAL, CHARACTER/STRING Composite types: Array of Integers Array of reals Etc. Pascal: Basic types: integer, real, character, Boolean Composite types: array of integers Array of characters Etc. Record of integers/reals/characters Etc. Set of…. File of… THERE ARE OPERATIONS ASSOCIATED WITH EACH TYPE AGGREGATING TOOLS: array, record, file 3 C: Basic types: int, real, char Composite types: arrays, structures WHAT ABOUT POINTERS? Pointer can be treated as a data type, but usually it’s treated as a DATA STRUCTURING FACILTY. 4 ABSTRUCT DATA TYPE (ADT) Set of objects plus a mathematical model with a collection of operations defined on the model Example: List a1, a2 ,…, an LIST (of integers) is an ADT with the following operations: 1. Calculate the length of the list 2. Get the fist member of the list and return null if empty 3. Retrieve the member at position P and return null if P doesn’t exist 4. Locate X in the list 5. Insert X into the list at position P 6. Delete the member at position P 5 P = 1 L = 50, 2 60, 3 4 5 6 7 8 23, 47, 21, 39, 60, 40 1. LENGTH (L) = 8 2. FIRST (L) = 50 3. RETRIEVE (4, L) = 47 RETRIEVE (9, L) = null 4. LOCATE (60, L) = 2 5. INSERT (30, 5, L) gives the result: L = 50, 60, 23, 47, 30, 21, 39, 60, 40 6. DELETE (3, L) gives the result L = 50, 60, 47, 21, 29, 39, 60, 40 ALL OPERATIONS ARE ATOMIC EXCEPT FIRST SINCE: FIRST (L) = RETRIEVE (1, L) 6 EXAMPLE: 1. 2. 3. 4. ADT STACK (OF INTEGERS) Retrieve the top element Delete the top element (POP) Insert x at the top (PUSH) Test if the stack is empty S= 27 40 32 1. TOP (S) = 27 2. POP(S) = results in 40 S= 32 3. PUSH (0, S) results in S= 3. EMPTY (S) = false 7 0 27 40 32 Example: ADT MATRIX (OF REALS) 1. Return number of rows 2. Return number of columns 3. Multiply matrices A and B 4. Add A and B 5. Compute the transpose of matrix A 6. Delete a rows/column 7. Add a row/column 8. Multiply matrix A by real number 6 OBSERVATIONS: 1. Domain of an operation may involve more than one ADT Type 2. Some operations are partial 3. Range of an operation may be a different ADT 8 A simple application of ADT – evaluation of arithmetic express a + b*c/d **e +f Algorithm: Value (x : expression); oprnd: STACK OF REALS optor: STACK OF CHARS x1, x2 : REAL i: INTEGER Initialize oprnd and optor for i:= to LEN (x) do case x[i] of real: PUSH (x[i], oprnd) char: if TOP (optor) < x[i] then PUSH (x[i], optor) else repeat x2: = TOP (oprnd); POP (oprnd); x1: = TOP (oprnd); POP (oprnd); x1: = x1 TOP (optor) x2; PUSH (x1, oprnd); POP (optor) until TOP (optor) < x[i]; PUSH (x[i], optor) end if endcase Value: = top (oprnd) 9 Comparison of ADT’s with procedure – the advantages 1. GENERALIZATION Procedures are generalization of primitive operations (e.g. +, -, *,….) ADT’s are generalizations of primitive data types. 2. ENCAPSULATION (OR MODULARITY) A procedure encapsulates all the statements relevant to a certain aspect of a program. An ADT encapsulates all the definitions and the operations relevant to a data type. How to implement an ADT? Note that a data structure doesn’t have to be associated with an ADT. 10 Data Structure: A collection of data objects connected in various ways. A data structure is always associated with a specific programming language. FORTRAN 77: the only data structuring facility is ARRAY PASCAL: we have: ARRAY, RECORD, and POINTER C: ARRAY, STRUCTURE, POINTER Some important terms: Cell: a box capable of holding a value drawn from some basic or composite data types (e.g. integer, record…) CELL IS BASIC BUILDING BLOCK OF DATA STRUCTURE Pointer: a cell whose value indicates another cell Cursor: an integer-valued cell, used as a pointer to an array 11 Example: A simple data structure is given below. It may be used in the implementation of ADT MATRIX. a11 a11 a11 a22 a22 a22 am1 am1 am1 A pointer 12 A CELL Type Cell type = record Element: real Down: cell type Right: cell type End 13 Example: A data structure below way be used in the implementation of ADT LIST 1.2 3 -1 ≡ uil pointer 1 cursor 0 ≡ uil pointer 3.4 0 2 5.6 2 3 7.8 1 4 A CURSOR L = 7.8, 1.2, 5.6, 3.4 4 2 Type Record type = Record Cursor = integer; Ptr: end 1.2 3 3.4 0 2 5.6 2 3 7.8 1 4 1 14 Record type ALGORITHM VERSUS PROGRAM An algorithm is a finite sequence of instructions satisfying the following criteria. 1) Definiteness : - each instructions must be clear and unambiguous 2) Finiteness: - the algorithm will terminate after a finite number of steps for all cases. 3) Effectiveness:- each instruction can be performed using a finite amount of resource (time and space) A (well-defined) program in principle is similarly described, but program: 1) is always associated with a specific programming Language 2) May not half (e.g. Operating systems) All the programs. We are interested in; half pseudo-Pascal is our chosen language WE WILL USE INTERCHANGABLE ALGORITHM 15 AND PROGRAM Examples: Proc search (x: integer; A : array [1…10] of integer) i=1; while x <> A[i] and I <= 10 do i:= i + 1; search: = i end proc print ( S: set) Print the elements of S end proc Pi print all the digits of Pi } never ends end 16 The running time of a program depends on 1) 2) 3) 4) Computer speed Compiler quality Input to program Program efficiency (or quality) The TIME COMPLEXITY of a program is defined as a function of input, usually the SIZE of input. 17 Program A is of worst case time complexity T(n) if the maximum running time of A on any input of size n is T(n). THE UNITS OF T (n) ARE UNSPECIFED. Although the constants in T (n) are important, we are more interested in the growth rate (or order) of T (n). e.g. 2n ≈ 10n + 1000 2n << n2 when n is large f (n) << g (n) ↔ lim f (n) n∞ g (n) 18 IMPORTANT DEFINITION T (n) is 0 (f(n)) if there are constants C and n0 such that T (n) ≤ C f (n) when n ≥ n0 Note : 0 (f(n)) actually denotes a class of functions of the same or slower growth rates, and it would be better to write T (n) Є 0 (f (n)) No 19 Stands for “is”, not “equals” Examples: 3n2 + 16n + 8 = 0 (n2) C=4 n > 17 n0 = 17 3n2 + 16n + 8 ≤ 4 n2 ------------------T (n) f (n) n logn = 0 (n2) n>0 n logn < n2, jo C = 1, n = 0 3n3 – 6n2 ≠ 0 (n2) V C Э n0 n > n0 3n3 – 6n2 > Cn2 If n > 0 THEN 3n3 -6n2 >Cn2 3n – 6 > C Hence if n > C + 6 then 3n3 – 6n2 > Cn2 3 20 k Σ ai n i = 0 (nk) when ak > 0 i= 0 106 = 0 (1) = 0 (2) 100n + 105 = 0 (n) n4 + n2 + n + 6 = 0(n4) 2n + n100 = 0(2n) 3n >> 0 (2n) log10 n = 0 (log2n) since log10n = log2n log210 0 (f(n)) is an upper bound of the at the growth rate order of T (n) if T (n) = 0 (f(n)) 21 To specify a lower bound, we use Ω. DEFINITION: T (n) is Ω (f(n)) if there is a constant C such that T (n) ≥ c f(n) infinitely of ten. ½ n + 100 = Ω (n) T (n) F (n) ½ n + 100 > C n C=½ T (n) = n n2 /100 n is odd & n ≥ 1 n is odd & n ≥ 1 T (n) = Ω (n2) C = 1/100 T (n) ≥ C n2 for n = 0, 2, 4, 6, 22 WHY IT IS IMPORTANT? 5n2 2n n3/2 100n 3000 2000 1000 5 10 15 20 Running times of 4 programs 1000 jek ≈ 17 minutes 23 n HOW LARGE A PROBLEM CAN WE SOLVE SUPPOSE THAT WE NOW WE BUY A MACHINE THAT RUN 10 TIMES FASTER AT NO ADDITIONAL COST. THEN FOR THE SAME COST WE CAN SPNED 104 SECONDS ON A PROBLEM WHRE WE SPENT 103 SEC BEFORE Running time T (n) Max Problem size for 103 sec Max problem size for 104 sec Increase in Max problem size 100 10 100 1000% 5n2 14 45 320% n 3/2 12 271 230% 2n 10 13 130% THE 0 (2n) PROGRAMS CAN SLOVE ONLY SMALL PROGRAMS NO MATTER HOW FAST THE UNDERLYING COMPUTER IS. 24 THEOREM IF T1 (n) = 0 (f(n)) AND T2 (n) = 0 (g(n)) THEN T1 (n) + T2 (n) = 0 (max (f (n)), g (n)). PROOF THERE ARE c1, n1, c2, n2 SUCH THAT n ≥ n1 T1 (n) ≤ c1 f (n) n ≥ n2 T2 (n) ≤ c2 g (n) LET n3 = max (n1,n2). THEN n ≥ n3 T1 (n) + T2 (n) ≤ c1 f (n) + c2 g (n) ≤ (c1 + c2) max (f(n), g (n)). ENDPROOF HENCE: 0 ( f (n)) + 0 (g (n)) = 0 (max (f (n),g (n))) 0 ( n2) + 0 (n3) = 0 (n3) 0 (n2) + 0 (2n2) = 0 (2n2) = 0 (n2) 25 THEOREM IF T1 (n) = 0 (f (n)) AND T2 (n) = 0 (g (n)) THEN T1 (n) T2 (n) = 0 (f (n) g(n)) PROOF THERE ARE c1, n1, c2, n2 SUCH THAT n ≥ n1 T1 (n) ≤ c1 f (n) n ≥ n2 T2 (n) ≤ c2 g (n) LET n3 = max (n1,n2). THEN n ≥ n3 T1 (n) T2 (n) ≤ c1 c2 f(n) g (n) ENDPROOF HENCE: 0 ( f (n)) 0 (g (n)) = 0 (f (n) g (n)) 0 ( n2) 0 (n5) = 0 (n7) 0 (n2) 0 (2nh) = 0 (n22h) = 0 (2h+2logh) OTHER IMPLICATIONS f(n) f (n) Σ 0 (g (i, n)) = 0 ( Σ i=1 i=1 g (i,n)) max (0 (f(n)), 0(g (n)) = 0 (max(f (n)), g(n))) 26 0 (f(n)) = 0 (g (n)) * ASYMMETRIC! ↨ 0 (f (n)) ≤ 0 ( g(n)) ═ Means IS MY CAT IS BLACK ≠ BLACK IS MY CAT 0 : FUNCTION → SET OF FUNCTIONS N2 n2 2 1000 n + 5 0 DEF: 0 (f (n) 0 (f (n) == 0 (f (n) Any Operator +, ., ETC. 27 g (n)) 0 (n2) OTHER USEFUL RULES f(n) ═ 0 (f (n)) C 0(f(n)) ═ 0 (f(n)) 0 (f(n)) + 0 (f(n)) ═ 0(f(n)) 0 (0 (f(n)) ═ 0 (f(n)) 0 (f(n))0(g(n)) ═ 0 (f(n))g(n)) 0 (f(n)g(n)) ═ f(n)0(g(n)) REMEMBER: ═ HERE IS ASYMMERIC! 28 CALCULATING COMPLEXITIES OF ALGORITHMS Procedure : bubble (var A : array [1…..n] of int); BUBBLE SORT A INTO INCREASING ORDER Var i, j, temp: interger: Begin 1 for i: = 1 to n -1 do 2 for j:=n down to it 1 do 3 if A[j-1] > A[j] then begin 4 temp: = A[j-1]; 5 A[j-1] := A[j]; swap A[j-1] and A[j] 6 A[j] := temp end (3) – (6) TAKES 0(1) (2) – (6) TAKES (n -1) 0(1) + 0(1) = 0(n-1) (1) – (6) TAKES: n Σ i=1 n [0(n-i) + 0(1) = 0 ﴾Σ (n-i) = 0 (n(n-1)) i=1 29 2 ═ 0(n2-n) ═ 0 (n2) 2 Function test (m: integer) : Boolean; TESTS IF m IS A POWER OF 2, I.E. M═2K FOR SOME k Begin if m =1 then test:= true else if (m mod 2 = 0) then test:= test (m/2) else test:=false end LET T(m) = time complexity of test 1→C1 2 →C2 3→C1 4→C2 + T(m/2) 5→C2 T (m) = C1 +C2 2c1 + c2 m =1 m odd, m >1 2C1 + C2 + T (m/2) m even A recurrence equation 30 Define a new function: C 1 + C2 m ≤1 T’(m) = 2C1 + C2 + T’ (m/2) m > 1 Then T(m) ≤ T ‘(m) for all the m > 0 i.e. T ‘(m) is an upper bound of T (m). Note: T ‘(m) is defined for all real numbers. T ‘(m) = 2C1 + C2 + T’ (m/2) = 2(C1 + C2 )+ T’ (m/22) = 3(2C1 + C2 ) + T’(m/23 ) … = [log2m] (2C1 + C2) + T’ = ( 2C1 + C2 ) [log2m] + C1 + C2 = 0(logm) THUS: T’(m) = 0 (logm), T(m) = 0 (logm) 31 m 2[log2m] Worst case occurs when M= 2k Ceiliuy : is the smallest integer ≥ x [x] → e.g. [1.5] = 2 [3.1] = 4 [3.0] = 3 NOTE THAT: 2[log2m] ≥ m IF m = 2k [log2m] = k log2m = k, and 2[log2m] = m m=6 log24 = 2 & log2 8 = 3 → 2 <log26 < 3 → [log26] =3 >m 32 → 2[log2m] Problem: What is T (m) ? Is m the length of input! M is 100, 15, 64, etc, just number!! IF m is BINARY and is the number of bits of m, THEN n = [log2m] And i.e. T (m) = 0 (log2m) → T (n) = T ([log2m]) = 0 (log2m) T (n) = 0 (n) M CAN BE TREATED AS : 000….0, m I.E. m UNITS, THEN THEN “LENGTH” OF M IS m, and T (n) = T (m) = 0 (log n) 33 DESIGN OF A PROGRAM TOP – DOWN / BOTTOM – UP APPROACH STEPWISE REFINEMENT, COOSE ADT’S AND DATA STRUCTURES CODING A REMARK ABOUT RUNNING TIME ALTHOUGH THE ORDER OF RUNNING TIME IS VERY IMPORTANT, WE SHOULD ALSO CONSIDER THE FOLLOWING FACTORS IN PRACTIC. 1. THE TIME IT TAKES TO WRITE AND DEBUG THE PROGRAM 2. READABILITY, MODULARITY, ETC. HOW HARD IS TO MAINTAIN THE PROGRAM 3. SOMETIMES CONSTANTS ARE ALSO IMPORTANT 4. SPACE (OR STORAGE) COMLEXITY 5. ACURACY 34 ADT LIST A list is a sequence of zero or more elements of a given type (element type). L = a1, a2, a3, …., an length = n first = a1 last = a1 ai is at postion i ai1 precedes ai ai followa ai1 END(L) = position n+1 some data type Operations INSERT (x,p,l); DELETE(p,L); LOCATE (x,L); RETRIEVE(p,L); MAKENULL(L): L ← Є FIRST(L); NEXT(p,L); PREVIOOUS(p,L) PRINT(L); LENGETH(L); REVERSE(L) CONCAT(L1, L2); etc. EMPTY(L) 35 Array implementation of lists Last 1 a1 2 a2 list an empty max Const max =?; type position = 1..max; LIST = record elements: array [positions] of elements type: last: o ..max end; function END (var L: LIST) : integer; begin END : = L.last+1 end; 36 last 1 2 3 p a1 a2 a3 ap max an-1 an Procedures INSERT (x: elements type; p: position; var L:LIST); Var q: position begin if L.last = max then error (‘list is full) else if (p>L.last+1) or (p>1) then error else begin for q:=L.list downto p do L.elements[q+1]:=L.elements[q]; //shifting to the right// L.last:=L.last+1; L.elements[p]:=x End End; Time co,plexity: INSERT, DELETE, LOCATE – O(n) RETIEVE, NEXT, PREVIOUS END, FIRST, MAKENULL – O(1) Avg. time INSERT, DELETE, LOCATE – O(n) 37 Pointer implementation (linked list) Cell 0 cell 1 cell 2 a1 a2 .. Header list L Type Celltype = record Element : elementtype; Next: ↑ ceeltype End; LIST = ↑ celltype; Position = ↑ celltype; Position i : a pointer to cell i -1, 1≤ i ≤n+1 Function END(L.LIST) : position Var q: position begin q:=L; while q. ↑. Next < > nil do q := q. ↑. Next; END:=q End; 38 cell n an . LIST: record first: ↑ celltype; last: ↑ celltype end; Insert x at p …… time O(1) a ….. b p Delete cell at p Time O(1) … a c b …. p Time O(n) L Header p PREVIOUS (p, L) 39 INSERT, DELETE, RETRIVE, NEXT, FIRST, MAKENULL – O(1) PREVIOUS, LOCATE, END – O(n) Compare the two implementations 1. maximum size of the list – array 2. waste of space – both 3. operation speeds array INSERT DELETE PREVIOUS END 4. O(n) O(n) O(1) O(1) pointer O(1) O(1) O(n) O(1) or O(n) pointer representation can be dangerous! e.g. q:=NEXT(p,L); INSERT (X,P,L) . . . IF q=NEXT(p,L) then 40 P q DOUBLY – LINK – LISTS Cell 1 cell 2 a1 q≠NEXT(p,L)! cell3 a2 a3 Type Cell type = record Element: elementtype; Next, previous: celltype End; Position: celltype; Position I: a pointer to cell I Function LAST (L) Begin LAST : = L.previous End; WHAT HAPPENS IF POINTERS AREN’T AVAILABLE? USE CURSOR! 41 cell n an PATTERN MATCHING IN STRINGS A = {a1, a2, …. , ak } ALPHABET SYMBOL/CHARACTER A STRING x = a1, a2, …. , an n 0 , ai A STRINGD A SPECIAL CAST OF LISTS PATTERN MACTHING: x = a1, a2, …. , an Pat = b1, b2, …. , bm Is pat a substring of x? i.e. ( I : 1 I n – m +1) ai ai+1 …. = b1b2 …bm x = aabbbabbbaaa pat = bab 1234567891011 x = aabbabbbaaa pat = bab yes i= 4 pat = abab => No 42 SIMPLE ALGORITHM x = aabbabbbaaa pat = bab aabbabbbaaa bab NO aabbabbbaaa bab NO aabbabbbaaa bab YES! BUT FOR pat = aaa WE NEED TO MOVE FROM aabbabbbaaa TO aabbabbbaaa aaa aaa SIMILARLY for pat = abab, from 1234567891011 aabbabbbaaa abab TO aabbabbbaaa abab 8 = 11 – 4 +1 43 WORST CASE x = a1, a2, …., am am+1 …..an-m+1 …. an b1 b2…bm b 1 b 2… b m n-m+1 passes EACH PASS TAKES O(m) comparisons, HENCE (n-m+1) O(m) = O(m(n-m+1)) = O (mn) procedure find (x, pat : STRING; var found : Boolean; var i : position) Found is set to false if pat doesn’t occur int x, otherwise found is set to take and I is set to the first position in x where pat begins) Var p,q : position; Begin If not EMPTY (x) and not EMPTY (pat) then Begin Found: = false; i:= FIRST(x); while not found and i END (x) do begin p:= i; q: = FORST(pat); while RETRIEVE (p,x) = RETRIEVE (q, pat) and not found do 44 begin p: = NEXT(p,x); q:= NEXT(q,pat) IF q= END (pat) then found : = true End; If not found then i:=NEXT(i,x) End; IF END(L) IS O(1) THEN T(n,M) = O(MN) END END THE KNUTH, MORRIS PRATT ALGORITHM ( KMP) X = abaababaabacabaababaabaab MISMATCH Pat= abaababaabaab 45 WHAT DO YOU DO NEXT? X= u w u c Pat = u w u a NEXT MOVE X= u w u w u c u a Start comparison X= abaababaabacabaababaabcab abaababaabaab math u abaababaaba u abaababaabacabaababaabaab abaaba baabaab start comparing & mismatch 46 abaaba abaababaabacabaababaabaab abaababaabaab START COMPARING & MISMATCH U aba u abaababaabacabaababaabaab abaababaabaab abaababaabacabaababaabaab ↕ abaababaabaab ↑ mismatch start comparing & mismatch abaababaabacabaababaabaab abaababaabaab COMPARING NUMBER OF COMPARISON IS O (n) BUT HOW TO FIND OUT WHAT IS U? 47 LET pat = b1b2 … bm OR EACH 1 ≤ j ≤ m, LET Largest i sud that 0 < i < j and b1… bi = bj –i+1 … bj f (j) = 0 if sud i does not exist f (j) < j FAILURE FUNCTION j 1 Pat = a f(j) 0 2 b 0 3 a 1 4 a 1 6 b 2 7 a 3 8 b 2 aba abaa abaab abaabab abaababa abaababaab abaababaaba abaababaabaa abaababaabaab 9 a 3 abaaba abaababaa 48 10 11 12 13 14 a b a a B 4 5 6 4 5 TIME COMPLEXITY: T (n, m) = O (n + complexity of defining g) = O (n + complexity of defining f) 0 f(j) = if j = 1 fs(j -1) +1 where s is the smallest I such that bfi(j-1) +1 = bj 0 if no such i exist f i (j -1) = f (f(… f(j -1)…..) i times 49 f 3 (j -1) = f (f (f (j-1))) T(j-1) T1 u j-1 a u a j f(j- 1) HERE f(j) = f (j -1) +1 j -1 U b u a i= f(j -1) j u j -1 a b a w w j f(i) = f (f(i-1)) = f2 (i-1) 50 f2(i-1) +1 proc fail (pat[1…m], vav f: away [1…m] of integer ) vav i, j = integer; begin f[1] : = 0; for j: = 2 to m do begin i:= f [j -1]; while (pat[j] ≠ pa[i+1} and i > 0) do i:= f[i]; if pat[j] = pat[i+1] then f[j]:= i+1 else f[j]:=0 and end T (m) = O (m) ! 51 Procedure KMP (x, pat, g, found, i); { x[1..n] , pat [1..m] are strings ; g[j] = g (j), 1 ≤ j ≤ m} var p, q : position begin if n ≠ 0 and m ≠ 0 then begin p:=1; q:=1; while (p ≠ n+1 and q ≠ m+1) do if x[p] = pat [9] then begin p: = p+1; q: = q+1; end else if q =1 then p:= p+1 else q: = g[q]; Time = 0(m) If q = m+1 then begin found : = true; i: = p-m end else found :=false end else …… end ; 52 ADT STACK “ LAST-IN-FIRST – OUT” LIST (LIFD) OPERATIONS: MAKENULL(S) : make stack S empty TOP(s) : Return the top element of s RETRIEVE (FIRST, S) TOP (s) = RETRIEVE (I, S) POP(S) : Delete the top element of S Sometimes POP is defined as function that returns the element being popped out DELETE (FIRST (s) , S) POP (s) = DELETE (I, S) PUSH (X,S); insert x at the top of S PUSH (x, s) = INSERT (x, I , s) INSERT (x, FIRST(S), S) EMPTY (s) : Return true if S is empty, false otherwise 53 A SIMPLE EXAMPLE: F: erase characters, if cancels the previous uncancelled character @: kill character, if cancels all previous characters on the line abc # d @ aa#b = ab Procedure EDIT Var S : STACK OF CHAR; C: CHAR Begin MAKENULL (S); Read (c); While not end ( c ) do Begin If c = ‘#’ then POP (s) Else if c = ‘@’ then MAKENULL(S) Else PUSH (c, S); Read (c) End; PRINT S IN REVERSE ORDER End 54 ARRAY IMPLEMENTATION OF STACKS TOP k 1 1 2 force K K 1ST ELEMENT 2ND ELEMENT max MAX LAST ELEMENT type : position = 1 … max; STACK = record Top : 1 .. max +1; Elements : away [position] of element type PUSH, POP, TOP – O ( 1) 55 stack MORE SPACE – EFFICIENT IMPLEMENTATION POINTER IMPLEMENTAION Stack a b c . MANY STACKS IN ONE ARRAY TOP 1 2 STACK 1 3 STACK 2 BOTTOM 1 2 3 STACK 3 Stack pace 56 tree ADT QUEUE A QUEUE IS A “First – in – First – Out” LIST > (FIFO) OPERATIONS: MAKENULL (Q); FRONT (Q) : return the first element of Q FRONT (Q) = retrieve (first (Q), Q) ENQUEUE (x, Q) : inserts x at the end of Q ENQUEUE (X, Q) = INSERT (X, END (Q), Q) DEQUEUE (Q): DELETES THE FIRST ELEMENT OF Q DEQUEUE (Q) = DELETE (FIRST (Q),Q) EMPTY (Q): 57 POINTER IMPLEMENTATION header a1 a2 … front near type celltype = record element : elementtype; next : ↑ celltype end; QUEUE = record Front, rear : ↑ celltype End; FUNCTION EMPTY (Q : QUEUE) : Boolean; Begin If Q. front = Q.rear then EMPTY: = true Else EMPTY : = false End 58 an EACH OPERATION – 0 (1) ARRAY IMPLEMANTATION FRONT TREE 1 1ST ELEMENT 2ND ELELMENT QUEUE REAR LAST ELELEMT MAX TREE 59 ENQUEUE – 0 (n) CIRCULAR ARRAY IMPLEMENTATION (BUFFER!) Max -1 max an real 1 …. . a2 . . a1 A1 60 2 HOW DO WE DISTRINGUSH BETWEEN FULL AND EMPTY MAINTAIN AN EXTRA BIT FULL ≡ (FRONT = addone(addone(real))) Mark [i, j] = 1 if have been to (i, j) 0 otherwise IF NO WAY OUT, BEACK UP ONE CELL AND TRY A DIFFERENT MOVE MUST STRORE THE CURRENT PATH SOMEWHERE A PATH: (i, j), (i2, j2), …., (is, js) 61 STACK (is, js) (is-1, js-1) . . . . (iz, jz) (i, j) NW N (i-j, j -1) NE (i-1, j) (i-1, j+1) W (j, j-1) WS (i + 1, j -1) (i,j) (i, j + 1) (i+1, j) (i+1, j+1) Type offsets = record X: -1 …1 Y: -1 …1 End 62 E SE Directions = (N, NE, E, SE, S, SW W, NW); Var move : away [directions] of offsets d Move[d] .x Move[d].y N NE E SE S SW W NW -1 -1 0 1 1 1 0 -1 Var 0 1 1 1 0 -1 -1 -1 maze : array [0 : m+1, 0 … n + 1] of 0 …1 Mark : array [1..m, 1…n] of 0…1 63 Type: dir = (N, NE, E, SE, S, SW,W,NW,D) Type: elementtype = recond X: 1 … m; Y: 1 … n; Start: dir End STACK = …….. Var path : STACK Fuy NEXTMOVE (loc : elementtype) : dir Var d: dir; S, r, i, j: out Found : bool; begin i = loc.x; j := loc.y; d:=loc.start; found:= false; while d# D ∩ not found do Begin s: = move [d].x; r: = move[d].y; if maze [i + s, j+r] = 0 and mark [ i + s, j +r] = 0 then found : = true else d:= Succ (d) end; NEXTMOVE : = d 64 DEAD END end proc var rat ( var: maze [0 … m+1, 0 … n+1] of 0…1); mark: array [1 …m, 1…n] of 0…1; path : STACK; location : elementtype; d: dir function NEXTMOVE (X: elementtype): dir; ….; begin mark: = (0); MAKENULL (path); initialzation (should be specified last) location:= (1,1, E) ; mark [1,1] : = 1; PUSH (location, path ); While not EMPTY (pathy) do begin location: = TOP (path) ; POP (path); d: = NEXTMOVIE (Location) if d = D then begin location.start: = succ(d); PUSH (location, path); location.x : = location.x + move [d].x; location.y : location.y + move [d].y; if location.x = m and location.y = n then begin print(path); return end else begin PUSH ((location.x, location.y, N), path); Mar(location.x, location.y) =1 End End End 65 end end TIME COMPLEXITY OF RAT : O (mn) SPACE COMPLEXITY OF RAT :O (mn) BUT WITHOUT SING MARK O (8mn) = O (2mn) An application of queues - breadth-first search in trees tree T V1 V2 a1 a2 V3 a3 V4 V5 a4 V6 a5 a6 V9 V7 a7 V8 a8 V10 a9 66 a10 V11 a11 binary LEFT (v), ║ left child of V RIGHT(v) ║ right child of V e.g., LEFT (v3 = null, RIGHT (V3) = V6 DATA(V1) = a1 ROOT(T) = v1 Searching in tree Given tree T and data x, find a node v of T s.t. DATA(v) = X. Possible approaches: 1. Breadth-first search try level 1, then level 2, then level 3, ...etc. 2. Depth-first search: search along the leftmost path until the leaf is reached, then backup, try the 2nd leftmost path, ...etc. 67 Breadth-first X = 20 V1 10 50 V2 V3 5 V4 V5 2 V6 60 20 V12 V10 V7 4 V8 2 V9 V11 5 20 Searching v1, v2, v3, v4, v5, v6, . . . Depth-first Searching order: v1, v2, v3, v4, v5, v6, . . 68 7 30 Procedure DSearch(x,T) begin if x = DATA(ROOT(T))’ then PrintROOT(T) Else DSearch left subtree; DSearch right subtree; end; nonrècursive version procedure var DSearch (x,T); path : STACK of nodes v: node; begin v := ROOT(T); MAKENULL (path); PUSH(v,path); while not empty (path) do begin v := TOP(path); pop(path); if DATA(v) = X then Print v elsel PUSH (LEFT(v)); PUSH(RIGHT(v)); e end end Time: 0(n) Space: 0(n) Space avg: 0(Iogn) 69 // swap// Procedur BSearch(x,T) Var level : QUEUE of nodes; V : node; begin v := ROOT(T); MAKENULL(level); ENQUEUE(vjevel); while not empty (level); begin V := FRONT(level); DEQUEUE(Ievel); if DATA(V) = x then Print v; stop else begin ENQUEUE(LEFT(v), level); ENQUEUE(RIGHT(v), level) end end end; Time:O(n) n =/T/ --------------size of T Space: 0(n) Avg :0(n) 70 Application – implement a DOS command cd:\ Cd:\ name – change current directory to subdirectory name A: job letters study WP 5.0 letters project homework letters What should cd:\letters do? BFS DFS? When do we use DFS? e.g., solution tree 71 Proc. A(x1, x2,....) Var y1, y2, … Begin . . A(a1, a2, …) L1 …. …. Proc. A(x1, x2,....) Var y1, y2, … begin . . . . . Proc. A(x1, x2,....) Var y1, y2, … begin . . . . . A(b1, b2, …) L3 . . . . . . B(c1, c2, …) L3 . . . . . . . 72 Proc. B(x1, x2,....) Var f1, f2, … begin . . . . . . . . . . . . . Ellmination of Recursion Sometimes it is absolutely necessary to eliminate recursive • recursive calls are not supported e.g., FORTRAN • speed is the first priority - do it by yourself Solution: STACK of activation records Generally, an activation record holds 1. current values of the parameters (pass by value) 2. current values of the local variables 3. a label indicating return address Assume that if procedure p(x1, x2 …. var y1, y2, ….) then the recursive call is p(a1, a2, …, y1, y2, ….) 73 General Rules: Procedure P (x1, x2: int; var y: int); Var i, j: int; Begin ____________________________________; ____________________________________; . . . P(a1, a2,y) _________________________________________; _________________________________________; . . . 74 end; Example 1 procedure Ackerrnann (m,n:integer, var A:int); 1. 2. 3. begin if n<O or m<O then wnteln(“error”) else if m=O then A:=n+1 else if n=O then Ackermann (m-1 ,1 ,A) else begin Ackermann(m,n-1 ,A); Ackermanfl(m-i ,A,A); end end; 75 Recursion Elimination procedure Ackerrnann(m,n:int; var A:int); label 1,2,3; var S : STACK of record m, n, l:int end; 1:2.3; begin MAKENULL(S); 1: if n<O or m<O then write!n(uerrorx) else if m=O then A:=n+1 else if n=O then begin PUSH((m,n, 3), S); m:=m- 1; n:=1; goto 1 end else begin PUSH((m,n,2), S); n:=n-1; goto 1; 2. PUSH((m, a, 3), S); m:=m-1; n:=A; goto l; end; 76 3. if not EMPTY(S) then begin (m, n,1):=TOP(S); POP(S); case 1 of 2: goto2; 3: goto 3; end end end; {Ackermann} More details in [AHU] pp. 64- 69. [HS] pp. 150-153. * The method works only when no pass-by-reference parameters or, same p-b-r parameters are passed each time (e.g., function) General case??? POINTER!!! p(...,var x:type1) p(...,xp:↑type1) 77 no global variables procedure R(x:integer var y,z: integer); var i: integer; begin --------------------- --------------------y:=x*i ------------------------------------R(a,i,y); ------------------- -----------------end; Trees 78 Basic Terminology 1 2 4 3 12 5 7 6 1. 2. 9 8 11 10 a single node is a tree, also the root. if T1,T2 are trees with roots n1, n2, …., nk. Then n nk T= n1 n2 T1 T2 Is a tree with root n. n1, n2, …, nk are the children of n. actually. A rooted tree or oriented siblings 79 TR a subtree of n(and of T) n is the parent of n1, n2, …, nk Note the every node (except root) has a unique parent. A node with no children is a leaf. A non-leaf node is also called an internal node. n1 n2 n3 nk n1, n2, n3,…., nk is a path of length k-1 from n1 to nk Note: n1 is a path of length 0 n1 is an ancestor of nk nk is a desendent of n1 height of n: length of the longest path from n to a leaf 80 height of a leaf is 0! depth of n: length of the unique path from root to n. depth of root is o! height (or depth) of a tree: height of root. Order of nodes in a tree, siblings are ordered from left-to-right (ordered) a ≠ a b c c b if n is to the left of n2 then all descendents of n1 are to the left of all descendents of n2 81 Tree Traversals n T T1 T2 Preorder traversal of T is n, DFS preorder traversal of T1 preorder traversal of T2 preorder traversal of Tk Inorder traversal of T is i.t. of T1 n, i.t. of T2 ..., i.t. of Tk Postorder traversal of T is p.t. of T1 p.t. of T2, ..., p.t. of Tk, n. ↑ evaluation of expression trees, divide-and conquer 82 TR Example 1 1 4 2 3 10 9 8 7 5 6 Preorder: 1, 2, 5, 3, 6, 7, 4, 8, 9, 10 Inorder: 5, 2, 1, 6, 3, 7, 8, 4, 9, 10 Postorder: 5, 2, 6, 7, 3, 8, 9, 10, 4, 1 Preorder: we list a node the first time we pass it Postorder: we list a node the last time we pass it Inorder: we list the first time, but list an interior node the second time we pass it 83 procedure Preorder (T:tree); var v: node; begin V := ROOT(T); Print v; for each subtree T of v, from left to right do Preorder (T) end; time complexity: O(|T|) ← number of nodes Pre/In/Post space complexity: 0 (height of T) ← stack 84 Procedure Preorder (T:tree); // no stack // var v: node; begin V := ROOT(T) while v ≠ null do begin print v; if v ≠ leaf then v := 1st child of v back up until v is not the last child of parent(v) else while v ≠ null and v = last child of Parent(v) do v := Parent(v); if v ≠ null then v := next sibling of v end end; time = O(|T|) if parent () is 0(1) space ÷? 85 Reconstructing a tree from its traversals Preorder and Postorder traversals are sufficient. Preorder and Inorder traversals aren’t sufficient. a a b c b e d c d Inorder and Postorder traversals aren’t sufficient. example trees? Any single traversal isn’t sufficient. (pre/in/post) 86 e Labelled Trees, Expression Trees n1 * + + n3 n2 a a b n4 n5 c n6 n7 n2 represents a+b n3 represents a+c n1 represents (a + b) * (a + c) Evaluation can be done by a postorder traversal. pre/in/post-order listings give prefix (Polish), infix, postfix (Reverse Polish) ↑ *+ab+ac ↑ a+b*a+c ↑ ab+ac+* 87 ADT TREE 1. PARENT (n,T).: node. If no parent return null node. 2. LEFTMOST-CHILD (n,T) : node 3. RIGHT-SIBLING (n,T): node returns the sibling immediately following n. 4. LABEL (n,T): label ≡ DATA(n,T) 5. ROOT(T) : node 6. MAKENULL(T) 7. CREATEL (v1, T1, T2 Ti ): tree; i=O,1,2,... v n T1 Ti T2 Alternative: ATTACH (T1 T2 ) : tree 8. DELETE (n,T) - delete the subtree rooted at n. 88 a1 n1 a2 a5 n2 a3 a4 n3 n4 a7 n5 a9 a6 a8 n6 n6 n8 LEFTMOST-CHiLD (n1, T = n2 ) RIGHT-SIBLING (n1, T) = n4 RIGHT-SIBLING (n7, T) = ^ procedure PREORDER (n:node); //list labels of descendents of n in T (global) in preorder!! var begin print LABEL (n,T); n := LEFTMOST-CHILD (n,T) while n ≠ ^ do begin PREORDER (n); n := RIGHT-SIBLING (n,T) end end; 89 n9 Array Implementation a a 1 3 b 2 b 4 c b a 9 5 6 10 8 a c 7 b 1 2 3 4 5 6 7 8 9 10 parent (10, T) 0 1 1 2 2 5 5 5 3 3 0=^ a b a c b a b c a b label node i is to the left of node j then i < j i.e., number siblings from left to right e.g., preorder, even inorder type node = 1 .. max cell = record parent : 0 … max; label : labletype end; 90 THREE = array [1… max] of cell function LEFTMOST-CHILD(n:node; T:TREE):node; var i:integer begin i : =1; while i ≤ max and T [i] . parent ≠ n do i := i+1; if i > max then LEFTMOST-CHILD := 0 else LEFTMOST-CH := i time: O(|T|) end; function RIGHT-SIBLING(n:node; T:TREE):node; var i:integer parent:node; begin time O(|T|) parent := T[n].parent; i := n+1; while i ≤ max and T [i]. parent ≠ parent do i := i+1; if i > max then RIGHT-SIBLING := 0 else RIGHT-SIBLING := 1 end; 91 Trees as lists of children Label children node right sibling 1 2 . . . . . . 3 4 5 6 7 8 9 10 6 node space type node = 1 .. max LIST = … TREE = record header : array [1..max] of LIST; labels : array [1..max] of labletype root : node end; no matter how LIST is implemented, LEFTMOST-CHILD; RIGHT- SIBLING _ 0(1) PARENT – 0(|T|) If want 0(1) for all, add parent field 92 7 Considering CREATE (n, T1, T2, …, Ti); node space 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 T1 T T2 A 6 C 4 . 12 . . B 10 G I D F E . H . . 2 . . 11 8 14 I A E T1 T2 B C F D 93 G H Simplified Leftmost=child & right-sibling representation A C B D Leftmost Child label right siblings 3 8 B 5 5 0 C 0 7 8 3 0 A D 0 0 Var cellspace : array [1..max] of record Label : labeltype; Leftmost-child, right-sibling:0 .. max End 94 SUMMARY 1. Array of Parents • PARENT--O(1) • LEFTMOST-CHILD, RIGHT-SIBLING - O(|T|) ALL-CHILDREN — 0(m) • simple, space-efficient 2. List of Children • LEFTMOST-CHILD - 0(1) • PARENT, RIGHT-SIBLING -- 0(|T|) • can store several trees, CREATE 3. Leftmost-child, Right-sibling • LEFTMOST-CHILD, RIGHT-SIBLING -- 0(1) • PARENT — O(|T|) • make tree, CREATE, slightly more space than (2) 95 BINARY TREES A node is a binary tree If T is a binary tree, v is a node, then V V T T If T1, T2 binary trees, v a node then V T2 T1 A binary tree is NOT a tree!!! A A B ≠ 96 B Binary Trees A child is either a left or right child Binary tree are not really trees full binary tree: every internal node has two children and leaves have the same depth complete binary tree: obtained from a full binary tree as follows: fix a leaf and delete all the leaves to the right of it • no. of nodes of depth i ≤ 2i i size of a binary tree of depth i ≤ Σ 2i = 2i+1 -1 j=0 If complete, 2i -1 < size ≤ 2i+1 -1 If full, size = 2i+1 -1 size-1 ≥ Depth ≤ log2 (size +1) -1 97 Binary tree traversals v T1 T2 Preorder (T): V, preorder (T1), preorder (T2) Inorder (T): * Inorder (T1), v, inorder (T2) v T2 Postorder (T): Postorder (T1), postorder (T2), v 98 How to reconstruct a binary tree from its traversals? Just Preorder (or inorder or postorder) traversal is not enough. Preorder & Postorder aren’t enough! a a b b Preorder and a1, a2, ….an 1. Inorder b1, b2, …, bn Find i s.t. a1 = bi Then T1 = Reconstruct (a2, … ai,bi, …., bi -1) T = Reconstruct (ai+1, …., an, bi+1, …, bn) a1 T= T1 T2 Posorder & inorder similar 99 Representation of binary trees A B . D . C . . . E . Type node = record label : labeltype; left, right : ↑ node end TREE ↑ node; Notes: 1. cursors may also be used. 2. if operation PARENT ( ) is crucial, a parent field could be included. 3. but if traversal is the only concern, then the parent field is not really needed. 100 F . procedure PREORDER(T:TREE); var temp, tempparent, tempchild; procedure BACKUP; var stop : boolean; begin // find the successor of temp in preorder traversal// stop false; temp:=tempparent; while temp ≠ nil and not stop do begin if temp ↑. tag = 0 then begin tempparent := temp ↑. left; temp ↑ .left := tempchild; if temp 1. right ≠ nil then begin tempchild := temp t right; temp ↑. right := tempparent; temp ↑. tag := 1 temparent : = then temp := tempchild; stop:= true; return; end else begin // tempt. tag = 1 // tempparent := temp ↑ .right; temp ↑. right := tempchitd end; tempchild := temp; temp := tempparent end end; // end of backup // 101 Begin // print nodes of T in preorder // temp := T; tempparent := nil; while temp ≠ nil do begin Print temp ↑. label; if temp ↑. left ≠ nil then begin tempchild := temp ↑ . left; temp ↑. left := tempparent; temp ↑ . tag := 0; tempparent := temp; temp := tempchild end else if temp ↑ . right ≠ nil then begin tempchild := temp ↑ . right; temp ↑. right := tempparent; temp ↑. tag := 1; tempparent := temp’ temp := tempchild end else //temp ↑ .left = temp ↑. right = nil // BACKUP end end; {end of PREORDER} 102 Threaded binary trees 0 0 0 0 . 1 1 lefttag righttag = = 0 1 1 1 0 1 → left = leftchild →left = leftthread (predecessor) in inorder 0 1 → right = right child right thread (successor) predecessor/successor in inorder can be found without using stack or flipping 103 . . Representation of complete binary trees A 1 2 B C 3 4 8 H 5 D 9 E 1 2 3 4 5 6 T F 6 10 G 7 7 8 9 10 11 12 13 A B C D E F G HI J ← largest integer ≤ i/2 parent of node i = [1/2], 1<i≤n left child of node I = 2i, 2i ≤ n right child of node I = 2i +1, 2i +1 ≤ n type THREE = record n : 0 .. max; labels : array [1..max] of labeltype end; 104 A B D C . . . . E . . H . Var temp, tempparent, tempchild : ↑ node; tag = 0→ left points to parent 1→ right points to parent type node = record label : labeltype; left, right: ↑ node; tag:0..1; end; TREE = ↑ node; 105 F . G . An a of binary trees - Huffman codes characters : {a1,a2, …. Ak} = A string or message : x1, x2,….xn є A p(ai) - the probability that a will appear in a message Encoding: assign a binary code c(ai) for each ai c(x1, x2…xn) = c(x1)… c(xn) Decoding: given code bib find the unique message x1,x2….xn such that C(x1,x2… xn) = b1b2 …bm Average code length: k Σ p(a1).| c(a1) | i =1 | c(a1)| : length of c(a1) 106 character a b c d e probability code 1 .30 .10 .10 .10 .40 average 000 001 010 011 100 3 code 2 code 01 0010 0011 000 1 00 01 10 000 1 2.1 3 1.7 Prefix property: c(ai ) is not prefix of c(ai ) for any j ≠ i e.g., Code 1 and Code 2 have prefix property, Code 3 doesn’t! Claim : prefix property makes decoding easy e.g., comsider Decoding code 000 …. Code 1 a … on-line decoding Code 2 d… Code 3 ?? 107 Huffman Code - an optimal (least average code length) prefix code Algorithm Huffman ({a1, a2, …. an}); //find Huffman code c(a for each ai // Let a and a be two characters such that p(ai) and p(aj) are the lowest among a1, a2, …., an; Let a be a new character and p(a’) = p(ai) + p(aj); Huffman ({a1, a2, …, an} - (ai,aj} + {a’}); c(ai) = c(a’) 0; c(aj) = c(a) 1; end; Example: {a,b,c} Hufiman ({a,b,c}) p(a) = 0.5, p(b) = 0.3, p(c) = 0.2 c(a)=0, Huff man ({a,[bc]})=> c(a) = 0, c([b]) = 1 108 c(b)=10, c(c)=11 Binary tree representation of prefix code 0 1 1 0 0 0 0 0 e a d 0 1 0 1 0 a b c d e code 1 0 b 1 c code 2 type node = record left, right :↑ node; probability : real ; character : {a1, a2, …., ak) end; used only in leaves A more efficient implementation is given in [AHU] pp.94 -101 109 example 1) a .10 b .20 c .05 d .05 e .10 .10 .20 .05 .05 a b c d .10 .20 .10 .10 f .30 g .10 h .10 .10 .30 .10 .10 e f g .30 .10 .10 h 2) a b e .05 f g h .05 c d (3) & (4) .20 .20 called a forest a .10 .10 .05 .05 c d 110 .10 .10 e g .20 .30 b f .10 h (5) .20 .20 called a forest a .10 .10 .10 .10 e g .05 c .30 d .30 .30 f .20 .10 b h (6) .20 .40 .10 .10 .10 .20 g e .10 a .05 c .05 d 111 (7) .20 .40 .10 .10 .10 .20 g e .05 .10 a c .05 d .60 .30 .30 f .20 b .10 h 112 (6) 1 0 1 0 1 0 f 0 0 1 0 0 1 1 1 e g b h a c d using a modified preorder listing, we can print the Huffman codes for the characters (using a stack) Algotithm Huffman-Tree; // construct a huffman tree for characters a1,a2,….,an// var forest: array [1… max] of THREE; p:real; begin 113 for i:=1 to n do begin new(forest[i]); forest[i] ↑. left := nil; forest[i] ↑. right:= nil; forest[i] ↑. probability := p(ai); forest[i] ↑. Character := ai end; while forest contains more than one tree do begin i := index of the tree with the smallest prob.; j := index of the tree with the second smallest prob.; p := forest[i] ↑. prob + forest [i] ↑. Prob.; forest [i] := CR EATE2( (p,-) ,forest[i],forest[j]); Delete tree forest[j] End End 114 A set is a collection of elements/members Notes: 1. An element can be a set! 2. A set can be infinite or empty. 3. Usually (in this course), members of a set are of the same type. 4. Members of a set are different (otherwise, a multiset). 5. Members could be nearly ordered. A relation is a linear order on some set S (i) (ii) for any a + b in S. exactly one of a<b, a+b, a>b is true. (Trichotomy) for a,b,c in S, a<b, b<c ==> a<c (Transitivity) 115 Some notation: S = { a1, a2, …an} or S = (x|x satisfies condition?) e.g. {1,2,...,10} = (x|x is an integer and 1 ≤ x ≤ 10) Ø = {} Membership: x є S, x ∉ S inclusion: S1⊆ S2 S1⊈ S2 (subset) S1⊆ S2 iff S1 ≠ S2 and S1 ⊆ S2 superset S1⊇ S2 proper superset: S1⊇ S2 Union: S1∪ S2 {1, 2}∪ (2, 3)={1,2,3} Intersection: S1∩S2 {1, 2} ∩{1, 3} = {2} Difference: S1-S2 {1, 2} – {2, 3} = {1} 116 ADT SET 1. MAKENULL(S): 2. INSERT(x,S): S:=S∪ (x} 3. DELETE(x,S): S:=S-{x} 4. MEMBER(x,S):true iff x ∊ S 5. ASSIGN(A,B): copy B into A 6. EQUAL(A,B): true iff A = B 7. UNION(A,B,C): 8. INTERSECTION(A,B,C): C:=A∪B 9. DWFERENCE(A,B,C): 10. MERGE(A,B,C): 11. MIN(S): returns the minimum element In S assuming S is linearly ordered 12. FIND(X): disjoint A1,A2 ,…An - global find the unique A1 St. X ∊ A1 13. S:=ø C:= A∪B C:=A-B if 4∩B=Ø, C:=A∪B otherwise C undefined SIZE(S). *SUBSET(A,B) COMPLEMENT(A) 117 SET with Union, Intersection, Difference Example – data-flow analysis B1 1. 2. 3. GEN = {1,2,3} KILL = {4, 5, 6, 7, 8, 9} t: = ? p:= ? q:= ? 4. 5. read (p) read (q) B2 GEN = {4,5} KILL = {2, 3, 7, 8} q ≤ p? GEN = KILL y B3 GEN = {6} KILL = {1, 9} 6. t : = p B4 7. 8. P:=q q:=t GEN = {7, 8} KILL = {2, 3, 4, 5} B6 GEN = KILL = ∅ P mod q =0 B6 y Write (q) 9. t : = pmodq B8 GEN = KILL = ∅ B7 GEN = {9} KILL = {1,6} GEN[i] = {data definition in B1} KILL[i] = {d|d ∊ Bi & ∊ d ė Bi defining same var as D} 118 DEF1NE[i] {d|∃ a path B1….BiBi, such that d is the last definition of the variable defined d in the path } reaching definitions of Bi DEFIN = (1,4,5) DEFIN = (4,5,6,7,8,9) GEN[i]= {data definitions in Bi } KILL[i] = (data definitions not in B), but defining the same variables as GEN[i] DEFOUT[i] = {d|(same as in DEFIN[I] except “Bi…BiBi”)} leaving definitions DEFOUT[i] = (DEFIN[i] – KILL[i]) ∪GEN[i] DEFIN[i] = ∪ DEFOUT[i] Bi is a predeceasor of Bi, i.e. there is an arc from Bi to Bi) 119 Algorithm dataflow ( GEN;KILL; var DEFIN); Var begin temp SET; i = integer; changed : boolean; no. of blocks for i:= 1 to n do begin MAKENULL(DEFIN[i]); MAKENULL(DEFOUT[i]) end; repeat changed := false; for i:= 1 to n do begin DIFFERENCE(DEFIN[i], KILL[i], temp); UNION (temp. GEN[i], temp); If not EQUAL (temp, DEFOUT[i]) then ASSIGN (DEFOUT[i], temp); Change : = true; End; For i:= 1 to n do begin MAKENULL(DEFIN[i]); for each predecessor Bi of Bi do UNION(DEFIN[i], DEFOUT[i],DEFIN[i]) end; until not changed; end; 120 Example B1 B2 1. 2. read (x) read (y) 3. 4. x: = x+y z: = 10.0 GEN[1] ={1,2} KILL[1]= {3,5} GEN[2] = {3 4} KILL[2] = {1} x z? B3 5. GEN[3] = KILL [3] =Ø y :=x*z GEN[4] ={5} KILL[4] = {2} B4 DEFOUT[I] = (DEFIN[I] – KILL GEN[I] DEFIN[I]= DEFOUT[j] Bj is a predecessor of BI iteration DEFIN[1] DEFOUT[1] DEFIN[2] DEFOUT[2] DEFIN[3] DEFOUT[3] DEFIN[4] DEFOUT[4] Ø 1 Ø Ø Ø Ø Ø Ø Ø Ø Ø 1,2 1,2 3,4 3,4 Ø Ø 5 2 3 4 3,4 2,3,4 2,3,4 1,2 1,2,4 1,2,4 1,2 1,2,4 1,2,4 2,3,4 2,3,4 2,3,4 2,3,4 2,3,4 2,3,4 3,4 2,3,4 2,3,4 3,4 2,3,4 2,3,4 5 3,4,5 3,4,5 121 BIT- VECTOR IMPLEMENATION {A,B,….Z} S {1,2,…N} 1 UNIVERSAL SET 2 I true iff i Є S const N =? Type SET = packed array [1..N] of boolean; Procedure UNIN (A,B: SET; var C:SET); var I = interger; begin for I:=1 to N do C[I]:=A[I] or B[I]; End; MEMBER. INSERT, DELETE, - O (1) MAKENULL, ASSIGN, EQUAL UNION, DIFFERENCE, INTERSECTION – O(N) EMPTY 122 N Linked –list implementation most general, size id unlimited efficient if the sets are ordering by “<” in that case, a set is represented as a shorted list a1,a2…, an a1 <a2 < …..an Unsorted MAKENULL, EMPTY – 0(N) INSERT, MEMBER, DELETE, ASSIGN, 0(n) EQUAL, UNION INTERSECTION, DIFFRENCE – 0(nm) SORTED MAKENULL, EMPLTY –(1) INSERT, MEMBER, DELETE, ASSIGN, EQUAL, UNION, INTERSECTION, DIFFERENCE – O(n) OR O(n+m) Can be improved to O(logn) If balanced search trees are used 123 where ADT Dictionary SET with IINSERT, DELETE, MEMBER, and MAKENULL Example Dean’s list data base Program deanlist(input, output); Type name = packed array[1…20] of char, Grade = -1 ..12 Var student :name; Average: grade; Database: DICTIOARY (of names) Begin MAKENULL(database); Readln(student, average); While student# ‘’ do begin Case avergage of 12..10 : INSERT (student, database) 9: 8.. 0 : DELETE(student, database) -1 : if MEMBER(student, database) then writein(‘yes’) else writeln(‘no’) endcase end end 124 A modified dictionary Type Elementtype = record Key : keytype; Data : datatype End; Then MAKENULL, INSERT, DELETE QUERY (x:keytype) : datatype; INSERT((key,data), dictionary) DELETE(KEY, dictionary) QUERY(key, dictionary) 125 Implementation of dictionary 1. Bit-vector if the universal ser is {1,2,…} INSERT, DELETE, MEMBER – O(1) 2. Sorted or unsorted o(n) INSERT O(n) DELETE or MEMBER O(logn) INSERT O(n) DELETE MEMBER – o(n) If set is ordered 1. Unsorted array (of some constant size) Type DICTIONARY = record Last : 0..max+1 Data : array [1..max] of element type end; Procedure MAKENULL (var : A DICTIONARY) Begin A last :=0 End; 0(1) 126 Function MEMBER(x:elementype; varA:DICTIONARY):boolean; Var i : integer; Begin For i:=1 A.last do O(n) if A.data[i] = x Then return (true); Return(false); End Procedure INSERT (x:elementtype; varA:dict…); Begin If not MEMBER(x,A) then If A.last <max then begin O(n) A.last := A.last +1 A.data[A.last]:x End Else error(‘full’) End; Proceudre DELETE(x,A);; Var i:= integer Begin Find the i s.t A.data[i] =x O(n) or I>a.last; If A.data[i] = then A.data[I]:=… 127 Hashing – O(1) time/operation on avg INSERT. DELETE, MEMBER Universal set To represent set S: put ai in cell I if ai Є a1 a2 a3 a4 . . . . an-1 an 0(1) time if rank (a) = i is o(1) time Generally, partition the elements into groups and let all elements in a group share a cell O(1) time if h(x) = I if x in a group I can be computed in O(1) time. Perfect if elements from a same group do not occur simultaneously! Good if it is unlikely two elements from a same group occur simultaneously! Okay if not TOO MANY elements from a same group occur simultaneously! Some hash functions: I mod p, sum of digits, h(135) = 9 128 Hashing Goal: O(1) / Operation of average INSERT, DELETE, MEMBER Pr(time > C) << 1.0 Open hashing buckets Partition elements in to B classes Hashing function h(x) = I if x Є class I 0 1 b-1 Bucket tables Headers list of elements in each bucket Avg time =1 + N/B per operation If N ≤ C*B., avg time ≤ 1+ C 129 Closed Hashing 0 1 B-1 Bucket table Insert: X x is placed in bucket h1(x) If bucket h1(x) is already taken collision Then try bucket h2(x) rehashing If bucket h2(x) is taken Then try bucket h3(x) . . . Member: X try bucket h1(x), h2(x) Until find it or an empty bucket is met 130 Example 2 Sorting using priority queue Key ≡ priority Pool: priority queue Procedure PQSort(var a array [1…n] of ….); Var pool:PRIORITY QUEUE of ….; I:Intger; Begin MAKENULL(pool); For i:=1 to n do INSERT(A[i], pool); For i:=1 to n do A[I]:= DELELTEMIN(pool) End; Obs: if INSERT, DELETEMIN- o(login), then PQSort – O(nlogn) 131 Previous implantation of sets Bit -vector – O (N) DELETEMIN Array - O (n) INSERT & DELETEMIN Linked list – unsorted O(n) DELETEMIN - sorted O(n) INSERT Hashing - DELETEMIN O(n) Solution – heap partially ordered tree in [AHU] 1 3 2 parent ≤ child 3 5 9 4 5 6 8 7 6 8 9 10 10 3 4 3 9 9 9 18 1 2 10 4 6 5 6 8 9 7 8 9 10 10 18 132 10 11 9 DELETEMIN: 9 5 9 6 8 9 10 10 18 5 9 9 6 8 9 10 10 18 5 6 6 9 9 8 9 10 18 Generally, time = O(depth of tree) = O(logn) (2depth ≤ n ≤ 2depth +1) 133 10 Insert (4, heap); 3 5 9 8 6 9 9 10 10 4 18 3 5 9 4 6 9 9 10 10 8 18 3 4 9 5 6 9 9 10 8 18 time = O(depth) = O(logn) 134 10 A linear order’<’ is a relation on elements; (i) for any two elements a and b a < b, a > b, or a = b (ii) a<b and b<c a <c A set is ordered if a linear order’<’ on its members exists, e.g. sets of integers reals character strings (by lexicographical order) Note: the appearance order of elements in a set representation is unimportant, e.g. (1,3,4) = (3,1,4} = {4,3,1} A sorted list: a1,a2 a3 a4,…. an-1, an 135 Representing ordered sets – binary search trees Elements are ordered by ‘<’ Interested In operations: MAKENULL, INSERT, MEMBER, DELETE, MIN Previous implementation: sorted linked list: MEMBER - 0(n) sorted array: INSERT, DELETE - 0(n) Solution: binary search tree 20 30 15 25 left subtree < parent 45 parent < right subtree 10 17 28 16 136 Pascal Implementation type elementtype = record key:real; data:datatype end; nodetypes = (leaf, interior) I twothreernode = record case kind : nodetypes of leaf: (element:elementtype); nterior (first,second,third: ↑ twothreenode; lowofseconcd,lowofthird:real end; SET = ↑ twothreenode need parent: ↑ Twothreenode? 2-3 three: 3-way B-tree 137 AVL – tree (Adelson – Velskii, landis) Balance binary search tree [HS pp.436-452] AVL tree AVL tree dL dR I dL – dR 1≤1 Empty, single nodes are also AVL trees. An AVL tree is also called a height-balance (or depth) binary tree. BF = dL- dR 12 BF =1 -1 15 -1 7 0 19 1 2 10 0 0 8 138 nd: minimum number of nodes in an AVL tree with depth d. Fact n0 = 1 n1 =2 nd = nd-1 + nd-2 +1 Similar to Fd = Fd-1 + Fd-2 Fibonacci number nd ≥ Fd = Cd/5 c = 1+2 5 >1 d ≤ log cnd + logc 5 depth O(logn) MEMBER, INSERT, DELETE — O(Iogn) 139 Sets with MERGE and FIND MERGE(A,B,C): it A∩B =∅ then C:=A∪B environment = A1,A2,…,Am FIND(x): the unique Ai s.t. x ∊ Ai Example Equivalence problem: Equivalence relation ‘≡’ on set S 1. a ≡a (reflexivity) 2. a ≡b b ≡ a (symmetry) 3. a≡b, b≡c a ≡c (transitivity) e.g. congruence modulo K i ≡kj iff(i-j) mod K = 0 equivalence classes: S = S1, ∪ S2∪ S3 ∪... s.t. a,b ∊ si a≡ b a ∊ si a ∊ sj a ≢ b, i≠j e.g. (0,k,2k,…} {1,k+1, 2k+1,…} …{k-1, 2k-1,…} 140 s = {a1,a2,a3,a4,a5,a6,a5,a6,a7} Fortran: EQUIVALENCE . . . a11≡a12 a13≡a14 . . . { a1 } { a2 } { a3 } { a4 } { a 5 } { a6 } { a7 } a1≡a1 { a1, a2} { a3 } { a4 } { a5 } { a6 } { a7 } a5≡a6 { a1, a2} { a3} { a4} { a5, a6} { a7 } a3≡a5 { a1, a2} { a3, a5, a6} { a4} { a7 } a4≡a7 { a1, a2} { a3,a5, a6} { a4 a7} ai≡aj A = FIND(ai; B = FIND(aj ); MERGE (A,B,A); MAKENULL(B); ∪ ={ a1, a2, …., an } =A1∪A2 ∪…∪=Am A Partition ADT MFSET { A1, A2,… Am } component 1. MERGE(A,B): A:=A∪B or B:=A∪B 2. FIND(X) 3. INITIAL(A,x): A:={x} 141 A simple implementation element-based Type MFSET = array[membertype]of set-id-type ∪ = {1, 2, …, 12} 1 2 3 4 5 6 7 8 9 10 11 12 2 1 1 2 3 2 4 1 4 3 4 3 = {(2, 3, 6}, {1, 4, 9}} {5, 8,12}, {7, 10, 11}} type set-id-type = integer membertype = 1…n function FIND(x:1..n; var C:MFSET); Begin O(1) FIND := c(x) End Procedure MERGE (A, B:integer; var C:MFSET); // A: A∪B// Var X:1..n; Begin For x:=1 to n do If C[x] = B then O(n) C[x] :=A End; 142 By some minor improvement N merges can be done in O(nlogn) time using member list). A tree implementation component-based A B 1 1 C 5 7 1 6 1 A = {1, 2, 3, 4} B={5, 6} MERGE (A,B) A C= {7} 1 2 5 3 6 4 Time=0(1) 143 FIND (x) – O(depth) i i *weight rule: if we always merge the smaller tree into the large tree, then depth ≤ log2n Root must conatin weight of the tree. Path compression 1 3 2 5 4 6 7 Find 6 8 1 3 3 4 144 5 6 7 8 With path compression only n consecutive FIND - O(n)time n Intermixed FIND and MERGE - O(nlogn) With both path compression and rule ( *) n intermixed FIND and MERGE - O(na(n)) α(n) = the least m s.t. n ≤ A(m,m) pseudo-Inverse of Ackermann’s ftn. In practice, α (n) ≤ 4 I since 2 A(4,4) = 2 . . 65536 . 2 2 145 Ordered Sets with MEREGE, FIND, SPLIT SLPIT (S, S1, S2, x): S1 := {a| a ∊ S and a < x} S2 : = {a| a ∊ S and a ≥ x} Longest Common Subsequence Problem Sequence = string e.g. abcdaaa A subsequence of a sequence x is obtained by removing zero or more (not necessarily contiguous) character from x e.g. ab, aaaa, ada are subsequences of the above sequence Longest common subsequence of x and y: A longest sequence that is a subsequence of both x and y 146 e.g. x=1214321 y=25134121 21421 is an LCS 21321 is another one Application : UNIX diff DNA analysis, ete. Solutions: x1, x2, …xn |x| = n 1. dynamic programming O(nm) y1,y2, …, ym |y| = m 2. O(plogn) Where p is the size of {(i,j) | ≤ n, 1 ≤i ≤n, 1≤i≤m, and x1 = y1} worst case p = O(mn) In practice p = O(m+n) 147 Key idea Input : A = a1a2…an B = b1b2…bm To find | LCS(A,B) | For j:=1 to m do Find | LCS( a1….ai,bi….bj) | Def. Sk = {i | |LCS(a1…aib1…bj)| = k A= 12345678 1214321 B= 25134121 s1 s2 J s0 s3 s4 s5 s6 s7 1 {1} {2,3,4,5,6,7} ∅ ∅ ∅ ∅ ∅ 2 {1} {2,3,4,5,6,7} ∅ ∅ ∅ ∅ ∅ 3 ∅ {1,2} {3,4,5,6,7} ∅ ∅ 4 ∅ {1,2} {3,4} {5,6,7} ∅ 5 ∅ {1,2} {3} {4, 5,6,7} ∅ 6 ∅ 7 ∅ 148 8 ∅ Def. PLACES (a) = {I| 1 ≤ i≤n, ai = a } All PLACES(a) can be obtained in O(n) time, assuming the alphabet is finite. If not, O(nlogn) e.g. if PLACES(a) = {i1,i2…., ik} i1 > i2 > ….ik PLACES [a] i1 ... i2 iR Hashing Intuitive fact: in iteration j (i.e. when considering bj), new matches happen at PLACES (bj) in A. These matches may have a position from sk to sk +1. Rule: r ∊ sr to sk+1 iff j -1 Move r to sk+1 iff 1. ar = bj (i.e., γ ∊ PLACES(bj)) 2. r-1 ∊ sk a1…ar-1 ar … b1 … bj-1 bj … 149 Procedure LCS: Begin Initialize s0 = [ 1, 2,…n} and Si = 0 for i=1,2,…n; for j:=1 to m do (1) (2) //compute sk’s for postion j // (3) (4) (5) (6) (7) for r in PLACES(bj) k:=FIND(r); if K = find(r -1) then begin SLPIT(sk, sk, s’k , r); MERGES(sk, sk+1, sk+1) End End End; Obs: if FIND, MERGE, SPLIT can be done in 0(logn), then the total time is m 0 ∑ | PLACES(bj).| logn j-1 0 = (p.logn) 150 Data structure for sets S0, S1, …, Sn 2 -3 trees ! ! ! 8 K 9 10 11 12 13 14 FIND ( r) : O(depth) = O(logn) MERGES (S’k, Sk+1, Sk+1): New Sk+1 S’k New Sk+1 Sk+1 Sk+1 S’k Similar to INSERT, repair Takes O(logn) time 151 S’k Sk+1 APLIT ( ) 6 7 8 9 10 11 12 r=9 split at 9 8 6 10 7 12 9 9 6 11 7 8 time = O(logn) 152 10 11 12 Graphs: A Math Model HW401 Hw401 Waterloo Toronto London HW6 QEW QEW Hw403 Hamilton Niagara falls Toronto Minneapolis New York Chicago New Orleans Miami Flight Map (Imaginary) KNOW Bob Mary Mary y Bob friends Alex Mark Alex Mark Sandy Sandy Misc: state transition diagrams 153 Directed Graphs (Digraphs) V1 = {1, 2, 3, 4, 5} E1 = {(1,2), (1,3), (2,3),(3,4),(4,5),(4,1),(5,1), (5,4)} G1 : = (V1, E1) 1 A digraph G = (V, E) 5 2 V: set of verices/nodes 3 4 E: set of arcs/directed edges The arc from vertex v to vertex w: (v,w) v≠w V➙w or Tail head w is adjacent to v |V| = n |E| ≤ n(n-1) = O(n2) A path v1,v2, …vm s.t. the arcs (v1,v2,(v2,v3),…,(vm-1,vm)exist. Length of the path : m-1 The path passes through v2, v3, …,vm-1 The path is simple if all vertices on the path are distinct, except possibly the first and last. 154 (Simple) cycle: a (simple) path of length at least one that begins and ends at the same vertex. e.g. 1, 2, 3, 4, 1, 3 is a path 1, 2, 3, 4, 5 is a simple path 1, 2, 3, 4, 5, 1 is a simple cycle 1, 2, 3, 4, 1, 3, 4 is a simple cycle labelled diagraph a b a b b b abab abbaaaba ... a a When the labels are numbers, the diagraph is also called a network or weighted diagraph. 155 Representation of diagraphs 1. List of edges e.g., (1,2), (1, 3), (2, 3),… 2. Adjacency matrix G = (V,E) V = {1,2, …, n} Adjacency matrix for G is an n x n Boolean matrix A[i,j] = true1 if (i, j) ∊ E = false0 otherwise space: 3. O(n2) even if | E | << o(n2) Adjacency list 2 3 4 1 1 1 2 3 4 5 5 156 . 3 . Space : O(|E|) to decide if i➙j, we need O(n) time ADT DIAGRAPH Single source shortest paths problem: Given G = (V,E) and source vertex 2 15 40 100 6 50 1 20 50 3 10 503 18 40 5 20 4 30 labels (costs) must be ≥ 0 costs (2,1) = +∞ We need to determine the cost of the shortest path from sources to every other vertex n-1 Cost(v1,v2,…vn) = ∑ cost(vi➙vi+1) i=1 e.g. source =1 to 2 min cost 70 157 3 4 5 6 60 40 10 30 Dijkstra’s algorithm Source vertex =1 G = (V,E) Distance V= {1, 2, 3} D(i) = cost of shortest path 1 to i Let S ⊆ V be a set of verticles, Ds (i) = cost of shortest path from 1 to i that only passes Through vertices in S S is called a restriction set 10 1 4 3 2 B(3) = 9 Ds(3) = 10 if S = {1} 5 Fact: Dv(i) = D(i) and D∅ (i) = cost(1➙i) Idea: Let S ⊆ V be some set s.t. 1∊ S Suppose we know Ds(i) for each i ≤ V. Then we can enlarge S as follows: 1. w ∊ V-S and Ds(w) is the minimum 158 2. S= S∪{w} 3. Ds(i): min(Ds(i),Ds(w) + cost (w, i)) i . . . S . . . . Ds(i) . | w . . Ds(w) Ds(x)≤ Ds(w) For any X ∊ S Algorithm Begin D[i]≡ Ds(i) S:= {1}; For i:=1 ton do D[i]:= cost(1,i); For i:= to n-1 do begin Find w in V-S s.t. D[w] is a minimum; S:=S∪{w}; For j:=2 to n do D[j]:=min(D[j], D[w] + cost(w, j)) End 159 . End; Obs. D[j] = D(i) if I ∊ S Ds(i) = D(i) for I ∊ S Thus no need to update D[i] if ∊ S Example 10 2 20 3 10 60 10 30 10 1 6 40 10 4 5 30 50 2 6 3 30 60 = d[3] 1 0 5 10 1 0 5 10 +∞ 40 6 3 2 30 4 3 60 230 60 4 1 0 +∞ 5 40 160 10 4 40 6 +∞ 6 3 2 30 1 1 0 5 10 6 3 2 30 +∞ 4 40 0 5 10 161 +∞ 4 40 Procedure Dijkstra: // C[i,j] = cost(i,j)// 1. 2. begin S := {1} For i :=2 to n do D[j] :=C[1, i]; For i := 2 to n-1 do begin Find a w in V-S such that D[w] is a minimum; S := S ∪{w}; for each vertex V in V-S do D[v] := min(D[v] + C[w,v]) End End; How to recover the Shortest paths? Time = O(n2) Adjacency lists of costs: W V1 C1 162 Priority Queue for V-S time: 0(|Ellogn) All-Pairs Shortest Paths Problem Given: a digraph with nonnegative arc costs Goal: for each pair v, w of vertices find the cost of the shortest path from v to w. Application: construction of shortest flying time table Solution 1: repeat Dijkstra’s algorithm with source = 1 ,2,...,n time: 0(n3) or 0(n | E | logn) Solution 2: Floyd’s algorithm let D(i,j) and Ds(i,j) be as before D(i,j): distance from i to j D distance from i to j under restrictions. 163 Floyd’s Idea: Let sk = {1, 2, …, k}, 0 ≤ k < n Suppose Dsk(i,j) is know for all 1≤ I, j, ≤ n Then, Let sk+1 = {1, 2, …k+1} Dsk+1 (i,j) = min Dsk(i,j) Dsk(I,k+1) + Dsk(k+1, j) For all i≤ j, j≤n Thus, we compute Ds0(i,j) Cost (i,j) Ds1(I,j),…., Dsn(i,j) D(i,j) 164 in the following procedure, A is an nxn matrix A[i,j] = Dsk,((i,j) after k-th iteration procedure Floyd(Var A:arraY[l ..n,1 ..n] of real; C:...); var i,j,k: integer, begin for i := i to n do for j:=1 to n do A[i,j] = C[i,j]; //A = Ds0// for i := 1 to n do A[i,j] :=0; for k := 1 to n do for i := 1 to n do for j:=1 to n do if A[i,k] + A[k,j] <A[i,j] then A[i,j] := A[k,j]+ A[k,j] // A[i,j]:= min(A[i,j], A[i,k] + A[k,j]) // end; time = 0(n3) 165 Recovering the paths Use an rrxn matrix P initially, P[i,j]:=0 1 ≤ 1,1 ≤n In procedure Floyd, insert red line If A[i,k] + A[k,j] <A [i,j]then begin A [i,j] := A[i,k] + A[k,j]; P [i,j] : = k end Meaning: the Shortest path from i to j passes through vertex k Procedure path (i,j:integer); // print a shortest path from I to j // var k: integer; begin k:= p[i,j]; if k ≠ 0 then begin path(i,k); writeln(k); path(k,j); end; // the path is not direct // end; 166 Transitive closure of adjacency matrix Given: digraph G = (V,E) represented by adjacency matrix C Goal: for each pair i,j, whether there exists a path from i to j A[i,j]= 1 true if apath from 0 false other for i toj 1≤i,j≤n A is called the transitive closure of C Solution 1: Use Floyd’s algorithm. Initialize A[i,j] = + ∞ if C[i,j] = 0 A[i,j] =1 other At the end, set A[i,j] =1 if A[i,j] ≠ + ∞ = 0 if .. = + ∞ Solution 2: Simplified Floyd’s algorithms - Warshall’s algorithm in iteration k A[i,j] := A[i,j] or A[i,k] and A[k,j] 167 procedure Warshall(var A:array[1…n, 1…n] of boolean; C:...); var i,j,k: integer; begin for i := 1 to n do for j := 1 to n do A[I, j] := C[i,j]; For k := 1 to n do for j := 1 to n do for j := i to n do If A[i,j] = 0 then A[i,j] = and A[k, i] end; lime = O(n3) A=C + C.C + C.C.C +…. + Cn-1 •‘: boolean multiplication, i.e. and it is known C.C can be done in 0(n2.376) to obtain A, compute c2, c4, c8,… cn-1 log2n time : O(logn *n2.376) < 0(n3) 168 2 1 Undirected graphs G = (V,E) 3 5 4 V={(1,2),(2,3)(3,4),(4,5),(5,1),1,3),(2,5)} (u,v) and (v,u) denotes the same edge (u,v) is incident upon u and v. V1,V2,…Vn is a path if the edges (V1,V2),(V2,V3)…,(Vn-1,Vn) exist. The path V1,V2,…,Vn connects V1 and Vn Definitions for simple path and cycle and the same of length ≥ 3 2 G1 = (V1,E1) is a subgraph of G2 = (V2,E2) if V1 and E1 E2 1 3 If E1 contains all edges (u,v) in E2 Such that u,v V1, G1 is called an induced subgraph of G2 2 1 169 3 Graph G is connected if every pair of G’s vertices is connected by some path Connected component of G: a maximal connected induced subgraph of G G is cyclic if G contains at least one cycle G is acyclic if G doesn’t contain any cycles Free tree: a connected acyclic graph Fact: 1. Every n-node free tree has n-1 edges 3. If we add any edge to a free tree, we get a cycle Claim: If n>1, there must be a vertex with degree (i.e., number of edges incident upon the vertex) =1 170 Proof of claim Let G be a free tree with > 1 node Suppose that G’s nodes all have degree >1 V1 V2 V3 Vi Vi+1 Vi+2 ∴ a cylcle esists. A contradiction!! Proof of (1): true If n = 1 Suppose (1) is true for n = k Let G = (V,E) be a k+1 – node free tree Let u be a vertex of dgree 1 and (u,w) be the only indicent edge G’ = (V-{u}. E-{(u,w)}I a free tree By induction hypothesis, G’ has k-1 edges ∴ G has k edges proof of (2): if no cycle then the graph is still free tree but number of edges = n. contradiction!!! 171 Vj Representation Adjacency matrix: symmetric, i.e. entry i,j = entry j,i Adjacency list: redundancy, i.e. if edge (u,v) exists, then u is on the list for v and v is on the list for u. Minimum-cost spanning tree G= (V,E) is connected. Each edge (u,v) E has a cost C(u,v) (=C(v,u)). A spanning tree of G is a subgraph of G which is a free connecting all vertices in V. The cost of a spanning tree is the sum of the costs of edges in the tree. 11 30 3 8 20 15 20 13 30 172 The MST Property: Let G = (V,E) be a connected graph U ⊆ V is a proper subset of V If (u,v) is an edge of the lowest cost s.t. u ⊆U and v V-U, then there is a minimum cost spanning tree that includes (u,v) as an edge. u . .v u . .v’ u v-u C(u,v) ≤ C(u’,v’) procedure Prim (G:graph;var T : set of edges); II Constructs a minimum-cost spanning tree T II var begin U : set of vertices; U,V:vertex; T : = ∅ ; ∪ := {1} while ∪≠ V do being find a lowest cost edge (u,v) s.t. u ∊ U and v ∊ U; T := T ∪ ((u,v)}; ∪ := U ∪ {v} end end; 173 1 An example: 9 5 9 2 7 3 5 1 8 8 4 3 8 6 U = {1} 2 5 3 4 U = {1,2} 1 7 5 8 2 4 3 U = {1,2,5} 1 8 2 7 5 3 3 4 U = {1,2,3,5} 8 1 2 7 5 3 3 174 5 4 Kruskal’s algorithms a connected component w.r.t. T procedure Kruskal (G:graph;var T:set of edges); var u,v : vertex; E’ : set of edges; begin E’ := E; T := ∅ ; while E’ ≠ ∅ do begin find a lowest cost edge (u,v) in E’; E’ := E’ – {(u,v)}; If u and v are not in the same connected component then T:= T∪{(u,v)}; end end; K1 := FIND (u) K2 := FIND (v) if K1, ≠ K2 then MERGE (k1, k2) … α(n) time E’: PRIORITY QUEUE; T: MFSET; Time:O(eloge), e = | E| 175 1 Example: 9 5 8 9 2 7 3 5 8 4 1 3 6 add(3,5) to T 2 5 3 3 4 add (4.5) discard (3,4) 1 5 3 2 5 4 3 Add (2,5) 1 Discard(2,3) 2 7 5 3 3 5 4 Add (1,2) discard (1,3),(1,5) 8 1 2 7 5 3 5 3 4 176 Graph Traversal and Search Digraphs - depth-first search go as far as you can following the arcs! type digraph = array [1 ..n] of adjacency list; vertex = 1..n var V : vertex mark: array (vertex] of (visited, unvisited); 0(e) for v.:= 1 to n do mark [V] := unvisited; for v := 1 to n do if mark [V] = unvisited then dfs (v); procedure dfs(v:vertex); var w: vertex; begin mark[v] := visited; Print v; // anything // for each vertex w on L[v] do I if mark [w] = unvisited then dfs(w) end; 177 3 1 Example 9 2 6 5 8 7 4 10 DFS order : 1, 2, 4, 7, 10, 5, 3, 6, 8, 9 Depth-first spanning forest: Dfnumber 1 7 1 5 2 2 3 6 8 4 7 5 6 4 10 forwars arc : ancestor ➙descendent (3,8) back arc: descendent ➙ ancestor (7, 1) Cross arc: all the other (7, 4) , (9,1) 178 3 tree arc 9 8 10 9 Fact: if (v, w) is a (1) tree/forward arc, dfnumber (v) dfnumber (w); (2) back/cross arc, dfnumber (v) dfnumber(w) An application- test for acyclicity Fact: a digraph is cyclic iff a back arc is encountered in any DFS. v Dfnumber (v) is the Smallest in the cycle . . w V1 V2 How to start a back arc? V3 In dfs, Include a dflNumber for each Node enoutered. Also, keep the current path in array. V4 V5 179 dfl 0 1 2 3 4 Breadth-first search go as broadly as possible procedure bfs(v); var Q: QUEUE of vertex x,y:vertex begin markivi := visited; print v // or anything // MAKENULL(Q) ENQUEUE(v,Q); while not EMPTY(Q) do begin x:= FRQNT(Q); DEQUEUE(Q); for each vertex y adjacent to do time = 0(e) if mark[y] = unvisited then begin mark[y] := visited; ENQUEUE(y,Q) End end; 2 BFS order: 1, 2, 5, 4, 6, 3, 7 1 5 3 bfnumber, bf spanning forest 4 7 6 180 (Undirected) Graphs DFS: very similar to DFS for digraphs. 7 5 2 8 10 4 1 9 3 6 DFS order: 1, 2, 4, 3, 6, 5, 7, 8, 10 , 9 dfs spanning forest 1 if connected df spanning tree 2 7 8 4 10 3 6 dfnumber(v): Tree edge: Back edge: (1,4), (4, 6), (2,5) No cross edges!!! 181 5 9 BFS: For the above graph, the BFS order is: 1,2,3,4,5,6,7,8,9,10 Applications of DFS and BFS: 1. Test for acyclicity acyclic 1ff no back edges 2. Test for connectivity connected 1ff only one tree In the DFS/BFS spanning forest generally, each tree in the forest gives a connected component. 3. Biconnected components (next lecture) 182 Articulation points and biconnected components Flight Map 1 2 3 5 4 8 7 6 Articulation vertex point : if removed the reaming graph becomes disconnected Def. A vertex v is called an articulation point or cutpoint if vertices x,w st. x≠ v, w≠x, x≠ and v is inevery path connected x and w. Def. A connected graph is biconnected if it does not have any articulation points. 183 Fact: The following are equivalent: 1. 0 is biconnected 2. Deletion of any single vertex fails to disconnect 0 3. Every pair of vertices are connected by two disjoint paths (n ≥ 3) Def. A connected graph Is k-connected If deletion of any k-i vertices fails to disconnect the graph Def. A connected graph is k edge-connected if deletion of any k-i edges fails to disconnect the graph Biconnected component (or bicomponent): a maximal Induced biconnected subgraph, e.g. the above graph has 5 bicomponents 2 5 4 5 6 3 5 8 1 5 3 7 184 Problem Given a connected graph G, identify all its articulation points and bicomponents. - Trivial algorithm: 0(n*e). We want 0(e)! To identity the articulation points: Step 1 Do a depth-first search of G. Note: 1. a single df spanning tree 3. only tree and back edges 185 Fact: 1. A leaf cannot be an articulation point 2. The root is an articulation point 1ff it has more than one child 3. Let v be an interior node other than the root. v Is an articulation point iff some subtree of v has no back edge incident upon a proper ancestor of v. Obs. Let w be any proper descendent of v and (w,x) be a back edge. x Is a proper ancestor of v iff dfnumber (x) <dfnumber (v). Def. low(v) = the smallest dfnumber of v or of any node reachable by following a back edge from some descendent of v (including v itself). 186 1 2 dfnumber 6 3 4 Dfnumber 1 2 3 4 5 6 7 8 9 10 11 Low 12115166911 5 7 8 9 10 11 dfnumber (v) Low(v) = min Dfnumber(x) s.t. (v,x) is back edge from any x Low(y) for any child y of v Step 2 Traverse the df spanning tree in postorder and compute low(v) for all nodes v. Note : if v is a leaf Low(v) = min dfnumber (v) Dfnumber(x) s.t. (v,x) is back edge Step 3 Identify articulation points by traversing the tree in postorder. (This step can be In parallel with Step 2.) 187 v is an articulation point i for some child w of v low (w) ≥ dfnumber(v) Step 4 In Step 3, whenever an articulation point v Is found, delete the subtree rooted at w and output the bicomponent given by the subtree and V. A B E D C F G H A Df spanninf tree And back edges 2 B D F Time : O(e) Matching in Graphs 188 1 3 4 C 5 E 6 G 7 8 H Teachers course 6 1 7 2 8 3 9 4 5 G = (V, E) is a graph A matching in g is a set of edges with no two edges incident upon same vertex A matching is maximal if the number of its edges in the maximum. A matching is complete/ perfect if every vertex in V is an endpoint of some edge in the matching. G is bipartite if V = V1 ∪ V2, V1 ∩V2 = ∅ each edge in E had one end in V1 and the other end in V2 . Problem: Given a bipartite G, find a maximal matching in G. Solution #1: (Brute force) Enumereate all possible matchings. Pick one that the largest number of edges. Time: O(n!) = O(nn) Solution #2: Augmenting paths 189 Time: O(ne) M = {(2,7),3(3,6),(4,9)} Let M be a matching A vertex V is matched if it is an endpoint if an edge in M, e.g. 2,3,4,6,7,9 Are matched An augmenting path relative to M: a path connecting two unmatched vertices in which alternate edges in the path are in M. e.g. 1 6 3 5 9 4 10 190 P2 10 P1 Fact: if P is an augmenting path relative to M, then M ⊗ P is a bigger matching. e.g. M ⊗ P {(2,7), (1,6), (3,9), (4,10)} M ⊗ P = {(3,6), (2,7), (4,9), (5,10)} (⊗ is also the Exclusive —Or on sets, i.e. A ⊗ B (A-B) ∪ (B-A) symmetric difference) Fact: M is maximal iff there is no augmenting path relative to M. Proof: “only if”: straight forward “if”: i.e., if M is not maximal then there must be an augmenting path. Let N be a matching s.t. |N| >| M|. Then each connected component of (V,N ⊗ M) must be one of the following: equal 1. a simple cycle with edges alternating between N and M 1more from M 2. an augmenting path relative to N 1 more from N 3. an augmenting path relative to M 4. a path with equal number of edges from N and M Since N ⊗ M has more edges from N than M... 191 Algorithm M:= ⊗ Repeat Find an augmenting path P relative to M; M:= M ⊗ P Until no more augmenting path exists 1 6 2 7 3 8 4 9 5 10 192 M ⊗ P 1 {(1,6)} 3 {(3,6),(1,8)} 2 {(2,8),(1,6), (3,9)} {(2,8),(1,6),(3,9),(4,7)} 6 6 8 4 7 5 7 1 8 1 6 4 3 9 10 Algorithm to find an augmenting path relative to matching M //G=(V E) V=V1∪ V2 // Build an augmenting graph level by level as follows: level 0 := repeat unmatched vertices In V1 level 2i+1 := new vertices that are adjacent to a vertex at through an edge not in M; also add the edge; level 2i +2:= level 2i+2:= new vertices that are adjacent to a vertex at level 21+1 through an edge in M; also add the edge; Stop when an unmatched vertex is added at an odd level or no more vertices can be added (i.e. no augmenting path exists) The path from the vertex to any vertex at level 0 is an augmenting path. 193 Example V1 V2 1 6 7 2 8 3 9 4 5 10 11 12 Level 0 2 5 1 7 8 9 1 3 4 2 3 6 194 The process is very similar to BFS Time: O(e) if adjacency lists are used Internal Sorting Internal: data are stored in the main memory which is a RAM. Thus, access to each data item takes constant time. Data Item: a record with one or more fields. One field contains the key of the record. ‘≤’ linear-ordering on keys (compare ‘<) Sorting: arrange a sequence of records so that the keys form a nondecreasing sequence., r1,r2,…rn ri1-,ri2,…rin s.t. ri1 .key ≤ri2.key ≤…≤ rin .key 195 Bubble Sort Move the Hghter records to the top for i := 1 to no do for j:=n down to i+1 do If A[j].key < a[j-1].key then swap (A[j], A[j-1]) In place Time: 0(n2) Bad sequence: descending Insertion Sort Insert A[i] into A[1], A[2], ..., A[i-1] at its rightful position A[0].key := - ∞ For i:= 2 to n do begin j:= I; while A[j] A[j-1] to begin swap (A[j], A[j-1]); inplace j:=j-1 end end; Time: 0(n2) at descending sequence 196 Selection Sort Select the smaHest record and place it at its tightful position. for i := i to n-i do select the smallest among A[i],…A[n] swap it with A[i] Time: 0(n2) In place Better than Bubble when reconi is large Shell Sort (diminishing-increment) Incr = 6 Incr =3 Time: O(n3/2) in place for some incr sequence 197 Heap Sort Q: PRIORITY QUEUE for i:=1 to n do INSERT (A[i],Q); for i := n down to 1 do A[i] := DELETEMIN(Q); Time: O(nlogn) in place if Q is implemented using array A[l ..n] Details in [AHU] Quick Sort If A[i..j] contains two distinct keys then find the larger of the first two distinct keys, v (called pivot); arrange A[i..j] so that k, i+1 ≤j; A[[],…., A[k] < v and A[k+1],…,A[j] ≤; quicksort (i,k-1); quicksort (k,j) 198 A[i],…A[k] < v, and A[k+1],…,A[j] ≥ v Example 5 7 2 1 4 3 9 5 1 7 Partition v = 7 5 5 1 1 2 2 1 1 4 4 3 3 5 Partition V =5 3 3 1 2 1 1 2 1 4 4 5 3 1 2 1 2 3 4 3 4 V= 2 1 1 1 3 2 4 9 7 7 5 5 4 3 7 5 v =4 1 2 9 7 7 partition v = 9 Partition v = 3 1 5 199 7 7 7 9 9 Worst time complexity: 0(n2) pivot (i,j) - 0(j-i+1) partition (i,j,pivot) – O(j-i+1) T(n) = 0(n) + T(n-1) =... = 0(n not in place! stack space can be made to O(logn) Average time complexity Assumptions: 1. all orderings are equally likely 2. the keys are distinct Pr(lst group s of size i) = Pr(A[1] is the i ÷1 St smallest and A[1] Is the pivot) + Pr(A[2] Is the i+1 st smallest and A[2] is the pivot) = 1/n i/n-1 + 1/n i/n-1 = 2i/n(n-1) 200 Tavg(1) = C0 n-1 Tavg(n) ∑ ≤ 2i I =1 [Tavg(i) + tavg(n-1)] +cn n(n-1) n-1 ≤2 n-1 ∑ [Tavg(i) +t cn I =1 Suppose tavg(i) ≤ k I logi for some constant k 2 ≤ I < n n-1 Tavg(n) ≤ 2 n-1 ∑ k i logi +cn I =1 n/2 = 2k n-1 ∑ n-1 i logi ∑ + i =1 2k ∑ + cn i=n/2+1 n/2 = I logn n-1 i (logn-1) + ∑ 201 i logn + cn n-1 i =1 i=n/2+1 ≤ knlogn – kn/4 – kn/2(n-1) +1 ≤ knlogn, if k is large enough Tavg(n) = 0(nlogn) 2-way merge sort • divide-and-conquer • can be used for external sorting • can be generalized to rn-way Algorithm Msort (A[l ..n]); if n> 1 then begin m:=[n/2]; Msort (A[1...m]); Msort (A[m+1..n]); Merge (A[1...m], A[m+l …n], B[1...n]); A[1...n] := B[1...n]; end; Let k=2[logn] (i.e. k is the smallest power of 2 that is greater than or equal to n) T(n) ≤ T(k) = T(k/2) + T (k/2) + ck = 2 T(k(2) + ck =4 T(k14)+ck+2 ck/2 if Merge takes 0(n) =8 T(k/8) + 3ck … = ck log2 k + k O(1) = O(nlogn) 202 The nonrecursive version Logn Recursive version logn NOTE : Merging order may be different in nonrecursive version. Type afile: array[1…max] of elementtype; 203 Merging two sorted lists Ι M m+1 i ι n j n k procedure merge (var X,Z :afile; l, m, n : integer); //Merge X[l…m] and X[m +1..n] into Z[l..n] // var i,j,k: integer; begin i:=1; j := m+1; k:=l while I ≤ m and j ≤ n do begin if x[i].key ≤ x[j].key then begin Z[k]:=x[i]; i:= i+1 End Else begin Z[k] :=X[j]; j:=j+1 End K := K +1 End; If i > m then 204 Union of ordered sets represented by sorted lists! Z[k…n] : = x[j…n] // move the reaming items// Else Z[k…n] : = x[j…m] End; Time: O(n-1) l Procedure onepass (var X,Y:afile; n;l:integer); //this procedure performs one pass of the merege sort. It merges adjacent pairs of segments of length l from list X to list Y. n= |X| // var i : integer; begin i =1; While i ≤ n -2l-1 do begin Merge (X, Y, i,l-1, i+2l -1); i:= I +2l; end; // merge remaining segments of length < 2 I // if (i+l-1) < n then Merge (X,Y,i+l-1,n) Else Y[i…n] := X[i…n] 205 End; Time : O(n) procedure Msort(var X:afile; n:integer); var l:integer Y:afile; Begin //l is the size of the segments currently being merged // l :=1; while l < n do begin Onepass(X,Y,n,I); l := 2*1; Onepass(Y,X,n,l); l =2*l; end; end; At most [1og2n] +1 passes. Each pass takes 0(n) time. Total: O(nlogn) 206 Example 3 5 6 4 5 93 7 2 8 4 6 1 X 1=1 3 5 4 6 5 9 3 7 2 8 4 6 1Y 1=2 3 4 4 5 5 6 3 5 7 9 2 4 6 8 1 X 1=4 3 3 5 6 7 9 1 2 4 6 8 Y 1 =8 1 2 3 3 4 4 5 207 5 6 6 7 8 9 Obs: The list X and Y are scanned sequentially from left to right [log2n] + 1 times Bin Sorting Is Ω(nlogn) the lower bound for sorting a elements? Yes, if we don’t make any assumption about the keytype and only use comparisons such as key l ≤ key2. What if we know 1 ≤ key ≤ n and the a elements have distinct keys? To sort such n elements, for i := 1 to n do B[A[i].key] := a[i]; 0(n) or for i:= to n do while A[i].key ≠ i do swap (A[i],A[A[i].key]); 0(n) 208 Example Sorting records that have a small number of distinct keys: n records, O(logn) distinct keys Can we do better than O(nlogn)? An algorithm using modified 2 - 3 tree: (AVL also okay) 5 6 4 2 7 4 9 5 6 209 7 11 9 11 . . . size of tree: O(logn) each insert: O(loglogn) Total time: O(nloglogn) Bin Sorting Key = 1..m (any finite and discrete type) 1 . 2 . . . m Bin table B Procedure binsort; Var i:= integer, v: keytype; begin for i = 1 to n do O(n) INSERT(A[i], END(B[A[i].key]), B[A[i],key]); 210 O(m) End; For v : = 2 to m do CONCAT(B[1],B[v]) Bin sorting when m = nk for some k Example k=2 keytype=0...n2 -1 Step 1: Place each integer i into bin i mod n Append Ito the end of the list for bin i mod n. Step 2: Concatenate the lists. Step 3: Place each integer i into bin [i/n]. Step 4: Concatenate the lists. Each step -- 0(n) Total time — 0(n) 211 n=10 Given: 45, 36, 21, 64, 60, 33, 12, 27, 30, 25 ImodlO= BIN CONTENTS 0 1 2 3 4 5 6 7 8 9 60,30 21 12 33 64 45,25 36 27 New list: 60, 30, 21, 12, 33, 64, 45, 25, 36, 27 BIN LW10J= 0 1 2 3 4 5 6 7 8 9 CONTENTS 12 21, 25, 27 30,33,36 45 60,64 212 Radix Sort type keytype = record f1: t1; f2:t2; . . finite, discrete fk: tk end; keyl = (a1, a2,…ak) key2 = (b1, b2,…,bk) keyl <key2 iff 1. a1 <b1, or 2. a1 = b1 and a2< b2, or k. a1 = b1 ,...,ak-l = bk-1, and ak < bk i.e. i, 0 ≤ i <k s.t. aj = bj i ≤j ≤i and ai+1 < bi+1 e.g. abc <aca (called lexicographic order) 213 Var Bi:array[ti] of linked listtype; Procedure radixsort; // binsort list A, first on fk, concatenate the bins in Bk, binsort of fk-1,and so on // begin for i := k down to 1 do begin for each value V of type ti do make Bi[v] empty; for each record r on list A do move r from A on to the end of bin Bi[r.fi]; // binssort on fi // foe each value V of type ti, from lowest to highest do concatenate Bi[v] onto the end of A end end; k k i=1 i =1 Time : o(|ti| +n) = o(kn+ |ti|) 214 Example A = hact, fact, sact, camp, duck, kuck, codd, less, more D E K P S T codd more sack, duck, kuck camp less hact, fact C D M V S sack, duck, kuck, hact, fact codd camp more less A E O U sack, hact, fact, camp less codd, more duck, kuck C D F H K L M S camp, codd duck fact hact kuck less more sack 215 Odd-even merge sort (Useful when you have a parallel computer) Algorithm Odd-even-merge-sort (a0,a1,…a2n-1) 1. Split the list a0,a1,…a2n-1 into two lists a0,a1,…an-1 and a0,an+1,…a2n-1 2. Odd-even-merge-sort (a0,a1,…an-1) 3. Odd-even-merge-sort (an,an+1,…a2n-1) 4. Odd-even-merge (a0,a1,…an-1, an,an+1,…a2n-1) Algorithm Odd-even-merge (a0,a1,…an-1, b0,b1,…bn-1) 1. c0,c1…cn-1 := Odd-even-merge (a0,a2,…an-2, b0,b2,…bn-2) 2. d0d1…dn-1 := Odd-even-merge (a1,a3,…an-1, b1,b3,…bn-1) 3. For all i > 0, compare c1 and di-1 and interchange i necessary 4. Return; c0 c1 d0 c2 d1 c3 d2 … cn-1 dn-2 dn-1 216 Example Odd-even-merge (4,5,8,11,20,25; 2,9,10,27,30,31) Odd-even-merge returns (4,8,20;2,10,30) 2,4,8,10,20,30 Odd-even-merge returns (5,11 ,25;9,27,31) 5,9,11,28,27,31 c: 2 4 8 10 20 30 d: 5 9 11 25 27 31 245 89 10 11 20 25 27 30 31 Sequential time complexity: Odd-even-merge – T1(n) T1(n) = 2T1(n/2) + cn . = O(ntogn) Odd-even-merge-sort – T2(n) T2(n) = 2T2(n/2) + c1nlogn . = O(nlog2n) In parallel, Odd-even-merge - O(logn) time 217 ….sort - O(log2n) Odd-even-mereg (a0a1a2a3, a4 a5 a6 a7) a0 a1 a2 a3 , a4 a5 a6 a7 a0 a2 a4 a6 , a1 a3 a5 a7 a0 a4 a2 a6 , a1 a5 a3 a7 b0 b1 b2 b3, b4 b5 b6 b7 c0 c1 c2 c3 , c4 c5 c6 c7 c0 c1 c4 c2 , c5 c3 c6 c7 218 d0 d1 d2 d3, d4 d5 d6 d7 Lower bound for sorting Defintion: Let B be a problem and f(n) a function. B requires Ω (f(n)) time if every algorithm for B has time complexity Ω (f(n)) (i.e., the running time is at least f(n) In the worst case for inputs of length n). f(n) is a time lower bound for B. Theorem : Sorting by comparisons requires Ω (nlogn) time. (in fact, Ω (nlogn) comparisons) Assumption: Only operations on keys are comparison of key values. Key 1 < key 2 ? no yes Without loss of generality, assume the keys are distinct. decision trees 219 Let P be any sorting algorithm. Denote the input by: A[1..n] : a1,a2,…,an Define a binary tree as follows A[i1,] < a[j1]? yes no A[i2,] < a[j2]? y A[i3,] < a[j3]? no y An outcome i.e. A sort list ar1,ar2,….,am 220 no called the decision tree for P on size n Decision tree for bubble sort with n =3 1 2 3 A For I : = 1 to 2 do For j :=3 down to I +1 do If A[j]< A[j-1] then swap (A[j],A[j-1]) A[1..3] = a b c Abc A[3] < a[2] ? y y Abc Abc A[2] < a[1] ? y A[2] < a[3] ? n y y Abc Abc Abc Abc A[3] < a[2] ? A[3] < a[2] ? A[3] < a[2] ? A[3] < a[2] ? y cba n cab n acb y bca 221 n bac n abc Fact: For any sorting algorithm A, the decision tree for a must have at least n! leaves. Proof There are ni outcomes when A sorts n elements. Fact The depth of the decision tree must be at least log Proof Let depth = d n! ≤ 2 d ≥[1og2(n!)] Corollary A requires at least 1og2(n!) comparisons in the worst case. . n! = (n/e) e = 2.71 83 1og2(n!) = n1og2(n/e) 222 n1og2(n/e) = Ω (nlogn) Sorting requires c compansors. Sorting requires Ω (nlogn) time. Average Time complexity for sorting = avg depth of leaves in decision tree Claim: Among the n! leaves, at least half of them have depth n1og 2 (n!). Proof: The maximum number of leaves with depth ≤ n1og2(n!)-1 is n1og2(n!/2) 2 n1og2(n!/2) =2 = n!/2 on average, sorting requires Ω 1og2(n!) 2 = Ω(nlogn) time 223 Problem Given a1,a2,…,an, s.t. a1 < a2 <…< an and x Find i s.t. a1 =x Binary search: O(logn) time Fact searching requires Ω (logn) time Proof any of a1,a2,…,an could be x there are at least n + 1 outcomes when we search x in a1,…,an a decision tree must have depth log2(n+1) Given a1,a2,…,an find the smallest element Problem ai < aj ai < aj ai < aj yes n-1 elements must all have lost some comparison! 224 no External sorting Assumption : The number of data items to be sorted is too large. The data items (records) are sorted on external storage devices in the form of (sequential) files. External storage device: Magnetic tape read/write head Operation: Read (B) Write (B) f.forward rewind ... BLOCK i BLOCK i +1 225 . .. inter-block gap Magnetic Disk A track R/W head A sector ( block) To access a sector: 1. locate the current track by shifting the R/W head 2. wait until the correct sector arrives the time needed to access a block: seek time + actual R/W time >>> main memory access time file : a sequence of block a fixed number of records 226 in external sorting, the dominating factor is the number of block accesses it is desirable to scan a file beginning to end The model File1 file2 file3 CPU . . . . MAIN MEMORY . . . . . . . . C BLOCKs Objective: sorting with minimum number of passes through the file (thus, minimum number of block accesses) 227 Bubble, insertion, …., Quick, heap sorts: require at least O(n) passes. 2-way merge sort: only require [log2n] passes! ASSUME THAT FILES ARE STORED ON DISKS. THUS, SEEK TIME IS THE “SAME” FOR ALL BLOCKS. EXAMPLE Sort file F = A1,A2,…A2100 A block = 100 records Working main memory space = 3 blocks ( used as buffers) 228 Step 1: internally sort three blocks (300 records at time. Store the resulting file on disk. Run1 run2 1-300 Step 2: 301-600 run3 run4 run5 run6 run7 601-900 901-1200 1201-1500 1501-1800 1801-2100 partition the main memory into three blocks. Two are used as input buffers and the third is used as an output buffer. Merge runs 1 and 2 . Alogrithm Merge (R1,R2) Read a block from R1; Read a block from R2; Merge the records in the input buffers and store the result in the output buffer; If the output buffer gets full, write the contents on to the disk and clear the buffer; If an output buffer gets empty, read a block from the same run. 229 Merge runs 3 and 4, then 5 and 6 , then copy run 7 The result of this is a file of 4 runs. Merge these runs and produce a file of two runs. Merge the two runs to obtain a single run (i.e., a sorted file). F F1 F2 F3 230 Notes: 1. If the number of initial runs is m, then [log2m] passes suffice. 2. if the device is tape, then we need four tapes. Tape 1: Tape 2: Tape 3: Tape 4: Run 1 Run 3 Run 5 Run 7 Run 2 Run 4 Run 6 Run 1 Run 3 Run 2 Run 4 Run 1 Tape 1: Tape 2: Run 2 Run 1 Tape 3: Tape 4: 3. Temp files can be discarded after being used. 231 4. k-way merge 3 way Generally, K-way merge sort requires [logkm] passes = [log2m/log2k] k +1 buffers; more comparison (k-1/record) in each pass for tapes, k-way merge requires 2k tapes. 232 General algorithm design techniques Divide-and-conquer Top-down, recursive e.g. merge sort quick sort Dynamic programming Bottom –up longest common subsequence Greedy Brute force shortest paths minimum-cost spanning tree Back tracking rat-in-maze Divide-and-conquer To solve problem A: If A is small enough Then solve it directly Else Break A into smaller problems A1,A2,..Ak; Solve Ai for each i = 1, 2, …, k; Combine the solutions for A1,..Ak To obtain the solution for A 233 smaller instances Of the same problem Example: Towers of Hanoi A B C Algorithm Move (n,A,B) //move n disk from A to B // if n =1 then move the disk B else begin move (n-1, A, C); move (1,A, B); move (n-1, C, B) end T(n) = C1 2T(n-1) + c2 if n =1 otherwise 234 = O(2n) Example Given n integers, find both the maximum and minimum Algorithm maximum (A[1..n], max, min) If n =1 then Max := min := a[1] Else if n =2 then ifA[1] < A[2] then max : = A[2] min : = A[1] else max : = A[1] min := A[2] else // n > 2// maximin (A[1..n/2], max1, min1); maximin (A[n/2 +1..n], max2, min2); if max1 < max2 then max : = max2 else max : = max1 if min1 ≤ min2 then min : = min1 else min : = min2 235 C (n) = n 2C( 2 ) +2 (comparisons) 1 if n =2 C(n) = 3/2n -2 (by induction) Dynamic programming There are situations where: (i) There is no way to divide a problem into a small number of Subproblems. (ii) The subproblems overlap each other (too much redundancy if d divide-and-conquer is used). (iii) The total number of subproblems to tackle is not large, i.e. polynomial (i.e. nk. Usually k =2,3). 236 Dynamic programming approach: Systematically solve all the subproblems, with the smallest ones first. Keep track of the solutions to the solved subproblems by means of a Table. Solutions to larger subproblems are found by combining solutions to smaller subproblems. Example Longest common subsequence problem (LCS) sequence : x = a1a2…an subsequence of x: a sequence obtained from x by deleting some characters ab cab ca bba b LCS Problem: given x = a1a2…an y = b1b2…bm find the length of an LCS of x and y 237 Previous solution ( using sets); O(plogn) time Where p = the number of paris of positions, One from each sequence, that have The same character In the worst case, p = O(nm) Time : O(nmlogn) Dynamic programming solution Given x = a1a2…an and y = b1b2…bm Define an (n +1) X (m +1) matrix L L[i,j] = the length of an LCS of a1a2…ai and b1b2…bi For all 0 ≤ i ≤ n, 0 ≤ j ≤ m Note : L[0,j] = L[i,0] = 0 0 ≤ i ≤ n, 0 ≤ j ≤ m L[n,m] is the length of the LCS of x and y Each L[i,j] A subproblem 238 L[0,j] = 0, 0≤j≤m L[0,j] = 0, 0≤i≤n L[i-1,j] L[i-j-1] i-1,j -1] +1 if a1 = bj L[i,j] = max 0 otherwise 1 ≤ I ≤ n, 1 ≤ j ≤ m a1… ai-1 ai b1… bj-1 bj 0 1 j -1 j 239 0 0 0 0 0 0 0 0 0 1 i -1 i n solution Algorithm LCS // evaluate matrix L row by row , with row 0 first // 240 for j : = 0 to m do L[0,j] : = 0; For i := 1 to n do L[i,0] : = 0; For i := 1 to n do for j : = 0 to m do if ai =bj then temp : = L(i-1,j-1] +1 else temp :=0 L[i,j] : = max (L(i-1,j], L[i,j-1], temp End; Writeln (L[n,m]) Time = O(nm) Space = O(min(n,m)) Recursive solution (divide – and-conquer) Algorithm LCS (n,m) 241 If n = 0 or m = o then LCS : = 0 Else begin I1:=LCS (n-1,m); I2:=LCS(n,m-1); if an = bm then I3 : LCS (n-1, m-1) +1 Else I3: =0; LCS : = max (I1,I2,I3) End End; T(n,m) = T(n-1,m) + T(n,m-1) +T(n-1,m-1) +C = O(3n+m) Dynamic programming example 2 World Series Odds Problem: Teams A and B play a match. Whoever wins n games first wins the match. 242 Assumption: A and B are equally competent, i.e., each has a 50% chance of winning a particular game. P(i,j): The probability that if A needs i games to win (i.e., A has won n-i games) and B needs j games to win, that A will eventually win the game, 0 ≤ i, j ≤ n. We want to compute P(s,t) for some particular 0 ≤ s.t. ≤ n. P(0,j) = 1 1≤j≤n P(i,0) = 0 1≤i≤n P(i,j) = P(i-1, j) + P (I, j-1) 0 ≤ i, j ≤ n 243 2 0 j -1 j 0 0 0 0 0 1 0 0 i -1 0 1 1 1 1 Order of evaluation: 1. row by row/column by column 2. diagonal Greedy Algorithm Setting : given n objects a1,a2,…an, Each with a weight ( or cost) w(ai) 244 i 0 We want to select a subset of objects a11,a12,…akm, subject to some constraint, such that m ∑ w (aij) j=1 is the minimum Example Coin Changing A1 = {c1,c2,…cn} is a set of distinct coin types c1 > c2 > cn ≥ 1 How do we make up an exact amount using a minimum Total number of coins? If cn =1 then greedy algorithms can be used. Algorithm coinchange (x); i = 1; while x ≠ 0 do begin if c1 ≤ x then begin //selsect the largest coin whose value is ≤ x // writeln(c1); 245 x : = x –c1 else i:= i +1 end e.g. c1 = 25¢ c2 = 10¢ c3 =5¢ c4 = 1¢ x =73¢ change : c1, c1, c2, c2, c4, c4, c4 Notes: 1. The algorithm doesn’t necessarily generate change with minimum total number of coins. e.g., c1 = 5, c2 = 4, c3 = 1 x =8 2. Does so if A1 = { kn-1, kn-2,…k0 } Matching. 246 6 1 7 2 8 3 4 9 5 10 1. Start with m 0 2. Find an augmenting path P relative to M and replace m by MP 3. Repeat (2) until no further augmenting path exists, and then M is maximal. 1 2 3 4 5 6 7 8 9 10 247 P = {1,6} M = 0 {1,6} = {1,6} 2 3 7 8 4 5 9 10 6 1 7 P = {(2,6), (1,6),(1,7)} M = {1,6} P = {(2,6), (1,7)} 3 4 6 7 5 248 P = {(3,7), (1,7),(1,6),(6,2),(2,9)} M = {(2,6),(1,7)} P = = {(3,7),(1,6),(2,9)} 4 5 6 8 9 10 249 1 2 3 3 1 7 3 3 Doesn’t not work 3 P = {(4,9), (2,9), (1,6),(1,8)} M = {(3,7),(1,6),(2,9)} P = {(4,9),(3,2),(1,8)} 2 6 5 9 10 250 P = {(2,9),(4,9),(4,10)} M = {(4,9), (1,8),(3,7)} P = = {(2,9),(1,8),(4,10),(3,7)} 5 6 251 P = {(5,6)} M = {(2,9),(1,8), (4,10),(3,7)} P= = {(2,9),(1,8),(4,10),(3,7),(5,6)} 252