Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Pattern recognition wikipedia , lookup
Factorization of polynomials over finite fields wikipedia , lookup
Gene expression programming wikipedia , lookup
Dijkstra's algorithm wikipedia , lookup
Algorithm characterizations wikipedia , lookup
Lattice model (finance) wikipedia , lookup
Computational phylogenetics wikipedia , lookup
History of the Church–Turing thesis wikipedia , lookup
Recursion Code Stack of Stack Frames Tree of Stack Frames Friends and Strong Induction Towers of Hanoi Check List Merge & Quick Sort Simple Recursion on Trees Generalizing the Problem Things not to do Heap Sort & Priority Queues Trees Representing Equations Pretty Print Parsing Recursive Images Divide And Conquer (an approach to faster algorithms) •DIVIDE my instance to the problem into smaller instances to the same problem. •Have a friend (recursively) solve them. Do not worry about it yourself. •GLUE the answers together so as to obtain the answer to your larger instance. Different Representations of Recursive Algorithms Views Pros Code - Implement on Computer Stack of Stack Frames - Run on Computer Tree of Stack Frames - View entire computation Friends & Strong Induction - Worry about one step at a time. Code Representation of an Algorithm MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f Pros and Cons? Code Representation of an Algorithm Pros: Cons: • Runs on computers • Precise and succinct • Perception that being able to code is the only thing needed to get a job. (false) • I am not a computer • I need a higher level of intuition. • Prone to bugs • Language dependent Different Representations of Recursive Algorithms Views Pros Code - Implement on Computer Stack of Stack Frames - Run on Computer Tree of Stack Frames - View entire computation Friends & Strong Induction - Worry about one step at a time. Stack of Stack Frames MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = bd = (a+b)(c+d) = XY = Stack Frame: A particular execution of one routine on one particular input instance. Stack of Stack Frames MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = Stack of Stack Frames MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = bd = (a+b)(c+d) = XY = Stack of Stack Frames MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = ? bd = (a+b)(c+d) = XY = Stack of Stack Frames MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = ? bd = (a+b)(c+d) = XY = X=2 Y=2 XY= Stack of Stack Frames MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = ? bd = (a+b)(c+d) = XY = X=2 Y=2 XY = 4 Stack of Stack Frames MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = (a+b)(c+d) = XY = Stack of Stack Frames MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = ? (a+b)(c+d) = XY = Stack of Stack Frames MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = ? (a+b)(c+d) = XY = X=1 Y=3 XY = 3 Stack of Stack Frames MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = XY = Stack of Stack Frames MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = ? XY = Stack of Stack Frames MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = ? XY = X=3 Y=5 XY = 15 Stack of Stack Frames MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = ? Stack of Stack Frames MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = 483 Stack of Stack Frames MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = 483 bd = (a+b)(c+d) = XY = Stack of Stack Frames MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = Stack of Stack Frames MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = ? bd = (a+b)(c+d) = XY = Stack of Stack Frames MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = ? bd = (a+b)(c+d) = XY = X=3 Y=1 XY = 3 Stack of Stack Frames MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = 3 bd = ? (a+b)(c+d) = XY = Stack of Stack Frames MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = 3 bd = ? (a+b)(c+d) = XY = X=3 Y=2 XY = 6 Stack of Stack Frames MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = ? XY = An so on …. Stack of Stack Frames Representation of an Algorithm Pros: Cons: • Trace what actually occurs in the computer • Concrete. • It is what students attempt to describe when told not to give code. • Described in words it is impossible to follow who is doing what. • Does not explain why it works. • Demonstrates for only one of many inputs. Different Representations of Recursive Algorithms Views Pros Code - Implement on Computer Stack of Stack Frames - Run on Computer Tree of Stack Frames - View entire computation Friends & Strong Induction - Worry about one step at a time. Tree of Stack Frames MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = 483 X=2 Y=2 XY=4 X=1 Y=3 XY=3 X=3 Y=5 XY=15 X = 2133 Y = 2312 ac = 483 bd = 396 (a+b)(c+d) = 1890 XY = 4931496 X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = 396 X=3 Y=1 XY=3 X=3 Y=2 XY=6 X = 54 Y = 35 ac = 15 bd = 20 (a+b)(c+d) = 72 XY = 1890 X=6 Y=3 XY=18 X=5 Y=3 XY=15 X=4 Y=5 XY=20 X=9 Y=8 XY=72 Stack of Stack Frames Representation of an Algorithm Pros: • View the entire computation. • Good for computing the running time. Cons: • Must describe entire tree. • For each stack frame • input instance • computation • solution returned. • who calls who • Structure of tree may be complex. Different Representations of Recursive Algorithms Views Pros Code - Implement on Computer Stack of Stack Frames - Run on Computer Tree of Stack Frames - View entire computation Friends & Strong Induction - Worry about one step at a time. Friends & Strong Induction MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = 483 X=2 Y=2 XY=4 X=1 Y=3 XY=3 X=3 Y=5 XY=15 X = 2133 Y = 2312 ac = 483 bd = 396 (a+b)(c+d) = 1890 XY = 4931496 X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = 396 X=3 Y=1 XY=3 X=3 Y=2 XY=6 One Friend for each stack frame. Each worries only about his job. X = 54 Y = 35 ac = 15 bd = 20 (a+b)(c+d) = 72 XY = 1890 X=6 Y=3 XY=18 X=5 Y=3 XY=15 X=4 Y=5 XY=20 X=9 Y=8 XY=72 Friends & Strong Induction MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = 483 X=2 Y=2 XY=4 X=1 Y=3 XY=3 X=3 Y=5 XY=15 X = 2133 Y = 2312 ac = 483 bd = 396 (a+b)(c+d) = 1890 XY = 4931496 X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = 396 X=3 Y=1 XY=3 X=3 Y=2 XY=6 Worry about one step at a time. Imagine that you are one specific friend. X = 54 Y = 35 ac = 15 bd = 20 (a+b)(c+d) = 72 XY = 1890 X=6 Y=3 XY=18 X=5 Y=3 XY=15 X=4 Y=5 XY=20 X=9 Y=8 XY=72 Friends & Strong Induction MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 33 Y = 12 ac = bd = (a+b)(c+d) = XY = •Consider your input instance Friends & Strong Induction MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 33 Y = 12 ac = ? bd = ? (a+b)(c+d) = XY = X=3 Y=1 XY = ? X=3 Y=2 XY = ? X=6 Y=3 XY = ? •Consider your input instance •Allocate work •Construct one or more sub-instances Friends & Strong Induction MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = X=3 Y=1 XY = 3 X=3 Y=2 XY = 6 •Consider your input instance •Allocate work •Construct one or more sub-instances +f •Assume by magic your friends give you the answer for these. X=6 Y=3 XY = 18 Friends & Strong Induction MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = 396 X=3 Y=1 XY = 3 X=3 Y=2 XY = 6 •Consider your input instance •Allocate work •Construct one or more sub-instances +f •Assume by magic your friends give you the answer for these. •Use this help to solve your own instance. X=6 Y=3 XY = 18 Friends & Strong Induction MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = 396 X=3 Y=1 XY = 3 X=3 Y=2 XY = 6 •Consider your input instance •Allocate work •Construct one or more sub-instances •Assume by magic your friends give you the answer for these. •Use this help to solve your own instance •Do not worry about anything else. •Who your boss is… •How your friends solve their instance… X=6 Y=3 XY = 18 Friends & Strong Induction MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = 396 X=3 Y=1 XY = 3 X=3 Y=2 XY = 6 X=6 Y=3 XY = 18 This technique is often referred to as Divide and Conquer Friends & Strong Induction MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Consider generic instances. Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f MULT(X,Y): MULT(X,Y): Break X into a,b and Y into c,d If |X| = |Y| = 1 then RETURN XY e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f ac bd (a+b)(c+d) Remember! Do not worry about •Who your boss is. •How your friends solve their instance. Friends & Strong Induction MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = 483 X=2 Y=2 XY=4 X=1 Y=3 XY=3 X=3 Y=5 XY=15 X = 2133 Y = 2312 ac = 483 bd = 396 (a+b)(c+d) = 1890 XY = 4931496 X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = 396 X=3 Y=1 XY=3 X=3 Y=2 XY=6 This solves the problem for every possible instance. X = 54 Y = 35 ac = 15 bd = 20 (a+b)(c+d) = 72 XY = 1890 X=6 Y=3 XY=18 X=5 Y=3 XY=15 X=4 Y=5 XY=20 X=9 Y=8 XY=72 Friends & Strong Induction Recursive Algorithm: •Assume you have an algorithm that works. •Use it to write an algorithm that works. Friends & Strong Induction Recursive Algorithm: •Assume you have an algorithm that works. •Use it to write an algorithm that works. If I could get in, I could get the key. Then I could unlock the door so that I can get in. Circular Argument! Friends & Strong Induction Recursive Algorithm: •Assume you have an algorithm that works. •Use it to write an algorithm that works. To get into my house I must get the key from a smaller house Friends & Strong Induction MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f X = 33 Y = 12 ac = ? bd = ? (a+b)(c+d) = XY = X=3 Y=1 XY = ? X=3 Y=2 XY = ? •Allocate work •Construct one or more sub-instances Each sub-instance must be a smaller instance to the same problem. X=6 Y=3 XY = ? Friends & Strong Induction Recursive Algorithm: •Assume you have an algorithm that works. •Use it to write an algorithm that works. Use brute force to get into the smallest house. Friends & Strong Induction MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a,b and Y into c,d e = MULT(a,c) and f =MULT(b,d) RETURN e 10n + (MULT(a+b, c+d) – e - f) 10n/2 + f MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Use brute force to solve the base case instances. Friends & Strong Induction Carefully write the specifications for the problem. <preCond>: Set of legal instances (inputs) <postCond>: Required output Why? Friends & Strong Induction Carefully write the specifications for the problem. <preCond>: Set of legal instances (inputs) •To be sure that we solve the problem for every legal instance. •So that we know –what we can give to a friend. <postCond>: Required output •So that we know –what is expected of us. –what we can expect from our friend. Related to Loop Invariants Induction nS ( n ) iS ( i ) S ( i 1) S ( 0) Strong Induction nS ( n ) i,[S ( 0), S (1), S ( 2),..., S ( i 1) S ( i )] S ( 0) Friends & Strong Induction •Induction Hypothesis: S ( n) ``The recursive algorithm works for every instance of size n.'' Base case S ( 0) size i nS ( n ) i,[S ( 0), S (1), S ( 2),..., S ( i 1) S ( i )] ``The algorithm works for every instance of any size.'' Friends & Strong Induction Representation of an Algorithm Pros: Cons: • Worry about one step • You expect too much from your at a time. friends. Each sub-instance must be • An abstraction within a smaller instance which to develop, think to the same problem. about, and describe algorithms in such way • You expect too much from your that their correctness is transparent. boss. You only know your instance. You do not know your boss’s instance. Output Try explaining what this algorithm does by tracing it out. It is much easier to TAKE ONE STEP AT A TIME Output n=1 X n=2 Y Output n=1 X n=2 Y n=3 A ? B ? C Output n=1 X n=2 Y n=3 A Y B X C Output n=1 X n=2 Y n=3 AYBXC Output n=1 X n=2 Y n=3 AYBXC n=4 A ? B ? C Output n=1 X n=2 Y n=3 AYBXC n=4 A AYBXC B Y C Output n=1 X n=2 Y n=3 AYBXC n=4 AAYBXCBYC Output n=1 X n=2 Y n=3 AYBXC n=4 AAYBXCBYC n=5 A ? B ? C Output n=1 X n=2 Y n=3 AYBXC n=4 AAYBXCBYC n=5 AAAYBXCBYCBAYBXCC Output n=1 X n=2 Y n=3 AYBXC n=4 AAYBXCBYC n=5 AAAYBXCBYCBAYBXCC T(2) = 1 Time: T(1) = 1 T(n) = T(n-1) + T(n-2) + 3 2T(n-1) +3 = 2Q(n) defined? Towers of Hanoi This job of mine is a bit daunting. Where do I start? And I am lazy. Towers of Hanoi At some point, the biggest disk moves. I will do that job. Towers of Hanoi To do this, the other disks must be in the middle. Towers of Hanoi How will these move? I will get a friend to do it. And another to move these. I only move the big disk. Towers of Hanoi Towers of Hanoi Get a job at the Towers of Hanoi company at level n=1,2,3,4,….. Or get transferred in as the president & you decide what your vice president does. Towers of Hanoi Time: T(1) = 1, T(n) = 1 + 2T(n-1) ≈ 2T(n-1) ≈ 2(2T(n-2)) ≈ 4T(n-2) ≈ 4(2T(n-3)) ≈ 8T(n-3) ≈ 2i T(n-i) ≈ 2n Evaluating: T(n) = aT(n-b)+f(n) Level Instance size Work in stack frame n f(n) # stack frames Work in Level 1 · f(n) a2 · f(n-2b) ai · f(n-ib) 0 1 2 n-b f(n-b) n-2b f(n-2b) 1 a a2 i n-ib f(n-ib) ai hh==n?/b n-hb T(0) n/ b a |base case| = 0 = n-hb Likely dominated by base cases n h = /b a · f(n-b) n/ b a · T(0) Exponential Evaluating: T(n) = 1T(n-b)+f(n) Level Instance size Work in stack frame n f(n) # stack frames Work in Level f(n) f(n-2b) 0 1 2 n-b f(n-b) n-2b f(n-2b) 1 1 1 i n-ib f(n-ib) 1 f(n-ib) hh==n?/b n-hb T(0) 1 f(0) f(n-b) Total Work T(n) = ∑i=0..h f(b·i) = θ(f(n)) or θ(n·f(n)) Evaluating: T(n) = aT(n/b)+f(n) Check Lists for Recursive Programs This is the format of “all” recursive programs. Don’t deviate from this. Or else! Check Lists for Recursive Programs This is input instance is your mission. You must return this output. (An x, y, & z in every computation path.) Focus on your mission. Check Lists for Recursive Programs The <preCond> & <postCond> must document these variables and what you must do. Check Lists for Recursive Programs To get help, you construct an instance for each friend It must be valid and smaller. And pass them to them. Check Lists for Recursive Programs Each friend returns a solution his sub-instance. Trust him to do this. But don’t expect him to do or know anything else. Check Lists for Recursive Programs Combine their solutions to construct a solution to your instance. Return your solution. Check Lists for Recursive Programs If your solution is too small to get solved by the general technique solve it yourself. Be sure you handle every legal instance. Check Lists for Recursive Programs You rarely need to change the values of these variable beyond their initial setting (this is not an iterative algorithm.) Your rarely need additional input, output, or local variable. But it you do document them. Check Lists for Recursive Programs Have NO global variables. If you change the value of a local variable n, neither your friends nor your boss get that value. Sorting Problem Specification • <preCond>: The input is a list of n values with the same value possibly repeated. • <postCond>: The output is a list consisting of the same n values in non-decreasing order. 88 52 14 31 25 98 30 23 62 79 14,23,25,30,31,52,62,79,88,98 Recursive Sorts • Given list of objects to be sorted • Split the list into two sub-lists. • Recursively have a friend sort the two sub-lists. • Combine the two sorted sub-lists into one entirely sorted list. Four Recursive Sorts Size of Sub-lists n/2,n/2 n-1,1 Minimal effort splitting Lots of effort recombining Merge Sort Insertion Sort Lots of effort splitting Minimal effort recombining Quick Sort Selection Sort Merge Sort 88 52 14 31 25 98 30 23 62 79 Divide and Conquer Merge Sort 88 52 14 31 25 98 30 23 62 79 Get one friend to sort the first half. 25,31,52,88,98 Split Set into Two (no real work) Get one friend to sort the second half. 14,23,30,62,79 Merge Sort Merge two sorted lists into one 25,31,52,88,98 14,23,25,30,31,52,62,79,88,98 14,23,30,62,79 Merge Sort Sort Time: T(n) = 2T(n/2) + Q(n) = Q(n log(n)) Total work at any level = Q(n) # levels = Q(log(n)) 2T(n/2) + Q(n) Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame n f(n) # stack frames Work in Level 1 · f(n) a2 · f(n/b2) ai · f(n/bi) 0 1 2 n/b f(n/b) n/b2 f(n/b2) 1 a a2 i n/bi f(n/bi) ai h = log n/log b n/bh T(1) a · f(n/b) 2i·Q(n/2i) = Q(n) log a/ log a/ log b n n log b · T(1) All levels basically the same. Total Work T(n) = ∑i=0..h aif(n/bi) = Q(n) · log(n) Quick Sort 88 52 14 31 25 98 30 23 62 79 Divide and Conquer Quick Sort Partition set into two using randomly chosen pivot 88 52 14 31 25 98 30 23 62 79 14 31 30 23 25 88 ≤ 52 ≤ 98 62 79 Quick Sort 14 31 30 23 25 Get one friend to sort the first half. 14,23,25,30,31 88 ≤ 52 ≤ 98 62 79 Get one friend to sort the second half. 62,79,98,88 Quick Sort 14,23,25,30,31 52 62,79,98,88 Glue pieces together. (No real work) 14,23,25,30,31,52,62,79,88,98 Quick Sort 88 52 14 31 25 98 30 23 62 79 14 30 23 25 Let pivot be the first element in the list? 88 ≤ 31 ≤ 62 98 52 79 Quick Sort 14,23,25,30,31,52,62,79,88,98 ≤ 14 ≤ 23,25,30,31,52,62,79,88,98 If the list is already sorted, then the slit is worst case unbalanced. Quick Sort Best Time: Worst Time: Expected Time: T(n) = 2T(n/2) + Q(n) = Q(n log(n)) Quick Sort Best Time: T(n) = 2T(n/2) + Q(n) = Q(n log(n)) Worst Time: T(n) = T(0) + T(n-1) + Q(n) = Q(n2) Expected Time: Quick Sort Best Time: T(n) = 2T(n/2) + Q(n) = Q(n log(n)) Worst Time: T(n) = T(0) + T(n-1) + Q(n) = Q(n2) Expected Time: T(n) = T(1/3n) + T(2/3n) + Q(n) = Q(n log(n)) Quick Sort Expected Time: T(n) = T(1/3n) + T(2/3n) + Q(n) = Q(n log(n)) Top work Q(n) 2nd level Q(n) 3rd level Q(n) 1/ 3 1/ 1/ 3 3 2/ # levels 3 2/ 3 2/ 3 2/ 3 = log / (n) Q(log(n)) 3 2 Kth Element Problem Specification • <preCond>: The input is a list of n values and an integer k. • <postCond>: The kth smallest in the list. k=3 88 52 14 31 25 98 30 23 62 79 Output: 25 14,23,25,30,31,52,62,79,88,98 Kth Element Problem Specification Partition set into two using randomly chosen pivot 88 52 14 31 25 98 30 23 62 79 14 31 30 23 25 88 ≤ 52 ≤ 98 62 79 Kth Element Problem Specification 14 31 30 23 25 88 ≤ 52 ≤ 98 62 79 k=3 r=5 elements on left Get one friend to find k=3rd element in first half. 14,23,25,30,31 Output: 25 Kth Element Problem Specification 14 31 30 23 25 88 ≤ 52 ≤ 98 62 79 k=8 r=5 elements on left Get one friend to find k=8-5=3rd element in second half. 52,62,79,98,88 Output: 79 Kth Element Problem Specification Best Time: T(n) = 1 T(n/2) + Q(n) = Q(n) log a/ log b Worst Time: Expected Time: # of base cases = n = nlog 1/log 2 = n0 = 1 n +n / 2 +n / 4 +n / 8 … +1 Kth Element Problem Specification Best Time: T(n) = 1 T(n/2) + Q(n) = Q(n) Worst Time: T(n) = 1 T(n-1) + Q(n) Expected Time: ≈ n+T(n-1) ≈ n+(n-1)+T(n-2) ≈ n+(n-1)+(n-2)+T(n-3) ≈ n+(n-1)+(n-2)+(n-3)+…+1 ≈ Q(n2) Kth Element Problem Specification Best Time: T(n) = 1 T(n/2) + Q(n) = Q(n) Worst Time: T(n) = 1 T(n-1) + Q(n) = Q(n2) Expected Time: T(n) = 1 T(2/3n) + Q(n) = Q(n) T(N) = 2T(N/2) + 1 = Q(N) = 2Q(n) Size = log(b) + log(N) N=7 b3 b 7 = b3 × b4 b4 N=3 N=1 N=4 N=2 N=1 N=1 N=2 N=1 N=1 N=2 N=1 N=1 N=7 b3 N=3 N=1 T(N) = 1T(N/2) + 1 = Q(log(N)) = Q(n) Size = log(b) + log(N) b7 = (b3)2 × b Recursion on Trees 3 8 1 3 6 2 2 5 4 7 9 1 Recursion on Trees A binary tree is: - the empty tree - a node with a right and a left sub-tree. tree because 3 8 1 3 6 2 2 5 4 7 9 1 Recursion on Trees A binary tree is: - the empty tree - a node with a right and a left sub-tree. node tree 3 tree 8 1 3 6 2 2 5 4 7 9 1 Recursion on Trees A binary tree is: - the empty tree - a node with a right and a left sub-tree. 3 tree because 8 1 3 6 2 2 5 4 7 9 1 Recursion on Trees A binary tree is: - the empty tree - a node with a right and a left sub-tree. 3 node 8 6 tree tree 1 3 2 2 5 4 7 9 1 Recursion on Trees A binary tree is: - the empty tree - a node with a right and a left sub-tree. 3 8 1 3 tree because 2 2 6 5 4 7 9 1 Recursion on Trees A binary tree is: - the empty tree - a node with a right and a left sub-tree. 3 8 1 3 tree 2 node 2 tree 6 5 4 7 9 1 Recursion on Trees A binary tree is: - the empty tree - a node with a right and a left sub-tree. 3 8 1 3 6 2 2 5 4 7 tree because empty 9 1 Recursion on Trees number of nodes = ? Get help from friends 5 3 6 8 1 3 6 2 2 5 4 7 9 1 Recursion on Trees number of nodes = number on left + number on right + 1 = 6 + 5 + 1 = 12 5 3 6 8 1 3 6 2 2 5 4 7 9 1 Recursion on Trees number of nodes Base Case ? 3 8 1 3 6 1 2 2 5 4 7 Are we done? 9 1 Recursion on Trees number of nodes Base Case ? 3 8 1 3 6 2 5 4 7 2 ? 9 1 Recursion on Trees number of nodes Base Case ? 3 8 1 3 6 2 2 0 5 4 7 Better base case! 9 1 Recursion on Trees number of nodes Base Case ? 3 8 1 3 6 1 2 2 5 4 7 9 1 Does this still need to be a base case? Recursion on Trees number of nodes = number on left + number on right + 1 =0+0+1=1 3 8 6 1 3 2 2 No! 0 5 7 4 0 9 1 Recursion on Trees Most programmers don’t use the empty tree as a base case. This causes a lot more work. 3 8 6 1 3 2 2 0 5 7 4 0 9 1 Recursion on Trees Recursion on Trees Designing Program/Test Cases n 1 + n2 + 1 n1 + 1 Try same code 3 n1 3 n2 generic generic Try same code 0+0+1=1 0 3 0 0 n1 generic Same code works! 0 Same code works! Recursion on Trees Time: T(n) = T(left) + T(right) + Q(1) = Q(n) One stack frame for each node in the tree Recursion on Trees sum of nodes = ? Get help from friends 25 3 23 8 1 3 6 2 2 5 4 7 9 1 Recursion on Trees sum of nodes = sum on left + sum on right + value at root = 23 + 25 + 3 = 51 25 3 23 8 1 3 6 2 2 5 4 7 9 1 Recursion on Trees sum of nodes = ? Base Case ? 3 8 1 3 6 2 2 0 5 4 7 9 1 Recursion on Trees Recursion on Trees Designing Program/Test Cases s 1 + s2 + 3 s1 + 3 3 s1 3 s2 generic generic 0+0+ 3 =3 0 3 0 0 s1 generic 0 Recursion on Trees max of nodes = ? Get help from friends 9 3 8 8 1 3 6 2 2 5 4 7 9 1 Recursion on Trees max of nodes = max( max on left, max on right, value at root) = max( 8, 9, 3) = 9 9 3 8 8 1 3 6 2 2 5 4 7 9 1 Recursion on Trees max of nodes = ? Base Case ? 3 8 1 3 6 2 2 ? 5 4 7 9 1 Recursion on Trees 3+4+2+8+… Sum so far is 17 Sum so far is 0 NewSum = OldSum + nextElement = 0 + 3 = 3 Recursion on Trees 3*4*2*3*… Product so far is 72 Product so far is 1 NewProd = OldProd × nextElement = 1 × 3 = 3 Recursion on Trees True and True and False and … Conclusion so far is True Conclusion so far is True NewProd = OldProd and nextElement = True and True = True Recursion on Trees Max(3,4,2,3,… Max so far is 4 Max so far is - NewMax = max(OldMax, next Element) = max( - , 3 ) = 3 Recursion on Trees Recursion on Trees height of tree = ? Get help from friends 4 3 3 8 1 3 6 2 2 5 4 7 9 1 Recursion on Trees height of tree = max(height of left,height on right) + 1 = max(3,4) + 1 = 5 4 3 3 8 1 3 6 2 2 5 4 7 9 1 Recursion on Trees height of tree Base Case ? 3 8 1 3 6 2 2 ? 5 4 7 9 1 Recursion on Trees height of tree? 2 nodes or 1 edge Base Case ? 3 8 1 3 6 2 2 0 nodes ? edges 5 4 7 9 1 Recursion on Trees Work it out backwards height = 0 edge = max(height of left,height on right) + 1 = max(?,?) + 1 3 8 1 3 ? 2 2 ? 6 ? edges 5 4 7 9 1 Recursion on Trees Work it out backwards height = 0 edge = max(height of left,height on right) + 1 = max(-1,-1) + 1 3 8 1 3 -1 -1 6 2 2 -1 edges 5 4 7 9 1 Recursion on Trees Recursion on Trees number of leaves = ? Get help from friends 2 3 3 8 1 3 6 2 2 5 4 7 9 1 Recursion on Trees number of leaves = number on left + number on right =3+2=5 2 3 3 8 1 3 6 2 2 5 4 7 9 1 Recursion on Trees number of leaves Base Case ? 3 8 1 3 6 2 2 0 5 4 7 9 1 Recursion on Trees L1 + L2 Designing Program/Test Cases L1 3 L1 3 L2 generic generic No! 1 0+0 = 0 0 3 0 0 L1 generic 0 Needs to be another base case. Recursion on Trees Recursion on Trees Recursion on Trees Recursion on Trees ? Drops the tree. Define Problem: Binary Search Tree • <preCond> – Key 25 – A binary search tree. 38 25 51 17 4 31 21 28 42 35 40 63 49 – <postCond> – Find key in BST (if there). 55 71 Binary Search Tree Its left children ≤ Any node ≤ Its right children 38 ≤ 25 ≤ 17 4 51 31 21 28 42 35 40 63 49 55 71 Define Loop Invariant • Maintain a sub-tree. • If the key is contained in the original tree, then the key is contained in the sub-tree. 38 key 17 25 51 17 4 31 21 28 42 35 40 63 49 55 71 Define Step • Cut sub-tree in half. • Determine which half the key would be in. • Keep that half. 38 key 17 25 51 17 4 If key < root, then key is in left half. 31 21 28 42 35 40 63 49 If key = root, If key > root, then key is then key is found in right half. 55 71 Recursion on Trees Recursion on Trees Recursion on Trees Time: T(n) = 2T(n/2) + Q( n) = Q(n log(n)) If tree very unbalanced? T(n) = 1T(n-1) + Q(n) = Q(n2) Computing max is too much work. Recursion on Trees Extra info from below Recursion on Trees Time: T(n) = 2T(n/2) + Q( n) = Q(n log(n)) n 1 Computing max is too much work. Recursion on Trees Extra info from above min min min 38 max root root max max - Things to Remember Things to Do and Things NOT TO DO I am obsessed with the Friends - Strong Induction View of Recursion. •Trust your friends to solve sub-instances. •The sub-instance given must be •smaller and •must be an instance to the same problem. •Combine solution given by friend to construct your own solution for your instance. •Focus on one step. •Do not talk of their friends friends friends. •Solve small instances on your own. Do it! Typical Test Answer Define pre & post conditions Don't have inputs or outputs that are not explained! Typical Test Answer Call recursively on the correct types of input k,num,v Typical Test Answer Call recursively returning the correct types of output Save the results (or don't bother calling) Typical Test Answer Combine solutions given by friends to construct your own solution. Typical Test Answer Return things of the correct types. Not an element the root? In every path through code Typical Test Answer Sub-instances need to be smaller. “Size” is size of the tree Wrong base case Typical Test Answer No Global Variables. What is the value of n here? Still zero. The friend has his own variable n. Typical Test Answer No Global Variables. Maybe they mean to pass the new n in and out. Please don’t use these to pass out values when I am marking. Typical Test Answer Looks like an iterative algorithm Looks like an recursive algorithm Which is it? Hard to write an iterative program to traverse on a recursive structure tree 3 8 1 3 6 2 2 5 4 7 9 1 Need pointers from each node to its parent. Hard to write a recursive program that implements an iterative algorithm. Writing a Recursive Program Call recursively on the correct types of input Writing a Recursive Program Call recursively on sub-instances which will give useful information. ? node in sub-tree 3 8 1 3 6 2 2 5 4 7 9 1 Writing a Recursive Program Call recursively on sub-instances which will give useful information. 3rd node in sub-tree ? node in whole tree 3 8 1 3 6 2 2 5 4 7 9 1 Writing a Recursive Program Call recursively on sub-instances which will give useful information. 3rd node in sub-tree nl +1 +3rd node in whole tree nl 3 8 1 3 6 2 2 5 4 7 9 1 Writing a Recursive Program Call recursively on sub-instances which will give useful information. 3rd node in sub-tree nl +1 +3rd node in whole tree 3 8 1 3 6 2 2 5 4 7 9 1 Writing a Recursive Program But who will count nodes in left sub-tree? nl My friend 3 8 1 3 6 2 2 5 4 7 9 1 Writing a Recursive Program But who will count nodes in left sub-tree? nl My friend 3 8 1 3 6 2 2 5 4 7 9 1 Writing a Recursive Program Call recursively returning the correct types of output Save the results (or don't bother calling) Writing a Recursive Program Combine solutions given by friends to construct your own solution. Writing a Recursive Program Sub-instances need to be smaller. “Size” is # nodes in the tree Writing a Recursive Program Sub-instances need to be smaller. When the instance sufficiently small solve on your own. Writing a Recursive Program Return things of the correct types. Writing a Recursive Program Running Time T(n) = 2T(n/2) + Q(1) = Q(n) Faster? Not possible because must count # nodes in left tree. Writing a Recursive Program Running Time T(n) = 2T(n/2) + Q(n) = Q(n logn) If tree very unbalanced? T(n) = 1T(n-1) + Q(n) = Q(n2) Heaps, Heap Sort, & Priority Queues Heap Definition •Completely Balanced Binary Tree •The value of each node each of the node's children. •Left or right child could be larger. Where can 9 go? Where can 1 go? Where can 8 go? Maximum is at root. Heap Data Structure Completely Balanced Binary Tree Implemented by an Array Make Heap Get help from friends Heapify Where is the maximum? ? Maximum needs to be at root. Heapify Find the maximum. Put it in place ? Repeat Heapify Heap Running Time: Make Heap Get help from friends Heapify Running time: T(n) = 2T(n/2) + log(n) = Q(n) Heap Another algorithm ? Heaps Heap ? Heap ? Heap ? Heap Running Time: 2log(n) -i log(n) -i i Priority Queues Selection Sort Selection Largest i values are sorted on side. Remaining values are off to side. 3 Exit 79 km Exit 75 km 5 1 4 < 6,7,8,9 2 Max is easier to find if a heap. Heap Sort Largest i values are sorted on side. Remaining values are in a heap. Exit 79 km Exit 75 km Heap Data Structure Heap 6 8 9 7 Array 5 3 4 2 1 Heap Array Heap Sort Largest i values are sorted on side. Remaining values are in a heap. Exit 79 km Exit 75 km Put next value where it belongs. ? Heap Heap Sort Heap Sort ? ? ? ? ? ? ? Sorted Heap Sort Running Time: Recursion on Trees Evaluate Equation Tree = ? Get help from friends 12 7 Recursion on Trees Evaluate Equation Tree = rootop( value on left, value on right ) = rootop(12,7) = 12 + 7 = 19 12 7 Recursion on Trees Evaluate Equation Tree Base Case ? 7 Derivatives Input: a function f. Output: Its derivative df/dx. Derivatives Derivatives Input: a function f. Output: Its derivative df/dx. Derivatives Input: a function f. Output: Its derivative df/dx. Output: Its derivative df/dx. Derivatives Input: a function f. g Simplify Input: a function f. Output: f simplified. Simplify Recursion on Trees Printing a Tree When I taught this one year, a student needed just this algorithm for his job. Recursion on Trees Typing line by line? Get help from friends Recursion on Trees One stack frame prints: His input gives: his tree whether left or right sting for each line Recursion on Trees He gives his friends: their trees whether left or right string for each line Parsing Input: s=6*8+((2+42)*(5+12)+987*7*123+15*54) Output: Parsing Parsing Parsing Algorithm: GetExp( s, i ) Input: s is a string of tokens i is a start index Output: p is a parsing of the longest valid expression j is the end index s=6*8+((2+42)*(5+12)+987*7*123+15*54) Parsing Algorithm: GetTerm( s, i ) Input: s is a string of tokens i is a start index Output: p is a parsing of the longest valid term j is the end index s=6*8+((2+42)*(5+12)+987*7*123+15*54) Parsing Algorithm: GetFact( s, i ) Input: s is a string of tokens i is a start index Output: p is a parsing of the longest valid factor j is the end index s=6*8+((2+42)*(5+12)+987*7*123+15*54) Algorithm: GetExp( s, i ) s=6*8+((2+42)*(5+12)+987*7*123+15*54) p<term,1> p<term,2> p<term,3> Algorithm: GetExp( s, i ) Exp p<term,1> + p<term,2> + …+ p<term,k> Algorithm: GetTerm( s, i ) s=6*8+((2+42)*(5+12)+987*7*123+15*54) p<fact,1> p<fact,2> Algorithm: GetTerm( s, i ) Term p<fact,1> * p<fact,2> * …* p<fact,k> Parsing Algorithm: GetFact( s, i ) s=6*8+((2+42)*(5+12)+987*7*123+15*54) Parsing Algorithm: GetFact( s, i ) Fact 42 Algorithm: GetFact( s, i ) s=6*8+((2+42)*(5+12)+987*7*123+15*54) p<exp> Algorithm: GetFact( s, i ) Fact ( p<exp> ) Parsing Stackframes nodes in parse tree Recursive Images if n=0, draw if n=1 n=0 else draw And recursively Draw here with n-1 Recursive Images if n=0, draw if n=2 n=1 else draw And recursively Draw here with n-1 Recursive Images if n=0, draw if n=3 n=2 else draw And recursively Draw here with n-1 Recursive Images if n=0, draw if n=30 else draw And recursively Draw here with n-1 Recursive Images if n=0 if n=5 n=1 n=2 n=3 n=4 Recursive Images if n=0 if n=5 n=1 n=2 n=3 n=4 Recursive Images L(n) = 4/3 L(n-1) = (4/3)n Ackermann’s Function How big is A(5,5)? Ackermann’s Function Ackermann’s Function Ackermann’s Function 3 Ackermann’s Function Ackermann’s Function Ackermann’s Function Ackermann’s Function Ackermann’s Function