* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Introduction to Probability Supplementary Notes 2 Recursion Instructor:
Survey
Document related concepts
Functional decomposition wikipedia , lookup
Mathematics of radio engineering wikipedia , lookup
List of regular polytopes and compounds wikipedia , lookup
Non-standard calculus wikipedia , lookup
Collatz conjecture wikipedia , lookup
Structure (mathematical logic) wikipedia , lookup
Non-standard analysis wikipedia , lookup
Large numbers wikipedia , lookup
Proofs of Fermat's little theorem wikipedia , lookup
Four color theorem wikipedia , lookup
Order theory wikipedia , lookup
Transcript
Introduction to Probability Supplementary Notes 2 Recursion MSIS 26:960:575:01 Instructor: Farid Alizadeh October 7, 2000 Topic 1: Recursive formulas based on one variable In many cases you will need to reason recursively in order to solve a problem. In general for an unknown function of n, say fn a recursive relation, instead of giving an explicit formula for fn , gives a relation which expresses fn in terms of fn−1 (or sometimes fn−1 and fn−2 and so on) and some other explicitly known function. You will also need initial values for instance f0 or f1 (or f0 and f1 if the recurrence relation depends on two previous value). With initial values and the recurrence formulas you can completely determine fn . Example1: How many subsets does the set An = {1, 2, · · · , n} have? Answer: We have already solved this problem and shown that the number is 2n . Let us get this again by recursive methods. To approach the problem, let us list the subsets of A0 = {}, A1 = {1}, A2 = {1, 2}, and A3 = {1, 2, 3}: A0 A1 A2 A3 = {} : = {1} : = {1, 2} : = {1, 2, 3} : {} {}, {1} {}, {1}, {2}, {1, 2} {}, {1}, {2}, {1, 2}, {3}, {1, 3}, {2, 3}, {1, 2, 3} 1 If you look at the list above you will see an algorithm: To get the list of subsets of An , first copy the list of subsets of An−1 , (because they are also subsets of An without the new element n in them) and then copy that same list again, except that this time add the new element n to each subset. This reasoning also gives you an immediate recurrence relation. If an is the number of subsets then we have shown: an = 2an−1 from here and the fact that a0 = 1 it should be easy to show by induction that an = 2n . Example 2: How many subsets does the set An = {1, 2, · · · , n} have so that no subset contains consecutive numbers? Answer: We attack this problem just like Example 1. First let us list all such subsets for a few values of n: A0 A1 A2 A3 A4 = {} : = {1} : = {1, 2} : = {1, 2, 3} : = {1, 2, 3, 4} : {} {}, {1} {}, {1}, {2} {}, {1}, {2}, {3}, {1, 3} {}, {1}, {2}, {3}, {1, 3}, {4}, {1, 4}, {2, 4} Again if you examine the list you will see the following pattern: Obviously the list of subsets of An−1 without consecutive elements are still valid subsets of An ; these are exactly those subsets that don’t have the new element n in them. To list those subsets that do have n in them observe that they cannot possibly have n − 1 in them, so all we have to do is to list the subsets of An−2 and add the new element n to them. This will account for all possible subsets of An with no consecutive numbers in them. Here again, if fn is the desired number, then we have actually proved the recurrence: fn = fn−1 + fn−2 with the initial conditions that f0 = 1 and f1 = 2. As you may know these numbers are called the Fibonacci numbers. Observe that the recurrence relation along with the initial numbers for f0 and f1 are all you need to determine the Fibonacci numbers. If I ask you the value of fn for any n you can easily write a program to calculate it. In this case it is also possible to 2 find an explicit formula for fn , we will derive it when we discuss generating functions. However, there does not seem to be any direct combinatorial proof for this explicit formula. Topic 2: Number of walks in graphs Consider the following directed multigraph: a 1 b h c d g i 2 3 e f We want to know how many walks of length exactly equal to n are there from vertex 1 to vertex 3. Notice that each walk is a sequence of directed edges and if there are two or more edges between two vertices, (as in between vertices 1 and 3 in our graph) then these edges are counted separately and therefore walks made up of one is different from walks made up of the other. Also loops that connect a vertex to itself, if taken, count as one step. So, for instance “abddef” is counted as a different walk of length 6 than, say “acdedf” even though the sequence of vertices visited are identical in both walks. Another way of viewing this problem is to ask for number of n–letter words over the alphabet {a, b, c, d, e, f, g, h, i} with restrictions on which letters can follow each letter, and theses rules are encoded in the graph above. Now to count such walks let us define an : number of walks of length n from vertex 1 to vertex 1, bn : number of walks of length n from vertex 1 to vertex 2, cn : number of walks of length n from vertex 1 to vertex 3. We are interested in cn . In order to obtain a recurrence relation, notice that for a walk to have length n and end in vertex 3, it needs to have walked a length of n − 1 and have ended up in vertices 1 or 2 and then have taken a 3 final step into vertex 3. Now if it was in vertex 1 before moving to vertex 3 then there are, by definition, an−1 ways of getting from 1 to 1 in n − 1 steps and then there are two ways of walking from 1 to 3. Therefore there are 2an−1 ways of getting from 1 to 3 with one to the last stop at 1. On the other hand if the last vertex before 3 is vertex 2, then there are, by definition, bn−1 ways of getting there, and then there is only one way of getting from 2 to 3. We conclude that the total number of ways of getting from 1 to 3 satisfies: cn = 2an−1 + bn−1 However, this is not enough to compute cn because we have no formulas or recurrences for an or bn , and so we need to discover such recurrences for them too. The logic is quite similar to the case of cn . For bn , we have an−1 ways of going from 1 to 1 in n − 1 steps and then 2 ways of going from 1 to 2, and so there are 2an−1 ways of getting from 1 to 2 with one to the last stop at 1. Similarly there are 2bn ways of going from 1 to 2 with one to the last stop at 2. Thus we get bn = 2an−1 + 2bn−1 Finally we need a recurrence for an and similar argument as before will give: an = an−1 + cn−1 Now we set a0 = 1 and b0 = c0 = 0. (There is one way of going from 1 to 1 without taking any steps at all, and that is taking the empty walk: Simply stay put! There is of course no way of getting from 1 to 2 or to 3 with zero steps.) We have now all the information to obtain an , bn and cn . There is a shorthand way of writing the three recurrences and that is by using matrix notation: an an−1 1 0 1 2 2 0 bn−1 = bn cn−1 cn 2 1 0 More succinctly, vn = Avn−1 = A2 vn−2 = · · · = An v0 where 1 0 1 A = 2 2 0 , 2 1 0 an−1 vn = bn−1 cn−1 and This formula can be generalized to any multigraph: 4 1 v0 = 0 0 Theorem 0.1 Let G be a multigraph with vertices {1, 2, · · · , n} and zero or more edges from each vertex i to each vertex j. Let A be an n×n matrix such that aij is the number of edges from vertex i to vertex j. Then the number of walks of length exactly equal to n from vertex i to vertex j is the ij entry of matrix An . Proof: The proof is by induction and it is essentially the adaptation of the way we obtained the recurrence relation for the three vertex graph above. Suppose the statement is true for all paths of length n − 1, that is the i, j entry of the matrix An−1 counts the number of walks of length n − 1 from vertex i to vertex j, for any pair of vertices i and j. Now, to go from vertex i to vertex j in n steps it is necessary to go from vertex i to some vertex in n − 1 steps and then from there to vertex j in one step. This one to the last vertex could be any one of vertices 1, 2, · · · , n. Let’s assume it is vertex k. Then the number of ways of getting from i to k in n − 1 steps, by induction hypothesis, is (An−1 )ik , that is the i, k entry of matrix An−1 . Now by definition of matrix A, the number of ways to go from vertex k to vertex j is Akj . Thus the total number of walks from vertex i to vertex j with one to the last stop at vertex k, by principle of multiplication, is (An−1 )ik Akj . Summing over all possible values of k from 1 to n we count all possible walks of length n from i to j: (An−1 )i1 A1j + (An−1 )i2 A2j + · · · + (An−1 )in Anj But by definition of matrix multiplication, this sum is nothing but the i, j entry of the matrix product An−1 .A, which is equal to An . And we are done! Topic 3: Two-variable recurrences Recursive formulas do not only apply to formulas with one parameter like n. One can have formulas depending on two or more parameters. In these cases, however, instead of initial conditions, one needs boundary conditions; let us look at a familiar example. Example 3: Let us forget that we know an explicit formula for n k n k and try to find a recursive formula for it. We know that counts the number of k–subsets of an n–set. Each such subset has either the element n in it 5 or not. If it does not then it is actually a k-subset of An−1 ; by definition n−1 there are k such subsets. If the subset has the element n in it, then upon removing n we get a subset of k − 1 elements of a set of n − 1 elements. By n−1 definition there are k−1 such subsets. Since these two cases account for all possibilities, we have actually proved that ! ! ! n n−1 n−1 = + k k k−1 Now this formula in itself is not enough to determine the values of need to have some initial conditions. We know that (Why?) And that n n = 1 and can now tabulate values of k ↓, n → 0 1 2 3 4 5 0 1 1 1 1 1 1 1 2 n k n 0 n k n k . We = 0 for k > n. = 1. With the latter two conditions we : 3 4 5 1 2 1 3 3 1 4 6 4 5 10 10 1 5 1 Here the boundary conditions–literally the two sides of the triangle–are n numbers 0 and nn . Example 4: Stirling numbers of the second kind. Remember that a partition of a set An = {1, 2, · · · , n} is a collection of subsets of An , say B1 , B2 , · · · , Bk such that i. Bi ∩ Bj = φ, ii. B1 ∪ B2 ∪ · · · ∪ Bn = An , and iii. Bi 6= φ. We are interested in the number of different partitions of a set of n elements into k blocks. Such(numbers are called Stirling numbers of the second kind ) n and are denoted by . To count these kinds of partitions, we focus again k 6 on the largest element of the set, that is n (or any other element, it really does not matter.) In particular we focus on the block of the partition that contains n. There are two possibilities: 1. The block that contains n is a singleton set {n}. Suppose we remove n from our set. Then this block disappears and we are left ( with a partition ) n−1 of a set of n − 1 elements into k − 1 blocks. There are such k−1 partitions by definition. And to each one we can add the singleton set {n} to make a partition of An into k blocks. 2. The block that contains n contains some other elements in it. Now if we remove n from An as before, the block that contains n will not disappear: Here we have a partition of a set of n − 1 elements into still ( ) n−1 k blocks. There are such partitions by definition. Now the k new element n can get into any one of the k blocks to make a partition of An -set into k blocks. The argument above immediately proves the recursion: ( n k ) = ( n−1 k−1 ) +k ( n−1 k ) What about the boundary conditions? Well, we know that ( n 1 ) = ( n n ) = 1. (Why?) boundary conditions and the recurrence are sufficient to generate ( These ) n for all values of n and k. One can create a triangle similar to the k triangle we obtained for nk : k ↓, n → 1 2 3 4 5 6 1 1 1 1 1 1 1 2 3 4 5 6 1 3 1 7 6 1 15 25 10 1 31 90 65 15 1 7 Example: Balls and bags revisited: Let us consider again the case of n balls and r bags. This time however, the bags are non-distinct and the balls are distinct. 1. How many arrangements are there such that no bag is empty? 2. How many arrangements are there with empty bags allowed? Answer: None of the bags are empty and the balls are distinct, but the bags are not distinct, so in the following picture the first and second arrangements are considered the same, but the third arrangement is different. 1 3 2 4 6 5 7 8 First arrangement 3 1 4 5 8 7 6 2 Second arrangement 6 2 1 3 4 5 7 Third arrangement 8 8 In this case observe that the bags essentially partition ( ) the set of n distinct n balls into r nonempty parts. Thus the answer is . r Now if some of the bags are empty, we decompose the problem into the case where exactly one bag is nonempty, or exactly two bags are nonempty, or exactly three bags are nonempty, ... or exactly r bags are nonempty. The answer therefore is equal to: ( n 1 ) + ( n 2 ) + ··· ( n r ) Topic 4: Recurrences with full memory So far the recurrences that we have seen relate an to an−1 or a few previous terms. There are however recurrences that need all of the values of ak for k = 1, 2, · · · , n − 1 to obtain an . Let us see a couple of examples: Example: Bell numbers. We are interested in the total number of partitions of a set of n elements. Let us call such numbers bn . These numbers are sometimes called Bell numbers. Of course, based on what we know so far we can compute these numbers: bn = ( n 1 ) + ( n 2 ) + ··· + ( n n ) (Why?) Let us derive a recurrence formula directly for these numbers. Take the set An+1 = {1, 2, · · · , n, n + 1}. We focus on the block that contains n + 1. This particular bloc, in addition to the element n + 1, contains k other n elements where k is either 0 or 1 or ... n. There are k ways of choosing the companions of n + 1 that go with it into a block. Then we are left with n − k other elements and we may partition these in any way we want and attach the resulting blocks to the block that contains n + 1 and its k companions. There are, by definition, bn−k such partitions. Thus, by principle of multiplication, n we have k bn−k such partitions. But this counts only the case k; ranging over all possible k = 0, · · · , n and adding up we obtain bn+1 ! ! ! ! n n n n bn bn−k + · · · + b1 + · · · + b0 + = 0 k n−1 n 9 Observing that n k = n n−k we get: bn+1 = n X k=0 bk n k ! We also agree that b0 = 1: There is only one partition of the empty set and that is the empty partition! Therefore we have all the necessary information to calculate the Bell numbers using this initial condition and the recurrence relation above. Note that in the recurrence relation bn+1 depends on all values of bk for k = 0, · · · n. Here is another example. Example: Catalan numbers. How many unlabeled binary search trees are there with exactly n nodes? Answer: First recall that a binary search tree is made up of a root and a right subtree and a left subtree each of which may be either empty or another nonempty binary search tree. Here is a list of such trees for n = 1, 2, 3: n= 1 n= 2 n= 3 Let the number of such trees be cn ; these are sometimes called the Catalan numbers. Let us try to find a recurrence relation for cn+1 . If a tree has n + 1 nodes and assuming its left subtree has k nodes, then necessarily its right subtree has n−k nodes. There are by definition ck ways to choose the k-node left subtree and cn−k ways to choose the (n − k)-node right subtree. Thus there are ck ×cn−k trees whose left subtree has exactly k nodes. Summing over all possibilities of k = 0, 1, 2, · · · , n we account for all cn+1 trees. Therefore 10 we have shown the recurrence: cn+1 = c0 cn + c1 cn−1 + c2 cn−2 + · · · + cn c0 We also know that c0 = 1: There is only one tree with no nodes and that is the empty tree. We have all the information to compute these numbers from our initial conditions and the recurrence relation. Again notice that in this recurrence, we need all previous values of ck to compute cn+1 . Later when we discuss generating functions, we will derive an explicit formula for these numbers. 11