* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Lecture 5 - McGill University
Survey
Document related concepts
Infinitesimal wikipedia , lookup
Abuse of notation wikipedia , lookup
Foundations of mathematics wikipedia , lookup
List of first-order theories wikipedia , lookup
Positional notation wikipedia , lookup
List of important publications in mathematics wikipedia , lookup
List of prime numbers wikipedia , lookup
Fundamental theorem of algebra wikipedia , lookup
Factorization wikipedia , lookup
Factorization of polynomials over finite fields wikipedia , lookup
Large numbers wikipedia , lookup
Proofs of Fermat's little theorem wikipedia , lookup
Mathematics of radio engineering wikipedia , lookup
Transcript
ALGEBRA+NUMBER THEORY +COMBINATORICS COMP 321 – McGill University These slides are mainly compiled from the following resources. - Professor Jaehyun Park’ slides CS 97SI - Top-coder tutorials. - Programming Challenges books. Outline • Algebra. • Number Theory • Combinatorics Numerical Bases • Binary • Base-2 numbers are made up of the digits 0 and 1. They provide the integer representation used within computers, because these digits map naturally to on/off or high/low states. • Octal • Base-8 numbers are useful as a shorthand to make it easier to read binary numbers, since the bits can be read off from the right in groups of three. Thus 101110012 = 3718 = 24910. • Why do programmers think Christmas is Halloween? Because 31 Oct = 25 Dec! • Decimal • We use base-10 numbers because we learned to count on our ten fingers. Numerical Bases • Hexadecimal • Base-16 numbers are an even easier shorthand to represent binary numbers • Alphanumeric • Base-36 numbers are the highest you can represent using the 10 numerical digits with the 26 letters of the alphabet. Real Numbers • The most important thing to remember about real numbers is that they are not real real numbers. • Floating point arithmetic has limited precision. • The fact that there always exists a number c between a and b if a < b. This is not true in real numbers as they are represented in a computer. • The associativity of addition guarantees that (a + b ) + c = a + (b + c ). Unfortunately, this is not necessarily true in computer arithmetic because of round-off errors. Types of Numbers • Integers • These are the counting numbers, −∞, . . . ,− 2,− 1, 0, 1, 2, . . . ,∞ . • Rational Numbers • These are the numbers which can be expressed as the ratio of two integers, i.e. c is rational if c = a/b for integers a and b • Irrational Numbers • There does not exist any pair of integers x and y such that x/y equals any of these numbers. • Examples include π = 3. 1415926 . . . ,√2 = 1. 41421 . . . , and e = 2. 71828 . . . . • They can be computed using Taylor series expansions, but for all practical purposes it suffices to approximate them using the ten digits or so. Dealing with Numbers • Rounding • Rounding is used to get a more accurate value for the least significant digit. • Truncating • Truncation is exemplified by the floor function, which converts a real number of an integer by chopping off the fractional part. Fractions • Exact rational numbers x/y are best represented by pairs of integers x, y, where x is the numerator and y is the denominator of the fraction. • Addition • We must find a common denominator before adding fractions • Substraction • Same as addition, since c − d = c + − 1 X d Fractions • Multiplication • Since multiplication is repeated addition • Division • To divide fractions you multiply by the reciprocal of the denominator • Note: It is important to reduce fractions to their simplest representation, i.e., replace 2/ 4 by 1/ 2 (use GCD). Manipulating Polynomials • Evaluation. • Brute Force: computing each term cixn independently and adding them together. It costs O(n2) multiplications. • Improvement 1: Note that xi= xi-1x , so if we compute the terms from smallest degree to highest degree we can keep track of the current power of x , and get away with two multiplications per term (xi-1 × x , and then ci × xi ). • Improvement 2: Employ Horner’s rule. Manipulating Polynomials • Addition/Substraction. • Easier than the same operations on long integers, since there is no borrowing or carrying. • Simply add or subtract the coefficients of the ith terms for all i from zero to the maximum degree. • Multiplication. • The product of polynomials P(x) and Q(x) is the sum of the product of every pair of terms, where each term comes from a different polynomial: • Such an all-against-all operation is called a convolution (check the fast Fourier transform O(n log n)) . Root Finding • Given a polynomial P(x) and a target number t, the problem of root finding is identifying any or all x such that P(x) = t . • If P(x) is a first-degree polynomial • If P(x) is a second-degree polynomial Root Finding • There are more complicated formulae for solving third- and fourth-degree polynomials. • Beyond quadratic equations, numerical methods are typically used. • Newton-Raphson (the basic idea is that of binary search) Logarithms • A logarithm is simply an inverse exponential function. • Logarithms are still useful for multiplication • A direct consequence of this is that • So how can we compute ab for any a and b using the exp(x) and ln(x) functions? • So the problem is reduced to one multiplication plus one call of each of these functions Sum of Powers • Pretty useful in many random situations Fast Exponentiation • Recursive computation of an Implementation • Running time O(log n). Linear Algebra • Gaussian elimination is useful for: • Solve a system of linear equations • Invert a matrix • Find the rank of a matrix • Compute the determinant of a matrix Outline • Algebra • Number Theory • Combinatorics Number Theory: Prime numbers • A natural number starting from 2: {2, 3, 4, . . .} is considered as a prime if it is only divisible by 1 or itself. • The first (and the only even) prime is 2. • The next prime numbers are: 3, 5, 7, 11, 13, 17, 19, 23, 29, . . . , and infinitely many more primes. • There are 25 primes in range [0 . . . 100], 168 primes in [0 . . . 1000], 1000 primes in [0 . . . 7919], 1229 primes in [0 . . . 10000], etc... Number Theory: Prime Testing Function • Test by definition. • Test if N is divisible by divisor ∈ [2 . . .N-1] • runs in O(N) • Improvement 1 • Test if N is divisible by a divisor ∈ [2 . . .√N] • if N is divisible by p, then N = p X q. If q were smaller than p, then q or a prime factor of q would have divided N earlier. • This is O(√N) • Improvement 2 • test if N is divisible by divisor ∈ [3, 5, 7 . . .√N] (only test odd numbers). • there is only one even prime number, i.e. number 2, which can be tested separately. • This is O(√N/2) Number Theory: Prime Testing Function • Improvement 3. • Test if N is divisible by prime divisors ≤√N • This is O(|#primes ≤√N|) • This improvement is already good enough for contest problems. • There are 500 odd numbers in [1 . . . √(106)], but there are only 168 primes in the same range. • This is O(√N/ln(√N)). Generating List of Prime Numbers • Use the ‘Sieve of Eratosthenes’ algorithm. • First, it sets all numbers in the range to be ‘probably prime’ but set numbers 0 and 1 to be not prime. • Then, it takes 2 as prime and crosses out all multiples of 2 • Then it takes the next non-crossed number 3 as a prime and crosses out all multiples of 3. • Then it takes 5 and crosses out all multiples of 5. • After that, whatever left uncrossed within the range [0 . . .N] are primes. • This is roughly O(N log logN) • opt sieve for smaller primes and reserve optimized prime testing function for larger primes Finding Prime Factors • A composite numbers N, i.e. the non-primes, can be written uniquely it as a multiplication of its prime factors. • N = 240 = 2 X 2 X 2 X 2 X 3 X 5 = 24 X 3 X 5 (the latter form is called prime-power factorization). • Divide and Conquer Spirit: • An integer N can be expressed as: N = PF X N’, where PF is a prime factor and N’ is another number which is N/PF – i.e. we can reduce the size of N by taking out its factor PF • We can keep doing this until eventually N = 1. Special case if N is actually a prime number. • This is O(π(√N)) = O(√N/ln√N). Greatest Common Divisor (GCD) • The largest positive integer d such that d | a and d | b where x | y implies that x divides y. • One practical usage of GCD is to simplify fraction, e.g. • Used very frequently in number theoretical problems. • Some facts: • gcd(a, b) = gcd(a, b − a) • gcd(a, 0) = a GCD: Euclidian Algorithm • Repeated use of gcd(a, b) = gcd(a, b − a) • Example: GCD: Euclidian Algorithm-Implementation • Running time: O(log(a + b)) LCM: Least Common Multiple. • The smallest positive integer l such that a | l and b | l • lcm(a, b) = a X b/gcd(a, b). • Implementation: • The GCD of more than 2 numbers, e.g. gcd(a, b, c) is equal to gcd(a, gcd(b, c)), etc, and similarly for LCM. • Both GCD and LCM algorithms run in O(log10 n), where n = max(a, b). Number Theory: Relative Prime • Two integers a and b are said to be relatively prime if gcd(a, b) = 1 • e.g. 25 and 42. • Problem of finding positive integers below N that are relatively prime to N. • Use Euler’s Totient (Phi) function. • For example: Number Theory: Solving Diophantine Equation • David buys apples and oranges at a total cost of 8.39CAD. If an apple is 25 cents and an orange is 18 cents, how many of each type of fruit does David buys? • This problem can be modeled as a linear equation with two variables: 25x + 18y = 839. • Since we know that both x and y must be integers, this linear equation is called the Linear Diophantine Equation. • We can solve Linear Diophantine Equation with two variables even if we only have one equation! Number Theory: Solving Diophantine Equation • Let a and b be integers with d = gcd(a, b). The equation ax + by = c has no integral solutions if d | c is not true. • But if d | c, then there are infinitely many integral solutions. • The first solution (x0, y0) can be found using the Extended Euclid algorithm, and the rest can be derived from x = x0 + (b/d)n, y = y0 − (a/d)n, where n is an integer. Number Theory: Extended Euclid Number Theory: Solving Diophantine Equation • For our problem above: 25x + 18y = 839, we have: • a = 25, b = 18, extendedEuclid(25, 18) = ((−5, 7), 1), or • 25 X (−5) + 18 X 7 = 1. • Multiplying the left and right hand side of the equation above by 839/gcd(25, 18) = 839, we have: • 25 X (−4195) + 18 X 5873 = 839. Thus, • x = −4195 + (18/1)n, y = 5873 − (25/1)n. • Since we need to have non-negative x and y, we have: • −4195 + 18n ≥ 0 and 5873 − 25n ≥ 0, or • 4195/18 ≤ n ≤ 5873/25, or • 233.05 ≤ n ≤ 234.92. • The only possible integer for n is 234. Number Theory: Solving Diophantine Equation • Thus • x = −4195 + 18 X 234 = 17 and y = 5873 − 25 X 234 = 23, • i.e. 17 apples (of 25 cents each) and 23 oranges (of 18 cents each) of a total of 8.39 CAD. Number Theory: Fibonacci Numbers • Leonardo Fibonacci’s numbers are defined as fib(0) = 0, fib(1) = 1, and fib(n) = fib(n − 1) + fib(n−2) for n ≥ 2. • This generates the following familiar patterns: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89 which can be derived with an O(n) DP technique. • (One of their properties) Zeckendorf’s theorem: • every positive integer can be written in a unique way as a sum of one or more distinct Fibonacci numbers such that the sum does not include any two consecutive Fibonacci numbers. Number Theory: Fibonacci Numbers • Implementation 1 (recursive solution) • Exponential time complexity. • Implementation 2 (Dynamic Programming) • O(n) complexity. Number Theory: Fibonacci Numbers • Implementation 3 (matrix exponentiation) • Use fast exponentiation to compute the matrix power Outline • Algebra. • Number Theory • Combinatorics Combinatorics • It is a branch of discrete mathematics concerning the study of finite or countable discrete structures. • Programming problems involving combinatorics usually titled ‘How Many [Object]’, ‘Count [Object]’, etc, • The code is usually short, but finding the recurrence formula takes some mathematics brilliance and patience. • In competitions, if such problem exists in the given problem set, ask one team member to derive the formula whereas the other two concentrates on other problems. Quickly code the usually short formula once it is obtained Basic Counting Techniques • Product Rule • if there are |A| possibilities from set A and |B| possibilities from set B, then there are |A| X |B| ways to combine one from A and one from B. • For example, suppose you own 5 shirts and 4 pants. Then there are 5X 4 = 20 different ways you can get dressed tomorrow. • Sum Rule • if there are |A| possibilities from set A and |B| possibilities from set B, then there are |A| + |B| ways for either A or B to occur (assuming the elements of A and B are distinct). • For example, given that you own 5 shirts and 4 pants and the laundry ruined one of them, there are 9 possible ruined items. Basic Counting Techniques • Inclusion-Exclusion Formula • The sum rule is a special case of a more general formula when the two sets can overlap, namely, • calculate the total number of colors given the number of color-matched garments. • The inclusion-exclusion formula generalizes to three sets and beyond. Counting Sets • Permutations • An arrangement of n items, where every item appears exactly once. There are n! different permutations. • The 3! = 6 permutations of three items are 123, 132, 213, 231, 312, and 32.1 • Subsets • A selection of elements from n possible items. There are 2n distinct subsets of n things. • The 23 = 8 subsets of three items, namely,1, 2, 3, 12, 13, 23, 123, and the empty set. • Strings • A sequence of items which are drawn with repetition. There are mn distinct sequences of n items drawn from m items • The 27 length-3 strings on 123 are 111, 112, 113 ….. 333 Other Counting Sequences • Catalan Numbers • The first several terms are 2, 5, 14, 42, 132, 429, 1430, . . . When C0 = 1. • How many ways are there to build a balanced formula from n sets of left and right parentheses? • there are five ways to do it for n = 3: ((())), ()(()), (())(), (()()), and ()()(). • number of triangulations of a convex polygon. • counting the number of rooted binary trees on n +1 leaves. • counting the number of paths across a lattice which do not rise above the main diagonal. Binomial Coefficients • is the number of ways to choose k objects out of n distinguishable objects. • same as the coefficient of xkyn−k in the expansion of (x + y)n • Hence the name “binomial coefficients” • Appears everywhere in combinatorics. Computing: Binomial Coefficients • Solution 1: Compute using the following formula: • Solution 2: Use Pascal’s triangle • Can use either if both n and k are small • Use Solution 1 carefully if n is big, but k or n − k is small Combinatorial Games • Turn-based competitive multi-player games • Can be a simple win-or-lose game, or can involve points • Everyone has perfect information • Each turn, the player changes the current “state” using a valid “move” • At some states, there are no valid moves • The current player immediately loses at these states Combinatorial Game Example • Settings: There are n stones in a pile. Two players take turns and remove 1 or 3 stones at a time. The one who takes the last stone wins. Find out the winner if both players play perfectly • State space: Each state can be represented by the number of remaining stones in the pile • Valid moves from state x: x -> (x − 1) or x -> (x − 3), as long as the resulting number is nonnegative • State 0 is the losing state Combinatorial Game Example • No cycles in the state transitions • Can solve the problem bottom-up (DP) • A player wins if there is a way to force the opponent to lose • Conversely, we lose if there is no such a way • State x is a winning state (W) if • (x − 1) is a losing state, • OR (x − 3) is a losing state • Otherwise, state x is a losing state (L) Combinatorial Game Example • DP table for small values of n: • See a pattern? • Let’s prove our conjecture Combinatorial Game Example • Conjecture: If n is odd, the first player wins. If n is even, the second player wins. • Holds true for the base case n = 0 • In general, • If n is odd, we can remove one stone and give the opponent an even number of stones. • If n is even, no matter what we choose, we have to give an odd number of stones to the opponent. More Complex Games • Settings: a competitive zero-sum two-player game • Zero-sum: if the first player’s score is x, then the other player gets −x • Each player tries to maximize his/her own score • Both players play perfectly • Can be solved using a minimax algorithm Minimax Algorithm • Recursive algorithm that decides the best move for the current player at a given state • Define f(S) as the optimal score of the current player who starts at state S • Let T1, T2, . . . , Tm be states can be reached from S using a single move • Let T be the state that minimizes f(Ti) • Then, f(S) = −f(T) • Intuition: minimizing the opponent’s score maximizes my score Memoization • (Not memorization but memoization) • A technique used to avoid repeated calculations in recursive functions • High-level idea: take a note (memo) of the return value of a function call. When the function is called with the same argument again, return the stored result • Each subproblem is solved at most once • Some may not be solved at all! Recursive Function without Memoization • fib(1) is called many times Recursive Function with Memoization • fib(1) is called less times Minimax Algorithm Pseudocode • Given state S, want to compute f(S) • If we know f(S) already, return it • Set return value x <− −∞ • For each valid next state T: • Update return value x <− max{x, −f(T)} • Write a memo f(S) = x and return x Minimax – Possible Extensions • The game is not zero-sum • Each player wants to maximize his own score • Each player wants to maximize the difference between his score and the opponent’s • There are more than two players • All of above can be solved using a similar idea Nim Game • Settings: There are n piles of stones. Two players take turns. Each player chooses a pile, and removes any number of stones from the pile. The one who takes the last stone wins. Find out the winner if both players play perfectly • Can’t really use DP if there are many piles, because the state space is huge Nim Game • Starts with heaps of 3, 4, 5 stones • – We will call them heap A, heap B, and heap C • Alice takes 2 stones from A: (1, 4, 5) • Bob takes 4 from C: (1, 4, 1) • Alice takes 4 from B: (1, 0, 1) • Bob takes 1 from A: (0, 0, 1) • Alice takes 1 from C and wins: (0, 0, 0) Nim Game • Given heaps of size n1, n2, . . . , nm • The first player wins if and only if the nim-sum • n1 n2 · · · nm is nonzero.( = bitwise XOR) • Why? • If the nim-sum is zero, then whatever the current player does, the nim-sum of the next state is nonzero • If the nim-sum is nonzero, it is possible to force it to become zero (not obvious, but true) Tips for Solving Game Problems • If the state space is small, use memoization • If not, print out the result of the game for small test data and look for a pattern • This actually works really well! • Try to convert the game into some nim-variant