Download Lecture 5 - McGill University

ALGEBRA+NUMBER THEORY
+COMBINATORICS
COMP 321 – McGill University
These slides are mainly compiled from the following
resources.
- Professor Jaehyun Park’ slides CS 97SI
- Top-coder tutorials.
- Programming Challenges books.
Outline
•  Algebra.
•  Number Theory
•  Combinatorics
Numerical Bases
•  Binary
•  Base-2 numbers are made up of the digits 0 and 1. They provide
the integer representation used within computers, because these
digits map naturally to on/off or high/low states.
•  Octal
•  Base-8 numbers are useful as a shorthand to make it easier to
read binary numbers, since the bits can be read off from the right in
groups of three. Thus 101110012 = 3718 = 24910.
•  Why do programmers think Christmas is Halloween? Because 31
Oct = 25 Dec!
•  Decimal
•  We use base-10 numbers because we learned to count on our ten
fingers.
Numerical Bases
•  Hexadecimal
•  Base-16 numbers are an even easier shorthand to represent binary
numbers
•  Alphanumeric
•  Base-36 numbers are the highest you can represent using the 10
numerical digits with the 26 letters of the alphabet.
Real Numbers
•  The most important thing to remember about real
numbers is that they are not real real numbers.
•  Floating point arithmetic has limited precision.
•  The fact that there always exists a number c between a and b if a < b.
This is not true in real numbers as they are represented in a computer.
•  The associativity of addition guarantees that (a + b ) + c = a + (b + c ).
Unfortunately, this is not necessarily true in computer arithmetic
because of round-off errors.
Types of Numbers
•  Integers
•  These are the counting numbers, −∞, . . . ,− 2,− 1, 0, 1, 2, . . . ,∞ .
•  Rational Numbers
•  These are the numbers which can be expressed as the ratio of two
integers, i.e. c is rational if c = a/b for integers a and b
•  Irrational Numbers
•  There does not exist any pair of integers x and y such that x/y
equals any of these numbers.
•  Examples include π = 3. 1415926 . . . ,√2 = 1. 41421 . . . , and
e = 2. 71828 . . . .
•  They can be computed using Taylor series expansions, but for all
practical purposes it suffices to approximate them using the ten
digits or so.
Dealing with Numbers
•  Rounding
•  Rounding is used to get a more accurate value for the least
significant digit.
•  Truncating
•  Truncation is exemplified by the floor function, which converts a
real number of an integer by chopping off the fractional part.
Fractions
•  Exact rational numbers x/y are best represented by pairs
of integers x, y, where x is the numerator and y is the
denominator of the fraction.
•  Addition
•  We must find a common denominator before adding fractions
•  Substraction
•  Same as addition, since c − d = c + − 1 X d
Fractions
•  Multiplication
•  Since multiplication is repeated addition
•  Division
•  To divide fractions you multiply by the reciprocal of the
denominator
•  Note: It is important to reduce fractions to their simplest
representation, i.e., replace 2/ 4 by 1/ 2 (use GCD).
Manipulating Polynomials
•  Evaluation.
•  Brute Force: computing each term cixn independently and adding
them together. It costs O(n2) multiplications.
•  Improvement 1: Note that xi= xi-1x , so if we compute the terms
from smallest degree to highest degree we can keep track of the
current power of x , and get away with two multiplications per term
(xi-1 × x , and then ci × xi ).
•  Improvement 2: Employ Horner’s rule.
Manipulating Polynomials
•  Addition/Substraction.
•  Easier than the same operations on long integers, since there is no
borrowing or carrying.
•  Simply add or subtract the coefficients of the ith terms for all i from
zero to the maximum degree.
•  Multiplication.
•  The product of polynomials P(x) and Q(x) is the sum of the product of
every pair of terms, where each term comes from a different
polynomial:
•  Such an all-against-all operation is called a convolution (check the fast
Fourier transform O(n log n)) .
Root Finding
•  Given a polynomial P(x) and a target number t, the
problem of root finding is identifying any or all x such that
P(x) = t .
•  If P(x) is a first-degree polynomial
•  If P(x) is a second-degree polynomial
Root Finding
•  There are more complicated formulae for solving third-
and fourth-degree polynomials.
•  Beyond quadratic equations, numerical methods are
typically used.
•  Newton-Raphson (the basic idea is that of binary search)
Logarithms
•  A logarithm is simply an inverse exponential function.
•  Logarithms are still useful for multiplication
•  A direct consequence of this is that
•  So how can we compute ab for any a and b using the
exp(x) and ln(x) functions?
•  So the problem is reduced to one multiplication plus one
call of each of these functions
Sum of Powers
•  Pretty useful in many random situations
Fast Exponentiation
•  Recursive computation of an
Implementation
•  Running time O(log n).
Linear Algebra
•  Gaussian elimination is useful for:
•  Solve a system of linear equations
•  Invert a matrix
•  Find the rank of a matrix
•  Compute the determinant of a matrix
Outline
•  Algebra
•  Number Theory
•  Combinatorics
Number Theory: Prime numbers
•  A natural number starting from 2: {2, 3, 4, . . .} is
considered as a prime if it is only divisible by 1 or itself.
•  The first (and the only even) prime is 2.
•  The next prime numbers are: 3, 5, 7, 11, 13, 17, 19, 23,
29, . . . , and infinitely many more primes.
•  There are 25 primes in range [0 . . . 100], 168 primes in
[0 . . . 1000], 1000 primes in [0 . . . 7919], 1229 primes in
[0 . . . 10000], etc...
Number Theory: Prime Testing Function
•  Test by definition.
•  Test if N is divisible by divisor ∈ [2 . . .N-1]
•  runs in O(N)
•  Improvement 1
•  Test if N is divisible by a divisor ∈ [2 . . .√N]
•  if N is divisible by p, then N = p X q. If q were smaller than p, then q or
a prime factor of q would have divided N earlier.
•  This is O(√N)
•  Improvement 2
•  test if N is divisible by divisor ∈ [3, 5, 7 . . .√N] (only test odd
numbers).
•  there is only one even prime number, i.e. number 2, which can be
tested separately.
•  This is O(√N/2)
Number Theory: Prime Testing Function
•  Improvement 3.
•  Test if N is divisible by prime divisors ≤√N
•  This is O(|#primes ≤√N|)
•  This improvement is already good enough for contest problems.
•  There are 500 odd numbers in [1 . . . √(106)], but there are only 168
primes in the same range.
•  This is O(√N/ln(√N)).
Generating List of Prime Numbers
•  Use the ‘Sieve of Eratosthenes’ algorithm.
•  First, it sets all numbers in the range to be ‘probably prime’ but set
numbers 0 and 1 to be not prime.
•  Then, it takes 2 as prime and crosses out all multiples of 2
•  Then it takes the next non-crossed number 3 as a prime and
crosses out all multiples of 3.
•  Then it takes 5 and crosses out all multiples of 5.
•  After that, whatever left uncrossed within the range [0 . . .N] are
primes.
•  This is roughly O(N log logN)
•  opt sieve for smaller primes and reserve optimized prime
testing function for larger primes
Finding Prime Factors
•  A composite numbers N, i.e. the non-primes, can be
written uniquely it as a multiplication of its prime factors.
•  N = 240 = 2 X 2 X 2 X 2 X 3 X 5 = 24 X 3 X 5 (the latter form is
called prime-power factorization).
•  Divide and Conquer Spirit:
•  An integer N can be expressed as: N = PF X N’, where PF is a
prime factor and N’ is another number which is N/PF – i.e. we can
reduce the size of N by taking out its factor PF
•  We can keep doing this until eventually N = 1. Special case if N is
actually a prime number.
•  This is O(π(√N)) = O(√N/ln√N).
Greatest Common Divisor (GCD)
•  The largest positive integer d such that d | a and d | b
where x | y implies that x divides y.
•  One practical usage of GCD is to simplify fraction, e.g.
•  Used very frequently in number theoretical problems.
•  Some facts:
•  gcd(a, b) = gcd(a, b − a)
•  gcd(a, 0) = a
GCD: Euclidian Algorithm
•  Repeated use of gcd(a, b) = gcd(a, b − a)
•  Example:
GCD: Euclidian Algorithm-Implementation
•  Running time: O(log(a + b))
LCM: Least Common Multiple.
•  The smallest positive integer l such that a | l and b | l
•  lcm(a, b) = a X b/gcd(a, b).
•  Implementation:
•  The GCD of more than 2 numbers, e.g. gcd(a, b, c) is
equal to gcd(a, gcd(b, c)), etc, and similarly for LCM.
•  Both GCD and LCM algorithms run in O(log10 n), where n
= max(a, b).
Number Theory: Relative Prime
•  Two integers a and b are said to be relatively prime if
gcd(a, b) = 1
•  e.g. 25 and 42.
•  Problem of finding positive integers below N that are
relatively prime to N.
•  Use Euler’s Totient (Phi) function.
•  For example:
Number Theory: Solving Diophantine
Equation
•  David buys apples and oranges at a total cost of
8.39CAD. If an apple is 25 cents and an orange is 18
cents, how many of each type of fruit does David buys?
•  This problem can be modeled as a linear equation with
two variables: 25x + 18y = 839.
•  Since we know that both x and y must be integers, this
linear equation is called the Linear Diophantine Equation.
•  We can solve Linear Diophantine Equation with two
variables even if we only have one equation!
Number Theory: Solving Diophantine
Equation
•  Let a and b be integers with d = gcd(a, b). The equation
ax + by = c has no integral solutions if d | c is not true.
•  But if d | c, then there are infinitely many integral
solutions.
•  The first solution (x0, y0) can be found using the Extended
Euclid algorithm, and the rest can be derived from x = x0 +
(b/d)n, y = y0 − (a/d)n, where n is an integer.
Number Theory: Extended Euclid
Number Theory: Solving Diophantine
Equation
•  For our problem above: 25x + 18y = 839, we have:
•  a = 25, b = 18, extendedEuclid(25, 18) = ((−5, 7), 1), or
•  25 X (−5) + 18 X 7 = 1.
•  Multiplying the left and right hand side of the equation
above by 839/gcd(25, 18) = 839, we have:
•  25 X (−4195) + 18 X 5873 = 839. Thus,
•  x = −4195 + (18/1)n, y = 5873 − (25/1)n.
•  Since we need to have non-negative x and y, we have:
•  −4195 + 18n ≥ 0 and 5873 − 25n ≥ 0, or
•  4195/18 ≤ n ≤ 5873/25, or
•  233.05 ≤ n ≤ 234.92.
•  The only possible integer for n is 234.
Number Theory: Solving Diophantine
Equation
•  Thus
•  x = −4195 + 18 X 234 = 17 and y = 5873 − 25 X 234 = 23,
•  i.e. 17 apples (of 25 cents each) and 23 oranges (of 18
cents each) of a total of 8.39 CAD.
Number Theory: Fibonacci Numbers
•  Leonardo Fibonacci’s numbers are defined as fib(0) = 0,
fib(1) = 1, and fib(n) = fib(n − 1) + fib(n−2) for n ≥ 2.
•  This generates the following familiar patterns: 0, 1, 1, 2, 3,
5, 8, 13, 21, 34, 55, 89 which can be derived with an O(n)
DP technique.
•  (One of their properties) Zeckendorf’s theorem:
•  every positive integer can be written in a unique way as a sum of
one or more distinct Fibonacci numbers such that the sum does not
include any two consecutive Fibonacci numbers.
Number Theory: Fibonacci Numbers
•  Implementation 1 (recursive solution)
•  Exponential time complexity.
•  Implementation 2 (Dynamic Programming)
•  O(n) complexity.
Number Theory: Fibonacci Numbers
•  Implementation 3 (matrix exponentiation)
•  Use fast exponentiation to compute the matrix power
Outline
•  Algebra.
•  Number Theory
•  Combinatorics
Combinatorics
•  It is a branch of discrete mathematics concerning the
study of finite or countable discrete structures.
•  Programming problems involving combinatorics usually
titled ‘How Many [Object]’, ‘Count [Object]’, etc,
•  The code is usually short, but finding the recurrence
formula takes some mathematics brilliance and patience.
•  In competitions, if such problem exists in the given
problem set, ask one team member to derive the formula
whereas the other two concentrates on other problems.
Quickly code the usually short formula once it is obtained
Basic Counting Techniques
•  Product Rule
•  if there are |A| possibilities from set A and |B| possibilities from set
B, then there are |A| X |B| ways to combine one from A and one
from B.
•  For example, suppose you own 5 shirts and 4 pants. Then there are 5X
4 = 20 different ways you can get dressed tomorrow.
•  Sum Rule
•  if there are |A| possibilities from set A and |B| possibilities from set
B, then there are |A| + |B| ways for either A or B to occur (assuming
the elements of A and B are distinct).
•  For example, given that you own 5 shirts and 4 pants and the laundry
ruined one of them, there are 9 possible ruined items.
Basic Counting Techniques
•  Inclusion-Exclusion Formula
•  The sum rule is a special case of a more general formula when the
two sets can overlap, namely,
•  calculate the total number of colors given the number of color-matched
garments.
•  The inclusion-exclusion formula generalizes to three sets and
beyond.
Counting Sets
•  Permutations
•  An arrangement of n items, where every item appears exactly
once. There are n! different permutations.
•  The 3! = 6 permutations of three items are 123, 132, 213, 231, 312, and
32.1
•  Subsets
•  A selection of elements from n possible items. There are 2n distinct
subsets of n things.
•  The 23 = 8 subsets of three items, namely,1, 2, 3, 12, 13, 23, 123, and
the empty set.
•  Strings
•  A sequence of items which are drawn with repetition. There are mn
distinct sequences of n items drawn from m items
•  The 27 length-3 strings on 123 are 111, 112, 113 ….. 333
Other Counting Sequences
•  Catalan Numbers
•  The first several terms are 2, 5, 14, 42, 132, 429, 1430, . . .
When C0 = 1.
•  How many ways are there to build a balanced formula from n sets
of left and right parentheses?
•  there are five ways to do it for n = 3: ((())), ()(()), (())(), (()()), and ()()().
•  number of triangulations of a convex polygon.
•  counting the number of rooted binary trees on n +1 leaves.
•  counting the number of paths across a lattice which do not rise above
the main diagonal.
Binomial Coefficients
• 
is the number of ways to choose k objects out of n
distinguishable objects.
•  same as the coefficient of xkyn−k in the expansion of
(x + y)n
•  Hence the name “binomial coefficients”
•  Appears everywhere in combinatorics.
Computing: Binomial Coefficients
•  Solution 1: Compute using the following formula:
•  Solution 2: Use Pascal’s triangle
•  Can use either if both n and k are small
•  Use Solution 1 carefully if n is big, but k or n − k is small
Combinatorial Games
•  Turn-based competitive multi-player games
•  Can be a simple win-or-lose game, or can involve points
•  Everyone has perfect information
•  Each turn, the player changes the current “state” using a
valid “move”
•  At some states, there are no valid moves
•  The current player immediately loses at these states
Combinatorial Game Example
•  Settings: There are n stones in a pile. Two players take
turns and remove 1 or 3 stones at a time. The one who
takes the last stone wins. Find out the winner if both
players play perfectly
•  State space: Each state can be represented by the
number of remaining stones in the pile
•  Valid moves from state x: x -> (x − 1) or x -> (x − 3), as
long as the resulting number is nonnegative
•  State 0 is the losing state
Combinatorial Game Example
•  No cycles in the state transitions
•  Can solve the problem bottom-up (DP)
•  A player wins if there is a way to force the opponent to
lose
•  Conversely, we lose if there is no such a way
•  State x is a winning state (W) if
•  (x − 1) is a losing state,
•  OR (x − 3) is a losing state
•  Otherwise, state x is a losing state (L)
Combinatorial Game Example
•  DP table for small values of n:
•  See a pattern?
•  Let’s prove our conjecture
Combinatorial Game Example
•  Conjecture: If n is odd, the first player wins. If n is even,
the second player wins.
•  Holds true for the base case n = 0
•  In general,
•  If n is odd, we can remove one stone and give the opponent an
even number of stones.
•  If n is even, no matter what we choose, we have to give an odd
number of stones to the opponent.
More Complex Games
•  Settings: a competitive zero-sum two-player game
•  Zero-sum: if the first player’s score is x, then the other
player gets −x
•  Each player tries to maximize his/her own score
•  Both players play perfectly
•  Can be solved using a minimax algorithm
Minimax Algorithm
•  Recursive algorithm that decides the best move for the
current player at a given state
•  Define f(S) as the optimal score of the current player who
starts at state S
•  Let T1, T2, . . . , Tm be states can be reached from S using
a single move
•  Let T be the state that minimizes f(Ti)
•  Then, f(S) = −f(T)
•  Intuition: minimizing the opponent’s score maximizes my score
Memoization
•  (Not memorization but memoization)
•  A technique used to avoid repeated calculations in
recursive functions
•  High-level idea: take a note (memo) of the return value of
a function call. When the function is called with the same
argument again, return the stored result
•  Each subproblem is solved at most once
•  Some may not be solved at all!
Recursive Function without Memoization
•  fib(1) is called many times
Recursive Function with Memoization
•  fib(1) is called less times
Minimax Algorithm Pseudocode
•  Given state S, want to compute f(S)
•  If we know f(S) already, return it
•  Set return value x <− −∞
•  For each valid next state T:
•  Update return value x <− max{x, −f(T)}
•  Write a memo f(S) = x and return x
Minimax – Possible Extensions
•  The game is not zero-sum
•  Each player wants to maximize his own score
•  Each player wants to maximize the difference between his score
and the opponent’s
•  There are more than two players
•  All of above can be solved using a similar idea
Nim Game
•  Settings: There are n piles of stones. Two players take
turns. Each player chooses a pile, and removes any
number of stones from the pile. The one who takes the
last stone wins. Find out the winner if both players play
perfectly
•  Can’t really use DP if there are many piles, because the
state space is huge
Nim Game
•  Starts with heaps of 3, 4, 5 stones
•  – We will call them heap A, heap B, and heap C
•  Alice takes 2 stones from A: (1, 4, 5)
•  Bob takes 4 from C: (1, 4, 1)
•  Alice takes 4 from B: (1, 0, 1)
•  Bob takes 1 from A: (0, 0, 1)
•  Alice takes 1 from C and wins: (0, 0, 0)
Nim Game
•  Given heaps of size n1, n2, . . . , nm
•  The first player wins if and only if the nim-sum
•  n1  n2 · · ·  nm is nonzero.( = bitwise XOR)
•  Why?
•  If the nim-sum is zero, then whatever the current player
does, the nim-sum of the next state is nonzero
•  If the nim-sum is nonzero, it is possible to force it to
become zero (not obvious, but true)
Tips for Solving Game Problems
•  If the state space is small, use memoization
•  If not, print out the result of the game for small test data
and look for a pattern
•  This actually works really well!
•  Try to convert the game into some nim-variant