Download Lecture 8 - McGill University

ALGEBRA+NUMBER THEORY
+COMBINATORICS
COMP 321 – McGill University
These slides are mainly compiled from the following
resources.
- Professor Jaehyun Park’ slides CS 97SI
- Top-coder tutorials.
- Programming Challenges books.
Outline
•  Algebra.
•  Number Theory
•  Combinatorics
Numerical Bases
•  Binary
•  Base-2 numbers are made up of the digits 0 and 1. They provide
the integer representation used within computers, because these
digits map naturally to on/off or high/low states.
•  Octal
•  Base-8 numbers are useful as a shorthand to make it easier to
read binary numbers, since the bits can be read off from the right in
groups of three. Thus 111110012 = 3718 = 24910.
•  Why do programmers think Christmas is Halloween? Because 31
Oct = 25 Dec!
•  Decimal
•  We use base-10 numbers because we learned to count on our ten
fingers.
Numerical Bases
•  Hexadecimal
•  Base-16 numbers are an even easier shorthand to represent binary
numbers
•  Alphanumeric
•  Base-36 numbers are the highest you can represent using the 10
numerical digits with the 26 letters of the alphabet.
Real Numbers
•  The most important thing to remember about real
numbers is that they are not real real numbers.
•  Floating point arithmetic has limited precision.
•  The fact that there always exists a number c between a and b if a < b.
This is not true in real numbers as they are represented in a computer.
•  The associativity of addition guarantees that (a + b ) + c = a + (b + c ).
Unfortunately, this is not necessarily true in computer arithmetic
because of round-off errors.
Types of Numbers
•  Integers
•  These are the counting numbers, −∞, . . . ,− 2,− 1, 0, 1, 2, . . . ,∞ .
•  Rational Numbers
•  These are the numbers which can be expressed as the ratio of two
integers, i.e. c is rational if c = a/b for integers a and b
•  Irrational Numbers
•  There does not exist any pair of integers x and y such that x/y
equals any of these numbers.
•  Examples include π = 3. 1415926 . . . ,√2 = 1. 41421 . . . , and
e = 2. 71828 . . . .
•  They can be computed using Taylor series expansions, but for all
practical purposes it suffices to approximate them using the ten
digits or so.
Dealing with Numbers
•  Rounding
•  Rounding is used to get a more accurate value for the least
significant digit.
•  Truncating
•  Truncation is exemplified by the floor function, which converts a
real number of an integer by chopping off the fractional part.
Fractions
•  Exact rational numbers x/y are best represented by pairs
of integers x, y, where x is the numerator and y is the
denominator of the fraction.
•  Addition
•  We must find a common denominator before adding fractions
•  Substraction
•  Same as addition, since c − d = c + − 1 X d
Fractions
•  Multiplication
•  Since multiplication is repeated addition
•  Division
•  To divide fractions you multiply by the reciprocal of the
denominator
•  Note: It is important to reduce fractions to their simplest
representation, i.e., replace 2/ 4 by 1/ 2 (use GCD).
Manipulating Polynomials
•  Evaluation.
•  Brute Force: computing each term cixn independently and adding
them together. It costs O(n2) multiplications.
•  Improvement 1: Note that xi= xi-1x , so if we compute the terms
from smallest degree to highest degree we can keep track of the
current power of x , and get away with two multiplications per term
(xi-1 × x , and then ci × xi ).
•  Improvement 2: Employ Horner’s rule.
Manipulating Polynomials
•  Addition/Substraction.
•  Easier than the same operations on long integers, since there is no
borrowing or carrying.
•  Simply add or subtract the coefficients of the ith terms for all i from
zero to the maximum degree.
•  Multiplication.
•  The product of polynomials P(x) and Q(x) is the sum of the product
of every pair of terms, where each term comes from a different
polynomial:
•  Such an all-against-all operation is called a convolution
Root Finding
•  Given a polynomial P(x) and a target number t, the
problem of root finding is identifying any or all x such that
P(x) = t .
•  If P(x) is a first-degree polynomial
•  If P(x) is a second-degree polynomial
Root Finding
•  There are more complicated formulae for solving third-
and fourth-degree polynomials.
•  Beyond quadratic equations, numerical methods are
typically used.
•  Newton-Raphson (the basic idea is that of binary search)
Logarithms
•  A logarithm is simply an inverse exponential function.
•  Logarithms are still useful for multiplication
•  A direct consequence of this is that
•  So how can we compute ab for any a and b using the
exp(x) and ln(x) functions?
•  So the problem is reduced to one multiplication plus one
call of each of these functions
Sum of Powers
•  Pretty useful in many random situations
Fast Exponentiation
•  Recursive computation of an
Implementation
•  Running time O(log n).
Outline
•  Algebra
•  Number Theory
•  Combinatorics
Number Theory: Prime numbers
•  A natural number starting from 2: {2, 3, 4, . . .} is
considered as a prime if it is only divisible by 1 or itself.
•  The first (and the only even) prime is 2.
•  The next prime numbers are: 3, 5, 7, 11, 13, 17, 19, 23,
29, . . . , and infinitely many more primes.
•  There are 25 primes in range [0 . . . 100], 168 primes in
[0 . . . 1000], 1000 primes in [0 . . . 7919], 1229 primes in
[0 . . . 10000], etc...
Number Theory: Prime Testing Function
•  Test by definition.
•  Test if N is divisible by divisor ∈ [2 . . .N-1]
•  runs in O(N)
•  Improvement 1
•  Test if N is divisible by a divisor ∈ [2 . . .√N]
•  if N is divisible by p, then N = p X q. If q were smaller than p, then q or
a prime factor of q would have divided N earlier.
•  This is O(√N)
•  Improvement 2
•  test if N is divisible by divisor ∈ [3, 5, 7 . . .√N] (only test odd
numbers).
•  there is only one even prime number, i.e. number 2, which can be
tested separately.
•  This is O(√N/2)
Number Theory: Prime Testing Function
•  Improvement 3.
•  Test if N is divisible by prime divisors ≤√N
•  This is O(|#primes ≤√N|)
•  This improvement is already good enough for contest problems.
•  There are 500 odd numbers in [1 . . . √(106)], but there are only 168
primes in the same range.
•  This is O(√N/ln(√N)).
Generating List of Prime Numbers
•  Use the ‘Sieve of Eratosthenes’ algorithm.
•  First, it sets all numbers in the range to be ‘probably prime’ but set
numbers 0 and 1 to be not prime.
•  Then, it takes 2 as prime and crosses out all multiples of 2
•  Then it takes the next non-crossed number 3 as a prime and
crosses out all multiples of 3.
•  Then it takes 5 and crosses out all multiples of 5.
•  After that, whatever left uncrossed within the range [0 . . .N] are
primes.
•  This is roughly O(N log logN)
•  opt sieve for smaller primes and reserve optimized prime
testing function for larger primes
Finding Prime Factors
•  A composite numbers N, i.e. the non-primes, can be
written uniquely it as a multiplication of its prime factors.
•  N = 240 = 2 X 2 X 2 X 2 X 3 X 5 = 24 X 3 X 5 (the latter form is
called prime-power factorization).
•  Divide and Conquer Spirit:
•  An integer N can be expressed as: N = PF X N’, where PF is a
prime factor and N’ is another number which is N/PF – i.e. we can
reduce the size of N by taking out its factor PF
•  We can keep doing this until eventually N = 1. Special case if N is
actually a prime number.
•  This is O(π(√N)) = O(√N/ln√N).
Greatest Common Divisor (GCD)
•  The largest positive integer d such that d | a and d | b
where x | y implies that x divides y.
•  One practical usage of GCD is to simplify fraction, e.g.
•  Used very frequently in number theoretical problems.
•  Some facts:
•  gcd(a, b) = gcd(a, b − a)
•  gcd(a, 0) = a
GCD: Euclidian Algorithm
•  Repeated use of gcd(a, b) = gcd(a, b − a)
•  Example:
GCD: Euclidian Algorithm-Implementation
•  Running time: O(log(a + b))
LCM: Least Common Multiple.
•  The smallest positive integer l such that a | l and b | l
•  lcm(a, b) = a X b/gcd(a, b).
•  Implementation:
•  The GCD of more than 2 numbers, e.g. gcd(a, b, c) is
equal to gcd(a, gcd(b, c)), etc, and similarly for LCM.
•  Both GCD and LCM algorithms run in O(log10 n), where n
= max(a, b).
Number Theory: Relatively Prime
•  Two integers a and b are said to be relatively prime if
gcd(a, b) = 1
•  e.g. 25 and 42.
•  Problem of finding positive integers below N that are
relatively prime to N.
•  Use Euler’s Totient (Phi) function.
•  For example:
Number Theory: Solving Diophantine
Equation
•  David buys apples and oranges at a total cost of
8.39CAD. If an apple is 25 cents and an orange is 18
cents, how many of each type of fruit does David buys?
•  This problem can be modeled as a linear equation with
two variables: 25x + 18y = 839.
•  Since we know that both x and y must be integers, this
linear equation is called the Linear Diophantine Equation.
•  We can solve Linear Diophantine Equation with two
variables even if we only have one equation!
Number Theory: Solving Diophantine
Equation
•  Let a and b be integers with d = gcd(a, b). The equation
ax + by = c has no integral solutions if d | c is not true.
•  But if d | c, then there are infinitely many integral
solutions.
•  The first solution (x0, y0) can be found using the Extended
Euclid algorithm, and the rest can be derived from x = x0 +
(b/d)n, y = y0 − (a/d)n, where n is an integer.
Number Theory: Extended Euclid
Number Theory: Solving Diophantine
Equation
•  For our problem above: 25x + 18y = 839, we have:
•  a = 25, b = 18, extendedEuclid(25, 18) = ((−5, 7), 1), or
•  25 X (−5) + 18 X 7 = 1.
•  Multiplying the left and right hand side of the equation
above by 839/gcd(25, 18) = 839, we have:
•  25 X (−4195) + 18 X 5873 = 839. Thus,
•  x = −4195 + (18/1)n, y = 5873 − (25/1)n.
•  Since we need to have non-negative x and y, we have:
•  −4195 + 18n ≥ 0 and 5873 − 25n ≥ 0, or
•  4195/18 ≤ n ≤ 5873/25, or
•  233.05 ≤ n ≤ 234.92.
•  The only possible integer for n is 234.
Number Theory: Solving Diophantine
Equation
•  Thus
•  x = −4195 + 18 X 234 = 17 and y = 5873 − 25 X 234 = 23,
•  i.e. 17 apples (of 25 cents each) and 23 oranges (of 18
cents each) of a total of 8.39 CAD.
Number Theory: Fibonacci Numbers
•  Leonardo Fibonacci’s numbers are defined as fib(0) = 0,
fib(1) = 1, and fib(n) = fib(n − 1) + fib(n−2) for n ≥ 2.
•  This generates the following familiar patterns: 0, 1, 1, 2, 3,
5, 8, 13, 21, 34, 55, 89 which can be derived with an O(n)
DP technique.
•  (One of their properties) Zeckendorf’s theorem:
•  every positive integer can be written in a unique way as a sum of
one or more distinct Fibonacci numbers such that the sum does not
include any two consecutive Fibonacci numbers.
Number Theory: Fibonacci Numbers
•  Implementation 1 (recursive solution)
•  Exponential time complexity.
•  Implementation 2 (Dynamic Programming)
•  O(n) complexity.
Number Theory: Fibonacci Numbers
•  Implementation 3 (matrix exponentiation)
•  Use fast exponentiation to compute the matrix power