Download My Slides - Department of Computer Sciences

Theorem
Note
1
Quotation
The laws of nature are but the mathematical thoughts of God. – E UCLID OF A LEXANDRIA Greek mathematician
(circa 300 BC)
1
Discrete Math: Theory
Theoretical Computer Science’s Holy Grail
• Does P = NP?
• Whatever the meaning of the equation, the answer is either “yes” or “no”
• Most experts believe the answer is “no”
• But no one knows the correct answer
Decision Problems
Questions with Yes or No Answers
√
• Is 2 a rational number?
• Is there a path from here to there?
• Is this path longer than the that path?
• A decision problem P is a question such as these
Decision Problems
• Given a decision problem we may be able
– To solve it
– To know it cannot be solved
– Not know if it can be solved or not
Algorithms and Resources Consumed
Turing Machines
• A decision algorithm A is a well-defined process that computes the correct answer to a decision problem
• Decision algorithms are modeled by Turing machines
ζ α β γ
α
c
r
s
t
Turing Machines
• Turing machines concepts
– Current state of the machine, call it c
– Scanned input, call it α
– Transition t to the next state s, new tape symbol β , and read/write head move m
t(c, α) = (s, β , m)
• The transition rules are the algorithm!
Turing Machines
• Writing Turing machine algorithms (transition rules) is difficult!
• Writing algorithms in a mathematical language is less difficult
• Writing algorithms in a high-level programming language may be less difficult
• Writing algorithms in a natural language may be less difficult, but more prone to misinterpretation
“An abstraction is one thing that represents several real things equally well” – E. W. D IJKSTRA, Danish computer
scientist (1930–2002) As quoted in a letter from David Lorge Parnas, Communications of the ACM, June 2007, page
7
Complexity
• When algorithm A executes it consumes resources
time, space, network bandwidth, . . .
• Time is measured by counting the “elementary steps” A executes
T (n) ≥ steps needed to solve instance of size n
• The time complexity function T (n) is often a common function
T (n) = n,
T (n) = n2 ,
T (n) = log(n),
T (n) = 2n
Determinism and Nondeterminism
Determinism
• An algorithm can behave deterministically or nondeterministically
• An algorithm acts deterministically when all transitions are functions from the (current state, input symbol) pair
to the (next state, new symbol, move) triple
s
α →β
left
• Unless qualified, an “algorithm” is deterministic
2
c
Nondeterminism
• An algorithm acts nondeterministically when all transitions are relations from (current state, input symbol) pair
to one of several (next state, new symbol, move) triples
• Nondeterministic algorithms always transitions to a correct next state, if there is a “correct” one
s
α →β
c
left
α →γ
right
t
Polynomial Time
Time Complexity: Polynomial
• Algorithm A has polynomial time when its time complexity function T (n) is bounded above by polynomial
T (n) ≤ p(n)
for all large values of n
• When problem P has a polynomial time algorithm P is said to belong to the class P of deterministic, polynomial
time problems
• When problem P has a polynomial time non-deterministic algorithm P is said to belong to the class NP of
non-deterministic, polynomial time problems
Problems Can Solved or Checked
• Problems in class P can be solved in polynomial time
– A correct answer can be computed in a polynomial number of steps
• Problems in class NP can be checked in polynomial time
– Given an answer, it can be validated in a polynomial number of steps
The P = NP? Question
• The P = NP? question asks
Are problems that can be checked quickly also quickly solvable?
• If the answer is yes, then every problem that can be checked quickly can also be solved quickly
• If the answer is no, then some problem can be checked quickly, but it cannot be solved quickly
2
Discrete Math: Practice
Data Types
Practical Data Types
• Although solving the P = NP? question may be impractical, there are practical things to learn
• Sequences that change discretely, e.g.,
h0, 1, 1, 2, 3, 5, . . .i
3
• Sums that count totals, e.g.,
n−1
∑ 2k = 1 + 2 + 4 + · · · + 2n−1 = 2n − 1
k=0
• Sets that collect things, e.g.,
N = {0, 1, 2, 3, 4, . . .}
Practical Data Types
• Relations that nondeterministically map a thing to one or more things
DivisorOf(n)
• Functions that deterministically map a thing to another thing
log n
• Strings that encoding meaning, e.g., a protein sequence
MYLWLKLLAFGFAFLDTEVFVTGQSPTPSPTDAY. . .
• Number systems, a particular string encoding
3.14159265
Procedures
Practical Activities
• Besides learning about interesting data types, it is important to accomplish practical acts
• Solve problems using algorithms
• Reason logically
• Determine when a statement is true or false
3
Introduction to Seqeuences
Quotation
Recursive: adj. See: Recursive – S TAN K ELLY-B OOTLE, “Devil’s DP Dictionary” English computer scientist and
author (1929 –)
Definition of a Sequence
Definition 1 (Sequence). A sequence is an ordered list, which may contain repetitions
4
Common Sequences
• Alice sequence ~A = h1, 1, 1, 1, . . . , 1, . . .i
~ = h0, 1, 2, 3, 4, . . . , n, . . .i
• Gauss sequence G
~ = h0, 1, 3, 7, 15, . . . , 2n − 1, . . .i
• Mersenne sequence M
E
D
• Fibonacci sequence ~F = 0, 1, 1, 2, 3, . . . , √15 φ n − φ̂ n , . . .
where
√
1+ 5
φ=
≈ 1.61803
2
and
φ̂ =
√
1− 5
≈ −0.61803
2
is the golden ratio
is the golden ratio’s conjugate
Sequence Notation
• Terms in a sequence are written using subscript notation, for example, a generic sequence can be written
~S = hs0 , s1 , s2 , s3 , . . . , sn , . . .i
• It is okay to write sequences using function notation
~S = hs(0), s(1), s(2), s(3), . . . , s(n), . . .i
Usefulness of Sequences
• Terms in a sequences count or compute “something”
• The On-Line Encyclopedia of Integer Sequences (OEIS) contains over 100, 000 integer sequences!
• We’ll only study an important few
4
Computing Terms in a Sequence
Quotation
Some archeologists believe that Stonehenge — the mysterious arrangement of enormous
elongated stones in England — is actually a crude effort by the Druids to build a computing device
Dave Barry, American writer and humorist (1947 –)
Computing Terms in a Sequence
• Sometimes the terms in sequences can be computed by
– Evaluating a function
– Solving a recurrence equation
– Executing an algorithm
• Sometimes the terms in sequences can be defined but cannot be computed
• Most often terms in sequences cannot even be described
5
Sequence Terms Computed by a Function
• Sometimes the terms in sequences can be computed by evaluating a function
• The Alice sequence: for all natural numbers n
an = 1
• The Gauss sequence: for all natural numbers n
gn = n
• The sequence of triangular numbers: for all natural numbers n
tn = n(n − 1)/2
Sequence Terms Computed by Recursion
• Sometimes the terms in sequences can be computed by solving a recurrence
• The factorial sequence
– Initial condition: 0! = 1
– Recurrence equation:
n! = n(n − 1)!,
n = 1, 2, 3, . . .
1
Hn = Hn−1 + ,
n
n = 1, 2, 3, . . .
• The harmonic sequence
– Initial condition: H0 = 0
– Recurrence equation:
Sequence Terms Computed Algorithmically
• Sometimes the terms in sequences can be computed by executing an algorithm
• The sequence of prime numbers:
~P = h2, 3, 5, 7, 11, . . .i
• The divisor sequence:
~D = h1, 2, 2, 3, 2, . . .i
which counts the number of divisors of the positive integer sequence
h1, 2, 3, 4, 5, . . .i
Sequences Whose Terms Cannot Be Computed
• Many sequences can be described but cannot be computed
• The halting sequence gives the number of steps Turing machine Tk executes on input I j
– If machine Tk does not halt on input I j , then define the number of steps to be the symbol ∞
• The busy beaver sequence gives the maximal number of steps an n state Turing machine can make on an initially
blank tape and halt
• Most sequences cannot even be described!
6
5
Important Sequences
Quotation
He was earnest about these objects. They were of eternal importance, like baseball or the
Republican Party.
Sinclair Lewis, “Babbitt” American author (1885 –1951)
Important Sequences
• There are many sequences that are important in computing, mathematics, science, and engineering
• A partial list of important sequences is presented here
Important Sequences: Alice and Gauss
• The Alice sequence
~A = h1, 1, 1, 1, 1, 1, 1, 1, . . .i
• Terms in the Alice sequence are computed by the function
∀n ∈ N = {0, 1, 2, 3, 4, . . .}
an = 1
• The Gauss sequence
~ = h0, 1, 2, 3, 4, 5, 6, 7, 8, . . .i
G
• Terms in the Gauss sequence are computed by the function
gn = n
∀n ∈ N
Important Sequences: Triangular and Mersenne
• The sequence of triangular numbers
~T = h0, 0, 1, 3, 6, 10, 15, 21, 28, 36, . . .i
• Terms in the triangular sequence are computed by the function
tn =
n(n − 1)
2
∀n ∈ N = {0, 1, 2, 3, 4, . . .}
• The Mersenne sequence
~ = h0, 1, 3, 7, 15, 31, 63, 127, 255, . . .i
M
• Terms in the Mersenne sequence are computed by the function
mn = 2n − 1
∀n ∈ N
Important Sequences: Factorial and Fermat
• The factorial sequence
~n! = h1, 1, 2, 6, 24, 120, 720, 5040, . . .i
• Terms in the factorial sequence can be computed by the recursion
n! = n(n − 1)! ∀n ∈ Z+ = {1, 2, 3, 4, . . .},
0! = 1
• The Fermat sequence
~R = h3, 5, 17, 257, 65537, 4294967297, . . .i
• Terms in the Fermat sequence are computed by the function
n
rn = 22 + 1
7
∀n ∈ N
Important Sequences: Fibonacci and Prime
• The Fibonacci sequence
~F = h0, 1, 1, 2, 3, 5, 8, 13, 21, . . .i
• Terms in the Fibonacci sequence can computed by the recursion
fn = fn−1 + fn−2 ,
n ∈ {2, 3, 4, . . .},
f0 = 0, f1 = 1
• The sequence of prime numbers
~P = h2, 3, 5, 7, 11, 13, 17, 19, . . .i
• There is no simple function that computes the primes, but they can be computed algorithmically
Important Sequences: Relation Counts and Zeno
• The relation count sequence
~L = h1, 2, 16, 512, 65536, 33554432, . . .i
• Terms in the relation count sequence are computed by the function
2
ln = 2n
∀n ∈ N
• A Zeno sequence
~Z = h1, 2, 4, . . . , 2n , . . .i
• Terms in a Zeno sequence are computed by the recursion
∀n ∈ Z+ ,
zn = 2zn−1 ,
z0 = 1
Important Sequences: Binomial and Catalan
• Some sequences are two-dimensional
• The sequence of binomial coefficient
~B = h1 ←- 1, 1 ←- 1, 2, 1 ←- 1, 3, 3, 1 ←- . . .i
• Terms in the binomial sequence can be computed by the expression
n
n!
=
∀n, m ∈ N, n ≥ m
m
m!(n − m)!
• The Catalan sequence is one-dimensional
~ = h1, 1, 2, 5, 14, 42, 132, 439, . . .i
C
• Terms in the Catalan sequence are computed using binomial coefficients
1
2n
cn =
∀n ∈ N
n+1 n
8
Important Sequences: Stirling of the First and Second Kind
• Stirling numbers are other examples of two-dimensional sequences
• Stirling numbers of the first kind
~S1 = h1 ←- 0, 1 ←- 0, 1, 1 ←- 0, 2, 3, 1 ←- 0, 6, 11, 6, 1 ←- . . .i
• Stirling numbers of the first kind can be computed by
n−1
n−1
n
=
+ (n − 1)
, n>0
m−1
m
m
• Stirling numbers of the second kind
~S2 = h1 ←- 0, 1 ←- 0, 1, 1 ←- 0, 1, 3, 1 ←- 0, 1, 7, 6, 1, ←- . . .i
• Stirling numbers of the second kind can be computed by
n
n−1
n−1
=
+m
, n>0
m
m−1
m
Important Sequences: Harmonic Numbers
• Sequences of rational numbers can also be important
• The sequence of harmonic numbers
~ = 0, 1 , 3 , 11 , 25 , 137 , . . .
H
1 2 6 12 60
• Terms in the sequence of harmonic numbers can computed by the recursion
1
∀n ∈ Z+ ,
n
(it is traditional to use capital “H” for the harmonic numbers)
Hn = Hn−1 +
H0 = 0
Arithmetic Sequences
• The Alice and Gauss sequences are examples of arithmetic sequences
• An arithmetic sequence has a slope m and a y-intercept c
• An arithmetic sequence is determined by the function
sk = mk + c, quadk = 0, 1, . . .
• The terms in a general arithmetic sequence are
hc, m + c, 2m + c, 3m + c, . . .i
Geometric Sequences
• The Zeno sequence is an example of a geometic sequence
• A geometic sequence has a ratio r and a y-intercept c
• An geometic sequence is determined by the function
sk = crk , quadk = 0, 1, . . .
• The terms in a general geometic sequence are
c, cr, cr2 , cr3 , . . .
9
Arithmetic and Geometric Sequences
• Consider the sequences
Arithmetic:
Geometic:
0
1
1 2
r r2
3
r3
4
r4
5
r5
6
r6
···
···
• Addition of terms in the arithmetic sequence corresponds to multiplication of terms in the geometric sequence
n + m ↔ rn rm = rn+m
• Subtraction of terms in the arithmetic sequence corresponds to division of terms in the geometric sequence
n − m ↔ rn /rm = rn−m
Arithmetic and Geometric Sequences
Arithmetic: 0
Geometic: 1
1 2
r r2
3
r3
4
r4
5
r5
6
r6
···
···
• Multiplication of terms in the arithmetic sequence corresponds to raising to a power in the geometric sequence
nm ↔ (rn )m = rnm
• Division of terms in the arithmetic sequence corresponds to extraction of roots in the geometric sequence
n/m ↔ (rn )1/m = rn/m
• This correspondence between opertations on terms between arithmetic and geometric sequences foreshadows
logarithm and exponential functions
6
Recursively Defined Sequences
Quotation
recursion 86, 139, 141, 182, 202, 269
Kernighan and Ritchie, “The C Programming Language” index entry on page 269 American
computer scientists (1942–) and (1941–)
Recursively Defined Sequences
• A sequence has a recursive definition when the next term sn is computed from the current term sn−1 , and perhaps
previous terms
• Recursively defined sequences require initial condition(s), which oddly, are also called terminating condition(s)
The Alice Recursion
• The Alice sequence
~A = h1, 1, 1, 1, . . . , 1, . . .i
is for simple counting or tallying
• The next term in the Alice sequence is the current term
an = an−1 ,
• The initial condition is
a0 = 1
10
n≥1
The Gauss Recursion
• The Gauss sequence
~ = h0, 1, 2, 3, . . . , n, . . .i
G
sums terms in the Alice sequence
• The next term in the Gauss sequence is 1 more than the current term
gn = gn−1 + 1,
n≥1
• The initial condition is
g0 = 0
The Triangular Recursion
• The sequence of triangular numbers
~T = 0, 0, 1, 3, 6, . . . , n(n − 1) , . . .
2
sums terms in the Gauss sequence
• The next term in the triangular sequence is n − 1 more than the current term
tn = tn−1 + (n − 1),
n≥1
• The initial condition is
t0 = 0
The Triangular Recursion
• Successive terms in the triangular sequence can be computed using the recurrence
tn = tn−1 + (n − 1),
t0 = 0
t1 = t0 + 0 = 0 + 0 = 0
t2 = t1 + 1 = 0 + 1 = 1
t3 = t2 + 2 = 1 + 2 = 3
t4 = t3 + 3 = 3 + 3 = 6
t5 = t4 + 4 = 6 + 4 = 10
t6 = t5 + 5 = 10 + 5 = 15
Polynomial Growth Rates
• Terms in the Alice, Gauss, and triangular sequences grow at polynomial rates
• Constant polynomials have the form
p(n) = a
• Linear polynomials have the form
p(n) = an + b
• Quadratic polynomials have the form
p(n) = an2 + bn + c
• Polynomials of degree k have the form
p(n) = ak nk + ak−1 nk−1 + · · · + a1 n + a0
11
Generalized Alice Sequences
• Terms in the Alice sequence are constant, they do not grow
• A generalized Alice sequence is
~S = ha, a, a, a, . . .i
• This general Alice sequence has recurrence equation
sn = sn−1 ,
n≥1
• And it has initial condition
s0 = a
Generalized Gauss Sequences
• Terms in the Gauss sequence grow linearly
• A generalized Gauss sequence is
~S = hb, a + b, 2a + b, 3a + b, . . .i
• This general Gauss sequence has recurrence equation
n≥1
sn = sn−1 + a,
• And it has initial condition
s0 = b
• Sequences of this form are called arithmetic
Generalized Triangular Sequences
• Terms in the triangular sequence grow quadraticly
• A generalized triangular sequence is
~S = hc, a + b + c, 4a + 2b + c, 9a + 3b + c, . . .i
• This general triangular sequence has recurrence equation
sn = sn−1 + a(2n − 1) + b,
n≥1
• And it has initial condition
s0 = c
The Zeno Recursion
• Many things grow at a rate that is faster than polynomial
• Terms in the Zeno sequence double at each step
~Z = h1, 2, 4, . . . , 2n , . . .i
• The next term in the Zeno sequence is twice the current term
zn = 2zn−1 ,
• The initial condition is
z0 = 1
12
n≥1
The Mersenne Recursion
• The Mersenne sequence
~ = h0, 1, 3, 7, . . . , 2n − 1, . . .i
M
sums terms in the Zeno sequence
• The next term in the Mersenne sequence is twice the current term plus 1
mn = 2mn−1 + 1,
n≥1
• The initial condition is
m0 = 0
The Mersenne Recursion
• Successive terms in the Mersenne sequence can be computed using the recurrence
mn = 2mn−1 + 1,
m0 = 0
m1 = 2m0 + 1 = 2 · 0 + 1 = 1
m2 = 2m1 + 1 = 2 · 1 + 1 = 3
m3 = 2m2 + 1 = 2 · 3 + 1 = 7
m4 = 2m3 + 1 = 2 · 7 + 1 = 15
m5 = 2m4 + 1 = 2 · 15 + 1 = 31
m6 = 2m5 + 1 = 2 · 31 + 1 = 63
The Fibonacci Recursion
• The Fibonacci sequence
~F = h0, 1, 1, 2, 3, 5, . . . , fn , . . .i
occurs in many fanciful applications
• The next term in the Fibonacci sequence is the sum of the current term and the previous term
fn = fn−1 + fn−2 ,
n≥2
• The initial conditions are
f0 = 0,
f1 = 1
The Fibonacci Recursion
• Successive terms in the Fibonacci sequence can be computed using the recurrence
fn = fn−1 + fn−2 ,
f0 = 0, f1 = 1
f2 = f1 + f0 = 1 + 0 = 1
f3 = f2 + f1 = 1 + 1 = 2
f4 = f3 + f2 = 2 + 1 = 3
f5 = f4 + f3 = 3 + 2 = 5
f6 = f5 + f4 = 5 + 3 = 8
13
Faster Than Polynomial Growth
• The Zeno, Mersenne, and Fibonacci sequence grow exponentially
• The Zeno and Mersenne sequences grow like 2n
zn = 2n
mn = 2 n − 1
• The Fibonacci sequence grows like φ n
fn ≈ φ n =
• The factorial sequence grows like
√
2πn
√ !n
1+ 5
2
n n
e
Slower Than Polynomial Growth
• Polynomial grow is called tractable
• It is easy to deal with a tractable problem
• Exponential grow is called intractable
• It is hard to deal with a intractable problem
• Fortunately, some sequences grow slower than the rate of a polynomial
• The Harmonic number sequence grows like ln(n)
7
Sequence Applications
Quotation
Every natural science always involves three things: the sequence of phenomena on which the
science is based; the abstract concepts which call these phenomena to mind; and the words in
which the concepts are expressed.
Antoine Laurent Lavoisier, “Traité Elémentaire de Chimie” French chemist (1743 – 1794)
Applications of Sequences and Recursion
• There are many sequence applications
• Root finding algorithms are numerical techniques to solve equations such as
f (x) = 0
where f is a function
• Interest calculations on loans or annuities are at the heart of financial applications
14
Newton’s Method
• A sequence known
√ to ancient mathematicians of Mesopotamia, the region of modern day Iraq and Iran, shows
how to compute 2
• Clay tablets from there, dated from around 1800 B.C. to 1600 B.C., have been found contain this sequence
• Consider an isosceles right triangle with legs of length 1
h
1
1
Newton’s Method
• By the Pythagorean theorem
12 + 12 = h2
√
2 ≈ 1.41421356 · · ·
√
• The Babylonians’ knew how to approximate the value of h = 2 to many decimals places (they used sexagesimal
notation)
• Therefore the hypotenuse h has length h =
• The method employed has been generalized and is now called Newton’s method
Newton’s Method
• Although it is not certain what reasoning the Babylonians’ used, their calculations indicate this is what they did:
√
– Let h0 = 1 be an initial approximation to 2
– Clearly 1 is too small as the Babylonians’ could easily measure
√
√ √
√
– If 1 = 2, then 1 · 1 = 2 · 2 = 2 and 2/1 = 2
– As it is, 2/1 is too large
– The average of the under estimate 1 and the over estimate 2/1 provides a better approximation to
this
1
2
1
2
3
h1 =
h0 +
=
1+
=
2
h0
2
1
2
Newton’s Method
• But h1 = 3/2 is too large as the Babylonians’ could measure
• If 3/2 were the exact square root, then 2/(3/2) = 4/3 would equal
√
2
• As it is, 4/3 ≈ 1.333 · · · is too small
• The average of the over estimate 3/2 and the under estimate 4/3 will provide a better approximation
1
2
1 3 4
17
h2 =
h1 +
=
+
=
≈ 1.41166 · · ·
2
h1
2 2 3
12
• But h2 = 17/12 is too small
15
√
2, call
Newton’s Method
• The Babylonians’ carried out this iteration a few more times computing the
places
√
2 accurately to at least 9 decimal
• That is, they next computed the average of h2 = 17/12 and 2/h2 = 24/17
1
2
h3 =
h2 +
2
h2
1 17 24
=
+
2 12 17
1 289 + 288
=
2
17 × 12
≈ 1.41421568628 · · ·
Newton’s Method
• Successive terms in Newton sequence can be computed using the recurrence
2
1
hn−1 +
, h0 = 1
hn =
2
hn−1
• The sequence hh0 , h1 , h2 , h3 , . . .i converges rapidly to
√
2
• To generalize, the square root of a can be computed from the recurrence
1
a
hn =
hn−1 +
, h0 = 1
2
hn−1
• Even more generally, the Newton sequence is
xn = xn−1 −
f (xn−1 )
f 0 (xn−1 )
where f (x) is a function, f 0 (x) is its derivative, and some initial value x0 is given
Compound Interest
• Pretend P dollars are deposited into an account every period for n periods
• For example, suppose P = 100 dollars are deposited every month for 40 years, that is n = 480 periods
• Pretend the amount in the account earns interest at a rate r each period
• We can describe the value of the account at the beginning of each period
Amount deposited + Previous value + Interest
Compound Interest
• Let Pn denote the value of the account at the beginning of period n
• That is,
Pn = P + Pn−1 + rPn−1
= P + (1 + r)Pn−1
(1 + r)n+1 − 1
=P
r
16
• For example, with P = 100, n = 480, and an interest rate of 1 percent (r = 0.01) the account will have value
(1.01)481 − 1
P480 = 100
(0.01)
= $1, 188, 342.02
Compound Interest: Formula
• Let’s show that
(1 + r)n+1 − 1
Pn = P
r
correctly computes the value of the account at the beginning of period n
• We show the formula is correct by
– Showing the formula is correct for the first value of n, that is, n = 0
– Showing that if the formula is correct for some value of n, then is it correct for n + 1, the next value
Compound Interest: Formula Initially Correct
• For the first period n = 0 and using the formula we find
(1 + r)0+1 − 1
P0 = P
r
(1 + r) − 1
=P
r
=P
Compound Interest: Formula Initially Correct
• As a check, for the second period n = 1 and using the formula we find
(1 + r)1+1 − 1
P1 = P
r
(1 + 2r + r2 ) − 1
=P
r
= P(2 + r)
= P + P + rP
which is also the correct value
Compound Interest: Inductive Step
• Now, let’s pretend that
(1 + r)n+1 − 1
Pn = P
r
correctly computes the value of the account at the beginning of period n to show it is correct at period n + 1
17
• For period n + 1 the value of the account is
Pn+1 = P + Pn + rPn
(1 + r)n+1 − 1
(1 + r)n+1 − 1
= P+P
+ rP
r
r
n+1
n+1
r + [(1 + r)
− 1] + r[(1 + r)
− 1]
=P
r
n+2
(1 + r)
−1
=P
r
which is matches formula
To Know More. . .
References
[Sloane, 2004] Sloane, N. J. A. (2004). On-Line Encyclopedia of Integer Sequences. Published electronically at
http://www.research.att.com/%7Enjas/sequences.
[Wikipedia, 2007] Wikipedia (2007).
The Free Encyclopedia:
http://en.wikipedia.org/wiki/Main_Page.
8
Sequences.
Published electronically at
Introduction to Summations
Quotation
We will occasionally use this arrow notation unless there is danger of no confusion. – RONALD G RAHAM,
"Rudiments of Ramsey Theory" American computer scientist, mathematician (1935 - )
Sums
• Addition is a fundamental arithmetic operation and we’ll want ways to denote and compute sums
• For example, it can be useful to add the natural numbers
0=0
0+1 = 1
0+1+2 = 3
0+1+2+3 = 6
Summation Notation
• Writing sums with a few terms is easy
• But what there there are a million terms in the sum?
• We could write
1 + 2 + 3 + · · · + 999, 999 + 1, 000, 000
• But even writing this semi-explicit form is tedious
18
Summation Notation
• We’ll work with sums of the form
a0 + a1 + · · · + an−1
• This sum can be expressed as
“The sum of a sub 0 plus a sub 1 plus up to a sub n − 1”
• Or by parameterizing the subscript we can say
“The sum of ak from k = 0 to n − 1”
• The “sum of ak from k = 0 to n − 1” is denoted
n−1
∑ ak
k=0
Summation Notation
• Parsing summation notation
n−1
∑ ak
k =0
• The summation symbol capital Greek sigma ∑
• The index of summation k
• The lower limit of the sum 0
• The upper limit of the sum n − 1
• The terms being added ak , k = 0, . . . , n − 1
• The implicit index set over which the index k ranges is {0, 1, 2, . . . , n − 1}
Summation Notation
• For n = 1, the sum is
1−1
n−1
∑ ak = ∑ ak = a0
k=0
k=0
• For n = 2, the sum is
2−1
n−1
∑ ak = ∑ ak = a0 + a1
k=0
k=0
• For n = 3, the sum is
3−1
n−1
∑ ak = ∑ ak = a0 + a1 + a2
k=0
k=0
Summation Notation
• For n < 1, the sum is empty and defined to be 0
• For example, for n = 0, the sum
0−1
n−1
∑ ak = ∑ ak = 0
k=0
k=0
19
Summation Notation
• Sums over index sets that are not consecutive integers can also be expressed
• For example, to denote the sum of the odd integers from 1 to 100 we could write
∑
k = 1 + 3 + 5 + · · · + 99
1≤k≤100
k odd
• For the sum of odd integers, we could sum over consecutive integers by writing
49
∑ (2k + 1) = 1 + 3 + 5 + · · · + 99
k=0
• To express the sum of the reciprocals of the primes less than n we can write
∑
p<n
1
p
p prime
Summation Is A Linear Operator
• The summation operator is linear, that is
n−1
n−1
n−1
∑ (cak + bk ) = c ∑ ak + ∑ bk
k=0
k=0
k=0
• Linearity can be useful in evaluating summations
• For example, if we knew
– How to add 1 from k = 0 to n − 1, and
– How to add k from k = 0 to n − 1
we could develop a formula for adding the odd integers
n−1
n−1
n−1
∑ (2k + 1) = 2 ∑ k + ∑ 1
k=0
9
k=0
k=0
Partial Sums of Sequences
Quotation
When an algorithm contains an iterative control construct such as a while or for loop, its running time can be
expressed as (a) sum – RONALD G RAHAM, "Rudiments of Ramsey Theory" American computer scientist,
mathematician (1935 - )
Partial Sums of Sequences
20
• Given a sequence ~S = ha0 , a1 , a2 , a3 , a4 , . . . , an , . . .i it is possible to form another sequence of partial sums
0
S0 =
∑ ak = a0
k=0
1
S1 =
∑ ak = a0 + a1
k=0
2
S2 =
∑ ak = a0 + a1 + a2
k=0
..
.
..
.
=
n
Sn =
∑ ak = a0 + a1 + a2 + · · · + an
k=0
Partial Sums and Recursion
• Let ~A = ha0 , a1 , a2 , a3 , a4 , . . . , an , . . .i be a sequence
• Consider the partial sum
n−1
Sn−1 =
∑ ak = a0 + a1 + a2 + · · · + an−1
k=0
• This sum is equivalent to the recurrence
S0 = a0
Sn = Sn−1 + an
Sample Sequences
• What are the partial sums of these sample sequences?
– Alice sequence
~A = h1, 1, 1, 1, . . . , 1, . . .i
– Fibonacci sequence
~F = 0, 1, 1, 2, 3, . . . , √1 φ n − φ̂ n , . . .
5
– Gauss sequence
~ = h0, 1, 2, 3, 4, . . . , n, . . .i
G
– The harmonic sequence
~h = 0, 1 , 1 , 1 , 1 , . . . , 1 , . . .
1 2 3 4
n
– Zeno sequence
~Z = h1, 2, 4, 8, 16, . . . , 2n , . . .i
Partial Sums of Alice and Gauss Sequences
• Alice sum: “the sum of 1 from 0 to k = n − 1 is n”
n−1
∑ 1 = 1| + 1 + 1{z+ · · · + 1} = n
k=0
n terms
• Gauss sum: “the sum of k from 0 to n − 1 is n(n − 1)/2”
n−1
∑ k = 0 + 1 + 2 + 3 + · · · + (n − 1) =
k=0
21
n(n − 1)
2
Partial Sums of Triangular and Fibonacci Sequences
• Triangular sum: “the sum of k(k − 1)/2 from 0 to n − 1 is n(n − 1)(n − 2)/6”
n−1
(n − 1)(n − 2) n(n − 1)(n − 2)
k(k − 1)
= 0+0+1+···+
=
2
2
6
k=0
∑
• Fibonacci sum: “the sum of fk from 0 to k = n − 1 is fn+2 − 1”
n−1
∑ fk = 0 + 1 + 1 + · · · + fn−1 = fn+1 − 1
k=0
Partial Sums of Zeno and Mersenne Sequences
• Zeno sum: “the sum of 2k from 0 to k = n − 1 is 2n − 1”
n−1
∑ 2k = 1 + 2 + 4 + · · · + 2n−1 = 2n − 1
k=0
• Mersenne sum: “the sum of 2k − 1 from 0 to k = n − 1 is 2n − (n + 1)”
n−1
∑ (2k − 1) = 0 + 1 + 3 + · · · + (2n−1 − 1) = 2n − (n + 1)
k=0
Partial Sums of the Harmonics
• The harmonics are the sequence of fractions
~h = 0, 1 , 1 , 1 , 1 , . . . , 1 , . . .
1 2 3 4
n
• The harmonic 1/n is the frequency of an anchored string plucked so as to have n waves
Partial Sums of the Harmonics
• Partial sums of harmonics are called Harmonic numbers
0
H0 =
k=1
1
H1 =
1
1
1
1
1
1
1
1
1
∑ k = 1+2
k=1
3
H3 =
1
∑k=1
k=1
2
H2 =
1
∑ k =0
1
∑ k = 1+2+3
k=1
..
.
..
.
=
n
Hn =
1
1
∑ k = 1 + 2 +···+ n
k=1
22
Harmonic Numbers
• There is no simple function that evaluates the Harmonic number
n
Hn =
1
1
1
1
∑ k = 1 + 2 +···+ n
k=1
• The harmonic numbers are well approximated by the natural logarithm
• Specifically,
n
Hn =
1
∑ k = ln n + γ + O(1/2n)
k=1
where γ ≈ 0.5772156649 . . . is the Euler-Mascheroni constant
10
Arithmetic Sums
Epigraph
‘Can you do Addition?’ the White Queen asked. ‘What’s one and one and one and one and one
and one and one and one and one and one?’ ‘I don’t know,’ said Alice. ’I lost count.’ ‘She can’t
do Addition,’ the Red Queen interrupted. ‘Can you do Subtraction? Take nine from eight.’
Lewis Carroll, "Alice’s Adventures in Wonderland" English author, logician, clergyman (1832 1898)
Linear Functions
• A linear function is given by the formula
y = f (x) = mx + b
where
– m is called the slope of the line; and
– b is the y-intercept
13
11
9
7
5
3
1
y = 2x + 1
0 2 4 6 8
Restricting the Domain to the Natural Numbers
• Let x be restricted to the natural numbers N = {0, 1, 2, 3, . . .} in the formula y = ax + b to generate an arithmetic
sequence
23
13
11
9
7
5
3
1
y = 2x + 1, x ∈ Z
0 2 4 6 8
Arithmetic Sequence: Definition
Definition 2 (Arithmetic Sequence). The linear function
y = f (x) = mx + b
with slope m and y-intercept b maps a real number x to a real number y and generates the arithmetic sequence
hb, m + b, 2m + b, 3m + b, 4a + b, . . .i
Arithmetic Sequence: Examples
• The odd natural numbers are generated by the linear function
f (n) = 2n + 1
2(0) + 1 = 1
2(1) + 1 = 3
2(2) + 1 = 5
• It is more efficient to use coherence to generate the terms of the sequence
• That is, add the slope 2 to the previous term to compute the next term
2(0) + 1 = 1
1+2 = 3
3+2 = 5
Arithmetic Sequence: Examples
• The Alice sequence is generated by the linear function
f (n) = 0 · n + 1 = 1
0(0) + 1 = 1
0(1) + 1 = 1
0(2) + 1 = 1
0(3) + 1 = 1
• Using coherence
0(0) + 1 = 1
1+0 = 1
1+0 = 1
1+0 = 1
Arithmetic Sequence: Examples
• The natural (Gauss) numbers are generated by the linear function
f (n) = 1 · n + 0 = n
1(0) + 0 = 0
1(1) + 0 = 1
1(2) + 0 = 2
1(3) + 0 = 3
• Using coherence
1(0) + 0 = 0
0+1 = 1
24
1+1 = 2
2+1 = 3
Arithmetic Sequence: Examples
• The generic arithmetic sequence is generated by the linear function
f (n) = m · n + b
mn + b
(mn + b) + m = 2m + b
(2m + b) + m = 3m + b
Arithmetic Sum Formula
• The sum of terms in an arithmetic sequence equals the number of terms times the average of the first and last
term
• Consider the sum
6
∑ (2k + 1)
k=0
– This sum has 6 − 0 + 1 = 7 terms
– The first term in this sum is 2(0) + 1 = 1
– The last term in this sum is 2(6) + 1 = 13
– The average of the first and last term is (1 + 13)/2 = 7
– The value of the sum is
6
∑ (2k + 1) = 7
k=0
1 + 13
2
= 49
Alice Sum Formula
• The Alice sums are
n−1
∑ 1 = 1| + 1 + 1{z+ · · · + 1}
k=0
n terms
= number of terms × average of first and last
1+1
= n×
2
=n
• The Alice sums generate terms in the Gauss sequence gn = n
General Alice
• Alice sequences and sums can be generalized
• A constant sequence is
hb, b, b, b, · · ·i
• The partial sum of terms in a constant sequence is
n−1
∑ b = b| + b + b{z+ · · · + b}
k=0
n terms
= number of terms × average of first and last
b+b
= n×
2
= bn
25
General Alice
• The partial sum of terms in a constant sequence can be generalized by changing the lower and upper summation
limits
n−1
∑ b = b| + b + b{z+ · · · + b}
k=m
n−m terms
= number of terms × average of first and last
b+b
= (n − m) ×
2
= b(n − m)
Gauss Sum Formula
• The Gauss sums are
n−1
∑ k = 0| + 1 + 2 +{z· · · + (n − 1)}
k=0
n terms
= number of terms × average of first and last
0 + (n − 1)
= n×
2
n(n − 1)
=
2
• The Gauss sums generate terms in the sequence of triangular numbers tn = n(n − 1)/2
General Gauss
• Gauss sequences and sums can be generalized
h0, a, 2a, 3a, · · ·i
• The partial sum of terms in this sequence is
n−1
∑ ak = 0| + a + 2a +{z· · · + a(n − 1)}
k=0
n terms
= number of terms × average of first and last
0 + a(n − 1)
= n×
2
n(n − 1)
=a
2
General Gauss
• For example, the partial sums of the sequence
h0, 5, 10, 15, · · ·i
is
n−1
∑ 5k = 5
k=0
26
n(n − 1)
2
General Gauss
• The partial sum of terms in a Gauss sequence can be generalized by changing the lower and upper summation
limits
n−1
+ a(m + 1) + a(m + 2) + · · · + a(n − 1)
∑ ak = am
{z
}
|
k=m
n−m terms
= number of terms × average of first and last
am + a(n − 1)
= (n − m) ×
2
(n − m)(n + m − 1)
=a
2
• For example, the partial sum
20
∑
5k = 5
k=−6
27(14)
2
Arithmetic Sum Formula
• The generic arithmetic sums are
n−1
∑ (ak + b) = b| + (a + b) + (2a + b){z+ · · · + [a(n − 1) + b]}
k=0
n terms
= number of terms × average of first and last
b + [a(n − 1) + b]
= n×
2
n(an − a + 2b)
=
2
General Arithmetic Sum Formula
• The partial sum of terms in a arithmetic sequence can be generalized by changing the lower and upper summation
limits
n−1
+ [a(n − 1) + b]
∑ (ak + b) = |[am + b] + · · · {z
}
k=m
n−m terms
= number of terms × average of first and last
[am + b] + [a(n − 1) + b]
= (n − m) ×
2
(n − m)(a(n + m − 1) + 2b
=
2
• For example, the partial sum
20
∑ (5k + 3) =
k=−6
27
27(14)
2
Summary: Arithmetic Sequences
• The Alice sequence
h1, 1, 1, 1, · · · , 1, · · ·i
• The Gauss sequence
h0, 1, 2, 3, 4, · · · , n, · · ·i
• Generic arithmetic sequence
hb, m + b, 2m + b, 3m + b, · · · , am + b · · ·i
• Successive values in an arithmetic sequence differ by a constant, the slope m
Summary: Arithmetic Sums
• The Alice sums
n
n
∑ ak = ∑ 1 = n + 1
k=0
• The Gauss sums
k=0
n−1
n−1
∑ gk = ∑ k =
k=0
k=0
n(n − 1)
2
Summary: Arithmetic Recurrences
• The Alice recurrence and initial condition
n = 0, 1, 2, · · · ,
an+1 = an ,
a0 = 1
• The Gauss recurrence and initial condition
gn+1 = gn + 1,
n = 0, 1, 2, · · · ,
g0 = 0
• A generic arithmetic recurrence and initial condition
sn+1 = sn + a,
11
n = 0, 1, 2, · · · ,
s0 = b
Geometric Sums
Epigraph
There is still a difference between something and nothing, but it is purely geometrical and there is
nothing behind the geometry.
Martin Gardner, “The Mathematical Magic Show” American logician, mathematician, puzzle
constructor (1914 – )
Exponential Functions
• An exponential function is given by the formula
y = f (x) = arx
where
– r is called the base or ratio
– a is the y-intercept
• Common bases are r = 2, r = e ≈ 2.718, and r = 10
28
Geometric Sequence: Definition
Definition 3 (Geometic Sequence). The exponential function
y = f (x) = arx
with ratio r and y-intercept a maps the natural numbers to a geometric sequence
D
E
a, ar, ar2 , ar3 , ar4 , ar5 , . . .
The Zeno Sequence
• The powers of 2 are generated by the exponential function
f (n) = 2n
20 = 1
21 = 2
22 = 4
23 = 8
24 = 16
25 = 32
26 = 64
The Powers of Two
• Powers of 2 are used to name amounts of storage
• Some use powers of 10, but I believe powers of 2 are the default
– Kilobyte KB, 210 = 1024 bytes = 8196 bits
– Megabyte MB, 220 = 1 048 576 bytes
– Gigabyte GB, 230 = 1 073 741 824 bytes
– Terabyte TB, 240 = 1 099 511 627 776 bytes
– Petabyte PB, 250 = 1 125 899 906 842 624 bytes
• Computers are also rated by how many instructions they can execute per second
– Teraflops: 240 floating point operations per seconds
– MIPS: 220 instructions per second
The Exponential
• The constant e is the base of the natural logarithm
• e is known as Napier’s constant, but the symbol e honors Euler
• The numerical value of e is approximately
e ≈ 2.718 . . .
29
The Exponential
• The value of e can be defined by the integral
Z e
dx
x
1
= ln e = 1
• The value of e can be defined by the limit
1 x
e = lim 1 +
x→∞
x
• The value of e can be defined by the infinite series
∞
e=
1
∑ k!
k=0
• One of the most beautiful and interesting identities in mathematics is
eiπ + 1 = 0
Factoring Polynomials
• You know that
x2 − 1 = (x − 1)(x + 1) or
x2 − 1
= x+1
x−1
• Perhaps you know that
x3 − 1 = (x − 1)(x2 + x + 1) or
x3 − 1
= x2 + x + 1
x−1
• Perhaps you know that
x4 − 1 = (x − 1)(x3 + x2 + x + 1) or
x4 − 1
= x3 + x2 + x + 1
x−1
Factoring Polynomials
• Do you see how to generalize?
xn − 1
= xn−1 + xn−2 + · · · + x2 + x + 1
x−1
• Do you see that the terms on the right-hand-side are terms in a geometric sequence?
1, x, x2 , . . . , xn−1 , . . .
• That is, the sum of terms in the geometric sequence is
n−1
∑ xn−k−1 = xn−1 + xn−2 + · · · + x2 + x + 1 =
k=0
30
xn − 1
=
x−1
Zeno Sum Formula
• The Zeno sums are the value of polynomial xn−1 + xn−2 + · · · + x2 + x + 1 at x = 2
−1
∑ 2k = 0
the empty sum
k=0
0
∑ 2k = 20 = 1
k=0
1
∑ 2k = 20 + 21 = 3
k=0
2
∑ 2k = 20 + 21 + 22 = 7
k=0
Zeno Sum Formula
3
∑ 2k = 20 + 21 + 22 + 23 = 15
k=0
4
∑ 2k = 20 + 21 + 22 + 23 + 24 = 31
k=0
5
∑ 2k = 20 + 21 + · · · + 25 = 63
k=0
6
∑ 2k = 20 + 21 + · · · + 26 = 127
k=0
7
∑ 2k = 20 + 21 + · · · + 27 = 255
k=0
Zeno Sum Formula
• In general, the sum of powers of 2 is
n−1
∑ 2k = 20 + 21 + · · · + 2n−1
k=0
2 × 2n−1 − 1
2−1
= 2n − 1
=
• The Zeno sums generate terms in the Mersenne sequence mn = 2n − 1
A Most Important Fact
• The largest unsigned binary number representable with n bits is
1 + 24 + 8 + · · · + 2n−1 = 2n − 1
• With a nibble (4 bits) the largest unsigned integer is
(1111)2 = 23 + 22 + 21 + 20 = 24 − 1 = 15
31
• With a byte (8 bits) the largest unsigned integer is
(1111 1111)2 = 27 + 26 + 25 + · · · + 22 + 21 + 20 = 28 − 1 = 255
• With 2 bytes (16 bits) the largest unsigned integer is
15
∑ 2k = 216 − 1 = 65 535
k=0
A Most Important Fact
• With a 4 byte word (32 bits) the largest unsigned integer is
31
∑ 2k = 232 − 1 = 4 294 967 295
k=0
(about the number of DNA bases in the human genome)
• With a 8 byte word (64 bits) the largest unsigned integer is
63
∑ 2k = 264 − 1 = 18 446 744 073 709 551 615
k=0
(about 18.5 quintillion, a big integer, but not the largest integer)
The Geometric Sum Formula
• The sum of terms in an geometric sequence equals its ratio times the last term, minus the first term, all divided
by its ratio minus one.
• Consider the example of a Zeno sum
6
∑ 2k = 20 + 21 + 22 + 23 + 24 + 25 + 26
k=0
– The ratio is r = 2
– The first term 20 = 1
– The last term is 26 = 64
– The value of the sum is
6
∑ 2k =
k=0
2 × 26 − 20
= 27 − 1 = 127
2−1
The Geometric Sum Formula
• Consider the example of sum of powers of 10
6
∑ 10k = 100 + 101 + 102 + 103 + 104 + 105 + 106
k=0
– The ratio is r = 10
– The first term 100 = 1
– The last term is 106 = 1 000 000
– The value of the sum is
6
∑ 10k =
k=0
10 × 106 − 100
107 − 1
=
= 1 111 111
10 − 1
9
32
The Geometric Sum Formula
• More generally, the sum
6
∑ rk = r0 + r1 + r2 + r3 + r4 + r5 + r6
k=0
is equal to
r × r6 − r0
r7 − 1
=
,
r−1
r−1
provided r 6= 1
• More generally still, the sum
n−1
∑ rk = r0 + r1 + r2 + · · · + rn−1
k=0
is equal to
rn − 1
r × rn−1 − r0
=
,
r−1
r−1
provided r 6= 1
The Geometric Sum Formula
• Even more generally still, the sum
n−1
∑ rk = r j + r j+1 + r j+2 + · · · + rn−1
k= j
is equal to
rn − r j
r × rn−1 − r j
=
,
r−1
r−1
provided r 6= 1
Example Geometric Sums
• Evaluate the sum
1
1
1
+ 2 + 1 + 1 + 21 + 22 + 23 + 24 + 25
3
2
2
2
– The ratio is r = 2
– The first term 2−3
– The last term 25
– The value of the sum is
2 × 25 − 2−3
1
= 26 − 2−3 = 64 −
2−1
8
Example Geometric Sums
• Evaluate the sum
1
1
+
+ · · · + 105
104 103
– The ratio is r = 10
– The first term 10−4
– The last term 105
– The value of the sum is
10 × 105 − 10−4
106 − 10−4
=
= 111 111.1111
10 − 1
9
33
Deriving the Geometric Sum Formula
• One way to derive the geometric sum formula is this subtraction trick
• Name the sum 1 + r + r2 + · · · + rn−1 : Call it S
n−1
S = 1 + r + r2 + · · · + rn−1 =
∑ rk
k=0
• Multiply S by r
rS = r(1 + r + r2 + · · · + rn−1 ) = r + r2 + r3 + · · · + rn
• Subtract S from rS
rS − S = (r + r2 + r3 + · · · + rn ) − (1 + r + r2 + · · · + rn−1 )
= rn − 1
S(r − 1) = rn − 1
rn − 1
S=
r−1
Deriving the Geometric Sum Formula
• Another way to derive the geometric sum formula is by multiplying polynomials
• Consider the polynomial
n−1
xn−1 + xn−2 + · · · + x2 + x + 1 =
∑ xn−1−k
k=0
• Multiply the polynomial by x − 1
xn
xn−1
xn−2
−xn−1
xn−1
−xn−2
xn−2
x2
···
· · · −x2
· · · x2
x
x
−x
x
xn
1
−1
−1
−1
Deriving the Geometric Sum Formula
• That is
(xn−1 + xn−2 + · · · + x2 + x + 1)(x − 1) = xn − 1
• Therefore
(xn−1 + xn−2 + · · · + x2 + x + 1) =
Recursion for Geometric Sums
• Name the sum
n−1
Sn−1 =
∑ 2k
k=0
• Then
n
Sn =
∑ 2k
k=0
!
n−1
=
∑2
k
k=0
= Sn−1 + 2n
34
+ 2n
xn − 1
x−1
• That is, the geometric sum ∑ 2k satisfies the recurrence equation
Sn = Sn−1 + 2n ,
with initial condition S0 = 0
Recursion for Geometric Sums
• More generally, name the sum
n−1
∑ rk
Sn−1 =
k=0
• Then
n
∑ rk
Sn =
k=0
!
n−1
=
∑r
k
+ rn
k=0
= Sn−1 + rn
• That is, the geometric sum ∑ rk satisfies the recurrence equation
Sn = Sn−1 + rn ,
with initial condition S0 = 0
Mathematical Induction Proof of Geometric Sum Formula
• The formula
n−1
Sn−1 =
∑ rk = 1 + r + r2 + · · · + rn−1 =
k=0
rn − 1
r−1
can be established by mathematical induction
• To establish a basis for induction
– Let n = 0
– Then the summation formula on the left is empty and defined to be 0
0−1
S−1 =
−1
∑ rk = ∑ rk = 0,
k=0
empty sum equals 0
k=0
– And the formula on the right is
r0 − 1 1 − 1
=
=0
r−1
r−1
– The left-hand-side equals the right-hand-side
Mathematical Induction Proof of Geometric Sum Formula
• We can also check n = 1
• Then the summation formula on the left is
S0 =
1−1
0
k=0
k=0
∑ rk = ∑ rk = r0 = 1
• And the formula on the right is
r1 − 1 r − 1
=
=1
r−1
r−1
• The left-hand-side equals the right-hand-side
35
Mathematical Induction Proof of Geometric Sum Formula
• Make an inductive hypothesis
– The formula
n−1
Sn−1 =
∑ rk = 1 + r + r2 + · · · + rn−1 =
k=0
rn − 1
r−1
is correct for some n ≥ 0
• We know this hypothesis is true: it was just established for n = 0 and n = 1
• Now complete the inductive step by proving that the inductive hypothesis implies
n
Sn =
∑ rk = 1 + r + r2 + · · · + rn−1 + rn =
k=0
rn+1 − 1
r−1
Mathematical Induction Proof of Geometric Sum Formula
• To complete the inductive step, note that
n
Sn ∑ r k
k=0
= 1 + r + r2 + · · · + rn−1 + rn
= 1 + r + r2 + · · · + rn−1 + rn
= Sn−1 + rn by recursion
rn − 1
+ rn by the inductive hypothesis
=
r−1
rn − 1
r−1
=
+ rn
r−1
r−1
rn+1 − 1
=
r−1
Change of Variable
• It can be useful to change the name of the index summation k
• The following sums are equivalent
n−1
n−1
n−1
∑ rk = ∑ ri = ∑ r j
i=0
k=0
j=0
• It can be useful to change the names of the lower an upper limits of summation
• The following sums are equivalent
n−1
n+3
∑ rk =
∑ rk−4
k=0
k=4
Change of Variable
• Notice that
n−1
∑ rk = r0 + r1 + r2 + · · · + rn−1
k=0
• While
n+3
∑ rk−4 = r4−4 + r5−4 + r6−4 + · · · + rn+3−4
k=4
36
Change of Variable
• Consider the sum
m
∑ 2k = 25 + 26 + · · · + 2m
k=5
• Rename k = i + 5
– When k = 5, i = 0
– When k = m, i = m − 5
• The following sums are equivalent
m
m−5
∑ 2k = ∑ 2i−5
i=0
k=5
Algorithm for Simplifying a Geometric Sum
• The geometric sum
ri + ri+1 + ri+2 + · · · + r j ,
is equal to the ratio r times the last term
rj
minus the first term
r 6= 1
ri
all divided by the ratio minus one
• That is,
ri + ri+1 + ri+2 + · · · + r j =
r · r j − ri
,
r−1
r 6= 1
Application of Geometric Sums
• There are many “things” that can be modeled by exponential growth.
• Compound interest: In = P × (1 + r)n , where In is the interested (earned or paid), P is the principal (loaned or
borrowed), r is the fixed interest rate, and n is the time period over which the interest in accumulated
• Malthus’ prediction of population growth: Pn = a(1.02)n , where P is world population, a is the world population
circa 1800, and n is time
• The (Richter) magnitude M of an earthquake is (almost) related to the amplitude A measured on a Richter scale
by the formula A = 10M
• Time required to search for a satisfying truth assignment for a Boolean expression.
Summary of Topics Covered
• Definition of a geometric sequence
a, ar, ar2 , ar3 , · · · , arn · · ·
• Ratios of successive values in an geometric sequence is the constant r
arn
=r
arn−1
• The Zeno sequence
h1, 2, 4, 8, · · · , 2n , · · ·i
37
Summary of Topics Covered
• The sum of terms in the Zeno sequence
1, 2, 4, 8, · · · , 2n−1 · · ·
is the Mersenne number
n−1
∑ 2k = 1 + 2 + 4 + · · · + 2n−1
k=0
=
2n − 1
2−1
• The largest unsigned integers that can be represented in binary with n bits is the Mersenne number 2n − 1
Summary of Topics Covered
• For arbitrary lower and upper limits
j
∑ 2k = 2i + 2i+1 + · · · + 2 j
k=i
= 2i (1 + 2 + · · · + 2 j−i )
= 2i (2 j−i+1 − 1)
• More generally, for arbitrary ratio r 6= 1
n−1
1 + r + r2 + · · · + rn−1 =
∑ rk =
k=0
rn − 1
r−1
• Even more generally, for r 6= 1
ri + ri+1 + · · · + r j = ri (1 + r + · · · + r j−i ) = ri
(r j−i+1 − 1)
(r − 1)
• A proof by mathematical induction.
12
Summations In Pictures
Quotation
See (sē) v., saw, seen, seeing . —v.t. . . . 5. to perceive (things) mentally; discern; understand: to see the point of an
argument. – T HE R ANDOM H OUSE D ICTIONARY OF THE E NGLISH L ANGUAGE
Summations In Pictures
• The use of diagrams to illustrate mathematical concepts has a ancient history
• Many of the sums can be “seen” by drawing the right picture
38
The Gauss Sum
1
0 + 1 + 2 + 3 + 4 + 5 + 6 = 21 = 7(7 − 1)
2
n−1
1
∑ k = 0 + 1 + 2 + · · · + (n − 1) = 2 n(n − 1)
k=0
The Zeno Sum
1 + 2 + 4 = 3 + 4 = 2 · 4 − 1 = 23 − 1
1
2
3
4
4
n−1
∑ 2k = 1 + 2 + 4 + · · · + 2n−1 = 2 · 2n−1 − 1 = 2n − 1
k=0
The Zeno Sum
1 + 2 + 4 + 8 = 3 + 4 + 8 = 7 + 8 = 2 · 8 − 1 = 24 − 1
1
2
4
8
3
4
8
7
8
n−1
∑ 2k = 1 + 2 + 4 + · · · + 2n−1 = 2 · 2n−1 − 1 = 2n − 1
k=0
39
The Fibonacci Sum
1 + 1 + 2 = 2 + (3 − 1) = 5 − 1
1
1
2 (3 − 1)
2
(5 − 1)
n−1
∑ fk = 0 + 1 + 1 + 2 + · · · + fn−1 = fn+1 − 1
k=0
The Fibonacci Sum
1 + 1 + 2 + 3 = 3 + (5 − 1) = 8 − 1
1 1 2 3
3 (5 − 1) (8 − 1)
n−1
∑ fk = 0 + 1 + 1 + 2 + · · · + fn−1 = fn+1 − 1
k=0
Sums of Harmonic Numbers
n−1
∑ Hk + n = nHn
k=1
40
1
1
1
1
1
1
1
1
H8
13
H8
H1
H8
H2
H8
H3
H8
H4
H8
H5
H8
H6
H8
H7
Introduction to Induction
Quotation
The concept of “mathematical induction” should be distinguished from what is usually called “inductive reasoning”
in science. – D ON K NUTH “The Art of Computer Programming: Fundamental Algorithms” American computer
scientist (1938 – )
Mathematical Induction
• Inductive reasoning is a logic used for discovery in science
– Repeated observation of consistent results leads to a conjecture
• The principle of mathematical induction is an axiom in the theory of natural numbers
– Mathematical induction is a template procedure used to establish facts about enumerable sets
– There is a basis for induction, an observation of the fact in some instance
– And there is a material implication that “if the fact is true in some instance, then it will be true in the next
instance”
Observations: Full Binary Trees
• The full binary tree of height h = 0 has 1 node
• The full binary tree of height h = 1 has 3 nodes
1
3
2
41
Observations: Full Binary Trees
• The full binary tree of height h = 2 has 7 nodes
1
3
2
4
5
6
7
Question
• What is the functional relationship between the height h of a full binary tree and the number of nodes n in the
tree?
Observations
Height Nodes
h=0
n=1
h=1
n=3
h=2
n=7
Hypothesis
• The full binary tree of height h has n = 2h+1 − 1 nodes
• This statement is true for h = 0, 1 and 2
Height
0
1
2
Nodes
1 = 2(0+1) − 1
3 = 2(1+1) − 1
7 = 2(2+1) − 1
• To prove the statement for every integer
– show that if it is true for some height h ≥ 0, then it will be true for height h + 1
Height h, Full Binary Tree Has 2h+1 − 1 Nodes
• A full binary tree of height h + 1 is constructed from
– 1 root node
– A full left subtree of of height h
– A full right subtree of of height h
root
height 1
left
right
subtree
subtree
42
height h
Height h, Full Binary Tree Has 2h+1 − 1 Nodes
root
height 1
height h
right
subtree
left
subtree
• If the left and right subtree both have 2h+1 − 1 nodes, then the nodes in the higher tree are
(1 root node) + (nodes in left) + (nodes in right)
1 + (2h − 1) + (2h+1 − 1) = 1 + 2(2h+1 − 1)
= 1 + (2h+2 − 2)
= 2h+2 − 1
Gauss Fools the Third Grade Teacher
• There is an often told story, with many variations, of how young Gauss irritated his third grade teacher
• The class was asked to compute the value of the sum
1 + 2 + 3 + · · · + 99 + 100
• Almost immediately Gauss answered 5050
• Perhaps Gauss knew the total would be the number of terms (100) times the average of the first and last term
1 + 100
= 50(101) = 5050
1 + 2 + 3 + · · · + 99 + 100 = 100
2
Generalizing the Problem
• Consider the more general sum
0 + 1 + 2 + 3 + · · · + (n − 2) + (n − 1)
• The sum has n terms
• The first term is 0, the last term is (n − 1), and their average is (n − 1)/2
• Is it true that
0 + 1 + 2 + 3 + · · · + (n − 2) + (n − 1) = n
Prove ∑ k =
n−1
?
2
n(n−1)
2
• To prove
0 + 1 + 2 + 3 + · · · + (n − 2) + (n − 1) = n
n−1
2
is true for all natural numbers n, show
Basis: For n = 0
• The sum on the left is empty, and so equal to 0
0 + 1 + · · · + (0 − 2) + (0 − 1) = 0
• The right hand side is also equal to 0 for n = 0
0
43
0−1
2
=0
(empty sum)
Prove ∑ k =
Induction:
n(n−1)
2
• Pretend
0 + 1 + 2 + · · · + (n − 2) + (n − 1) =
n(n − 1)
2
for some n ≥ 0
• Then the next sum is:
n(n − 1)
+n
2
n(n − 1) 2n
+
=
2
2
n(n + 1)
=
2
[0 + 1 + · · · + (n − 1)] + n =
• Which shows the next sum equals the function at the next natural number
Inductive Template For Functional Equality
• To prove “ f (n) = g(n)” is true for all natural numbers n
Basis: Show: both functions map 0 to the same value, that is,
f (0) = g(0)
Induction: Show: if f (n) = g(n) for some n ≥ 0, then f (n + 1) = g(n + 1)
14
Induction on Sequences
Quotation
The so-called law of induction cannot possibly be a law of logic, since it is obviously a
proposition with a sense. Nor, therefore, can it be an a priori law.
Ludwig Wittenstein, "Tractatus Logico-Philosophicus" Austrian-British philosoper (1889 - 1951)
Checking Solutions to Algebraic Equations
• Pretend you want to check that x = 4 is a solution to the linear equation
3x − 5 = 7
• You can do this by substitiuting 4 for x on the left-hand side and show the result is the right-hand side
3(4) − 5 = 12 − 5 = 7
Checking Solutions to Algebraic Equations
√
• Pretend you want to check that x = (1 + 5)/2 is a solution to the quadratic equation
x2 = x + 1
√
• You can do this by substitiuting (1 + 5)/2 for x in both the left-hand and right-hand sides and show the results
are equal
44
– The left-hand side is
√
√
√ !2
1+2 5+5 3+ 5
1+ 5
=
=
2
4
2
– The right-hand side is
√
√
√
1+ 5
1+ 5 2 3+ 5
+1 =
+ =
2
2
2
2
Checking Solutions to Recurrence Equations
• The same basic idea can be used to check a proposed solution to a recurrence equation
• Suppose you want to show that mn = 2n − 1 is a solution to the recurrence
mn = 2mn−1 + 1
• You can do this by substitiuting 2n − 1 for mn in the left-hand side and 2n−1 − 1 for mn−1 in the right-hand sides
and show the results are equal
– The left-hand side is
2n − 1
– The right-hand side is
2 2n−1 − 1 + 1 = (2n − 2) + 1 = 2n − 1
Checking Solutions to Recurrence Equations
• There’s is one problem with this
• For any constant c, you can show mn = c2n − 1 is a solution to the recurrence
mn = 2mn−1 + 1
– The left-hand side would be
c2n − 1
– And the right-hand side would be
2 c2n−1 − 1 + 1 = c2n − 1
• An initial value nails down the solution: If m0 = 0, then c20 − 1 = c − 1 = 0 forces c to be 1
Induction on The Triangular Sequence
• Consider the recurrence for the triangular numbers
tn = tn−1 + (n − 1)
• To show that tn = n(n − 1)/2 solves the recurrence substitute into the left-hand and right hand sides
– The left-hand side is
n(n − 1)/2
– The right-hand side is
(n − 1)(n − 2)
(n − 1)(n − 2) 2(n − 1)
+ (n − 1) =
+
2
2
2
(n − 1)(n − 2) + 2(n − 1)
=
2
n(n − 1)
=
2
• To complete the induction, establish the basis: t0 = 0(0 − 1)/2 = 0
45
Induction on The Mersenne Sequence
• Consider the recurrence for the Mersenne numbers
mn = 2mn−1 + 1
or, relabeling subscripts
mn+1 = 2mn + 1
• If mn = 2n − 1, then
mn+1 = 2mn + 1 = 2(2n − 1) + 1
• Which can be rewritten as
mn+1 = 2n+1 − 1
• The function 2n − 1 maps 0 to 0 which matches the initial Mersenne number m0 = 0
• Since both the basis and the inductive conditional are both true, the Mersenne numbers mn can be computed by
the function m(n) = 2n − 1 for all natural numbers n
Induction on The Fermat Sequence
• Consider the recurrence for the Fermat numbers
rn = (rn−1 − 1)2 + 1
or, relabeling subscripts
rn+1 = (rn − 1)2 + 1
n
• If rn = 22 + 1, then
n
rn+1 = ((22 + 1) − 1)2 + 1
• Which can be rewritten as
rn+1 = 22
n+1
+1
n
• The function 22 + 1 maps 0 to 3 which matches the initial Fermat number r0 = 3
• Since both the basis and the inductive conditional are both true, the Fermat numbers rn can be computed by the
n
function r(n) = 22 + 1 for all natural numbers n
15
Induction on Sums
Quotation
MacPherson told me that my theorem can be viewed as blah blah blah Grothendick blah blah
blah, which makes it much more respectable. I think some intuition leaks out in every step of an
induction proof.
J IM P ROPP, A MERICAN MATHEMATICIAN
46
Induction on Sums
• To prove a summation ∑ ak can be expressed as a function g(·) on the natural numbers,
– Establish a basis for induction, that is, show
0
∑ ak = a0 = g(0)
k=0
– Prove the inductive conditional: If
n−1
∑ ak = g(n),
If
k=0
then
"
n
#
n−1
∑ ak = ∑ ak
k=0
+ an
k=0
= g(n) + an
= g(n + 1)
Alice Sum Math Induction Proof
• The Alice sum formula
n−1
∑1=n
k=0
can be established by mathematical induction
Basis: For n = 0
• The sum
0−1
n−1
∑1= ∑1
k=0
k=0
is empty and equal to 0
• The value on the right hand, n, is 0 too
Alice Sum Math Induction Proof
Induction: Show that
• If
n−1
∑ 1 = n,
k=0
then
n
∑ 1 = n+1
k=0
By computing
"
n
n−1
∑1= ∑1
k=0
k=0
= n+1
47
#
+1
Gauss Sum Math Induction Proof
• The Gauss sum formula
n−1
∑ k = 0 + 1 + 2 + 3 + · · · + (n − 1) =
k=0
n(n − 1)
2
can be established by mathematical induction
Basis: For n = 0 the sum on the left is empty, and so equal to 0. And the right hand side is 0(0 − 1)/2 = 0 too
Induction: If ∑n−1
k=0 k = n(n − 1)/2, then
"
n
n−1
#
∑k= ∑k
k=0
+ n = (n + 1)n/2
k=0
Zeno Sum Math Induction Proof
• The Zeno sum formula
n−1
∑ 2k = 1 + 2 + 4 + · · · + 2n−1 = 2n − 1
k=0
can be established by mathematical induction
Basis: For n = 0 the sum on the left is empty, and so equal to 0. And the right hand side n is 0 too
k
n
Induction: If ∑n−1
k=0 2 = 2 − 1, then
"
n
∑2
#
n−1
k
=
k=0
k
∑2
+ 2n = 2n+1 − 1
k=0
Sum of the Odd Integers is a Square
• The ancient formula
n
∑ (2k − 1) = 1 + 3 + 5 + · · · + (2n − 1) = n2
k=1
can be established by mathematical induction
Basis: For n = 0 the sum on the left is empty, and so equal to 0. And the right hand side n is 0 too.
Sum of the Odd Integers is a Square
n
∑ (2k − 1) = 1 + 3 + 5 + · · · + (2n − 1) = n2
k=1
Induction: If
∑nk=1 (2k − 1) = n2 ,
then
n+1
"
n
#
∑ (2k − 1) = ∑ (2k − 1)
k=1
k=1
2
+ 2(n + 1) − 1
= n + 2(n + 1) − 1
= n2 + 2n + 1
= (n + 1)2
48
Sum of Cubes is a Squared Triangular Number
• The formula
n−1
∑k
3
=
k=1
2
1
n(n − 1)
2
can be established by mathematical induction
Basis: For n = 0 the sum on the left is empty, and so equal to 0. And the right hand side n is 0 too.
Sum of Cubes is a Squared Triangular Number
n−1
∑k
3
=
k=1
2
1
n(n − 1)
2
3
2
2
Induction: If ∑n−1
k=1 k = n (n − 1) /4, then
"
n
∑k
#
n−1
3
=
k=1
∑k
3
+ n3
k=1
2
1
n(n − 1) + n3
2
2
2 (n − 1)
+n
=n
4
2 (n + 2n + 1)
=n
4
2
1
n(n + 1)
=
2
=
Sum of Fibonacci Numbers
• Recall the Fibonacci sequence
~F = h0, 1, 1, 2, 3, 5, 8, 13, 21, . . .i
• Also recall the recurrence for the Fibonacci numbers
fn = fn−1 + fn−2 ,
n ∈ {2, 3, 4, . . .},
f0 = 0, f1 = 1
• Let’s write the sequence using the names of the numbers rather than their values
~F = h f0 , f1 , f2 , f3 , f4 , f5 , f6 , f7 , f8 , . . .i
Sum of Fibonacci Numbers
~F = h0, 1, 1, 2, 3, 5, 8, 13, 21, . . .i
~F = h f0 , f1 , f2 , f3 , f4 , f5 , f6 , f7 , f8 , . . .i
• Consider the sequence of partial sums
S0 = f 0 = 0
S1 = f 0 + f 1 = 1
S2 = f 0 + f 1 + f 2 = 2
S3 = f 0 + f 1 + f 2 + f 3 = 4
S4 = f0 + f1 + f2 + f3 + f4 = 7
S4 = f0 + f1 + f2 + f3 + f4 + f5 = 12
• Can you make a conjecture about these sums?
49
Sum of Fibonacci Numbers
n−1
∑ fk = fn+1 − 1
k=0
Basis: For n = 0
• The sum on the left-hand side is empty and so equal to 0
• The formula on the right-hand side is also equal to 0
f0+1 − 1 = f1 − 1 = 1 − 1 = 0
• This establishes the basis for induction
Sum of Fibonacci Numbers
n−1
∑ fk = fn+1 − 1
k=0
Induction: Pretend that
n−1
∑ fk = fn+1 − 1
k=0
is true for some n ≥ 0
• Then
"
n
#
n−1
∑ fk = ∑ fk
k=0
+ fn
k=1
= [ fn+1 − 1] + fn
= [ fn+1 + fn ] − 1
= fn+2 − 1
Pascal’s Triangle
1
1
1
1
1
1
1
1
1
..
.
1
2
3
4
5
6
7
8
..
.
1
3
6
10
15
21
28
..
.
1
4
10
20
35
56
..
.
1
5
15
35
70
..
.
1
6
21
56
..
.
1
7
28
..
.
1
8 1
.. ..
. .
Pascal’s Triangle: Sum of Rows
• There are many identities that can be found in Pascal’s triangle
• Perhaps the most famous one is found by summing values in a row
50
..
.
1
1
1
1
1
1
1
+1
+2
+3
+4
+5
+6
+1
+3 +1
+6 +4
+1
+10 +10 +5 +1
+15 +20 +15 +6
+1
=1
=2
=4
=8
= 16
= 32
= 64
Pascal’s Triangle: Sum of Rows
• We can make the conjecture
– The sum of values in row n of Pascal’s triangle is 2n for n = 0, 1, 2, 3, . . .
• When you notice how a row is computed from the previous row, it becomes clear why the row sums double
• Every term, except the first and last 1, is added twice to generate the next row
Pascal’s Triangle: Sum of Rows
• Consider row 6 of Pascal’s Triangle
1
6
15
20
15
6
1
• It can be used to generate row 7
1
7
21
35
35
21
7
1
• Sum adjacent numbers in row 6 yields
1
1+6
6 + 15
15 + 20
20 + 15
15 + 6
6+1
1
1
7
21
35
35
21
7
1
• Adding in the initial and terminal 1 in row 7 shows each term in row 6 is summed twice when summing row 7
Pascal’s Triangle: Sum of Rows
• To make the preceding argument precise we must name things
• We’ll name the rows and columns by the natural numbers
0
1
2
3
4
5
6
7
8
..
.
0
1
1
1
1
1
1
1
1
1
..
.
1
2
3
4
5
6
7
8
1
2
3
4
5
6
7
8
..
.
1
3
6
10
15
21
28
..
.
1
4
10
20
35
56
..
.
1
5
15
35
70
..
.
1
6
21
56
..
.
1
7
28
..
.
1
8
..
.
1
..
.
51
···
Pascal’s Triangle: Binomial Coefficients
• We’ll name terms in row n, column m by the symbol
• The symbol mn is called a binomial coefficient
• The symbol mn is read “n choose m”
n
m
• The binomial coefficient “n choose m” can be expressed in terms of factorials
n!
n
=
m
m!(n − m)!
Pascal’s Triangle
0
1
2
3
4
5
6
7
8
..
.
0
0
0
1
0
2
0
3
0
4
0
5
0
6
0
7
0
8
0
..
.
1
2
3
4
5
6
7
8
1
1
2
1
3
1
4
1
5
1
6
1
7
1
8
1
2
2
3
2
4
2
5
2
6
2
7
2
8
2
3
3
4
3
5
3
6
3
7
3
8
3
4
4
5
4
6
4
7
4
8
4
5
5
6
5
7
5
8
5
6
6
7
6
8
6
7
7
8
7
8
8
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
···
.
Pascal’s Identity
• The relationship between terms in row n − 1 and row n is called Pascal’s identity
• Pascal’s identity states that
n−1
n−1
n
+
=
m−1
m
m
• That is, the sum of terms in columns m − 1 and m of row n − 1 equals the term in column m row n
Pascal’s Identity
• An example of Pascal’s identity is
7
7!
7×6×5
=
=
= 35
4
4!3! 3 × 2 × 1
and
Therefore
7
7!
7×6
=
= 21
=
5
5!2! 2 × 1
8
7
7
=
+
= 35 + 21
5
4
5
52
Pascal’s Identity
• Another example of Pascal’s identity is
12!
12
= 495
=
8!4!
8
and
12
12!
=
= 660
9
9!3!
Therefore
13
= 495 + 660 = 1155
9
Pascal’s Triangle Row Sum
• Armed with Pascal’s identity, we can use induction to establish the sum of values in row n is 2n
• To sum the elements in row n we write
n n
n
n
n
n
n
=
+
+
+
·
·
·
+
+
∑ k
0
1
2
n
−
1
n
k=0
• Leave the first and last terms alone, but use Pascal’s identity on the middle terms
n
n−1
n−1
n−1
n−1
+
+
+
+
0
0
1
1
2
n−1
n−1
n
+···+
+
+
n−2
n−1
n
Pascal’s Triangle Row Sum
• Collect the terms that appear twice
n
n−1
n−1
n−1
n−1
n
+
+2
+···+2
+
+
0
0
1
n−2
n−1
n
• Recognize that the first two and last two terms are equal to 1 and so can be replaced as
n−1
n−1
n−1
n−1
2
+2
+···+2
+2
0
1
n−2
n−1
• Therefore, if the sum of terms in row n − 1 is 2n−1 , the sum of terms in row n is 2n
Epigraph
φ is an H OF A LOT COOLER THAN π .
Stettner, in Dan Brown’s “The Da Vinci Code”
Cassini’s Identity
Theorem 4 (Cassini’s Identity).
Fn+1 Fn−1 − Fn2 = (−1)n ,
for n > 0
• Interpret Cassini’s identity as stating the area of a rectangle with sides Fn−1 and Fn+1 is plus or minus one the
area of a square with sides Fn
• Arithmetically, multiply every other Fibonacci number and subtract the square of the number in the middle
• The result alternates between −1 and +1
53
Cassini’s Identity
n
Fn
0
0
1
1
2
1
3
2
4
3
5
5
6
8
7
13
8
21
9
34
...
...
0 · 1 − 12 = −1
2 · 1 − 12 = 1
3 · 1 − 22 = −1
5 · 2 − 32 = 1
8 · 3 − 52 = −1
Lewis Carroll’s Favorite Trick
• Cassini’s identity is the basis of an absurdity attributed Lewis Carroll
• Cut the 8 × 8 square along the lines indicated below and arrange the pieces into a 5 × 13 rectangle to conclude
that 64 = 65.
Inductive Proof of Cassini’s Identity
Fn+1 Fn−1 − Fn2 = (−1)n ,
for n > 0
• Replace Fn−1 by Fn+1 − Fn in the left-hand side of Cassini’s proposed identity to obtain
Fn+1 Fn−1 − Fn2 = Fn+1 (Fn+1 − Fn ) − Fn2
2
= Fn+1
− Fn+1 Fn − Fn2
2
= Fn+1
− Fn (Fn+1 + Fn )
2
= Fn+1
− Fn Fn+2
2
= −(Fn Fn+2 − Fn+1
)
2
)
= −(Fn+2 Fn − Fn+1
Inductive Proof of Cassini’s Identity
Fn+1 Fn−1 − Fn2 = (−1)n ,
for n > 0
• Thus if
Fn+1 Fn−1 − Fn2 = (−1)n
• Then
2
Fn+2 Fn − Fn+1
= (−1)n+1
• Since F2 F0 − F1 = 1 · 0 − 1 = (−1)1 , the basis for induction is true, so Cassini’s identity holds for all n.
54
Fundamental Theorem of Arithmetic
• The fundamental theorem of arithmetic states that every positive integer greater than 1 can be written as a
(unique) product of primes
• As a basis 2 is prime, hence a product of primes
• Pretend that for some n ≥ 2, every integer k = 2, 3, . . . , n can be written as the product of primes
• Consider n + 1
– If n + 1 is prime, then it is the product of primes
– If n + 1 is composite, then n + 1 = ab where 1 < a < n + 1 and 1 < b < n + 1
– Therefore, by the inductive assumption, a and b are the product of primes, and so n + 1 is the product of
primes
The Fundamental Theorem of the Sum and Difference Calculus
• Define the falling factorial power nm by
nm = n(n − 1) · · · (n − m + 1)
• The following identity is true
n−1
m ∑ km−1 m =
k=0
n−1
∑ k(k − 1) · · · (k − m + 2) = nm
k=0
• As a basis for induction, when n = 0 the left-hand side equals the right-hand side
The Fundamental Theorem of the Sum and Difference Calculus
• Pretend
n−1
m ∑ km−1 = nm
k=0
is true for some n ≥ 0
• Then
n
n−1
m ∑ km−1 = m ∑ km−1 + mnm−1
k=0
k=0
m
= n + mnm−1
= [n(n − 1) · · · (n − m + 1)] + [m(n(n − 1) · · · (n − m + 2))]
= n(n − 1) · · · (n − m + 2)[(n − m + 1) + m]
= (n + 1)n(n − 1) · · · (n − m + 2)
= (n + 1)m
The Fundamental Theorem of the Sum and Difference Calculus
• The fundamental theorem of the sum and difference calculus is
n−1
m ∑ km−1 = nm
k=0
• Notice the analogy with the fundamental theorem of calculus
Z
m
xm−1 dx = xm
55
Exponential Beat Powers
• The exponent 2n grows faster than the polynomial n2
• Prove n2 ≤ 2n for all n > 3
– For n = 4 we have 42 = 1624
– Pretend n2 ≤ 2n for some n ≥ 4
– Notice that
(n + 1)2 ≤ 2n2 ,
when n > 3
since
0 ≤ 2n2 − (n + 1)2 = n2 − 2n − 1 = (n − 1)2 − 2
– Therefore
(n + 1)2 = 2n2
≤ 2 · 2n
= 2n+1
Induction Over Products
• Define the product notation ∏ ak by
n−1
∏ ak = a0 a1 · · · an−1
k=0
• Prove by induction that
n
n+1
1
∏ 1 − k2 = 2n
k=2
for n ≥ 2
• For a basis, let n = 2, and find
2
1
2+1
1
1
−
= 1− 2 =
∏
2
k
2
2·2
k=2
Induction Over Products
• Pretend that
n
1
n+1
1
−
=
∏
2
k
2n
k=2
for n ≥ 2
for some n ≥ 2
• Then notice that
n+1 ∏
k=2
1
1− 2
k
"
# 1
1
= ∏ 1− 2
1−
k
(n + 1)2
k=2
n+1
1
=
1−
2n
(n + 1)2
1
1
=
(n + 1)2 −
2n
(n + 1)
2
1 n + 2n
=
2n (n + 1)
n+2
=
2(n + 1)
n
56
16
Introduction to Sets
Epigraph
Every mathematician agrees that every mathematician must know some set theory; the
disagreement begins in trying to decide how much is some.
Paul Halmos, "Naive Set Theory" American mathematician (1916 - 2006)
Definition of a Set
Definition 5. Set A set is, well, . . . , an undefined term, just as point and line are undefined in geometry.
What is a Set?
• A set is a collection of things
• The things in a set are unordered
• A thing occurs only once in a set
Common Sets
• There are several sets of special interest
• The Natural Numbers
N = {0, 1, 2, 3, 4, 5, 6, . . .}
• The Integers
Z = {0, ±1, ±2, ±3, ±4, ±5, ±6, . . .}
• The Positive Integers
Z+ = {1, 2, 3, 4, 5, 6, . . .}
• The Negative Integers
Z− = {−1, −2, −3, −4, −5, −6, . . .}
• The Rational Numbers
Q = {a/b : a, b ∈ Z, b 6= 0}
Common Sets
• The Real Numbers
R
• The Irrationals
Q = R−Q
• The Complex Numbers
C = {x + iy : x, y ∈ R}
57
Common Sets
• The Bits
B = {0, 1}
• The Octets
O = {0, 1, 2, 3, 4, 5, 6, 7}
• The Digits
D = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}
• The Hexits
H = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D, E, F}
• The Empty Set
0/
• A Universal Set (that depends on context (state))
U
Describing a Set
• A set can be defined by listing its elements or members
– The elements are enclosed in braces { } and separated by commas
– For example,
{chocolate, vanilla, strawberry}
• When a clear pattern exists, ellipsis (. . .) can be used to denote the elements in a set
{a, b, c, d, . . . , z}
• A set can also be defined by a predicate, the property that describes all and only elements of the set
Sets Defined by a Predicate Statements
• The set of lower case English letters
L = {α : α is a lower case English letter}
• The set of prime numbers
P = {p : p ∈ Z+ ∧ p has exactly two divisors}
= {2, 3, 5, 7, 11, 13, 17, 19, 23, · · · }
• The Zeno numbers
Z = {w : w ∈ N && w is a power of two}
= {1, 2, 4, 8, 16, 32, 64, 128, 256, 512, · · · }
58
Sets Defined by a Predicate Statements
• The set of Mersenne numbers
M = {m : m = 2n − 1 for some n ∈ N}
= {0, 1, 3, 7, 15, 31, 63, 127, 255, · · · }
• The set of triangular numbers
n(n − 1)
T= t :t =
for some n ∈ N
2
= {0, 1, 3, 6, 10, 15, 21, 28, · · · }
• The set of Fibonacci numbers
(
F=
)
√
√
(1 + 5)n − (1 − 5)n
√
for some n ∈ N
f:f=
52n
= {0, 1, 2, 3, 5, 8, 13, 21, · · · }
Set Operations
• Operations can be performed on sets
• Operations can be classified as
– Boolean operations: union, intersection, set complement
– Relational operations: Cartesian product, subset, element of
– Counting operations: cardinality, power set
Boolean Operation: Union
• The union A ∪ B creates a set by combining elements that are members of either A or B
A ∪ B = {x : x ∈ A ∨ x ∈ B}
• If A = {0, 2, 4} and B = {2, 4, 6}, then
A ∪ B = {0, 2, 4, 6}
Boolean Operation: Union
• Let A be set of integers with remainder 3 when divided by 4, that is,
A = {. . . , −9, −5, −1, 3, 7, 11, . . .}
= {4n + 3 : n ∈ Z}
• Let B be set of integers with remainder 2 when divided by 3, that is,
B = {. . . , −7, −4, −1, 2, 5, 8, . . .}
= {3n + 2 : n ∈ Z}
• The union of A and B is
A ∪ B = {4n + 3 or 3n + 2 : n ∈ Z}
59
Algebraic Laws for Union
• The union operation is commutative that is,
A∪B = B∪A
• The union operation is associative that is,
A ∪ (B ∪ C) = (A ∪ B) ∪ C
Boolean Operation: Intersection
• The intersection A ∩ B creates a set by selecting elements that are members of both A and B
A ∩ B = {x : x ∈ A ∧ x ∈ B}
• If A = {0, 2, 4} and B = {2, 4, 6}, then
A ∩ B = {2, 4}
Boolean Operation: Intersection
• Let A be set of integers with remainder 3 when divided by 4, that is,
A = {. . . , −9, −5, −1, 3, 7, 11, . . .}
= {4n + 3 : n ∈ Z}
• Let B be set of integers with remainder 2 when divided by 3, that is,
B = {. . . , −7, −4, −1, 2, 5, 8, . . .}
= {3n + 2 : n ∈ Z}
Algebraic Laws for Intersection
• The intersection operation is commutative that is,
A∩B = B∩A
• The intersect operation is associative that is,
A ∩ (B ∩ C) = (A ∩ B) ∩ C
Algebraic Laws for Distributing Intersections and Union
• Intersection distributes over union that is,
A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C)
• Union distributes over intersection that is,
A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C)
A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C)
B
A
B
C
A
60
C
• B∪C
• A∩B
• A ∩ (B ∪ C)
• (A ∩ B) ∪ (A ∩ C)
A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C)
B
A
B
C
A
C
• B∩C
• A∪B
• A ∪ (B ∩ C)
• (A ∪ B) ∩ (A ∪ C)
Algebraic Laws for Union and Intersection: Absorption
• Union absorbs intersecion that is,
A ∪ (A ∩ B) = A
• Intersection absorbs union that is,
A ∩ (A ∪ B) = A
Boolean Operation: Set Complement
• The complement A creates the set of all elements not in A
A = {x : x 6∈ A}
Algebraic Laws for Set Complement
• A set and its complement partition the universal set that is,
A∪A = U
and
A ∩ A = 0/
• The complement of a complement is the set itself
A=A
Additional Algebraic Identities for Sets
• Union and intersection are idempotent that is,
A∪A = A
and
A∩A = A
61
• The empty set is the identity for union
A ∪ 0/ = A
• The universal set is the identity for intersection
A∩U = A
• The empty set and universal set are complements of one another
U = 0/
0/ = U
DeMorgan’s Laws
• The complement of a union is the intersection of complements that is,
A∪B = A∩B
• The complement of an intersection is the union of complements that is,
A∩B = A∪B
Relational Set Operation: Cartesian Product
• The Cartesian product A × B creates a relation between elements of A and B
A × B = {(a, b) : a ∈ A ∧ b ∈ B}
• The Cartesian product A × B is read “A cross B” or A times B”
• The Cartesian product of two sets is a set of ordered pairs
Relational Set Operation: Cartesian Product
• The xy plane, with which you are familiar, is an example of a Cartesian product
• The Cartesian plane is a visual representation of the Cartesian product
R × R = {(x, y) : x ∈ R, y ∈ R}
x
y
Examples of Cartesian Products
• Let X = {0, 1, 2} and Y = {3, 4}
• Then
X × Y = {(0, 3), (0, 4), (1, 3), (1, 4), (2, 3), (2, 4)}
• Let X = {2, 3, 5} and Y = {0, 1, 3}
• Then
X × Y = {(2, 0), (2, 1), (2, 3), (3, 0), (3, 1), (3, 3),
(5, 0), (5, 1), (5, 3)}
• Do you see a relationship between the number of elements in A, B, and A × B?
62
Examples of Cartesian Products
• Often one takes the Cartesian product of a set with itself
• Let B = {0, 1} be the set of bits
• Then
B × B = {(0, 0), (0, 1), (1, 0), (1, 1)}
Subset
Definition 6 (Subset). Set A is a subset of set B if and only every element x in A is also an element of B
A⊆B
Subset Examples
• Let B, O, D, and H be the set of bits, octets, decimals, and hexits.
• Then
– B⊆O
– O 6⊆ B
– B⊆D
– H 6⊆ O
– D⊆H
– D 6⊆ O
– 0/ ⊆ H
– B 6⊆ 0/
Subset Laws
• The empty set is a subset of every set
0/ ⊆ A
for every set A
A⊆U
for every set A
• Every set is a subset of the universal set
(this assumes the context is fixed for the involved sets)
Subset Laws
• Every set is a subset of itself (subset is reflexive)
A⊆A
for every set A
• If one set is a subset of another, and vice versa, then the sets are equal (subset is antisymmetric)
If A ⊆ B and B ⊆ A
then
A=B
• If one set is a subset of another which is subset of a third, then the first set is a subset of the third (subset is
transitive)
If A ⊆ B and B ⊆ C
then
A⊆C
63
Proper Subset
Definition 7 (Proper Subset). Set A is a proper subset of set B, written A ⊂ B, if and only
A⊆B
and
B 6⊆ A
Relations
• Subsets of a Cartesian product are called relations
• The equality relation on B × B is
E = {(0, 0), (1, 1)}
• The less than relation is
L = {(0, 1)}
• The greater than or equal is
T = {(0, 0), (1, 0), (1, 1)}
• Being politically incorrect, the Appalachian relation is
A ⊆ B × B = {(0, 0), (0, 1), (1, 0), (1, 1)}
where every member is related to every other member
Counting Operation: Cardinality
• The cardinality |A| counts the number of elements in a set
• Let A = {3, 5, 17, 257} be the first few Fermat numbers
• Then
|A| = 4
• Let L be the lower case English letters
• Then
|L| = 26
• Let H be the hexadecimal digits
• Then
|H| = 16
Cardinality
• Two sets A and B have the same cardinalitiy if there is a one-to-one and onto function f mapping elements in A
to B
• Finite sets can be put into correspondence with a set {0, 1, 2, . . . , n} for some natural number n
• The nucleic acids alphabet {AC, T, G} can be put in correspondence with the set {0, 1, 2, 3} by the function
f (0) = A
f (1) = C
f (2) = T
f (3) = G
64
Cardinality: Countable Sets
• Countable sets, that are not finite, can be put into correspondence with a set N of natural numbers
• The even integers are countable: They can be put in correspondence with the natural numbers by the function
f (n) = 2n
• The integers are countable: They can be put in correspondence with the natural numbers by the function
f (2n) = n,
f (2n + 1) = −n − 1
That is,
f (0) = 0, f (1) = −1, f (2) = 1, f (3) = −2, f (4) = 2, . . .
Cardinality: Countable Sets
• The rational numbers are countable
• Can you find a one-to-one function that maps the natural numbers onto the rationals?
• The cardinality of a countable set that is not finite is called aleph-naught, ℵ0
Cardinality: Uncountable Sets
• The real numbers are uncountable
• There is no one-to-one functions that maps the natural numbers onto the real numbers
• The cardinality of the real numbers is called aleph-one, ℵ1
Power Set of a Set
• Power set 2A creates the set of all subsets of A
2A = {X : X ⊆ A}
• For example, let B = {0, 1} be the set of bits
• The power set of B (the set of all subsets of B) is
2B = {0,
/ {0}, {1}, {0, 1}
17
Introduction to Logic
Quotation
Mathematical logic, then, is a branch of mathematics which has much the same relation to the
analysis and criticism of thought as geometry does to the science of space.
Haskell Curry, American mathematician, (1900–1982)
65
Propositions — Statements
• A proposition is a something that is either true or false
• Propositions are also called statements
• “George Washington chopped down a cherry tree.” is a proposition
• “Did George Washington chop down a cherry tree?” is not a proposition
• “George, chop down the cherry tree!” is not a proposition
Arguments, Premises, and Conclusions
• An argument is a collection of propositions in which some propositions, called the premises, are reasons for
accepting the proof of another proposition, called the conclusion
• A famous argument, attributed to Aristotle is
– Major premise: All humans are mortal
– Minor premise: Socrates is human
– Conclusion: Socrates is mortal
Valid, Invalid, and Sound Arguments
• The premises and conclusions, being propositions, are either true or false
• An argument is valid when the conclusion must be true if the premises are true
• An argument is invalid when the conclusion can be false even if the premises are true
• An argument is sound if it is valid and the premises are true
Valid and Sound Arguments
• Consider the premises below, both of which are true
– Major premise: Paris is in France
– Minor premise: France is in the Western hemisphere
• Consider the conclusion
– Conclusion: Paris is in the Western hemisphere
• The premises are true
• The conclusion must be true if the premises are true
• Logicians would say this is a valid argument
• Logicians would say the argument is sound
66
Valid but Unsound Arguments
• Consider the premises below, both of which are false
– Major premise: Paris is in China
– Minor premise: China is in the Western hemisphere
• Consider the conclusion
– Conclusion: Paris is in the Western hemisphere
• The conclusion is true
• Logicians would say this is a valid argument even though the premises are false
• Logicians would say the argument is unsound
Valid but Unsound Arguments
• Consider the premises below, both of which are false
– Major premise: Paris is in China
– Minor premise: China is in the Western hemisphere
• Consider the conclusion
– Conclusion: Paris is in China
• The conclusion is false
• But if the premises were true, then the conclusion must be true
• Logicians would say this is a valid argument even though all the statements in it are false
• Since the premises are false, logicians would say the argument is unsound
Valid Arguments
• A valid argument can have
– True premises and a true conclusion, in which case the argument sound
– False premises and a true conclusion, in which case the argument unsound
– False premises and a false conclusion, in which case the argument unsound
• A valid argument can not have true premises and a false conclusion
Invalid Arguments
• Consider the premise below
– Premise: The pyramids are in Egypt or Cairo is in Egypt
• Consider the conclusion
– Conclusion: The pyramids in in Egypt
• Logicians say this is a invalid argument even thought all the statements in it are true
• The form of the argument
P OR Q
is not always true
67
→P
Invalid Arguments
• Invalid arguments are always unsound
• An invalid argument can have instances where
– The premises are true and conclusion is true
– The premises are true and conclusion is false
– The premises are false and conclusion is true
– The premises are false and conclusion is false
Why is logic important?
• Logic is the study of sound arguments
• Philosophical reasons
– Making sound deductions
– Making reasoned inductions
Why is logic important?
• Programming control structure reasons
– All control structures can be expressed in terms of four basic forms
1.
2.
3.
4.
Procedure call (proc(input);)
Sequence (statement; statement;)
Alternative (if (proposition) then statement;)
Iteration (while (proposition) do statement;)
Why is Logic Important?
• Programming data structure reasons
– Arithmetic on the integers can be implemented in digital logic
– Bitwise addition with carry can be computed with an exclusive OR and AND
x
0
0
1
1
y
0
1
0
1
– The sum of bits x and y is x ⊕ y
– The bit to carry when x and y are added is x ∧ y
68
x⊕y
0
1
1
0
x∧y
0
0
0
1
18
Logical Operations
Operations On Proposition
• Operations can be performed on propositions to create new propositions
• Given the truth value of input propositions, we want to compute the truth value of the output proposition formed
by the operation
• Let P, Q, and R be variables representing propositions
• There are a few basic operations
– The negation of P is denoted ∼ P
– The conjunction of P and Q is denoted P ∧ Q
∗ Conjunction is called and
– The disjunction of P and Q is denoted P ∨ Q
∗ Disjunction is called or
– The conditional of Q on P is denoted P =⇒ Q
∗ The conditional P =⇒ Q can also be written as ∼ P ∨ Q
– The equivalence of P and Q is denoted P ≡ Q
Negation of a Proposition
• Let P be a proposition
• Logician denote the negation of P by
∼P
• Other notation for negation
– !P (used in programming)
– P̄ or P0 (used in digital logic)
• If P is true, then !P is false
• If P is false, then !P is true
• Logic gate and truth table representations of negation
P
P
¬P
0
1
Conjunction of Propositions
• Let P and Q be propositions
• Logician denote the conjunction of P and Q by
P∧Q
or simply
• Other notation for conjunction
– P AND Q
– P && Q (used in programming)
69
PQ
!P
1
0
– P • Q (used in digital logic)
• P ∧ Q is true only when both P and Q are true
• If either P or Q or both are false, then P ∧ Q is false
• Logic gate and truth table representations of conjunction
P
Q
P∧Q
P
Q
P∧Q
0
0
1
1
0
1
0
1
0
0
0
1
Disjunction of Propositions
• Let P and Q be propositions
• Logician denote the disjunction of P and Q by
P∨Q
• Other notation for disjunction
– P OR Q
– P k Q (used in programming)
– P + Q (used in digital logic)
• P ∨ Q is true when either P or Q or both are true
• If both P and Q are false, then P ∨ Q is false
• Logic gate and truth table representations of disjunction
P
Q
P∨Q
P
Q
P∨Q
0
0
1
1
0
1
0
1
0
1
1
1
P
Q
0
0
1
1
0
1
0
1
The Conditional Statement
• Let P and Q be propositions
• Logician denote the conditional statement “if P then Q” by
P
⊃Q
• Other notation for conditional
– P → Q
– P =⇒ Q
• P =⇒ Q is true when P is false or both P and Q are true
• If P is true and Q are false, then P =⇒ Q is false
• Logic gate and truth table representations of the conditional
P
Q
P
→Q
70
P
⊃Q
1
1
0
1
Reasoning About Conditional Statements
• The conditional P =⇒ Q is read
– P implies Q, or
– if P, then Q
• Recall P =⇒ Q is true, except when P is true and Q is false
• The conditional P =⇒ Q is equivalent to the disjunction
!P ∨ Q
• Notice !P ∨ Q is true, except when P is true and Q is false
• Using DeMorgan’s law, we can write the disjunction !P ∨ Q as
!(P∧!Q)
• Notice !(P∧!Q) is true, except when P is true and Q is false
Reasoning About Conditional Statements
• Consider the propositions
– P = James scores above 90% on the final
– Q = James earns an A in the class
• Pretend I make the statement
– R = (P =⇒ Q) = If James scores above 90% on the final, then James earns an A in the class
• Based on the truth value of P and Q, when is my statement r true and when is it false?
Reasoning About Conditional Statements
• Pretend “James scores above 90% on the final” is true
– Pretend “James earns an A in the class” is true
∗ My statement appears to be true
– Pretend “James earns an A in the class” is false
∗ My statement appears to be false, a lie
Reasoning About Conditional Statements
• Pretend “James scores above 90% on the final” is false
– Pretend “James earns an A in the class” is true
∗ My statement does not appears to be a lie
∗ I can award an A to James even if he does not score above 90% on the final
∗ If my statement is not a lie, it must be true
– Pretend “James earn an A in the class” is false
∗ My statement does not appears to be a lie
∗ If my statement is not a lie, it must be true
71
Reasoning About Conditional Statements
• The conditional statement
If TRUE, then TRUE
is true
• The conditional statement
If TRUE, then FALSE
is false
• The conditional statement
If FALSE, then TRUE
is true
• The conditional statement
If FALSE, then FALSE
is true
The Equivalence Statement
• Let P and Q be propositions
• Logician denote the equivalence statement “P if and only if Q” by
P
≡Q
• Other notation for equivalence or bi-conditional
– P↔Q
– P⇔Q
• Logic gate and truth table representations of equivalence
P
Q
P
≡Q
The Equivalence Statement
• The equivalence P ≡ Q can be written as
(P =⇒ Q) ∧ (Q =⇒ P)
• The equivalence P ≡ Q can be written as
(!P ∨ Q) ∧ (!Q ∨ P)
• Using DeMorgan’s laws the equivalence P ≡ Q can be written as
!(P∧!Q)∧!(Q∧!P)
• Applying DeMorgan’s laws again, we can write P ≡ Q as
![(P∧!Q) ∨ (Q∧!P)]
72
P
Q
0
0
1
1
0
1
0
1
P
≡Q
1
0
0
1
The Not And (NAND) Operation
• Let P and Q be propositions
• The NAND of P and Q the negation of their conjunction
!(P ∧ Q)
• Logic gate and truth table representations of NAND
P
Q
!(P ∧ Q)
!(P ∧ Q)
1
1
1
0
P
Q
0
0
1
1
0
1
0
1
P
Q
0
0
1
1
0
1
0
1
!(P ∨ Q)
1
0
0
0
P
Q
P⊕Q
0
0
1
1
0
1
0
1
0
1
1
0
The Not Or (NOR) Operation
• Let P and Q be propositions
• The NOR of P and Q the negation of their disjunction
!(P ∨ Q)
• Logic gate and truth table representations of NOR
P
Q
!(P ∨ Q)
The Exclusive Or (XOR) Operation
• Let P and Q be propositions
• The exclusive or (XOR) of P and Q is denoted by
P⊕Q
• Logic gate and truth table representations of XOR
P
Q
P⊕Q
• Notice that exclusive or is the not equivalent relation
73
Counting Logical Operations
• We listed 1 logical operation with one input
– N EGATION !P
• There are 3 additional one-input logical operations: True; False; B UFFER (pass input through to output)
• Consider the truth table with one input column and four output columns
N EGATION
1
0
P
0
1
True
1
1
False
0
0
B UFFER
0
1
1
• There are 4 = 22 logical formulas on one variable
Counting Logical Operations
• We have listed 7 logical operations with two input signals
– Conjunction P ∧ Q
– Disjunction P ∨ Q
– Conditional P → Q
– Equivalence P ≡ Q
– NAND !(P ∧ Q)
– NOR !(P ∨ Q)
– XOR P ⊕ Q
• There are 9 other two-input logical operations:
True; False;
→ P; P 6→ Q;
Q
Q
P;
6→ P
!P;
Q;
!Q;
2
• There are 16 = 22 logical formulas on two variable
Counting Logical Operations
• We did not list any 0 input logical operations, but there are in fact two of them
– True
– False
• What is the function that maps the number of input signals to the number of logical operations?
• We’ve computed the following values
Input signals
0
1
2
Logical Operations
2
4
16
74
Counting Logical Operations
• A logical operation on n variables is called a Boolean function
• The domain X of a Boolean function f depends on n, the number of variables f act on
• The range Y of a Boolean function f is always the set {0, 1}
• The number of functions from X to Y is the cardinality of Y raised to the cardinality of X
|Y||X| = 2|X|
Counting Logical Operations
• Let f (P) be a Boolean function on one variable P
– The domain of f is the set X = {0, 1}, the values P can take
– Number of one variable Boolean functions is
2|X| = 22 = 4
• Let f (P, Q) be a Boolean function on two variables P and Q
– The domain of f is the set of values P and Q can take
X = {(0, 0), (0, 1), (1, 0), (1, 1)}
– Number of two variable Boolean functions is
2|X| = 24 = 16
Counting Logical Operations
• Let f (P, Q, R) be a Boolean function on three variables P, Q, and R
– The domain of f is the set of values P, Q, and R can take
X = {(0, 0, 0), (0, 0, 1), (0, 1, 0), (0, 1, 1),
(1, 0, 0), (1, 0, 1), (1, 1, 0), (1, 1, 1)}
– Number of three variable Boolean functions is
2|X| = 28 = 256
Counting Logical Operations
• Let f (P0 , P1 , . . . , Pn−1 ) be a Boolean function on n variables
– The domain X of f is the set n-tuples over 0 and 1
– The cardinality of the domain of f is 2n , the number of n-tuples over 0 and 1
– The range of f is Y = {0, 1}, which has cardinality 2
– The number of functions from X to Y is
|Y||X|
– The number of n variable Boolean functions is
n
22
75
19
A Brief History of Logic
Quotation
I’m sorry to say that the subject I most disliked was mathematics. I have thought about it. I think
the reason was that mathematics leaves no room for argument. If you made a mistake, that was all
there was to it.
Malcolm X “The Autobiography of Malcolm X” American civil rights activist, (1925–1965)
Aristotle’s Logic
• Analysis and criticism of thought occurred in many ancient cultures
• Aristotle’s (384–322 BC) logic had a significant impact on Western thought
• Aristotle reasoned about relationships between subject-predicate statements of the form
1. All things S are of type P
2. No things S are of type P
3. Some things S are of type P
4. Some things S are not of type P
• S is the subject of these statements
• P is the predicate describing a property of the subject
• Notice there are 6 combinations for statements of the form
All (No) (Some)things P are (not) of type Q
Question
• Which two statements were missing from the four listed previously?
• Is it okay to not list them? Explain your “yes” or “no” answer
• Why is it necessary to list the given four?
Aristotle’s Statements and Graphical Representation
• Universal Affirmative: “All S are P”
• Universal Negative: “No things S are of type P”
• Existential Affirmative: “Some things S are of type P”
• Existential Negative: “Some things S are not of type P”
• Euler and Venn diagrams are used to illustrate these statements
76
The Universal Affirmative Statement
• All P are Q
– Venn Diagram
U
Q
Shade P outside Q to indicate
that there are no things of type P
outside of Q
P
– Euler Diagram
U
Q
Draw P inside of Q to indicate
every thing P has property Q
P
The Universal Negative Statement
• No P are Q
– Venn Diagram
U
Q
Shade P inside Q to indicate that
there are no things of type P inside Q
Y
P
– Euler Diagram
U
Separate P and Q to indicate the
subject and predicate no members
in common
Q
P
The Existential Affirmative Statement
• Some P are Q
– Venn Diagram
U
Q
X
P
Check the region of P that overlaps Q to indicate entities exist
there
77
– Euler Diagram
U
Q
Shade the region of P inside Q to
indicate that it is not empty
Y
P
The Existential Negative Statement
• Some P are not Q
– Venn Diagram
U
Q
Check the region of P where it
does not overlap Q to indicate
entities exist there
X
P
– Euler Diagram
U
Q
P
Shade P outside Q to indicate that
it is not empty
Aristotelian or Categorical Syllogisms
• Major premise:
– All humans are mortal (subject=human, predicate=mortal)
– Human and mortal are classes (or sets) of things
• Minor premise
– Socrates is human (subject=Socrates, predicate=human)
– Socrates is a particular instance of a human thing
• human appears in both premises
– It is called the middle term, labelled M
– In syllogistic arguments there is always a middle term shared by the premises.
– The middle term can be either the subject or the predicate
78
Aristotelian or Categorical Syllogisms
• Conclusion
– Socrates is mortal (subject=Socrates, predicate=mortal)
• Socrates as the subject in the conclusion
– Label it S
– S will appear as either the subject or predicate in the minor premise
• mortal appears as the predicate in the conclusion
– Label it P
– P will appear as the either the subject or predicate the major premise
Counting Syllogistic Moods
• The major and minor premises, and conclusion in a syllogism is one of four forms:
1. All things S are of type P
2. No things S are of type P
3. Some things S are of type P
4. Some things S are not of type P
• There are 3 statements in a syllogism: Major and minor premises, and conclusion
• Each statement can be one of four forms
• 64 = 43 = 26 syllogistic moods
Syllogistic Figures
• There are 4 different ways to distribute the middle term M and the conclusion’s subject S and predicate P across
Statement
First
Second
Third
Fourth
Major Premise
M–P
P–M
M–P
P–M
the major and minor premises
Minor Premise
S–M
S–M
M–S
M–S
Conclusion
∴S – P ∴S – P ∴S – P ∴S – P
Counting Syllogisms
• A syllogism can have one of 64 different moods
• Terms in syllogism statements can be arranged in 4 figures
• There are
256 = 64 × 4
standard syllogistic forms
• Of these syllogisms, only 15 of them are valid.
Inductive Logic
• Inductive logic uses controlled observations to make inferences
• Inductive logic is credited with sparking the scientific revolution
• Francis Bacon is often credited with establishing the origins of modern inductive logic
• Discussion beyond the scope
79
20
Truth Tables
Epigraph
No matter how correct a mathematical theorem may appear to be, one ought never to be satisfied
that there was not something imperfect about it until it also gives the impression of being
beautiful.
George Boole, British mathematician, (1815–1864)
Recursive Definition of a Logical Formula
.
Definition 8 (Logical Formula). A logical formula is
1. A logical constant: True or False
2. A logical variable: P, Q, R, . . .
3. If P and Q are logical formulas, then so are
(a) Their conjunction P ∧ Q
(b) Their disjunction P ∨ Q
(c) The negation !P
4. Nothing else is a logical formula
Logical Formulas
.
• The following are logical formulas
– P =⇒ !Q
– (!P ∨ Q) ∧ (!Q ∧ R)
– (P ∧ Q)∨!Q
– (!P =⇒ Q) =⇒ R
– (P∧!Q) ≡ (Q =⇒ P)
– !(P∧!Q) ≡ (P =⇒ Q)
– P =⇒ (!Q ∨ R)
– !P =⇒ [(P =⇒ Q) =⇒ Q]
• When are these formulas true?
• When are they formulas false?
Truth Tables
• A truth table shows all possible combinations of truth values for a statement
• Truth tables for “not,” “and,” “or,” “if. . . then. . . , ” and “if and only if” are basic forms from which more complex
statements can be analyzed
• Truth tables can be used to show that a logical formula is always true (a tautology)
• Truth tables can be used to show that a logical formula is always false (a contradiction)
• Most often a logical formula will be true sometimes and false at other times; truth tables will show this
80
Truth Table for Negation
• The truth table for “NOT” is
!P
1
0
P
0
1
• The input column P is white
• The output column !P is gold
Truth Table for Conjunction
• The truth table for “AND” is
P
Q
P∧Q
0
0
1
1
0
1
0
1
0
0
0
1
P
Q
P∨Q
0
0
1
1
0
1
0
1
0
1
1
1
• The input columns P and Q are white
• The output column P ∧ Q is gold
Truth Table for Disjunction
• The truth table for “OR” is
• The input columns P and Q are white
• The output column P ∨ Q is gold
Truth Table for Conditional
• The truth table for “If. . . Then. . . ” is
P
Q
0
0
1
1
0
1
0
1
• The input columns P and Q are white
• The output column P =⇒ Q is gold
81
P
=⇒ Q
1
1
0
1
Truth Table for Equivalence
• The truth table for “If and Only If” is
P
Q
0
0
1
1
0
1
0
1
P
≡Q
1
0
0
1
• The input columns P and Q are white
• The output column P ≡ Q is gold
Constructing a Logical Formula from Its Truth Table
• Given a truth table its logical formula can be easily constructed
• Consider the truth table for the unknown formula f (P, Q)
P
Q
0
0
1
1
0
1
0
1
f (P, Q)
1
0
1
1
• We want to find a formula for the output column f (P, Q)
Constructing a Logical Formula from Its Truth Table
• For every row where f (P, Q) = 1 construct a term (x ∧ y) where
(
P
if P = 1
x=
!P if P = 0
and
(
y=
Q
!Q
if Q = 1
if Q = 0
• The formula f (P, Q) is the disjunction of all of these terms
Constructing a Logical Formula from Its Truth Table
• For the truth table with unknown formula f (P, Q)
P
Q
0
0
1
1
0
1
0
1
• f is true
82
f (P, Q)
1
0
1
1
– when P is false and Q is false, or
– when P is true and Q is false, or
– when P is true and Q is true
• Therefore the formula for the truth table is
f (P, Q) = (!P∧!Q) ∨ (P∧!Q) ∨ (P ∧ Q)
Constructing a Logical Formula from Its Truth Table
• This algorithm for constructing logical formulas can be extended to more variables
• For every row where f (P, Q, R) = 1 construct a term (x ∧ y ∧ z) where
(
P
if P = 1
x=
!P if P = 0
(
y=
and
Q
!Q
(
z=
R
!R
if Q = 1
if Q = 0
if R = 1
if R = 0
• The formula f (P, Q, R) is the disjunction of all of these terms
Constructing a Logical Formula from Its Truth Table
• Consider the truth table for the unknown formula f (P, Q, R)
P
Q
R
0
0
0
0
1
1
1
1
0
0
1
1
0
0
1
1
0
1
0
1
0
1
0
1
f (P, Q, R)
1
1
0
0
0
1
0
1
• The formula for the truth table is
f (P, Q, R) = (!P∧!Q∧!R) ∨ (!P∧!Q ∧ R) ∨ (P∧!Q ∧ R) ∨ (P ∧ Q ∧ R)
Constructing a Logical Formula from Its Truth Table
• In the study of digital logic one studies how to minimize logical formulas that are disjunctions of conjunctions
• It is easy to write formulas that are conjunctions of disjunctions using rows where f is False
• These topics are beyond the scope of this class
83
Tautologies Statements
• Logical formulas that are always True are important
• A logical formula that is always True are called tautologies
• Tautologies can be use in any proof
• Tautologies used in proofs are called inference rules
Example Tautologies
• The following are tautologies (always true)
• The constant True
• Non-contradiction: P∨!P
• Modus ponens: [P ∧ (P =⇒ Q)] =⇒ Q
• Modus tollens: [!Q ∧ (P =⇒ Q)] =⇒ !P
• Hocus pocus: There is no hocus pocus
Example Tautologies
• The following are tautologies (always true)
• Double negation: !(!P) ≡ P
• Reductio ad absurdum: [(P =⇒ Q) ∧ (P =⇒ !Q)] =⇒ !P
• Simplification: (P ∧ Q) =⇒ P
• Resolution: [(P =⇒ Q) ∧ (!P =⇒ R)] =⇒ (Q ∨ R)
Example Tautologies
• The following are tautologies (always true)
• De Morgan’s Laws:
– !(P ∧ Q) ≡!P∨!Q
– !(P ∨ Q) ≡!P∧!Q
• Hypothetical Syllogism: [(P =⇒ Q) ∧ (Q =⇒ R)] =⇒ (P =⇒ R)
• Disjunctive Syllogism: [(P ∨ Q)∧!P] =⇒ Q
Modus Ponens Is A Tautology
• Modus ponens: [P ∧ (P =⇒ Q)] =⇒ Q
• A truth table for modus ponens is
P
Q
0
0
1
1
0
1
0
1
[P∧ (P =⇒ Q)]
0
1
0
1
0
0
1
1
=⇒ Q
1
1
1
1
• Because the gold column (corresponding to the major implication) is all True, modus ponens is a tautology
84
Modus Tollens Is A Tautology
• Modus tollens: [!Q ∧ (P =⇒ Q)] =⇒ !P
• A truth table for modus tollens is
P
Q
0
0
1
1
0
1
0
1
[!Q∧ (P =⇒ Q)]
1
1
0
1
0
0
0
1
=⇒
1
1
1
1
!P
1
1
0
0
• Because the gold column corresponding to the major implication is all True, modus tollens is a tautology
Reductio ad Absurdum Is A Tautology
• Reductio ad Absurdum:
[(P =⇒ Q) ∧ (P =⇒ !Q)] =⇒ !P
• A truth table for reductio ad absurdum is
P
Q
0
0
1
1
0
1
0
1
[(P =⇒ Q) ∧ (P =⇒ !Q)]
1
1
1
1
1
1
0
0
1
1
0
0
=⇒
1
1
1
1
!P
1
1
0
0
• Because the gold column corresponding to the major implication is all True, reductio ad absurdum is a tautology
Contradictions
• Logical formulas that are always False are important
• A logical formula that is always False are called contradictions
• If f (P, Q, . . .) is a contradiction, then its negation ! f (P, Q, . . .) is a tautology
Example Contradictions
• The following are contradictions (always false)
• The constant False
• Contradiction: P∧!P
• The negation of any tautology is a contradiction
85
Example Contradictions
• Construct a truth table to show that
P ∧ [( Q
=⇒ !P) ∧ Q]
is a contradiction
• A truth table for this formula is
P
Q
P∧
0
0
1
1
0
1
0
1
0
0
0
0
[(Q =⇒ !P)
1
1
1
1
1
0
0
0
∧Q]
0
1
0
0
• Because the gold column corresponding to the major ANDis all False, P ∧ [(Q =⇒ !P) ∧ Q] is a contradiction
Contingencies
• Logical formulas that are sometimes True and sometimes False are called contingencies
• A truth table for P =⇒ (Q∨!R) is
P
Q
R
0
0
0
0
1
1
1
1
0
0
1
1
0
0
1
1
0
1
0
1
0
1
0
1
P
=⇒ (Q∨!R)
1
1
1
0
1
1
1
1
1
1
0
0
1
1
1
1
Satisfiability
• A classic problem in computing is called satisifiability, SAT for short
• The SAT problem asks:
– Given a logical formula does it have a truth assignment?
– That is, an assignment of True or False to its variable to make it true
• SAT is a decision problem: It has a “yes” or “no” answer
• SAT is an non-deterministic polynomial time (complete) problem
– A given satisfying truth assignment can be checked for correctness quickly
– No one has ever found an efficient algorithm for finding a satisfying truth assignment
Counting Rows in A Truth Table
• A truth table on n variables will have 2n rows
– We’ve seen
∗ 1 variable formulas have 2 row truth tables: labeled 0 and 1
∗ 2 variable formulas have 4 row truth tables: labeled 00, 01, 10, 11
∗ 3 variable formulas have 8 row truth tables: labeled 000 through 111
– In general, each variable can take two values: 0 and 1
– There are 2n combinations of n things over two values
86
Counting Rows in A Truth Table
• In scheduling and other problems it is not uncommon to find solutions depend on logical formulas that have
tens, if not hundreds, of variables
• Consider a logical formula with 100 variable
• A truth table for such a formula would have 2100 rows
Counting Rows in A Truth Table
• Using the approximation 210 ≈ 103 , find that
2100 ≈ 1030
• Pretend it took one nanosecond (10−9 seconds) to compute the value in a row truth table for a 100 variable
formula
f (P0 , P1 , . . . , P99 )
• It would take about 1030 × 10−9 = 1021 seconds to fill the table
• How long is 1021 seconds?
Counting Rows in A Truth Table
• There are π billion seconds in century, give or take a few.
• A back-of-the-envelope calculation shows
60 minutes 24 hours 365 days 100 years
60 second
×
×
×
×
1 minute
1 hour
1 day
1 year
1 century
seconds
= 3, 153, 600, 000
century
≈ π billion seconds/century
• 1021 seconds is about (3 × π)109 × 1011 seconds
• 1021 seconds is about 3 × 1011 billion centuries
• Filling all 2100 rows in a 100 variable truth table at one row per nanosecond would take about 3 × 1011 billion
centuries
Counting Logical Formulas
• Truth tables provide an easy way to count logical formulas on n variables.
• We’ve seen there are
– 2 logical formulas on 0 variables: True and False
– 4 logical formulas on 1 variables: True, False, P, and !P
– 16 logical formulas on 2 variables
87
Counting Logical Formulas
• Consider how many output columns a truth table on n variables can have (not including columns for scratch
work)
• For each row there are two values that can be written in a column: 0 or 1
• There are 2n rows
n
• So there are 22 output columns that can be constructed
n
• There are 22 logical functions on n variables
Counting Logical Formulas
• Physicist estimate there are about 1080 atoms in the universe
• For what value of n are there more logical formulas than atoms in the universe?
n
• That is, for what value of n is 22 greater than 1080 ?
Counting Logical Formulas
• Consider the inequality
n
22 > 1080
• Take the logarithm of both sides
n
log 22 = 2n log(2) > 80 = log 1080
• Use the approximation log(2) ≈ 0.3010 to find
2n > 266
• Conclude: There are more 9 input logic formulas (logic gates) than there are atoms in the universe!
21
Venn Diagrams
Quotation
Logic (and therefore probability as a branch of logic) is not concerned with what men do believe,
but what they ought to believe, if they are to believe correctly.
John Venn, British philosopher (1834–1923)
Venn Diagrams
• Venn diagrams illustrate relations between sets
• Venn diagrams illustrate relations between predicates
• Represent one set by a circle labeled by its name
U
X
• Enclose the circle within a rectangle representing the universe of elements
• Two regions are constructed: X and X
88
Venn Diagrams
• Represent two sets by overlapping circles labeled by their names
U
Four regions are constructed:
Y
X
X ∩ Y,
X∩Y
X ∩ Y,
X∩Y
Venn Diagrams
• Represent three sets by overlapping circles labeled by their names
U Eight regions can be identified:
Y
Z
X
X ∩ Y ∩ Z,
X ∩ Y ∩ Z,
X ∩ Y ∩ Z,
X ∩ Y ∩ Z,
X ∩ Y ∩ Z,
X∩Y∩Z
X ∩ Y ∩ Z,
X∩Y∩Z
• Do you see a pattern?
• Do you see a relationship between set operations and Boolean operations?
Venn Diagrams
• The 16 regions than can be constructed from 4 sets cannot be represent by 4 overlapping circles
• Do you see why from this attempt to do so?
U
Y
Z
X
W
Venn Diagram: All X are Y
• To represent the proposition “All X are Y” shade the region of X that is not in Y to represent that it is empty
U
Y
X
Shade X outside Y to indicate that
it is empty
• Lack of symmetry in the diagram shows “All X are Y” and “All Y are X” are not logically equivalent
• “All integers are real numbers” versus “All real numbers are integers”
89
Venn Diagram: No X are Y
• To represent the proposition “No X are Y” shade the region X ∩ Y to represent that it is empty
U
Y
X
• By symmetry, this diagram also represents “No Y are X”
• The diagram shows “No X are Y” and “No Y are X” logically equivalent
• “No cats are dogs” and “No dogs are cats”
Venn Diagram: Some X are Y
• To represent the proposition “Some X are Y” mark X ∩ Y to represent that it is not empty
U
Y X X
• By symmetry, this diagram also represents “Some Y are X”
• The diagram shows “Some X are Y” and “Some Y are X” are logically equivalent
• “Some Mersenne numbers are prime” and “Some primes are Mersenne numbers”
Venn Diagram: Some X are not Y
• To represent the proposition “Some X are not Y” mark X ∩ Y to represent that it is not empty
U
Y
XX
• Lack of symmetry in the diagram shows “Some X are not Y” and “Some Y are not X” are not logically equivalent
• “Some dogs numbers are not collies” versus “Some collies are not dogs”
90
Venn Diagram: No non-X are not non-Y
• To represent the proposition “No non-X are not non-Y” shade X ∩ Y to represent that it is empty
U
Y
X
Venn Diagram: Syllogistic Arguments
• Consider a syllogism of the form
– Major premise: All X are Y
U
Y
X
Z
– Minor premise: All Z are X
U
Y
X
Z
– Conclusion: All Z are Y
U
Y
X
22
Z
Predicate Logic
Epigraph
Sometimes I seem to see a difficulty, but then again I don’t see it.
G OTTLOB F REGE , G ERMAN MATHEMATICIAN ,
Predicate Statements
• Predicate logic studies proposition on variables over sets
• Statements in predicate logic have forms such as
– All S are P
– No S are P
– Some S are P
– Some S are not P
91
LOGICIAN , PHILOSOPHER
(1848–1925)
Predicate Statements: Propositions over Sets
• These statements can be written as quantified statements about sets
–
–
–
–
For all x, if x is (in) S, then x is (in) P
For all x, if x is (in) S, then x is not (in) P
For some x, x is (in) S and x is (in) P
For some x, x is (in) S and x is not (in) P
Classes
• In logic, the sets in predicate statements are called classes
• Examples might include, the class of
– athletes
– rock bands
– mammals
– functions on the integers
– natural numbers
– permutations
– green things
– logicians
– floating point numbers
Structure of a Predicate Statement
• Predicate statements, in standard form have four parts
–
–
–
–
–
A quantifier: All, No, or Some
A subject class term: S
Some form of the verb to be: is, are, was, will be, . . .
A predicate class term: P
For example
Quantifier subject verb
predicate
All
squares are plane figures
Quantification
• The quantifier in a predicate statement can be
– Universal: For all values of the variable
– Existential: For some (at least one) values of the variable
Quantification
• Universal statements can be stated
– Positively
∗ All members of the subject class are in the predicate class
– Negatively
∗ No members of the subject class are in the predate class
• Existential statements can be stated
– Positively
∗ At least one member of the subject class is in the predicate class
– Negatively
∗ At least one member of the subject class is not in the predicate class
92
Universal Quantification
• The symbol for universal quantification is ∀, which is read “for all,” “for any,” or similar phrases
• Suppose we were discussing classes: Frogs and Green Things
• A statement such as “Frogs are green” can be written in predicate logic as
(∀x)(Fx =⇒ Gx)
• The affirmative predicate statement (∀x)(Fx =⇒ Gx) means for all things x, if x is a frog, then x is green
Universal Quantification
• Suppose we were discussing classes: Prime numbers and Integers
• An affirmative statement such as “Primes are integers” can be written in predicate logic as
(∀x)(x ∈ P =⇒ x ∈ Z)
• The predicate statement (∀x)(x ∈ P =⇒ x ∈ Z) means for all things x, if x is a prime number, then x is an integer
Universal Quantification
• Suppose we were discussing classes: Dogs and Green Things
• A negative statement such as “No dogs are green” can be written in predicate logic as
(∀x)(Dx =⇒ !Gx)
• The predicate statement (∀x)(Dx =⇒ !Gx) means for all things x, if x is a dog, then x is not green
Universal Quantification
• Suppose we were discussing classes: Irrational numbers and Integers
• A negative statement such as “No irrational numbers are integers” can be written in predicate logic as
(∀x)(x ∈ Q =⇒ x 6∈ Z)
• The predicate statement (∀x)(x ∈ Q =⇒ x 6∈ Z) means for all things x, if x is an irrational number, then x is not
an integer
Existential Quantification
• The symbol for existential quantification is ∃, which is read “for some,” “for at least one,” or similar phrases
• Suppose we were discussing classes: Frogs and Green Things
• An affirmative statement such as “Some frogs are green” can be written in predicate logic as
(∃x)(Fx ∧ Gx)
• The predicate statement (∃x)(Fx ∧ Gx) means for some things x, x is a frog and x is green
93
Existential Quantification
• Suppose we were discussing classes: Prime numbers and Integers
• An affirmative statement such as “Some integers are prime” can be written in predicate logic as
(∃x)(x ∈ Z ∧ x ∈ P)
• The predicate statement (∃x)(x ∈ Z ∧ x ∈ P) means for some things x, x is an integer and x is a prime number
Existential Quantification
• Suppose we were discussing classes: Dogs and Green Things
• A negative statement such as “Some dogs are not green” can be written in predicate logic as
(∃x)(Dx∧!Gx)
• The predicate statement (∃x)(Dx∧!Gx) means for some things x, x is a dog and x is not green
Existential Quantification
• Suppose we were discussing classes: Prime numbers and Integers
• A negative statement such as “Some integers are not prime” can be written in predicate logic as
(∃x)(x ∈ Z ∧ x 6∈ P)
• The predicate statement (∃x)(x ∈ Z ∧ x 6∈ P) means for some things x, x is an integer and x is not a prime number
Existential Quantification: Relation to Sets
• An affirmative existential statement such as
(∃x)(x ∈ S ∧ x ∈ P)
states that the intersection of S and P is not empty
• A symbolic representation for (∃x)(x ∈ S ∧ x ∈ P) is
S ∩ P 6= 0/
• Notice the symmetry: if some S are P, then some P are S
Existential Quantification: Relation to Sets
• A negative existential statement such as “Some S are not P,” symbolically
(∃x)(x ∈ S ∧ x 6∈ P)
states that the intersection of S and the complement of P is not empty
• A symbolic representation for (∃x)(x ∈ S ∧ x 6∈ P) is
S ∩ P 6= 0/
• Notice that if some S are not P it does not follow that some P are not S
94
Universal Quantification: Relation to Sets
• An affirmative universal statement such as
(∀x)(x ∈ S =⇒ x ∈ P)
states that the intersection of S and the complement of P is empty
• The (∀x)(x ∈ S =⇒ x ∈ P) is equivalent to
(∀x)(x 6∈ S ∨ x ∈ P)
Universal Quantification: Relation to Sets
• By De Morgan’s law (∀x)(x 6∈ S ∨ x ∈ P) is equivalent to
(∀x)(!(x ∈ S ∧ x 6∈ P))
• And (∀x)(!(x ∈ S ∧ x 6∈ P)) is equivalent to
(∀x)(x 6∈ (S ∩ P))
• This last expression states that S ∩ P is empty
• A symbolic representation for (∀x)(x ∈ S =⇒ x ∈ P) is
S ∩ P = 0/
Universal Quantification: Relation to Sets
• A negative universal statement such as
(∀x)(x ∈ S =⇒ x 6∈ P)
states that the intersection of S and P is empty
• The statement (∀x)(x ∈ S =⇒ x 6∈ P) is equivalent to
(∀x)(x 6∈ S ∨ x 6∈ P)
Universal Quantification: Relation to Sets
• By De Morgan’s law (∀x)(x 6∈ S ∨ x 6∈ P) is equivalent to
(∀x)(!(x ∈ S ∧ x ∈ P))
• This last expression states that S ∩ P is empty
• A symbolic representation for (∀x)(x ∈ S =⇒ x 6∈ P) is
S ∩ P = 0/
95
Well-Formed Formulas
• To express the ideas of mathematics more complex predicate statements must be created
• To form these more complex expressions we need an alphabet of symbols and a grammar for combining symbols
• There are logical symbols
– Logical operators: ∧, ∨, !, =⇒ , and ≡
– Quantifiers: ∀ and ∃
– Variables: u, v, w, x, y, z ranging over elements in a class
– Delimiters: (, ), [, ]
• And what are called non-logical symbols
– Constants: a, b, c, d naming particular elements in a class
– Predicates: S, P, A naming classes
Well-Formed Formulas
• Well-formed formulas are defined recursively
• An atomic formula is a predicate letter followed zero or more constants
• A predicate letter with no following constant is a proposition
• Well-formed formulas can be defined recursively
– An atomic formula is well-formed
– If φ is well-formed, so is its negative !φ
– If φ and ψ are a well-formed, so are
∗
∗
∗
∗
The conjunction φ ∧ ψ
The disjunction φ ∨ ψ
The condition φ =⇒ ψ
The equivalence φ ≡ ψ
– If x is a variable and φ is well-formed, then
∗ (∀x)(φ x) is a well-formed formula
∗ (∃x)(φ x) is a well-formed formula
Why is Quantification Theory Needed?
• Consider Aristotle’s famous syllogism
– Major premise: All humans are mortal
– Minor premise: Socrates is human
– Conclusion: Socrates is mortal
• Let H and M denote the classes of humans and mortal things
• The major premise is a proposition: H =⇒ M
• The minor premise is a proposition: Hs
• The conclusion is a proposition: Ms
96
Why is Quantification Theory Needed?
• As propositional statements Aristotle’s argument is
[(H =⇒ M) ∧ Hs] =⇒ Ms
• Propositional logic is not capable of showing this argument is a valid tautology
• Propositional logic is does not have the expressive power to represent Aristotle’s argument
• There is no propositional logic truth table for Aristotle’s syllogism
Why is Quantification Theory Needed?
• Consider the statement
“Every philosopher admires some logician.”
• There is ambiguity in the statement with which Aristotelian logic cannot cope
1. If x is a philosopher, there is some logician y and x admires y
2. There is some logician y and all philosophers x admire y
• Quantification theory, at least, allows unambiguous expression of these two interpretations
1. (∀x)(Px =⇒ (∃y)(Ly ∧ xAy))
2. (∃y)(Ly ∧ (∀x)(Px =⇒ xAy))
Examples of Predicate Statements
• Some dogs are animals that jump
(∃x)[Dx ∧ (Ax ∧ Jx)]
• If all dogs are mammals, then some dogs are not reptiles
(∀x)(Dx =⇒ Mx) =⇒ (∃x)(Dx∧!Rx)
• Twin prime conjecture: There are infinitely many prime pairs (p, p + 2)
(∀n)[(∃p)((p > n) ∧ (p, p + 2 ∈ P))]
• Goldbach’s conjecture: every even integer greater than 2 can be written as the sum of two primes
(∀n)[(n > 2) ∧ (n % 2 = 0) =⇒ (∃p, q)(p, q ∈ P ∧ n = p + q)]
Examples of Predicate Statements
• Function f : X −→ Y is onto
(∀y)(y ∈ Y ∧ (∃x ∈ X)( f (x) = y))
• Function f : X −→ Y is one-to-one
(∀x0 )(∀x1 )[(x0 6= x1 ) =⇒ ( f (x0 ) 6= f (x1 ))]
97
Inference Rules for Predicate Logic
• Inference rules are needed to prove theorems about predicate statements
• There are four basic inference rules for quantified statements
1. Universal Instantiation
2. Universal Generalization
3. Existential Instantiation
4. Existential Generalization
Universal Instantiation
• If a quantified statement is true for all values of its variable, then it is true for some particular constant value
• For example, if all squared real numbers are non-negative then c2 ≥ 0 for some constant c
• Or, in syncopated notation
(∀x)(x ∈ R =⇒ x2 ≥ 0)
• In general, if c be a constant, then
(∀x)(P(x) =⇒ P(c))
Existential Instantiation
• If a quantified statement is true for at least one value of its variable, then it is true for some particular value
• For example, if some Mersenne number is prime, then 2n − 1 is prime for some natural number n
• Or, in syncopated notation
(∃x)[(x ∈ M ∩ P) =⇒ (∃c)[(c ∈ M) ∧ (c ∈ P)]
• In general, if c be a constant, then
(∃x)(P(x) =⇒ (∃c)(P(c)))
Universal Generalization
• If a unquantified statement is true for an individual variable, then it is true for all values of the variable
• For example, to prove the square of all even integers are divisible by 4
– Let n = 2k be an even integer
– Then n2 = 4k2 is divisible by 4
• Since no restriction was made on n except that n = 2k, we can generalize to reach the conclusion: all evens
squared are divisible by 4
• The syncopated statement of universal generalization is
P (y)
=⇒ (∀x)(P(x))
where y is an arbitrary variable
98
Existential Generalization
• If a unquantified statement is true for a particular value its variable, then it is true for some value of the variable
• For example, if (−2)2 − 4 = 0, then there exists an x such that x2 − 4 = 0
• In syncopated statement of existential generalization is,
(P(c)) =⇒ (∃x)(P(x)))
where c is a constant
Equivalence of Quantifiers Via Negation
• Negating an universal statement about P(x) yields an existential statement about !P(x)
[!(∀x)(P(x))] ≡ [(∃x)(!P(x))]
• For example, if it is not true that x2 > 0 for all x, then there exists an x such that x2 ≤ 0, and vice versa
• Negating an existential statement about P(x) yields an universal statement about !P(x)
[!(∃x)(P(x))] ≡ [(∀x)(!P(x))]
• For example, if there does not exist an x such that x2 < 0, then for all x x2 ≥ 0, and vice versa
Commutative Rules for Quantifiers
• When can the order of quantifiers be exchanged?
• Consider the three cases
?
1. (∀x)(∀y)(P(x, y)) = (∀y)(∀x)(P(x, y))
?
2. (∃x)(∃y)(P(x, y)) = (∃y)(∃x)(P(x, y))
?
3. (∀x)(∃y)(P(x, y)) = (∃y)(∀x)(P(x, y))
(∀x)(∀y) = (∀y)(∀x)
• Consider the statement
(∀x)(∀y)(P(x, y))
• For each value of x and each value of y, the statement P(x, y) is true
• Since “and” is commutative, for each value of y and each value of x, the statement P(x, y) is true
• That is,
(∀x)(∀y)(P(x, y)) = (∀y)(∀x)(P(x, y))
(∀x)(∀y) = (∀y)(∀x)
• Consider the statement
(∀x)(∀y)[((x < 0) ∧ (y < 0)) =⇒ (xy > 0)]
• xy > 0 is true for all negative numbers x and y
• The order of the quantifiers (∀x)(∀y) does not affect the meaning or truth value of the statement
99
(∃x)(∃y) = (∃y)(∃x)
• Consider the statement
(∃x)(∃y)(P(x, y))
• For some value of x there is a value of y for which the statement P(x, y) is true
• This is equivalent to saying for this value of y there is the value x such that P(x, y) is true
• That is,
(∃x)(∃y)(P(x, y)) = (∃y)(∃x)(P(x, y))
(∃x)(∃y) = (∃y)(∃x)
• Consider the statement
(∃x)(∃y)[(x2 > y) ∧ (x < y)]
• For some value of x, say x = −2, there is a value of y, say y = 1, such that (−2)2 > 1 and −2 < 1
• For this same y, the same x satisfies the statement
(∃x)(∀y) 6= (∀y)(∃x)
• In general, the order of ∀ and ∃ cannot be changed
• The statement
(∀x)(∃y)(x + y = 0)
is true
• For every x there is a number y = −x such that x + y = x + (−x) = 0
• The statement
(∃y)(∀x)(x + y = 0)
is false
• There is no number y such that x + y = 0 for every x
• For a given instance of y, say y = 5, x + 5 = 0 cannot be true for all values of x
23
Relations
Epigraph
I did not have $*xu@! relations with that woman.
Bill Clinton, January 26, 1998 address to the public,” American president (1946 –)
Relation: Definition
• A relation r on set X is a subset of the Cartesian product X × X
r ⊆ X×X
• A relation r from X to Y is a subset of X × Y.
r ⊆ X×Y
• The set X is called the domain of the relation r
• The set Y is called the range of the relation r
• If x ∈ X and y ∈ Y are related by r, write xry
100
Common Relations
• Equals: x = y
– Equality is the set of ordered pairs
== {(x, y) : x = y}
– Equality on the integers is the set of ordered pairs
== {(0, 0), (1, 1), (−1, −1), (2, 2), (−2, −2), . . .}
• Less than or equal: x ≤ y
– Less than or equal is the set of ordered pairs
≤= {(x, y) : x ≤ y}
– Less than or equal on the reals is the set of ordered pairs
≤= {(x, y) : x, y ∈ R && x ≤ y}
Common Relations
• Subset: A ⊆ B
– Subset is the set of ordered pairs
⊆= {(A, B) : A ⊆ B}
– Subset on the set of sets {0,
/ {0}, {1}, {0, 1}} is the set of ordered pairs
⊆ = {(0,
/ 0),
/ (0,
/ {0}), (0,
/ {1}), (0,
/ {0, 1}),
({0}, {0}), ({0}, {0, 1}),
({1}, {1}), ({1}, {0, 1}),
({0, 1}, {0, 1})}
Common Relations
• Divides: a|b
– Divides is the set of ordered pairs
| = {(x, y) : y = cx for some c ∈ N}
– Divides on the natural numbers is the set of ordered pairs
| = { (0, 0),
(1, 0), (1, 1), (1, 2), (1, 3), . . .
(2, 0), (2, 2), (2, 4), (2, 6), . . .
(3, 0), (3, 3), (3, 6), (3, 9), . . .
(4, 0), (4, 4), (4, 8), (4, 12), . . .
···}
101
Common Relations
• Congruence mod m: a ≡ b (mod m)
– Congruence mod m is the set of ordered pairs
(mod m) = {(x, y) : x − y = cm for some c}
– Congruence mod 2 on the integers is the set of ordered pairs
(mod 2) = {(2n, 2m), (2n + 1, 2m + 1) : n, m ∈ Z}
– Congruence mod 3 on the integers is the set of ordered pairs
(mod 3) = {(3n, 3m), (3n + 1, 3m + 1), (3n + 2, 3m + 2) : n, m ∈ Z}
Counting Relations From X to Y
• Let n = |X| and m = |Y|
• The Cartesian product X × Y has nm elements
• A set with nm elements has 2nm subsets
• There are 2nm = 2|X||Y| relations from X to Y
Counting Relations On X
• Let n = |X|
• The Cartesian product X × X has n2 elements
2
• A set with n2 elements has 2n subsets
2
2
• There are 2n = 2|X| relations on X
Counting Relations
• Pretend we are interested in relations on the set
X = {0, 1, 2, 3, 4}
• The cardinality of X is |X| = 5
• The cardinality of X × X is 5 × 5 = 25
• There are 225 subsets of X × X
• Each subset of X × X is a relation on X
• There are 225 relations on X
Properties of Relations
Epigraph
At the time, Nixon was normalizing relations with China. I figured that if he could normalize
relations, then so could I.
E. F. (Ted) Codd, British computer scientist (1923 – 2003)
102
Reflexive Property
Definition 9 (Reflexive). A relation r on set X is reflexive if
xrx
for all x ∈ X
• In English: “A relation is reflexive if (every element is related to itself)”
Reflexive Relations
• The following relations are reflexive
– Equality: a = a
– Less than or equal: a ≤ a
– Divides: a|a
– Subset: A ⊆ A
– Congruence modulo m: a ≡ a (mod m)
– Parallel Lines: LkL
– Implication: P =⇒ P
Not Reflexive Relations
• The following relations are not reflexive
– Less than: a < a is false
– Proper subset: A ⊂ A is false
– Perpendicular Line: L ⊥ L is false
Symmetric Property
Definition 10 (Symmetric). A relation r on set X is symmetric if
for all x, y ∈ X
xry =⇒ yrx,
• In English: “A relation is symmetric when (a is related to b implies b is related to a)”
Symmetric Relations
• The following relations are symmetric
– Equality: if a = b, then b = a
– Not Equal if a 6= b, then b 6= a
– Congruence modulo m: if a ≡ b (mod m), then b ≡ a (mod m)
– Parallel Lines: if LkM, then MkL
– Perpendicular Lines: if L ⊥ M, then M ⊥ L
Not Symmetric Relations
• The following relations are not symmetric
– Less than or equal: a ≤ b does not imply b ≤ a
– Subset: A ⊆ B does not imply B ⊆ A
103
Antisymmetric Property
Definition 11 (Antisymmetric). A relation r on set X is antisymmetric if
(xry ∧ yrx) =⇒ x = y,
for all x, y ∈ X
• In English: “A relation is symmetric when (a is related to b and b is related to a implies a is equal to b)”
Antisymmetric Relations
• The following relations are antisymmetric
– Equality: If a = b and b = a, then a = b
– Divides on the natural numbers: If a|b and b|a, then a = b
– Subset on the power set of a set: If A ⊆ B and B ⊆ A, then A = B
Not Antisymmetric Relations
• The following relations are not antisymmetric
– Inequality: “If a 6= b and b 6= a, then a = b” is false
– Congruence mod m: “If a ≡ b (mod m) and b ≡ a (mod m), then a = b” is false
– Parallel lines: “If AkB and BkA, then A = B” is false
Transitive Property
Definition 12 (Transitive). A relation r on set X is transitive if
(xry ∧ yrz) =⇒ xrz,
for all x, y, z ∈ X
• In English: “A relation is transitive when (a is related to b and b is related to c implies a is related to c)”
Transitive Relations
• The following relations are transitive
– Equality: If a = b and b = c, then a = c
– Divides on the natural numbers: If a|b and b|c, then a|c
– Subset on the power set of a set: If A ⊆ B and B ⊆ C, then A ⊆ C
– Congruence mod m: If a ≡ b (mod m) and b ≡ c (mod m), then a ≡ c (mod m)
Not Transitive Relations
• The following relations are not transitive
– Inequality: “If a 6= b and b 6= c, then a 6= c” can be false
– Perpendicular lines: “If A ⊥ B and B ⊥ C, then A ⊥ C” is false
Relations Among People
• Describing relations among people can be problematic
• But they can be fun, and perhaps instructive
• Consider these relations: Father, Mother, Sibling, Friend, Married
• Can you determine if these relations are reflexive, symmetric, antisymmetric, or transitive?
104
The Father Relation
• Fatherhood is reflexive if everyone is there own father
• Fatherhood is symmetric if x is the father of y implies y is the father of x
• Fatherhood is antisymmetric if x is the father of y and y is the father of x implies x is y
– I think this is true because there are no x’s or y such that x is the father of y and y is the father of x
• Fatherhood is transitive if x is the father of y and y is the father of z implies x is the father of z
The Sibling Relation
• Sibling is reflexive if everyone is there own sibling (brother or sister)
• Sibling is symmetric if x is a sibling of y implies y is a sibling of x
• Sibling is antisymmetric if x is the sibling of y and y is a sibling of x implies x is y
• Sibling is transitive if x is a sibling of y and y is a sibling of z implies x is a sibling of z
The Friend Relation
• Friendship is reflexive if everyone is there own friend
• Friendship is symmetric if x is a friend of y implies y is a friend of x
• Friendship is antisymmetric if x is a friend of y and y is friend of x implies x is y
• Friendship is transitive if x is a friend of y and y is a friend of z implies x is a friend of z
The Married Relation
• Married is reflexive if everyone is married to themselves
• Married is symmetric if x is a married to y implies y is married to x
• Married is antisymmetric if x is married to y and y is married to x implies x is y
• Married is transitive if x is married to y and y is married to z implies x is married z
The Sex Relation
• Sex is reflexive if everyone has sex with themselves
• Sex is symmetric if x has sex with y implies y has sex with x
• Sex is antisymmetric if x sex with y and y has sex with x implies x is y
• Sex is transitive if x has sex with y and y has sex with z implies x sex with z
105
24
Graphs to Represent Relations
Visualization of Relations
• It can be useful to visualize a relation
• Cartesian sketches are well known visualizations
• If x is related to y, place a mark point (x, y) in a Cartesian coordinate system
• Here are a few examples
y = f (x)
1 x
x20
f (x) = sin
ex
x
Node-and-Edge Graphs
• In computer science it is more common to draw a graph as a collection nodes with connecting edges
• Nodes are also called vertices
• Edges can be directed or undirected
– Undirected edges are drawn as lines without arrow heads
– Directed edges are drawn as arrows and called arcs
Graphs of Common Relations: Less Than
• Here’s one way to draw the graph of “less than” on the set {0, 1, 2}
2
2
1
1
0
0
Graphs of Common Relations: Divides
• Here’s one way to draw the graph of “divides” on the set {1, 2, 3, 4, 5, 6}
106
6
6
5
5
4
4
3
3
2
2
1
1
Graphs of Common Relations: (mod 3)
• Here’s one way to draw the graph of (mod 3) on the set {0, 1, 2, 3, 4, 5, 6}
6
6
5
5
4
4
3
3
2
2
1
1
0
0
The Degree of a Node
• The out-degree of a node is the number of arcs coming out of a node domain (on the left)
– The out-degree of node v is denoted by deg+ (v)
• The in-degree is the number of arcs coming into a node in the range (on the right)
– The in-degree of node v is denoted by deg− (v)
• Can you list the out-degree and in-degree of each node in the above diagrams?
Important Properties of Relations
• There are several important properties that can be defined in terms of in-degree and out-degree
• If the in-degree of each range node is 1 or more, the relation is said to be onto
• If the out-degree of each domain node is 1, the relation is said to be a function
• If a relation is a function and the in-degree of each range node is 1 or less, the relation is said to be one-to-one
107
Graphs of Common Relations: Subset
• Consider the subset relation on the power set of B = {0, 1}
{0, 1}, deg− = 4
deg+ = 1, {0, 1}
deg+ = 2, {1}
{1}, deg− = 2
deg+ = 2, {0}
{0}, deg− = 2
0,
/ deg− = 1
deg+ = 4, 0/
• This subset relation is
– Onto its range (all range nodes have in-degree ≥ 1)
– Not one-to-one (some range nodes have in-degree > 1)
– Not a function (some domain nodes have out-degree > 1)
Graphs of Common Relations: Divides
• Consider the divides relation on the set {2, 3, 5, 7}
deg+ = 1, 7
7, deg− = 1
deg+ = 1, 5
5, deg− = 1
deg+ = 1, 3
3, deg− = 1
deg+ = 1, 2
2, deg− = 1
• This relation is
– Onto its range (all range nodes have in-degree ≥ 1)
– One-to-one (all range nodes have in-degree ≤ 1)
– A function (all domain nodes have out-degree = 1)
Graphs of Common Relations: Squared
• Consider the squared relation on the set {−2, −1, 0, 1, 2} to the set {0, 1, 4}
deg+ = 1, 2
deg+ = 1, 1
deg+ = 1, 0
4, deg− = 2
deg+ = 1, −1
1, deg− = 2
deg+ = 1, −2
0, deg− = 1
108
• This relation is
– Onto its range (all range nodes have in-degree ≥ 1)
– Not one-to-one (some range nodes have in-degree > 1)
– A function (all domain nodes have out-degree = 1)
Graphs of Common Relations: Square Root
• Consider the square root relation on the set {0, 1, 4} to the set {−2, −1, 0, 1, 2}
2, deg− = 1
1, deg− = 1
deg+ = 2, 4
0, deg− = 1
deg+ = 2, 1
−1, deg− = 1
deg+ = 1, 0
−2, deg− = 1
• This relation is
– Onto its range (all range nodes have in-degree ≥ 1)
– Not one-to-one (since it is not a function)
– Not a function (some domain nodes have out-degree > 1)
25
Incidence Matrices to Represent Relations
Matrix Representation of Relations
• Relations can be represented as a matrix
• Label rows by elements in X
• Label columns by elements in Y
• Matrix entry in row x, column y is 1 if x is related to y
• If x is not related to y, then the entry in row x, column y is 0
Matrix Representation of Relations
• The matrix representation for “less than” on the set
X = {0, 1, 2, 3, 4}
0
1
2
3
4
0
0
0
0
0
0
1
1
0
0
0
0
2
1
1
0
0
0
3
1
1
1
0
0
4
1
1
1
1
0
• When the row value (0, 1, 2, 3, 4) is less than the column value the matrix entry is 1
109
Identifying Relation Properties from Its Matrix
• The relation properties: reflexive, symmetric, and antisymmetric can be readily identified from a matrix
• Reflexive relations have 1’s along their main diagonal
• Symmetric relations have equal values in symmetric entries: (i, j) and ( j, i)
– Symmetric relations have matrices that equal their transpose
• Antisymmetric relations have matrices that do not have 1’s in symmetric entries
Identifying Relation Properties from Its Matrix
• The relation properties: onto, function, and one-to-one can be readily identified from a matrix
• Onto relations have at least one 1 in every column
• Functions have exactly one 1 in every row
• One-to-one functions have exactly one 1 in every row no more than one 1 is every column
Reflexive Property of a Matrix
• When each element is related to itself, the relation is said to be reflexive
• A reflexive relation has 1’s along its main diagonal
• Consider relation represented by the matrix below
x
y
z
u
v
w
x
1
0
1
0
0
0
y
0
1
0
1
0
0
z
0
0
1
0
1
1
u
1
0
1
1
0
1
v
0
0
1
0
1
0
w
0
1
0
1
0
1
4
1
0
1
0
1
0
5
0
1
0
1
0
1
• This relation is reflexive: There is a 1 along the main diagonal
Reflexive Property of a Matrix
• Consider the (mod 2) relation on the set {0, 1, 2, 3, 4, 5}
• Its matrix representation is
0
1
2
3
4
5
0
1
0
1
0
1
0
1
0
1
0
1
0
1
2
1
0
1
0
1
0
3
0
1
0
1
0
1
• The (mod 2) relation is reflexive: There is a 1 along the main diagonal
110
Symmetric Property of a Matrix
• When x is related to y implies y is related to x, the relation is said to be symmetric
• A relation is symmetric if the value in row i column j is always the same as the value in row j column i
• Consider relation represented by the matrix below
x
0
0
0
1
0
0
x
y
z
u
v
w
y
0
1
0
0
1
0
z
0
0
0
0
0
1
u
1
0
0
1
0
1
v
0
1
0
0
0
0
w
0
0
1
1
0
1
Relation Properties from Its Matrix: Symmetry
• Consider the (mod 3) relation on the set {0, 1, 2, 3, 4, 5, 6}
0
1
0
0
1
0
0
1
0
1
2
3
4
5
6
1
0
1
0
0
1
0
0
2
0
0
1
0
0
1
0
3
1
0
0
1
0
0
1
4
0
1
0
0
1
0
0
5
0
0
1
0
0
1
0
6
1
0
0
1
0
0
1
• The the (mod 3) relation is symmetric: If entry (i, j) is 0, then so is entry ( j, i), while if entry (i, j) is 1, then
so is entry ( j, i)
Antisymmetric Property of a Matrix
• When x is related to y and y is related to x implies x = y, the relation is said to be antisymmetric
• A relation is antisymmetric when the value in row i column j is a 1, then value in row j column i always 0
• Consider relation represented by the matrix below
x
y
z
u
v
w
x
1
0
1
0
0
1
y
0
1
0
1
0
0
z
0
0
1
0
1
0
u
1
0
0
1
0
0
v
0
0
0
0
1
0
w
0
1
0
1
0
1
• This relation is antisymmetric: There is never a 1 in symmetric off-diagonal entries
Relation Properties from Its Matrix: Antisymmetry
111
• Consider the “divides” relation on the set {0, 1, 2, 3, 4, 5, 6}
0
1
2
3
4
5
6
0
1
1
1
1
1
1
1
1
0
1
0
0
0
0
0
2
0
1
1
0
0
0
0
3
0
1
0
1
0
0
0
4
0
1
1
0
1
0
0
5
0
1
0
0
0
1
0
6
0
1
1
1
0
0
1
• The the divides relation is antisymmetric: If entry (i, j) is 1, then entry ( j, i) does not, unless i = j
Relation Properties from Its Matrix: Onto
• A relation is onto if every column contains at least one 1
• The relation defined by the matrix
0
1
1
1
1
0
1
2
3
1
0
1
0
0
2
0
1
1
0
3
0
1
0
1
4
0
0
0
1
is onto (every column contains a 1)
• This relation is not onto
0
1
2
3
0
1
1
1
1
1
0
1
0
0
2
0
0
0
0
3
0
1
0
1
Relation Properties from Its Matrix: Function
• A relation is a function if every row contains exactly one 1
• The relation defined by the matrix
0
1
2
3
0
1
1
0
0
1
0
0
0
0
2
0
0
0
1
3
0
0
1
0
0
1
2
3
0
1
0
1
1
1
0
1
0
0
2
0
0
0
0
3
0
1
0
1
is a function
• The relation defined by the matrix
is not a function
112
Matrix Property: Onto Function
• Let f be a function from X to Y
• The function f cannot be onto when the cardinality of Y is greater than the cardinality of X,
• This is an example of the pigeonhole principle
– We are required to place only one 1 in each row
– There are more columns than rows
– Some column will not contain a 1
0
1
2
3
0
1
1
0
0
1
0
0
0
0
2
0
0
0
1
3
0
0
1
0
4
0
0
0
0
Matrix Properties: One-to-One Function
• A function is one-to-one if every row contains exactly one 1 and no more than one 1 in any column
• The relation defined by the matrix
0
1
2
3
0
1
0
0
0
1
0
0
0
1
2
0
1
0
0
3
0
0
1
0
0
1
2
3
0
1
0
0
0
1
0
1
1
0
2
0
0
0
0
3
0
0
0
1
is a one-to-one function
• The relation defined by the matrix
is not a one-to-one function
Counting Relations By Counting Matrices
• Pretend X has n elements (the cardinality of X is n, denoted |X| = n)
• A matrix to represent a relation on X will have n rows and n columns
• A matrix to represent a relation on X will have n × n = n2 entries
• There are two choices for the value of a matrix entry: 0 or 1
2
• Therefore there are 2n different relations on X
Counting Relations On the Empty Set
• There is one relation that can be defined on the empty set: The empty relation
• A relation on the the empty set is a subset of 0/ × 0/
• The only subset of 0/ × 0/ is the empty set 0/
2
• The number of relations on a 0 element set is 20 = 1
113
Counting Relations On a One Element Set
• There is two relation that can be defined a one element set {x}
• The empty relation is one of them
– The empty relation on a one element set is represented by the matrix
x
0/ ⊆ {(x, x)}
x
0
• The identity relation is the second
– The identity relation on a one element set is represented by the matrix
x
{(x, x)} ⊆ {(x, x)}
x
1
2
• The number of relations on a 1 element set is 21 = 2
Counting Relations On a Two Element Set
• There is 16 relation that can be defined a two element set X = {x, y}
• The Cartesian product is
X × X = {(x, x), (x, y), (y, x), (y, y)}
• There are 24 = 16 subsets of X × X
• Each of the 16 subsets is a relation on X
• There are 24 = 16 relations on X
Counting Relations On a Two Element Set
• There are 41 = 4 relations with one 1 and three 0’s in their matrix representation
x
y
• There are
4
3 =4
x
1
0
y
0
0
x
y
x
0
0
y
1
0
x
y
x
0
1
y
0
0
x
y
x
0
0
y
0
1
relations with three 1’s and one 0 in their matrix representation
x
y
x
1
1
y
1
0
x
y
x
1
0
y
1
1
x
y
x
0
1
y
1
1
x
y
x
1
1
y
0
1
Counting Relations On a Two Element Set
• There are 42 = 6 relations with two 1’s and two 0’s in their matrix representation
x
y
x
1
0
y
0
1
x
y
x
0
1
y
0
1
x
y
x
0
1
y
1
0
x
y
x
1
1
y
0
0
114
x
y
x
1
0
y
1
0
x
y
x
0
0
y
1
1
Counting Relations On a Two Element Set
• The empty relation on a two element set is represented by the matrix
x
y
x
0
0
y
0
0
• The Appalachian (pardon me) relation on a two element set is represented by the matrix
x
y
x
1
1
y
1
1
2
• The number of relations on a 2 element set is 22 = 16
Counting Relations On a 3, 4, 5, . . . Element Set
2
• There are 23 = 29 = 512 relations on a 3 element set (too many matrices to list)
2
• There are 24 = 216 = 65536 relations on a 4 element set
2
• There are 25 = 225 = 33 554 432 relations on a 5 element set
Counting Relations On a 3 Element Set
• A relation on a 3 element set is represented by a 3 × 3 matrix
• The number of 3 × 3 matrices with only 0’s or 1 entries is the number of relations on the set
• A 3 × 3 matrix has 9 elements
Counting Relations On a 3 Element Set
• There is 90 = 1 matrices without a 1 (the empty relation)
• There is 91 = 9 matrices with one 1
• There is 92 = 9(8)/2 = 36 matrices with two 1’s
• Continuing the patterns, there are
9
9
9
9
9
=
+
+
·
·
·
= 29
∑ k
0
1
9
k=0
different matrices
• There are 29 = 512 relations on a 3 element set
Counting Relations On an n Element Set
• A matrix representing a relation on X will have
– n rows
– n columns
– n × n = n2 entries
• Each of the n2 entries can take on one of two values: 0 or 1
• There are 2 choices for each of the n2 entries in an n × n incidence matrix
2
• There are 2n different incidence matrices for an n element set
2
• There are 2n different relations on an n element set
115
Counting Relations With Special Properties
• The matrix representation of a relation allows easy counting of relations with special properties
• Let X have cardinality n = |X|
• An n × n incidence matrix representing a relation on X has
– n entries along its diagonal
– 1 + 2 + · · · + (n − 1) = n(n − 1)/2 entries above the diagonal
– 1 + 2 + · · · + (n − 1) = n(n − 1)/2 entries below the diagonal
– Note that n + n(n − 1)/2 + n(n − 1)/2 = n + n(n − 1) = n2
Counting Reflexive Relations
• Consider a matrix for a reflexive relation on a 2 element set
x
1
b
x
y
y
a
1
• The incidence matrix of a reflexive relation has a 1 along its diagonal
– There are 2 choices (0 or 1) for the off-diagonal entries x and y
– There are 22 = 2 reflexive relations on a 2 element set
x
y
x
1
0
y
0
1
x
y
x
1
1
y
1
1
x
y
x
1
0
y
1
1
x
y
x
1
1
y
0
1
Counting Reflexive Relations
• Consider a matrix for a reflexive relation on a 3 element set
x
y
z
x
1
c
e
y
a
1
f
z
b
d
1
• There are 2(1 + 2) = 3(3 − 1) = 6 off-diagonal entries for a 3 × 3 matrix
– There are 2 choices (0 or 1) for the 6 off-diagonal entries: a, b, c, d, e, f
– There are 26 = 64 reflexive relations on a 3 element set
• Consider a matrix for a reflexive relation on a n element set
• There are 2(1 + 2 + · · · + (n − 1)) = n(n − 1) off-diagonal entries for an n × n matrix
• There are 2n(n−1) reflexive relations on an n element set
116
Counting Symmetric Relations
• Consider a matrix for a symmetric relation on a 2 element set
x
a
b
x
y
y
b
c
• The incidence matrix of a symmetric relation equal symmetric off-diagonal entries
• There are 2 diagonal entries
• There are 2(2 − 1)/2 = 1 entries above the diagonal
• There are 1 + 2 = 2(2 + 1)/2 = 3 entries on or above the diagonal
• For a symmetric relation, the entries above the diagonal determine the entries below the diagonal
Counting Symmetric Relations
• There are 22+1 = 8 symmetric relations on a 2 element set
x
y
x
1
0
y
0
1
x
y
x
0
0
y
0
0
x
y
x
0
0
y
0
1
x
y
x
1
0
y
0
0
x
y
x
1
1
y
1
1
x
y
x
0
1
y
1
0
x
y
x
0
1
y
1
1
x
y
x
1
1
y
1
0
Counting Symmetric Relations
• Consider a matrix for a symmetric relation on a 3 element set
x
a
b
c
x
y
z
y
b
d
e
z
c
e
f
• There are 3 diagonal entries
• There are 3(3 − 1)/2 = 3 entries above the diagonal
• There are 1 + 2 + 3 = 3(3 + 1)/2 = 6 entries on or above the diagonal
• For a symmetric relation, the entries above the diagonal determine the entries below the diagonal
• There are 23+3 = 26 = 64 symmetric relations on a 3 element set
Counting Symmetric Relations
• Consider a matrix for a symmetric relation on an n element set
• There are n diagonal entries
• There are 1 + 2 + · · · + (n − 1) = n(n − 1)/2 entries above the diagonal
• There are 1 + 2 + · · · + (n − 1) + n = n(n + 1)/2 entries on or above the diagonal
• There are 2n+n(n−1)/2 = 2n(n+1)/2 symmetric relations on an n element set
117
Counting Antisymmetric Relations
• Consider matrices for an antisymmetric relation on a 2 element set
x
y
x
a
0
y
0
b
x
a
0
x
y
y
1
b
x
y
x
a
1
y
0
b
• There are 22 = 4 ways to choose diagonal entries
Counting Antisymmetric Relations
• There are 3 choices for symmetric off-diagonal entries of an antisymmetric relation
– Both are 0
– The upper entry is 0 and the lower entry is 1
– The upper entry is 1 and the lower entry is 0
• There are 4 · 3 = 12 antisymmetric relations on a 2 element set
x
y
x
0
0
y
0
0
x
y
x
1
0
y
0
0
x
y
x
0
0
y
0
1
x
y
x
1
0
y
0
1
x
y
x
0
1
y
0
0
x
y
x
1
1
y
0
0
x
y
x
0
1
y
0
1
x
y
x
1
1
y
0
1
x
y
x
0
0
y
1
0
x
y
x
1
0
y
1
0
x
y
x
0
0
y
1
1
x
y
x
1
0
y
1
1
Counting Antisymmetric Relations
• Consider a matrix for an antisymmetric relation on a 3 element set
x
a
d
e
x
y
z
y
b
f
h
z
c
g
i
• There are 3 diagonal entries and 23 ways to choose them
• There are 1 + 2 = 3(3 − 1)/2 = 3 entries above the diagonal
• There are 1 + 2 + 3 = 3(3 + 1)/2 = 6 entries on or above the diagonal
• For a antisymmetric relation, the entries above the diagonal determine the entries below the diagonal
• There are 23 33 = 8 · 27 antisymmetric relations on a 3 element set
Counting Antisymmetric Relations
• Consider a matrix for a symmetric relation on an n element set
• There are n diagonal entries
• There are 2n ways to choose these diagonal entries
• There are 1 + 2 + · · · + (n − 1) = n(n − 1)/2 entries above the diagonal
• There are 3n(n−1)/2 ways to choose the entries above the diagonal
• There are 2n 3n(n−1)/2 antisymmetric relations on an n element set
118
Counting Functions
• Consider a matrix for function on a 2 element set
x
a
c
x
y
y
b
d
• For each row, there is a 1 in one and only one column
• There are 21 = 2 columns where a 1 can be placed
• The remaining column must contain a 0
• There are 22 = 4 functions from a 2 element set to a 2 element set
x
1
1
x
y
y
0
0
x
y
x
1
0
y
0
1
x
y
x
0
1
y
1
0
x
y
x
0
0
y
1
1
Counting Functions
• Consider a matrix for function from a 2 element set to a 3 element set
x
y
x
a
c
y
b
d
z
e
f
• For each row, there is a 1 in one and only one column
• There are 31 = 3 columns where a 1 can be placed
• The remaining column must contain a 0
Counting Functions
• There are 32 = 9 functions from a 2 element set to a 3 element set
x
y
x
1
1
y
0
0
z
0
0
x
y
x
1
0
y
0
1
z
0
0
x
y
x
1
0
y
0
0
z
0
1
x
y
x
0
1
y
1
0
z
0
0
x
y
x
0
0
y
1
1
z
0
0
x
y
x
0
0
y
1
0
z
0
1
x
y
x
0
1
y
0
0
z
1
0
x
y
x
0
0
y
0
1
z
1
0
x
y
x
0
0
y
0
0
z
1
1
Counting Functions
• Consider a matrix for function from a n element set to an m element set
• For each row, there is a 1 in one and only one column
• There are m1 = 3 columns where a 1 can be placed
• The remaining column must contain a 0
• There are mn functions from an n element set to an m element set
119
Counting One-to-One Functions
• Consider a matrix for one-to-one function on a 2 element set
x y
x a b
y c d
• For each row, there is a 1 in one and only one column
• For each column, there is a 1 in at most one row
• For the first row, there are 21 columns where a 1 can be placed
• For the second row, there is only one column where a 1 can be placed
• There are 2 · 1 = 2 one-to-one functions on a 2 element set
x
y
x
1
0
y
0
1
x
y
x
0
1
y
1
0
Counting One-to-One Functions
• Consider a matrix for one-to-one function from a 2 element set to a 3 element set
x y z
x a b e
y c d f
• For each row, there is a 1 in one and only one column
• Once a column contains a 1, it can have no other 1’s
Counting One-to-One Functions
• There are 3 · 2 = 6 one-to-one functions from a 2 element set to a 3 element set
x
y
x
1
0
y
0
1
z
0
0
x
y
x
0
0
y
1
0
z
0
1
x
y
x
1
0
y
0
0
z
0
1
x
y
x
0
1
y
0
0
z
1
0
x
y
x
0
1
y
1
0
z
0
0
x
y
x
0
0
y
0
1
z
1
0
Counting One-to-One Functions
• Consider a matrix for function from a n element set to an m element set
• For each row, there is a 1 in one and only one column
• Once a 1 has been placed in a column, that column cannot have another 1 placed in it
• For the first row, there are m columns where a 1 can be placed
• For the second row, there are m − 1 columns where a 1 can be placed
• For the kth row, there are m − k + 1 columns where a 1 can be placed
• There are
m(m − 1)(m − 2) · · · (m − n + 1) =
one-to-one functions from an n element set to an m element set
120
m!
m
= n!
(m − n)!
n
Counting Onto Functions
• This is harder
26
Equivalence Relations
Epigraph
Numerorum congruentiam hoc signo, ≡, in posterum denotabimus, modulum ubi opus erit in
clausulis adiungentes, −16 ≡ 9 (mod 5), −7 ≡ 15 (mod 11).
Carl Friedrich Gauss, German mathematician (1777 – 1855)
The sign ≡ shows congruence between numbers for a given modulus, as illustrated by the
examples −16 ≡ 9 (mod 5) and −7 ≡ 15 (mod 11).
My un-educated translation
Partitions
• To partition: to separate into parts
• A set can be separated into disjoint parts
• Let X = {0} be a 1 element set
– There is 1 partition of X
{0}
• Let B = {0, 1} be a 2 element set
– There is 2 partition of B
{0, 1}
{0}, and {1}
or
Partitions
• Let T = {0, 1, 2} by a 3 element set
• There are 5 partition of T
{0, 1, 2}
{0, 1},
{2}
{0, 2},
{1}
{1, 2},
{0}
or
{0},
{1},
121
{2}
Partitions
• A partition of X is a set of disjoint subsets whose union is all of X
• ForT = {0, 1, 2}
1. {{0, 1, 2}} is a partition
2. {{0, 1}, {2}} is a partition
3. {{0, 2}, {1}} is a partition
4. {{1, 2}, {0}} is a partition
5. {{0}, {1}, {2}} is a partition
Partitions
• A partition defines a relation on the underlying set
• For example, consider the set F = {0, 1, 2, 3}
– The partition
{0, 2},
{1, 3}
defines a relation we might class “even-odd”
– We’d say 0 and 2 are equivalent because they are both even
– We’d say 1 and 3 are equivalent because they are both odd
Partitions
• The partition
{0, 3},
{1},
{2}
on {0, 1, 2, 3} defines a relation we might call “remainder on division by 3”
• We’d say 0 and 3 are equivalent because they are both divisible by 3
• We’d say 1 is equivalent only to itself: It has a remainder of 1 when divided by 3
• And, we’d say 2 is equivalent only to itself: It has a remainder of 2 when divided by 3
Last Name’s the Same Equivalence
• Say two people are equivalent if they share the same last name
– George W. Bush is equivalent to Jay Bush of bean fame, who is equivalent to Sam Bush, the famous
mandolin player
• Note that “last name’s the same” is reflexive: Every person has their own last name
• “last name” is symmetric: If X has the same last name as Y , then Y has X’s last name
• “last name” is transitive: If X has Y ’s last name and Y has Z’s last name, then X has Z’s last name
• People can be partitioned into disjoint sets based on their last name
122
Projective Space
• Consider the set of all points in the “punctured plane:” The Cartesian plane excluding the origin
R × R − {(0, 0)}
• Say two points (x0 , y0 ) and (x1 , y1 ) in this set are equivalent if they lie on the same straight line passing through
the origin
y
(x0 , y0 )
x
(x1 , y1 )
Projective Space
• Projective space defines an equivalence relation on the punctured plane
– Reflexive:
(x, y) lies on the same line through the origin as itself
– Symmetry:
If (x0 , y0 ) and (x1 , y1 ) lie on a line through the origin, then (x1 , y1 ) and (x0 , y0 ) lie on a line through the
origin
– Transitive:
If (x0 , y0 ) and (x1 , y1 ) lie on a line through the origin, and if (x1 , y1 ) and (x2 , y2 ) lie a line through the
origin, then (x0 , y0 ) and (x2 , y2 ) lie on a line through the origin
Projective Space
• Projective space partitions the points in the punctured plane into disjoint subsets defined by the lines through
the origin
• Every point (x, y) on the unit circle (x2 + y2 = 1) defines an equivalence class (a unique line through the origin)
for this relation
• That is, the equivalence class [(x, y)] of a point in the plane is the set
[(x, y)] = {(a, b) : a = cx, b = cy for some constant c 6= 0}
• Understanding projective space is essential to understanding three dimensional views in computer graphics
Kernel of a Function
• Let
f : X −→ Y
be a function
• Define an relation on X by
x0 Kx1
if and only if f (x0 ) = f (x1 )
• The relation K, called the “kernel of f ”
• The relation K is an equivalence relation
123
Kernel of a Function
• The kernel of f is reflexive
(∀x)(x ∈ X =⇒ xKx) since f (x) = f (x)
• The kernel of f is symmetric
(∀x)(∀y)(xKy =⇒ yKx)
since
f (x) = f (y) =⇒ f (y) = f (x)
• The kernel of f is transitive
(∀x)(∀y)(∀z)((xKy ∧ yKz) =⇒ xKz)
since
( f (x) = f (y) and f (y) = f (z)) =⇒ f (x) = f (z)
The Equal Sum Equivalence
• Let D = {0, 1, . . . , 9} be the set of digits
• Let 2D be the power set of D
• That is 2D is the set of all subsets
2D = {0,
/ {0}, {1}, . . . , {9}, {0, 1}, . . . , D}
• Define two subsets of D to be equivalent if their elements sum to the same value
• For example
{1} ≡ {0, 1}
{1, 2, 3} ≡ {1, 5}
{8} ≡ {0, 8} ≡ {1, 7} ≡ {1, 3, 4}
Equivalence Classes
• An equivalence relation partitions a set into disjoint equivalence classes
• The partition
{0, 2},
{1, 3}
of the set F = {0, 1, 2, 3} has 2 equivalence classes
– We can name these 2 classes
[0]
and
[1]
{1},
{2}
after representative elements from the partition
• The partition
{0, 3},
on F has 3 equivalence classes
– We can name these 3 classes
[0]
[1]
124
and
[2]
People Equivalence Classes
• The “last name’s the same” defines many equivalence classes: One for each possible last name
[Bush], [Smith], [Jones], [Carter],. . .
• We can define other equivalence classes on people, for example
[Male], [Female], [Hermaphrodite]
Projective Space Equivalence Classes
• Each point (x, y) on the unit circle x2 + y2 = 1 defines an equivalence class for projective space
• The equivalence classes can be denoted by
[(x, y)]
• The equivalence class [(x, y)] represents the set of point
{(ax, ay) : a ∈ R ∧ a 6= 0}
• For example, the equivalence class [(1, 0)] is the punctured x-axis
Kernel of a Function Equivalence Classes
• The equivalence classes for the kernel of f can be denoted by [x], where
[x] = {x0 : f (x0 ) = f (x)}
• [x] is called the inverse image of f (x)
• For example, for the trigonometric function sin x the equivalence class
[x] = {x + 2nπ : n ∈ Z} ∪ {−x + (2n + 1)π : n ∈ Z}
The Equality Equivalence Relation
• The most simple equivalence relation is equality
• Consider equality on the set of rational numbers
• Represent a rational number by an ordered pair of integers
(a, b),
where a, b ∈ Z and b 6= 0
• Then, for example,
(1, 2) ≡ (2, 4) ≡ (3, 6) ≡ (−4, −8) ≡ · · ·
since each ordered pair represents the same rational number 1/2
• Equality is reflexive: (x, y) ≡ (x, y)
• Equality is symmetric: if (x, y) ≡ (u, v), then (u, v) ≡ (x, y)
• Equality is transitive: if (x, y) ≡ (u, v) and (u, v) ≡ (s, t), then (x, y) ≡ (s, t)
125
The Equality Equivalence Relation
• Wait a minute, isn’t
where a, b ∈ Z and b 6= 0
(a, b),
exactly the same as the projective space example?
• Well, almost
– Projective space points are pairs of real numbers
– Rationals are pairs of integers
• But otherwise the mathematics is the same
Matrix Representation of Equivalence Relations
• Equivalence relations are reflexive
– Its matrix has 1’s along the main diagonal
• Equivalence relations are symmetric
– If its matrix has a 1 in row i column j, then it has a 1 in row j column i
• Equivalence relations are transitive
– If its matrix has a 1 in row i column j and row j column k, then it has a 1 in row i column k
Matrix Representation of Equivalence Relations
• On D = {0, 1, . . . , 9}, the partition
{0, 3, 6, 9},
{1, 4, 7},
{2, 5, 8}
defines an equivalence relation with matrix
0
1
2
3
4
5
6
7
8
9
0
1
0
0
1
0
0
1
0
0
1
1
0
1
0
0
1
0
0
1
0
0
2
0
0
1
0
0
1
0
0
1
0
3
1
0
0
1
0
0
1
0
0
1
4
0
1
0
0
1
0
0
1
0
0
5
0
0
1
0
0
1
0
0
1
0
6
1
0
0
1
0
0
1
0
0
1
7
0
1
0
0
1
0
0
1
0
0
8
0
0
1
0
0
1
0
0
1
0
9
1
0
0
1
0
0
1
0
0
1
Graph Representation of Equivalence Relations
• Equivalence relations are reflexive
– Its graph has self-loops at each node (an edge from a node back to itself)
• Equivalence relations are symmetric
– Its graph is undirected (edges can be traversed in both directions)
• Equivalence relations are transitive
– If its graph has an edge from node n to node m and an edge from node m to node p, then it has an edge
from node n to node p
126
Graph Representation of Equivalence Relations
• On D = {0, 1, . . . , 9}, the partition
{0, 3, 6, 9},
{1, 4, 7},
{2, 5, 8}
defines an equivalence relation with graph
6
9
3
0
7
8
4
5
1
2
Congruence Modulo 2
• Congruence modulo 2 partitions the integers into “evens” and “odds”
• There are 2 equivalence classes modulo 2
• These equivalence classes are denoted
[0]
and
[1]
• The equivalence class [0] is the set of even integers
[0] = {. . . , −6, −4, −2, 0, 2, 4, 6, . . .}
= {0, ±2, ±4, ±6, . . .}
= {2n : n ∈ Z}
• If a and b are both even we write
a≡b
(mod 2)
Congruence Modulo 2
• The equivalence class [1] is the set of odd integers
[1] = {. . . , −5, −3, −1, 1, 3, 5, 7, . . .}
= {1, 1 ± 2, 1 ± 4, 1 ± 6, . . .}
= {2n + 1 : n ∈ Z}
• If a and b are both odd we write
a≡b
(mod 2)
• If one of a and b is even and the other is odd we write
a 6≡ b
127
(mod 2)
Congruence Modulo 2
• One useful way to “see” the even and odd numbers is to list them in rows, two columns to a row
..
.
−4
−2
0
2
4
6
8
10
12
..
.
..
.
−2
−1
1
3
5
7
9
11
13
..
.
Congruence Modulo 2
• The concept that captures the evenness and oddness is
– The difference of any two even integers is even, and
– The difference of any two odd integers is even
• This relationship among integers called congruence modulo 2
• To denote that n is even write
n≡0
(mod 2)
n≡1
(mod 2)
• To denote that n is odd write
Congruence Modulo 2
• In general,
a≡b
(mod 2) if and only if (∃k)(k ∈ Z ∧ a − b = 2k)
• For example
–
–
–
–
−8 ≡ 10 (mod 2) since −8 − 10 = 2(−9)
11 ≡ 7 (mod 2) since 11 − 7 = 2(2)
14 ≡ −32 (mod 2) since 14 − (−32) = 46 = 2(23)
41 ≡ −23 (mod 2) since 41 − (−23) = 64 = 2(32)
• And 5 6≡ 6 (mod 2) since 5 is odd and 6 is even
Congruence Modulo 3
• There is nothing magical about the evens and odds, we could have just as easily used 3 to partitioned the integers
into 3 classes
– The multiples of 3 (0 remainder when divided by 3)
[0] = {0, ±3, ±6, ±9, . . .} = {3n : n ∈ Z}
– The multiples of 3 plus 1 (1 remainder when divided by 3)
[1] = {1, 1 ± 3, 1 ± 6, 1 ± 9, . . .} = {3n + 1 : n ∈ Z}
– The multiples of 3 plus 2 (2 remainder when divided by 3)
[2] = {2, 2 ± 3, 2 ± 6, 2 ± 9, . . .} = {3n + 2 : n ∈ Z}
128
Congruence Modulo 3
• One useful way to “see” the integers with remainders 0, 1, and 2 is to list them in rows, three columns to a row
..
..
..
.
.
.
−6 −5 −4
−3 −2 −1
0
1
2
3
4
5
6
7
8
9 10 11
12 13 14
..
..
..
.
.
.
Congruence Modulo 3
• The concept that captures the modulo 3 is
– The difference of two equivalent integers mod 3 is a multiple of 3
• To denote that n is a multiple of 3 write
n≡0
(mod 3)
n≡1
(mod 3)
n≡2
(mod 3)
• To denote that n is a multiple of 3 plus 1 write
• To denote that n is a multiple of 3 plus 2 write
Congruence Modulo 3
• In general,
a≡b
(mod 3) if and only if (∃k)(k ∈ Z ∧ a − b = 3k)
• For example
– −8 ≡ 10 (mod 3) since −8 − 10 = 3(−6)
– 11 ≡ 8 (mod 3) since 11 − 8 = 3(1)
– 14 ≡ −31 (mod 3) since 14 − (−31) = 45 = 3(15)
– 41 ≡ −22 (mod 3) since 41 − (−22) = 63 = 3(21)
• And 5 6≡ 6 (mod 3) since 5 − 6 6= 3k for any integer k
Congruence Modulo m
• The idea of partitioning integers into equivalence classes based on the divisors of their differences is called
congruence
• The general definition is
Definition 13 (Congruence modulo m). a is congruent to b modulo m if a − b is a multiple of m
a ≡ b (mod m) if and only if a − b = mk for some k ∈ Z
129
Congruence Modulo 4
• Writing the integers in rows of so many columns illustrates the partitioning of the integers
..
..
..
.
.
.
−4 −3 −2
0
1
2
4
5
6
8
9 10
12 13 14
16 17 18
20 21 22
..
..
..
.
.
.
..
.
−1
3
7
11
15
19
23
..
.
Congruence Modulo 5
• Here is the partitioning induced by (mod 5) partitioning of the integers
..
..
..
..
..
.
.
.
.
.
−5 −4 −3 −2 −1
0
1
2
3
4
5
6
7
8
9
10 11 12 13 14
15 16 17 18 19
20 21 22 23 24
25 26 27 28 29
..
..
..
..
..
.
.
.
.
.
Congruence Modulo m
• Congruence modulo m is an equivalence relation on Z
• If a − b = mk for some k ∈ Z, write
a≡b
(mod m)
and say a is equivalent to b mod m
• if a − b is a multiple of m
• if m divides a − b
• if a and b have the same remainder on division by m
Congruence Modulo m
• Congruence modulo m is reflexive
(∀a)(a ∈ Z =⇒ a ≡ a
(mod m))
– This is true since m divides 0 = a − a for all a ∈ Z
• Congruence modulo m is symmetric
(∀a)(∀b)(a ≡ b (mod m) =⇒ b ≡ a
– This is true since if m divides a − b, then m divides b − a
130
(mod m))
Congruence Modulo m
• Congruence modulo m is transitive
(∀a)(∀b)(∀c)((a ≡ b
(mod m) ∧ b ≡ c
(mod m))
=⇒ a ≡ c (mod m))
– This is true since if m divides a − b and m divides b − c, then m divides a − c = (a − b) + (b − c)
Congruence Modulo m
• Congruence modulo m partitions the integers into a collection m equivalence or residue classes
• The equivalence class [a] is the set of all integers that have the same remainder as a does when divided by m
[a] = {a, a ± m, a ± 2m, a ± 3m, . . .}
= {b : b = a + mk for some k ∈ Z}
The Integers Congruence Modulo m
• The set of integers modulo m is denoted
Zm = {0, 1, 2, . . . , m − 1}
where m ∈ Z+
• The set of integers modulo m is the set of remainders when an integer is divided by m
• For example,
Z2 = {0, 1}
Z3 = {0, 1, 2}
Z5 = {0, 1, 2, 3, 4}
• In the modular numbers section of the notes we’ll define an arithmetic on these remainders
Stirling Numbers of the Second Kind
Stirling Numbers of the Second Kind
• An interesting question is
– On n elements, how many partitions are there with k subsets?
• Stirling numbers of the second kind count these numbers
• There are other applications of Stirling numbers of the second kind
• A Stirling number of the second kind is denoted
n
k
• The Stirling number
n
• The Stirling number
n
k
k
counts the number of partitions of an n element set into k subsets
is read
“n subset k”
131
Partitions
• Recall two sets X0 , and X1 are disjoint if their intersection is empty
X0 ∩ X1 = 0/
• Recall a partition of a set X is a collection of disjoint subsets X0 , X1 , . . . , Xn−1 , such that
X=
n−1
[
Xk
k=0
• For example,
X0 = {0, 2},
X1 = {1, 3}
is a partition of
X = {0, 1, 2, 3}
Partitions
• On the set X = {0, 1, 2, 3} there is 1 partition with 1 subset
4
{0, 1, 2, 3}
=1
1
• There are 7 partitions with 2 subsets
{0, 1, 2}, {3}
and
{0, 1, 3}, {2}
{0, 2, 3}, {1}
and
{1, 2, 3}, {0}
{0, 1}, {2, 3}
and
{0, 2}, {1, 3}
{1, 2}, {0, 3}
4
=7
2
Partitions
• On the set X = {0, 1, 2, 3} there is 6 partitions with 3 subsets
{0, 1}, {2}, {3}
and
{0, 2}, {1}, {3}
{1, 2}, {0}, {3}
and
{0, 3}, {1}, {2}
{0}, {1, 3}, {2}
and
{0}, {1}, {2, 3}
4
=6
3
• There is 1 partition with 4 subsets
{0}, {1}, {2}, {3},
4
=1
4
Stirling Numbers of the Second Kind
• There is a triangle of Stirling numbers of the second kind
• Successive rows are computed from previous rows by a recursive relation that is similar to Pascal’s identity
• Before deriving the identity, let’s compute a few rows directly
132
Stirling Numbers of the Second Kind
• On the empty set there is one partition called the empty partition
• Therefore,
0
=1
0
Stirling Numbers of the Second Kind
• On a singleton {a}
– There are no partitions with no subsets, so we say “1 subset 0 is 0” and write
1
=0
0
– There is 1 partition with 1 subset: {a}, so we say “1 subset 1 is 1” and write
1
=1
1
• The first two rows of Stirling’s triangle of the second kind are
0
1
0
0
1
1
1
Stirling Numbers of the Second Kind
• On a two element set {a, b}
– There are no partitions with no subsets: 2 subset 0 is 0 and
2
=0
0
– There is 1 partition with 1 subset
{a, b}
– We say 2 subset 1 is 1 and write
2
=1
1
– There is 1 partition with 2 subsets
{a}, {b}
– We say 2 subset 2 is 1 and write
2
=1
2
Stirling Numbers of the Second Kind
• The first three rows of Stirling’s triangle of the second kind are
0
1
2
0
1
0
0
133
1
2
1
1
1
Stirling Numbers of the Second Kind
• On a three element set {a, b, c}
– There are no partitions with no subsets:
3
0
=0
– There is 1 partition with 1 subset
{a, b, c}
– 3 subset 1 is 1 and
3
=1
1
– There are 3 partitions with 2 subset
{a, b}, {c}
{a, c}, {b}
{a}, {b, c}
and
– We say 3 subset 2 is 3 and write
3
=3
2
– There is 1 partition with 3 subsets
{a}, {b}, {c}
3
=1
3
and
Stirling Numbers of the Second Kind
• On a four element set {a, b, c, d} we find
• 40 = 0
• There is one 1-subset partition
{a, b, c, d}
• There are seven 2-subset partitions are
{a, b, c}, {d}
{a, b, d}, {c}
and
{a, b}, {c, d}
{a, c, d}, {b}
and
{a, c}, {b, d}
{a, d}, {b, c}
and
{a}, {b, c, d}
Stirling Numbers of the Second Kind
• These seven 2-subset partitions can be constructed from the 1-subset and 2-subset partitions on {a, b, c}
– Include the subset {d} to the 1-subset partition
{a, b, c} −→ {a, b, c}, {d}
– Insert d into each of the subsets from the 2-subset partitions
{a, b}, {c} −→ {a, b, d}, {c} and
{a, b}, {c, d}
{a, c}, {b} −→ {a, c, d}, {b}
and {a, c}, {b, d}
{a}, {b, c} −→ {a, d}, {b, c}
and {a}, {b, c, d}
134
Stirling Numbers of the Second Kind
• There are six 3-subset partitions are
{a, b}, {c}, {d}
{a, c}, {b}, {d}
{a, d}, {b}, {c}
{a}, {b, c}, {d}
{a}, {b, d}, {c}
{a}, {b}, {c, d}
• There is one 4-subset partition is {a}, {b}, {c}, {d}
• Therefore
4
= 0,
0
4
= 1,
1
4
= 7,
2
4
= 6,
3
4
=1
4
Triangle of Stirling Numbers of the Second Kind
• The first rows of Stirling’s triangle of the second kind are
0
1
2
3
4
5
6
0
1
0
0
0
0
0
0
1
2
3
4
5
6
1
1
1
1
1
1
1
3
7
15
31
1
6
25
90
1
10
65
1
15
1
• Like binomial coefficients, Stirling numbers of the second kind form two dimensional array
• Do you notice the recursion that generates rows of Stirling numbers of the second kind?
Recursion for Stirling Numbers of the Second Kind
• Stirling’s identity of the second kind is
n
n−1
n−1
=
+k
k
k−1
k
• The boundary conditions on the identity are
n
= 0, for n > 0 and
0
n
=1
n
Numerical Examples for Stirling Numbers
• Here’s a portion of the triangle
4
5
6
0
0
0
0
1
1
1
1
2
7
15
31
• Let’s compute some examples
– 51 = 40 + 1 41 = 0 + 1 · 1 = 1
– 62 = 51 + 2 52 = 1 + 2 · 15 = 31
– 64 = 53 + 4 54 = 25 + 4 · 10 = 65
– 65 = 54 + 5 55 = 10 + 5 · 1 = 15
– 73 = 62 + 3 63 = 31 + 3 · 90 = 301
– 74 = 63 + 4 64 = 90 + 4 · 65 = 350
135
3
6
25
90
4
1
10
65
5
6
1
15
1
Why Stirling’s Identity is Correct
• Here’s an example showing why the identity is correct
• Let {a, b, c, d, e, f , g, h} be an 8 element set
• To count the number of partitions with 5 subsets
– Consider the partitions with 4 subsets on the 7 element set {a, b, c, d, e, f , g}
– For example, here is one of them
{a, b}, {c, d}, {e, f }, {g}
– For this partition a 5-subset partition can be constructed by appending the subset {h}
{a, b}, {c, d}, {e, f }, {g}, {h}
– There are
7
4
five-subset partitions that can be constructed in the manner
Why Stirling’s Identity is Correct
• Consider the partitions with 5 subsets on the 7 element set {a, b, c, d, e, f , g}
• There are 75 five-subset partitions on a 7 element set
• For example, here is one of them
{a, b}, {c}, {d}, {e, f }, {g}
• For this partition the letter h can be inserted into each of the 5 subsets generating a partition on 8 elements
{a, b, h}, {c}, {d}, {e, f }, {g}
{a, b}, {c, h}, {d}, {e, f }, {g}
{a, b}, {c}, {d, h}, {e, f }, {g}
{a b}, {c}, {d}, {e, f , h}, {g}
{a, b}, {c}, {d}, {e, f }, {g, h}
Why Stirling’s Identity is Correct
• For each of the 75 five-subset partitions on 7 objects we can generate 5 five subset partitions on 8 objects
• This is accomplished by inserting the eighth object into one of 5 subsets
• Therefore, there are 5 75 five-subset partitions that can be constructed in the manner
Why Stirling’s Identity is Correct
• Five subset partitions on 8 objects can be created in
7
4 from 4 subset partitions on 7 objects
7
• Five subset partitions on 8 objects can be created in 5 5 from 5 subset partitions on 7 objects
• Adding these counts: 74 and 5 75 gives
8
7
7
=
+5
5
4
5
ways to construct 5 subset partitions on 8 objects
136
Stirling’s Triangle of the Second Kind
0
1
2
3
4
5
6
7
0
1
0
0
0
0
0
0
0
1
2
1
1
1
1
1
1
1
1
3
7
15
31
63
3
4
5
6
7
1
6
1
25 10
1
90 65 15 1
301 350 140 21
1
Facts in Stirling’s Triangle of the Second Kind
• That except for the first entry, the zero column is all zeros
0
n
= 1, but
= 0,
0
0
for n > 0
• The 1 column is always 1
n
=1
1
• The 2 column holds Mersenne numbers
n
= 2n−1 − 1
2
• The diagonal is always has entry 1
n
=1
n
Facts in Stirling’s Triangle of the Second Kind
• The sub-diagonal holds the triangular numbers
n
n(n − 1)
n
=
= tn =
n−1
2
2
• The sum of values in row n is the number of partitions of (or equivalence relations on) an n element set
27
Functions
Quotation
Mind like parachute — only function when open.
Charlie Chan, fictional Chinese-American detective
Function: Definition
Definition 14 (Function). A function f from X to Y is a relation such that for each x ∈ X there is a unique y ∈ Y such
that
f (x) = y
Definition 15 (Function). A function f from X to Y is a set of ordered pairs
f = {(x, y) : x ∈ X, y ∈ Y}
such that if (x, y0 ) ∈ f and (x, y1 ) ∈ f then y0 = y1
137
Function Notation and Terminology
• If f is a function and f (x) = y, we say
– f maps x to y, or
– the value of f at x is y, or
– y is the image of x
• If f is a function from X to Y we write
f : X −→ Y
• The set X is called the domain of f
• The set Y is called the co-domain of f
• The set f (X) = { f (x) : x ∈ X} is called the range or image of f
The Domain
• The domain of f is the set X
• Elements in the domain are input values for the function f
• The domain has a type, for example,
– The square root function
f (x) =
√
x
has input type Double
– The AND function
f (P, Q) = P ∧ Q
has input type Boolean × Boolean
– The mod function f (n, m) = n mod m has input type Integer × Integer
The Domain
• A function f is said to be total when it is defined for all values of x in its domain
• That is, f is a total function when
(∀x ∈ X)(∃y ∈ Y)((x, y) ∈ f )
or, written differently
(∀x ∈ X)(∃y ∈ Y)( f (x) = y)
• When f is not defined for all elements in its domain, it is called partial
• That is, f is a partial function when
(∃x ∈ X)(∀y ∈ Y)((x, y) 6∈ f )
138
Co-Domain and Range
• The co-domain of f is the set Y
• Elements in the co-domain are output values for the function f
• The co-domain has a type, for example,
– The square root function
f (x) =
√
x
has output type Double
– The AND function
f (P, Q) = P ∧ Q
has output type Boolean
– The mod function f (n, m) = n mod m has output type Integer
Co-Domain and Range
• The range of f is the subset of Y that f maps onto
• That is, the range of f is the set
{ f (x) : x ∈ X}}
Example Total Functions
• The following are total functions from the real numbers to the real numbers
Example Partial Functions
• The following are partial functions from the real numbers to the real numbers
Function: Onto and One-To-One
• A function f is said to be onto when every element in its range is the image of some element in its domain
• That is, f is an onto function when
(∀y ∈ Y)(∃x ∈ X)((x, y) ∈ f )
or, written differently
(∀y ∈ Y)(∃x ∈ X)( f (x) = y)
• A function is always onto its image
• A function f is one-to-one when no different element in its domain map to the same element in its range
• That is, f is an one-to-one function when
(∀x0 ∈ X)(∀x1 ∈ X)((x0 6= x1 ) =⇒ ( f (x0 ) 6= f (x1 )))
or, written in the contrapositive
(∀x0 ∈ X)(∀x1 ∈ X)(( f (x0 ) = f (x1 )) =⇒ (x0 = x1 ))
Example Onto Functions
• The following are functions from the real numbers onto the real numbers
139
Example Functions that Are Not Onto
• The following are functions from the real numbers that are not onto to the real numbers
Example One-to-One Functions
• The following are one-to-one functions from the real numbers to the real numbers
Example Functions That Are Not One-to-One Functions
• The following are not one-to-one functions from the real numbers to the real numbers
The Inverse of a Function
• When a function f : X −→ Y is one-to-one it has an inverse
f −1 : Y −→ X
• For example, the function y = f (x) = 2x − 1 is one-to-one
– The inverse of f is
x = f −1 (y) =
y+1
2
• Another one-to-one function is y = f (x) = x3 which has inverse
– The inverse of f is
x = f −1 (y) =
√
3 y
• An important example is the exponential function y = 2x which has inverse x = lg y
The Inverse of a Function
• Pretend function f is one-to-one and is given by the set of ordered pairs
f = {(x, y) : x ∈ X, y ∈ Y}
• For each y there is at most one x such that f (x) = y
• Interchange the order x and y in f to get a new set
f −1 = {(y, x) : x ∈ X, y ∈ Y}
• Since f is one-to-one, the set f −1 satisfies the definition of a (partial) function
• If f is also onto, then f −1 is a total function
140
Common Functions: Polynomials
• Linear polynomials
y = mx + b
where
– m is the slope or rate of change
– b is the y-intercept
– x = −b/m is the zero, provided m 6= 0
• Quadratic polynomials
y = ax2 + bx + c
– 2ax + b is the derivative or rate of change
– c is the y-intercept
– There are two zeros
√
2c
−b ± b2 − 4ac
√
=
x± =
2a
−b ∓ b2 − 4ac
Common Functions: Polynomials
• General polynomials
y = p(x) = an−1 xn−1 + an−2 xn−2 + · · · a1 x + a0
• Example polynomials
– y = x3 − 1
– y = 2x5 − 7x3 + 3x2 − 5
– y = x4 + x3 + x2 + x + 1
– y = x6 − 3x4 + 9x3 − 17
• A polynomial of degree n − 1 has n − 1 zeros, some of which may be complex numbers
Common Functions: Transcendentals
• Logarithms
y = log x
y = ln x
y = lg x
y = ex
y = 2x
• Exponentials
y = 10x
• Trigonometric functions
y = cos x,
y = sin x,
y = tan x
y = cosh x,
y = sinh x,
y = tanh x
• Hyperbolic functions
Common Functions: Permutations
• A permutation is a one-to-one function from a set onto itself
• A permutation orders or re-orders a set of elements
• For example, there are 3! = 6 permutations on {a, b, c} are
(a, b, c)
(a, c, b)
(b, a, c)
(b, c, a)
(c, a, b)
(c, b, a)
• There are n! permutations of a set whose cardinality is n
141
The Set of All Functions
• The set of all functions from X to Y is denoted YX
YX = { f : f : X −→ Y}
• There are |Y||X| total functions from X to Y
28
Polynomials
Polynomials
• Low degree polynomials p(x)
• Their equations p(x) = 0,
• And their graphs p(x) = y
• Are mastered in what is sometimes called “college algebra”
Polynomials
• There are several polynomials we can name
– Constant polynomials
– The golden polynomial g(x)
– The complex polynomial C (x)
• Polynomials can be used to model quite complex curves
Constant Polynomials
• ζ = 0, Zero
• γ ≈ 0.5772, The Euler-Mascheroni constant gamma
• ω = 1, One
• φ ≈ 1.618, The golden mean phi
• e ≈ 2.718, Euler’s or Napier’s constant
• π ≈ 3.14, Pi
The Golden Polynomial
• The golden polynomial
• Named because one of its roots, where g(x) = 0, is the golden mean
√
1+ 5
≈ 1.6180
φ=
2
• The other root is the negative reciprocal of φ
√
1− 5
φ̂ =
≈ −0.6180
2
• The golden polynomial is related to the Fibonacci recurrence
Fn = Fn−1 + Fn−2 ,
142
F0 = 0, F1 = 1
The Complex Polynomial
• The complex polynomial
C (x) = x2 + 1
• To have roots complex numbers ±i are introduced
• Complex numbers help describe electricity and magnetism, among other things
Classes of Polynomials
• There are also classes of polynomials that arise in practice
• The standard basis
• The geometric sums
• The difference of powers
• The falling factorials
The Standard Basis Polynomials
• The standard basis functions
βn (x) = xn
• Their recurrence relation
βn (x) = xβn−1 (x)
• Useful fact: Their first derivative is
d(xn )/dx = nxn−1
Geometric Sum Polynomials
• The geometric sum polynomials
n−1
γn (x) = xn−1 + xn−2 + · · · + x + 1 =
∑ xk
k=0
• Their recurrence relation
γn (x) = xn−1 + γn−1 (x)x
• Useful fact: At x = 2, their value is the Mersenne number mn
n−1
2n − 1 = γn (2) = 2n−1 + 2n−2 + · · · + 2 + 1 =
∑ 2k
k=0
Difference of Powers Polynomials
• The difference of powers polynomials
δn (x) = xn − 1
• Their recurrence relation
δn (x) = δn−1 (x) + xn (x − 1)
143
Geometric Sums and Differences of Powers
• There is an important functional relationship between geometric sum and difference of powers polynomials
•
γn (x)(x − 1) = δn+1 (x)
• Or, as it is often written, in rational form
n
∑ xk = γn (x) =
k=0
δn+1 (x) xn+1 − 1
=
x−1
x−1
Falling Factorial Polynomials
• The falling factorial polynomials
xn = x(x − 1) · · · (x − n + 1)
• Their recurrence relation
xn = (x − n + 1)xn−1
• Useful facts
– At x = n,
nn = n!
– Their forward difference is analogous to differentiation
4xn = (x + 1)n − xn = nxn−1
– They form an appropriate basis for the sum and difference calculus
28.1
Horner’s Method for Polynomial Evaluations
Quotation
Conjunction junction, what’s your function?
Bob Dorough, “Schoolhouse Rock” American singer, song writer, citizen of the world (1923 –)
Evaluating Polynomials: Horner’s Method
• Horner’s method is an efficient algorithm for evaluating a polynomial p(x) at a given value of x
• Here’s an example
– Evaluate y = 5x − 3 at x = 4
5
5
−3
20
17
– Horner’s method yields y = 17 at x = 4
Evaluating Polynomials: Horner’s Method
• Evaluate y = 3x2 − 4x + 7 at x = −2
3
3
−4
−6
− 10
• Horner’s method yields y = 27 at x = −2
144
7
20
27
Analyzing Horner’s Method
• Horner’s method can be interpreted using nested parenthesis
• For example, to evaluate 3x2 − 4x + 7 at a given x, write
((3 · x) − 4) · x + 7
• More generally, to evaluate
an−1 xn−1 + an−2 xn−2 + · · · + a1 x + a0
write
(· · · (((an−1 · x) + an−2 ) · x) + · · · + a1 ) · x + a0
Programming Horner’s Method
• Horner’s method can be implemented in a programming language
• Here’s pseudocode for Horner’s method to evaluate p(x) = an−1 xn−1 + · · · + a0
[1] Hornerdouble a[ ], int n, double x double p ← a[n − 1] Initialize the return value (inti ← n − 2; i >= 0; i − −)
p ← p · x + a[i] return p The value of p(x)
Time Complexity Analysis of Horner’s Method
• The time complexity of Horner’s method is big-O of n where n − 1 is the degree of the polynomial p(x)
• The notation for big-O of n is written O(n)
• The for loop in line ?? of Horner’s method is executed n − 1 times
• Within each execution 1 multiply and 1 addition are performed in line ??
• Therefore Horner’s method executes n − 1 multiplications and n − 1 additions
Evaluating Polynomials: Horner’s Method
• Here is another example polynomial to evaluate y = 3x5 − 2x3 + x − 4 at x = 2
3
3
0
6
6
−2 0 1
12 20 40
10 20 41
−4
82
78
• Horner’s method yields y = 78 at x = 2
• See the use of 0 as a place-holder for missing powers of x
29
The Floor and Related Functions
The Floor Function
• The floor or greatest integer integer function maps the real numbers R onto the integers Z
• The floor function is useful in describing how real numbers are rounded or truncated
• The floor truncates a real number discarding its fractional part
Definition 16 (Floor function). The floor of x is the greatest integer less than or equal to x
bxc = max{n : n ∈ Z ∧ n ≤ x}
Evaluating the Floor Function
• Examples of the floor function
145
–
–
–
–
–
–
–
– b−πc = −4
b2.75c = 2
b−2.5c = −3
b18.5c = 18
b−12.6c = −13
b5c = 5
b−5c = −5
bπc = 3
– bec = 2
– b−ec = −3
j √ k
– 1+2 5 = 1
–
j
√ k
− 1+2 5 = −2
Graph of the Floor Function
• The floor is a step function whose graph is shown below
The Ceiling Function
• The ceiling or least integer integer function maps the real numbers R onto the integers Z
• The ceiling rounds up a real number padding its fractional part
Definition 17 (Ceiling function). The ceiling of x is the least integer greater than or equal to x
dxe = min{n : n ∈ Z ∧ n ≥ x}
Evaluating the Ceiling Function
• Examples of the ceiling function
–
–
–
–
–
–
–
d2.75e = 3
d−2.5e = −2
d18.5e = 19
d−12.6e = −12
d5e = 5
d−5e = −5
dπe = 4
– d−πe = −3
– dee = 3
– d−ee = −2
l √ m
– 1+2 5 = 2
–
Graph of the Ceiling Function
• The ceiling is a step function whose graph is shown below
The Round Function
• The round of a real number x returns the integer nearest to x
Definition 18 (Round Function). The round of x is
e x = bx + 0.5c
146
l
√ m
− 1+2 5 = −1
The Round Function
• When x − bxc = 0.5 there both bxc and dxe are equal distance from x
• For example, when x = 1.5 we could round to 1 or 2
• By default, always round up
Evaluating the Round Function
• Examples of the round function
– e 2.75 = 3
– eπ = 3
– e −2.5 = −2
– e −π = −3
– e 18.5 = 19
– ee = 3
– e −12.6 = −13
– e −e = −3
– e5 = 5
– e 1+2
– e −5 = 5
–
√
5
=2
√
e − 1+2 5
= −2
Graph of the Round Part Function
• The round is a step functions whose graph is shown below
The Fractional Part Function
• The fractional part of a real number x is what remains when the floor of x is subtracted from x
Definition 19 (Fraction Part Function). The fractional part of x is
{x} = x − bxc
The Fractional Part Function
• Because the floor of x is always less than or equal to x, the fractional part is always greater than or equal to 0
{x} = x − bxc ≥ 0
• The notation {x} is somewhat unfortunate because it can be interpreted as
– The fractional part of x
– The set containing x
• But {x} is Knuth’s notation, so it must be correct!
Evaluating the Fractional Part Function
• Examples of the fractional part function
– {2.75} = 0.75
– {−2.5} = 0.5
– {18.5} = 0.5
– {−12.6} = 0.4
– {5} = 0.0
– {−5} = 0.0
147
–
–
–
–
{π} ≈ 0.1416
{−π} ≈ 0.8584
{e} ≈ 0.7183
{−e} ≈ 0.2817
–
–
n
√ o
1+ 5
≈ 0.6180
2
n
√ o
{− 1+2 5 ≈ 0.3820
Graph of the Fractional Part Function
• The fractional part is discontinuous and its graph is shown below
30
Logarithms and Exponentials
Logarithms and Exponentials
• Logarithms and exponentials are inverses of one another
• Logarithms and exponentials have a base b
• The usual bases are 2, e ≈ 2.718, and 10
• Base 2 is important in computer science
• Base e is important in mathematical analysis
• Base 10 is used to learn about logarithms because the arithmetic is simple
lg x and 2x
• The logarithm of x base 2 is written y = lg x
• The inverse of the log base 2 is the exponential function is y = 2x
• If y = lg x then x = 2y
• Here are some example computations
x
lg x
x
lg x
1
4
= 2−2
−2
√
2 = 21/2
1
2
1
2
= 2−1
1
√
4
2
−1
2 = 21
1
= 2−1/4
− 41
4 = 22
2
• The graphs of the log base 2 and 2 to the x are shown
y
• Do you see the symmetry about the line y = x?
148
√
3
2 = 21/3
0
1
3
64 = 26
6
lg x and 2x
f (x) = 2lgx x
1 = 20
x
256 = 28
8
2m
m
ln x and ex
ln x and ex
• The graphs of the natural log and e to the x are shown
y
f (x) = ex
f (x) = ln x
x
log x and 10x
• The logarithm of x base 10 is written log(x)
• If 10x = y, then x = log y
– 100 = 1,
0 = log 1
– 101 = 10,
– 102 = 100,
–
103
–
10−1
= 1000,
=
– 10−2 =
– 10−3 =
– 10−4 =
1
10 ,
1
100 ,
1
1000 ,
1
10000 ,
1 = log 10
2 = log 100
3 = log 1000
1
−1 = log 10
1
100
1
−3 = log 1000
1
−4 = log 10000
−2 = log
Facts About Logarithms
• An logarithm function grows slowly
• Therefore its inverse, an exponential, grows quickly
• For any base b 6= 0, b0 = 1
• The logarithm base b of 1 is 0
– 20 = 1, lg(1) = 0
– 100 = 1, log(1) = 0
– e0 = 1, ln(1) = 0
– b0 = 1, logb (1) = 0
• Since logarithms are inverses of exponentials
– 2lg(x) = x and lg 2x = x
– eln(x) = x and ln ex = x
– 10log(x) = x and log 10x = x
– blogb (x) = x and logb bx = x
149
Exponential Rules
• There are several rules for computing with exponentials
• The examples below are for base 2, but are valid for any base b > 0
• The zero exponent rule: 20 = 1
• The negating exponent rule:
1
= 2n
2−n
and
2−n =
1
2n
• The product rule: In products, exponents add
2n 2m = 2n+m
• The quotient rule: In quotients, exponents subtract
2n
= 2n−m
2m
• The exponential to an exponent rule: In exponentiating, exponents multiply
(2n )m = 2nm
Logarithm Rules
• The exponential rules have corresponding logarithm rules
• The log of 1 rule: lg 1 = 0
• The reciprocal rule:
lg (1/x) = − lg x
• The log of a product is the sum of logs
lg(xy) = lg x + lg y
• The log of a quotient is the difference of logs
lg(x/y) = lg x − lg y
• The log of a power is the power times the log
lg xy = y lg x
• The log cabin rule
Z
dcabin
= log cabin
cabin
150
Log of a Product is the Sum of Logs
• The log of a product is the sum of logs
logb xy = logb x + logb y
• To prove this, let z be a name for logb xy
z = logb xy
or
bz = xy
• Let w be the name of logb (x) so that
w = logb (x)
or
bw = x
u = logb (y)
or
bu = y
• Let u be the name of logb (y) so that
• Therefore
bz =xy =bw bu = bw+u
• And
z =w + u =logb xy = logb x + logb y
Log of a Product is the Sum of Logs
• Here are some examples of the product rule
– log 50 = log(5 · 5 · 2) = log 5 + log 5 + log 2 = 2 log 5 + log 2
– lg 36 = lg(2 · 2 · 3 · 3) = lg 2 + lg 2 + lg 3 + lg 3 = 2 + 2 lg 3
– ln 210 = ln(2 · 3 · 5 · 7) = ln 2 + ln 3 + ln 5 + ln 7
– lg 200 = lg(8 · 25) = 3 + 2 lg 5
Log of n!
• Finding an alternative expression for the log of n! is an important example fo the product rule for logarithms
• The log of n! is
lg n! = lg(n(n − 1)(n − 2) · · · 2 · 1)
• Using the log of a product is the sum of logs gives
lg(n(n − 1)(n − 2) · · · 2 · 1) = lg(n) + lg(n − 1) + lg(n − 2) + · · · + lg(2) + lg(1)
• Expressed in summation notation
n
lg n! =
∑ lg(k)
k=1
• The largest term in the sum on the right-hand side is lg(n)
• Replacing the smaller terms by lg(n) find
n
lg n! ≤
∑ lg(n) = n lg(n)
k=1
151
Log of a Quotient is the Difference of Logs
• The log of a quotient is the difference of logs
logb
• To prove this, let z be a name for logb
x
= logb x − logb y
y
x
y
z = logb
x
y
or
bz =
x
y
• Let w be the name of logb (x) so that
w = logb (x)
or
bw = x
u = logb (y)
or
bu = y
• Let u be the name of logb (y) so that
Log of a Quotient is the Difference of Logs
• Using bz = x/y, bw = x, and bu = y find that
x bw
bz = = u = bw−u
y b
• And thus
x
z =w − u =logb
= logb x − logb y
y
Log of a Quotient is the Difference of Logs
• Here are some examples of the product rule
1
10
– log
– lg
5
4
– ln
25
9
– lg
256
81
= log(1) − log(10) = 0 − 1 = −1
= lg(5) − lg(4) = lg(5) − 2
= ln 25 − ln 9 = 2 ln(5) − 2 ln 3
= lg(256) − lg(81) = 8 − 4 lg 3
152
Log of a Power is the Power Times the Log
• The log of a product is the product times the log
logb (xy ) = y logb x
• To prove this, let z be a name for logb (xy )
z = logb (xy )
or
bz = xy
w = logb (x)
or
bw = x
• Let w be the name of logb (x) so that
• Therefore
xy = (bw )y
or
byw = bz
• And thus
z = yw = logb (xy ) = y logb x
Log of a Power is the Power Times the Log
• Here are some examples of the product rule
– lg (36) = lg(62 ) = 2 lg 6 = 2(lg 2 + lg 3) = 2 + 2 lg 3
– lg
1
√ 2 = lg 2 2 = 12 lg 2 =
– log
1
2
1
√
3
10 = log 10 3 = 13 log(10) =
1
3
Changing Bases
• It is important and easy to change the base of a logarithm
• The logarithm of x base b is the logarithm of x base c divided by the logarithm of b base c
logb x =
logc x
logc b
• By naming terms, it is easy to prove this, let
z = logb (x)
• Then, converting to exponential form
bz = x
• Take the logarithm base c of both sides of bz = x to obtain
logc (bz ) = z logc b = logb x logc b = logc x
• That is
logb x =
153
logc x
logc b
Changing Bases
• Here are some examples of changing bases
log x
log 2
ln x
lg x =
ln 2
lg x
log x =
lg 10
ln x
log x =
ln 10
lg x
ln x =
lg e
log x
ln x =
log e
lg x =
Logarithms Approximations
• Let’s approximate some values of logarithms using the above rules
• Since
210 = 1024 ≈ 1000 = 103
• We can take the logarithm base 2 of both sides to obtain
10 = lg 210 = lg(1024) ≈ lg(1000) = lg 103 = 3 lg(10)
• That is
10
= 3.333 · · · ≈ lg 10 = 3.32192809 · · ·
3
Logarithms Approximations
• Alternatively, take the logarithm base 10 of both sides of the approximation
210 = 1024 ≈ 1000 = 103
10 log 2 = log 210 = log(1024) ≈ log(1000) = 3
• That is
0.301029995663 · · · = log 2 ≈
3
= 0.30
10
Logarithms Approximations
• Here’s another example
• We know
128 = 27 ≈ 53 = 125
• It follows that
2 ≈ 53/7
• Take the logarithm base 5 to get
0.430676558 · · · = log5 (2) ≈
3
= 0.428571 · · ·
7
• We could have written
27/3 ≈ 5
• So that
7/3 = 2.333 = · · · ≈ lg 5 = 2.321928 · · ·
154
Why Are Logarithms Important?
• Napier and others invent logarithms in the sixteenth century
• History tells why they might have been interested in logarithms
• The 15th through the 17th centuries were ages of discover
• Navigators were sailing the oceans
• Astronomers were observing the skies
• Navigational and astronomical calculations are extensive
• Logarithms simplify calculation
– Logarithms change products into sums
– Logarithms change quotients into difference
• Books of tables useful for astronomy and celestial navigation could be more quickly assembled using logarithmic calculations
Why Are Logarithms Important?
• A naive person may conclude that with computers and calculators, there is no reason to understand logarithms
• Yet there are almost unlimited applications where logarithms are used
– Calculating concentrations in chemistry
– Radioactive half-life calculations
– Calculating noise (decibel) levels
– Calculating the intensity (Richter level) of an earthquake
– Calculating annuities on loans and investments
– Describing the logarithmic spiral that occurs in the natural growth of many living creatures
– And, most importantly in computer science, describing the time complexity of many, many algorithms:
binary search, quick sort, . . .
31
Permutations
Permutations
• A permutation is a one-to-one function from a set onto itself
• A permutation orders or re-orders a set of elements
• For example, the 2! = 2 permutations on {a, b} are
(a, b)
and
155
(b, a)
Permutations
• There are 3! = 3 · 2 · 1 = 6 permutations on {a, b, c}
(a, b, c)
(a, c, b)
(b, a, c)
(b, c, a)
(c, a, b)
(c, b, a)
• If the objects are people, say: Alice, Bob and Carol, we see they can be placed in 6 = 3 · 2 · 1 = 3! different
orders
• I would have included Ted, but there would have been 24 = 4! permutations: Too many to list.
• And Ted is usually called Diane!
• The importance of the order becomes apparent if order determines a job held by the person: Butcher, Baker,
Candlestick Maker
Permutations
• There are n! permutations of a set whose cardinality is n
– There are n ways to choose the first element
– Then there are n − 1 ways to choose the second element
– Then there are n − 2 ways to choose the third element
– And so on, until there is only 1 choice for the nth element
• There are 10! = 362 880 permutations of the digits 0 through 9
• The 26 letters in the English alphabet can be permuted in
26! = 403 291 461 126 605 635 584 000 000
ways
• The 52 cards in a standard deck can be put into 52! orders: A large number
Permutations on n Things taken k at a Time
• In some applications, you are required to choose k of n elements, and then permute the element chosen
• For example, there are about 17 people running for the presidency of the United State
• Let’s assume that one will be elected President, one will be Vice President, and one will be appointed Secretary
of State
• In how many ways can this happen?
• There are 17 choices for President, then 16 choices for Vice President, then 15 choices for Secretary of State
• Therefore, there are
17 · 16 · 15
• Notice that
ways to make these assignments to office
17!
17
17!
=
· 3! =
· 3!
17 · 16 · 15 =
14! 3!14!
3
156
Permutations on n Things taken k at a Time
• To count the number permutations on n things taken k at a time
– There are
n
n!
=
k!(n − k)!
k
ways to choose k elements from a set of n elements
– There are k! permutations of k elements
– Therefore, there are
n!
n
n!
k! =
k! =
k!(n − k)!
(n − k)!
k
ways to permute n elements chosen k at a time
• Notice that
n!
= n(n − 1)(n − 2) · · · (n − k + 1)
(n − k)!
An Application from Genetics
• DNA is a string of letters over the base alphabet {A, C, T, G}
• Codons are triples of bases that encode amino acids
• How many codons are there without repeated bases
– There are
4!
4
4!
=
=4
=
3!(4 − 3)! 3!1!
3
ways to choose 3 of the four bases
– These 4 choices of 3 bases are
{A, C, T }
{A, C, G}
{A, T, G}
{C, T, G}
– There are 3! = 6 ways to permute each of these four choices
– Therefore, there are 4 · 6 = 24 codons with 3 different bases
An Application from Genetics
• Codons can have repeated bases
• That is, there are 4 · 4 · 4 = 43 = 64 different codons
• This can be visualize by using a quad-tree: A tree with 4-way branching
• A partially filled quad-tree for constructing codons might look like this
•
A
C
T
C
A
T
157
G
G
An Application from Genetics
• How many codons have repeated bases?
• There are 43 3! = 4 · 6 = 24 codons without repeated bases
• There are 4 · 4 · 4 = 43 = 64 codons
• Therefore, there are 64 − 24 = 40 codons with repeated bases
An Example on English Strings
• A string over the English alphabet is a finite sequence of letters
• How many strings of length 5 are there when all of the letters are distinct?
– Five different letters can be chosen in
26 · 25 · 24 · 23 · 22 7893600
26
26!
=
=
=
= 65780 ways
5!21!
120
120
5
– The chosen letters can be arranged in 5! = 120 ways
– Therefore, there are
26 · 25 · 24 · 23 · 22 = 7 893 600
5 letter strings with no repeated letter
Cyclic Notation
• Cyclic notation can be used to describe how elements are permuted
• Consider {a, b, c} and list them in their normal order
(a, b, c)
• Cyclic notation to denote this order is
[a][b][c]
• The notation [a][b][c] is read
– a goes to a
– b goes to b
– c goes to c
Cyclic Notation
• Cyclic notation for the order (a, c, b) is
[a][bc]
• The notation [a][bc] is read
– a goes to a
– b goes to c
– c goes to b
• The notations [a][bc] and [a][cb] are equivalent
158
• Cyclic notation for the order (b, a, c) is
[ab][c]
• The notation [ab][c] is read
– a goes to b
– b goes to a
– c goes to c
• The notations [ab][c] and [ba][c] are equivalent
Cyclic Notation
• Cyclic notation for the order (c, a, b) is
[abc]
• The notation [abc] is read
– a goes to b
– b goes to c
– c goes to a
• The notations [abc], [bca], and [cab] are equivalent
• Cyclic notation for the order (b, c, a) is
[acb]
• The notation [acb] is read
– a goes to c
– c goes to b
– b goes to a
• The notations [acb], [cba], and [bac] are equivalent
Cyclic Notation
• Use cyclic notation to describe the permutation (a, c, d, b) on the set {a, b, c, d}
• Cyclic notation for permutation (a, c, d, b) is
[a][bdc]
• Use cyclic notation to describe the permutation (d, a, b, c) on the set {a, b, c, d}
• Cyclic notation for permutation (d, a, b, c) is
[abcd]
• Use cyclic notation to describe the permutation (e, c, b, a, d) on the set {a, b, c, d, e}
• Cyclic notation for permutation (e, c, b, a, d) is
[ade][bc]
159
Cyclic Notation
• Given the cyclic notation [adc][b][e], describe the permutation
• The cyclic notation [adc][b][e] determines the permutation
(c, b, d, a, e)
• Given the cyclic notation [ade][b f c], describe the permutation
• The cyclic notation [ade][b f c] determines the permutation
(e, c, f , a, d, b)
• Given the cyclic notation [ac][be][d f ], describe the permutation
• The cyclic notation [ac][be][d f ] determines the permutation
(c, e, a, f , b, d)
Cyclic Notation
• In cyclic notation every bracketed pair is called a cycle
• The permutation [abcd] has 1 cycle
• The permutation [ab][cd] has 2 cycles
• The permutation [a][b][cd] has 3 cycles
• The permutation [a][b][c][d] has 4 cycles
• The permutation [a f ][bg][ch][di][e j] has 5 cycles
Stirling Numbers of the First Kind
• An interesting question is
– On n elements, how many permutations are there with k cycles?
• Stirling numbers of the first kind count these numbers
• There are other applications of Stirling numbers of the first kind
• A Stirling number of the first kind is denoted
n
k
• The Stirling number
n
• The Stirling number
n
k
k
counts the number of permutations of n elements with k cycles
is read
“n cycle k”
160
Stirling Numbers of the First Kind
• Consider the word FLORIDA
• There are 7! = 5040 permutations of the letters in FLORIDA
• The identity permutation leaves the letters in their original order
[F][L][O][R][I][D][A]
• This is the only permutation with 7 cycles
• Therefore, 7 cycle 7 equals 1 and we write
7
=1
7
Stirling Numbers of the First Kind
• The reverse permutation permutes the letters so they are read backward: ADIROLF
• The reverse permutation on FLORIDA is
[FA][LD][OI][R]
• This is a permutation with 4 cycles
• There are 735 permutation of 7 objects with 4 cycles
• We say 7 cycle 4 equals 735 and write
7
= 735
4
• How is
7
4
computed?
Stirling Numbers of the First Kind
• Recall Pascal’s triangle
0
1
2
3
4
5
6
7
8
..
.
0
1
1
1
1
1
1
1
1
1
..
.
1
2
3
4
5
6
7
8
1
2
3
4
5
6
7
8
..
.
1
3
6
10
15
21
28
..
.
1
4
10
20
35
56
..
.
1
5
15
35
70
..
.
1
6
21
56
..
.
1
7
28
..
.
1
8
..
.
1
..
.
161
Stirling Numbers of the First Kind
• Successive rows in Pascal’s triangle are computed from values in the previous row
• For example, from the row
8
0
1
1
8
0
1
1
9
2
36
2
28
3
56
4
70
5
56
6
28
7 8
8 1
we can calculate the next row
9
3
84
4
126
5
126
6
84
7
36
8 9
9 1
• The sum of two adjacent entries in row 8 gives a value in row 9
• Pascal’s triangle has boundary conditions: Each row starts and ends with 1
Stirling Numbers of the First Kind
• The entries in Pascal’s triangle are called binomial coefficients
• The entry row n column k is denoted by the binomial coefficient
n
n!
=
k
k!(n − k)!
• Pascal’s identity expresses the recursive relation for computing the entries in row n from entries in row n − 1
n
n−1
n−1
=
+
k
k−1
k
Stirling Numbers of the First Kind
• There is a triangle of Stirling numbers of the first kind
• Successive rows are computed from previous rows by a recursive relation that is similar to Pascal’s identity
• Before deriving the identity, let’s compute a few rows directly
Stirling Numbers of the First Kind
• On the empty set there is one permutation with 0 cycles: The empty permutation
• Therefore,
0
=1
0
Stirling Numbers of the First Kind
• On a single element set {a}
– There are no permutations with no cycles
– We say “1 cycle 0 is 0” and write
1
=0
0
– There is 1 permutation with 1 cycle: [a]
162
– We say “1 cycle 1 is 1” and write
1
=1
1
• The first two rows of Stirling’s triangle of the first kind is
0
1
0
0
1
1
1
Stirling Numbers of the First Kind
• On a two element set {a, b}
– There are no permutations with no cycles: 2 cycle 0 is 0 and
2
=0
0
– There is 1 permutation with 1 cycle
[ab]
– We say 2 cycle 1 is 1 and write
2
=1
1
– There is 1 permutation with 2 cycles
[a][b]
– We say 2 cycle 2 is 1 and write
2
=1
2
Stirling Numbers of the First Kind
• The first three rows of Stirling’s triangle of the first kind is
0
1
2
0
1
0
0
3
=0
1
2
1
1
1
Stirling Numbers of the First Kind
• On a three element set {a, b, c}
– There are no permutations with no cycles:
0
– There are 2 permutations with 1 cycle
[abc]
[acb]
and
– 3 cycle 1 is 2 and
3
=2
1
– There are 3 permutations with 2 cycles
[ab][c]
[ac][b]
163
and
[a][bc]
– We say 3 cycle 2 is 3 and write
3
=3
2
– There is 1 permutation with 3 cycles
[a][b][c]
and
3
=1
3
Stirling Numbers of the First Kind
• On a four element set {a, b, c, d} we find
• 40 = 0
• There are six 1-cycle permutations are
[abcd]
[abdc]
[adbc]
[acbd]
[acdb]
[adcb]
• These six 1-cycle permutations can be constructed from the 1-cycle permutations on {a, b, c}
– Insert d into one of 3 places in [abc]: Last, next to last, and next to first
– Insert d into one of 3 places in [acb]
Stirling Numbers of the First Kind
• There are eleven 2-cycle permutations are
[abc][d]
[acb][d]
[abd][c]
[adb][c]
[ab][cd]
[acd][b]
[adc][b]
[ac][bd]
[ad][bc]
[a][bcd]
[a][bdc]
• These eleven 2-cycle permutations can be constructed from the 1-cycle and 2-cycle permutations on {a, b, c}
– Append the cycle [d] to the 1-cycle permutations
[abc][d]
and
[acb][d]
– Insert d into 3 places in each of [ab][c], [ac][b], and [a][bc]
[abd][c]
[adb][c]
[ab][cd]
[acd][b]
[adc][b]
[ac][bd]
[ad][bc]
[a][bcd]
[a][bdc]
Stirling Numbers of the First Kind
• There are six 3-cycle permutations are
[ab][c][d]
[ac][b][d]
[ad][b][c]
[a][bc][d]
[a][bd][c]
[a][b][cd]
• There is one 4-cycle permutation is [a][b][c][d]
• Therefore
4
= 0,
0
4
= 6,
1
4
= 11,
2
164
4
= 6,
3
4
=1
4
Triangle of Stirling Numbers of the First Kind
• The first rows of Stirling’s triangle of the first kind is
0
1
2
3
4
5
6
0
1
0
0
0
0
0
0
1
2
3
1
1
2
6
24
120
1
3
11
50
274
1
6
35
225
4
5
6
1
10 1
85 15
1
• Like binomial coefficients, Stirling numbers of the first kind form two dimensional array
• Do you notice the recursion that generates rows of Stirling numbers of the first kind?
Recursion for Stirling Numbers of the First Kind
• Stirling’s identity of the first kind is
n
n−1
n−1
=
+ (n − 1)
k
k−1
k
• The boundary conditions on the identity are
n
= 0, for n > 0 and
0
n
=1
n
Numerical Examples for Stirling Numbers
• Here’s a portion of the triangle
4
5
6
0
0
0
0
1
6
24
120
2
11
50
274
• Let’s compute some examples
– 51 = 40 + 4 41 = 0 + 4 · 6 = 24
– 62 = 51 + 5 52 = 24 + 5 · 50 = 274
– 64 = 53 + 5 54 = 35 + 5 · 10 = 85
– 65 = 54 + 5 55 = 10 + 5 · 1 = 15
– 73 = 62 + 6 63 = 274 + 6 · 225 = 1624
– 74 = 63 + 6 64 = 225 + 6 · 85 = 735
165
3
6
35
225
4 5
1
10 1
85 15
6
1
Why Stirling’s Identity is Correct
• Here’s an example showing why the identity is correct
• Let {a, b, c, d, e, f , g, h} be an 8 element set
• To count the number of permutations with 5 cycles
– Consider the permutations with 4 cycles on the 7 element set {a, b, c, d, e, f , g}
– For example, here is one of them
[ab][cd][e f ][g]
– For this permutation a 5 cycle permutation can be constructed by appending the cycle [h]
[ab][cd][e f ][g][h]
– There are
7
4
five cycle permutations that can be constructed in the manner
Why Stirling’s Identity is Correct
• Consider the permutations with 5 cycles on the 7 element set {a, b, c, d, e, f , g}
• There are 75 five cycle permutations on a 7 element set
• For example, here is one of them
[ab][c][d][e f ][g]
• For this permutation the letter h can be inserted into one of 7 positions
[abh][c][d][e f ][g]
[ahb][c][d][e f ][g]
[ab][ch][d][e f ][g]
[ab][c][dh][e f ][g]
[ab][c][d][e f h][g]
[ab][c][d][eh f ][g]
[ab][c][d][e f ][gh]
Why Stirling’s Identity is Correct
• For each of the 75 five cycle permutations on 7 objects we can generate 7 five cycle permutations on 8 objects
• This is accomplished by inserting the eighth object into one of 7 positions
• Therefore, there are 7 75 five cycle permutations that can be constructed in the manner
Why Stirling’s Identity is Correct
• Five cycle permutations on 8 objects can be created in
7
from 4 cycle permutations on 7 objects
• Five cycle permutations on 8 objects can be created in 7 75 from 5 cycle permutations on 7 objects
• Adding these counts: 74 and 7 75 gives
8
7
7
=
+7
5
4
5
ways to construct 5 cycle permutations on 8 objects
166
4
Facts to See in Stirling’s Triangle of the First Kind
• That except for the first entry, the zero column is all zeros.
0
n
= 1, but
= 0,
0
0
• The 1 column is n − 1 factorial
for n > 0
n
= (n − 1)!
1
• The diagonal is always has entry 1
n
=1
n
• The sub-diagonal holds the triangular numbers
n
n
n(n − 1)
=
= tn =
2
2
n−1
Useful Permutations
• The queue permutation
• The stack permutation
32
Remainder and Mod Functions
The Division Theorem
Theorem 20 (Division Theorem). Let a and b > 0 be integers. There are unique integers q and r such that
0≤r<b
a = bq + r,
• The integer q is called the quotient
• The natural number r is called the remainder
The Quotient
• The quotient q is the floor of a divided by b
q=
• For example
– q = 24
8 =3
24 – q= 7 =3
– q = −24
= −4
7
−37 – q = 5 = −8
167
jak
b
The Remainder
• The remainder r is a mod b
r=a
(mod b)
• For example
– r = 24 (mod 8) = 0
– r = 24 (mod 7) = 3
– r = −24 (mod 7) = 4
– r = −37 (mod 5) = 3
The Mod Function
• The mod function return the non-negative remainder r when a is divided by b > 0
• That is,
r = a mod b ≥ 0
• Here are some examples mod 5
– 22 mod 5 = 2 because 22 = 5 · 4 + 2
– 53 mod 5 = 3 because 53 = 5 · 10 + 3
– −13 mod 5 = 2 because −13 = 5 · (−3) + 2
– −41 mod 5 = 4 because −41 = 5 · (−9) + 4
The C Remainder Function
• Languages in the C family tree (C++, JAVA, and C#) define a remainder operator: %
• This operator acts as the mod function on positive values
• But gives negative remainders on negative input
• That is, for d > 0
(
a mod d
a%d =
−a mod d + d
if a ≥ 0
if a < 0
• Although for a < 0, in C, the behavior is machine dependent
The C Remainder Function
• If you write programs and use the remainder operator % you should check its behavior on negative values
• It is always true that
(a/b) ∗ b + a % b = a,
• Here’s some examples of how % computes in JAVA
– 22 % 5 = 2, because 22 = 5 · 4 + 2
– −13 % 5 = −3, because −13 = 5 · (−2) − 3
– −41 % 5 = −1, because −41 = 5 · (−8) − 1
– −41 % 6 = −5, because −41 = 6 · (−6) − 5
168
if b 6= 0
33
Name Spaces
Epigraph
We do what we must, and call it by the best names.
Ralph Waldo Emerson, American essayist and poet (1803 - 1882)
Name Spaces
• A name space is a collection of objects where each has a unique name
• Names are usually assigned by rules
– The Internet Assigned Number Authority (IANA) assigns 2 letter country codes, e.g., us, fr, iq, . . . from
an alphabet of 26 letters
– The New York and American Stock Exchanges use ticker symbols that are 1, 2, or 3 letters long
– NASDAQ uses ticker symbols that are 4 letters long
– North American telephone numbers have the format NXX-NXX-XXXX, where N is a digit in the range
2–9 and X is any digit 0–9
Alphabets
• An alphabet is a finite set of characters
– English letters E = {a, b, . . . , z}
– Greek letters G = {α, β , . . . , ω}
– The digits D = {0, 1, . . . , 9}
– The bits B = {0, 1}
– The hexits H = {0, 1, . . . , 9, A, B, C, D, E, F}
• Let Σ denote a generic alphabet with characters
{g0 , g1 , . . . gm−1 }
Strings
• A string s over alphabet Σ is either
– The empty string λ
– A character g ∈ Σ
– Or the concatenation gs of a character g with a string s
Enumerating Name Spaces
Theorem 21. Let alphabet Σ have m characters. There are mn strings over Σ of length n
• When a set has m elements, we say its cardinality is m and write
|Σ| = m
169
Example Enumerations
• The total namespace for IANA two letter country codes is
26 × 26 = 676
– There are about 250 country codes
– The fill factor is about 0.37
• The total namespace for NYSE/AMEX is
26 + 262 + 263 = 18, 278
– There are about 4000 active stock symbols
– The fill factor is about 0.22
Number System Name Spaces
• There are 10n decimal numbers with n digits
– There are 10 single digit decimal numbers
{0, 1, 2, 3, 4, 5, 6, 7, 8, 9}
– There are 102 = 100 two digit decimal numbers
{00, 01, . . . , 09, 10, 11, . . . , 98, 99}
(padding numbers with leading (leftmost) zeros is okay)
– There are 103 = 1000 three digit decimal numbers
{000, 001, . . . , 988, 999}
Binary Number System
• There are 2n binary numbers with n bits
– There are 2 single bit binary numbers
{0, 1}
– There are 22 = 4 two bit binary numbers
{00, 01, 10, 11}
– There are 23 = 8 three bit binary numbers
{000, 001, 010, 011, 100, 101, 110, 111}
Hexademcial Number System
• There are 16n hexadecimal numbers with n hexits
– There are 16 single numeral hexadecimal numbers
{0, 1, . . . , 9, A, B, C, D, E, F}
– There are 162 = 28 = 256 two numeral hexadecimal numbers
{00, 01, . . . , FE, FF}
– There are 163 = 212 = 4096 three numeral hexadecimal numbers
{000, 001, . . . , FFE, FFF}
170
Languages
• A language is a subset of strings
• The strings in a language are called words
• Typically, words have some meaning
• The English language word semantics has a definition describing what it means
• The string hsilgne is (not yet) a word in the English language
Number Systems
• When the alphabet is a collection of numerals; bits, digits, hexits, all strings have meaning
• The meaning of a string of numerals can change based on context
• A binary string such as 101 could mean the natural number 5 or the integer −3 depending on context
34
Positional Number Systems
Epigraph
The creator of the universe works in mysterious ways. But he uses a base ten counting system and
likes round numbers.
Scott Adams, American cartoonist (1957 - )
Decimal Numbers
• Decimal (base 10) numbers are formed over the alphabet
D = {0, 1, 2, . . . , 8, 9}
• The position of a digit gives meaning to a number
• For example, 314 means 4 ones, 1 ten, and 3 hundreds
• Numbers are interpreted in polynomial form
314 = 3 · 102 + 1 · 10 + 4
• 3 · 102 + 1 · 10 + 4 is the polynomial
p(x) = 3x2 + x + 4
evaluated at x = 10
Decimal Numbers
• With n digits 10n unsigned integers can be named
0
to
10n − 1 = 99
· · 999}
| · {z
n nines
• The largest unsigned integer that can be represented with n digits is formed by adding the terms in the geometric
sequence
9, 90, 900, . . . , 9 × 10n−1
• The largest unsigned integer that can be represented with n digits is the geometric sum
n−1
10n − 1
∑ 9 · 10k = 9 10 − 1
k=0
171
= 10n − 1
Binary Numbers
• Binary (base 2) numbers are formed over the alphabet
B = {0, 1}
• The position of a bit gives meaning to a number
• For example, 101 means 1 ones, 0 twos, and 1 four
• Numbers are interpreted in polynomial form
101 = 1 · 22 + 0 · 2 + 1
• 1 · 22 + 0 · 2 + 1 is the polynomial
p(x) = x2 + 1
evaluated at x = 2
Binary Numbers
• With n bits 2n unsigned integers can be named
0
to
2n − 1 = 11
· · 11}
| ·{z
n ones
• The largest unsigned integer that can be represented with n bits is formed by adding the terms in the geometric
sequence
1, 10, 100, . . . , 1 × 2n−1
• The largest unsigned integer that can be represented with n bits is the geometric sum
n−1
∑ 2k =
k=0
2n − 1
= 2n − 1
2−1
Hexadecimal Numbers
• Hexadecimal (base 16) numbers are formed over the alphabet
H = {0, 1, 9, . . . , A, B, C, D, E, F}
• Here A = 10, B = 11, C = 12, D = 13, E = 14, and F = 15
• The position of a hexit gives meaning to a number
• For example, CAFE means 14 ones, 15 sixteens, 10 two-hundred-and-fifty-sixes, and 12 four-thousand-andninety-sixes
• Numbers are interpreted in polynomial form
CAFE = C · 163 + A · 162 + F · 16 + E
• 12 · 163 + 10 · 162 + 15 · 16 + 14 is the polynomial p(x) = 12x3 + 10x2 + 15x + 14 evaluated at x = 16
172
Hexadecimal Numbers
• With n hexits 16n unsigned integers can be named
0
to
16n − 1 = FF
· · FF}
| ·{z
n Fs
• The largest unsigned integer that can be represented with n hexits is formed by adding the terms in the geometric
sequence
F, F0, F00, . . . , F × 16n−1
• The largest unsigned integer that can be represented with n hexits is the geometric sum
n−1
16n − 1
∑ F · 16k = F 16 − 1
= 16n − 1
k=0
Positional Number Systems
• Numbers written in polynomial form are called positional
• When necessary, a notation such as
(314)10
can be used to denote the base of the number is 10
• The hexadecimal number (314)16 is equal to the decimal number
3 · 162 + 16 + 4 = (788)10
Positional Number Systems
• There a simple algorithms for arithmetic on positional numbers
• Decimal numbers are well known and arithmetic is first taught in this system
• Computers use binary numbers to simplfy the circuitry and logic gates
• Hexadecimal numbers are used to expose the binary numbers to human readers
– Hex strings are shorter than binary strings
• There are simple algorithms for converting between number systems
35
Conversions Between Number Systems
Epigraph
God created the integers, all the rest is the work of man.
Leopold Kronecker, Prussian mathematician (1823 - 1891)
173
Horner’s Method
• Let a = (11001101)2 be a binary number
(11001101)2 = 27 + 26 + 23 + 22 + 1
• The brute force algorithm to convert this number to decimal is to add 1 and 4 and 8 and 64 and 128
• But use Horner’s method instead
– Horner is faster
– Horner is fool-proof
– For binary-to-decimal Horner only requires doubling and adding 1
Pseudocode for Horner’s Algorithm
[1] Array a[ ] holds the numerals of a base b number Element a[i] multiplies bi
Hornerunsigned int a[ ] int p ← a[n − 1] Initialize the return value (i ← n − 2 downto 0) p ← p · b + a[i] return p
The decimal of the number
Example of Horner’s Method
• Convert (101)2 to decimal
• Write the bits 101 in a row leaving space for some arithmetic
1
1
0
1
2
4
2
5
• Bring down the leading (most significant, left-most) 1
• Multiply by the base 2 and place it under the next bit
• Add the next bit to the product
• Repeat the last two steps until all bits have been processed
• Therefore (101)2 = (5)10
Example of Horner’s Method
• Convert (11101101)2 to decimal
1
1
1
1
0
1
1
0
1
2
6
14
28
58
118
236
3
7
14
29
59
118
• Therefore (11101101)2 = (237)10
174
237
A Base Eight (Octal) Example
• Convert (3727)8 to decimal
3
3
7
2
7
24
248
2000
31
250
2007
• Therefore (3724)8 = (2007)10
A Base Sixteen (Hexadecimal) Example
• Convert (ACE)16 to decimal
10
10
12
14
160
2752
172
2766
• Therefore (ACE)16 = (2766)10
Conversion from Decimal to Another Base
• Horner’s method can used to convert from decimal to another base
• However, this requires arithmetic in the new base
• Arithmetic calculation in base 2 or 8 or 16 may be uncomfortable
• Another, less efficient, but easy to understand algorithm is useful for hand calculation
Division and Remainders
• When you divide a natural number
– By 2 the remainder is either 0 or 1
– By 3 the remainder is either 0, 1, or 2
– By 8 the remainder is either 0, 1, . . . , or 7
– By 16 the remainder is either 0, 1, . . . , or 15
• Successive remainders can be strung together to represent a decimal value in a given base
Pseudocode for Decimal to Base b Conversion
[1] Convert decimal number d ≥ 0 to base b Return an array of numerals
RepeatedDivisionunsigned int d int i ← 0 (d > 0) numerals[i] ← d % b Store the remainder d ← d/b Reduce the
problem size i + + return numerals Numerals in the base b number
175
Decimal to Binary Conversion
• Convert the decimal number 47 to binary
• Repeatedly compute the remainder upon division by 2
• Bits are computed from least (right) to most (left) significant place
47 % 2 = 1
47 ÷ 2 = 23
23 % 2 = 1
23 ÷ 2 = 11
11 % 2 = 1
11 ÷ 2 = 5
5%2 = 1
5÷2 = 2
2%2 = 0
2÷2 = 1
1%2 = 1
1÷2 = 0
• Therefore
47 = (101111)2 = 25 + 0 · 24 + 23 + 22 + 21 + 20
Decimal to Binary Conversion
• Convert the decimal number 134 to binary
134 % 2 = 0
134 ÷ 2 = 67
67 % 2 = 1
67 ÷ 2 = 33
33 % 2 = 1
33 ÷ 2 = 16
16 % 2 = 0
16 ÷ 2 = 8
8%2 = 0
8÷2 = 4
4%2 = 0
4÷2 = 2
2%2 = 0
2÷2 = 1
1%2 = 1
1÷2 = 0
• Therefore
134 = (10000110)2 = 27 + 22 + 21
Decimal to Octal Conversion
• Convert the decimal number 123 to octal
• Repeatedly compute the remainder upon division by 8
• Octal numerals are computed from least (right) to most (left) significant place
123 % 8 = 3
123 ÷ 8 = 15
15 % 8 = 7
15 ÷ 8 = 1
1%8 = 1
1÷8 = 0
• Therefore
123 = (173)8 = 82 + 7 · 8 + 3
176
Binary To Another Power of Two Conversion
• To convert from binary to another power of 2 group bits from right to left
– Two bits can be grouped to form a base 4 (quaternary) number
00 = 0,
01 = 1,
10 = 2,
11 = 3
– Three bits can be grouped to form a base 8 (octal) number
000 = 0,
001 = 1,
010 = 2,
011 = 3
100 = 4,
101 = 5,
110 = 6,
111 = 7
– Four bits can be grouped to form a base 16 (hexadecimal) number
0000 = 0,
0001 = 1,
0010 = 2,
0011 = 3
0100 = 4,
0101 = 5,
0110 = 6,
0111 = 7
1000 = 8,
1001 = 9,
1010 = A,
1011 = B
1100 = C,
1101 = D,
1110 = E,
1111 = F
Binary To Another Power of Two Conversion
• Convert (11011000)2 to quaternary (base 4)
11 01 10 00 = 3 1 2 0
(11011000)2 = (3120)4
• Convert (11011000)2 to octal (base 8)
011 011 000 = 3 3 0
(11011000)2 = (330)8
• Convert (11011000)2 to hexadecimal (base 16)
11011000 = D 8
(11011000)2 = (D8)16
Power of Two to Binary Conversion
• To convert from a power of 2 to binary expand the numerals to binary
• Convert (4567)8 to binary
4567 = 100 101 110 111
(4567)8 = (100101110111)2
• To convert (FACE)16 to binary
FACE = 1111 1010 1100 1110
(FACE)16 = (1111101011001110)2
177
36
Signed Integers
Representation of Signed Integers
• There are several ways to represent (signed) integers
– Signed-magnitude
– One’s complement (for binary)
– Two’s complement (for binary)
– Biased notation
Signed-Magnitude Notation
• Signed-magnitude is the familiar notation where negative integers have a leading minus (−) sign
• Positive integers have a leading plus (+) sign
• To represent positive and negative integers, the digit alphabet must be expanded by these new symbols
D± = {+, −, 0, 1, 2, . . . , 9}
• Alternatively, we can declare the first (leading, leftmost) digit encodes the sign
– Positive integers have a leading 0, 1, 2, 3, or 4
– Negative integers have a leading 5, 6, 7, 8, or 9
– For example
0 314 = + 314,
1 2718 = + 2718,
3 5772 = + 5772
5 314 = - 314,
6 2718 = - 2718,
7 5772 = - 5772
Signed-Magnitude Notation
• Binary signed-magnitude declares
– Positive integers have a leading 0
– Negative integers have a leading 1
– For example
0 000 = + 0,
1 0000 = - 0,
1 111 = - 7,
0 1111 = + 15
• Given a binary string, for example 10110110, you must be given context (state) to interpret its value
– unsigned int 10110110 = 182
– signed-magnitude int 1 0110110 = - 54
• Signed-magnitude requires one set of circuits for addition and another for subtraction, so it is not used in
computers
178
Converting Binary Signed-Magnitude to Decimal
• To convert a signed-magnitude binary integer to decimal
– Apply Horner’s algorithm to all but the leading bit
– If leading bit is 0 return Horner’s result
– Else return negative of Horner’s result
• Convert the sign-magnitude integer (1 101 1011)2 to decimal
1
1
0
1
1
0
1
1
2
4
10
22
44
90
2
5
11
22
45
91
• Therefore (1 101 1011)2 = −91 (since leading bit is 1)
Binary Two’s Complement Notation
• Two’s complement is the notation used in computers to represent signed integers
• Let n be an unsigned integer written in binary
• Positive integers n have a leading 0 in two’s complement notation
unsigned
n = 1011 → signed
n = 0 1011
• Negative integers n have a leading 1 in two’s complement notation
• To construct a negative integer n
– Prepend a leading 0
unsigned
n = 1011 → signed
n = 0 1011
– From right-to-left write the bits up to and including the first 1, then flip (0 becomes 1, 1 becomes 0) the
remaining bits
signed − n = 1 0101
Binary Two’s Complement Notation
• Write the decimal integer 36 in two’s complement notation
• Therefore (36)10 =
0 10 0100
2
36 % 2 = 0
36 ÷ 2 = 18
18 % 2 = 0
18 ÷ 2 = 9
9%2 = 1
9÷2 = 4
4%2 = 0
4÷2 = 2
2%2 = 0
2÷2 = 1
1%2 = 1
1÷2 = 0
in two’s complement
179
Binary Two’s Complement Notation
• Write the decimal integer −36 in two’s complement notation
• We know positive (36)10 = (0100 100)2 in two’s complement
• To express (−36)10 write the bits from right-to-left up to and including the first 1
100
• Flip the remaining bits
(−36)10 =
1 011 100
2
Binary Two’s Complement Notation
• The leading (leftmost) bit of a positive two’s complement binary integer is 0
• Each of these two’s complement integers is positive
0101,
000100,
00101010
• The leading (leftmost) bit of a negative two’s complement binary integer is 1
• Each of these two’s complement integers is negative
1101,
100100,
10101010
Negate a Two’s Complement Integer
• To negate a two’s complement integer write the bits from right-to-left up to and including the first 1, flip the
remaining bits
• The negative of 010 1 is 101 1
• The negative of 000 100 is 111 100
• The negative of 101010 10 is 010101 10
• The negative of 10011 1 is 01100 1
Convert a Two’s Complement to Decimal
• To convert a two’s complement integer n to decimal
– Apply Horner’s algorithm
– If n < 0, subtract 2m where m is the number of bits in n
• Convert n = 1011 to decimal
1
0
1
2
4
10
1
2
5
11
• Since n is negative, its decimal value is
n = 11 − 24 = 11 − 16 = −5
180
1
Convert a Two’s Complement to Decimal
• Convert n = 0110 to decimal
0
1
1
0
0
2
6
0
1
3
6
• Since n is positive, its decimal value is n = 6
Convert a Two’s Complement to Decimal
• Convert n = 1110 0110 to decimal
1
1
1
1
0
0
1
1
0
2
6
14
28
56
114
230
3
7
14
28
56
115
230
• Since n is negative, its decimal value is
n = 230 − 28 = 230 − 256 = −26
Computer Architecture
• Bits in a computer are grouped into larger chunks
• 4 bits is called a nibbles
• 8 bits is called a byte
• A word is the size of a register
– In the early 1970’s the Intel 4004 had 4 bit words
– In the mid 1970’s the Intel 8008 had 8 bit words
– In the late 1970’s the Intel 8086 had 16 bit words
– In the mid 1980’s the Intel 80386 had 32 bit words
– Today 64 bit words are common
Sign Extension Rule
• A sign extension rule is used to fill chunk (a nibble, byte, or word)
• The rule is to extend the sign to the left to fill the chunk
– Extend −2 = 10 to a nibble: 1110
– Extend 31 = 01 1111 to a byte: 31 = 0001 1111
– Extend −1 = 11 to a byte: −1 = 1111 1111
– Extend −31 = 10 0001 to a byte: −31 = 1110 0001
181
Binary Arithmetic
• Binary Addition
0
+0
0
0
+1
1
1
+0
1
1
+1
10
0
×0
0
0
×1
0
1
×0
0
1
×1
1
• Binary Multiplication
Arithmetic in Two’s Complement Notation
• Pretend a computer word is a byte (8 bits) wide
• Adding a integer to its negative in two’s complement arithmetic gives
n + (−n) = 28
which requires a 9 bit word
• For example, if n = 0000 0100, then −n = 1111 1100
• Adding n + (−n)
carries
n
−n
1
1
0
1
1 0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
0
1
1
0
0
0
0
0
0
0
0
• Discarding the last carry bit shows n + (−n) = 0
Arithmetic in Two’s Complement Notation
• Overflow occurs when a sum is too large to hold in a computer word
• Overflow occurs if the carry into the last bit does not equal the carry out
• For example, 127 = 0111 1111 plus 1 = 0000 0001 yields
carries
127
1
0
0
1
0
0
1
1
1
0
0
1
1
0
0
1
1
0
0
1
1
0
0
1
1
0
0
1
1
0
0
• The sum is wrongly computed as 1000 0000 = −128
• The carry into the last bit 1 does not match the carry out of the last bit 0
Graphical Representation of Unsigned Integers
0
000
1
001
2
010
3
011
4
100
182
5
101
6
110
7
111
1
1
0
Graphical Representation of Signed-Magnitude Integers
−3
−2
−1
111
110
101
0
000
100
1
001
2
010
3
011
Graphical Representation of Two’s Complement Integers
37
−4
−3
−2
−1
100
101
110
111
0
000
1
001
2
010
3
011
Floating Point Numbers
Epigraph
Every well-rounded programmer ought to have a knowledge of what goes on during the
elementary steps of floating point arithmetic.
Don Knuth, “The Art of Computer Programming: Seminumerical Algorithms American
computer scientist (1938 – )
Rational Numbers
• Rational numbers are those that can be written as ratios of integers
n
o
a
Q = r : r = : a ∈ Z, b ∈ Z, b 6= 0
b
• Rational numbers could be represented as a data structure of two signed integer
type Rational { int numerator; int denominator; }
• Some problems with this idea include
– Multiple representations of the same rational
– Restricted representational range: Using 64 bits
∗ the largest positive rational is only 263 − 1 ≈ 9.2 × 1018
∗ the smallest positive rational is only 1/(263 − 1) ≈ 1.1 × 10−19
IEEE 754 Floating Point Numbers
• Floating point numbers are a subset of rationals
F⊂Q
• The set of floating point numbers depends on the architecture of a computer
• The IEEE 754 Floating Point standard specifies expectations of computer architectures
• A float number type is 32 bits
• A double number type is 64 bits
• IEEE 754 floating point has too many bits and other intricacies to describe fully
• The basics of floating point numbers can be learned by studying more simple systems
183
Decimal Scientific Notation
• Scientific notation should be a familiar
• Here’s some examples as a brief review
Decimal Number
0.0000031415
161803.399
− 0.0005772
− 2.71828183
0.0060221415 × 1026
Scientific Notation
3.1415 × 10−6
1.61803399 × 105
− 5.772 × 10−4
− 2.71828183 × 100
6.0221415 × 1023
Decimal Scientific Notation
• When written in decimal scientific notation, a rational number r has the form
r = ±(i. f ) × 10e
where:
– i is a non-zero digit (1, 2, 3, . . . , 9)
∗ When i is non-zero are said to be normalized
– f is an integer representing a fractional part
– e is an integer power chosen such that
r = ±(i. f ) × 10e
Scientific Notation in Binary
• When 2 is the base, scientific notation is similar, but only bits and powers of 2 are used
Binary Number
0.0000011011
101000.011
− 0.0001001
− 1.11001110
0.0010001111 × 226
Scientific Notation
1.1011 × 2−6
1.01000011 × 25
− 1.001 × 2−4
− 1.11001110 × 20
1.0001111 × 223
Decimal and Binary Fractional Parts
• Decimal and binary fractional parts are written in positional notation
• The decimal 0.278 means “2 tenths” plus “7 hundredths” plus “8 thousandths”
0.278
=
2
7
8
+
+
10 100 1000
= 2 · 10−1 + 7 · 10−2 + 8 · 10−3 =
278
1000
• The binary 0.1001 means “1 half” plus “0 quarters” plus “0 eighths” plus “1 sixteenth”
0.1001 =
1 0 0
1
+ + +
2 4 8 16
= 1 · 2−1 + 0 · 2−2 + 0 · 2−3 + 1 · 2−4 =
184
9
16
Decimal and Binary Fractional Parts
• Here are some examples fractions in binary and their equivalent decimal representations
B INARY
0.1000
0.1100
0.1010
0.1110
0.1001
0.1101
0.1011
0.1111
S UM
1
2
+
1
2
1
1
2+4
1
1
2+8
1
1
1
2+4+8
1
1
2 + 16
1
1
1
+
2
4 + 16
1
1
1
2 + 8 + 16
1
1
1
4 + 8 + 16
D ECIMAL F RACTION
8/16
12/16
10/16
14/16
9/16
13/16
11/16
15/16
Horner’s Method for Rational Numbers
• Horner’s method should be used to convert a binary number to decimal
• First recall a decimal number such as 3.1415 means
31415
104
• Similarly binary number such as 1.1011 means
11011
24
• To convert a binary number i. f to decimal
1. Ignore the binary point
2. Convert i f to decimal by Horner’s method
3. Divide the result by 2n where n is the number of bits in f
Horner’s Method for Rational Numbers
• Convert the binary rational 1100.1001 to a decimal fraction
• Ignoring the binary point, convert 11001001 to decimal by Horner’s algorithm
1
1
1 0 0 1 0
0
2 6 12 24 50 100
3 6 12 25 50 100
• The binary rational 1100.1001 equals
9
201 201
=
= 12 +
24
16
16
• Note: the string 1100 from 1100.1001 is 12
• Note: the string 1001 from 1100.1001 is 9
185
1
200
201
Horner’s Method for Rational Numbers
• Convert the binary rational 110.01001 to a decimal fraction
• This is the same example as in the previous slides, except the binary point has been moved
• Now, we’ll divide the result by 25
1
1
1 0 0 1 0
0
2 6 12 24 50 100
3 6 12 25 50 100
1
200
201
• The binary rational 110.01001 equals
201 201
9
=
= 6+
25
32
32
Conversion of Decimal Fractions into Binary
• Recall to convert decimal integers to binary you repeatedly divide by 2 and collect remainders
• To convert fractions, repeatedly multiply by 2 extracting a bit on each multiplication
0.703125 × 2 = 1.40625
(0.1 . . .)2
0.40625 × 2 = 0.8125
(0.10 . . .)2
0.8125 × 2 = 1.625
(0.101 . . .)2
0.625 × 2 = 1.25
(0.1011 . . .)2
0.25 × 2 = 0.5
(0.10110 . . .)2
0.5 × 2 = 1.0
(0.101101)2
Conversion of Decimal Fractions into Binary
• Notice, writing these equations in reverse order, we have
1.0 = 0.5 × 2
= (0.25 × 2) × 2
= ((0.625 × 2 − 1) × 2) × 2
= (((0.8125 × 2 − 1) × 2 − 1) × 2) × 2
= ((((0.40625 × 2) × 2 − 1) × 2 − 1) × 2) × 2
= (((((0.703125 × 2 − 1) × 2) × 2 − 1) × 2 − 1) × 2) × 2
Conversion of Decimal Fractions into Binary
• These calculations can then be unrolled as a binary fraction.
1
2
1
1+ 2
2
1
1
1+ + 3
2 2
1
1
1
1+ 2 + 3 + 5
2
2
2
1
1
1
1
+ 3+ 4+ 6
2 2
2
2
= ((((0.703125 · 2 − 1) · 2) · 2 − 1) · 2 − 1) · 2
= (((0.703125 · 2 − 1) · 2) · 2 − 1) · 2
= (0.703125 · 2 − 1) · 2 · 2
= 0.703125 · 2
= 0.703125
186
Conversion of Decimal Fractions into Binary
• Convert 1/3 = 0.3333 · · · to binary
• Therefore
0.3333 · · · × 2 = 0.6666 · · ·
(0.0 . . .)2
0.6666 · · · × 2 = 1.3333 · · ·
(0.01 . . .)2
0.3333 · · · × 2 = 0.6666 · · ·
(0.010 . . .)2
0.6666 · · · × 2 = 1.3333 · · ·
(0.0101 . . .)2
1
= (0.3333 · · ·)10 = (0.0101 · · ·)2
3
Conversion of Decimal Fractions into Binary
• If you understand infinite series, you’ll appreciate that
3
(1.111 · · · )
10
3 ∞ −k
=
∑ 10
10 k=0
0.3333 · · · =
=
1
3
1
10 1 − 10
3 10
10 9
1
=
3
=
• And the binary representation 0.0101 · · · is the series
1
0.0101 · · · = (1.010 · · · )
4
1 ∞
= ∑ 4−k
4 k=0
=
1 1
4 1 − 14
14
43
1
=
3
=
Scientific Notation in Binary
• When written in binary scientific notation a rational number r has the form
r = ±(1. f ) · 2e
where:
– f is an integer representing a fractional part
– e is an integer power chosen such that
r = ±(1. f ) × 2e
• Numbers of the form 1. f are said to be normalized
• The 1 in (1. f ) is implicit and not explicitly written
187
Binary Floating Point Numbers
• Floating point numbers are represented as a string of bits
• The scheme used to represent a rational r is
– A sign bit s, the leftmost, determines if r < 0 or r ≥ 0
– Followed by n bits that represent an exponent e
– With the final m bits representing a fractional part f
• That is, a floating point number is parsed as
1 sign bit
s
n exponent bits
e
m fractional part bits
f
Binary Floating Point Numbers
• Let’s interpret the decimal meaning of the binary floating point number 0 111 1101
• The sign bit is s = 0
– A sign bit of 0 means the rational number is positive
– A sign bit of 1 means the rational number is negative
• The fraction part is 1101
• Prepending the implicit 1, evaluate 1.1101 by Horner’s method
1
1
1 1 0
2 6 14
3 7 14
1
28
29
1.1101 =
29
13
= 1+
16
16
Binary Floating Point Numbers
• Binary floating point 0 111 1101 represents
+
29
29
× 2e = + × 2111
16
16
• The three bit exponent 111 needs to be interpreted as a decimal e
• Floating point exponents are written in biased notation
Biased Notation
• Biased notation presents an alternative to two’s complement for representing integers (negative, zero, and positive whole numbers)
• The idea is to use the natural numbers in the range
0 ≤ k ≤ N = {0, 1, 2, . . . , N − 1, N}
to represent the range
−b ≤ k ≤ N − b = {−b, 1 − b, 2 − b, . . . , +N − b − 1, N − b}
• That is, the number k stands for the value k − b
• The number b is called the bias
188
Decimal Biased Notation
• In decimal, pretend that 3 digits are used to represent an exponent
• That is, decimal strings from 000 to 999 encode an exponent
• Select a bias b near the middle of this range, say b = 500
• Let 000 ≤ k ≤ 999 and declare that k represents the value k − b = k − 500
Decimal Biased Notation
• For example
Biased Representation: k
000
007
200
454
700
836
999
“Real” Value: k − 500
−500
−493
−300
−46
200
336
499
Binary Biased Notation
• In binary, pretend that 3 bits can be used to represent an exponent
• That is, binary strings from 000 to 111 encode an exponent
• Select a bias b near the middle of this range, say b = 4 = 100
• Let 000 ≤ k ≤ 111 and declare that k represents the value k − b = k − 4
Binary Biased Notation
• For example
Biased Representation: k
(000)2 = 0
(001)2 = 1
(010)2 = 2
(011)2 = 3
(100)2 = 4
(101)2 = 5
(110)2 = 6
(111)2 = 7
“Real” Value: k − 4
−4
−3
−2
−1
0
1
2
3
Completing the Example
• Recall, the binary floating point 0 111 1101 represents
+
29
29
× 2 p = + × 2111
16
16
• The three bit exponent 111 is a biased number with bias b = 4
• The exponent 111 is interpreted as representing
(111)2 − 4 = 7 − 4 = 3
• Therefore, the binary floating point 0 111 1101 represents
+
29
29
× 23 = +
16
2
189
Another Example
• Interpret the binary floating point 1 010 0100 as a decimal rational number r
• The sign bit is s = 1
– The rational number r is negative
• The exponent is e = 010
– With a bias of b = 4, the “real” value of e is
e = (010)2 − 4 = 2 − 4 = −2
Another Example
• The fractional part is f = 0100
– Appending a leading 1 and evaluating 1.0100 yields
1
1
0 1 0
2 4 10
2 5 10
0
20
20
1.0100 =
1
20 5
= = 1+
16 4
4
• Therefore, the binary floating point 1 010 0100 represents
5
5
− × 2−2 = −
4
16
An Example with 16 Bits
• What is the decimal value of the binary floating point
1 010010 100110101
• The sign bit is s = 1
– The decimal number is negative
• The exponent is e = 010010, 6 bits long
– With 6 bits, the range of unsigned integers is from 0 to 26 − 1 = 63
– Choose a bias in the middle of this range, say b = 32
– With a bias of b = 32, the “real” value of e is
e = (010010)2 − 32 = 18 − 32 = −14
An Example with 16 Bits
• The fractional part is f = 100110101
– Appending a leading 1 and evaluating 1.100110101 yields
1
1 0 0 1 1
0
1
0
2 6 12 24 50 102 204 410
1 3 6 12 25 51 102 205 410
821
309
= 1+
512
512
• Therefore, the binary floating point 1 010010 100110101 represents
1.100110101 =
−
821
821
× 2−14 = − 23
512
2
190
1
820
821
Normalized and Denormalized Floating Points
• A problem with our scheme is that we can not represent 0
• Normalized floating point numbers always have the form
r = ±(1. f ) × 2e
• With the leading 1 in 1. f they can never equal 0
• A denomalized floating point has the form
r = ±(0. f ) × 2e
• There are reasons to allow denormalized floats
• We will declare the zero string 0000 0000 . . . to be denormalized and represents 0
• All other strings are bits are normalized floats
Distribution of Floating Point Numbers
• It is insightful to see how floating point numbers are distributed along a number line
• Consider the non-negative floats that can be represented using 9 bits with 1 sign bit, 3 exponent bits, and 5
fractional part bits
• The biased exponents range from 000 to 111, that is from 0 to 7.
• Using a bias of b = 4 the “real” range is from −4 to 3
• The normalized values 1. f range from 1.00000 to 1.11111
• Expressed in decimal the normalized values range from
1
to
1+
31 63
=
32 32
Distribution of Floating Point Numbers
• The normalized values (1. f ) increment in steps of 1/32 as the values increase
1.00000 = 1
1
32
2
1.00010 = 1 +
32
3
1.00011 = 1 +
32
..
.
. = ..
30
1.11110 = 1 +
32
31
1.11111 = 1 +
32
1.00001 = 1 +
191
33
32
34
=
32
35
=
32
=
62
32
63
=
32
=
Distribution of Floating Point Numbers
• The smallest exponential scale is 2−4
• At this scale, the normalized values increment is steps of
1
1
1
×
=
32 16 512
• And at this smallest scale, the representable positive floats are
0 000 00001 =
0 000 00010 =
..
.
=
0 000 11110 =
0 000 11111 =
33
512
34
512
..
.
62
512
63
512
Distribution of Floating Point Numbers
• At this smallest exponential scale the floats range from
33
512
63
512
• That is, these float range from their smallest positive value, a little bigger than 1/16, to almost 1/8
• The representable floats in this range are “fairly” tightly bunched: Separated by steps of 1/512
Distribution of Floating Point Numbers
• There are 63 − 33 + 1 = 31 floating point numbers between
33
512
63
512
• There is a large gap in representable numbers from 0 to 33/512
0
33
512
63
512
Distribution of Floating Point Numbers
• The next smallest exponential scale is 2−3
• At this scale, the normalized values increment is steps of
192
1
(32)(8)
=
1
256
x
• And at this scale, the representable floats are
0 001 00000 =
0 001 00001 =
..
.
=
0 001 11110 =
0 001 11111 =
32
256
33
256
..
.
62
256
63
256
Distribution of Floating Point Numbers
• At this next smallest exponential scale the floats range from
32
256
63
256
• That is, 1/8 to almost 1/4
• The representable floats in this range are less tightly bunched: Separated by steps of 1/256
Distribution of Floating Point Numbers
• There are 63 − 32 + 1 = 32 floating point numbers between
63
256
32
256
0
33
512
32
256
63
256
Distribution of Floating Point Numbers
• The next smallest exponential scale is 2−2
• At this scale, the normalized values increment is steps of
1
(32)(4)
=
• And at this scale, the representable floats are
0 010 00000 =
0 010 00001 =
0 010 00010 =
..
.
=
0 010 11110 =
0 010 11111 =
193
32
128
33
128
34
128
..
.
62
128
63
128
1
128
x
Distribution of Floating Point Numbers
• At exponential scale 2−2 the floats range from
63
128
32
128
• That is, from 1/4 to almost 1/2
• The representable floats in this range are still less tightly bunched: Separated by steps of 1/128
Distribution of Floating Point Numbers
• There are 63 − 32 + 1 = 32 floating point numbers between
32
128
0
33
512
32
256
63
128
32
128
63
128
x
Distribution of Floating Point Numbers
• The largest exponential scale is 23
• At this scale, the normalized values increment is steps of
8
32
=
1
4
• And at this scale, the representable floats are
0 111 00000 =
0 111 00001 =
..
.
=
0 111 11110 =
0 111 11111 =
32
4
33
4
..
.
62
4
63
4
Distribution of Floating Point Numbers
• At the largest exponential scale the floats range from
32
4
63
4
• That is, from 8 to their largest possible value 63/4
• The representable floats in this range are not tightly packed: They are separated by steps of 1/4
194
Distribution of Floating Point Numbers
• The smallest positive float is
0 000 00001 =
33
33
× 2−4 =
32
512
• The largest positive float is
0 111 11111 =
63
63
× 23 =
32
4
Distribution of Floating Point Numbers
• Here is a complete graph of the representable floating point numbers using this scheme
0 1 2
4
8
63
4
A Smaller Example
• Consider the floating point numbers that can be represented with
– 1 sign bit
– 2 exponent bits
– 3 fractional part bits
• These float point numbers have the form
s
e0 e1
f0 f1 f2
• 0 is represented by the denormalized string
0 00 000
• The smallest positive float is represented by
0 00 001
• The largest positive float is represented by
0 11 111
A Smaller Example: The Biased Exponents
• The 2-bit biased exponent ranges from 00 to 11
• Choosing a bias of b = 2 gives a “real” or “unbiased” exponent range from −2 to 1
• And the normalized fractional parts (1. f ) can be scaled by
2−2 ,
2−1 ,
195
20 ,
21
x
A Smaller Example: The Normalized Fractional Parts
• The normalized fractional parts (1. f ) range from
1.000
to
1.111
• Convert the values in this range to decimal notation
9
8
11
1.011 =
8
13
1.101 =
8
15
1.111 =
8
1.000 = 1
1.001 =
10
8
12
1.100 =
8
14
1.110 =
8
1.010 =
A Smaller Example: The Smallest Scale
• At the smallest scale, 2−2 , the smallest representable float is
9
9
0 00 001 = + × 2−2 =
8
32
• The other representable float point numbers at this scale are
10
× 2−2 =
8
12
× 2−2 =
8
14
× 2−2 =
8
10
32
12
32
14
32
11
× 2−2 =
8
13
× 2−2 =
8
15
× 2−2 =
8
11
32
13
32
15
32
A Smaller Example: The Smallest Scale Graph
• A number line graph of these values is
0
• Any computed number x between 0 and
9
32
9
32
12
32
will be rounded to either 0 or
15
32
9
32
• Any computed x between representable float point numbers will be rounded
• These are called floating point rounding errors
196
x
A Smaller Example: The Largest Scale
• At the largest scale, 21 , the first representable float is
0 11 000 = +1 × 21 = 2 =
8
4
• The other representable float point numbers at this scale are
9
× 21 =
8
11
× 21 =
8
13
× 21 =
8
15
× 21 =
8
9
4
11
4
13
4
15
4
10
× 21 =
8
12
× 21 =
8
14
× 21 =
8
10
4
12
4
14
4
A Smaller Example: The Largest Scale Graph
• A number line graph of these values is
0
8
4
12
4
15
4
x
• Any computed number x between these representable float point numbers will be rounded-off
A Smaller Example: The Intermediate Scales
• At the intermediate scale, 2−1 , the first representable float is
0 01 000 = +1 × 2−1 =
1
8
=
2 16
• The other representable float point numbers at this scale are
9
× 2−1 =
8
11
× 2−1 =
8
13
× 2−1 =
8
15
× 2−1 =
8
9
16
11
16
13
16
15
16
10
× 2−1 =
8
12
× 2−1 =
8
14
× 2−1 =
8
197
10
16
12
16
14
16
A Smaller Example: The Intermediate Scales
• At the intermediate scale, 20 , the first representable float is
0 10 000 = +1 × 20 = 1 =
8
8
• The other representable float point numbers at this scale are
9
× 20 =
8
11
× 20 =
8
13
× 20 =
8
15
× 20 =
8
9
8
11
8
13
8
15
8
10
× 20 =
8
12
× 20 =
8
14
× 20 =
8
10
8
12
8
14
8
A Smaller Example: The Line Graph
• A number line graph for all of the non-negative representable float point numbers is
0
9
32
1
2
1
2
15
4
x
• Any computed number x between these representable float point numbers will be rounded-off
A Smaller Example: Floating Point Errors
• Floating point arithmetic is not precise: Errors will be made
• A novice programmer mistake is to test a floating point number against 0
if (x == 0.0) ...
• Equivalently: Testing if two computed float point numbers are equal is a mistake
if (x == y) ...
• Such a test can never be more precise that the machine epsilon
A Smaller Example: Floating Point Errors
• The machine epsilon is the largest value ε = 2e such that
1.0 + ε = 1.0
• For our small example
ε=
• Note that 1 + 18 =
1
• But 1 + 16
=
17
16
9
8
1
16
is a representable float
is not representable, and the sum will be rounded to 1
198
A Smaller Example: Floating Point Errors
• Floating point arithmetic does not obey common arithmetic principles
• For example, floating point arithmetic is not associative
– The sum
1+
5
5
+
16 16
= 1+
10
16
=
26
16
is computed correctly: All values are representable floating point numbers
– However, the sum
5
20
24
5
5
1+
=
=
+
+
16
16
16
16 16
is not computed correctly
21
20
16 is not representable and rounded to 16
5
25
24
16 = 16 is not representable and rounded to 16
5
=
∗ 1 + 16
∗
20
16
+
IEEE 754 Floating Point Standard
• The IEEE 754 Floating Point Standard provides a model to make errors minimal and provide consistency across
programming languages and computer architectures
• A good summary by David Goldberg is
What Every Computer Scientist Should Know About Floating Point Arithmetic
• The IEEE standard defines four different precisions
• Most hardware provides for single (type float) and double precision (type double)
IEEE Single Precision
• The format parameters for single precision are
– Word length: 32 bits
– 1 sign bit
– 8 exponent bits
– 23 fractional part bits
• The exponent bias is 127
– The unbiased exponent range is from −127 to 128
– But the exponents −127 and 128 are used for special qualities
– The practical range of exponents is from emin = −126 to emax = 127
– Since 210 ≈ 103 , floats can scale from about
2−126 = (210 )−12.6 ≈ (103 )−12.6 ≈ 10−38
to
2127 = (210 )12.7 ≈ (103 )12.7 ≈ 1038
199
IEEE Single Precision
• Because of the hidden bit in (1. f ), there are 24 bits of precision in floats
• Since 210 ≈ 103 , 24 bits of precision gives about 7 digits of decimal precision
• That is
224 = (210 )2.4 ≈ (103 )2.4 ≈ 107
• At the smallest scale, floats are separated by increments of
2−23 × 2−126 = 2−149
• At the largest scale, floats are separated by increments of
2−23 × 2127 = 2104
IEEE Double Precision
• The format parameters for double precision are
– Word length: 64 bits
– 1 sign bit
– 11 exponent bits
– 52 fractional part bits
• The exponent bias is 1023
– The unbiased exponent range is from −1023 to 1024
– The exponents −1023 and 1024 are used for special qualities
– The practical range of exponents is from emin = −1022 to emax = 1023
– Since 210 ≈ 103 , doubles can scale from about
2−1022 = (210 )−102.2 ≈ (103 )−102.2 ≈ 10−306
to
21023 = (210 )102.3 ≈ (103 )102.3 ≈ 10309
IEEE Double Precision
• Because of the hidden bit in (1. f ), there are 53 bits of precision in doubles
• Since 210 ≈ 103 , 53 bits of precision gives about 16 digits of decimal precision
• That is
253 = (210 )5.3 ≈ (103 )5.3 ≈ 1016
• At the smallest scale, doubles are separated by increments of
2−52 × 2−1022 = 2−1074
• At the largest scale, doubles are separated by increments of
2−52 × 21023 = 2971
200
IEEE Special Qualities
• When the exponent e is at its extremes, special qualities of the number are defined
– Recall, for single precision the exponent e can range from emin − 1 = −127 to emax + 1 = 128
– For double precision the exponent can range from emin − 1 = −1023 to emax + 1 = 1024
• When the exponent takes on its most negative value
– The float represents 0 if the fractional part f is 0
– The float represents the denormalized number ±(0. f ) × 2emin if f 6= 0
• When the exponent takes on its most positive value
– The float represents infinity (∞) if the fractional part f is 0
– The float represents “Not a Number” (NaN) if f 6= 0
How fine is floating-point arithmetic?
• Here Nick Trefethen’s answer to the density of floating point numbers
• According to molecular physics, there are approximately 3 × 108 molecules per meter in a gas at atmospheric
conditions
– Essentially the cube root of Avogadro’s number: 6.02 × 1023
• The circumference of the earth is 4 × 107 meters,
How fine is floating-point arithmetic?
• So in a circle around the earth, there are around (3 × 108 ) × (4 × 107 ) ≈ 1016 molecules
• In IEEE double precision arithmetic, there are 252 numbers between 1 and 2, which is also about 1016
• So if we put a giant circle around the earth with a distance coordinate ranging from 1 to 2, the spacing of the
floating-point numbers along the circle will be about the same as the spacing of the air molecules.
38
Modular Numbers
Epigraph
Mathematics is the queen of the sciences and number theory is the queen of mathematics.
Carl Friedrich Gauss, German mathematician (1777 – 1855)
Modular Numbers
• We want to define a different type of integer [r], called a modular integer that:
– Behaves arithmetically as ordinary integers do with respect to addition and multiplication
– Yet a modular integer [r] counts circularly, from 0 to m − 1, then back to 0
• Here are some familiar modular systems
–
–
–
–
Clocks are mod 12 systems, with 12 equivalent to 0
Odometers count up to some value, say 999, 999 then rollover to 0
Combination locks roll through a fixed range, then back to 0
Addition on a fixed number of bits or digits
201
The Mod 2 Numbers
• Define the integer modulo 2, to be
Z2 = {[0], [1]}
• Define addition on the mod 2 numbers by
+
0
1
0
0
1
1
1
0
+
0
1
0
0
0
1
0
1
• Define multiplication on the mod 2 numbers by
The Mod 2 Numbers
• The mod 2 integer [0] represents the set of even integers
[0] = {0, ±2, ±4, ±6, . . .} = {2n : n ∈ Z}
• The mod 2 integer [1] represents the set of odd integers
[1] = {1, 1 ± 2, 1 ± 4, 1 ± 6, . . .} = {2n + 1 : n ∈ Z}
• The mod 2 addition table represents the facts
[0] + [0] = [0]
– “Even plus even is even”
– “Even plus odd is odd”
[0] + [1] = [1] and [1] + [0] = [1]
– “Odd plus odd is even”
[1] + [1] = [0]
• The mod 2 multiplication table represents the facts
[0] · [0] = [0]
– “Even times even is even”
[0] · [1] = [0] and [1] · [0] = [0]
– “Even times odd is even”
[1] · [1] = [1]
– “Odd times odd is odd”
Congruence Mod 2
• Two integers x and y are congruent modulo 2 if either
– Both x and y are even, or
– Both x and y are odd
• Two integers x and y are congruent modulo 2 if
– Both x and y have the same remainder when divided by 2
• Two integers x and y are congruent modulo 2 if
– Their difference x − y is a multiple of 2
202
Congruence Mod 2
• When x is congruent to y modulo 2, we write
x ≡ y mod 2
• The following statements are true
2 ≡ 0 mod 2,
−4 ≡ 18 mod 2,
5 ≡ 21 mod 2
• When x is not congruent to y modulo 2, we write
x 6≡ y mod 2
• The following statements are true
2 6≡ 3 mod 2,
−4 6≡ 17 mod 2,
8 6≡ 21 mod 2
The Mod 3 Numbers
• Define the integer modulo 3, to be
Z3 = {[0], [1], [2]}
• Define addition on the mod 3 numbers by
+
0
1
2
0
0
1
2
1
1
2
0
2
2
0
1
+
0
1
2
0
0
0
0
1
0
1
2
2
0
2
1
• Define multiplication on the mod 3 numbers by
The Mod 3 Numbers
• The mod 3 integer [0] represents the set of integers that are multiples of 3
[0] = {0, ±3, ±6, ±9, . . .} = {3n : n ∈ Z}
• The mod 3 integer [1] represents the set of integers that have a remainder of 1 when divided by 3
[1] = {1, 1 ± 3, 1 ± 6, 1 ± 9, . . .} = {3n + 1 : n ∈ Z}
• The mod 3 integer [2] represents the set of integers that have a remainder of 2 when divided by 3
[2] = {2, 2 ± 3, 2 ± 6, 2 ± 9, . . .} = {3n + 2 : n ∈ Z}
Congruence Mod 3
• The mod 3 addition table represents the facts
–
–
–
–
–
–
3n + 3m = 3(n + m)
3n + (3m + 1) = 3(n + m) + 1
3n + (3m + 2) = 3(n + m) + 2
(3n + 1) + (3m + 1) = 3(n + m) + 2
(3n + 1) + (3m + 2) = 3(n + m + 1)
(3n + 2) + (3m + 2) = 3(n + m + 1) + 1
([0] + [0] = [0])
([0] + [1] = [1])
([0] + [2] = [2])
([1] + [1] = [2])
([1] + [2] = [0])
([2] + [2] = [1])
203
Congruence Mod 3
• The mod 3 multiplication table represents the facts
– 3n · 3m = 3(3nm)
([0] · [0] = [0])
– 3n · (3m + 1) = 3(3nm + n)
([0] · [1] = [0])
– 3n · (3m + 2) = 3(3nm + 2n)
([0] · [2] = [0])
– (3n + 1) · (3m + 1) = 3(3nm + n + m) + 1
([1] · [1] = [1])
– (3n + 1) · (3m + 2) = 3(3nm + 2n + m) + 2
([1] · [2] = [2])
– (3n + 2) · (3m + 2) = 3(nm + 2n + 2m + 1) + 1
([2] · [2] = [1])
Congruence Mod 3
• Two integers x and y are congruent modulo 3 if
– Both x and y have the same remainder when divided by 3
• Two integers x and y are congruent modulo 3 if
– Their difference x − y is a multiple of 3
Congruence Mod 3
• When x is congruent to y modulo 3, we write
x ≡ y mod 3
• The following statements are true
3 ≡ 0 mod 3,
−4 ≡ 17 mod 3,
6 ≡ 21 mod 3
• When x is not congruent to y modulo 3, we write
x 6≡ y mod 3
• The following statements are true
2 6≡ 3 mod 3,
−4 6≡ 16 mod 3,
8 6≡ 21 mod 3
Modular Numbers
• Given an integer m > 1, called the modulus, define sets of modular integers,
Zm = {[0], [1], [2], . . . , [m − 1]}
• A modular integer [r] is not really an integer, but a set of periodic integers
• The brackets [ ] can be dropped when the context is clear
• When the context is clear simply write r as a modular integer
• Modular arithmetic behaves more or less like arithmetic on the integers
• In practice, m is typically large and prime
204
Modular Numbers
• A modular integer [r] is a set of integers
[r] = {r, r ± m, r ± 2m, r ± 3m, . . .}
= {n : n = mk + r, where k ∈ Z}
• The set [r] is called an equivalence class, or in this context a residue class
Modular Numbers
• For example, the set of integers mod 5 is
[0] = {0, ±5, ±10, ±15, . . .}
= {n : n = 5k, where k ∈ Z}
[1] = {1, 1 ± 5, 1 ± 10, . . .}
= {n : n = 5k + 1, where k ∈ Z}
[2] = {2, 2 ± 5, 2 ± 10, . . .}
= {n : n = 5k + 3, where k ∈ Z}
[3] = {3, 3 ± 5, 3 ± 10, . . .}
= {n : n = 5k + 3, where k ∈ Z}
[4] = {4, 4 ± 5, 4 ± 10, . . .}
= {n : n = 5k + 4, where k ∈ Z}
The Mod 5 Number System
• You can “see” the mod 5 numbers by listing them in rows, 5 columns to a row
−5
−4
−3
−2
−1
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
The Mod 5 Number System
• For the modular number
Z5 = {[0], [1], [2], [3], [4]}
– [0] stands for multiples of 5
– [1] stands for multiples of 5 plus 1
– [2] stands for multiples of 5 plus 2
– [3] stands for multiples of 5 plus 3
– [4] stands for multiples of 5 plus 4
• Mod 5 partitions the integers Z into 5 disjoint residue classes
The Mod 7 Number System
• Let m = 7 and consider the integers mod 7
Z7 = {[0], [1], [2], [3], [4], [5], [6]}
205
• Let’s start with an addition table
+
0
1
2
3
4
5
6
0
0
1
2
3
4
5
6
1
1
2
3
4
5
6
0
2
2
3
4
5
6
0
1
3
3
4
5
6
0
1
2
4
4
5
6
0
1
2
3
5
5
6
0
1
2
3
4
6
6
0
1
2
3
4
5
The Mod 7 Number System
• Here are some numeric examples of addition modulo 7
–
–
–
–
–
[0] + [3] = [3] because [7c] + [7d + 3] = [7(c + d) + 3]
[2] + [4] = [6] because [7c + 2] + [7d + 4] = [7(c + d) + 6]
[4] + [5] = [2] because [7c + 4] + [7d + 5] = [7(c + d + 1) + 2]
[3] + [4] = [0] because [7c + 3] + [7d + 4] = [7(c + d + 1)]
[5] + [5] = [3] because [7c + 5] + [7d + 5] = [7(c + d + 1) + 3]
• In general, [a] + [b] = [a + b] = [r] where r ≡ (a + b) mod 7
The Mod 7 Number System
• In modular arithmetic, large positive numbers are replaced by their smaller remainders
• For example, mod 7 we compute
–
–
–
–
[34] = [6] because [34] = [7(4) + 6]
[56] = [0] because [56] = [7(8) + 0]
[74] = [4] because [74] = [7(10) + 4]
[195] = [6] because [195] = [7(27) + 6]
The Mod 7 Number System
• Here are some numeric examples of negating integers modulo 7
–
–
–
–
–
−[0] = [0] because −[7c] = [7(−c)]
−[2] = [5] because −[7c + 2] = [7(−c) − 2] = [7(−c − 1) + 5]
−[4] = [3] because −[7c + 4] = [7(−c) − 4] = [7(−c − 1) + 3]
−[5] = [2] because −[7c + 5] = [7(−c) − 5] = [7(−c − 1) + 2]
−[6] = [1] because −[7c + 6] = [7(−c) − 6] = [7(−c − 1) + 1]
• In general, −[a] = [−a + 7c], where 7c is chosen to make −a + 7c ≥ 0
The Mod 7 Number System
• Absolutely large negative numbers are replaced by their smaller non-negative remainders
• For example, mod 7 we compute
–
–
–
–
[−68] = [2] because [−68] = [7(−10) + 2]
[−36] = [6] because [−36] = [7(−6) + 6]
[−74] = [3] because [−74] = [7(−11) + 3]
[−195] = [1] because [195] = [6]
206
The Mod 7 Number System
• The modulus m is often a prime number
• When the modulus m is prime each modular integer a 6= 0 has a multiplicative inverse a−1
• Here’s the mod 7 multiplication table (with bracket [ ] dropped)
×
0
1
2
3
4
5
6
0
0
0
0
0
0
0
0
1
0
1
2
3
4
5
6
2
0
2
4
6
1
3
5
3
0
3
6
2
5
1
2
4
0
4
1
5
2
6
3
5
0
5
3
1
6
4
4
6
0
6
5
4
3
2
1
The Mod 7 Number System
• Here’s some of the arithmetic behind this multiplication table
– 2 × 3 = 6 = 6 mod 7
– 3 × 5 = 15 = 1 mod 7
– 3 × 6 = 18 = 4 mod 7
– 4 × 3 = 12 = 5 mod 7
– 5 × 5 = 25 = 4 mod 7
– 5 × 6 = 30 = 2 mod 7
– 6 × 6 = 36 = 1 mod 7
The Mod 7 Number System
• In mod 7 multiplication every non-zero modular integer r has a “multiplicative inverse” or “reciprocal” r−1 such
that
r × r−1 = 1
– 2 × 4 = 8 = 1 mod 7, so 2−1 = 4 and 4−1 = 2
– 3 × 5 = 15 = 1 mod 7, so 3−1 = 5 and 5−1 = 3
– 6 × 6 = 36 = 1 mod 7, so 6−1 = 6
• When the modulus m is a prime number, every non-zero modular integer r has a multiplicative inverse
The Mod 6 Number System
• When the modulus m is not prime, some non-zero modular integer r will not have multiplicative inverses
• Consider mod 6 multiplication
×
0
1
2
3
4
5
0
0
0
0
0
0
0
1
0
1
2
3
4
5
207
2
0
2
4
0
2
4
3
0
3
0
3
0
3
4
0
4
2
0
4
2
5
0
5
4
3
2
1
The Mod 6 Number System
• Notice that 2 does not have a multiplicative inverse mod 6
2x 6= 1
for any mod 6 integer
• The numbers 2 and 6 have common divisor: 2
• Notice that 3 does not have a multiplicative inverse mod 6
3x 6= 1
for any mod 6 integer
• The numbers 3 and 6 have common divisor: 3
• Notice that 4 does not have a multiplicative inverse mod 6
4x 6= 1
for any mod 6 integer
• The numbers 4 and 6 have common divisor: 2
• Notice that 5 does have a multiplicative inverse mod 6
5 · 5 = 25 = 1 mod 6
• The numbers 5 and 6 have only 1 as a common divisor
Linear Congruence Equations
• A linear congruence equation has the form
ax = b mod m
• Recall a linear equation has the form
ax = b
• And the equation ax = b is solved by multiplying both sides by the multiplicative inverse (reciprocal) of a
a−1 ax = x = a−1 b
Linear Congruence Equations
• Pretend a has a multiplicative inverse mod m
• That is, pretend there is an integer a−1 such that
a · a−1 = 1 mod m
• Then the linear congruence equation ax = b mod m can be solved by multiplying both sides of the equation by
a−1
– 2 has inverse 4 mod 7; 2 · 4 = 8 = 1 mod 7
– Therefore 2x = 5 mod 7 has solution x = 6, that is
x = 4 · 2x = 4 · 5 = 6 mod 7
• 5 has inverse 5 mod 6; 5 · 5 = 25 = 1 mod 6
• Therefore 5x = 4 mod 6 has solution x = 2, that is
x = 5 · 5x = 5 · 4 = 20 = 2 mod 6
208
Linear Congruence Equations
• If a does not have a multiplicative inverse mod m, then linear congruence equation ax = b mod m may not have
a solution
• If a does not have a multiplicative inverse mod m, then the equation ax = b mod m may have multiple solutions
– 2 does not an inverse mod 6; 2x = 0, 2, 4 mod 6
– Therefore 2x = 3 mod 6 has no solution, that is,
2x 6= 3 mod 6
for any x
– But the 2x = 4 mod 6 has 2 solutions
x=2
and
x=5
Linear Congruence Equations
• A brute force algorithm to solve the linear congruence ax = b mod m would be to complete a row of the mod m
multiplication table
• For example, to solve 3x = 4 mod 7 create the table
× 0
3 0
1 2 3 4 5 6
3 6 2 5 1 4
• From the table we can see that 3 · 6 = 4 mod 7, so x = 6
• We can see that 3 · 5 = 1 mod 7, so the equation can be solved by multiplying by 5
x = 5 · 3x = 5 · 4 = 20 = 6 mod 7
Linear Congruence Equations
• The brute force algorithm can become impractical when the modulus m is large
• For example, to solve 3x = 4 mod 35 create the table
× 0 1 2 3
3 0 3 6 9
4
12
5
15
6 · · · 34
24 · · · 32
• Filling the entire table is tedious
• Better algorithms rely on ideas from number theory
Number Theory
• There are ideas from number theory that must be understood in solving linear congruence equations
• These ideas are
– The fundamental theorem of arithmetic
– The divides relation
– The greatest common divisor of two integers
– And integers that are relatively prime
209
The Fundamental Theorem of Arithmetic
Theorem 22 (Fundamental Theorem of Arithmetic). Every integer n > 1 can be represented as a product of prime
factors in only one way, apart from the order of the factors
The Fundamental Theorem of Arithmetic
• For example
– 30 can be factored as
30 = 2 · 3 · 5
– 154 can be factored as
154 = 2 · 7 · 11
– 108 can be factored as
108 = 2 · 2 · 3 · 3 · 3 = 22 · 33
– 288 can be factored as
288 = 2 · 2 · 2 · 2 · 2 · 3 · 3 = 25 · 32
– 10! can be factored as
10! = 28 · 34 · 52 · 7
The Fundamental Theorem of Arithmetic
• Unfortunately, factoring appears to be a hard problem
• Unfortunately, third grade teachers don’t seem to know this (my apologies to these teachers)
• Factoring an integer into primes is so difficult it is the basis for the RSA cryptography algorithm
The Divides Relation
• The divides relation is defined on the integers
• An integer d divides n if there is an integer c such that
n = cd
• For example
– 5 divides 35 because 35 = 7 · 5
– 13 divides 39 because 39 = 3 · 13
The Divides Relation
• If d divides n, we write
d|n
• If d divides n, we also say
– n is a multiple of d
– d is a divisor of n
– d is a factor of n
• If d does not divide n, we write
d-n
• For example
– 5 - 36 because 36 6= 5 · c for any integer c
– 13 - 38 because 38 6= 13 · c for any integer c
210
Properties of the Divides Relation
• Divisibility has the following properties
• Reflexive: For all integers n
n|n
• Antisymmetric: For natural numbers n and d
(d | n
and n | d) =⇒ d = n
• Transitive: For all integers n, m, and d
((d | n) ∧ (n | m)) =⇒ d | m
Properties of the Divides Relation
• 1 divides every integer n:
1|n
• Every integer n divides 0:
n|0
• Zero only divides 0:
(0 | n) =⇒ n = 0
• Linearity:
((d | n) ∧ (d | m)) =⇒ d | (an + bm)
Common Divisors
• If d divides a and b, then d is called a common divisor of a and b
• 1 is a common divisor of every pair of integers a and b
• Every pair of integers a and b has a greatest common divisor which can be expressed as a linear combination of
a and b
gcd(a, b) = au + bv, for some integers u and v
Greatest Common Divisor
Definition 23 (Greatest Common Divisor). The greatest common divisor of integers a and b is the number d with the
properties
1. d ≥ 0 (d is non-negative)
2. d | a and d | b (d is a common divisor of a and b)
3. e | a and e | b implies e | d (every common divisor divides d)
Greatest Common Divisor
• The greatest common divisor of a and b is denoted
gcd(a, b)
• Here are some small examples
– gcd(24, 36) = 12 and 12 = 36(1) + 24(−1)
– gcd(17, 22) = 1 and 1 = 17(−9) + 22(7)
– gcd(27, 42) = 3 and 3 = 27(−3) + 42(2)
• How can the greatest common divisor be computed?
• How can the linear combination be found?
211
Greatest Common Divisor
• The greatest common divisor of a and b can be computed by factoring a and b, and collecting common factors
• For example,
24 = 23 · 3
36 = 22 · 32
and
• See that 22 = 4 and 3 are common factors of 24 and 36
• Therefore 4 · 3 = 12 is the greatest common divisor of 24 and 36
• This brute force algorithm can be very expensive
• Fortunately, there is an ancient efficient algorithm: Euclid’s algorithm
39
The Greatest Common Divisor and Euclid’s Algorithm
Epigraph
We have learnt that nothing is simple and rational except what we ourselves have invented; that
God thinks in terms neither of Euclid nor of Riemann; that science has “explained” nothing; that
the more we know the more fantastic the world becomes and the profounder the surrounding
darkness.
Aldous Huxley, (English Novelist and Critic, 1894-1963)
Fractured History
• Imagine how Egyptian’s might re-measured their fields after the annual flood of the Nile
• Using a measuring rod 1 cubit (1 c) long, lay out the rod along the field’s edge counting the number of cubits
• Chances are the rod will not evenly measure the length
• Suppose the remainder after the last full rod is an amount r
r
Fractured History
• The remainder r can be compared against the cubit
r0
1c
2r
• To find r does not evenly divide the cubit
• The new remainder r0 can be compared against r to find r0 measures r exactly
212
r
2r0
• We have
2r0 = r,
r + r0 = c,
3c + r = 1 field length
0
r = c − r = c − ( f − 3c) = 4c − f
3c + r = 4r + 3r0 = 5.5r = 11r0 = f
Fractured History
3 23 cubits
5 12 r
11r0
• This procedure for finding a common measure r0 is the basis of Euclid’s algorithm
• Imagine the confusion that occurred when people realized they could not exactly measure the diagonal of a
square this way
Euclid’s Algorithm
• Euclid’s algorithm computes the greatest common divisor of two integers
• Euclid’s algorithm repeatedly computes the remainder r when a is divided b
– If the remainder r = 0 the algorithm halts and returns b
– Otherwise a is assigned b’s value, b is assigned r’s value, and a new remainder is computed
Numerical Examples
• Find gcd(105, 33)
Dividend Divisor
105
33
33
6
6
3
Remainder
Scratch Work
105 mod 33 = 6 105 = 33 × 3 + 6
33 mod 6 = 3
33 = 6 × 5 + 3
6 mod 3 = 0
6 = 2×3+0
• Once a 0 remainder is found, the last divisor is the greatest common divisor
gcd(105, 33) = 3
A Numerical Example
• The steps can be reversed
Dividend Divisor
105
33
33
6
6
3
Remainder
Scratch Work
105 mod 33 = 6 105 = 33 × 3 + 6
33 mod 6 = 3
33 = 6 × 5 + 3
6 mod 3 = 0
6 = 2×3+0
213
• Starting from the scratch work in the next to last line, we find
3 = 33 − 6 × 5
Solve for the remainder 3
= 33 − (105 − 33 × 3) × 5
Substitute for remainder 6
= 33(16) + 105(−5)
Collect divisors and dividends
• This shows that the greatest common divisor 3 can be written as a linear combination of 33 and 105
Another Numerical Example
• Find gcd(83, 17)
Dividend
83
17
15
2
Divisor
Remainder
Scratch Work
17 83 mod 17 = 15 83 = 17 × 4 + 15
15 17 mod 15 = 2 17 = 15 × 1 + 2
2
15 mod 2 = 1
15 = 2 × 7 + 1
1
2 mod 1 = 0
2 = 1×2+0
• Once a 0 remainder is found, the last divisor is the greatest common divisor
gcd(83, 17) = 1
Another Numerical Example
• The steps can be reversed
Dividend
83
17
15
2
Divisor
Remainder
Scratch Work
17 83 mod 17 = 15 83 = 17 × 4 + 15
15 17 mod 15 = 2 17 = 15 × 1 + 2
2
15 mod 2 = 1
15 = 2 × 7 + 1
1
2 mod 1 = 0
2 = 1×2+0
• Starting from the scratch work in the next to last line, we find
1 = 15 − 2 × 7
= 15 − (17 − 15 × 1) × 7
= 15(8) + 17(−7)
= (83 − 17 × 4)(8) + 17(−7)
= 83(8) + 17(−39)
• This shows that the greatest common divisor 1 can be written as a linear combination of 83 and 17
A Third Numerical Example
• Find gcd(90, 12)
Dividend
90
12
Divisor Remainder
Scratch Work
12 90 % 12 = 6 90 = 12 × 7 + 6
6 12 % 6 = 0 12 = 6 × 2 + 0
• Once a 0 remainder is found, the last divisor is the greatest common divisor
gcd(90, 12) = 6
214
A Third Numerical Example
• The steps can be reversed
Dividend
90
12
Divisor Remainder
Scratch Work
12 90 % 12 = 6 90 = 12 × 7 + 6
6 12 % 6 = 0 12 = 6 × 2 + 0
• Starting from the next to last line, we find
6
= 90 − 12 × 7
• This shows that the greatest common divisor 6 can be written as a linear combination of 90 and 12
Programming Euclid’s Algorithm
• Euclid’s method can be implemented in a programming language
• Here’s pseudocode for Euclid’s algorithm to compute the greatest common divisor of non-negative integers a
and b
[1] Euclidunsigned int a, unsigned int b unsigned int r (b! = 0) r ← a % b a ← b b ← r return a
Programming Euclid’s Algorithm
• The recursive version of Euclid’s method is even shorter
[1] Euclidunsigned int a, unsigned int b b == 0 return a E UCLID(b, a mod b)
The Extended Euclid Algorithm
• An extension of Euclid’s algorithm computes d, u, and v where
d = gcd(a, b) = au + bv
[1] Extended-Euclidunsigned int a, unsigned int b b == 0 return (a, 1, 0) (d 0 , u0 , v0 ) ←E XTENDED -E UCLID(b, a mod
b) (d, u, v) ← (d 0 , v0 , u0 − ba/bcv0 ) return (d, u, v)
Congruence Relations
• Given a and m, s suppose we use the extended Euclid algorithm to find d, u, and v such that
d = gcd(a, m) = au + mv
• We can then say that au − d is a multiple of m
au − d = −mv
• Or au is congruent to d mod m
au ≡ d
(mod m)
• In particular, when 1 is the greatest common divisor, we have
au ≡ 1
• Or a and u are multiplicative inverses mod m
215
(mod m)
Linear Congruence Equations
• To solve the linear congruence equation
ax = b mod m
• Use the extended Euclid algorithm to find d, u, and v such that
d = gcd(a, m) = au + mv
• If d = 1, then ax = b mod m has exactly one solution
x = ub mod m
• If d - b, then ax = b mod m has no solutions
• If d | b, then ax = b mod m has d solutions
x = t, t +
m
m
m
, t + 2 , . . . , t + (d − 1)
d
d
d
where t is the unique solution to
a
b
m
x=
mod
d
d
d
Linear Congruence Equations
• Solve the linear congruence equation
3x = 4 mod 7
• Use the extended Euclid algorithm to find d, u, and v such that
d = gcd(3, 7) = 3u + 7v
• We find 1 = 3(−2) + 7(1)
• The multiplicative inverse of 3 is −2 = 5 mod 7
• Multiply the equation through by 5
x = 5 · 3x = 5 · 4 = 20 = 6 mod 7
Linear Congruence Equations
• Solve the linear congruence equation
2x = 7 mod 17
• Use the extended Euclid algorithm to find d, u, and v such that
d = gcd(2, 17) = 2u + 17v
• We find 1 = 2(−8) + 17(1)
• The multiplicative inverse of 2 is −8 = 9 mod 17
• Multiply the equation through by 9
x = 9 · 2x = 9 · 7 = 63 = 12 mod 17
216
Linear Congruence Equations
• Solve the linear congruence equation
19x = 6 mod 141
• Use the extended Euclid algorithm to find d, u, and v such that
d = gcd(19, 141) = 19u + 141v
• We find 1 = 2(−8) + 17
Dividend Divisor
Remainder
Scratch Work
141
19 141 % 19 = 8 141 = 19(7) + 8
19
8
19 % 8 = 3
19 = 8(2) + 3
8
3
8%3 = 2
8 = 3(2) + 2
3
2
3%2 = 1
3 = 2+1
Linear Congruence Equations
• Reversing the equations and substituting for the remainder yields
1 = 3−2
= 3 − (8 − 3(2))
= 8(−1) + 3(3)
= (8(−1) + (19 − 8(2))(3)
= (141 − 19(7))(−7) + 19(3)
= 8(−7) + 19(3)
= 141(−7) + 19(52)
• The multiplicative inverse of 19 is 52 mod 141
• Multiply the equation through by 52
x = 52 · 19x = 52 · 6 = 312 = 30 mod 141
40
Proof by Contradiction
Proof by Contraction
• A Boolean expression is a contradiction if it is never true
• A Boolean expression is a contradiction if it is always false
• Suppose you could prove the Boolean expression
(P =⇒ Q) ∧ (P =⇒ !Q)
is true
• That is, from the truth table
P
Q
0
0
1
1
0
1
0
1
[(P =⇒ Q) ∧ (P =⇒ !Q)]
1
1
1
1
1
1
0
0
1
1
0
0
• If (P =⇒ Q) ∧ (P =⇒ !Q) is true, then P is false
217
Proof by Contraction
• The Boolean expression
(P =⇒ Q) ∧ (P =⇒ !Q) ∧ P
is a contradiction
• One way to prove a hypothesis H is true is to
–
–
–
–
Name P =!H
Prove that (P =⇒ Q) ∧ (P =⇒ !Q) is true
Conclude that P =!H is false
That is, H is true
• This process is called a proof by contradiction
Classic Example:
√
2 is Irrational
• The proof, found in Euclid’s Elements, that
–
–
–
–
√
2 is irrational is an example of a contradiction proof
Prove H = “The square root of 2 is irrational” is true
Assume not H, that is, P = “The square root of 2 is rational”
√
Then 2 = a/b where a and b are integers and b 6= 0
Without loss of generality, we can assume a/b has been reduced
∗ That is, a and b have no common factors (other than 1)
∗ That is, gcd(a, b) = 1
∗ That is, a and b are relatively prime
√
Classic Example: 2 is Irrational
√
√
• If 2 = a/b, then 2b = a
√
• If 2b = a, then 2b2 = a2
• If 2b2 = a2 , then a2 is even
• If a2 is even, then a is even
• If a is even, then a = 2k for some integer k
• If a = 2k and 2b2 = a2 , then 2b2 = 4k2
• If 2b2 = 4k2 , then b2 = 2k2
• If b2 = 2k2 , then b is even
• We’ve shown a is even and b is even
• But a and b are relatively prime: They cannot both be even!
Classic Example: There Is No Largest Prime Number
• The proof, found in Euclid’s Elements, that there is no largest prime is a classic example of a contradiction proof
–
–
–
–
–
Prove H = “There is no largest prime number ” is true
Assume not H, that is, P = “There is a largest prime”
Call this largest prime p
Let p0 = 2, p1 = 3, . . . , pn−1 = p be all of the primes
Let q = p0 p1 · · · pn−1 + 1 be the product of all primes plus 1
218
Classic Example: There Is No Largest Prime Number
• Then q = p0 p1 · · · pn−1 + 1 is not prime because q > p = pn−1 and p is the largest prime
• But q is not composite because no prime p0 , p1 , . . . pn−1 divides q
• That q is not prime and q is not composite is a contradiction
• Therefore, the assumption that there is a largest prime is false
A Modern Classic: The Real Numbers are Uncountable
• Cantor showed that there are degrees of infinity
• In particular, that the cardinality of the natural numbers is less than the cardinality of the real numbers
• The cardinality of the natural numbers is called aleph naught, written ℵ0
• The cardinality of the real numbers is called aleph one, written ℵ1
• Cantor’s diagonalization argument proves that
ℵ0 < ℵ1
Finite Sets
• A set X is finite with cardinality n if there is a one-to-one function f mapping X onto the set Zn = {0, 1, . . . , n −
1}
– Here, Z0 is defined to be the empty set 0/
– And define λ () to be the empty function mapping 0/ onto 0/
• Common finite sets we use are: the bits B, the digits D, the hex numerals H, and the English alphabet E
Countable Sets
• The natural numbers
N = {0, 1, 2, . . .}
are not finite
• The natural numbers are said to be countable
• The cardinality of N is denoted ℵ0
• ℵ0 is a transfinite number
• Cantor showed that ℵ0 is the first transfinite number in that there is no set X with cardinality greater than every
natural number n, but less than ℵ0
(!∃ℵ)(∀n)(n < ℵ < ℵ0 )
Sets with Cardinality ℵ0
• Two sets have the same cardinality if there is a one-to-one function from one set onto the other
• The natural numbers N and the integers Z have the same cardinality
– Let f : N −→ Z be defined by
(
f (n) =
n
2
− n+1
2
if n is even
if n is odd
– This function shows the natural numbers can be put in a one-to-one correspondence with the integers
– This is true even though the natural numbers are a proper subset of the integers
219
Sets with Cardinality ℵ0
• Let’s construct a one-to-one function f (n) = (xn , yn ) that maps the natural numbers onto all the ordered pairs of
natural numbers
• After doing so, we can infer that the rational numbers Q are countable
• A good idea is plot the ordered pairs
(xn , yn )
and name them in diagonal order
Naming Ordered Pairs of Natural Numbers
6
(0, 3)
3
7
(0, 2)
(1, 2)
1
4
8
(0, 1)
(1, 1)
(2, 1)
0
2
5
9
(0, 0)
(1, 0)
(2, 0)
(3, 0)
• Up the y-axis, 0, 1, 3, 6, . . . are triangular numbers
• Along the x-axis, 0, 2, 5, 9, . . . are triangular numbers minus 1
• The sum of coordinates along a diagonal is a constant
Naming Ordered Pairs of Natural Numbers
• The numbers up the y-axis are the triangular numbers
tk+1 =
k(k + 1)
,
2
k = 0, 1, 2, . . .
• The numbers along the x-axis are one less than the next triangular number, that is, they are
tk+2 − 1 =
(k + 1)(k + 2)
k(k + 3)
−1 =
,
2
2
k = 0, 1, 2, . . .
• Along diagonal k the coordinates sum to k
diagonal 0
xn + yn = 0
for n = 0
diagonal 1
xn + yn = 1
for n = 1, 2
diagonal 2
xn + yn = 2
for n = 3, 4, 5
diagonal k
xn + yn = k
for n = tk+1 , . . . , tk+2 − 1
220
Naming Ordered Pairs of Natural Numbers
• Create a table enumerating
–
–
–
–
The natural numbers n = 0, 1, 2, . . .
The coordinate xn , n = 0, 1, 2, . . .
The coordinate yn , n = 0, 1, 2, . . .
Mark the rows where diagonals start
∗ For k = 0, 1, 2, 3, . . ., mark rows that start diagonals,
∗ That is, rows numbered
n = tk+1 = k(k + 1)/2 = 0, 1, 3, 6, . . .
Naming Ordered Pairs of Natural Numbers
• The start of a table of this data is
k
n
0 t1 = 0
1 t2 = 1
2
2 t3 = 3
4
5
3 t4 = 6
7
8
9
xn
0
0
1
0
1
2
0
1
2
3
yn
0
1
0
2
1
0
3
2
1
0
Naming Ordered Pairs of Natural Numbers
• When n varies from tk+1 to tk+2 − 1,
– The sum of coordinates is k
xn + yn = 0
for n = 0, k = 0, t1 = 0, t2 − 1 = 0
=1
for n = 1, 2, k = 1, t2 = 1, t3 − 1 = 2
=2
for n = 3, 4, 5, k = 2, t3 = 3, t4 − 1 = 5
=k
for n = tk+1 , . . . , tk+2 − 1
– The value of xn varies from 0 up to k
– The value of yn varies from k down to 0
Naming Ordered Pairs of Natural Numbers
• The function we seek is
f (n) = ( j, k − j)
where
n = tk+1 + j =
k(k + 1)
+j
2
for
j = 0, 1, . . . k,
and k = 0, 1, . . .
• As k ranges over the natural numbers and j varies from 0 to k, n goes from 0 to infinity in unit steps bunched in
diagonal groups from
k(k + 1)
tk+1 =
to
tk+1 + k = tk+2 − 1
2
221
Cantor’s Diagonalization Argument
• Cantor showed that there is no one-to-one function from the natural numbers onto the unit interval
[0, 1] = {x : 0 ≤ x ≤ 1}
• His proof is by contradiction
• Let P =”There is a one-to-one and onto function f : N −→ [0, 1]”
• That is,
– For every natural number n, f (n) is a real number between 0 and 1
– No two unequal natural number n0 6= n1 map to the same real number
– Every real number between 0 and 1 is the image of some natural number n
Diagonalization
• Every real number in [0, 1] can be represented as a binary string, for example,
– 0 is the never-ending string 0.000 . . .
– 1 is the never-ending string 0.111 . . .
– 0.5 is the never-ending string 0.1000 . . .
– 1/3 is the never-ending string 0.0101 . . .
– 2/3 is the never-ending string 0.1010 . . .
• Let 0∗ denote an infinite string of 0s
– So 0 can be written as 0.0∗
• Let 1∗ denote an infinite string of 1s, and so 1 = 0.1∗
• Similarly 1/3 = 0.0101 . . . can be written 0.(01)∗
• And 4/5 = 0.1100 . . . can be written 0.(1100)∗
Diagonalization
• Suppose
f (0) = 0.0∗ ,
f (1) = 0.1∗ ,
f (2) = 0.10∗ ,
f (3) = 0.(01)∗ ,
f (4) = 0.(10)∗ ,
f (5) = 0.(1100)∗
• Create a table that looks like this
0
1
2
3
4
5
..
.
0
0
0
0
0
0
0
..
.
1
0
1
1
0
1
1
..
.
2
0
1
0
1
0
1
..
.
222
3
0
1
0
0
1
0
..
.
4
0
1
0
1
0
0
..
.
5
0
1
0
0
1
1
..
.
···
···
···
···
···
···
···
..
.
Diagonalization
• The value in row k column j is bit j in the binary expansion of f (k)
• The assumption is every real number between 0 and 1 lies in some row of the table
• Consider the number formed by concatenating bits along the diagonal
d = b(0,0) .b(1,1) b(2,2) b(3,3 b(4,4) b(5,5) . . .
• From the example table
d = 0.10001 · · ·
Diagonalization
• Define the diagonal number d¯ by taking the one’s complement of (flipping) each bit after the binary point
• That is, b(k,k) is the bit in row k column k of the table then the bit in position k of d¯ is
1 − b(k,k)
• From the example table d = 0.10001 · · · , so
d¯ = 0.01110 · · ·
Diagonalization
• The diagonal number d¯ is precisely defined (assuming the function f exists)
• The diagonal number d¯ is a real number between 0 and 1
• Therefore d¯ lies in some row of the created table
• Suppose d¯ lies in row k > 0
– That is, f (k) = d¯
– The bit in row k column k is b(k, k) , say this bit is 1
– The bit position k of d¯ is 1 − b(k, k) , which will be is 0
• Therefore d¯ does lie in row k
• Conclude that the table is incomplete
• That is, no one-to-one function f from the natural numbers to the unit interval exists
• That is, the cardinality of the natural numbers is less than the cardinality of the real numbers
Diagonalization
• Gödel modified Cantor’s diagonalization to show that arithmetic is incomplete
– There are true statements about addition and multiplication on the integers than cannot be proven true from
even a countable, consistent set of axioms
• Turing modified Cantor’s diagonalization to show that there are unsolvable problems
– There are decision problems (questions with “yes” or “no” answers) whose answer cannot be computed in
even a countable number of steps
223
41
Sequence
Common Sequences
• Alice sequence ~A = h1, 1, 1, 1, . . . , 1, . . .i
~ = h0, 1, 2, 3, 4, . . . , n, . . .i
• Gauss sequence G
E
D
,
.
.
.
• Triangular sequence ~T = 0, 0, 1, 3, 6, . . . , n(n−1)
2
D
• Fibonacci sequence ~F = 0, 1, 1, 2, 3, . . . ,
E
φ n√
−φ̂ n
,...
5
• Zeno sequence ~Z = h1, 2, 4, 8, 16, . . . , 2n , . . .i
~ = h0, 1, 3, 7, 15, . . . , 2n − 1, . . .i
• Mersenne sequence M
• Generic sequence ~S = hs0 , s1 , s2 , s3 , s4 , . . . , sn , . . .i
Definition of a Sequence
• A sequence is an ordered list of objects
• In practice a sequence is always finite
~S = hs0 , s1 , s2 , s3 , s4 , . . . , sn−1 i
• In theory a sequence is almost always infinite
~S = hs0 , s1 , s2 , s3 , s4 , . . . , sn−1 , . . .i
• In practice the terms sk , k = 0, 1, 2, . . . can represent almost anything
• In theory the terms sk , k = 0, 1, 2, . . . will almost always be integers
• A sequence is a function from the natural numbers
N = {0, 1, 2, 3, . . .}
to the objects
hs0 , s1 , s2 , s3 , s4 , . . . , sn−1 , . . .i
Functional Notation for Sequences
• Let sk or s(k), k = 0, 1, . . ., denote term k in sequence ~S
• Often there are simple functions for computing terms sk
– Alice sequence ak = 1
– Gauss sequence gk = k
– Triangular sequence tk = k(k − 1)/2
√
√
√
– Fibonacci sequence fk = ((1 + 5)k − (1 − 5)k )/2k 5
– Zeno sequence zk = 2k
– Mersenne sequence mk = 2k − 1
224
Algorithmic Representation of Sequences
• Some sequences have terms that are not determined by simple functions
• But there may be simple algorithms for computing terms sk
– The prime sequence pk
– The digits of π
– The divisor sequence dk
• Some sequences can be described but their terms can not be computed by any algorithm
• Some sequences can not even be described
Recurrence Representation of Sequences
• Some sequences can be describe by a recurrence relation and initial conditions
• A recurrence relation defines sn in terms of previous terms sn−1 , sn−2 , . . .
• Initial conditions provide beginning values for the recurrence
√
• For example, the Newton recurrence for 2
sn =
sn−1
1
+
,
2
sn−1
Alice and Gauss Recurrence Relations
• Alice numbers
– Initial condition a0 = 1
– Recursion an = an−1 for n ≥ 1
– Solution an = 1
• Gauss numbers (natural numbers)
– Initial condition g0 = 0
– Recursion gn = gn−1 + 1 for n ≥ 1
– Solution gn = n
Triangular and Fibonacci Recurrence Relations
• Triangular numbers
– Initial condition t0 = 0
– Recursion tn = tn−1 + (n − 1) for n ≥ 1
– Solution tn = n(n − 1)/2
• Fibonacci numbers
– Initial conditions f0 = 0, f1 = 1
– Recursion fn = fn−1 + fn−2 for n ≥ 2
√
√
√
– Solution fn = ((1 + 5)n − (1 − 5)n )/2n 5
225
with s0 = 1
Pascal’s Triangle
• The Alice, Gauss, and triangular sequences form row 0, 1 and 2 of Pascal’s triangle
n=0
1
2
3
4
5
6
7
8
..
.
k=0
1
1
1
1
1
1
1
1
1
..
.
1
2
3
4
5
6
7
1
2
3
4
5
6
7
8
..
.
1
3
6
10
15
21
28
..
.
1
4
10
20
35
56
..
.
1
5
15
35
70
..
.
1
6
21
56
..
.
1
7
28
..
.
1
8 1
.. ..
. .
• The Fibonacci numbers hide in Pascal’s triangle too (can you find them?)
Zeno and Mersenne Recurrence Relations
• Zeno numbers
– Initial condition z0 = 1
– Recursion zn = 2zn−1 for n ≥ 1
– Solution zn = 2n
• Mersenne numbers
– Initial condition m0 = 0
– Recursion mn = 2mn−1 + 1 for n ≥ 1
– Solution mn = 2n − 1
42
Summations
Summation Notation
• The “sum of f (k) from k = 0 to n − 1” is denoted
n−1
∑ f (k)
k=0
which stands for the total
f (0) + f (1) + f (2) + · · · + f (n − 1)
Summation Notation
• Parsing summation notation
n−1
∑ f (k)
k =0
• The summation symbol capital sigma ∑
226
8
..
.
• The index of summation is k
• The lower limit of the sum 0
• The upper limit of the sum n − 1
• The terms being added f (k), k = 0, . . . , n − 1
• A sum is empty and equal to 0 when the upper limit is less than the lower limit
Partial Sums of Alice and Gauss Sequences
• Alice sums
“the sum of 1 from k = 0 to k = n − 1 is n”
n−1
∑ 1 = 1| + 1 +{z· · · + 1} = n
k=0
n terms
• Gauss sums
“the sum of k from k = 0 to n − 1 is n(n − 1)/2”
n−1
∑ k = 0 + 1 + 2 + 3 + · · · + (n − 1) =
k=0
n(n − 1)
2
Partial Sums of Triangular Numbers
• Triangular sums
“the sum of k(k − 1)/2 from k = 0 to n − 1 is n(n − 1)(n − 2)/6”
n−1
k(k − 1)
(n − 1)(n − 2)
= 0+0+1+···+
2
2
k=0
∑
=
n(n − 1)(n − 2)
6
Partial Sums of Triangular Numbers
n−1
k(k − 1) n(n − 1)(n − 2)
=
2
6
k=0
∑
• The triangular numbers are column 2 in Pascal’s triangle
• The triangular sum formula states that summing 2k from k = 0 to k = n − 1 totals
n
3
• To form a 3 element subset from an n element set
– For k = 0 to n − 1, construct all 2 element sets Xk that can be formed from {0, 1, . . . , k − 1}
– Insert n − 1 into set each set Xk creating a 3 element set
227
Partial Sums of the Fibonacci Sequence
• Fibonacci sums
“the sum of fk from k = 0 to k = n − 1 is fn+1 − 1”
n−1
∑ fk = 0 + 1 + 1 + · · · + fn−1 = fn+1 − 1
k=0
• If the formula is correct, then adding in the next term fn gives
"
#
n
∑
n−1
fk =
k=0
∑ fk
+ fn
k=0
= ( fn+1 − 1) + fn
= fn+2 − 1
Partial Sums of Zeno and Mersenne Sequences
• Zeno sums
“the sum of 2k from k = 0 to k = n − 1 is 2n − 1”
n−1
∑ 2k = 1 + 2 + 4 + · · · + 2n−1 = 2n − 1
k=0
• Mersenne sums
“the sum of 2k − 1 from k = 0 to k = n − 1 is 2n − (n + 1)”
n−1
∑ (2k − 1) = 0 + 1 + 3 + · · · + (2n−1 − 1) = 2n − (n + 1)
k=0
43
Induction
Mathematical Induction
• To prove “ f (n) = g(n)” is true for all natural numbers n, show
Basis: Show that f (0) = g(0)
Induction: Show that
• “If f (n) = g(n), then f (n + 1) = g(n + 1)” is true
Alice Sum Formula Induction Proof
• The Alice sum formula
n−1
∑1=n
k=0
Basis: For n = 0 the sum on the left is empty, and so equal to 0 And the right hand side n is 0 too
Induction: If
n−1
∑1=n
k=0
then
n
"
n−1
∑1= ∑1
k=0
k=0
228
#
+1 = n+1
Gauss Sum Formula Induction Proof
• The Gauss sum formula
n−1
∑ k = 0 + 1 + 2 + 3 + · · · + (n − 1) =
k=0
n(n − 1)
2
Basis: For n = 0 the sum on the left is empty, and so equal to 0 And the right hand side is 0(0 − 1)/2 = 0 too
Induction: If ∑n−1
k=0 k = n(n − 1)/2, then
"
n
n−1
#
∑k= ∑k
k=0
+ n = (n + 1)n/2
k=0
Zeno Sum Formula Induction Proof
• The Zeno sum formula
n−1
∑ 2k = 1 + 2 + 4 + · · · + 2n−1 = 2n − 1
k=0
Basis: For n = 0 the sum on the left is empty, and so equal to 0 And the right hand side n is 0 too
k
n
Induction: If ∑n−1
k=0 2 = 2 − 1, then
#
"
n
n−1
∑2
k
k
=
k=0
∑2
+ 2n = 2n+1 − 1
k=0
Zeno Recurrence Induction Proof
• The recurrence for terms in the Zeno sequence is
zn = 2zn−1 , n ≥ 1
and z0 = 1
• Prove that the formula zn = 2n solves the recurrence
Basis: For n = 0 the formula 2n equals 1 which matches the initial condition z0 = 1
Induction: If zn−1 = 2n−1 for some n ≥ 1, then
zn = 2zn−1 = 2(2n−1 ) = 2n
Triangular Recurrence Induction Proof
• The recurrence for terms in the triangular sequence is
tn = tn−1 + (n − 1), n ≥ 1
and t0 = 0
• Prove that the formula tn = n(n − 1)/2 solves the recurrence
Basis: For n = 0 the formula n(n − 1)/2 equals 0 which matches the initial condition t0 = 0
Induction: If tn−1 = (n − 1)(n − 2)/2 for some n ≥ 1, then
tn = tn−1 + (n − 1)
(n − 1)(n − 2)
+ (n − 1)
2
n(n − 1)
=
2
=
229
44
Sets
Set Operations
• The union of A and B is the set A ∪ B of elements x where x ∈ A or x ∈ B.
A ∪ B = {x : x ∈ A ∨ x ∈ B}
• The intersection of A and B is the set A ∩ B of elements x where x ∈ A and x ∈ B.
A ∩ B = {x : x ∈ A ∧ x ∈ B}
• The complement of A is the set A of elements x not in A
A = {x : x 6∈ A}
Subset of a Set
• A is a subset of B if x is in A implies x is in B
A⊆B
• The empty set 0/ is a subset of every set A
• Every set A is a subset of the universal set U
• Every set A is a subset of itself
• Every set A is a subset of A ∪ B
Counting Subsets
• The binomial coefficient
n
k
counts the number of k element subsets that can be formed from an n element set
• Binomial coefficients can be expressed using factorial notation
n(n − 1) · · · (n − k + 1)
n
n!
=
=
k!(n − k)!
k!
k
• Binomial coefficients satisfy the recurrence relation
n
n−1
n−1
=
+
k
k−1
k
with boundary conditions
n
n
=
=1
0
n
Counting Subsets
• To form a k element subset from an n element set
{0, 1, 2, . . . , (n − 1)}
– Form a k − 1 element set from {0, 1, 2, . . . , (n − 2)} in n−1
k−1 ways
– Then insert the element (n − 1) into each of these sets
– Or form a k element set from {0, 1, 2, . . . , (n − 2)} in n−1
ways
k
• The total
n−1
n−1
+
k−1
k
gives the number of ways to create a k element set from an n element set
230
Pascal’s Triangle
n=0
1
2
3
4
5
6
7
8
..
.
k=0
1
1
1
1
1
1
1
1
1
..
.
1
2
3
4
5
6
7
8
1
2
3
4
5
6
7
8
..
.
1
3
6
10
15
21
28
..
.
1
4
10
20
35
56
..
.
1
5
15
35
70
..
.
1
6
21
56
..
.
1
7
28
..
.
1
8 1
.. ..
. .
..
.
n
k
• Pascal’s triangle lists the binomial coefficients
Cardinality of a Set
• The cardinality |A| of a finite set A is the number of elements in A
• The integers Z have cardinality ℵ0 , a “transfinite” number
• The reals R have cardinality ℵ1 , another “transfinite” number
• There are “fewer” integers than reals: ℵ0 < ℵ1
Power Set of a Set
• The set of all subsets of A is called the power set of A
• The power set of A is the set
2A = {X : X ⊆ A}
• The cardinality of the power set 2A is 2n where n = |A|
Counting Sets
• One non-empty set A0 partitions the universe into two subsets
A0
A0
• Two non-empty sets A0 and A1 can partition the universe into four subsets
A 0 ∩ A1 ,
A0 ∩ A1
A 0 ∩ A1 ,
A0 ∩ A1
• Three non-empty sets A0 , A1 , and A2 can partition the universe into eight subsets
– Intersect each of the four sets above with A2
– Intersect each of the four sets above with A2
• n non-empty sets can partition the universe into 2n subsets
231
Counting Sets
• More complex subsets can be formed by taking or not taking the union of the 2n subsets that partition the
universe
• The maximum number of subsets that can be formed from n sets using union, intersection, and set complement
operations is
n
22
45
Logic
Propositions
• A proposition is a declarative statement: A statement that is either true or false
• A propositional variable P can take one of two logical values
P
= True
or
P
= False
• In computing these logical values are called Boolean values and denoted by numeric values
True = 1
and
Logical Operations
• There are three basic logical operations
– Not, denoted !P (or ¬P)
– And, denoted P ∧ Q (or P • Q)
– Or, denoted P ∨ Q (or P|Q)
More Logical Operations
• There several other useful logical operations
– If. . . then, denoted P =⇒ Q (or P ⊃ Q)
– If and only if, denoted P ≡ Q (or P ⇔ Q)
– Not And, denoted !(P ∧ Q) (or !P∨!Q)
– Not Or, denoted !(P ∨ Q) (or !P∧!Q)
– Exclusive Or, denoted P ⊕ Q (or ¬(P ≡ Q)
Boolean Expressions
• The constants 0 and 1 are Boolean expressions
• Boolean variables P, Q, R, . . . , are Boolean expressions
• If A and B are Boolean expressions, then
– (A) is a Boolean expression
– !A is a Boolean expression
– A ∧ B is a Boolean expression
– A ∨ B is a Boolean expression
232
False = 0
Arguments
• An argument is a sequence of propositions
• The last proposition is called the conclusion
• The leading propositions are called premises
p , p ,..., p , c
| 0 1 {z n−1} |{z}
premises
conclusion
Valid and Sound Arguments
• An argument is valid when the conclusion is True if its premises are True
• A valid argument can have false premises, in which case the conclusion can be either true or false
• As a general rule, arguments with true premises and a true conclusion are most useful
• A sound argument is a valid argument where all the premises are true
Rules of Inference
• A rule of inference is a logical proposition that is always true
• Rules of inference can be used as premises in sound arguments
• Rules of inference consist of
– Premises or assumptions
– A conclusion which follows from the premises
Rule of Inference: Modus Ponens
• Modus ponens
– “If P is true and if P implies Q is true, then Q is true”
[P ∧ (P =⇒ Q)] ∴ Q
• Modus ponens is used in direct proofs
– To show Q is true, show P is true and show P =⇒ Q
Rule of Inference: Modus Tollens
• Modus tollens
– “If Q is true and if P implies ¬Q is true, then P is false”
[¬Q ∧ (P =⇒ Q)] ∴ ¬P
• Modus tollens is used in indirect proofs
– To show ¬P is true, show ¬Q is true and show P =⇒ Q
233
Rule of Inference: Reductio ad Absurdum
• Reductio ad absurdum
– “If P implies Q and P implies ¬Q are true, then ¬P is true.”
[(P =⇒ Q) ∧ (P =⇒ ¬Q)] ∴ ¬P
• Reductio ad absurdum is used in proofs by contradiction
– To show P is true, show ¬P =⇒ (Q ∧ ¬Q) is true
Rule of Inference: Resolution
• Resolution
– If P ∨ Q is true and ¬P ∨ R is true, then Q ∨ R is true.
[(P ∨ Q) ∧ (¬P ∨ R)] ∴ Q ∨ R
• Resolution is used to automate reasoning, and is the basis for logical programming languages such as PROLOG
Truth Tables
• Truth tables enumerate all truth values of a Boolean expression
• Truth table for Not (negation)
!P
1
0
P
0
1
• Truth table for And (conjunction)
P
Q
P∧Q
0
0
1
1
0
1
0
1
0
0
0
1
P
Q
P∨Q
0
0
1
1
0
1
0
1
0
1
1
1
Truth Tables
• Truth table for Or (disjunction)
• Truth table for If. . . then (conditional)
P
Q
0
0
1
1
0
1
0
1
234
P
=⇒ Q
1
1
0
1
Truth Tables
• Truth table for If and Only If (equivalence)
P
Q
0
0
1
1
0
1
0
1
≡Q
1
0
0
1
P
Q
P⊕Q
0
0
1
1
0
1
0
1
0
1
1
0
P
• Truth table for Exclusive Or
Counting Truth Tables
• Let B be a Boolean expression in n variables
• That is, B(x0 , . . . , xn−1 ) is a function
– B maps Bn to B
– The domain the set of all n-tuples of zeros and ones
∗ Assigning a truth value (0 or 1) to each variable xk , k = 0, . . . , n − 1 creates a bit string of length n
∗ There are 2n bit strings of length n
– The cardinality of the domain is 2n
• There are 2n rows in the truth table for B
• There is one row for each assignment of truth values to the variables
Counting Truth Tables
• The values in the column labeled B define the output of a function for the given input
• Here is an n = 3 variable example
x0
0
0
0
0
1
1
1
1
x1
0
0
1
1
0
0
1
1
x2
0
1
0
1
0
1
0
1
• B maps a triple (x, y, z) to a bit 0 or 1
• The string of bits describing all outputs of b has length 2n
235
B
1
0
0
1
1
1
0
0
Counting Truth Tables
• For each row there are two possible output values: 0 or 1
• There are 2n rows in the truth table
n
• There are 22 different output strings of length 2n
n
• There are 22 Boolean expressions in n variables
Subjects and Predicates
• A singular proposition has the form
[subject term] is [predicate term]
• For example “A wiki is a web application”
• The predicate is often true for many subjects x
• A simple predicate is written W x, meaning x is a W
Quantification
• Simple predicates that are true for all subjects can be universally quantified
• If all x are W ’s write
(∀x)(W x)
• Simple predicates that are true for at least one subject can be existentially quantified
• If some x is a W write
(∃x)(W x)
46
Relations
Relation: Definition
• A relation r on set X is a subset of the Cartesian product X × X
r ⊆ X×X
• A relation r from X to Y is a subset of X × Y.
r ⊆ X×Y
• The set X is called the domain of the relation r
• The set Y is called the range of the relation r
• If x ∈ X and y ∈ Y are related by r, write xry
Common Relations
• Equals: a = b
• Less than or equal: a ≤ b
• Subset: A ⊆ B
• Divides: a | b
• Modulo m: a ≡ b (mod m)
236
Counting Relations
• Let n = |X|
2
• There are 2n = 2|X||X| relations on X
• Let m = |Y|
• There are 2nm = 2|X||Y| relations from X to Y
Reflexive Property
• A relation r on set X is reflexive if
xrx
for all x ∈ X
Symmetric Property
• A relation r on set X is symmetric if
xry =⇒ yrx,
for all x, y ∈ X
Antisymmetric Property
• A relation r on set X is antisymmetric if
(xry ∧ yrx) =⇒ x = y,
for all x, y ∈ X
Transitive Property
• A relation r on set X is transitive if
(xry ∧ yrz) =⇒ xrz,
for all x, y, z ∈ X
Graph Representation of Relations
• Relations can be represented as graphs where the nodes are elements in X ∪ Y and edges are lines or arrows
between nodes.
• The relation a | b on the set {0, 1, 2, 3, 4} is represented by the graph
1
3
2
4
0
237
Matrix Representation of Relations
• Relations can be represented as a matrix with rows labeled by elements in X and columns labeled by elements
in Y. Matrix entries in row a, column b are 1 if there is an edge from a to b and 0 otherwise.
• The mod 3 relation on {0, 1, 2, 3, 4, 5, 6} is represented by the matrix
0
1
2
3
4
5
6
0
1
0
0
1
0
0
1
1
0
1
0
0
1
0
0
2
0
0
1
0
0
1
0
3
1
0
0
1
0
0
1
4
0
1
0
0
1
0
0
5
0
0
1
0
0
1
0
6
1
0
0
1
0
0
1
Classification of “Equivalent” Objects
• An equivalence relation is reflexive, symmetric, and transitive
• Examples of equivalence relations
– Equality
– Congruence mod m
Equivalence Relations
• An equivalence relation partitions a sets into disjoint subsets
• The Stirling number of the second kind nk counts the number of partitions of an n element set into k subsets
• Stirling numbers of the second kind satisfy the recurrence
n
n−1
n−1
=
+ (n − 1)
k
k−1
k
with boundary conditions
For n > 0
n
= 0,
0
(∀n)
n
=1
n
Putting Objects in Order
• A partial order is a relation that is reflexive, antisymmetric, and transitive
• Examples of partial orders
– Less than or equal
– Subset
– Divides
238
47
Functions
Function: Definition
Definition 24 (Function). A function f from X to Y is
1. A relation such that for each x ∈ X there is a unique y ∈ Y such that
f (x) = y
2. A set of ordered pairs
f = {(x, y) : x ∈ X, y ∈ Y}
such that if (x, y0 ) ∈ f and (x, y1 ) ∈ f then y0 = y1
Function: Domain and Range
• The domain of f is the set X
• The range of f is the set Y
• A function f is said to be total when it is defined for all values of x in its domain
• Function f is total when
(∀x ∈ X)(∃y ∈ Y)((x, y) ∈ f )
or, written differently
(∀x ∈ X)(∃y ∈ Y)( f (x) = y)
• When f is not defined for all elements in its domain, it is called partial
Function: Onto and One-To-One
• A function f is onto when every element in its range is the image of some element in its domain
• Function f is onto when
(∀y ∈ Y)(∃x ∈ X)((x, y) ∈ f )
• Or, written using function notation
(∀y ∈ Y)(∃x ∈ X)( f (x) = y)
One-to-one Functions
• A function f is said to be one-to-one when different elements in its domain do not map to the same element in
its range
• Function f is one-to-one when
(∀x0 ∈ X)(∀x1 ∈ X)((x0 6= x1 ) =⇒ ( f (x0 ) 6= f (x1 )))
• Or, written in the contrapositive
(∀x0 ∈ X)(∀x1 ∈ X)(( f (x0 ) = f (x1 )) =⇒ (x0 = x1 ))
239
Example Functions
• The function y = f (x) = x from the reals numbers to the real numbers is
– Total, onto, and one-to-one
• The function y = f (x) = x2 − x − 1 from R to R is
– Total, not onto, and not one-to-one
• The function y = f (x) = sin x from R to R is
– Total, not onto, and not one-to-one
• The function y = f (x) = ex from R to R is
– Total, not onto, and one-to-one
Example Functions
• The function y = f (x) = lg x from R to R is
– Partial, onto, and one-to-one
• The function y = f (x) = tan x from R to R is
– Partial, onto, and not one-to-one
• The function y = f (x) = 1/(x − 1)2 from R to R is
– Partial, not onto, and not one-to-one
• The function y = f (x) = x(x − 1)(x − 2) from R to R is
– Total, onto, and not one-to-one
The Inverse of a Function
• A one-to-one function f : X −→ Y has an inverse
f −1 : Y −→ X
• If f is given by
f = {(x, y) : x ∈ X, y ∈ Y}
• Then the inverse function f −1 is
f −1 = {(y, x) : x ∈ X, y ∈ Y}
• A function that is not one-to-one has a formal inverse too: But is will be a relation, not a function
Polynomials
• Polynomials
y = p(x) = an−1 xn−1 + an−2 xn−2 + · · · a1 x + a0
– Evaluate at value x = c using Horner’s method
– There are more advanced evaluation ideas, beyond the scope of these notes
– Polynomials are used to approximate more complex functions
– A polynomial p(x) of degree n has n roots
– Newton’s method is useful for computing roots
240
Exponentials
• Exponentials
y = 2x ,
y = ex ,
y = 10x
• An exponential to a sum is the product of exponentials
bu+v = bu bv
• An exponential to a power is the exponential to the product of powers
(bu )v = buv
• An exponential of a negative is the reciprocal of the exponential
b−u = 1/bu
• A product of exponentials is the product to the power
au bu = (ab)u
Logarithms
• Logarithms
y = lg x,
y = ln x,
y = log x
• A log of a product is the sum of logs
lg(xy) = lg(x) + lg(y)
• A log of a power is the power times the log
lg(xy ) = y lg(x)
• A log of a reciprocal is the negative of the log
lg(1/x) = − lg(x)
• A products of logs is a log
loga (x) logb (a) = logb (x)
Logarithmic Arithmetic
• To multiply x by y
– Compute lg(x) and lg(y)
– Add lg(x) + lg(y)
– Compute 2lg(x)+lg(y)
• To divide x by y
– Compute lg(x) and lg(y)
– Subtract lg(x) − lg(y)
– Compute 2lg(x)−lg(y)
• To extract a root x1/r
– Compute lg(x)
– Multiply lg(x) by 1/r
– Compute 2lg(x)/r
241
Integer Functions
• Floor bxc
b−πc = −4
• Ceiling dxe
d−πe = −3
• Fractional part {x}
{−π} = −π − (−4) ≈ 0.8584
• Round e x
e −π = −3
Permutations
• Permutations
– A one-to-one function from a set onto itself
– There are n! permutations on a set with n elements
– Cyclic notation [02][13][56][47] describes the permutation
(2, 3, 0, 1, 7, 6, 5, 4)
of the set {0, 1, 2, 3, 4, 5, 6, 7}
– The Stirling number of the first kind
n
k
counts the number of permutations of n objects with k cycles
– Stirling numbers of the first kind satisfy the recurrence
n
n−1
n−1
=
+k
k
k−1
k
with boundary conditions
For n > 0
n
= 0,
0
(∀n)
n
=1
n
The Set of All Functions
• The set of all functions from X to Y is denoted YX
YX = { f : f : X −→ Y}
• There are |Y||X| total functions from X to Y
• Let n = |X| and m = |Y | with n ≤ m
• The subset of all one-to-one functions has cardinality
n
m!
=
m!
m(m − 1) · · · (m − n + 1) =
(m − n)!
m
48
Positional Numbers
Name Spaces
• A name space is a collection of objects where each has a unique name
• Unsigned Integer
– Finite strings over alphabet D = {0, 1, 2, . . . 9}
– Finite strings over alphabet B = {0, 1}
242
Enumerating Name Spaces
Theorem 25. Let alphabet Σ have m characters. There are mn strings over Σ of length n
Number System
• There are 10n unsigned integers between 0 and 10n − 1
• There are 2n unsigned integers between 0 and 2n − 1
• There are 16n unsigned integers between 0 and 16n − 1
• Whether the alphabet is digits, bits, or hexadecimal numerals
• The meaning of a numeral string depends on context (state)
Positional Number Systems
• Numbers written in polynomial form are called positional
• When necessary, a notation such as
(2.718)10
can be used to denote the base of the number is 10
• There a simple algorithms for arithmetic on positional numbers
• Computers use binary numbers to simplify the circuitry and logic gates
• Hexadecimal numbers are used to expose the binary numbers to human readers
• There are simple algorithms for converting between number systems
Horner’s Method
• Use Horner’s method to convert a number in some base to decimal
• Let x = (1001 1010)2 be a binary number
a = (1100 1101)2 = 27 + 26 + 23 + 22 + 1
• Convert (1001 1010)2 to decimal
1
1
0 0 1 1 0 1
2 4 8 18 38 76
2 4 9 19 38 77
0
154
154
Horner’s Method
• Horner’s method could be used to convert decimal to some other base
• But it requires arithmetic in the target base
• And we’ve not been wired for arithmetic in any base other than 10
243
Horner’s Method
• A conversion from decimal to binary by Horner’s method would have this type of arithmetic
a = (53)10
= 5 · 101 + 3 · 100
= (101)2 101 + (011)2
= (101)2 (1010)2 + (011)2
101
101
011
110010
110101
Division and Remainders
• When you divide a natural number
– By 2 the remainder is either 0 or 1
– By 3 the remainder is either 0, 1, or 2
– By 8 the remainder is either 0, 1, . . . , or 7
– By 16 the remainder is either 0, 1, . . . , or 15
• Successive remainders can be strung together to represent a decimal value in a given base
Decimal to Binary Conversion
• Convert the decimal number 74 to binary
74 % 2 = 0
74 ÷ 2 = 37
37 % 2 = 1
37 ÷ 2 = 18
18 % 2 = 0
18 ÷ 2 = 9
9%2 = 1
4%2 = 0
4÷2 = 2
2%2 = 0
2÷2 = 1
1%2 = 1
1÷2 = 0
9÷2 = 4
• Therefore
74 = (100 1010)2
Hexadecimal Numbers
• Used in low-level machine programming
• Each hex numeral is 4 bits: Zero = 0000 to Fifteen = 1111
• Two hex numerals are a byte
(CAFE DB)16 = (1100 1010 1111 1110 1101 1011)2
244
Counting Wheels: Unsigned Integers
• Consider the radix 16 counting wheel for unsigned integers
0
F
E
D
1111
1110
1
0000
2
0001
0010
1101
0011
C 1100
0100
1011
B
0101
1010
1001
A
1000
9
3
4
5
0110
0111
6
7
8
• Laying out the wheel on a number line yields
0
1
2
3
4
5
7
6
8
9
10 11 12 13 14 15
A B C D E F
Representation of Signed Integers
• There are several notational systems to represent signed integers
–
–
–
–
Binary two’s complement, Decimal tens’s complement, . . .
Biased notation
Binary one’s complement, Decimal nine’s complement, . . .
Sign-magnitude: ±m
• Let m ↔ x denote that integer m is represented by the string x in some notational system
• m can be positive, negative, or zero
• Let −x be the string representing −m, the negative of m
• Let n denote the length of x or −x
Counting Wheels: Two’s Complement
• Consider the radix 16 counting wheel for two’s complement
0
−1
−2
1111
1110
−3
0000
1
2
0001
0010
1101
0011
−4 1100
0100
1011
−5
0101
1010
−6 1001
−7
1000
−8
3
4
5
0110
0111
6
7
• The strings 8 through 15 are shifted by −16:
0
1
2
3
4
5
6
7
8 9 10 11 12 13 14 15
−8 −7 −6 −5 −4 −3 −2 −1
• In each case
x + (−x) = 16
245
Two’s Complement 4-Bit Example
0
1
2
3
4
5
6
7
8 9 10 11 12 13 14 15
−8 −7 −6 −5 −4 −3 −2 −1
• For example
x = 0000 = 0 ↔ 0
0 represents 0
x = 0111 = 7 ↔ 7
7 represents 7
x = 1000 = 8 ↔ −8
8 represents −8
x = 1001 = 9 ↔ −7
9 represents −7
x = 1111 = 15 ↔ −1
15 represents −1
Two’s Complement 4-Bit Example
0
1
2
3
4
5
6
7
8 9 10 11 12 13 14 15
−8 −7 −6 −5 −4 −3 −2 −1
• Using 4 bits
– Natural numbers m in the range 0 to 7 = 23 − 1
∗ Have a 0 as their leftmost bit
∗ Represent their natural values in two’s complement notation
– Natural numbers (−m) in the range 23 to 24 − 1
∗ Have a 1 as their leftmost bit
∗ Represent the negative value
(−m) − 16
– Integer m and its negative (−m) are related by the equation
m + (−m) = 16 = 24
Binary Two’s Complement
• Two’s complement notation is determined by the equation
x + (−x) = 2n
• For example
– x = 0100 1100 represents 76
– −x = 1011 0100 represents −76
– Adding the two’s complement binaries gives
0100 1100 + 1011 0100 = 1 0000 0000 = 28 ↔ 76 + (−76)
Decimal Ten’s Complement
• Ten’s complement notation is determined by the equation
x + (−x) = 10n
• For example
– x = 4321 represents 4321
– −x = 5679 represents −4321
– Adding the ten’s complement decimals gives
4321 + 5679 = 1 0000 = 104 ↔ 4321 + (−4321)
246
Binary Two’s Complement Notation
• Two’s complement is the notation used in computers to represent signed integers
• Using n bits, two’s complement represents numbers in the range −2n−1 to 2n−1 − 1
• Positive integers start with a 0 sign bit
• Positive integers are evaluated as unsigned integers
• Negative integers start with a 1 sign bit
• Positive integers are evaluated as unsigned integers
Binary Two’s Complement Notation
• Write the decimal integer 36 in two’s complement notation
36 % 2 = 0
36 ÷ 2
= 18
18 % 2 = 0
18 ÷ 2
=9
9%2 = 1
9÷2
=4
4%2 = 0
4÷2
=2
2%2 = 0
2÷2
=1
1%2 = 1
1÷2
=0
• Therefore (36)10 = (0100100)2 in two’s complement
Binary Two’s Complement Notation
• Write the decimal integer −36 in two’s complement notation
• We know positive (36)10 = (010 0100)2 in two’s complement
• To express (−36)10 write the bits from right-to-left up to and including the first 1
100
• Flip the remaining bits
(−36)10 = (101 1100)2
Binary Two’s Complement Notation
• The leading (leftmost) bit of a positive two’s complement binary integer is 0
• Each of these two’s complement integers is positive
0101,
00 0100,
0010 1010
• The leading (leftmost) bit of a negative two’s complement binary integer is 1
• Each of these two’s complement integers is negative
1101,
10 0100,
247
1010 1010
Negate a Two’s Complement Integer
• To negate a two’s complement integer write the bits from right-to-left up to and including the first 1, flip the
remaining bits
• The negative of 0101 is 1011
• The negative of 00 0100 is 11 1100
• The negative of 1010 1010 is 0101 0110
• The negative of 100 111 is 011 001
Convert a Two’s Complement to Decimal
• To convert a two’s complement integer x to decimal
– Apply Horner’s algorithm
– If x is negative, subtract 2n where n = |x|
• Convert x = 1011 to decimal
1
1
0 1
2 4
2 5
1
10
11
• Since x is negative, its decimal value is
m = 11 − 24 = 11 − 16 = −5
Convert a Two’s Complement to Decimal
• Convert x = 0110 to decimal
0
0
1 1
0 2
1 3
0
6
6
• Since x is positive, its decimal value is m = 6
Convert a Two’s Complement to Decimal
• Convert x = 1110 0110 to decimal
1
1 1 0 0 1
1
2 6 14 28 56 114
1 3 7 14 28 57 115
0
230
230
• Since x is negative, its decimal value is
m = 230 − 28 = 230 − 256 = −26
Sign Extension Rule
• A sign extension rule is used to fill chunks of memory (a nibble, byte, or word)
• The sign extension rule
Extend the sign bit leftward to fill the chunk
• For example
–
–
–
–
Extend −2 = 10 to a nibble: 1110
Extend 31 = 01 1111 to a byte: 31 = 0001 1111
Extend −1 = 11 to a byte: −1 = 1111 1111
Extend −31 = 100 0001 to a byte: −31 = 1100 00001
248
Counting Wheels: Biased Notation
• Consider the radix 16 counting wheel for bias 8 notation
1
−2
1111
1110
−3
−8
0000
−7
−6
0001
0010
1101
0011
4 1100
5
−5
−4
0100
1011
0101
1010
1001
6
1000
7
−3
0110
0111
−2
−1
0
• All numbers are translated
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
−8 −7 −6 −5 −4 −3 −2 −1 −0 −1 2 3 4 5 6 7
(−x) = x − b
Biased Representation
• Biased notation is determined by the equation
or (−x) = x − b
x + (−x) = 2b
• The parameter b is called the bias
• For example
–
–
–
–
Let the bias be b = 128
x = 112 represents 112 − 128 = −16
−x = 144 represents 144 − 128 = 16
Adding the biased decimals gives
112 + 144 = 256 = 28 ↔ 16 + (−16)
Counting Wheels: Sign-Magnitude
• Consider the radix 16 counting wheel for sign-magnitude
0
−7
−6
1111
1110
−5
0000
1
2
0001
0010
1101
0011
−4 1100
0100
0101
1011
−3
1010
−2 1001
−1
1000
−0
3
4
5
0110
0111
6
7
• The numbers 8 through 15 are reflected and translated
0
1
2
3
4
5
6
7
8 9 10 11 12 13 14 15
−0 −1 −2 −3 −4 −5 −6 −7
• In each case :
(−x) = 8 + x
249
Sign-Magnitude 4-Bit Example
0
1
2
3
4
5
7
6
8 9 10 11 12 13 14 15
−0 −1 −2 −3 −4 −5 −6 −7
• The representation scheme is
x = 0000 = 0 ↔ 0
0 represents 0
x = 0111 = 7 ↔ 7
7 represents 7
x = 1000 = 8 ↔ −0
8 represents −0
x = 1001 = 9 ↔ −1
9 represents −1
x = 1111 = 15 ↔ −7
15 represents −7
Sign-Magnitude Representation
• Sign-magnitude: ±k
• Binary sign-magnitude notation is determined by the equation
x + (−x) = 2n−1 + 2x
(−x) = 2n−1 + x
or
• For example
– x = 01001100 represents +76
– −x = 11001100 = 27 + 76 = 204 represents −76
– Adding the signed binaries gives
01001100 + 11001100 = 1 0001 1000
– As an unsigned integer (1 0001 1000)2 = 256 + 16 + 8 = 280
– And 280 = 128 + 152 = 27 + 2(76)
Counting Wheels: One’s Complement
• Consider the radix 16 counting wheel for one’s complement
0
−0
−1
1111
1110
−2
0000
1
2
0001
0010
1101
0011
−3 1100
0100
1011
−4
0101
1010
−5 1001
−6
1000
−7
3
4
5
0110
0111
6
7
• The numbers 8 through 15 are translated
0
1
2
3
4
5
6
7
8 9 10 11 12 13 14 15
−7 −6 −5 −4 −3 −2 −1 −0
(−x) = 15 − x
250
Binary One’s Complement
• Let x be an n-bit string representing an integer
• One’s complement notation is determined by the equation
x + (−x) = 2n − 1
• For example
– x = 0100 1100 represents 76
– −x = 1011 0011 represents −76
– Adding the one’s complement binaries gives
0100 1100 + 1011 0011 = 11111111 = 28 − 1 ↔ 76 + (−76)
Decimal Nine’s Complement
• Let x be an n-digit string representing an integer
• Nine’s complement notation is determined by the equation
x + (−x) = 10n − 1
• For example
– x = 4321 represents 4321,
– −x = 5678 represents −4321
– Adding the nine’s complement decimals gives
4321 + 5678 = 9999 = 104 − 1 ↔ 4321 + (−4321)
49
Modular Numbers
50
Proofs by Contradiction
51
Discrete Mathematics Applications
Name Spaces
• A name space is a collection of objects where each has a unique name
• Names are usually assigned by rules
– The Internet Assigned Number Authority (IANA) assigns 2 letter country codes, e.g., us, fr, iq, . . . from
an alphabet of 26 letters
– The New York and American Stock Exchanges use ticker symbols that are 1, 2, or 3 letters long
– NASDAQ uses ticker symbols that are 4 letters long
– North American telephone numbers have the format NXX-NXX-XXXX, where N is a digit in the range
2–9 and X is any digit 0–9
251
Alphabets
• An alphabet is a finite set of letters
– English letters E = {a, b, . . . , z}
– Greek letters G = {α, β , . . . , ω}
– The digits D = {0, 1, . . . , 9}
– The bits B = {0, 1}
• Let Σ denote a generic alphabet with letters
{g0 , g1 , . . . gm−1 }
Strings
• A string s over alphabet Σ is either
– The empty string λ
– A letter g ∈ Σ
– Or the concatenation gs of a letter with a string
Enumerating Name Spaces
Theorem 26. Let alphabet Σ have cardinality m. There are mn strings over Σ of length n
Example Enumerations
• The total namespace for IANA two letter country codes is
26 × 26 = 676
– There are about 250 country codes
– The fill factor is about 0.37
• The total namespace for NYSE/AMEX is
26 + 262 + 263 = 18, 278
– There are about 4000 active stock symbols
– The fill factor is about 0.22
Rootfinding: Newton’s Method
• To compute a value x such that f (x) = 0
– Pick an initial guess x0 as the solution
– Construct the sequence
xn = xn−1 −
252
f (xn−1 )
f 0 (xn−1 )
Compound Interest and Annuities
• The future value F of an investment with present value P is
F = P(1 + i)n
where i is the periodic interest rate and n is the number of periods
• An annuity is where payments are made each period
• The variables that determine the value of an annuity are
– r = the annual interest rate
– m = the number of payments per year
– i = r/m the periodic interest rate
– n = the number of payments
– P = the principal
• The value of an annunity is
n−1
F=
∑ P(1 + i)k = P
k=0
(1 + i)n − 1
i
Counting Tic-Tac-Toe Boards
• A completely filled 3 × 3 tic-tac-toe board will have five ×’s and four 0’s
• There are 95 = 9!/(5!4!) = 126 different tic-tac-toe board
• However, some of these boards will be congruent
• Two boards are congruent if one can be obtained from another by rotation or reflection
• Congruence on tic-tac-toe boards is an equivalence relation
Counting Tic-Tac-Toe Boards
• Permutations can be used to describe congruent boards
• Consider naming the squares in a tic-tac-toe board
1
4
7
2
5
8
3
6
9
• A quarter-turn clockwise produces the congruent board
7
8
9
4
5
6
1
2
3
• This change in boards can be expressed as a permutation in cyclic notation
[1397][2684][5]
253
51.1
RSA Public Key Cryptosystems
RSA Public Key Cryptosystems
• Alice and Bob want to share a secret message M
• Both have public and secret keys
– Alice public and secret keys are functions, named PA (M) and SA (M)
– Bob’s functions are named PB (M) and SB (M)
• Bob uses Alice’s public key to encrypt message M
C = PA (M)
encrypted message
• The encrypted message C is called the cypertext
• Alice uses her secret key to decrypt cypertext C
M = SA (C) = SA (PA (M))
Implementation of the Key Functions
• Select two large primes p and q, where p 6= q
– The size of these primes might be 512 bits or more
∗ 512 bits ≈ lg p = log p/ log 2 ≈
∗ That is, p about
3
10 512
10
3
log p digits
≈ 150 digits long
• Set a modulus m to the product of p and q, that is, m = pq
• Select a small odd integer e relatively prime to (p − 1)(q − 1)
Implementation of the Key Functions
• Use Euclid’s extended algorithm to compute d, the multiplicative inverse of e mod m
ed = 1
(mod m)
• Publish as the public key function
P(M) = M e
(mod m)
S(M) = Cd
(mod m)
• Keep as the secret key function
An RSA Example
• Let p = 11 and q = 13, so that n = pq = 11 · 13 = 143
• Let e = 7 and note that gcd(7, (p − 1)(q − 1)) = gcd(7, 120) = 1
• Consider a message: 2, 6, 1, 8
• Encrypt each number
c2 = 27 mod 143 = 128
c6 = 67 mod 143 = 85
c1 = 17 mod 143 = 1
c8 = 87 mod 143 = 57
• Send 128, 85, 1, 57 as the cyphertext
254
An RSA Example
• We need d to decrypt the cyphertext 128, 85, 1, 57
• d is the multiplicative inverse of 7 mod 120
• d can be computed by the extended Euclid algorithm
120 % 7 = 1,
1 = 120 + 7(−17)
• So that d = −17 = 103 mod 120
• The decrypted message is
m128 = 128103 mod 143 = 2
m85 = 85103 mod 143 = 6
m1 = 1103 mod 143 = 1
m57 = 57103 mod 143 = 8
255