Download A+B

Chapter 3
3.1 Algorithms
3.2 The Growth of Functions
3.3 Complexity of Algorithms
3.4 The Integers and Division
3.5 Primes and Greatest Common Divisors
3.6 Integers and Algorithms
3.7 Applications of Number Theory
3.8 Matrices
1
Chapter 3
3.1 Algorithms
– Searching Algorithms
– Greedy Algorithms
– The Halting Problem
2
Algorithm
• Definition 1: An algorithm is a finite set of precise
instructions for performing a computation or for
solving a problem.
• Example 1: Describe an algorithm for finding the
maximum (largest) value in a finite sequence of
integers.
3
• We perform the following steps
1. Set the temporary maximum equal to the first
integer in the sequence. (the temporary maximum
will be the largest integer examined at any stage of
the procedure.)
2. Compare the next integer in the sequence to the
temporary maximum, and if it is larger than the
temporary maximum, set the temporary maximum
equal to this integer.
3. Repeat the previous step if there are more integers
in the sequence.
4. Stop when there are no integers left in the
sequence. The temporary maximum at this point is
the largest integer in the sequence.
4
Pseudocode
• Pseudocode provides an intermediate step between an
English language description of an algorithm and an
implementation of this algorithm in a programming
language.
• Algorithm 1: Finding the maximum element in a finite
sequence.
procedure max(a1, a2, . . . ,an: integers)
max := a1
for i: =2 to n
if max < ai then max := ai
{max is the largest element}
5
Property of Algorithm
Input.
Output.
Definiteness. The steps of an algorithm must be
defined precisely.
Correctness.
Finiteness.
Effectiveness.
Generality. The procedure should be applicable
for all problems of the desired form, not just for
a particular set of input values.
6
Searching Algorithms
Search Problem: Locating an element in an
(ordered) list.
• Linear search
• Binary search (ordered list)
7
The linear search
• Algorithm 2 : the linear search algorithm
procedure linear search (x: integer, a1, a2, …,an: distinct
integers)
i :=1;
while ( i ≤n and x ≠ ai)
i := i + 1
If i ≤ n then location := i
Else location := 0
{location is the subscript of the term that equals x , or is
0 if x is not found}
8
The binary search
• Algorithm 3: the binary search algorithm
Procedure binary search (x: integer, a1, a2, …,an: increasing integers)
i :=1 { i is left endpoint of search interval}
j :=n { j is right endpoint of search interval}
While i < j
begin
m := (i+j)/2
if x > am then i := m+1
else j := m
Example 3:
end
to search for 19 in the list
If x = ai then location := i
1 2 3 5 6 7 8 10 12 13 15 16 18 19 20 22
else location :=0
{location is the subscript of the term equal to x, or 0 if x is not
found}
9
Sorting
• Sort:
– Sorting is putting elements into a list in which the
elements are in increasing order.
• E.g.
1) 7,2,1,4,5,9 -> 1,2,4,5,7,9
2) d,h,c,a,f -> a,c,d,f,h.
•
•
Bubble sort
Insertion sort
10
Bubble Sort
• ALGORITHM 4: The Bubble Sort
procedure bubble sort (a1, a2, …,an: real numbers with n ≥2)
for i := 1 to n-1
for j := 1 to n- i
if aj > aj+1 then interchange aj and aj+1
{a1, a2, …,an is in increasing order}
• Example 4: Use the sort to put 3, 2, 4, 1, 5 into
increasing order.
11
Bubble Sort
Insertion Sort
• Algorithm 5: The Insertion Sort
procedure insertion sort (a1, a2, …,an: real numbers with n ≥2)
for j := 2 to n
begin
i := 1
while aj > ai
Example 5:
i := i + 1
Use the insertion sort to put the
elements of the list 3, 2, 4, 1, 5
m := aj
into increasing order.
for k :=0 to j-i-1
aj-k := a j-k-1
ai := m
end {a1, a2, …,an are sorted}
13
Greedy Algorithm
• Optimization Problem: find the best solution.
• Algorithms that make what seems to be the
best choice at each step are called greedy
algorithms.
14
• Example 6: Consider the problem of making n cents
change with quarters, dimes, nickels, and pennies, and
using the least total number of coins.
• Algorithm 6: Greedy Change-Marking Algorithm
procedure change (c1, c2, …, cr: values of denominations of
coins, where c1 > c2 > … > cr ; n: a positive integer)
for i := 1 to r
while n ≥ ci
begin
add a coin with value ci to the change
n := n – ci
end
15
The Halting Problem
• There is a problem that cannot be solved using any
procedure.
• That is, there are unsolvable problems.
• Halting Problem
FIGURE 2 Showing that the Halting Problem is Unsolvable.
16
Chapter 3
3.1 Algorithms
3.2 The Growth of Functions
3.3 Complexity of Algorithms
3.4 The Integers and Division
3.5 Primes and Greatest Common Divisors
3.6 Integers and Algorithms
3.7 Applications of Number Theory
3.8 Matrices
17
Chapter 3
•
–
–
–
–
3.2 The Growth of Functions
Big-O Notation
Some Important Big-O Results
The Growth of Combinations of Functions
Big-Omega and Big-Theta Nation
18
The Growth of Functions
We quantify the concept that g grows at least as fast as f.
What really matters in comparing the complexity of
algorithms?
• We only care about the behavior for large problems.
• Even bad algorithms can be used to solve small
problems.
• Ignore implementation details such as loop counter
incrementation, etc. we can straight-line any loop.
19
Big-O Notation
• Definition 1: let f and g functions from the set of integers or
the set of real numbers to the set of real number. We say
that f(x) is O(g(x)) if there are constants C and k such that
|f(x)| ≤ C |g(x)| whenever x > k.
• This is read as “ f(x) is big-oh of g(x) ”.
• The constants C and k in the definition of big-O notation are
called witnesses to the relationship f(x) is O(g(x)).
• Note:
– Choose k
– Choose C ; it may depend on your choice of k
– Once you choose k and C, you must prove the truth of the
implication (often by induction).
• Example 1: show that f(x)= x2+ 2x + 1 is O(x2)
20
Big-O Notation
FIGURE 1 The Function x2 + 2x + 1 is O(x2).
21
Big-O Notation
FIGURE 2 The Function f(x) is O(g(x)).
22
Big-O Notation
• Example 2: show that 7x2 is O( x3 ).
• Example 4: Is it also true that x3 is O(7x2)?
• Example 3: show that n2 is not O(n).
23
Little-O Notation
• An alternative for those with a calculus background:
f ( n)
0
g ( n)
• Definition: if lim
then
f
is
o(g),
n 
called little-o of g.
24
• Theorem: if f is o(g) then f is O(g).
• Proof: by definition of limit as n goes to infinity,
f(n)/g(n) gets arbitrarily small.
That is for any ε >0 , there must be n integer N such
that when n > N, | f(n)/g(n) | < ε.
Hence, choose C = ε and k= N . Q.E.D.
It is usually easier to prove f is o(g)
• Using the theory of limits
• Using L’Hospital’s rule
• Using the properties of logarithms
etc
25
• Example : 3n + 5 is O(n2).
3n  5
 0 using
• Proof: it’s easy to show lim
2
n
n 
the theory of limits.
Hence, 3n+5 is o(n2) and so it is O(n2).
Q.E.D.
26
Some Important Big-O Results
f ( x)  an x n  an 1 x n 1    a1 x  a0
• Theorem 1: let
where a0, a1, . . .,an-1 , an are real numbers
then f(x) is O(xn) .
• Example 5: how can big-O notation be used to
estimate the sum of the first n positive
integers?
27
Some Important Big-O Results
• Example 6: give big-O estimates for the
factorial function and the logarithm of the
factorial function, where the factorial function
f(n) =n! is defined by
n! = 1* 2 * 3 * . . .*n
Whenever n is a positive integer, and 0!=1.
28
Some Important Big-O Results
• Example 7: In Section 4.1 ,we will show that n <2n
whenever n is a positive integer.
Show that this inequality implies that n is O(2n) ,
and use this inequality to show that log n is O(n).
29
The Growth of Combinations of Functions
1
logn
n
n log n
n2
2n
n!
FIGURE 3 A Display of the Growth of Functions Commonly Used in Big-O Estimates.
30
Important Complexity Classes
O(1)  O(log n)  O(n)  O(n log n)
 O(n 2 )  O(n j )  O(c n )  O(n!)
Where j > 2 and c> 1.
• Example :Find the complexity class of the function
(nn!3n 2  3n100 )( n n  n2n )
• Solution: this means to simplify the expression.
Throw out stuff which you know doesn’t grow as fast.
We are using the property that if f is O(g) then f + g is O(g).
31
Important Complexity Classes
if a flop takes a nanosecond, how big can a
problem be solved (the value of n ) in
a minute?
a day?
a year?
For the complexity class O(n n! nn)
32
Important Complexity Classes
a minute= 60*109=
6*1010 flops
a day=
24*60*60=
8.65*1013 flops
a year=
365*24*60*60*109= 3.1536*1016 flops
We want to find the maximal integer so that
n*n!*nn < 6*1010
n*n!*nn < 8.65*1013
n*n!*nn < 3.1536*1016
33
Important Complexity Classes
Maple Program:
for k from 1 to 10 do (k,k*factorial(k)*kk)end do;
1, 1
2, 16
3, 486
4, 24576
5, 187500
6, 201553920
7, 29054597040
8, 5411658792960
9, 1265284323434880
10, 362880000000000000
So, n=7,8,9 for a minute, a day, and a year.
34
The Growth of Combinations of Functions
• Theorem 2: suppose that f1(x) is O(g1(x)) and
f2(x) is O(g2(x)). Then (f1 + f2)(x) is
O(max( |g1(x)| , |g2(x)| )).
• Corollary 1: suppose that f1(x) and f2(x) are
both O(g(x)). Then (f1 + f2)(x) is O(g(x)).
35
• Theorem: If f1 is O(g1) and f2 is O(g2) then
1. f1 f2 is O(g1g2)
2. f1+f2 is O(max {g1 ,g2})
36
The Growth of Combinations of Functions
• Theorem 3 :suppose that f1(x) is O(g1(x)) and f2(x) is
O(g2(x)).
Then (f1f2)(x) is O(g1(x) g2(x)).
• Example 8: give a big-O estimate for
f(n)=3n log(n!) + (n2 +3) log n
where n is a positive integer.
• Example 9: give a big-O estimate for
f(x)=(x+1)log(x2+1) + 3x2
37
Properties of Big-O
• f is O(g) iff O( f )  O( g )
• If f is O(g) and g is O(f) then O( f )  O( g )
• The set O(g) is closed under addition:
if f is O(g) and h is O(g) then f+h is O(g)
• The set O(g) is closed under multiplication by a scalar a
(real number):if f is O(g) then af is O(g)
That is ,O(g) is a vector space. (The proof is in the book.)
Also, as you would expect,
• If f is O(g) and g is O(h), then f is O(h) .
In particular
O ( f )  O ( g )  O ( h)
38
• Note : we often want to compare algorithms in the
same complexity class
• Example:
Suppose
Algorithm 1 has complexity n2 – n +1
Algorithm 2 has complexity n2/2 + 3n + 2
Then both are O(n2) but Algorithm 2 has a smaller
leading coefficient and will be faster for large
problems.
Hence we write
Algorithm 1 has complexity n2 +O(n)
Algorithm 2 has complexity n2/2 + O(n)
39
Big-Omega and Big-Theta Nation
• Definition 2: Let f and g be functions from the set of integers
or the set of real numbers to the set of real numbers.
• We say that f(x) is Ω(g(x)) if there are positive constants C and
k such that |f(x)|≥ C|g(x)|
Whenever x > k. ( this is read as “f(x) is big-Omega of g(x)” .)
• Example 10 :The function f(x) =8x3+ 5x2 +7 is Ω(g(x)) , where
g(x) is the function g(x) =x3.
• This is easy to see because f(x) =8x3+ 5x2 +7 ≥ x3 for all
positive real numbers x. this is equivalent to saying that
g(x) = x3 is O(8x3+ 5x2 +7 ) ,which can be established directly
by turning the inequality around.
40
• Definition 3: Let f and g be functions from the set of
integers or the set of real numbers to the set of real
numbers.
• We say that f(x) is Θ(g(x)) if f(x) is O(g(x)) and f(x) is Ω(g(x)).
• When f(x) is Θ(g(x)) , we say that” f is big-Theta of g(x)” and
we also say that f(x) is of order g(x).
• Example 11: we showed (in example 5) that the sum of the
first n positive integers is O(n 2). Is this sum of order n 2?
• Example 12: show that 3x2 + 8x(logx) is Θ(x2).
41
• Theorem 4: let f ( x)  an x n  an 1 x n 1    a1 x  a0
, where a0, a1, . . .,an-1 , an are real numbers with
an≠0 . Then f(x) is of order xn .
• Example 13: the ploynomials
3x8+10x7+221x2+1444
x19-18x4-10112
-x99+40001x98+100003x
are of orders x8, x19 and x99 ,respectively.
42
Chapter 3
3.3 Complexity of Algorithms
– Time Complexity
– Understanding the complexity of Algorithms
43
Complexity of Algorithm
• Computational Complexity (of the Algorithm)
• Time Complexity: Analysis of the time required.
• Space Complexity: Analysis of the memory
required.
44
Time Complexity
• Example 1: Describe the time complexity of
Algorithm 1 of section 3.1 for finding the maximum
element in a set (in terms of number of comparisons).
• Algorithm 1: Finding the maximum element in a
finite sequence.
procedure max(a1, a2, . . . ,an: integers)
max := a1
for i: =2 to n
if max < ai then max := ai
{max is the largest element}
45
• Example 2: Describe the time complexity of the
linear search algorithm.
• Algorithm 2 : the linear search algorithm
procedure linear search (x: integer, a1, a2, …,an: distinct integers)
i :=1;
while ( i ≤n and x ≠ ai)
i := i + 1
If i ≤ n then location := i
Else location := 0
{location is the subscript of the term that equals x , or is 0 if x is
not found}
46
• Example 3: Describe the time complexity of the binary search
algorithm in terms of the number of comparisons used .
(and ignoring the time required to compute m= (i  j ) / 2 in
each iteration of the loop in the algorithm)
• Algorithm 3: the binary search algorithm
Procedure binary search (x: integer, a1, a2, …,an: increasing integers)
i :=1 { i is left endpoint of search interval}
j :=n { j is right endpoint of search interval}
While i < j
begin
m := (i  j ) / 2
if x > am then i := m+1
else j := m
end
If x = ai then location := I
else location :=0
{location is the subscript of the term equal to x, or 0 if x is not found}
47
• Example 4: Describe the average-case performance
of the linear search algorithm, assuming that the
element x is in the list.
• Example 5: What is the worst-case complexity of
the bubble sort in terms of the number of
comparisons made?
• ALGORITHM 4: The Bubble Sort
procedure bubble sort (a1, a2, …,an: real numbers with n ≥2)
for i := 1 to n-1
for j := 1 to n- i
if aj > aj+1 then interchange aj and aj+1
{a1, a2, …,an is in increasing order}
48
• Example 6: What is the worst-case complexity of the
insertion sort in terms of the number of comparisons
made?
• Algorithm 5: The Insertion Sort
procedure insertion sort (a1, a2, …,an: real numbers with n ≥2)
for j := 2 to n
begin
i := 1
while aj > ai
i := i + 1
m := aj
for k :=0 to j-i-1
aj-k := a j-k-1
ai := m
end {a1, a2, …,an are sorted}
49
Understanding the complexity of
Algorithms
50
• Solvable (in polynomial time, or in exponential time)
• Tractable: A problem that is solvable using an
algorithm with polynomial worst-case complexity.
• Intractable: The situation is much worse for
problems that cannot be solved using an algorithm
with worst-case polynomial time complexity. The
problems are called intractable.
• NP problem.
• NP-complete problem.
• Unsolvable problem: no algorithm to solve them.
51
• Big-O estimate on the time complexity of an algorithm provides
an upper, but not a lower, bound on the worst-case time
required for the algorithm as a function of the input size.
• Table 2 displays the time needed to solve problems of various
sizes with an algorithm using the indicated number of bit
operations. Every bit operation takes nanosecond. Times of
more than 10100 years are indicated with an asterisk.
52
Chapter 3
–
–
–
–
–
3.4 The Integers and Division
Division
The Division Algorithm
Modular Arithmetic
Applications of Congruences
Cryptology
53
Division
• Definition 1: if a and b are integers with a≠0, we say
that a divides b if there is an integer c such that b=ac.
When a divides b we say that a is a factor of b and
that b is a multiple of a. the notation a|b denotes
that a divides b. we write a | b when a does not
divide b.
• Example 1: Determine whether 3|7 and whether
3|12.
• Example: Determine whether 3|0.
54
• Theorem 1: let a, b, and c be integers. Then
1. If a|b and a|c, then a|(b+c)
2. If a|b and a|bc for all integer c
3. If a|b and b|c, then a|c
•
Corollary 1: If a, b, c are integers such that a|b and
a|c , then
a| mb + nc
whenever m and n are integers.
55
The Division Algorithm
• Theorem 2 the division algorithm :let a be an integer
and d a positive integer. Then there are unique
integers q and r, with 0 ≤ r < d, such that
a= dq+r
• Definition 2: In the equality give in the division
algorithm, d is called the divisor, a is called the
dividend, q is called the quotient, and r is called the
remainder. This notation is used to express the
quotient and remainder.
q = a div d, r = a mod d.
• Example 4: What are the quotient and remainder
when -11 is divided by 3?
56
Modular Arithmetic
• Definition 3: if a and b are integers and m is a
positive integer, then a is congruent to b modulo m if
m divides a - b.
• we use the notation a≡b (mod m) to indicate that a is
congruent to b modulo m.
• if a and b are not congruent modulo m, we write
/ (mod m) .
a ≡b
57
Modular Arithmetic
• Theorem 3: let a and b be integers, and let m
be a positive integer. Then a≡b (mod m) if and
only if a mod m = b mod m .
• Example 5: determine whether 17 is congruent to 5
modulo 6 and whether 24 and 14 are congruent
modulo 6.
58
Modular Arithmetic
• Theorem 4 : let m be positive integer. The integers a and b
are congruent modulo m if and only if there is an integer k
such that a = b + km .
• Theorem 5: let m be a positive integer. If a≡b(mod m ) and
c ≡d (mod m), then
a+c≡b+d (mod m) , ac ≡ bd (mod m)
• Example 6: because 7≡2 (mod 5) and 11≡1 (mod 5) , it
follows from theorem 5 that
18=7+11 ≡2+1=3(mod 5) , and that
77=7*11 ≡2*1=2 (mod 5)
59
• Corollary 2: let m be a positive integer and let
a and b be integers. Then
(a+b) mod m = ((a mod m)+(b mod m)) mod m
And
ab mod m =((a mod m)(b mod m)) mod m.
60
Applications of Congruences
• Hashing Functions
• Pseudorandom Numbers
• Cryptology
61
Hashing Functions
• How can memory locations be assigned so
that customer records can be retrieved quickly?
• Hashing function and key
• h(k) = k mod m; m is the number of available
memory locations.
• Collision: one way to re solve a collision is to
assign the first free location.
62
Pseudorandom Numbers
• The numbers generated by systematic method are
not truly random, they are called pseudorandom
numbers.
• Linear Congruential Method(m, a, c, x0 :integers):
• Modulus m
• Multiplier a, 2  a < m
• Increment c, 0  c < m
• Seed x0 , 0  x0 < m
• xn+1= (axn+c) mod m
• For example: m=9, a=7, c=4, x0 =3, then
(x1, x2, x3, x4, x5, x6, x7, x8, x9)=(7, 8, 6, 1, 2, 0, 4, 5, 3)
x10=x1
63
Cryptology
•
•
•
•
Important Application of Congruences
Earliest known uses by Julius Caesar.
Shifting each letter three letters forward in the alphabet.
To express the process mathematically:
• Let U={0,.., 25}, V={A, .., Z} and g: V -> U is a bijection
function defined as the table below.
• Define function f : U -> U, where f(p)=(p+3) mod 26.
• The Encryption function h:V->V, where h(x)=g-1( f(g(x) ) )
• The decryption function f-1(p)=(p-3) mod 26.
0 1 2 3 4 5 6 7 8 9
A B C D E F G H I
D E F G H I
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
J K L M N O P Q R S T U V W X Y Z
J K L M N O P Q R S T U V W X Y Z A B C
64
Applications of Congruences
• Example 9:
• What is the secret message produced from
the message “MEET YOU IN THE PARK” using
the Caesar cipher.
• HW: Example 10, p208
65
Chapter 3
3.5 Primes and Greatest Common
Divisors
‒ Primes
‒ Greatest common divisors and least
common multiples
66
Primes
• Definition 1: A positive integer p greater than 1 is
called prime if the only positive factors of p are 1 and
p. A positive integer that is greater than 1 and is not
prime is called composite.
• Remark: The integer n is composite if and only if there
exists an integer a such that a|n and 1< a < n.
• Example 1: The integer 7 is prime because its only positive
factors are 1 and 7, whereas the integer 9 is composite
because it is divisible by 3.
67
Primes
• Theorem 1: The fundamental theorem of arithmetic
Every positive integer greater than 1 can be written uniquely
as a prime or as the product of two or more primes where
the prime factors are written in order of nondecreasing size.
• Example 2: The prime factorizations of 100, 641 , 999 and
1024 are given by
100=2*2*5*5=2252
641=641
999=3*3*3*37=33*37
1024=2*2*2*2*2*2*2*2*2*2=210
68
Primes
• Theorem 2: If n is a composite integer , then n
has a prime divisor less than or equal to n .
• Example 3: Show that 101 is prime.
• Example 4: Find the prime factorization of 7007.
69
Primes
• Theorem 3: There are infinitely many primes .
• Proof:
We will prove this theorem using a proof by
contradiction. We assume that there are only
finitely many primes, p1, p2, … , pn.
Let Q= p1  p2  ...  pn  1
70
Greatest Common Divisors
• Definition 2: Let a and b be integers, not both zero.
• The largest integer d such that d|a d|b is called the
greatest common divisor of a and b.
• The greatest common divisor of a and b is denoted
by gcd(a,b).
• Example 10: what is the greatest common divisor of
24 and 36?
71
Greatest Common Divisors
• Definition 3: The integers a and b are relatively
prime if their greatest common divisor is 1.
• Example 12: Prove that y the integers 17 and 22
are relatively prime.
72
Greatest Common Divisors
• Definition 4: The integers a1,a2 …,an are pairwise
relatively prime if gcd(ai , aj)=1 whenever 1≦i <j ≦n.
• Example 13: determine whether the integers 10 , 17
and 21 are pairwise relatively prime and whether the
integers 10 , 19 and 24 are pairwise relatively prime.
• Example 14: Because the prime factorizations of 120
and 500 are 120=23*3*5 and 500=22*53, the greatest
common divisor is
gcd(120,500)=2 min(3 , 2) 3 min(1 , 0) 5 min(1,3)=223051=20
73
Least Common Multiples
• Definition 5: The least common multiple of the
positive integers a and b is the smallest positive
integer that is divisible by both a and b.
• The least common multiple of a and b is
denoted by
lcm(a , b).
• Example 15: What is the gcd and lcm of 233572 and
2 4 33 ?
74
Greatest Common Divisors and
Least Common Multiples
• Theorem 5: Let a and b be positive integers.
Then
ab = gcd(a ,b)* lcm(a , b)
75
Chapter 3
3.6 Integers and Algorithms
‒ Representations of integers
‒ Algorithms for integer operations
‒ Modular Exponentiation
‒ The Euclidean Algorithm
76
Representations of integers
• Theorem 1: Let b be a positive integer greater
than 1. Then if n is a positive integer, it can be
expressed uniquely in the form
n  ak b k  ak 1b k 1  ...  a1b  a0
where k is a nonnegative integer, a0, a1, …,ak
are nonnegative integers less than b, and ak ≠0.
77
• Example 1: What is the decimal expansion of
the integer that has (1 0101 1111)2 as its
binary expansion?
• Example 2: What is the decimal expansion of
the hexadecimal expansion of (2AE0B)16 ?
78
• Example 3: Find the base 8, or octal, expansion
of (12345)10
• Example 4: Find the hexadecimal expansion of
(177130)10?
79
• Algorithm 1: Construction Base b Expansions
procedure base b expansion(n:positive integer)
q: = n
k: =0
while q ≠ 0
begin
ak : =q mod b
q: = q / b 
k: =k+1
end {the base b expansion of n is (ak-1 . . . a1 a0)b}
80
81
Algorithms for integer operations
• Algorithm 2: Addition of Integers
Procedure add(a , b:positive integers)
{the binary expansions of a and b are (an-1 . . . a1 a0)2 and
(bn-1 . . . b1 b0)2 respectively}
c : =0
for j: =0 to n-1
Begin
d : = (a j  b j  c) / 2
sj : = aj+bj+c-2d
c : =d
end
sn:=c {the binary expansion of the sum if (sn sn-1. . . s1 s0)2 }
82
Algorithms for integer operations
Example 7:
Add a=(1110)2 and b=(1011)2.
83
Algorithms for integer operations
• Algorithm 3 : Multiplying Integers
procedure multiply(a, b : positive integers)
{the binary expansions of and b are(an-1 . . . a1 a0)2 and
(bn-1 . . . b1 b0)2 respectively}
for j:=0 to n-1
Begin
if bj =1 then cj=a shifted j places
else cj:=0
end
{c0 c1 . . . cn-1 are the partial products}
p :=0
for j:=0 to n-1
p: = p +cj
{p is the value of ab}
84
Algorithms for integer operations
Example 9: Find the product of
a= (110)2 and b=(101)2
85
Algorithms for integer operations
•
Algorithm 4 : Computing div and mod
procedure division algorithm(a :integers ,d: positive integer)
q: =0
r: =|a|
while r≧d
begin
r := r-d
q :=q+1
end
if a<0 then
if r=0 then q:=-q else
begin
r := d-r
q := -(q+1)
end
{q = a div d is the quotient, r = a mod d is the remainder}
86
Modular Exponentiation
• In cryptography it is important to be able to find bn mod
m efficiently, where b, n and m and large integers. It’s
impractical to first compute bn and then find its
remainder when divided by m because bn will be a huge
number. Instead, we can use an algorithm that employ
expansion of the exponent n , say n = (ak-1 . . . a1 a0)2 .
• Before we present this algorithm, we illustrate its basic
idea. We will explain how to use the binary expansion of
n to compute bn .First , note that
b b
n
ak 1 2k 1 ...  a1 2 a0
b
ak 1 2k 1
 ...  b
a1 2
b
a0
87
Modular Exponentiation
• To compute bn , we find the values of b, b2,(b2)2=b4,
2k
4
2
8
(b ) =b , . . . , b .
2j
• We multiply the terms b in this list, where aj=1 .
This gives us b n .
• For example, to compute 311 we first note that 11 =
(1011)2, so that 311= 383231.
• By successively squaring, we find that 32=9, 34=81,
38=6561.
• Consequently,311=383231=6561*9*3= 177,147
88
Modular Exponentiation
• Algorithm 5: Modular Exponentiation
procedure modular exponentiation(b:integer ,
n=(ak-1 . . . a1 a0)2 ,m: positive integer)
x: = 1
Example 11: Use Algorithm 5
power := b mod m
to find 3644 mod 645.
for i=0 to k-1
begin
for ai =1 then x :=(x*power) mod m
power :=(power*power) mod m
End
{x equals bn mod m}
89
The Euclidean Algorithm
• Lemma 1: Let a=bq+r ,where a, b, q, and r are
integers. Then gcd(a,b)=gcd(b,r).
• Algorithm 6: The Euclidean Algorithm
procedure gcd(a.b:integers)
x: = a
y: = b
while y0
begin
r := x mod y
x := y
y := r
end {gcd(a,b) is x}
90
The Euclidean Algorithm
• Example 12: Find the GCD of 414 and 662
using the Euclidean Algorithm.
91
Chapter 3
3.7 Applications of Number Theory
‒
‒
‒
‒
‒
‒
Some Useful Results
Linear Congruences
The Chinese Remainder Theorem
Computer Arithmetic with Large Integers
Pseudoprimes
Public Key Cryptography
92
Some Useful Results
• Theorem 1: If a and b are positive integers,
then there exist integers s and t such that
gcd(a ,b) = sa+tb .
• Example 1: express gcd(252 , 198) =18 as a
linear combination of 252 and 198 .
93
Some Useful Results
• Lemma 1: If a, b, and c are positive integers
such that gcd(a , b) = 1 and a|bc, then a|c .
• Lemma 2 : If p is a prime and p|a1a2. . .an,
where each ai is an integer , then p|ai for
some i.
• Theorem 2: Let m be a positive integer and let
a, b ,and c be integers. If ac≡ bc (mod m) and
gcd(c, m) = 1 , then a≡b (mod m).
94
Linear Congruences
• A congruence of the form ax≡b (mod m) where m is a
positive integer , a and b are integers , and x is variable, is
called a linear congruence.
• Such congruences arise throughout number theory and
its applications.
• How can we solve the linear congruence ax≡b (mod m) ?
That is, find the x that satisfy this congruence.
• One method that we will describe uses an integer ā such
that aā≡1 (mod m), if such an integer exist.
• Such an integer ā is said to be an inverse of a modulo m.
• Theorem 3 guarantees that an inverse of a modulo m
exists whenever a and m are relatively prime.
95
Linear Congruences
• Theorem 3: If a and m are relatively prime integers
and m>1, then an inverse of modulo m exist.
Furthermore, this inverse is unique modulo m.
(there is a unique positive integer ā less than m that is an
inverse of a modulo m and every other inverse of a modulo
m is congruent to ā modulo m.)
When we have an inverse of a modulo m, that is, ax≡1 (mod
m) , we can easily solve the congruence ax≡b (mod m).
96
The Chinese Remainder Theorem
• Example 3: Find an inverse of 3 modulo 7?
Theorem 5, section 3.4, p204.
Let m be a positive integer. If a≡b (mod m) and c≡d (mod m), then
a+c≡b+d (mod m) and ac≡bd (mod m).
• Example 4: What are the solutions of the liner congruence 3x ≡4
( mod 7)?
• Example 5: In the first century, the Chinese mathematician SunTsu asked:
There are certain things whose number is unknown. When
divider by 3, the remainder is 2; when divided by 5, the
remainder is 3; and when divided by 7 , the remainder is 2.
What will be the number of things?
97
The Chinese Remainder Theorem
• Theorem 4: The Chinese Remainder Theorem
Let m1, m2, . . . ,mn be pairwise relative prime positive
integers and a1, a2,. . . ,an arbitrary integers. Then the
system
x≡a1 ( mod m1)
x≡a2 ( mod m2)
…
x≡an ( mod mn)
has a unique solution modulo m= m1, m2, . . . ,mn .
(That is , there is solution x with 0 ≦x ＜m, and all other
solutions are congruent modulo m to this solution.)
98
射雕英雄傳 第一千四比二十七首
瑛姑說道: 『. . . 今有物不知其數，三三數支謄二，
五五數之謄三，七七數之謄二，問物幾何?』
黃蓉笑道: 『這容易得緊，以三三數之，餘數乘以七
十; 五五數之，餘數乘以二十一，七七數之，餘數
乘以十五。三者相加，如不大於一百零五，即為
答數; 否則須減去一百零五或其倍數。』
黃蓉道: 『也不用這般硬記，我念一首詩給你聽，那
就容易記了:三人同行七十稀，五樹梅花二一枝，
七子團員正半月，餘百零五便得知。』
99
The Chinese Remainder Theorem
• Example 6: Solve the system of congruences in
Example 5 by using theorem 4.
• Example 5:there are certain things whose number is
unknown. When divider by 3, the remainder is 2;
when divided by 5, the remainder is 3; and when
divided by 7 , the remainder is 2. What will be the
number of things?
100
Computer Arithmetic with Large Integers
• Suppose that m1, m2, . . . ,mn are pairwise
relatively prime integers greater than or equal to
2 and let m be their product. By the Chinese
Remainder Theorem, we can show that an
integer a with 0≤ a < m can be uniquely
represented by the n-tuple consisting of its
remainders upon division by mi , i= 1, 2,. . .,n.
• We can uniquely represent a by
(a mod m1, a mod m2, . . ., a mod mn)
101
Computer Arithmetic with Large Integers
• Example 7: What are the pairs used to represent
the nonnegative integers less than 12 when they
are represented by the ordered pair where the
first component is the remainder of the integer
upon division by 3 and the second component is
the remainder of the integer upon division by 4?
102
Pseudoprimes
• Theorem 5: Fermat’s Little Theorem
If p is prime and a is an integer not divisible by p, then
ap-1 ≡1 (mod p)
Furthermore, for every integer a we have
ap ≡a (mod p)
• Unfortunately, there are composite integer n, such
that 2n-1≡1 (mod p). Such integers are called
pseudoprimes to the base 2.
• Example 9: Explain why the integer 341 is a
pseudoprime to the base 2.
103
Computer Arithmetic with Large Integers
• Definition 1: Let b be a positive integer. If n is a
composite positive integer, and bn-1 ≡1 (mod n), then
n is called a pseudoprime to the base b.
• Definition 2: A composite integer n that satisfies the
congruence bn-1 ≡1 (mod n) for all positive integers b
with gcd(b , n)=1 is called a Carmichael number.
• (This numbers are named after Robert Carmichael, who studied
them in the early twentieth century)
• Example 10: The integer 561 is a Carmichael
number.
104
Private Key Cryptography
Private key cryptosystems (Section 3.4, Example 9,
p207)
• c=(p+k) mod 26, where p, c represent a letter,
k is an encryption key.
• Everybody knowing this key can both encrypt
and decrypt messages easily.Private
• Two people need to securely exchange the key
in advance.
105
Public Key Cryptography
• In 1976, three researchers at M.I.T. – Ronald Rivest,
Adi Shamir, and Leonard Adleman – introduced to
the world a public key cryptosystem, known as the
RSA system.
• The RSA cryptosystem is based on modular
exponentiation modulo the product of two large
primes, which can be done rapidly using Algorithm 5
in section 3.6.
• Each individual has an encryption key consisting of a
modulus n=pq, where p and q are large primes, say,
with 200 digits each, and an exponent e that is
relatively prime to (p-1)(q-1).
106
Public Key Cryptography
• To produce a usable key, two large primes must be
found. This can be done quickly on a computer using
probabilistic primality test. (Example 16, Section 6.2,
p 412-413 text book)
• However, the product of these primes n=pq, with
approximately 400 digits, cannot be factored in a
reasonable length of time. This is an important
reason why decryption cannot be done quickly
without a separate decryption key.
107
RSA Encryption
• In the RSA encryption method, messages are
translated into sequences of integers.
• These integers are grouped together to form larger
integers, each representing a block of letters.
• The encryption proceeds by transforming the integer
M, representing the plaintext (the original message),
to an integer C, representing the ciphertext (the
encryption message), using the function C=Me mod n.
108
RSA Encryption
• Example 11: Encrypt the message STOP using the RSA
cryptosystem with p=43 and q=59, so that n=43 x 59
= 2537, and with e=13. Note that
Gcd(e, (p-1)(q-1)) = gcd(13, 42 x 58)=1.
109
RSA Decryption
• The plaintext message can be quickly recovered
when the decryption key d, an inverse of e modulo
(p-1)(q-1), is known. Such inverse exist because gcd(e,
(p-1)(q-1))=1).
• de≡1 (mod (p-1)(q-1)), there exist an integer k, such
that de=k(p-1)(q-1)+1.
• It follows that Cd≡(Me)d=Mde=M1+k(p-1)(q-1) (mod n).
• By Fermat’s Little Theorem (theorem 5)[assuming
that gcd(M,p)=gcd(M,q)=1, which holds except in
rare cases], it follows that Mp-1≡1 (mod p) and Mq-1
≡1 (mod q).
110
RSA Decryption
• Consequently,
Cd ≡M(Mp-1)k(q-1) ≡M (mod p)
and
Cd ≡M(Mq-1)k(p-1) ≡M (mod q)
• Because gcd(p,q)=1, it follows by the Chinese
Remainder Theorem that
Cd ≡M (mod pq)
111
RSA Decryption
• Example 12: We receive the encrypted message 0981
0461. What is the decrypted message if it was
encrypted using the RSA cipher form example 11.
112
Chapter 3
3.8 Matrices
‒
‒
‒
‒
Matrix Arithmetic
Algorithms for Matrix Multiplication
Transposes and Powers of Matrices
Zero-One Matrices
113
Matrix Arithmetic
• Definition 1:
• A matrix is a rectangular array of numbers.
• A matrix with m rows and n columns is called an
m × n matrix.
• The plural of matrix is matrices. A matrix with the
same number of rows as columns is called square.
• Two matrices are equal if they have the same
number of rows and the same number of columns
and the corresponding entries in every position are
equal.
114
Matrix Arithmetic
• Definition 2: Let
 a11
a
A   21
 

an1
a12
a22

an 2
• The ith row of A is the
1 x n matrix [ai1,ai2,. . .,ain].
 a1n  • The jth column of A is
 a2 n 
the n x 1 matrix  a1 j 
a 
 2j
  


a

The (i, j)th element or entry of is the element aij ,
 nj 

that is , the number in the ith row and jth column of A.
A convenient shorthand notation for expressing the
matrix A is to write A =[aij], which indicates that A is the
matrix with its (i, j)th element equal to aij.


 

ann 
115
Matrix Arithmetic
• Definition 3: Let A=[aij] and B=[bij] be m x n
matrices. The sum of A and B, denoted by A+B,
is the m x n matrix that has aij+bij as its (i, j)th
element. In other words, A+B= [aij+bij].
• Example 2: we have
1
2

3
0
2
4
 1  3
 3   1
0   1
4
3
1
4
 1
0   3

2 
2
4  2
 1  3
5
2 
116
Matrix Arithmetic
• Definition 4:
• Let A be an m x k matrix and B be k x n matrix.
• The product of A and B, denoted by AB, is the m x n
matrix with its (i , j )th entry equal to the sum of the
products of the corresponding elements from the
ith row of A and the jth column of B.
• In other words, if AB=[cij], then
cij = ai1b1j + ai2b2j +. . . +aikbkj
117
Matrix Arithmetic
118
Algorithms for Matrix Multiplication
• Algorithm 1 : Matrix
Multiplication
procedure matrix multiplication
(A, B: matrices)
for i := 1 to m
for j := 1 to n
begin
cij :=0
for q := 1 to k
cij :=cij + aiqbqj
end
{C= [cij] is the product of A and B}
• Example 6: In which order
should the matrices A1, A2,
and A3, where
• A1 is 30x20 , A2 is 20x40 ,
A3 is 40x10,
• all with integer entries –
be multiplied to use the
least number of
multiplications of integers?
119
Transposes and Powers of Matrices
• Definition 5:
the identity matrix of order n is the n x n matrix
In = [δij]
where δij =1 if i = j and δij = 0 if i ≠ j. Hence,
1 0 0 0 
0 1 0 0 

In  
   


0 0 0 1 
120
Transposes and Powers of Matrices
• Definition 6: Let A=[aij] be an m x n matrix.
• The transpose of A, denoted by At, is the n x m
matrix obtained by interchanging the rows and
columns of A .
• In other words, if At=[bij], then bij = aji for i=1,2,. . .,n
and j = 1,2,. . .,m .
• Definition 7: A square matrix A is called symmetric
if A = At.
• Thus A =[aij] is symmetric if aij = aji for all i and j with
1≤ i ≤ n and 1 ≤ j ≤ n .
121
Symmetric Matrix
122
Zero-One Matrices
• Definition 8: Let A=[aij] and B=[bij] be m x n zeroone matrices.
• Then the join of A and B is the zero-one matrix with
(i , j )th entry aij v bij.
The join of A and B is denoted by A v B.
• The meet of A and B is the zero-one matrix with (i ,
j )th entry aij Λ bij.
The meet of A and B is denoted by A Λ B.
123
Zero-One Matrices
• Definition 9: Let A=[aij] be an m x k zero-one matrix
and B=[bij] be a k x n zero-one matrix .
• Then the boolean product of A and B,denote by A⊙B ,
is the m x n matrix with with (i , j)th entry cij where
cij = (a i1  b1j )  ( a i2  b 2j )  . . .  (a ik  b kj )
• Example 10: find the Boolean product of A and B,
1 0
where
1 1 0


A  0 1, B  

0
1
1


1 0
124
Zero-One Matrices
• Algorithm 2: The Boolean Product
procedure Boolean product(A, B: zero-one matrices)
for i := 1 to m
for j := 1 to n
begin
cij :=0
for q := 1 to k
cij := cij  (a iq  b qj )
end
{C= [cij] is the Boolean product of A and B}
125
Zero-One Matrices
• Definition 10: Let A be a square zero-one
matrix ant let r be a positive integer.
• The rth Boolean power of A is the Boolean
product of r factors of A. The rth Boolean
product of A is denoted by A[r]
• Hence, [r]
A A
C⊙
A
⊙
A
⊙


⊙

r times
• (this is well defined because the Boolean product of matrices
is associative.)
• We also define A[0] to be In
126
Zero-One Matrices
• Example 11: Let
0 0 1 .


A  1 0 0
1 1 0
Find A[n] for all positive integers n.
127