Download Sequences - UNM Computer Science

Sequences
Shuang (Sean) Luan
Department of Computer Science
University of New Mexico
Albuquerque, NM 87131
E-mail: [email protected]
1
What is a Sequence?
Definition 1 A sequence is a list of objects arranged in a definite order: a first element, a second
element, a third element, ..., and so on. The elements may be different and may be the same.
Comparing sequence and set, there are several distinct differences. First, in sequence we emphasize
on the order, while set is an unordered collection of objects. Secondly, in sequence we can have
repeating elements, while in a set, the collection of the objects must be well-defined and therefore
distinct.
2
Why sequence?
We are particularly interested in a number of special mathematical sequences and the analytical
equation for the sum. This has applications when we analyze running time of computer algorithms.
Generally speaking, an algorithm is a step by step procedure for accomplishing a task. In algorithmic theory, we are interests in the running time of the algorithm. However, since the actual
performance of the algorithm can only be determined after implementation, and depends on a variety
of factors such as the programming language used, the configuration of the physical machine, etc. One
way to get around of all these complications is to come up with an abstract way to model running
times.
In algorithmic theory, we use the number of discrete steps as a function of the input size to
characterize running times of an algorithm. The input size is also abstracted as the number of discrete
units of storage, where a unit of discrete storage is something that can be stored in constant (or finite)
amount of memory. Examples of data that requires only a discrete unit storage are standard type
integers, standard floating point numbers, double precision numbers, etc. (Note, the storage used to
store a number of variable precision is not a discrete storage unit.) Generally speaking, a discrete
step is something that can be executed by the CPU in constant (or finite) time. For examples, the
addition of two standard type integers, the multiplication between two standard type double precision
numbers, and the comparison between two standard type integers all take a discrete step.
Example 1 Consider the following computing problem. You are given a sequence S of standard type
integers: s1 , s2 , s3 , .... Design an algorithm to sort the numbers in non-decreasing order.
One simple sorting algorithm is insertion sort. Insertion sort iterates over the input, while maintaining/growing a sorted sequence of the current encountered numbers. In each iteration, insertion
1
sort takes a new number from the input data, finds the location it belongs within the sorted list, and
inserts it there. It terminates when all input numbers are sorted. This algorithm can be described
using the following (pseudo-code) algorithm.
Input: A list of n numbers denoted by s1 , s2 , s3 , ..., sn .
Output: The same list of numbers sorted in non-decreasing order.
Line 1: for i = 2 to n do:
Line 2:
for j = i to 2
Line 3:
if (sj < sj−1 ), swap sj and sj−1 .
Line 3:
else break.
So, what is the running time of the above algorithm? Recall that the running time is a function
of the input size. What is the input size? The input size is n because we only need n units of storage
for the n numbers given. Thus the running time is a function f (n). Now we need to count how many
discrete steps the algorithm take. Observe that the algorithm runs in two loops. The outside loop for
i is executed n − 1 times. The inside loop is executed for i − 1 times. Each loop only takes constant
time. Now if we use ti to denote the time it takes for the outside loop for each i, then ti = i − 1.
The total running time is the sum t2 + t2 + ... + tn = 1 + 2 + 3 + ... + n − 1. Thus we are essentially
calculating the sum of a mathematical sequence: 1, 2, 3, ..., n − 1.
3
Arithmetic Sequence
Definition 2 A sequence of numbers is said to be arithmetic if the difference between adjacent numbers
is constant.
Example 2 The sequence 1, 2, 3, 4, ..., n is an arithmetic sequence where the difference between adjacent numbers is 1.
Example 3 The sequence n, n − 1, n − 2, n − 3, ..., 1 is an arithmetic sequence where the difference
between adjacent numbers is −1.
Example 4 The sequence 4, 6, 8, ..., 2n is an arithmetic sequence where the difference between adjacent
numbers is 2.
We are now developing the analytical equation for calculating the sum of an arithmetic sequence.
For ease of discussion, we will first introduce a general representation of an arithmetic sequence.
Let a1 , a2 , a3 , ..., an , ..., be an arithmetic sequence. Let d be the constant difference between adjacent
P
elements. We shall use Sn to denote the sum of the first n numbers in the sequence, i.e., S = nj=1 aj =
P
a1 + a2 + ... + an , where
is the summation symbol.
Since the difference between adjacent elements is the constant d, we have:
a1 = a1
a2 = a1 + d = a1 + (2 − 1)d
a3 = a2 + d = (a1 + (2 − 1)d) + d = a1 + (3 − 1)d
a4 = a3 + d = (a1 + (3 − 1)d) + d = a1 + (4 − 1)d
2
...
aj = a1 + (j − 1)d
...
Thus, we can represent any number aj in the sequence using a1 and d.
The trick for calculating the sum is what we called the “flip-and-add” trick. Consider the sum
Sn = a1 + a2 + a3 + ... + an . Observe that if we flip the order of the aj ’s and add them backward from
an to a1 , we still end up with the same sum. In other words sn = an + an−1 + an−1 + a1 . Now what
if we add the two equations together?
Sn = a1 +
a2
+
a3
+ ... + an
+ Sn = an + an−1 + an−2 + ... + a1
Replacing the aj ’s with their representation using a1 and d, we have:
Sn =
a1
+
a1 + d
+
a1 + 2d
+...+ a1 + (n − 1)d
+ Sn = a1 + (n − 1)d + a1 + (n − 2)d + a1 + (n − 3)d +...+
a1
2Sn = 2a1 + (n − 1)d + 2a1 + (n − 1)d + 2a1 + (n − 1)d +...+ 2a1 + (n − 1)d
= (2a1 + (n − 1)d)n
= (a1 + (a1 + (n − 1)d))n)
= (a1 + an )n
n )n
Sn = (a1 +a
2
Thus the general equation for the sum of arithmetic sequence is: Sn = a1 + a2 + ... + an =
4
(a1 +an )n
.
2
Geometric Sequence
Definition 3 A sequence of numbers is said to be a geometric sequence if the ratio between adjacent
numbers is constant.
Example 5 The sequence 2, 2, 2, ... is a geometric sequence where the ratio between adjacent numbers
is 1. In fact the sequence is an arithmetic sequence where the difference between adjacent numbers is
0.
When the ratio between adjacent numbers is 1, the geometric sequence becomes “degenerate”
where all the numbers are the same. The situation becomes trivial. Thus in our discussion, we shall
assume that the constant ratio for a geometric sequence is not equal to 1.
Example 6 The sequence 20 , 21 , 22 , ..., 2n is a geometric sequence where the ratio between adjacent
numbers is 2.
Example 7 The sequence 2n , 2n−1 , 2n−2 , ..., 20 is a geometric sequence where the ratio between adjacent numbers is 21 .
Example 8 The sequence
4
numbers is 32 .
35 39 31 3
34n+1
2 , 23 , 25 , ..., 22n+1
is a geometric sequence where the ratio between adjacent
Similar to the arithmetic sequence, there is also a general representation of a geometric sequence.
Let a1 , a2 , a3 , ..., an , ..., be a geometric sequence. Let r be the constant difference between adjacent
P
elements. We shall use Sn to denote the sum of the first n numbers in the sequence, i.e., S = nj=1 aj =
a1 + a2 + ... + an .
Since the ratio between adjacent numbers is the constant r, we have:
a1 = a1
3
a2 = a1 · r = a1 · r2−1
a3 = a2 · r = (a1 · r2−1 ) · r = a1 · r3−1
a4 = a3 · r = (a1 · r3−1 ) · r = a1 · r4−1
...
aj = a1 · rj−1
...
Thus, we can represent any number aj in the sequence using a1 and r.
The trick for calculating the sum is what we called “shift-and-subtract” or “annihilator”. Consider
the sum Sn = a1 + a2 + a3 + ... + an . Observe that if we multiply the ratio r on both sides, we get
Sn ·r = a1 ·r+a2 ·r+a3 ·r+...+an ·r. Since aj ·r = aj+1 , we end up with: Sn ·r = a2 +a3 +a4 +...+an+1 .
Observe that the righthand is appears to be shifted to the right.
Now if we subtract the original sum Sn from Sn · r = a2 + a3 + a4 + ... + an+1 , the items a2 , a3 , ..., an
all cancel out, or annihilated. This is illustrated in the table below:
Sn
=
a1
+
a2
+
a3
+...+
an
Sn · r
= a1 · r + a2 · r + a3 · r +...+ an · r
Sn · r
=
+
a2
+
a3
+...+ a + n an+1
Sn
=
a1
+
a2
+
a3
+...+
an
sn · (r − 1) = −a1 +
+
+...+
an+1
sn · (r − 1) = (an+1 − a1 )
−a1
Sn
= an+1
r−1
·r−a1
Sn
= anr−1
·r−a1
Thus the general equation for the sum of geometric sequence is: Sn = a1 + a2 + ... + an = anr−1
.
5
Harmonic Sequence
Definition 4 The sequence 1, 21 , 31 , 41 , ... is called the harmonic sequence.
The trick for calculating the sum of a harmonic sequence is integration. Let Sn = 1+ 12 + 31 + 14 +...+ n1 .
Observe that:
1
1
1
1
1
+
+
+ ... +
+
Sn = 1 +
n−1
n
R 2 21
R 3 31
R 4 41
R n−1
R
n
1
1
≤ 1 + R1 x dx +
+
+...+
+
2 x dx
3 x dx
n−2 x dx
n−1 x dx
n 1
= 1 +
1 x dx
On the other hand,
1
1
1
1
1
Sn =
1
+
+
+
+ ... +
+
R2 1
R 3 21
R 4 31
R 5 41
R n n−11
R n+1n 1
≥
+
+
+
+...+
+
x dx
x dx
2 x dx
3 x dx
4 x dx
n−1 x dx
n
R1n+1
1
=
x dx
R n+1 1 1
Rn 1
x dx ≤ Sn ≤ 1 + 1 x dx. Thus ln(n + 1) ≤ Sn ≤ 1 + ln n.
1
6
Sequences and Recursive Functions
The concept of sequence is closely related to recursive functions. Before we formally define recursive
function, let’s first explore the relation between mathematical sequence and function.
Recall that a mathematical sequence is a list of numbers arranged in a definite order: the first
number, the second number, the third number, ... Thus a mathematical sequence is in fact a mapping
from the natural numbers N = {1, 2, 3, 4, ...} to the set of real numbers R. Because of this, we can
represent a mathematical sequence as a list of values for some function f : N → R as f (1), f (2), f (3),
... Thus any function from N to R is in fact a mathematical sequence.
4
Notationwise, we often use lower case letters a, b, c, ... to represent mathematical sequences.
However, instead of write a(n) like a function, we often put the index as the subscript. Thus, a
mathematical sequence is often denoted as a1 , a2 , a3 , ..., where an is used to denote the n−th term in
the sequence.
Note that some textbooks defines a mathematical sequence as a function from non-negative integers,
i.e., {0, 1, 2, 3, ...} to real numbers. In this case, the first element of the sequence is denoted a0 , a1 , a2 ,
... In this lecture, we will use both conventions, which should be clear in the context of the discussion.
Although it is worthwhile to realize that mathematical sequence are essentially functions whose
domains are natural numbers, most of the sequences that we work with are not general functions but
a special type of function called recursive functions.
Definition 5 A recursive function is a function whose definition involves the function itself and the
exit (or initial) conditions.
Example 9 The arithmetic sequence 1, 2, 3, ... can also be defined recursively as follows: an =
an−1 + 1 for n ≥ 2 and a1 = 1. In this definition, the term a1 is the exit (or initial) condition of the
recursive function. Without it, the sequence is not well-defined.
Example 10 The mathematical sequence defined recursively as: an = an−1 + 1 for n ≥ 2 and a1 = 10
is also arithmetic. We can iteratively obtain the sequence starting from the term a1 : 10, 11, 12, ...
Example 11 The mathematical sequence defined recursively as: an = an−1 + 2.5 for n ≥ 2 and a1 = 2
is also arithmetic. We can iteratively obtain the sequence starting from the term a1 : 2, 4.5, 7, ...
Example 12 The geometric sequence 20 , 21 , 22 , ... can also be defined recursively as follows: an =
an−1 · 2 for n ≥ 2 and a1 = 20 . famous Fibonacci sequence is most often defined recursively as:
Example 13 The famous Fibonacci sequence is most often defined recursively as: Fn = Fn−1 + Fn−2
for n ≥ 2, and F0 = 0 and F1 = 1. With the initial terms F0 and F1 , we can calculate the Fibonacci
sequence: 0, 1, 1, 2, 3, 5, ...
An important topic in studying the recursive functions is its solution, i.e., the analytic representation of each term in the sequence. The iterative expansion, where we obtain the first several elements
iteratively and try to deduce the trend of the progression, is a powerful technique, but proves to be
too cumbersome and impractical for most recursively defined sequences.
example, a analytical
√ For
√ 1+ 5
2
n
−
√
5
1− 5
2
n
. It is very difficult
representation of the Fibonacci sequence is the following: an =
to come up with such a solution by simply examining the first several numbers. We will not study
the solution to a general recursive function in this course. The topic will be discussed in details in
an algorithm class. We will however turn to another topic for the rest of this lecture. Assume that
using iterative expansion or other methods, we come up with a general analytical representation of a
mathematical sequence, how do one prove that the analytical equation is indeed correct? For this, we
will need a proof technique called mathematical induction.
7
Mathematical Induction
Theorem 1 (Induction Theorem) Given a statement P with a parameter n. To prove that P is
correct for any natural number n, it suffices to do the following:
5
Basis Step: Show that P is true for n = 1.
Induction Step: Assume P is correct for n = k, which is called the induction hypothesis. Show
that with the induction hypothesis, and show that P is correct for n = k + 1.
Example 14 Consider mathematical sequence defined as: an = an−1 +1 for n ≥ 2 and a1 = 10. Show
that an = 9 + n for n = 1, 2, ...
Proof:
Basis Step: For n = 1, an = 10 = 9 + 1. Done.
Induction Step: Assume claim is correct for n = k. In other words, ak = 9 + k. Then for
b = k + 1, ak+1 = ak + 1 = (9 + k) + 1 = 9 + (k + 1). Hence the claim is also correct for n = k + 1.
2
In some situations, the following strong induction theorem may come in handy for proving statements with respect to natural numbers.
Theorem 2 (Strong Induction Theorem) Given a statement P with a parameter n. To prove
that P is correct for any natural number n, it suffices to do the following:
Basis Step: Show that P is true for n = 1.
Induction Step: Assume P is correct for n ≤ k, which is called the strong induction hypothesis.
Show that with the induction hypothesis, and show that P is correct for n = k + 1.
Example 15 Consider mathematical sequence defined as: an = 2an−1 − an−2 for n ≥ 2, a0 = 0, and
a1 = 1. Show that an = n for n = 1, 2, ...
Proof:
Basis Step: For n = 0 and n = 1, the claim is obviously correct.
Induction Step: Assume claim is correct for n ≤ k. In other words, a0 = 0, a1 = 1, ..., ak = k.
Then for n = k + 1, we have ak+1 = 2ak − ak−1 = 2k − (k − 1) = k + 1. Hence the claim is also
correct for n + 1.
2
8
Martingale Betting Strategy and its Limitations
6