Download DdsLect99

Lecture Notes (Draft)
COMS100B, 1999
Department of Computer Science
University of the Witwatersrand, Johannesburg
Data and Data Structures (DDS)
Table of Contents
1.
2.
Introduction
1.1. General
1.2. Objectives of this course
1.3. Contents of this course
3
3
3
3
Some basic definitions and terms
5
3.
Representing numerical data values
3.1. Classical systems for representing numerical values
3.2. Positional notation
3.3. Converting integers between positional number systems with different radices
3.4. Converting fractions between positional number systems with different radices
3.5. Representing integers in a computer memory
3.6. Floating point systems
3.7. Precision and accuracy
8
8
9
10
13
16
19
22
4.
Representing non-numerical data values
4.1. Standards for representing individual characters
4.2. Representing a value as a sequence of characters or symbols
23
23
24
5.
Arrays, lists, vectors, sequences
5.1. Computer memory
5.2. One-dimensional arrays and their representation in a computer memory
5.3. Records
5.4. Files
5.5. Multidimensional arrays
26
26
27
30
32
32
6.
Association by links and pointers
6.1. Linked lists
6.2. Pointers and links
6.3. Diagrams of linked lists
6.4. Searching a linked list
6.5. Inserting an element into a linked list
6.5.1. Case 1: inserting a new item at the beginning of a list
6.5.2. Case 2: inserting a new item after an existing element of a list
6.6. Deleting an element from a linked list
6.6.1. Case 1: deleting the first element from a list
6.6.2. Case 2: deleting an element other than the first from a list
6.7. List of available elements
35
35
38
38
39
42
42
44
46
46
48
50
DDS Lecture Notes (draft)
-1-
Robert L. Baber, 1999 July
7.
Queues
7.1. Pipelines
7.2. Stacks
7.3. Priority Queues
52
52
55
58
8.
An introduction to trees
8.1. Definitions and terminology
8.2. Traversing a tree
8.3. Creating a tree
60
60
61
66
9.
Polish notation
9.1. Expressions in infix, Polish prefix, Polish postfix and tree form
9.2. Converting expressions between infix, Polish notation and tree form
9.2.1. Converting from fully parenthesized infix to Polish postfix notation
9.2.2. Converting from fully parenthesized infix to Polish prefix notation
9.2.3. Converting from Polish postfix to fully parenthesized infix notation
68
68
69
69
70
71
10. Other types of trees
10.1. Binary search trees
10.2. Heaps
10.3. B-Trees
73
73
75
77
11. Comparison of data structures and their algorithms
78
12. Selected other data structures
12.1. Graphs
12.2. Hash tables
12.3. Permutation arrays
80
80
80
81
13. Summary and conclusions
84
14. References
85
DDS Lecture Notes (draft)
-2-
Robert L. Baber, 1999 July
1. Introduction
1.1. General
1.
2.
3.
4.
5.
introduction of lecturer
recommended reading list
meeting times and places
teaching vs. learning
administrative announcements
1.2. Objectives of this course
The objectives of this course are:
1. to familiarize the student with the basic and most important types of data structures,
2. to familiarize the student with their representation and manipulation in a computer system,
3. to develop the student’s understanding of these data structures and their associated
algorithms,
4. to develop the student’s ability to analyze them and
5. to make the student aware of the relative advantages and disadvantages of the various data
structures and algorithms and of the tradeoffs arising when selecting and designing data
structures and algorithms for different purposes.
While the student will write several programs as part of the assigned exercises, acquiring
detailed knowledge of and developing skills in any particular programming language are not
objectives of this course. Stress will be placed on learning the various concepts independently
of any programming language and its special features intended to facilitate the manipulation
of certain data structures, as such special features often hide the very mechanisms and
algorithms we wish to study in this course. The student should, therefore, pay particular
attention in this course to the distinction between fundamental concepts vs. implementation
details. Emphasis should be placed on the fundamental concepts.
1.3. Contents of this course
Viewed abstractly, two main aspects characterize data: values and associations between
values. Such values and associations can be represented in a number of ways, i.e. by various
types of symbols and of data structures.
A computer memory typically consists of one or a few sequences of cells, each cell being
capable of storing one symbol selected from a limited set.
This course examines
 various data structures (ways of representing values and associations between values),
 how these data structures can be represented in a computer memory,
 algorithms for manipulating these data structures and
 important characteristics (e.g. efficiency, time and space complexity) of these data
structures and algorithms.
We will often look at a specific data structure and consider how it can be implemented using
other, simpler, more concrete and less abstract data structures. This corresponds to the designer’s task of implementing a data structure not directly supported by the target system in
terms of data structures which are implemented by the available system. Ultimately, of
course, the higher level data structure must be implemented in the data structure actually
available in the hardware — typically a sequence of cells.
DDS Lecture Notes (draft)
-3-
Robert L. Baber, 1999 July
The main specific subjects covered in this course are:
 data items, data values, data types,
 ways of representing numerical values: positional notation; binary, decimal, hexadecimal
and other radix representation; integers; floating point,
 converting values between different representational systems, e.g. converting numbers
between different radix systems, between integer and floating point representation,
 data structures such as linear sequences (lists, vectors), one dimensional arrays, multidimensional arrays, lists of lists, stacks (LIFO queue), pipeline (FIFO queue), linked lists,
pointers, trees, heaps, Polish notation, graphs, hash tables, records, files,
 algorithms (recursive and non-recursive) for searching, sorting, evaluating expressions in
Polish notation, converting between different data structures, inserting and deleting
elements of the various data structures, dynamic storage allocation,
 analyzing data structures and algorithms, e.g. regarding their efficiency, time and space
complexity, etc. and
 examples of the practical application of the above, e.g. in compilers, operating systems
and application systems.
These subjects will not be presented strictly in the above sequence. Together with each data
structure the relevant algorithms will be studied and analyzed and examples considered.
Algorithms for converting between specific data structures will be examined at appropriate
times throughout the block.
DDS Lecture Notes (draft)
-4-
Robert L. Baber, 1999 July
2. Some basic definitions and terms
A data item consists of a name and a data value, or simply value. Often we will also
explicitly consider the set from which the values may be selected, in which case the data item
consists of a name, a value and a set. In the latter case, the value must be an element of the set
in question. A data item can also be thought of as a variable, as it has many of the
characteristics of a variable in mathematics.
A data value, or simply value, is one of the components of a data item (see the paragraph
above). A value is represented by a symbol or a (possibly empty) sequence of symbols. The
set of symbols used for representing values, e.g. in a particular computer system, is usually
rather limited. Ultimately, at the lowest level in a computer system, such a set typically
consists of only two elements, often represented in written form by the symbols 0 and 1.
It is sometimes important to distinguish between the meaning or interpretation of a value and
its representation. Within a computer system only the representation is of concern. The
designers and users of a computer system usually interpret the symbols or sequences of
symbols representing values in some way, that is, they attach meanings to such
representations of values. The connection between meaning and representation will be of
concern to both designers and users, but such interpretation takes place outside the computer
system itself. The computer system itself processes and manipulates symbols representing
data values according to fixed rules, without regard to the meanings humans attach to those
values or their representations.
For example, the number five may be represented by the single symbol 5, the symbol V, the
sequence of letter symbols “five”, the sequence of bits 101, the sequence of tertiary digits 12,
etc. All of these representations could be interpreted to mean the same thing, i.e. a certain
quantity of unspecified things.
A data type, also called abstract data type or ADT, consists of a set (of values) and a
collection of operations on that set of values.
An association between values is any sort of logical connection between them. An
association may be explicitly represented or it may be implicit, i.e. an association only in the
minds of humans such as the designers or users of the computer system in question. In either
case, such associations are the basis for data structures.
A (one dimensional) array is a sequence of values. Other terms are also used to refer to a
sequence of values, such as list and vector. (Note that the term “vector” here is used in the
computing sense, not the purely mathematical sense. These are related, but are not the same.)
Sometimes, within the context of a specific programming language, these different terms
imply different methods of implementing the sequence. For our purposes here, however, the
sequential ordering between elements of the array, list or vector is the essential characteristic
of interest. Mathematically, an array, a list, a vector or a sequence can be thought of as a
function which maps an integer to a value. The natural ordering of the integers (sequence
number, index) defines the ordering of the values in the sequence.
The ordering of the elements (values) in a sequence can serve to associate those elements
with one another. Another possibility is to associate elements in the same position of two or
more different sequences with each other.
Example: The following table expresses certain associations between the values “George”,
“Themba”, 22 and 25:
DDS Lecture Notes (draft)
-5-
Robert L. Baber, 1999 July
Sequence number
1
2
Name
George
Themba
Age
22
25
Here, the name “George” and the age 22 are associated with one another by being on the
same line (spatially) as well as both having the same sequence number (index). Similarly, the
name “Themba” is associated with the age 25 and the sequence number 2. In a computer
system, these data might be stored in two one dimensional arrays (e.g. Name and Age) such
that
Data item name
Name(1)
Name(2)
Age(1)
Age(2)
Data item value
George
Themba
22
25
The values “George” and 22 are associated by their common index value (1) in the two arrays
Name and Age. The values “Themba” and 25 are associated with one another in the same
way.
Note that the association between the arrays Name and Age (i.e. between each element of the
array Name and the corresponding element of the array Age) is implicit. Nevertheless, this
association serves to associate, and hence structure, the data contained in the two arrays.
Alternatively, the above associations can be represented by forming sequences (sometimes
called lists or vectors) of the values. For example, the above data collection could be written
as a list of lists:
( (George 22) (Themba 25) )
Here each person is represented by a list of data values (his or her name and age). The lists
for the individual persons are then put into a sequence to form a higher level list.
Alternatively, these associations between data values could be represented in the structure
( (George Themba) (22 25) )
in which a list of names is followed by a list of ages. Each name is associated with the
number (age) appearing in the corresponding position in the other list.
Still another alternative for structuring this data is in the form of two separate lists
(George Themba)
(22 25)
which are not explicitly associated with each other. They are instead only implicitly
associated by the way they are interpreted by the designers and users of the system in which
they are embedded, as in the case of the two arrays above. This implicit association will be
reflected in the way the two lists are referred to in the program and its documentation.
Presumably, this data will be interpreted to mean that some person named George is 22 years
old and that some other person named Themba is 25 years old. This interpretation of the
various data values and their associations is, however, external to the system and exists only
in the mind of the reader. Note that this interpretation is based on several implicit associations
not expressed in any way in the tables above:
DDS Lecture Notes (draft)
-6-
Robert L. Baber, 1999 July
 the usual meanings of the sequences of symbols “Name” and “Age” in the English
language
 the sequences of symbols “George” and “Themba” are common names of persons
 the sequences of symbols “22” and “25” are usually interpreted as decimal numbers
 ages of persons are usually measured in years and the numbers appearing above are
plausible ages of human beings
To someone with no knowledge of the English language and having lived only in a society in
which the names George and Themba are unknown, this interpretation would be neither
obvious nor natural. To people in ancient Rome, not even the numbers 22 and 25 would have
been recognizable as such; only the sequences of symbols “XXII” and “XXV” would have
conveyed these meanings to them.
A data structure is, in short, a collection of values and associations among them. We will
study specific types of data structures during this course.
In summary, the following terms were introduced in this section.
 data item
 data value, or simply value
 array, list, vector (sequence)
 meaning
 representation
 data type
 association (between values)
 interpretation (both of values and of associations)
 data structure
The student should review these terms, their meanings and the relationships between them.
These terms will recur frequently in this and subsequent topics.
DDS Lecture Notes (draft)
-7-
Robert L. Baber, 1999 July
3. Representing numerical data values
3.1. Classical systems for representing numerical values
In early human societies numbers were often represented (if at all) using systems in which
each symbol represented one specific number. A symbol was repeated and written together
with other symbols as necessary to represent any particular number. The numerical value of a
string of such symbols is determined by adding the values of the individual symbols or, in
some systems and in some cases, subtracting or even multiplying the symbols’ values. For
example, in the Roman number system, XIV = 10–1+5 = 14, and in the Attic system used in
Greece around 450 BC,  = 10+5+1+1+1 = 18, but  = 5*10 = 50 and  = 5*100 =
500 [Kline].
In such systems only a limited set of symbols is available. If a limit is placed on how many
times any one symbol may be repeated, there is, in turn, a limit to the numbers that can be
represented; larger numbers simply cannot be written in the system.
Today the Roman number system is perhaps the best known number system of these types. In
a sense it is a simple system, but certain operations are difficult to perform in it. Adding is not
too difficult (add, for example, XVI (16) and VI (6) by writing them together to obtain
XVVII which is normalized to XXII), but try multiplying these numbers together. Dividing is
even more complicated.
To overcome these limitations and shortcomings, e.g. to make it possible to represent any
number, no matter how large, and to simplify arithmetic operations, the Arabic positional
number system was introduced. In such a system the value associated with any symbol in a
string is the basic value of the symbol multiplied by a factor depending upon its position in
the number. That factor is usually an integral power of some number, called the radix of the
system in question. For example, in the decimal (radix ten) positional system currently in
widespread use throughout the world, the symbol 3 means either three (3*100), thirty (3*101),
three hundred (3*102), three thousand (3*103), etc., depending upon the position in which it
appears.
Some older number systems included both non-positional and positional characteristics. The
Akkadian system used in Babylonia around 2000 BC, for example, employed several
symbols which were used to write numbers up to 59 in a non-positional scheme. Positional
notation was then used to represent larger numbers, so that the system amounted to a
positional system with radix 60. Initially, this system had no symbol for zero, so strings of
symbols could be ambiguous, depending upon the position intended. E.g. the symbol for 1
followed by the symbol for 20 could mean either 80 (1*601+20) or 3620 (1*602+20). Spacing
was often used to indicate no symbol in a particular position, but this did not eliminate the
possiblity of misinterpreting the string of symbols. Much later a symbol indicating a space
(i.e. the lack of a numerical symbol in the position in question) was introduced. [Kline]
Today we would recognize such a separation symbol as the symbol for the numerical value
zero.
A system in use in Greece from about 300 BC used a similar mixture of non-positional and
positional notation to represent numbers. A symbol-additive scheme was used to indicate
numbers up to 999. Larger numbers were written in a positional notation. The system was, in
effect, a positional system with radix 1000, but, like the Akkadian system outlined above,
with a sequences of symbols in each position, rather than only a single symbol in each
position as in our current system. [Kline]
DDS Lecture Notes (draft)
-8-
Robert L. Baber, 1999 July
3.2. Positional notation
The system for writing numbers which has crystallized out of the experience gained with
using various systems over several thousand years has the following key characteristics:
 A limited set of symbols is used.
 Each symbol has its own inherent numerical value. These values are zero, one, two, etc.
 A number is represented by a sequence of symbols. Each sequence is bounded in length
(“finite”) but no upper limit is placed on that length.
 A positional value is associated with each position in the sequence. This value is the radix
of the number system raised to the power corresponding to the position in the sequence.
These positions are counted from right to left, beginning with zero.
 The number of symbols used is the same as the radix of the system.
 The numerical value represented by any particular symbol is the product of its own
inherent value and the value associated with its position in the number string.
 The value represented by the sequence of symbols is the sum of the numerical values
represented by the individual symbols.
The radix of the system in question is a positive integer often represented by R. Thus, a
particular positional number system is characterized by its radix R and a set of R symbols
representing the values 0, 1, 2, … R-1.
The value of the number represented by the sequence of symbols
sn sn–1 ... s1 s0
is defined to be
n
i=0 val(si) * Ri
where the function val maps each symbol to its own inherent numerical value. Usually each
symbol is interpreted to be that value itself, so the above can also be written as
n
i=0 si * Ri
Such a positional notational system can represent any non-negative integer and only nonnegative integers. Negative integers are usually represented by appending another special
symbol, e.g. “–”, to the sequence of symbols. Usually the minus sign is written before the
sequence of symbols representing the number, but sometimes after.
The above system can be extended easily to non-integer values by allowing the sequence to
continue to the right and by placing another special symbol (e.g. “.” or, in some countries, “,”
or “-”, etc.) to separate the integer part from the fractional part of the sequence. For example,
the value of the number represented by the sequence of symbols
sn sn–1 ... s1 s0 . s–1 s–2 ... s–(m–1) s–m
is defined to be
n
i=–m si * Ri
The system in common use today in the world is the positional notational system with radix
ten, called the decimal system. It is often said that ten was chosen because human beings
DDS Lecture Notes (draft)
-9-
Robert L. Baber, 1999 July
normally have ten fingers. However, with ten fingers one can represent 11 different symbols
(in a non-positional scheme) by using no fingers for zero, so one might claim that if this had
been the reason for choosing a radix, 11 would or should have been chosen instead of 10.
Probably the presumed reason is correct but it was not applied completely consequently.
Note that the above description of positional notation defines not one system, but a family of
systems. Each system in the family is characterized completely by its radix R.
Note also that if R=1, only one symbol may be used and its own inherent numerical value
must be zero. Thus, in the positional number system with radix 1, only the value zero can be
represented. This system is, therefore, of no practical interest and will not be considered
further. Only systems with radix 2 or greater are of significance. Below we will deal only
with these.
The values of the symbols used are 0, 1, ... R–1. Given a number and a radix, the
representation of that number in that radix system is unique except for leading left zeroes
before and trailing right zeroes after a “.” separating symbol. I.e., one and only one sequence
of symbols not beginning with a zero and not ending with a zero to the right of a “.” symbol
exists which represents the given number for the given radix.
Sample exercise: Consider the sequence of symbols 3245. If this is written in the octal (R=8)
system, what decimal number does it mean? If it is written in the system of radix 6? In the
hexadecimal (R=16) system? In the system of radix 4?
Sample exercise: Consider the number 110101 written in the binary (R=2) system. What
decimal number does it mean?
Sample exercise: Consider the decimal number 34. How would it be written in the binary
(R=2) system?
3.3. Converting integers between positional number systems with different radices
We will begin by considering the problem of converting integers between two different radix
systems. The algorithms can be extended to include fractional numbers.
Consider first some examples of such conversions.
Example: (binary to decimal, computing in decimal) Let 10110 be the binary
representation (R=2) of some number. We can calculate the value, and hence the decimal
representation, of this number by applying the definition above of the value represented by a
sequence of symbols. We must only take care to perform all our calculations in the decimal
system, the system in which we want the answer. Applying the definition
n
i=0 si * Ri
of the value to the specific binary number 10110, we have
n
i=0 si * Ri
=
0*20 + 1*21 + 1*22 + 0*23 + 1*24
=
0*1 + 1*2 + 1*4 + 0*8 + 1*16
=
DDS Lecture Notes (draft)
- 10 -
Robert L. Baber, 1999 July
0 + 2 + 4 + 0 + 16
=
2 + 4 + 16
=
22
This is the value of the original number expressed in decimal notation.
The same computation can be organized differently and the individual steps performed in a
different order. This alternative is in many cases computationally more efficient because it
effectively generates all the various powers of R in one single process while at the same time
multiplying them by the corresponding si. It is based on the observation that
n
i=0 si * Ri = ... ((sn*R + sn–1)*R + sn–2) ... + s0
e.g. when n=4
4
i=0 si * Ri
=
s4 * R4 + s3 * R3 + s2 * R2 + s1 * R1 + s0 * R0
=
(((s4*R + s3)*R + s2)*R + s1)*R + s0
Using this approach, the above conversion of 10110 to decimal would be done as follows.
We begin with the first binary symbol on the left (s4, which is 1), multiply it by R (2) to
obtain 2 and add the next symbol (s3, which is 0) to obtain 2 as the result of the first iterative
step.
We then multiply this 2 by R, obtaining 4, and add the next symbol (s2, which is 1) to obtain
5 as the result of this second step in the computation.
We then multiply this 5 by R, obtaining 10, and add the next symbol (s1, which is 1) to obtain
11 as the result of this third step in the computation.
We then multiply this 11 by R, obtaining 22, and add the next symbol (s 0, which is 0) to
obtain 22 as the result of this fourth step in the computation. Having reached and added s0,
the last symbol, into the accumulated result, we are finished. The answer is 22.
Example: (decimal to binary, computing in binary) Consider the decimal number 22. We
can convert it to its binary representation in exactly the same way used above if we perform
all calculations in binary, the target system. We must first convert each decimal symbol
(digit) into its binary representation. We must also write ten, the radix of the decimal system,
in its binary form. The decimal digit 2 is, in binary, written as 10. The binary representation
of the number ten is 1010.
Proceeding in the same way outlined above, we begin with the left most digit 2, multiply its
binary representation (10) by R (ten, or 1010 in binary) to obtain 10100 and add the next
symbol (2, or 10 in binary) to obtain 10110. Having reached and added the right most digit
into the accumulated result, we are finished. The answer is 10110.
DDS Lecture Notes (draft)
- 11 -
Robert L. Baber, 1999 July
Example: (decimal to binary, computing in decimal) Again consider the decimal number
22. We wish to convert it to binary, but this time performing the calculations in decimal (the
source system), not binary (the target system). We wish to find a sequence of binary symbols
tk tk–1 ... t1 t0 such that
k
i=0 ti * 2i = 22
where the symbol 2 and the number 22 are understood to be in decimal notation.
We can take the 0th term out of the series and rewrite
k
i=0 ti * 2 i
as
k
i=1 ti * 2i + t0 * 20
=
k
i=1 ti * 2i + t0
Notice that each term still remaining in the series contains at least one factor of 2. The entire
series is, therefore, divisible by 2. The last term, t0, is a binary symbol which must be less
than 2 and is, therefore, not divisible by 2. In other words, if we divide the original number
by 2, the remainder will be the right most binary symbol in the representation of the number.
Dividing 22 by 2 gives 11 with a remainder of 0. Therefore, t0 = 0.
We must find the remaining binary symbols such that
k
i=1 ti * 2 i–1 = 11
where 11 is the quotient resulting above from dividing 22 by 2.
This remaining task is basically the same as the original one. Dividing 11 by 2 results in a
quotient of 5 and a remainder of 1. Therefore, t1 = 1.
The same process is repeated: 5 divided by 2 yields 2 and a remainder of 1, so t2 = 1.
The process is repeated again: 2 divided by 2 yields 1 with a remainder of 0, so t3 = 0.
The process is repeated again: 1 divided by 2 yields 0 with a remainder of 1, so t4 = 1.
The remaining quotient is zero, so repeating the above process will give rise to only leading
zeroes, which can be omitted. In other words, we are finished. We have converted the
decimal number 22 into its binary representation 10110 (t4 t3 t2 t1 t0).
[end of example]
From these examples we generalize and conclude that to convert a number from radix system
Rs to radix system Rt, one can either
DDS Lecture Notes (draft)
- 12 -
Robert L. Baber, 1999 July
 perform all calculations in the target radix system Rt and, by applying the definition of the
numerical value of a sequence of symbols, repetitively multiply by Rs and add the next
symbol or
 perform all calculations in the source radix system Rs and repetitively divide by Rt, the
remainders becoming the symbols in the target system, from right to left.
The following table summarizes these processes.
Calculations performed in
Process for converting an integer
target system Rt
repetitively multiply by Rs and add the next symbol,
final sum is result
source system Rs
repetitively divide by Rt, remainders generate result
Table: Converting integers from one radix system to another
Sample exercise: Convert the octal (R=8) number 377 to decimal notation. To binary. To
hexadecimal (R=16), using the symbols A, B, C, D, E and F to represent the values ten,
eleven, twelve, thirteen, fourteen and fifteen respectively. Perform each computation in both
the source and in the target system.
Sample exercise: Convert the decimal number 134 to binary, octal and hexadecimal notation.
Questions: What similarities do you notice between the binary, octal and hexadecimal
representations of a number? Why do these similarities arise? How can this observation be
used to facilitate the conversion of numbers between these radix systems?
3.4. Converting fractions between positional number systems with different radices
One obvious way to convert a fraction or a number with a fractional part such as
sn sn–1 ... s1 s0 . s–1 s–2 ... s–(m–1) s–m
is to rewrite this as
sn sn–1 ... s1 s0 s–1 s–2 ... s–(m–1) s–m * Rs–m
and convert the sequence of symbols, which now represents an integer, as described above.
The resulting representation in the target radix system is then multiplied by Rs –m, the
multiplication being performed in the target radix system Rt, of course.
This method is obviously applicable only to numbers whose original representation is of
bounded length. However, because in practice all computation is limited to bounded
representations, the above method can generally be applied in practice after truncating the
original sequence of symbols at an appropriate place on the right.
It is possible, of course, to convert the fractional part of a number from one radix system to
another directly, e.g. in ways comparable to those illustrated in section 3.3 above.
Example: (binary to decimal, computing in decimal) Let 0.1011 be the binary
representation (R=2) of some number. By performing the calculations in decimal arithmetic
DDS Lecture Notes (draft)
- 13 -
Robert L. Baber, 1999 July
we can calculate the value, and hence the decimal representation, of this number by applying
the definition in section 3.2 above of the value represented by a sequence of symbols.
The appropriate form of the definition is that the value represented by the sequence of
symbols
0 . s–1 s–2 ... s–(m–1) s–m
is
–1
i=–m si * Ri
which is equivalent to
m
i=1 s–i * R–i
Observing that
m
i=1 s–i * R–i = ( ... ((s–m/R + s–(m–1))/R + s–(m–2)) ... + s–1)/R
suggests an algorithm structurally like one encountered earlier: repetitively divide the already
accumulated value by R and add the next symbol (here proceeding from right to left). This is
basically the reverse of that earlier procedure for converting an integer to a different radix
system; here we divide instead of multiply and proceed from right to left instead of left to
right.
E.g. when m=4, the above equation becomes
4
i=1 s–i * R–i = (((s–4/R + s–3)/R + s–2)/R + s–1)/R
Using this approach, the above conversion of 0.1011 to decimal would be done as follows,
where all arithmetic will be done in decimal.
We begin with the first binary symbol on the right (s–4, which is 1) and divide it by R (2) to
obtain 0.5 as the result of the first iterative step.
We then add the next symbol (s–3, which is 1), obtaining 1.5, and divide by R to obtain 0.75
as the result of this second step in the computation.
We then add the next symbol (s–2, which is 0), obtaining 0.75, and divide by R to obtain
0.375 as the result of this third step in the computation.
We then add the next symbol (s–1, which is 1), obtaining 1.375, and divide by R to obtain
0.6875 as the final result.
Example: (decimal to binary, computing in binary) Consider the decimal number 0.6875.
We can convert it to its binary representation in exactly the same way used above if we
perform all calculations in binary, the target system. We must first convert each decimal
symbol (digit) into its binary representation. We must also write ten, the radix of the decimal
system, in its binary form. The decimal digits 6, 8, 7 and 5 are, in binary, written as 110,
1000, 111 and 101 respectively. The binary representation of the number ten is 1010.
DDS Lecture Notes (draft)
- 14 -
Robert L. Baber, 1999 July
Proceeding in the same way outlined above, we begin with the right most digit 5 and divide
its binary representation (101) by R (ten, or 1010 in binary) to obtain 0.1.
We then add the next symbol (7, or 111 in binary), obtaining 111.1, and divide this 111.1 by
R (1010 in binary) to obtain 0.11 as the result of this step.
We then add the next symbol (8, or in binary 1000), obtaining 1000.11, and divide this
1000.11 by R (1010 in binary) to obtain 0.111.
We then add the next symbol (6, or in binary 110), obtaining 110.111, and divide this
110.111 by R (1010 in binary) to obtain 0.1011 as the final result.
Example: (decimal to binary, computing in decimal) Again consider the decimal number
0.6875. We wish to convert it to binary, but this time performing the calculations in decimal
(the source system), not binary (the target system). We wish to find a binary representation
0 . t–1 t–2 ... t–(m–1) t–m
such that
m
i=1 t–i * 2–i = 0.6875
where the symbol 2 and the number on the right hand side are understood to be in decimal
notation.
The first term in the series above is t–1 * 2–1. If we were to multiply the series by 2, this first
term would become simply t–1, a non-negative integer. The rest of the series would still be a
fraction. We could then deduce a value for t–1.
This observation suggests multiplying both sides of the equation above by 2 to obtain
m
i=1 t–i * 2–(i–1) = 1.375
Taking the first term out of the series and writing the number on the right hand side as the
sum of an integer and a fraction, we obtain
m
t–1 + i=2 t–i * 2–(i–1) = 1 + 0.375
=
m–1
t–1 + j=1 t–(j+1) * 2–j = 1 + 0.375
This suggests requiring that t–1 be 1 and that
m–1
j=1 t–(j+1) * 2–j = 0.375
This decomposition of a real number into an integer and a fractional part on each side of the
equation is unique. Why?
Finding the remaining t–(j+1) is basically the same task as our original one above. Thus we can
find the binary representation of a fraction by multiplying the fraction to be converted by 2
(the target radix), taking the integer part of the product as the next bit (the target symbol) and
DDS Lecture Notes (draft)
- 15 -
Robert L. Baber, 1999 July
repeating this process on the remaining fraction. When the remaining fraction is zero, the
procedure is finished.
Question: Must this process terminate? Why or why not?
Thus, to convert 0.6875 from decimal to binary, computing in decimal, we begin by
multiplying 0.6875 by 2, obtaining 1.375. The first bit t–1 is, therefore, 1. The remaining
fraction is 0.375.
Next, we multiply 0.375 by 2, obtaining 0.75. The next bit t–2 is, therefore, 0. The remaining
fraction is 0.75.
Next, we multiply 0.75 by 2, obtaining 1.5. The next bit t–3 is, therefore, 1. The remaining
fraction is 0.5.
Next, we multiply 0.5 by 2, obtaining 1. The next bit t–4 is, therefore, 1. The remaining
fraction is 0 and we are therefore finished.
The result is 0.1011.
[end of example of converting 0.6875 from decimal to binary, computing in decimal]
Notice that the processes for converting integers and fractions from one radix system to
another are, in a certain sense, the opposites of each other.
Calculations performed in
Process for converting
an integer
Process for converting a
fraction
target system Rt
from left to right
repetitively multiply by Rs
and add the next symbol,
final sum is result
from right to left
repetitively divide by Rs
and add the next symbol,
final sum divided by Rs is
result
source system Rs
repetitively divide by Rt,
remainders generate result
from right to left
repetitively multiply by
Rt, integer parts of
products generate result
from left to right
Table: Converting integers and fractions from one radix system to another
Sample exercise: Convert the binary number 0.1011 to decimal, computing in binary.
Sample exercise: What is the binary representation of one tenth? What are some of the
implications of the binary representation of one tenth?
Additional exercises: Convert various numbers, both integers and fractions, between binary,
octal, decimal, hexadecimal and other number systems such as radix 3, radix 5, etc. systems.
3.5. Representing integers in a computer memory
Typically integers are represented in a computer memory using the positional notation introduced in section 3.2 and a fixed number of symbols in the radix system chosen (e.g. decimal
DDS Lecture Notes (draft)
- 16 -
Robert L. Baber, 1999 July
digits, binary bits, etc.). The most straightforward interpretation of each possible combination
of symbols is as an unsigned (non-negative) integer.
Example: The following table illustrates such an interpretation for a binary system (R=2)
with three bits.
Bit pattern
Numerical value in
binary
decimal
111
+111
+7
110
+110
+6
101
+101
+5
100
+100
+4
011
+011
+3
010
+010
+2
001
+001
+1
000
(+)00
(+)0
Table: Representing unsigned (non-negative) integers
For many applications such a system is not adequate, as negative values are also required.
Various schemes can be used to represent integer values which can be positive or negative. In
the specific case of binary representation, three of the most obvious or common schemes are:
 One bit is interpreted as the sign (+ or ) and the remaining bits are interpreted as the
magnitude of the number (signed magnitude method).
 One bit is interpreted as the sign (+ or ) and the remaining bits are interpreted as follows.
If the sign is positive, the remaining bits represent the magnitude of the number directly. If
the sign is negative, the one’s complement of the remaining bits represent the magnitude
of the number (one’s complement method).
 One bit is interpreted as the sign (+ or ) and the remaining bits are interpreted as follows.
If the sign is positive, the remaining bits represent the magnitude of the number directly. If
the sign is negative, the two’s complement of the remaining bits represent the magnitude
of the number (two’s complement method).
The following table illustrates these three different interpretations for a binary system (R=2)
with three bits.
DDS Lecture Notes (draft)
- 17 -
Robert L. Baber, 1999 July
Signed magnitude,
meaning in
One’s complement,
meaning in
Two’s complement,
meaning in
binary
decimal
binary
decimal
binary
decimal
111
11
3
()00
()0
01
1
110
10
2
01
1
10
2
101
01
1
10
2
11
3
100
()00
()0
11
3
100
4
011
+11
+3
+11
+3
+11
+3
010
+10
+2
+10
+2
+10
+2
001
+01
+1
+01
+1
+01
+1
000
(+)00
(+)0
(+)00
(+)0
(+)00
(+)0
Bit pattern
Table: Some different ways of interpreting signed integers
Note that both the simple signed method and the one’s complement system have two representations for zero, shown above as (+)0 and ()0. Mathematically and numerically there is,
of course, no difference; a sign associated with the value zero has no meaning or consequence. Thus, these systems waste one possibility of representing a number. The representational difference between +0 and 0 can also sometimes lead to confusion (e.g. what result
should a test for equality yield?). For these and other reasons, these systems are no longer
frequently implemented, although they can sometimes be found in practice.
Note also the break between 011 and 100 in the sequence of the numbers represented.
The one’s and the two’s complement systems form, in effect, cyclical systems with only one
break in sequence, i.e. between 111 and 000. Because of their cyclical nature, arithmetic is
logically somewhat simpler in these two systems than in the simple signed method.
For the reasons mentioned above, the unsigned and the two’s complement methods are probably the ones most commonly implemented today. But it is important to note that they are not
the only possible ones, and for a special application some other scheme may be better. E.g., a
particular application may be best served by a representational scheme in which a much
larger range of positive integers is required than of negative integers.
A particular method of fixed length for representing integers in a computer memory can be
specified by stating (1) the radix, (2) the number of symbols in the representation and (3) the
function to be used for interpreting each sequence of symbols as an integer (e.g. whether
one’s or two’s complement).
Still another method of representing negative numbers should be mentioned here because it is
often used within binary floating point systems. Structurally it is similar in some respects to
the unsigned integer and the two’s complement methods described above. Using this method
DDS Lecture Notes (draft)
- 18 -
Robert L. Baber, 1999 July
a signed number sn is represented by the unsigned number un = sn + bias, where bias is a
suitable non-negative integer. This method is called the bias or offset method.
Example: The following table illustrates several different specific instances of such a bias or
offset system for representing signed numbers using three bits. The values of the various bit
patterns are shown in decimal notation.
Bit pattern
Value as integer
unsigned
signed,
bias = 4
signed,
bias = 3
signed,
bias = 1
signed,
bias = 0
111
+7
+3
+4
+6
+7
110
+6
+2
+3
+5
+6
101
+5
+1
+2
+4
+5
100
+4
0
+1
+3
+4
011
+3
1
0
+2
+3
010
+2
2
1
+1
+2
001
+1
3
2
0
+1
000
0
4
3
1
0
Table: The bias or offset method for representing signed integers
Note that the unsigned integer representation is just a special case of the bias method with the
bias equal to zero.
Notice also the cyclical structure of biased systems for representing signed integers — just
like that of the one’s and especially of the two’s complement systems. The major difference
is the position of the break, here between the top and the bottom of the list, i.e. at the point in
the cycle where all 1s change to all 0s.
Exercise: Extend the above table for a system using 4 bits. 8 bits.
Exercise: Given the biased representations bx and by for two numbers x and y, how would
you calculate the biased representation bsum for their sum x+y? the biased representation
bdiff for their difference xy? What conditions must the arithmetic system and/or intermediate results satisfy in order to ensure that the results of your calculations are correct?
3.6. Floating point systems
Numbers in positional notation form as presented in section 3.2 above are represented most
simply in sequences of fixed length, whereby both the integer part and the fractional part of
the number are of predefined lengths. I.e., the position of the decimal (or binary, etc.) point is
fixed; hence the term “fixed point”. This, of course, imposes limits on the range of numbers
which can be represented. For many scientific and technical computations these limits can
DDS Lecture Notes (draft)
- 19 -
Robert L. Baber, 1999 July
have severe consequences. For such computations a more flexible scheme for representing
numbers is required.
A common and useful solution to this problem is floating point representation. This relies on
the fact that any number x can be represented by three numbers s, m and e with the following
relationship:
x = (1)s * m * Re
where R is the radix of the number system being used, e is an integer, s is either 0 or 1, and m
is a non-negative number. The number m is called the “mantissa” or “significand”, s is the
“sign”, and e is called the “exponent”. The exponent is typically a signed integer, i.e. an
integer which can take on positive and negative values.
The value of e effectively defines the position of the “decimal” point in the representation of
x, hence the name “floating point”.
Many such representations exist for any number x. If the value of m is restricted to the range
1m<R
(except for the representation of 0), m can be represented in fixed point format without
imposing any bound on the values of x which can be represented. If the range of values of e is
also restricted, a corresponding limit is placed on the values of x which can be represented,
but this limit is much less severe than if x itself were to be represented in fixed point format.
If R=2, i.e. if the binary system is used for representing the numbers, then the restriction
1m<R
implies that the integer part (the high order bit) of m is always 1. Since this is a constant, it
need not be stored, saving one bit in memory. Perhaps more importantly, greater precision is
achieved without increasing the number of bits which must be stored. Advantage has been
taken of this possibility in some implemented systems.
The restriction 1  m < R is arbitrary. Any corresponding restriction can be imposed with the
same effect. A number of systems have been implemented with the restriction
R–1  m < 1
i.e. m is a fraction and its first digit (or bit, etc.) is not zero.
A floating point representation which fulfils a specified restriction such as R–1  m < 1 is said
to be in normalized form.
A floating point system (together with its corresponding arithmetic operations) is a very
useful approximation to the real number system with its arithmetic operations, but floating
point arithmetic does not share all of the important characteristics of the corresponding
mathematical operations. This has important consequences when analyzing the behaviour of
floating point arithmetic. Total accuracy is not generally achieved and should never be
assumed.
Example: Consider for the purposes of illustration a small floating point system with a two
bit mantissa, a one bit sign and a three bit exponent, in which the exponent is represented
using the bias method. The following table shows the non-negative values (in decimal
DDS Lecture Notes (draft)
- 20 -
Robert L. Baber, 1999 July
notation) which can be represented in this floating point system. The corresponding negative
values are also representable but are not shown.
exponent (biased representation, bias = 4)
000
001
010
011
100
101
110
111
mantissa
decimal
values
4
3
2
1
0
+1
+2
+3
1.1
1.5
0.09375
0.1875
0.375
0.75
1.5
3
6
12
1.0
1.0
0.0625
0.125
0.25
0.5
1
2
4
8
0.1
0.5
0.03125
0.0625
0.125
0.25
0.5
1
2
4
0.0
0
0
0
0
0
0
0
0
0
Table: Non-negative values representable in a small floating point system
(two bit mantissa, three bit exponent)
The values representable in this floating point system are:
0, 0.03125, 0.0625, 0.09375, 0.125
0.1875, 0.25
0.375, 0.5
0.75, 1
1.5, 2
3, 4
6, 8
12
and their negatives.
Notice how the precision (the inverse of the difference between two neighboring values)
decreases with increasing value. This is a consequence of the definition of the value (see
formula above) and is, therefore, a characteristic of all floating point systems. The precision
is determined by the value of the exponent, and as it increases, the precision decreases.
Because the precision varies with value, the attainable accuracy of arithmetic operations
varies.
Floating point arithmetic is only an approximation to mathematical arithmetic. For example,
when two representable values are added, the most accurate result possible is the true
mathematical result rounded to the closest representable value. Sometimes floating point
arithmetic (e.g. floating point addition ) is exact, for example
0.125  0.25 = 0.375, and 0.125 + 0.25 = 0.375
but sometimes it is not exact, for example
3  1.5 = 4, but 3 + 1.5 = 4.5
Floating point addition is sometimes associative, for example
(0.75  1)  1.5 = 2  1.5 = 4 and
DDS Lecture Notes (draft)
- 21 -
Robert L. Baber, 1999 July
0.75  (1  1.5) = 0.75  3 = 4
but sometimes it is not associative, for example
(6  0.75)  0.75 = 6  0.75 = 6, but
6  (0.75  0.75) = 6  1.5 = 8
[end of example]
In the design of a floating point system a number of detailed decisions must be made, e.g.
regarding the length of the mantissa, the range of the exponent, rounding, representing results
of arithmetic operations which are too large or too small to be represented, etc. Standards
have been agreed upon for some of these aspects of floating point systems, e.g. [IEEE 1985,
1987].
Typically, at least 8 bits are used to represent the exponent and at least 4 bytes (32 bits) for
the entire floating point number. The IEEE standard for binary floating point arithmetic, for
example, specifies a 4 byte format for single precision and an 8 byte format for double
precision floating point numbers. In these formats, the high order bit of the mantissa is not
stored; its value is implied by the exponent. Also, this standard calls for using the bias
method to represent the exponent. [IEEE 1985]
3.7. Precision and accuracy
In section 3.6 above on floating point representation of numbers, it was pointed out that the
precision and accuracy with which a number can be represented varies with the value being
represented. It is often impossible to represent a desired numerical value precisely. Many
such values can be represented only approximately.
Complete precision is not achievable when one attempts to represent an arbitrary element of a
continuous space (e.g. a real number or a rational number) by an element of a finite space,
e.g. using a sequence (of bounded length) of symbols selected from a finite set.
In some contexts exact (i.e. completely precise and accurate) representation is possible. This
is typically the case when representing integers selected from a finite set. Within the limits set
by the bounds of the set of integers in question, arithmetic will also typically be exact, i.e.
will yield a mathematically correct and precisely represented result.
Note that floating point representation and arithmetic cannot in general be assumed to give
exact results. Correspondingly, one may not, in general, assume convenient mathematical
properties such as associativity. Only if one restricts the set of values actually used in a computation (e.g. to integers within a certain range) can one sometimes ensure that all results of
floating point operations are exact. In such cases, the burden of proof clearly lies on the
designer of the program in question.
Documentation on specific programming language systems typically contains information on
the precision and accuracy of the various implemented methods for representing numbers.
Integer types will normally be exact within the range of values represented. Floating point
types will generally not give exact results.
The documentation of the programming language Scheme [Kelsey] distinguishes explicitly
between exact and inexact numbers, independently of the particular representation used. See
section 6.2.2. Exactness in [Kelsey].
DDS Lecture Notes (draft)
- 22 -
Robert L. Baber, 1999 July
4. Representing non-numerical data values
In chapter 3 above numbers were represented by sequences of symbols, each symbol
representing an integer between 0 and R1 inclusive, where R is the radix of the number
system in question, or separating the integer and fractional parts of the number.
More generally, any data value can be represented by a sequence of symbols selected from
some finite set. The sequence may be of any length, including 0 (the empty sequence).
The need to represent text (sequences of letters, digits and other special symbols, e.g.
punctuation marks) or other non-numerical data in a computer memory arises often and in
many types of applications. Because computer hardware memories are typically organized as
sequences of cells, each cell storing a byte consisting of eight bits, it has become usual to
represent each character (symbol) in a body of text by a pattern of eight bits, i.e. in one byte
(cell) in such a memory. Because 28 = 256, any one of 256 different characters can be
represented in one byte of computer memory.
4.1. Standards for representing individual characters
Earlier, 5 and 6 bit representations of such characters were standard, e.g. in teletype networks
and in computer systems. With 6 bits, 26 = 64 characters can be represented, which is not
enough for both lower and upper case letters in the English alphabet, the ten digits and a
number of special characters (e.g. punctuation marks). The ASCII 7 bit encoding standard
was, therefore, defined. The following is an extract of the ASCII standard code table:
DDS Lecture Notes (draft)
- 23 -
Robert L. Baber, 1999 July
Binary representation
Hexadecimal
representation
Decimal
representation
Character(s)
000 0000 - 001 1111
00 - 1F
0 - 31
control codes such as carriage
return, line feed, other teletype
functions, etc.
010 0000
20
32
the blank character (space)
010 0001 - 010 1111
21 - 2F
33 - 47
special characters (e.g. *, +, -)
011 0000
30
48
digit 0
011 0001 - 011 1000
31 - 38
49 - 56
digits 1 - 8
011 1001
39
57
digit 9
011 1010 - 100 0000
3A - 40
58 - 64
special characters
100 0001
41
65
capital letter A
100 0010 - 101 1001
42 - 59
66 - 89
capital letters B - Y
101 1010
5A
90
capital letter Z
101 1011 - 110 0000
5B - 60
91- 96
special characters
110 0001
61
97
lower case letter a
110 0010 - 111 1001
62 - 79
98 - 121
lower case letters b - y
111 1010
7A
122
lower case letter z
111 1011 - 111 1111
7B - 7F
123 - 127
special characters
Table: Extract of the ASCII standard encoding table
Several eight bit extensions have been defined to the ASCII encoding table. They define
codes for letters appearing in European languages other than English (e.g. á, ä, å, æ, ç, é, ð, ñ,
ö, ù, ü, þ, etc.).
4.2. Representing a value as a sequence of characters or symbols
As stated in section 4 above, a data value can be represented by a sequence of characters or
symbols. Assuming that the starting position of the sequence in computer memory is
indicated in some appropriate way, a way of indicating the final position of the data value in
question is still needed. Two common methods are
 to use a special symbol (a terminating character) to delimit the end of the sequence and
 to indicate the length of the sequence in the form of a number, either stored at the
beginning of the sequence or somewhere else, e.g. together with the indication of the
starting position of the sequence.
For example, the blank character can be used to mark the end of data value. In effect, this was
the method used in section 3 above. Any other character not otherwise used in the sequence
of symbols representing the data value in question can be used as an end marker.
DDS Lecture Notes (draft)
- 24 -
Robert L. Baber, 1999 July
Another common method uses a length indicator placed at the beginning of the sequence of
symbols representing the data value. In many systems, a single 8 bit (one byte) integer is
used. This limits the length of possible sequences to 255 (if 0 is used to represent an empty
sequence). If this poses an unreasonable limitation, two or more bytes can be used for the
length. Schemes for representing such a length without imposing any upper limit exist. One
possibility involves prefixing the sequence by its length in some radix system, prefixing that
length by the length of the length, etc., until a predefined length (e.g. 1) is reached [Baber].
These and still other ways for representing such sequences in a computer memory utilize data
structures of more general applicability, bringing us to the subject of representing data
structures in a computer memory.
Exercise: Determine the binary representation for the letter “c”. the sequence of letters “cde”.
the sequence “f2xyz”. In each case use the 7-bit ASCII representation defined in the above
table.
DDS Lecture Notes (draft)
- 25 -
Robert L. Baber, 1999 July
5. Arrays, lists, vectors, sequences
As mentioned in chapter 2 above, an array, a list or a vector is a sequence of values or data
items, grouped together logically and usually also physically in a computer memory. The
main structural aspect of an array, a list or a vector is its linear sequence, often expressed by
physical adjacency. This structure is the logical basis of most computer hardware memories.
5.1. Computer memory
Physically a computer memory typically consists of a linear sequence of cells. Each cell is
identified by a numerical address and can store a symbol selected from a fixed set of symbols. In most modern computing systems, a cell consists of 8 bits (also called a byte), but
sometimes a multiple of 8 bits. The term word is sometimes used as a synonym for cell,
especially in older computer systems and in systems with a cell size larger than 8 bits.
The addresses associated with the cells of a computer memory are typically integers beginning with 0. The following diagram illustrates the structure of a typical computer memory.
Contents of the cells are shown, interpreted as characters in a defined set of symbols.
Address
Contents
0
d
1
g
2
K
3
1
4
2
5
3
6
F
7
&
8
%
9
@
10
?
11
/
12
+
...
...
The electronic hardware comprising the memory is constructed in such a way that the address
is the access key to the contents of the corresponding cell. That is, if the address 10 is sent to
the memory circuitry, the character ? (more precisely, the bit representation of the character
DDS Lecture Notes (draft)
- 26 -
Robert L. Baber, 1999 July
?) will be returned. If we consider the entire memory to be an array which we call cmem, then
the values of various elements of this array are:
cmem(0) = “d” (the symbol or character d)
cmem(3) = “1” (the symbol or character 1)
cmem(10) = “?” (the symbol or character ?)
cmem(12) = “+” (the symbol or character +)
etc.
The cells of the computer memory are structured in a linear sequence by the linear sequence
inherent in their addresses, which are consecutive non-negative integers. This may or may not
correspond to the physical locations in which the values are actually stored.
Similarly, the individual elements of a one-dimensional array are structured in a linear sequence by the linear sequence inherent in their indices, which are typically integers (and
usually consecutive integers). For example, our model above of a computer memory as an
array called cmem is a one-dimensional array with indices selected from the set of consecutive integers beginning with 0.
5.2. One-dimensional arrays and their representation in a computer memory
A one dimensional array is typically stored (represented, mapped) in a computer memory in a
sequence corresponding to the sequence defined by the values of the indices of the various
elements of the array.
Example: Consider an array called Age with 3 elements. If the computer memory has free
available space beginning at memory address 75 and if each element in the array Age is an
unsigned integer stored in one cell (byte) of the computer memory, the array Age might be
stored in the computer memory as follows:
Addresses in computer memory
Contents
Element of array
...
...
75
22
Age(1)
76
25
Age(2)
77
75
Age(3)
...
...
Array
Age
The address in computer memory of any value in the array Age can obviously be calculated
in the following way:
address in computer memory = index in array Age + 74
Expressed differently, the value of the array element Age(i) is stored in cmem(i+74), for i =
1, 2, 3.
Consider another array called Name, also with 3 elements. If each element in the array Name
is a sequence of up to ten characters, each of which is stored in one cell (byte) of the computer memory, the array Name might be stored in the computer memory as follows:
DDS Lecture Notes (draft)
- 27 -
Robert L. Baber, 1999 July
Addresses in computer memory
Contents
Element of array
...
...
75
22
Age(1)
76
25
Age(2)
77
75
Age(3)
78-87
George
Name(1)
88-97
Themba
Name(2)
98-107
Gogo
Name(3)
...
...
Array
Age
Name
The address in computer memory of the first cell of any value in the array Name can be
calculated in the following way:
starting address in computer memory = (index in array Name - 1) * 10 + 78
Expressed differently, the value of the array element Name(i) is stored in
cmem((i1)*10+78), cmem((i1)*10+79), ... cmem((i1)*10+87),
for i = 1, 2, 3.
If a third array called City is added to this data structure, it might be stored in the computer
memory as follows:
DDS Lecture Notes (draft)
- 28 -
Robert L. Baber, 1999 July
Addresses in computer memory
Contents
Element of array
...
...
75
22
Age(1)
76
25
Age(2)
77
75
Age(3)
78-87
George
Name(1)
88-97
Themba
Name(2)
98-107
Gogo
Name(3)
108-122
Johannesburg
City(1)
123-137
Pretoria
City(2)
138-152
Nelspruit
City(3)
...
...
Array
Age
Name
City
The address in computer memory of the first cell of any value in the array City can be calculated in the same way, but with different constants in the formula:
starting address in computer memory = (index in array City - 1) * 15 + 108
The value of the array element City(i) is stored in
cmem((i1)*15+108), cmem((i1)*15+109), ... cmem((i1)*15+122),
for i = 1, 2, 3.
A one dimensional array reflects the linear structure of an array and, therefore, lends itself
well to representing such a data structure and manipulating it simply and straightforwardly.
The following exercises deal with searching, inserting a new item in an already sorted sequence and sorting.
In some programming languages (such as C) formulae for addressing elements in arrays must
be explicitly written into expressions in program statements referring to array elements. Such
formulae will be of essentially the same form as the ones above.
In many other programming languages (such as Pascal and Basic) the mapping from the array
index to the memory address is generated automatically by the compiler or interpreter. I.e., in
the program the expression “City(i)” would appear. It would automatically be converted to
“cmem((i1)*15+108)” as in the example above. In fact, the constant 108, and possibly also
the constant 15, would be unknown (and unavailable) to the programmer.
Logically, a list in a language like Scheme is structurally equivalent to an array as described
above. The Scheme structure “vector” is even closer in intent to the concept of an array. The
manner in which a list or vector is actually stored in a computer memory is often not speciDDS Lecture Notes (draft)
- 29 -
Robert L. Baber, 1999 July
fied in the documentation of the language and is purposely hidden from the programmer. In
languages based on Lisp concepts a list is typically actually stored in a tree structure. (Trees
will be covered later in this course.) Even when lists are actually stored in a tree structure,
program statements or procedures are usually provided for accessing elements of a list or
vector in essentially the same ways as accessing array elements in the above example. For
example, the expression (list-ref City i) or (vector-ref City i) in Scheme corresponds to the
reference City(i) above to an array element.
Exercise (searching 1): Data is stored in an array x with index values ranging from 0 to n
inclusive. The values of the array elements x(0), ... x(n) are not necessarily sorted in any
particular order. Design an algorithm (procedure) which will determine whether or not the
value d exists in this array and if it does, the index of its first occurrence. Your algorithm
will, presumably, involve a loop. What is your loop invariant, i.e. what condition is always
true before and after each execution of the body of the loop? Use a diagram to describe your
loop invariant. Does your algorithm require that the array is not empty, i.e. that n  0?
Exercise (searching 2): Data is stored in an array x as stated in exercise 1 above. The values
of the array elements x(0), ... x(n) are in ascending order with duplicates allowed. Design an
algorithm (procedure) as specified above, but which takes advantage of the fact that the values in the array x are sorted. If the value d does not exist in the array, your algorithm should
determine where in the array it would have been. Hint: decide upon a suitable loop invariant
before designing your algorithm and use it to guide your thinking leading to your design.
Exercise (inserting a value into a sorted array): Data is stored in an array x with index
values ranging from 0 to n inclusive. The values of the array elements x(0), ... x(n) are in
ascending order with duplicates allowed. Design an algorithm which will insert the value of
array element x(n+1) into the correct place in the array so that the entire array from x(0) to
x(n+1) inclusive is sorted into ascending order. What is your loop invariant? How many
times will the body of the loop be executed in the best possible case? in the worst case? on
average? Under what condition will the best case occur? the worst case? What did you assume in determining the average case?
Exercise (sorting): Data is stored in an array x with index values ranging from 0 to n inclusive. The values of the array elements x(0), ... x(n) are not necessarily sorted. Design an
algorithm (procedure) which will sort these values into ascending order. What is your loop
invariant?
5.3. Records
To access the values of Age(3), Name(3) and City(3) in the example in section 5.2 above,
several multiplications and additions are required in order to calculate the relevant addresses
in computer memory. Furthermore, if this data is stored on a disk or similar electromechanical device, accessing these three data values might involve significant mechanical movements
with attendant time delays. Especially if each of these arrays is large, e.g. with thousands of
elements each, the delay can become excessive. Storing these three arrays in a different way
can reduce this potential problem considerably.
In section 5.2 above, each array was stored in contiguous cells in computer memory, one
array after the other. This was an arbitrary choice. We might have chosen to organize the
storage of these arrays differently, for example in the following way.
DDS Lecture Notes (draft)
- 30 -
Robert L. Baber, 1999 July
Addresses in computer memory
Contents
Element of array
...
...
75
22
Age(1)
76-85
George
Name(1)
86-100
Johannesburg
City(1)
101
25
Age(2)
102-111
Themba
Name(2)
112-126
Pretoria
City(2)
127
75
Age(3)
128-137
Gogo
Name(3)
138-152
Nelspruit
City(3)
...
...
Now, addresses can be calculated as follows.
Array element
begins in computer memory address
ends in computer memory address
Age(i)
(i1) * 26 + 75
(i1) * 26 + 75
Name(i)
(i1) * 26 + 76
(i1) * 26 + 85
City(i)
(i1) * 26 + 86
(i1) * 26 + 100
Notice that the term (i1)*26 is common to all address calculations. Thus, only one multiplication is requred to calculate the addresses of all three array elements for a given index value.
Since multiplication is a relatively time consuming operation, three associated array elements
(e.g. Age(3), Name(3) and City(3)) can usually be accessed more quickly if the arrays are
stored interwoven with each other as shown above. If the data is stored on a disk or similar
electromechanical device, the values of these three array elements, being stored adjacent to
one another, can usually be accessed much more quickly than if they had been stored in
widely separate locations, e.g. if the several arrays had been stored one after the other as in
the earlier example above.
Thus, if corresponding elements of different arrays tend to be accessed together, e.g. because
they are logically associated with one another in the context of the application, it is often
advantageous to store the related array elements together and allow each array to become
dispersed. When related array elements are grouped in this way, each group is often called a
record. Sometimes the group is given a name; in the above example, the group or record
might be named Person.
DDS Lecture Notes (draft)
- 31 -
Robert L. Baber, 1999 July
A component of a record is often called a field. I.e., the above record called Person consists
of the fields Age, Name and City. When arrays are involved, a reference to either a record as
a whole or to one of the component fields must be qualified by the index, e.g. the suffix “(i)”
in Age(i).
In programming languages implementing record structures explicitly, the individual components are not often explicitly called or thought of as arrays or array elements, but logically
they are arrays or array elements nevertheless.
In many programming language systems, all of the address calculations referred to above are
performed automatically within the relevant compiler, run time routines belonging to the
programming language system or object code generated by the system.
As other data structures, arrays may be viewed as abstract data types. The term “abstract”
expresses the idea that details of the implementation (e.g. organization of storage and the
representation of the data in a computer memory) are abstracted out of consideration. One
concentrates one’s attention instead on the essence of the data viewed at a higher, more aggregate level. In the case of the field or array Age, the only aspects of interest would be the
fact that the values of the array elements are selected from the set of integers {0, 1, ... 255}
and the operations defined on this set, if any. Also, one could view each element of the array
Age as an ADT or one could view the entire array as a single ADT, in which case the set
associated with the array Age would be, in the above example, {0, 1, ... 255}3. All of these
different views are valid. For different analytical purposes different views may be appropriate.
5.4. Files
The term file is used in computing and business data processing in several different ways. In
its most generic sense, it means any collection of interrelated data.
Probably its most common meaning is any data structure which is physically stored in an
auxiliary or external storage device such as disk, diskette, tape, etc. Such storage devices are
typically much slower than the central memory (e.g. RAM) of a computing system. Data in
such auxiliary or external storage cannot normally be processed directly, but can only be
accessed (read) and rewritten. Because of the speed difference and the fact that data in auxiliary or external storage cannot be processed directly, it is often useful to distinguish between
data in central memory vs. data in auxiliary or external storage; hence the introduction of the
term file.
One of the more specific meanings of the term file is a collection of records (see section 5.3
above), especially when the fields of each record are stored in adjoining areas of memory.
When records are stored in auxiliary or external storage devices, their fields are normally
stored in adjoining areas of the memory device. Thus, the last two meanings mentioned above
are related to one another.
5.5. Multidimensional arrays
A one dimensional array, as presented above, is an array each element of which is identified
by a single index value. More generally, a multidimensional array is an array each element of
which is identified by more than one index values.
A multidimensional array can be viewed as an array of an array ... etc. For example, a two
dimensional array is, in effect, a (one dimensional) array of (one dimensional) arrays. A three
DDS Lecture Notes (draft)
- 32 -
Robert L. Baber, 1999 July
dimensional array is a (one dimensional) array of two dimensional arrays, i.e. a (one dimensional) array of (one dimensional) arrays of (one dimensional) arrays.
Example: Data on the number of votes cast for each of 5 candidates in each of 20 regions
may be structured in the form of a two dimensional array calles Votes. The individual elements of this array would be, in many systems, referenced in the form Votes(c, r), where c is
the index (identification number) of the candidate in question and r, the index (identification
number) of the region in question.
If the five candidates are identified by the numbers 1, 2, 3, 4 and 5 and the 20 regions are
identified by the numbers 11, 12, ... 30 (for whatever reason), then the value of the index c
would range from 1 to 5 inclusive and the value of the index r, from 11 to 30 inclusive.
This two dimensional array Votes could be mapped onto a one dimensional array Oda with
index values ranging from, say, 1 to 100 in different ways. One possibility would be:
Votes(c, r)
Oda(i)
Votes(1, 11)
Oda(1)
Votes(1, 12)
Oda(2)
...
...
Votes(1, 30)
Oda(20)
Votes(2, 11)
Oda(21)
Votes(2, 12)
Oda(22)
...
...
Votes(2, 30)
Oda(40)
...
...
Votes(5, 11)
Oda(81)
Votes(5, 12)
Oda(82)
...
...
Votes(5, 30)
Oda(100)
The index i in the array Oda can be calculated straightforwardly from the indices c and r in
the array Votes by applying the following formula:
i = 1 + (c1)*20 + r11
Questions: How can one derive this formula? How can this formula be generalized for any
given starting and ending values for c and r (e.g. cs, ce, rs, re) and starting value for i (e.g.
is)? What then is the value of the ending value for i (e.g. ie) in terms of the other values?
DDS Lecture Notes (draft)
- 33 -
Robert L. Baber, 1999 July
Another possibility for mapping the two dimensional array Votes to the one dimensional
array Oda is:
Votes(c, r)
Oda(j)
Votes(1, 11)
Oda(1)
Votes(2, 11)
Oda(2)
...
...
Votes(5, 11)
Oda(5)
Votes(1, 12)
Oda(6)
Votes(2, 12)
Oda(7)
...
...
Votes(5, 12)
Oda(10)
...
...
Votes(1, 30)
Oda(96)
Votes(2, 30)
Oda(97)
...
...
Votes(5, 30)
Oda(100)
In this case the index j in the array Oda can be calculated from the indices c and r in the array
Votes by the formula:
j = 1 + (c1) + (r11)*5
A two dimensional array is a natural structure for storing a matrix. The value in column c and
row r of a matrix M is typically stored in the array element M(c, r), although the choice between this and M(r, c) is arbitrary. Whichever convention one chooses, one must be careful to
be consistent.
Exercise: Design an algorithm for multiplying two matrices A and B to form the product C =
A*B. Matrix A has ac columns and ar rows. The range of every index begins with 0. Similarly, matrix B has bc columns and br rows. What condition must be met if the product A*B is
to be defined?
The above approach to defining two dimensional arrays and storing them in a computer
memory (a linear sequence of cells) can be extended in the obvious way to arrays of any
higher dimension.
DDS Lecture Notes (draft)
- 34 -
Robert L. Baber, 1999 July
6. Association by links and pointers
In the preceding sections an association between two or more data values has been realized
either implicitly (i.e. in the minds of the designers and users only) or explicitly by physical
proximity of the data values in the computer memory or by the value of an index to arrays or
lists. These mechanisms, while convenient and simple, do not always suffice, e.g. when the
need arises to express associations between one data value and several others not in corresponding positions of other arrays. A common and more flexible method for expressing such
associations is with the help of “links” and “pointers”.
Different types of programming languages handle the link or pointer mechanism differently.
(1) Some make no particular provision at all for them, leaving pointers and related operations
entirely up to the programmer to implement when needed. (2) Other programming languages
provide data types especially for pointers and procedures for performing related operations.
(3) Still others use such mechanisms extensively but entirely automatically, hiding pointers
and related operations from the programmer. Among the examples of the first type are most
dialects of Basic. Pascal is a typical example of the second type. Scheme falls into the third
type.
Because neither the first nor the third type of programming languages provides explicit pointer mechanisms and related operations, they may appear to be similar or the same to the casual
programmer. However, the programmer concerned with efficiency of execution must be fully
aware of the difference and of various characteristics of these mechanisms and operations,
especially time complexity.
This chapter will examine links, pointers and related operations primarily from the standpoint
of the first and second types of programming languages outlined above. The reader should
keep in mind, however, that implementations of programming languages of the third type
(e.g. Scheme) utilize the mechanisms and operations examined in this chapter extensively,
even if typically invisibly. Understanding these mechanisms is essential if the Scheme programmer, for example, is to understand why some procedures are easy to express in the language but may be inefficient in execution.
This link or pointer method for expressing associations between data values will first be
explained and illustrated using a linear linked list as an example. In later sections, it will be
extended to branching structures, such as trees and graphs.
6.1. Linked lists
In section 5.2 above, a linear sequence of data values was represented in a one dimensional
array in which the sequence of the data values was determined by the natural order of the
integers used as indices to the array. Using this method, the sequence of 6 integers {11, 13,
14, 14, 37, 42} might be represented in the array x as follows:
DDS Lecture Notes (draft)
- 35 -
Robert L. Baber, 1999 July
index i
value of x(i)
1
11
2
13
3
14
4
14
5
37
6
42
Another way of representing this sequence of integers eliminates the structural dependency
upon the indices to the array x and expresses the sequence in a second array, called next
below. In addition, a single variable is needed to indicate the first value in the sequence, here
the variable start, whose value is 4.
index i
x(i)
next(i)
1
13
5
2
...
...
3
...
...
4
11
1
5
14
6
6
14
10
7
...
...
8
42
-1
9
...
...
10
37
8
11
...
...
Note several characteristics of such a linked list: The value of a particular variable (here start)
is the index of the first element in the list (sequence). The list is stored in two (or more) arrays. Each element of the list contains one (or more) data values and the index of the following element in the list. One particular value, which is not a valid index to the arrays in question, indicates the end of the list; here the value -1 is used as this end marker. Not all elements of the arrays need be occupied (used). Adjacent elements in the list (sequence) may,
but need not, be stored in adjacent elements of the arrays.
DDS Lecture Notes (draft)
- 36 -
Robert L. Baber, 1999 July
Variations and extensions of the above characteristics are sometimes desirable. For example,
in some applications it may be desirable to maintain a pointer to the last element in a list,
along with the pointer to the first element.
Question: How might an empty list be represented?
Clearly, representing a sequence as a list instead of in a single array requires more memory.
Some operations on the list are more difficult or time consuming to perform, but others are
easier and less time consuming. The list representation is more flexible in some (but not all)
respects. Tradeoffs are involved in making a decision how to store a sequence of data values;
for some purposes the linked list has definite advantages and for other purposes the simple
array is better.
Such a linked list may have more than one data value and may be simultaneously ordered in
different ways. For example, consider the following set of arrays and the two variables startforward and startbackward whose values are 4 and 8 respectively.
index i
x(i)
y(i)
next(i)
previous(i)
1
13
2
5
4
2
...
...
3
...
...
4
11
1
1
-1
5
14
8
6
1
6
14
3
10
5
7
...
8
42
9
...
10
37
11
...
...
6
-1
10
...
6
8
6
...
This list is structured both in ascending and in descending order simultaneously. The list
beginning at index 4 is in ascending order and the list beginning at index 8 is in descending
order. Both lists consist of the same data elements.
Exercise: Add another set of links to the above data structure so that the new links sequence
the data elements in ascending order on the values of y. Where does the new list begin (with
what index value)?
DDS Lecture Notes (draft)
- 37 -
Robert L. Baber, 1999 July
6.2. Pointers and links
In such lists, the index values (stored in the arrays next and previous above) are often called
pointers, as they “point to” the next element in the list. Note that a pointer is nothing other
than an index to an array (or to several arrays).
The only substantial difference between an index to an array and a pointer in such a list is that
indices to an array are usually selected from a set of consecutive integers, while pointer values are typically selected from a set of integers but not necessarily consecutive integers. This
difference is usually of no great significance and is often overemphasized.
Many specific programming languages introduce a syntactical difference between an index to
an array and a pointer to an element in a list. For example, in Pascal, the pointer notation i^.x
is used as x(i) was used above. Semantically these two references are synonyms for each
other: both are interpreted to mean the value of x identified by the value of the index (pointer)
i. In these two cases, the identifier x would be declared in syntactically different ways, but
semantically the effect is the same. Other programming languages follow still other conventions for writing references to variables indexed by pointers.
The connection or association established by a pointer is called a link.
The sequence of links along one path is often called a chain. There are two chains in the
above data structure, one beginning at startforward and the other beginning at startbackward.
The exercise above adds a third chain to the data structure.
An element in a list is sometimes also called a node. Thus, a link connects two nodes.
6.3. Diagrams of linked lists
The arrays in which a list is stored and, in particular, whether some parts of the arrays are
occupied or not are usually not the essential characteristics of a list. The tabular representation used in section 6.1 above is, therefore, not a particularly appropriate form for describing
such a list. Instead, diagrams which highlight the links are typically employed. The following
is such a diagram for the above list:
11
1
13
2
14
8
14
3
37
6
42
6
end
end
startforward
startbackward
Note that the actual values of the pointers are not of interest and, hence, are not shown on this
diagram.
Both forward and backward pointer chains are often shown on one diagram. Adding other
chains, such as the third one added in the exercise above, typically gives the diagram a confused appearance, with chains crossing over each other. Different chains structuring the data
in different ways are usually best shown on separate diagrams.
DDS Lecture Notes (draft)
- 38 -
Robert L. Baber, 1999 July
6.4. Searching a linked list
Frequently one wants to search a linear sequence of data items, be it in the form of a one
dimensional array or in the form of a linked list. This is usually accomplished by scanning
over unwanted elements of the list until the desired element is found or it becomes clear that
the desired element is not in the list. The latter condition applies if the end of the list is
reached or, in the case of a list known to be sorted, an element later in the ordering sequence
than the desired element is encountered.
We will design two searching algorithms. When designing the first one, we will assume that
we do not know whether the list is initially sorted or not. When designing the second one, we
will assume that the list is initially sorted. An important question will be whether utilizing
this knowledge improves the search in any way.
Example (list not necessarily initially sorted): Consider the list in the preceding examples.
We seek the first element in the list whose value of x is equal to the value of the variable key.
When the search terminates, we require that the result variable f points to the desired element
in the list, if it exists, otherwise to the “end” of the list. In order to ensure that the element
found is the first one satisfying the search criterion, we also require that all elements in the
list before the one to which f points not satisfy the search criterion. Somewhat more formally,
we require that when the search terminates the following condition will be fulfilled:
(f  1 and x(f) = key
or f = 1)
[desired element found]
[end of list encountered; desired element therefore not in list]
and every element in the list before the one to which f points has a value of x not equal
to the value of key
This formulation of the final condition does not presuppose that the list is initially sorted in
any order. Duplicate values are permitted.
Question: Why is it not necessary that the list be initially sorted?
The above condition represents the goal of our search algorithm. It, together with the necessity of starting the search at the beginning of the list, leads to the following algorithm:
f := startforward
while f  1 and x(f)  key do
f := next(f)
endwhile
Question: Why does this program segment satisfy the required final condition above?
Question: What condition is true before and after every execution of the body of the loop,
i.e. what is the loop invariant?
Example (list initially sorted): Consider the same list as in the above examples, but now
assume that the list is in ascending order on the values of x. We seek the first element in the
list whose value of x is equal to the value of the variable key. When the search terminates, we
require that the result variable f points to the desired element in the list, if it exists, otherwise
to the element beyond the place where the desired element would be, or to the “end” of the
list if no such place in the list exists. In order to ensure that the element found is the first one
satisfying the search criterion, we also require that all elements in the list before the one to
DDS Lecture Notes (draft)
- 39 -
Robert L. Baber, 1999 July
which f points not satisfy the search criterion. Somewhat more formally, we require that
when the search terminates the following condition will be fulfilled:
(f  1 and x(f) = key
or f  1 and x(f) > key
or f = 1)
[desired element found]
[higher element found; desired element therefore not in list]
[end of list encountered; desired element therefore not in list]
and every element in the list before the one to which f points has a value of x less than
the value of key
The above final condition can obviously be simplified to
(f  1 and x(f)  key
or f = 1)
[location found, whether or not desired element present]
[end of list encountered; desired element therefore not in list]
and every element in the list before the one to which f points has a value of x less than
the value of key
This formulation of the final condition clearly presupposes that the list is initially sorted in
ascending sequence of x, whereby duplicate values are permitted.
Question: Why is it necessary that the list be initially sorted?
The above condition represents the goal of our search algorithm. It, together with the necessity of starting the search at the beginning of the list, leads to the following algorithm:
f := startforward
while f  1 and x(f) < key do
f := next(f)
endwhile
Question: Why does this program segment satisfy the required final condition above?
Question: Is the second algorithm (the one based on the assumption that the list is initially
sorted) better than the first? If so, in what way?
The two algorithms above deliver as their result the value of the variable f, which points to
the first element in the list which satisfies the termination condition (x(f)  key or x(f) = key,
depending on the algorithm). In some applications a pointer to the preceding element in the
list is needed as a result of the search algorithm. This requirement can be added to the above
specifications in a fairly simple way. Firstly, we introduce an additional result variable prev
and require that it point to the predecessor of the element to which f points. Viewed differently but equivalently, prev should point to the element in the list which in turn points to the
same element to which f points.
DDS Lecture Notes (draft)
- 40 -
Robert L. Baber, 1999 July
...
...
...
...
prev
f
An obvious strategy for achieving this new goal is to continually keep track of the predecessor of the node to which f points by adding appropriate statements to the algorithms above.
I.e. prev and f should continually (that is, at most points in the algorithm) satisfy the condition
f = next(prev)
This expression, of course, makes sense only if prev is not the end value, i.e. if prev  1.
This raises the question what the relationship between f and prev should be otherwise. Thus,
our condition becomes
prev  1 and f = next(prev)
or prev = 1 and ?
Note that the initialization of the two algorithms establishes the truth of the condition f =
startforward. But if f = startforward and f = next(prev), then startforward = next(prev), which
can be satisfied only if the first element in the list has a predecessor, which is a contradiction.
Thus, the subcondition f = startforward (or something which implies it) is needed somewhere
in the condition we are constructing. The place marked ? above is a possibility which immediately presents itself and which appears to (and, in fact, does) suffice. We add, therefore, the
condition
prev  1 and f = next(prev)
or prev = 1 and f = startforward
to the postconditions and to the loop invariants of the two algorithms above. Wherever f is
assigned a value in those algorithms, appropriate statements assigning a suitable value to prev
must be added. Thus, the algorithms become:
prev := 1; f := startforward
while f  1 and x(f)  key do
prev := f; f := next(f)
endwhile
[algorithm not requiring the list to be sorted]
[note: next(f) could be replaced by next(prev)]
and
prev := 1; f := startforward
while f  1 and x(f) < key do
prev := f; f := next(f)
endwhile
DDS Lecture Notes (draft)
[algorithm requiring the list to be sorted]
[note: next(f) could be replaced by next(prev)]
- 41 -
Robert L. Baber, 1999 July
where the new statements are highlighted in italics. Note that after every execution of the
body of the loop, prev will not be equal to 1 and f will be equal to next(prev). Therefore, if
the algorithm terminates with prev = 1, then the body will not have been executed at all;
either the list is empty or the first element in the list satisfied the termination condition.
Note how a thorough analysis of the problem leads relatively simply and quickly to the necessary program statements. This observation applies often in program design. Such types of
analyses can be systematized more extensively than has been done here, but that is a subject
for another course.
Question: The new body of the loop was written as
prev := f; f := next(f)
and not
f := next(f); prev := f
Why? (Hint: consider the new condition to be satisfied.)
6.5. Inserting an element into a linked list
When processing data represented in a linked list, it is often necessary to insert a new data
item into an existing list. At first glance, there appear to be four special cases: inserting a data
item into (A) an empty list, (B) at the beginning of an existing list, (C) between two elements
of an existing list and (D) at the end of an existing list. In fact, only two cases need be handled, because the structure of the sublist following the insertion point is not relevant for the
insertion operation. Cases A and B reduce to one situation (called case 1 below) and cases C
and D reduce to another situation (called case 2 below). A question for the student to ponder
is, “How can the remaining two cases be reduced still further to only one?”
6.5.1. Case 1: inserting a new item at the beginning of a list
In this case a new data item is to be inserted at the beginning of a list (which may or may not
be empty). Let the variable start point to the beginning of a list and the variable pnew point to
an item to be inserted at the beginning of the list.
The following diagram illustrates the conditions prevailing before the insertion:
pnew
...
...
...
...
start
DDS Lecture Notes (draft)
- 42 -
Robert L. Baber, 1999 July
and the following diagram illustrates the conditions required after the insertion:
pnew
...
...
...
...
start
By comparing the two diagrams above defining the initial and final situations, we can deduce
the changes which must be made:
pnew
...
...
insert
...
...
delete
insert
start
From these diagrams, especially the last one above, it follows that this program segment will
accomplish the insertion:
next(pnew) := start
start := pnew
Questions: Why was this sequence of statements selected and not the reverse? The above
program segment contains two statements, each of which inserts a link. How does it delete
the link from start to the (originally) first element in the list?
Will this program segment also correctly insert the element to which new points if the list is
initially empty? Why or why not? Draw the appropriate diagrams to convince yourself and
others that your answer is correct.
DDS Lecture Notes (draft)
- 43 -
Robert L. Baber, 1999 July
6.5.2. Case 2: inserting a new item after an existing element of a list
In this case a new data item is to be inserted after an existing element of a list, i.e. either
between two elements of an existing list or at the end of an existing non-empty list. Let the
variable p point to the list element immediately after which the new element is to be inserted.
Let the variable pnew point to the item to be inserted.
The following diagram illustrates the conditions prevailing before the insertion:
pnew
...
...
...
...
...
...
p
DDS Lecture Notes (draft)
- 44 -
Robert L. Baber, 1999 July
and the following diagram illustrates the conditions required after the insertion:
pnew
...
...
...
...
...
...
p
By comparing the two diagrams above defining the initial and final situations, we can deduce
the changes which must be made:
pnew
...
...
insert
insert
...
...
...
...
delete
p
From these diagrams, especially the last one above, it follows that this program segment will
accomplish the insertion:
DDS Lecture Notes (draft)
- 45 -
Robert L. Baber, 1999 July
next(pnew) := next(p)
next(p) := pnew
Questions: Why was this sequence of statements selected and not the reverse? The above
program segment contains two statements, each of which inserts a link. How does it delete
the link from the element to which p points to the subsequent element in the list?
Will this program segment also correctly insert the element to which pnew points if the element to which p points was initially the last element in the list? Why or why not? Draw the
appropriate diagrams to convince yourself and others that your answer is correct.
Question: Can the remaining two cases be reduced still further to only one? If so, how? If
not, why not?
6.6. Deleting an element from a linked list
When processing data represented in a linked list, it is also often necessary to delete a data
item from an existing list. Again here, two different situations arise which must be handled
by different program segments. They are (1) deleting the first element from the list and (2)
deleting an element other than the first from the list.
Question: Why do only these two cases arise? Why does the deletion of the last element of a
list not require special consideration?
6.6.1. Case 1: deleting the first element from a list
In this case, the first element of a (non-empty) list is to be deleted, i.e. removed, from the list.
Let the variable start point to the beginning of the list in question. After deletion, the variable
pdeleted must point to the item which was deleted from the list.
The following diagram illustrates the conditions prevailing before the deletion:
...
...
...
...
start
DDS Lecture Notes (draft)
- 46 -
Robert L. Baber, 1999 July
and the following diagram illustrates the conditions required after the deletion:
pdeleted
...
...
...
...
start
By comparing the two diagrams above defining the initial and final situations, we can deduce
the changes which must be made:
pdeleted
insert
...
...
(delete)
...
...
insert
delete
start
From these diagrams, especially the last one above, it follows that this program segment will
delete the first element from the list and record in pdeleted the location (index) of the element
that was deleted from the list:
pdeleted := start
start := next(start)
Questions: Why is the one link labelled “(delete)”, i.e. “delete” in parentheses? Is it really
necessary to delete this link? Why or why not? What does “delete” mean in this specific
context?
Why was this sequence of statements selected and not the reverse? The above program segment contains two statements, each of which inserts a link. How does it delete the link from
start to the (originally) first element in the list?
DDS Lecture Notes (draft)
- 47 -
Robert L. Baber, 1999 July
Will this program segment also correctly delete the first element in the list if it is also the last
element in the list, i.e. if it is the only element in the list? Why or why not? Draw the appropriate diagrams to convince yourself and others that your answer is correct.
If the above program segment also correctly deletes the only element in a list, why does it not
matter whether the element to be deleted from the list is the last element in the list or not?
6.6.2. Case 2: deleting an element other than the first from a list
In this case, an element other than the first is to be deleted from a list. This statement of the
problem implies that the element to be deleted from the list has a predecessor in the list. Let
the variable p point to that predecessor; i.e., next(p) points to the element to be deleted from
the list. After deletion, the variable pdeleted must point to the item which was deleted from
the list.
The following diagram illustrates the conditions prevailing before the deletion:
...
...
...
...
...
...
p
DDS Lecture Notes (draft)
- 48 -
Robert L. Baber, 1999 July
and the following diagram illustrates the conditions required after the deletion:
pdeleted
...
...
...
...
...
...
p
By comparing the two diagrams above defining the initial and final situations, we can deduce
the changes which must be made:
pdeleted
...
...
delete
(delete)
...
...
...
...
insert
p
From these diagrams, especially the last one above, it follows that the program segment
pdeleted := next(p)
next(p) := next(next(p))
DDS Lecture Notes (draft)
- 49 -
Robert L. Baber, 1999 July
will
1. delete an element other than the first from the list, in particular, will delete the successor of
the element to which p points, and will
2. record in pdeleted the location (index) of the element that was deleted from the list:
Notice that “next(p)” appears here exactly where “start” appeared in the algorithm for case 1
above (see section 6.6.1 above).
Questions: What preconditions must be met before the program segment above is applied to
a list? How can one deduce these preconditions from the statement of this case of the problem? from the program segment above? E.g., what is the minimum permissible length of a list
to which the above program segment is applied? What other conditions must be fulfilled
before this program segment is executed? Express these conditions both in a verbal form
related to the problem description and in the form of mathematical expressions.
Why is the one link labelled “(delete)”, i.e. “delete” in parentheses? Is it really necessary to
delete this link? Why or why not? What does “delete” mean in this specific context?
Why was this sequence of statements selected and not the reverse? The above program segment contains two statements, each of which inserts a link. How does it delete the link from
start to the (originally) first element in the list?
Will this program segment also correctly delete an element in the list if that element is also
the last element in the list? Why or why not? Draw the appropriate diagrams to convince
yourself and others that your answer is correct.
If the above program segment also correctly deletes the last element in a list, why does it not
matter whether the element to be deleted from the list is the last element in the list or not?
Question: Can the above two cases be reduced still further to only one? If so, how? If not,
why not?
6.7. List of available elements
The functions of inserting and deleting elements of a linked list as described in sections and
above give rise to the questions
 where does the memory space for the new list elements come from and
 what happens with the space freed by deleting old list elements?
Many programming language systems which support linked list structures maintain a list of
available, i.e. unused, list elements, sometimes called the list of free elements or, more simply, the free list. Such a list can be implemented as any other list, using the techniques and
mechanisms described earlier in this chapter. If newly freed list elements are inserted into the
free list at its beginning and list elements required for new data items are taken from the
beginning of the free list, the insertion and deletion operations involve only one case each to
consider and are correspondingly simple.
Example: The arrays implementing the list in section 6.1 above could be extended to include
a free list. The following table illustrates one such possible extension:
DDS Lecture Notes (draft)
- 50 -
Robert L. Baber, 1999 July
index i
x(i)
next(i)
nextfree(i)
1
13
5
...
2
...
...
7
3
...
...
2
4
11
1
...
5
14
6
...
6
14
10
...
7
...
...
11
8
42
-1
...
9
...
...
-1
10
37
8
...
11
...
...
9
Here, the free list begins at index 3. Therefore, if the variable startfree is used to indicate the
beginning of the free list, its value in this example would be 3.
Questions: How can one determine whether free list elements are available or not in the
above table?
In some systems the need for determining the number of free elements available arises frequently. One way of determining this number is, of course, to scan the list, counting the
number of elements traversed until the end of the free list is reached. Describe briefly a way
of determining this number more quickly. What invariant condition must then be maintained
by any program segment operating on the free list?
[end of questions]
Other mechanisms for monitoring and controlling available memory exist and are used in
practice. Some are variants of the basic linked list techniques and methods described above.
Exercise: A number of data items no longer needed are organized in a list. The variable start
points to the first element of this list and the variable last points to the last element of the list.
All of the data items in this list are to be returned to the free list. Design an algorithm which
performs this operation. Under what initial conditions will your algorithm function correctly?
DDS Lecture Notes (draft)
- 51 -
Robert L. Baber, 1999 July
7. Queues
In various types of computer systems the need arises often for storing a number of data values
in such a way that they are retrieved in the same order as they were stored. Such a queue, or
buffer, is often called a first-in-first-out (FIFO) queue. In computer science jargon it is often
called a pipeline.
Alternatively, one may want to retrieve the values in a queue in the opposite order than that in
which they were stored. Such a queue, or buffer, is often called a last-in-first-out (LIFO)
queue. In computer science jargon such a queue is often called a stack.
A queue of either of these types can be realized in a computer system in various ways. The
most obvious and natural of these are a one dimensional array and a linked list.
The two primary operations on a queue (either a pipeline or a stack) are to
 place a data item into the queue (also called store, insert, enqueue, put, push, etc.) and to
 retrieve a data item from the queue (also called fetch, remove, dequeue, serve, get, pop,
etc.).
Viewed mathematically as functions, put maps a queue and a data item to a queue, while get
maps a queue to a queue and a data item:
put: Q  D  Q
get: Q  D  Q
where Q is the set of queues and D is the set of data items. A queue here is a sequence of data
items. The data items are of an unspecified type (i.e. are elements of an unspecified set).
In the case of a stack, the put and get operations defined above are usually called push and
pop respectively.
In addition, other operations and functions are typically defined to create a queue, to determine whether a queue is empty or not and, if the length of the queue is bounded (as it always
is in practice), to determine whether the queue is full or not. Still other functions can be defined to satisfy special needs arising in particular applications (e.g. to determine the number
of elements in a queue).
7.1. Pipelines
The distinguishing characteristics of a pipeline (a first-in-first-out queue) are:
 the put operation applied to a queue q and a data item d yields as its result a queue q' in
which d has been appended to the end of the queue q. I.e., q' = q & [d], where & stands for
concatenation and
 the get operation applied to a queue q returns the data item d, which is the first element in
the queue q, and a queue q' which is the rest of q after removing its first element. These are
sometimes written d = first(q) and q' = tail(q). The relationship between q, d and q' is given by the equation q = [d] & q'.
Example of a pipeline: If q = [a, b, 2, 5t5] and d = z, then the result of performing the put
operation is the queue (sequence) [a, b, 2, 5t5, z].
If q = [a, b, 2, 5t5, z], then performing the get operation yields the pair of results d and q',
where d = a and q' = [b, 2, 5t5, z].
[end of example]
DDS Lecture Notes (draft)
- 52 -
Robert L. Baber, 1999 July
Note that the results of the get operation are defined above only for a non-empty queue. I.e.,
the domain of the get function is the set of non-empty queues.
A pipeline can be realized in a straightforward way with a one dimensional array. Beginning
at, say, index 0, items are placed into the pipeline at consecutively higher positions. Correspondingly, they are retrieved, beginning at 0, from consecutively higher positions. This can
be represented by the following diagram:
out
in
index
0
1
2
3
4
5
6
7
8
9
10
11
12
...
queue
...
...
C
D
E
F
G
H
I
J
...
...
...
...
The variables out and in keep track of the positions at which the next retrieval and insertion
operations respectively are to be performed. This pipeline currently contains the queue [C, D,
E, F, G, H, I, J]. The pipeline contains data from index out to index in1 inclusive. The number of data items in the pipeline is given by the expression inout. This number must always
be zero or positive, i.e. the condition out  in must always be met. If out = in, the pipeline is
empty and no retrieval operation may be performed.
The put operation (as defined above) can be performed by the program segment
queue(in) := d; in := in + 1
[put]
and the get operation as defined above, by
d := queue(out); out := out + 1
[get]
A precondition for invoking this get operation is the condition alluded to above, namely that
the queue is not empty. More precisely, this precondition is out < in (and, of course, that the
values of out and in must be integers).
These operations are efficient in terms of time, but not in terms of memory occupation.
Enough memory must be provided to store the entire sequence passed through the queue, not
just the maximum number of items actually stored in the queue. Memory once used but no
longer needed (i.e. the region to the left of out) is never reused. Another shortcoming is that,
in practice, the length of the array is always bounded, and as soon as that amount of data has
passed through the pipeline, no more data can be placed into it, even though it might be empty.
These shortcomings can sometimes be overcome by resetting out and in to 0 if and when the
pipeline becomes empty. Depending upon the nature of the application using the pipeline,
this may occur often, seldom, or never. Thus, this approach may, but need not, overcome the
shortcomings identified above.
The shortcomings above can always be overcome by shifting the data in the pipeline to the
left each time a data item is retrieved. Then, the array need only be as long as the longest
sequence ever actually held in the queue. This would be comparatively efficient in terms of
memory occupation, but the need to shift a possibly long sequence to the left each time a
single data item is retrieved would result in unnecessarily long execution times.
Here we see an example of time vs. memory tradeoff which arises in design tasks.
DDS Lecture Notes (draft)
- 53 -
Robert L. Baber, 1999 July
Another approach, which avoids the shortcomings of both of the above algorithms, is to
realize the pipeline by a circular “array”, which is, in turn, realized in a one dimensional
array. The array queue with indices ranging from 0 to N1 is considered to be a circular
structure, with queue(0) being the successor of (i.e. following) queue(N1). More generally,
the index of the array element following queue(i) is queue(i1), where  stands for addition
modulo N. Modulo addition gives the array the circular structure desired. Also here, as in the
example above, the variables out and in indicate the positions at which the next retrieval and
insertion operations respectively are to be performed. The following diagram illustrates this
data structure for a pipeline.
N1 0
circular array
queue
out
in
The contents of the pipeline are the data items in array elements queue(out) (inclusive)
through queue(in) (exclusive). In other words, the pipeline contains all data items from and
including queue(out) to but excluding queue(in), in that order.
A certain ambiguity arises in the case that out = in, in particular, whether the pipeline is full
(contains N data items) or is empty (contains 0 data items). There are several ways of resolving this design question. Normally, it is necessary that an empty pipeline be provided for, so
all generally useful alternatives must foresee the possibility of an empty queue.
Perhaps the simplest solution is to define out = in to mean that the pipeline is empty. This
implies, in turn, that when in1 = out, no more data items may be placed into the pipeline,
for doing so would result in increasing in to be equal to out, effectively emptying the pipeline
and losing all N data items (the N1 items previously in the pipeline and the one being inserted). In other words, in this case the pipeline is effectively full when in1 = out, even
though it is not physically full then. Thus, at most N1 data items would ever be stored in the
pipeline at any one time, effectively wasting 1 element in the array. For large N, this is not
really significant.
Another solution would be to maintain an additional variable indicating whether the pipeline
is empty or not (or whether the pipeline is full or not). If out = in, the value of the additional
DDS Lecture Notes (draft)
- 54 -
Robert L. Baber, 1999 July
variable will determine whether the pipeline is full or empty. When in becomes equal to out,
one can always distinguish between a full pipeline or an empty pipeline, so the correct value
of this additional variable can always be maintained. If in becomes equal to out while placing
a data item into the pipeline (i.e. while incrementing in), the pipeline becomes full (not empty). If in becomes equal to out while removing a data item from the pipeline (i.e. while incrementing out), the pipeline becomes empty (not full).
Still another solution is to indicate an empty pipeline by a special value of out and/or in, the
special value being outside the range of allowed array index values. I.e., this special value
would be less than 0 or greater than N1. One such suitable special value would be 1.
If we choose the convention that out = in always means that the pipeline is empty, the following program segment will implement the put operation:
queue(in) := d; in := in  1
[put]
The get operation will be implemented by
d := queue(out); out := out  1
[get]
A precondition for invoking this put operation is the condition that the pipeline is not full, i.e.
that in1  out. A precondition for invoking this get operation is the condition that the queue
is not empty, i.e. that in  out.
Summarizing the full and empty conditions in the design convention selected here, the pipeline is full iff in1 = out and it is empty iff in = out.
7.2. Stacks
As already stated on page 52 in section 7 above, in the case of a stack, the put and get operations defined above for pipelines are usually called push and pop respectively.
The defining characteristics of a stack (a last-in-first-out queue) are: if a push operation is
followed immediately by a pop operation,
 the stack is returned to its original condition and
 the data item retrieved by the pop operation is the same as the one inserted by the push
operation.
Viewed in another but equivalent way, the distinguishing characteristics of a stack are:
 the push operation applied to a queue (stack) q and a data item d yields as its result a
queue q' in which d has been appended to the end (also called “top” in the context of a
stack) of the queue q. I.e., q' = q & [d], where & stands for concatenation and
 the pop operation applied to a queue q returns the data item d, which is the last element in
the queue q, and a queue q' which is the rest of q after removing its last element.
Example of a stack: If q = [a, b, 2, 5t5] and d = z, then the result of performing the push
operation is the queue (stack) [a, b, 2, 5t5, z].
If q = [a, b, 2, 5t5, z], then performing the pop operation yields the pair of results d and q',
where d = z and q' = [a, b, 2, 5t5].
[end of example]
Sometimes the definition of a stack appears in a different form in which the newly pushed
item is prefixed to the sequence, in which case an item is popped from the beginning of the
sequence. This aspect of the definition is immaterial; important is only that both operations
DDS Lecture Notes (draft)
- 55 -
Robert L. Baber, 1999 July
are performed on the same end of the queue representing the stack, ensuring that the last item
pushed (stored) onto the stack will be the first (next) one to be popped (retrieved).
Note that the results of the pop operation are defined above only for a non-empty stack. I.e.,
the domain of the pop function is the set of non-empty stacks.
A stack can be realized in a straightforward way with a one dimensional array. Beginning at,
say, index 0, items are placed onto the stack at consecutively higher positions. Correspondingly, they are retrieved, beginning at the top, from consecutively lower positions. This can
be represented by the following diagram:
top
index
0
1
2
3
4
5
6
7
8
9
10
11
12
...
queue
A
B
C
D
E
F
G
H
I
J
...
...
...
...
The variable top keeps track of the “top” of the stack, i.e. the position at which the last data
item pushed onto the stack was inserted. This stack currently contains the queue [A, B, C, D,
E, F, G, H, I, J]. The stack contains data from index 0 to index top inclusive. The number of
data items in this stack is given by the expression top+1. This number must always be zero or
positive, i.e. the condition 1  top must always be met.
The stack is empty iff top = 1, in which case no pop (retrieval) operation may be performed.
The push operation (as defined above) can be performed by the program segment
top := top + 1; queue(top) := d
[push]
and the pop operation as defined above, by
d := queue(top); top := top  1
[pop]
A precondition for invoking this pop operation is the condition mentioned above, namely that
the stack is not empty. More precisely, this precondition is 1 < top (or, equivalently, that 0 
top), and, of course, that the value of top must be an integer.
In practice, the amount of computer memory is always bounded and, therefore, each array,
such as queue above, is limited in size. This must be taken into account when designing
and/or using an implemented stack mechanism.
Question: Assume that in the above illustration of an implementation of a stack, the array
queue has been declared to have N elements, i.e. that the range of indices is from 0 to N1
inclusive. What additional condition(s) must be placed on invoking the push and/or pop
operations? What other changes, if any, must be made in the above design of the stack system?
What mathematical expression is equivalent to the statement “the stack is full”? How did you
deduce or derive this expression and why is it valid?
Exercise: Implement the push and pop operations as above, but using a linked linear list to
represent the stack.
DDS Lecture Notes (draft)
- 56 -
Robert L. Baber, 1999 July
Stacks are used in computer systems for various purposes. One of the common uses is in the
implementation of recursive subprograms. All of the data associated with each invocation
(activation) of a recursive subprogram is typically stored in the stack: the return address (the
location of the next instruction to be executed after the execution of the subprogram is complete), variables local to the subprogram, the arguments (input data) of the invocation in
question and any other information defining the context of the computation.
Example: Consider the following recursive procedure for calculating the factorial of a number:
procedure factorial(n)
if n=0
then return 1 as the result
else return n*factorial(n-1) as the result
endif
end procedure
Typically this subprogram would be compiled to machine code which satisfies the following
(or a very similar) specification: If
 the top item in the stack is a non-negative integer n,
[precondition]
 the second item in the stack is the label (location, address) of a program statement and
 control is transferred to the label “factorial”,
then
 the top two items will be popped (removed) from the stack,
[postcondition]
 the factorial of n will be pushed (placed) onto the stack and
 control will be transferred to the program statement identified by the label which was
originally the second item in the stack.
 The values of the program variables n, f and next and the stack items mentioned above
may have been modified by the execution of the procedure beginning at the label “factorial”, but no other changes will have been made to the data environment, in particular, other
items lower in the stack are left unchanged.
A program segment satisfying the above specification can be used to calculate the factorial of
a number n by executing the following sequence of program statements:
push “end”
push n
goto “factorial”
end: stop
Upon termination (reaching the stop command) the top item in the stack will be the factorial
of n, the value pushed onto the stack by the statement “push n” above.
DDS Lecture Notes (draft)
- 57 -
Robert L. Baber, 1999 July
The above procedure “factorial” would typically be compiled to the following code or its
equivalent:
procedure factorial(n)
if n=0
then return 1 as the result
else return n*factorial(n-1) as the result
factorial: pop n
if n=0
then pop next
push 1
goto next
else push n
push “continue”
push n1
goto factorial
continue: pop f
pop n
pop next
push n*f
goto next
endif
end procedure
Notice how the program code on the right ensures that the specification is satisfied, in particular, regarding the state of the stack. Note also how the value of n, which must be multiplied
by the value of the factorial of n1, as well as the return address are maintained over the
internal recursive activation of the procedure by placing these values on the stack.
Notice also how the specification of the procedure factorial is used in the design and verification of the program code on the right. This corresponds to the inductive step in a proof by
induction. The then part of the if construct corresponds to the base step in a proof by induction.
No limitation on the stack size has been considered above for didactical reasons. In practice,
appropriate additions to the above program code would have to be made to ensure that such a
limitation is not exceeded. This would require extending the specification appropriately.
Exercise: Trace through the execution of the above stack oriented program for n = 0, 1, 2,
etc.
Other applications of stacks include traversing trees and converting between Polish and infix
notation for mathematical expressions. These subjects are treated in later sections below.
7.3. Priority Queues
Queues may also be defined in such a way that neither the first nor the last data item inserted
into the queue is necessarily the next item which will be retrieved. Some other criterion can
be defined to determine the retrieval priority. Such queues are sometimes called priority
queues or highest-priority-in/first-out queues (HPIFO).
Logically, a priority queue amounts to a linear queue coupled with a sorting mechanism. A
particularly simple way to implement such a queue is to put (insert) each new data item into
the queue in a position corresponding to its priority. A get operation then retrieves the data
item at the high-priority end of the queue.
DDS Lecture Notes (draft)
- 58 -
Robert L. Baber, 1999 July
Question: How many data items must be examined when entering a new data item into a
priority queue implemented in a one dimensional array or a linear linked list? (Give the minimum, maximum and average number and state any assumptions needed to determine these
quantities.)
Tree structures can enable priority queues to be implemented so that the insertion time is
considerably less than that for a linear queue. Tree structures will be covered in the next
section.
DDS Lecture Notes (draft)
- 59 -
Robert L. Baber, 1999 July
8. An introduction to trees
8.1. Definitions and terminology
A tree is defined as a non-empty collection of nodes and edges exhibiting the following
characteristics:
1. A node (also called a vertex) can contain information (data item(s)).
2. An edge (also called an arc, a branch or a link) is a directed connection between two distinct (different) nodes. The direction of an edge defines a predecessor-successor (parentchild) relationship between the two nodes it connects.
3. There is a unique node, called the first or root node, which has no predecessor (parent).
4. Every node except the root node has exactly one predecessor (parent).
5. A node may have any number of successors (children).
6. For every node other than the root node, there exists a path (see definition below) from the
root node to that other node.
Additional useful terms and conventions relating to trees include:
 A node with no successors (children) is called a leaf (end, terminal) node.
 A node which is neither a root nor a leaf node is sometimes called an internal node.
 A path is a sequence of vertices in which every pair of adjacent vertices is connected by an
edge in the tree, whereby all such pairs are related in the same parent-child direction. Alternatively, a path can be viewed as the sequence of edges connecting these vertices.
 The root node of a tree is usually drawn as a circle at the top of the diagram. Each edge is
drawn in a downward direction from the predecessor to the successor.
 An edge is sometimes drawn with an arrowhead to indicate the direction of the predecessor-successor relationship. If the above mentioned convention of the downward direction
is systematically followed, however, the arrowhead is superfluous (see point above) and,
therefore, can be omitted.
 A descendant of a node N is any node which is a child of N or a child of a child of N, ...
etc.
 Two or more nodes are siblings of each other if they have the same parent.
 Any node can be thought of as the root of a subtree consisting of itself and all of its descendants.
Additional characteristics of a tree following from the above definition include:
 There is exactly one path from the root node to any given node in the tree.
 There are no cycles (loops) in the tree, i.e. there is no path from any node back to itself.
 A unique number can be associated with each node, that number being the number of
edges along the path from the root to the node in question. This number defines the hierarchical level upon which the node in question is located. This hierarchical level of a node is
sometimes called the depth of the node. The maximum level number over all nodes is
called the height (depth) of the tree.
DDS Lecture Notes (draft)
- 60 -
Robert L. Baber, 1999 July
Example: Consider the following tree:
A
B
E
Level 0
C
F
D
G
K
L
H
M
I
Level 1
J
Level 2
Level 3
Node A is the root node. Its children are nodes B, C and D. Nodes K, L and M are children of
node G. Node D is the parent of nodes H, I and J. The root node (here A) is always on level 0.
Nodes B, C and D are on level 1. Nodes E, F, G, H, I and J are on level 2. Nodes K, L and M
are on level 3. Nodes G, K, L and M are the descendants of node C. The tree is of height 3.
The leaf nodes are E, F, K, L, M, H, I and J. Nodes B, C, D and G are internal nodes. Nodes
H, I and J are siblings. The descendants of node A are the nodes B, C, D, E, ... M.
[end of example]
A binary tree is defined as a tree in which each node has at most two successors (children).
More generally, an n-tree is defined as a tree in which each node has at most n successors
(children).
Trees are used to represent various types of data in quite different kinds of applications. They
are used to represent mathematical expressions in compilers and other programs which manipulate such expressions. In computer systems used for supporting product design, development and manufacturing trees are used to represent the hierarchical composition of a product through its various subsystems. Such information is needed, for example, to produce bills
of materials for production planning.
8.2. Traversing a tree
A process involving inspecting nodes and processing the data associated with them is often
called traversing a tree. The term traversing expresses the notion that many or all of the
nodes must be “visited” or scanned in this process. The nodes of tree may be scanned and
processed in different sequences; different sequences give rise to different ways of traversing
the tree. Three common ways of traversing a tree are inorder, preorder and postorder traversals. They differ in the order in which the parent and children nodes are processed.
In a preorder transversal of a (sub)tree, the root node of the (sub)tree in question is processed
first, then its subtrees are traversed in order, beginning with the leftmost subtree and ending
with the rightmost.
DDS Lecture Notes (draft)
- 61 -
Robert L. Baber, 1999 July
In a postorder transversal of a (sub)tree, first its subtrees are traversed in order, beginning
with the leftmost subtree and ending with the rightmost subtree. Finally, the root node of the
(sub)tree in question is processed.
In an inorder transversal of a binary (sub)tree, its left subtree is transversed first, then the root
node of the (sub)tree in question is processed, and finally, its right subtree is transversed. For
trees other than binary trees, this procedure must be suitably generalized, depending upon the
interpretation of the data structure represented by the tree. One possibility, which is often
meaningful, is to process the root node of the (sub)tree in question between the traversals of
its subtrees.
Variants of the three traversal procedures are also found in practice. For example, an inorder
traversal is sometimes supplemented by an initial and a final procedure. I.e., the initial procedure is first performed, then the sequence of subtree traversals and root node processing is
executed as described above, and, lastly, the final procedure is performed.
Example (mathematical expression): Consider the following tree:
*
x
+
*
a
c
b
Clearly this tree could be interpreted as representing the arithmetic expression (x*(a*b + c)).
Question: Why?
Consider the following variant of an inorder traversal algorithm. Its parameter is a pointer to
the root node of a (sub)tree.
DDS Lecture Notes (draft)
- 62 -
Robert L. Baber, 1999 July
procedure outputexpression(treeroot)
if node treeroot has children nodes
then output “(“
outputexpression(leftsubtree(treeroot))
output the data (arithmetic operation) at node treeroot
outputexpression(rightsubtree(treeroot))
output “)”
else (node treeroot is a leaf node)
output the data (variable name) at node treeroot
endif
endprocedure
This procedure will output a fully parenthesized expression with the same arithmetic meaning
as the tree structure beginning at the node to which the input parameter points.
Applied to the tree in the preceding diagram, this procedure outputexpression will result in
the following output string:
(x*((a*b)+c))
The following procedure is a postorder traversal algorithm for such a tree:
procedure outputpost(treeroot)
if node treeroot has children nodes
then outputpost(leftsubtree(treeroot))
outputpost(rightsubtree(treeroot))
output the data (arithmetic operation) at node treeroot
else (node treeroot is a leaf node)
output the data (variable name) at node treeroot
endif
endprocedure
Applied to the tree in the preceding diagram, this procedure outputpost will output the following string:
xab*c+*
We will see in section 9 that an expression in this form (called Polish postfix notation) is
another meaningful way of describing or formulating the computation represented by the tree
and by the expression (x*((a*b)+c)).
The following procedure is a preorder traversal algorithm for such a tree:
procedure outputpre(treeroot)
if node treeroot has children nodes
then output the data (arithmetic operation) at node treeroot
outputpre(leftsubtree(treeroot))
outputpre(rightsubtree(treeroot))
else (node treeroot is a leaf node)
output the data (variable name) at node treeroot
endif
endprocedure
DDS Lecture Notes (draft)
- 63 -
Robert L. Baber, 1999 July
Applied to the tree in the preceding diagram, this procedure outputpre will output the following string:
*x+*abc
In section 9 we will see that also this expression is a meaningful way of describing the computation represented by the other expressions and the tree above. This form for an expression
is called Polish prefix notation.
Example (bill of materials): Consider a production planning activity concerned with making
desks. In part of the planning process the total quantities of the various materials required to
produce the desired number of desks must be calculated. This calculation is often called a
“bill of materials explosion”. A tree structure for representing the data upon which a bill of
materials explosion is based lends itself well to the hierarchical composition of such a product consisting of various subassemblies and components.
In this simplified example we will assume that:
Each desk consists of
2 pedestals and
1 top.
Each pedestal consists of
1 pedestal frame,
2 feet and
3 drawers.
Each pedestal frame is an end item (e.g. externally procured).
Each foot is an end item.
Each drawer consists of
1 side and bottom trough,
1 back panel and
1 front panel.
Each side and bottom trough is an end item.
Each back panel is an end item.
Each front panel is an end item.
DDS Lecture Notes (draft)
- 64 -
Robert L. Baber, 1999 July
Each top is an end item.
The above data could be represented by the following tree:
desk
2
pedestal
1 2 3
pedestal frame
top
foot
1
trough
1
drawer
1 1
back panel
front panel
The following procedure will traverse such a tree and output a list of the quantities of the end
items required to make a given number of desks. Its parameters are (1) the quantity of the
product required and (2) a pointer to the root node of the (sub)tree for the product required.
procedure listenditems(quantityrequired, treeroot)
if node treeroot has children nodes
then for each child of the node treeroot
listenditems(quantityrequired*quantityperproduct(treeroot, child),
pointertochild(treeroot, child))
endfor
else (node treeroot is a leaf node)
output quantityrequired and the data (name of the end item) at node treeroot
endif
endprocedure
In this tree, each node contains the name of the product or subassembly in question and, for
each child, (1) the quantity of the child (subsidiary component) required to make one unit of
the product or subassembly in question and (2) a pointer to the subtree defining the child and
its composition.
If the above procedure were invoked with a value of 10 for the parameter quantityrequired
and a pointer to the node for “desk” for the parameter treeroot, the following list would be
produced:
20 pedestal frame
40 foot
60 back panel
60 front panel
DDS Lecture Notes (draft)
- 65 -
Robert L. Baber, 1999 July
60 trough
10 top
These end items are required in these quantities in order to make 10 desks.
In an actual production planning application additional data would be contained in the various nodes, such as the cost of each component or subassembly, the cost of assembling each
product (i.e. the cost of labor, energy, amortization of manufacturing equipment, etc.), production times, etc. The structure of the data collection would be the same, only more details
would be provided and processed. Furthermore, the above example has been simplified;
many parts such as fasteners, packaging materials, etc. have been omitted. Again, a realistic
application system would exhibit the same structure, only more details of the same type as
already included above would be added.
Question: How could a drawing of the desk be represented by a data structure of the above
type? What data would be required in each node?
Question: Typical programming languages require that the number of data items (including
pointers to other nodes) associated with each type of node be bounded by a fixed number.
How can a data structure in which a node may have any number of children be represented in
such a system?
8.3. Creating a tree
Various algorithms can be designed to create a tree from equivalent expressions in different
forms. An expression in the form corresponding to the output of a preorder scan is particularly convenient as a basis for generating a tree.
When creating a tree, nodes with empty data fields must be made available. In programming
language systems supporting tree structures, a system function is typically provided which,
when called, will allocate memory for a new node and return a pointer to that node. Such a
system function often removes the memory required from a list of free memory as described
earlier in section 6.7 above.
Example: The following algorithm (in pseudocode form) will generate the tree corresponding to an input expression in Polish prefix notation (e.g. *x+*abc, cf. the example of the
mathematical expression in section 8.2 above). The result parameter pointer will point to the
root node of the tree generated. The input expression must be a non-empty, syntactically
correct expression in Polish prefix notation.
Note that the example expression involves only binary operators (operators with exactly two
arguments each). Thus its corresponding tree will be a binary tree. If operators with a larger
number of arguments could appear in the expression, the corresponding tree would no longer
be a binary tree.
The basic idea underlying the following algorithm is that when it is invoked, a node is created
for the first symbol in the expression (either an operator or a value or variable name). The rest
of the expression consists of the expressions for the left subtree and the right subtree. These
two expressions are separated and the subtree for each is generated by (recursively) calling
the procedure for generating a tree. Links from the parent node to the subtrees are established.
Question: When is such an expression syntactically correct and in Polish prefix notational
form? Give a rule for determining whether a string of symbols meets this requirement.
DDS Lecture Notes (draft)
- 66 -
Robert L. Baber, 1999 July
procedure buildtree(expression, rootnode)
Separate expression into firstsymbol, leftsubexpression and rightsubexpression.
Allocate a new node and set rootnode to point to the new node.
Copy the value of firstsymbol into the data field of node rootnode.
if firstsymbol is an operator (e.g. +, *, etc.)
then
buildtree(leftsubexpression, subnode)
In the node to which rootnode points, set the pointer to the left subtree to subnode.
buildtree(rightsubexpression, subnode)
In the node to which rootnode points, set the pointer to the right subtree to subnode.
else {firstsymbol is a value or variable}
In the node to which rootnode points, set the pointer to the left subtree to the end
marker value.
[see note below]
In the node to which rootnode points, set the pointer to the right subtree to the end
marker value.
[see note below]
endif
endprocedure
Note: Some systems will provide the newly created node with these pointer fields already
initialized to the end marker value. In such cases, these statements (and hence the entire else
part of the if statement) are superfluous and may be omitted.
Question: Is termination of the above algorithm guaranteed? Why?
Questions: How can a Polish prefix expression be separated into the first symbol and its two
subexpressions (i.e. on the basis of what criteria)? How does this process relate to the criterion for a syntactically correct expression (cf. previous question above). How does this criterion relate to the structure of the algorithm above and, in particular, its correctness?
Question: Separating an expression in Polish prefix notation into the first symbol and two
subexpressions (cf. the first step in the above algorithm) will presumably involve scanning
the expression, or at least a significant part of it, symbol by symbol. How can the algorithm
above be reorganized to eliminate the need for this process and, thereby, reduce the time
complexity of generating the tree?
DDS Lecture Notes (draft)
- 67 -
Robert L. Baber, 1999 July
9. Polish notation
9.1. Expressions in infix, Polish prefix, Polish postfix and tree form
In section 8.2 above it was mentioned that the tree
*
x
+
*
a
c
b
and the expressions
*x+*abc
[Polish prefix notation]
xab*c+*
[Polish postfix notation]
(x*(a*b + c))
[infix notation]
can all be interpreted to have the same meaning and are equivalent to each other in this sense.
The first two expressions above are in a form called Polish notation. Notice that they are both
free of parentheses, which gives rise to one of their advantages: the order of computation is
specified by the sequence of symbols only.
The more familiar and, by people at least, more commonly used notation is called infix notation to distinguish it from the two Polish notational forms. The term infix was suggested by
the fact that the symbols for the mathematical operations are interspersed within the expression, between their operands. When writing an expression in infix notation, the sequence of
symbols representing the variables and the operations is not sufficient to specify the intended
mathematical function; parentheses are often also needed to specify the order of computation.
The tree clearly suggests the following sequence of computations:
1. multiply the values of the variables a and b together
2. add the product obtained in step 1 above and the value of the variable c
3. multiply the value of the variable x and the sum obtained in step 2 above to give the
final result
The expression in infix form above also indicates the same sequence of computational steps.
DDS Lecture Notes (draft)
- 68 -
Robert L. Baber, 1999 July
An expression in Polish postfix notation is often interpreted with the help of a stack. The
expression is scanned from left to right. Each variable or value encountered is pushed onto
the stack. Whenever a symbol for an operation is encountered, the appropriate number (in the
above example, 2) of variables is popped from the stack and the operation applied to them.
The result is pushed onto the stack and the scan of the expression continued. After the expression has been completely scanned, the result is the only item left in the stack. Alternatively, instead of performing the calculation, machine language instructions to perform the calculation can be easily generated; this technique is often used in compilers and interpreters.
9.2. Converting expressions between infix, Polish notation and tree form
We have already encountered several algorithms for transforming expressions between tree,
infix, Polish prefix and Polish postfix forms; they are illustrated in the following diagram.
The number on each arrow representing a conversion indicates the section in which the algorithm appears.
tree
8.2
8.3
8.2
infix
8.2
Polish
prefix
Polish
postfix
In the sections below algorithms for other conversions between these forms will be given.
They should be considered as examples, not the only or the best way of performing the conversion in question.
9.2.1. Converting from fully parenthesized infix to Polish postfix notation
The following algorithm will convert an expression from fully parenthesized infix form to the
equivalent expression in Polish postfix notation. The logic in this algorithm depends critically
on the input expression being fully parenthesized, that is, a pair of parentheses encloses each
operator together with its operands. Furthermore, only a single pair of parentheses may enclose each such group. Expressed differently, the number of left parentheses, the number of
right parentheses and the number of operator symbols must all be equal to each other.
The input to the algorithm is the expression in fully parenthesized infix form. The output is
an initially empty string of symbols. An initially empty stack is used in the process of generating the postfix expression from the infix expression.
DDS Lecture Notes (draft)
- 69 -
Robert L. Baber, 1999 July
The input expression in fully parenthesized infix form is scanned from left to right. For each
symbol scanned an action is performed as follows:
If the symbol scanned is
a value or variable
an operator
right parenthesis )
left parenthesis (
the action below is performed.
the value or variable is appended to the output string
the operator is pushed onto the stack
the operator at the top of the stack is popped and appended
to the output string
no action (the left parenthesis is skipped over)
The algorithm terminates after the last symbol in the input expression has been scanned and
processed, i.e. when there is no more symbol in the input string.
Basically, this algorithm moves each operator to the right of its right hand subexpression,
immediately preceding its corresponding right parenthesis. In the resulting expression, the
parentheses would be superfluous, so they are removed in the conversion process.
Question: What is the time complexity of this algorithm? memory space complexity?
Question (optional): How can an expression in infix form but not necessarily fully parenthesized be converted into Polish postfix notation? Consider operator precedence rules in the
absence of corresponding parentheses.
9.2.2. Converting from fully parenthesized infix to Polish prefix notation
An expression can be converted from fully parenthesized infix form to Polish prefix notation
by an algorithm basically symmetric to the one in section 9.2.1 above. Again here, the logic
in this algorithm depends critically on the input expression being fully parenthesized, that is,
a single pair of parentheses encloses each operator together with its operands. The number of
left parentheses, the number of right parentheses and the number of operator symbols must all
be equal to each other.
The input to the algorithm is the expression in fully parenthesized infix form. The output is
an initially empty string of symbols. An initially empty stack is used in the process of generating the prefix expression from the infix expression.
The input expression in fully parenthesized infix form is scanned from right to left. For each
symbol scanned an action is performed as follows:
If the symbol scanned is
a value or variable
an operator
left parenthesis (
right parenthesis )
the action below is performed.
the value or variable is prefixed to the output string
the operator is pushed onto the stack
the operator at the top of the stack is popped and prefixed
to the output string
no action (the right parenthesis is skipped over)
The algorithm terminates after the leftmost symbol in the input expression has been scanned
and processed, i.e. when there is no more symbol in the input string.
Basically, this algorithm moves each operator to the left of its left hand subexpression, immediately to the right of its corresponding left parenthesis. In the resulting expression, the
parentheses would be superfluous, so they are removed in the conversion process.
Question: What is the time complexity of this algorithm? memory space complexity?
DDS Lecture Notes (draft)
- 70 -
Robert L. Baber, 1999 July
Question (optional): How can an expression in infix form but not necessarily fully parenthesized be converted into Polish prefix notation? Consider operator precedence rules in the
absence of corresponding parentheses.
9.2.3. Converting from Polish postfix to fully parenthesized infix notation
In this algorithm we will make use of a stack, each element of which is a string. Each element
of the stack will be a fully parenthesized infix expression or a single value or variable. Because a single value or variable contains no operation and no parentheses, it can be viewed as
a special case of a fully parenthesized infix expression. Thus, every element in the stack can
be considered to be a fully parenthesized infix expression.
The input to the algorithm is the expression in Polish postfix notation. The output is a string
of symbols as the only element in the stack. An initially empty stack is used in the process of
generating the fully parenthesized infix expression from the input expression in Polish postfix
notation.
The input expression in Polish postfix notation is scanned from left to right. For each symbol
scanned an action is performed as follows:
If the symbol scanned is
a value or variable
an operator
the action below is performed.
The value or variable is pushed onto the stack.
The top of the stack is popped to the working variable
rightexpression. The next element in the stack is popped to
the working variable leftexpression. The string formed by
concatenating a left parenthesis, leftexpression, the operator
being scanned, rightexpression and a right parenthesis (in
that order) is pushed onto the stack.
This algorithm terminates after the last symbol in the input expression has been scanned and
processed, i.e. when there is no more symbol in the input string. Upon termination, there will
be exactly one element in the stack and it will contain the fully parenthesized infix expression
equivalent to the input expression in Polish postfix notation.
Basically, this algorithm scans the expression in Polish postfix notation and for each operator,
forms the corresponding fully parenthesized infix expression, retaining it in the stack. The
final contents of the stack will be the fully parenthesized infix expression for the last operator
in the input expression, which is the desired expression in fully parenthesized infix notation.
Question: What is the time complexity of this algorithm? memory space complexity?
Having added the above three algorithms to our collection, we now have algorithms for the
following conversions:
DDS Lecture Notes (draft)
- 71 -
Robert L. Baber, 1999 July
tree
8.2
8.2
infix
8.2
9.2.2
9.2.1
8.3
9.2.3
Polish
prefix
Polish
postfix
As before, the number on each arrow representing a conversion indicates the section in which
the algorithm appears.
This set of conversion algorithms forms a complete set in the sense that it enables us to convert an expression in any of these notational forms to an expression in any other, either directly or indirectly. Of course, algorithms for the other direct conversions can be designed.
DDS Lecture Notes (draft)
- 72 -
Robert L. Baber, 1999 July
10. Other types of trees
Basic and fundamental aspects of tree data structures were presented in chapter 8. One special type of tree was introduced there: a binary tree. There are, of course, many other special
cases of trees, some of which will be examined in this chapter.
By placing additional restrictions on tree structures, certain desirable characteristics and
properties can be obtained, enabling special goals to be achieved and/or facilitating processing the tree.
10.1. Binary search trees
A binary search tree is a binary tree (see the definition in section 8.1 above) which satisfies
the following additional conditions:
1. Each node’s data value is an element of a linearly ordered set (i.e., an order relation is
defined on the set of the nodes’ data values).
2. For every node N in the tree
2.1. the value of every node L in the left subtree of N is less than the value of N and
2.2. the value of every node R in the right subtree of N is greater than the value of N.
Requirement 2.1 above can be modified by replacing the relation “less than” by “less than or
equal to”. Similarly, requirement 2.2 above can be modified by replacing the relation “greater
than” by “greater than or equal to”.
Phrases such as “the data value of node A” or “the data value at node A” occur often and
sometimes give rise to lengthy and even clumsy sentences. Such phrases are often shortened
to “node A” or even just “A” when it is clear from the context precisely what is meant. I.e.,
the node identification alone often stands for the data value at (of) that node. Rephrased in
this way the conditions above become:
1. Each node is an element of a linearly ordered set.
2. For every node N in the tree
2.1. every node L in the left subtree of N is less than (less than or equal to) N and
2.2. every node R in the right subtree of N is greater than (greater than or equal to) N.
Example:
18
6
40
3
1
10
4
DDS Lecture Notes (draft)
7
25
13
19
- 73 -
60
28
42
85
Robert L. Baber, 1999 July
Questions: How many nodes can a tree of height 2 contain? height 3? height h?
Questions: What is the minimum height of a tree containing n nodes? the maximum height?
Questions: What is the minimum number of nodes which must be examined to find a node
containing a given value? the maximum number of nodes? the average number of nodes?
Note that when the tree is balanced the maximum number of nodes which must be searched is
significantly less than when the tree is unbalanced. Thus, the better balanced the tree is, the
higher the searching efficiency.
Questions: Given that a binary search tree contains 7 data elements (and hence nodes), what
is the minimum possible height of the tree? the maximum possible height? if the tree contains
n data elements?
Note that a non-leaf node in a binary search tree need not have 2 children; it may have only 1.
In fact, it may occur that every non-leaf node has only 1 child. The data in the tree in the
above diagram could, for example, be organized differently so that no node has more than 1
child.
Examples:
85
60
42
40
...
or
DDS Lecture Notes (draft)
- 74 -
Robert L. Baber, 1999 July
85
1
60
42
...
Exercise: Design an algorithm which searches a binary search tree for a node containing a
specific value. Assume that the variable top points to the root node of the binary tree and that
the value of the variable searchkey is the value to be located in the tree. The algorithm should
set the variable nodefound to point to the node containing the value sought. If that value is
not present in the tree, the algorithm should set the variable nodefound to the end marker
value (endmark).
Exercise: Outline an algorithm which deletes a specific node in a binary search tree. What
information must be available to such an algorithm?
Exercise: Outline an algorithm which inserts a given value into a binary search tree.
Questions: How many nodes must be examined or processed when searching a binary search
tree and inserting and deleting nodes? Distinguish between the best, average and worst cases.
10.2. Heaps
A heap is a binary tree (see the definition in section 8.1 above) which satisfies the following
additional conditions:
1. Each node is an element of a linearly ordered set. I.e., an order relation is defined on the
set of the nodes’ data values.
2. The tree is complete*, that is, every level has the maximum possible number of nodes
except possibly the lowest (deepest) level, and the nodes at the lowest level are in their left
most positions.
3. Each node in the tree is less than every one of its children. (This requirement can be modified by replacing the relation “less than” by “less than or equal to”, “greater than” or
“greater than or equal to”).
* Note: The mathematical and computing science literature contains different definitions of
the term “complete tree” which are inconsistent with one another. There is no one single
universally accepted meaning of this term. The above definition will be used throughout this
document.
Example: The following diagram shows a data collection arranged as a heap.
DDS Lecture Notes (draft)
- 75 -
Robert L. Baber, 1999 July
4
6
[1]
[2]
40 [4]
30 [3]
49 [5]
33 [6]
55
43
50
52
39
[8]
[9]
[10]
[11]
[12]
80 [7]
The numbers in the circles representing the nodes are the data values.
The numbers in square brackets next to or below the circles representing the nodes are sequence numbers. Note that these sequence numbers form a set of consecutive integers, each
of which identifies the exact location of its associated node, i.e. both the level number and the
position within that level. Note further that
de(i) < de(2i) and
de(i) < de(2i+1)
for all applicable values of i.
Question: If the heap contains n nodes (data elements), what are the applicable values of the
sequence numbers i?
Question: Why is it possible to assign sequence numbers with the above properties to the
nodes of a heap? Why not to other types of binary trees?
Because the sequence numbers form a set of consecutive integers, a heap can be implemented
efficiently in an array. No array cells will be left unused. Furthermore, pointers from one
node to another are superfluous and can be eliminated, saving storage space.
Question: For what function is the heap particularly suited? Which restriction in the definition of a heap gives rise to its special advantage for this function?
Question: How can a given data value be found in a heap? a new data value inserted into a
heap? deleted from a heap? (Outline the algorithms.)
Exercise: Compare the binary search tree and the heap with regard to the efficiency (speed
and memory requirements) of the following operations:
 searching for a node with a given data value,
 inserting a new data element,
 deleting a specific data value and
 sorting.
DDS Lecture Notes (draft)
- 76 -
Robert L. Baber, 1999 July
10.3. B-Trees
In the tree structures described in the preceding sections of this chapter, each node contained
one data element (value) of significance to the structure of the tree. In the case of a B-tree this
restriction is relaxed; more than one such value may be present in any node. The number of
values in any one node is, however, bounded to facilitate implementation of the B-tree.
Let d be a positive integer. A B-tree of order d is a tree satisfying the following conditions:
1. The root node contains between 0 and 2d (inclusive) data elements.
2. If the root node contains 0 data elements, then it has no children and the B-tree is empty.
3. Each node other than the root node contains between d and 2d (inclusive) data elements.
4. The number of children of any node is either zero or one more than the number of data
elements it contains. That is, if a node contains k data elements, then that node has either 0
or k+1 children.
5. Each data element (value) in a node is an element of a linearly ordered set. I.e., an order
relation is defined on the set of the nodes’ data elements.
6. The data elements within each node are sorted in ascending order. I.e., if a node contains k
data elements called de(1), de(2), ... de(k), then de(1)  de(2)  ...  de(k).
7. If node N contains k data elements (called de(1), ... de(k) below) and k  1, then
7.1. every element in the first (left most) subtree of node N is less than or equal to de(1),
7.2. every element in the jth subtree of node N is between de(j-1) and de(j) inclusive, for
1 < j  k, and
7.3. every element in the k+1st (last, right most) subtree of node N is greater than or equal
to de(k).
In basic concept the B-tree is a generalization of a binary search tree with additional constraints imposed which tend to keep the tree balanced (but do not guarantee it). This, in turn,
tends to maintain efficiency, e.g. of searching.
Exercise: Compare the B-tree with the binary search tree and the heap with regard to the
efficiency (speed and memory requirements) of the following operations:
 searching for a node with a given data value,
 inserting a new data element,
 deleting a specific data value and
 sorting.
DDS Lecture Notes (draft)
- 77 -
Robert L. Baber, 1999 July
11. Comparison of data structures and their algorithms
The data structures with their attendant algorithms can be compared with regard to a number
of criteria. Two of the most important criteria for assessing algorithms in general are time and
memory complexity, i.e. the asymptotic behaviour of the time or memory required as a function of the “size” of the problem, e.g. the number of data elements in the collection to be
stored and processed.
Most of the algorithms discussed in this document have the same or similar memory complexity, so we will not consider this aspect of these algorithms further. With regard to time
complexity, however, the algorithms associated with the data structures introduced in this
document vary considerably with regard to time complexity. The following table indicates
the time complexity of various functions for several data structures. Where different complexities are given, they are for different common algorithms or for best, average and worst
cases. Also, in some cases of algorithms for operating on trees, the time complexity depends
upon how well balanced the tree is.
Data structure
search
insert
delete
sort
linear array
(not sorted)
O(n)
constant
O(n)
O(n*log(n))
O(n1.5)
O(n2)
linear array
(sorted)
O(log(n))
O(n)
O(n)
O(n)
n.a.
linked list
(not sorted)
O(n)
constant
constant
O(n2)
linked list
(sorted)
O(n)
constant
constant
n.a.
pipeline, stack
n.a.
constant
constant
n.a.
binary search tree
O(log(n))
O(n)
constant
O(log(n))
O(n)
n.a.
heap
O(n)
O(log(n))
O(log(n))
n.a.
B-tree
O(log(n))
O(n)
constant
O(log(n))
O(n)
constant
O(log(n))
O(n)
n.a.
In the cases of inserting and deleting an item above, it is assumed that the location in question
is known. If this is not the case, the time for a search must be considered also.
DDS Lecture Notes (draft)
- 78 -
Robert L. Baber, 1999 July
Exercise: The suitability of the several data structures for various processing functions also
varies significantly. Assess the several data structures with regard to the functions (e.g.
searching, inserting, deleting, sorting, buffering a data stream, etc.) for which they are particularly suited or not suited and fill in the table below correspondingly.
Data structure
Well suited for
Suited or adequate for
Not suited for
linear array
(not sorted)
linear array
(sorted)
linked list
(not sorted)
linked list
(sorted)
pipeline, stack
binary search tree
heap
B-tree
tree (general)
multi-dimensional
array
DDS Lecture Notes (draft)
- 79 -
Robert L. Baber, 1999 July
12. Selected other data structures
A number of other data structures can be defined and are useful for various applications.
Most are variants, generalizations, extensions, etc. of the basic types described in the sections
above.
The following sections briefly introduce some of these other data structures. A more detailed
treatment of these data structures is beyond the scope of this course. Extensive literature on
these and other data structures exists.
12.1. Graphs
A graph is a collection of nodes and edges (cf. trees, section 8.1) but without most of the
restrictions imposed upon a tree. In contrast to a tree, a graph does not necessarily have a root
node, may have cycles (loops) and need not be connected (there may be two nodes not connected by any path). An edge connects two nodes, but need not have a direction.
A graph is frequently represented in a computer system by nodes and links of the same type
as used for linked lists and trees as described in several sections above.
For example, the following data structure is a graph, but obviously does not satisfy the definition of a tree.
X
B
P
E
T
R
L
U
F
P
D
Question: Which conditions in the definition of a tree are violated by this graph? (Cf. section
8.1.)
12.2. Hash tables
In all of the data structures described above, the procedure for locating a data item with a
given value requires, in general, examining a number of data items. Because such searches
can take some time and are performed often in many computerized systems, it would be
desirable to have a data structure in which a data item with any given value can be directly
found and accessed. The hash table provides such access, at least under certain conditions and
within certain limits.
In its basic form, a hash table is an array of records, each containing one data item or data
group. When a data item or group is to be stored in the array, an index is calculated based on
the value of one or more of the fields in the record to be stored. In the simple case, the record
is then stored in that element of the array referenced by the index value calculated. The function which maps data values to index values is called the hashing function.
DDS Lecture Notes (draft)
- 80 -
Robert L. Baber, 1999 July
Early implementations based on this idea calculated the index of a record by adding together
the individual characters in the field or fields in question, each viewed as an integer, and
either discarding high order carries or wrapping them around. The total derived in this way
was called a hash total or hash sum, hence the term hash table for a data structure based on
such a mapping function.
If a data item is to be stored in an already occupied array element, a collision is said to have
occurred. The new record is then stored in some other place; determining which is a design
feature of the system which can have a significant effect on the performance of the system,
especially with regard to its speed.
If a data item is to be located, the hash function is first calculated to give the index of the
array element in which the data item would normally be stored. If that element is empty or
contains some other element, other locations must possibly be examined, depending upon the
insertion algorithm used when collisions arise and also upon the algorithm employed for
deleting items.
The two main design criteria of such a data storage system are usually resources required
(memory space) and performance (time). An ideal hash table system consists, therefore, of an
array just large enough to contain the data to be stored and a hashing function which distributes the actual data items uniformly over the available index space without collisions. Usually
a design tradeoff is involved. The more densely occupied the array is, the higher the collision
rate will be. I.e. memory space can be reduced at the cost of increasing the collision rate and,
hence, processing time. An optimum balance between memory cost and processing time is
the design goal.
12.3. Permutation arrays
Linear sequences of data items arise in many applications. It is often desirable to sort the
sequence into various different orders. In order to avoid repeated sorting into different orders
and then back again while avoiding the obvious alternative of maintaining a number of copies
of the data in different sequences, the data may be effectively sorted by employing one or
more permutation arrays. A permutation array is an array of pointers to the original data
collection with the property that it permutes the original sequence (which need not be in any
particular order).
A data collection stored in an array or in a group of arrays may have many permutation arrays
associated with it. In this way, the data collection can be sorted in several different orders at
one time. Because the data collection is, in effect, stored in a sorted array, the binary search
algorithm can be used to find a data element with a given value.
DDS Lecture Notes (draft)
- 81 -
Robert L. Baber, 1999 July
For example, consider the following related data arrays Name and Age. The permutation
array P1 effectively orders the data alphabetically by name. The permutation array P2 orders
the data by age.
Index i
P1(i)
P2(i)
Name(i)
Age(i)
1
4
1
George
22
2
1
5
Gogo
75
3
2
4
Pieter
52
4
3
3
Christa
33
5
5
2
Themba
25
Note how P1 sorts the arrays Name and Age into alphabetical order on the names:
Index i
P1(i)
Name(P1(i))
Age(P1(i))
1
4
Christa
33
2
1
George
22
3
2
Gogo
75
4
3
Pieter
52
5
5
Themba
25
and how, at the same time, P2 sorts the arrays Name and Age by age:
Index i
P2(i)
Name(P2(i))
Age(P2(i))
1
1
George
22
2
5
Themba
25
3
4
Christa
33
4
3
Pieter
52
5
2
Gogo
75
while the original arrays Name and Age remain unchanged and are in no particular order. The
arrays P1 and P2 effectively permute the data values in the arrays Name and Age into the
desired sequences. Viewed more mathematically, P1 and P2 are one-to-one functions from
the set of index values onto itself — the general property characteristic of a permutation.
DDS Lecture Notes (draft)
- 82 -
Robert L. Baber, 1999 July
Thus, with the help of permutation arrays, a single collection of data can be in different orders at one and the same time.
Questions: How can such a data structure be searched for an entry (record) with a particular
name? with a particular age? for an entry satisfying some other criterion?
Questions: How can a new data record be added to such a data structure? a record deleted
from such a data structure?
DDS Lecture Notes (draft)
- 83 -
Robert L. Baber, 1999 July
13. Summary and conclusions
Typical computer hardware provides for only one type of data structure — a sequence of data
values, each consisting of a small group of bits (currently most commonly 8 bits, sometimes
somewhat larger groups). We have seen in this course that it is possible to realize a wide
variety of data structures with only this simple basis to build upon.
In your future work you will encounter many different types of data structures, some of
which have not yet been developed. In this course, you have been introduced to most of the
fundamental data structures in common use today. Many of the others not covered here can
be viewed as variants, extensions and specializations of the data structures presented in this
course.
No one data structure is optimum, even suitable, for all applications. Each data structure has
its advantages and disadvantages, its strengths and weaknesses for the various applications. In
particular, we have seen that the time complexity for any one type of operation varies from
one data structure to another significantly. The designer’s task is to determine the data structure or the combination thereof which results in the best overall performance in the application in question, taking into account the usage patterns which can be expected.
DDS Lecture Notes (draft)
- 84 -
Robert L. Baber, 1999 July
14. References
Baber, Robert L.; “A method for representing data items of unlimited length in a computer
memory”, IEEE Transactions on Software Engineering, Vol. SE-7, No. 6, Nov. 1981, p.
590-593.
IEEE; An American National Standard, IEEE Standard for Binary Floating-Point Arithmetic,
ANSI/IEEE Std 754-1985, IEEE, New York, 1985.
IEEE; An American National Standard, IEEE Standard for Radix-Independent FloatingPoint Arithmetic, ANSI/IEEE Std 854-1987, IEEE, New York, 1987.
Kelsey, Richard; Clinger, William; Rees, Jonathan (eds.); Revised5 Report on the Algorithmic
Language
Scheme,
http://www.cena.dgac.fr/~decrock/langages/scheme/r5rs/r5rshtml/r5rs_toc.html, 20 February 1998.
Kline, Morris; Mathematical Thought from Ancient to Modern Times, Oxford University
Press, New York, 1972.
Recommended reading
The following is a small selection of the many books which contain sections dealing with
subjects covered in this course:
Abelson, Harold; Sussman, Gerald Jay; Structure and Interpretation of Computer Programs,
MIT Press, Cambridge, Massachusetts, 1985.
Amsbury, Wayne; Data Structures from Arrays to Priority Queues, Wadsworth Publishing Co.,
Belmont, CA, 1985.
Beidler, John; An Introduction to Data Structures, Allyn and Bacon, Inc., Boston, 1982.
Brunskill, David; Turner, John; Understanding Algorithms and Data Structures, McGraw-Hill,
London, 1996.
Kruse, Robert L.; Data Structures and Program Design, Prentice-Hall, Englewood Cliffs, NJ,
1984.
Manber, Udi; Introduction to Algorithms: A Creative Approach, Addison-Wesley, Reading,
MA, 1989.
Manis, Vincent S.; Little, James J.; The Schematics of Computation, Prentice Hall, Englewood
Cliffs, New Jersey, 1995.
Patterson, David A.; Hennessy, John L.; Computer Organization & Design: the Hardware/Software Interface, Morgan Kaufmann, San Francisco, 1994.
Stubbs, Daniel F.; Webre, Neil W.; Data Structures with Abstract Data Types and Pascal,
Brooks/Cole Publishing Co., Monterey, CA, 1985.
Tenenbaum, Aaron M.; Augenstein, Moshe J.; Data Structures Using Pascal, Prentice-Hall,
Englewood Cliffs, NJ, 1986.
Warford, J. Stanley; Computer Science, Volumes 1 and 2, D. C. Heath, Lexington, MA, 1991.
Weiss, Mark Allen; Data Structures and Algorithm Analysis, Benjamin/Cummings, Redwood
City, CA, 1992.
DDS Lecture Notes (draft)
- 85 -
Robert L. Baber, 1999 July