Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Lecture Notes (Draft) COMS100B, 1999 Department of Computer Science University of the Witwatersrand, Johannesburg Data and Data Structures (DDS) Table of Contents 1. 2. Introduction 1.1. General 1.2. Objectives of this course 1.3. Contents of this course 3 3 3 3 Some basic definitions and terms 5 3. Representing numerical data values 3.1. Classical systems for representing numerical values 3.2. Positional notation 3.3. Converting integers between positional number systems with different radices 3.4. Converting fractions between positional number systems with different radices 3.5. Representing integers in a computer memory 3.6. Floating point systems 3.7. Precision and accuracy 8 8 9 10 13 16 19 22 4. Representing non-numerical data values 4.1. Standards for representing individual characters 4.2. Representing a value as a sequence of characters or symbols 23 23 24 5. Arrays, lists, vectors, sequences 5.1. Computer memory 5.2. One-dimensional arrays and their representation in a computer memory 5.3. Records 5.4. Files 5.5. Multidimensional arrays 26 26 27 30 32 32 6. Association by links and pointers 6.1. Linked lists 6.2. Pointers and links 6.3. Diagrams of linked lists 6.4. Searching a linked list 6.5. Inserting an element into a linked list 6.5.1. Case 1: inserting a new item at the beginning of a list 6.5.2. Case 2: inserting a new item after an existing element of a list 6.6. Deleting an element from a linked list 6.6.1. Case 1: deleting the first element from a list 6.6.2. Case 2: deleting an element other than the first from a list 6.7. List of available elements 35 35 38 38 39 42 42 44 46 46 48 50 DDS Lecture Notes (draft) -1- Robert L. Baber, 1999 July 7. Queues 7.1. Pipelines 7.2. Stacks 7.3. Priority Queues 52 52 55 58 8. An introduction to trees 8.1. Definitions and terminology 8.2. Traversing a tree 8.3. Creating a tree 60 60 61 66 9. Polish notation 9.1. Expressions in infix, Polish prefix, Polish postfix and tree form 9.2. Converting expressions between infix, Polish notation and tree form 9.2.1. Converting from fully parenthesized infix to Polish postfix notation 9.2.2. Converting from fully parenthesized infix to Polish prefix notation 9.2.3. Converting from Polish postfix to fully parenthesized infix notation 68 68 69 69 70 71 10. Other types of trees 10.1. Binary search trees 10.2. Heaps 10.3. B-Trees 73 73 75 77 11. Comparison of data structures and their algorithms 78 12. Selected other data structures 12.1. Graphs 12.2. Hash tables 12.3. Permutation arrays 80 80 80 81 13. Summary and conclusions 84 14. References 85 DDS Lecture Notes (draft) -2- Robert L. Baber, 1999 July 1. Introduction 1.1. General 1. 2. 3. 4. 5. introduction of lecturer recommended reading list meeting times and places teaching vs. learning administrative announcements 1.2. Objectives of this course The objectives of this course are: 1. to familiarize the student with the basic and most important types of data structures, 2. to familiarize the student with their representation and manipulation in a computer system, 3. to develop the student’s understanding of these data structures and their associated algorithms, 4. to develop the student’s ability to analyze them and 5. to make the student aware of the relative advantages and disadvantages of the various data structures and algorithms and of the tradeoffs arising when selecting and designing data structures and algorithms for different purposes. While the student will write several programs as part of the assigned exercises, acquiring detailed knowledge of and developing skills in any particular programming language are not objectives of this course. Stress will be placed on learning the various concepts independently of any programming language and its special features intended to facilitate the manipulation of certain data structures, as such special features often hide the very mechanisms and algorithms we wish to study in this course. The student should, therefore, pay particular attention in this course to the distinction between fundamental concepts vs. implementation details. Emphasis should be placed on the fundamental concepts. 1.3. Contents of this course Viewed abstractly, two main aspects characterize data: values and associations between values. Such values and associations can be represented in a number of ways, i.e. by various types of symbols and of data structures. A computer memory typically consists of one or a few sequences of cells, each cell being capable of storing one symbol selected from a limited set. This course examines various data structures (ways of representing values and associations between values), how these data structures can be represented in a computer memory, algorithms for manipulating these data structures and important characteristics (e.g. efficiency, time and space complexity) of these data structures and algorithms. We will often look at a specific data structure and consider how it can be implemented using other, simpler, more concrete and less abstract data structures. This corresponds to the designer’s task of implementing a data structure not directly supported by the target system in terms of data structures which are implemented by the available system. Ultimately, of course, the higher level data structure must be implemented in the data structure actually available in the hardware — typically a sequence of cells. DDS Lecture Notes (draft) -3- Robert L. Baber, 1999 July The main specific subjects covered in this course are: data items, data values, data types, ways of representing numerical values: positional notation; binary, decimal, hexadecimal and other radix representation; integers; floating point, converting values between different representational systems, e.g. converting numbers between different radix systems, between integer and floating point representation, data structures such as linear sequences (lists, vectors), one dimensional arrays, multidimensional arrays, lists of lists, stacks (LIFO queue), pipeline (FIFO queue), linked lists, pointers, trees, heaps, Polish notation, graphs, hash tables, records, files, algorithms (recursive and non-recursive) for searching, sorting, evaluating expressions in Polish notation, converting between different data structures, inserting and deleting elements of the various data structures, dynamic storage allocation, analyzing data structures and algorithms, e.g. regarding their efficiency, time and space complexity, etc. and examples of the practical application of the above, e.g. in compilers, operating systems and application systems. These subjects will not be presented strictly in the above sequence. Together with each data structure the relevant algorithms will be studied and analyzed and examples considered. Algorithms for converting between specific data structures will be examined at appropriate times throughout the block. DDS Lecture Notes (draft) -4- Robert L. Baber, 1999 July 2. Some basic definitions and terms A data item consists of a name and a data value, or simply value. Often we will also explicitly consider the set from which the values may be selected, in which case the data item consists of a name, a value and a set. In the latter case, the value must be an element of the set in question. A data item can also be thought of as a variable, as it has many of the characteristics of a variable in mathematics. A data value, or simply value, is one of the components of a data item (see the paragraph above). A value is represented by a symbol or a (possibly empty) sequence of symbols. The set of symbols used for representing values, e.g. in a particular computer system, is usually rather limited. Ultimately, at the lowest level in a computer system, such a set typically consists of only two elements, often represented in written form by the symbols 0 and 1. It is sometimes important to distinguish between the meaning or interpretation of a value and its representation. Within a computer system only the representation is of concern. The designers and users of a computer system usually interpret the symbols or sequences of symbols representing values in some way, that is, they attach meanings to such representations of values. The connection between meaning and representation will be of concern to both designers and users, but such interpretation takes place outside the computer system itself. The computer system itself processes and manipulates symbols representing data values according to fixed rules, without regard to the meanings humans attach to those values or their representations. For example, the number five may be represented by the single symbol 5, the symbol V, the sequence of letter symbols “five”, the sequence of bits 101, the sequence of tertiary digits 12, etc. All of these representations could be interpreted to mean the same thing, i.e. a certain quantity of unspecified things. A data type, also called abstract data type or ADT, consists of a set (of values) and a collection of operations on that set of values. An association between values is any sort of logical connection between them. An association may be explicitly represented or it may be implicit, i.e. an association only in the minds of humans such as the designers or users of the computer system in question. In either case, such associations are the basis for data structures. A (one dimensional) array is a sequence of values. Other terms are also used to refer to a sequence of values, such as list and vector. (Note that the term “vector” here is used in the computing sense, not the purely mathematical sense. These are related, but are not the same.) Sometimes, within the context of a specific programming language, these different terms imply different methods of implementing the sequence. For our purposes here, however, the sequential ordering between elements of the array, list or vector is the essential characteristic of interest. Mathematically, an array, a list, a vector or a sequence can be thought of as a function which maps an integer to a value. The natural ordering of the integers (sequence number, index) defines the ordering of the values in the sequence. The ordering of the elements (values) in a sequence can serve to associate those elements with one another. Another possibility is to associate elements in the same position of two or more different sequences with each other. Example: The following table expresses certain associations between the values “George”, “Themba”, 22 and 25: DDS Lecture Notes (draft) -5- Robert L. Baber, 1999 July Sequence number 1 2 Name George Themba Age 22 25 Here, the name “George” and the age 22 are associated with one another by being on the same line (spatially) as well as both having the same sequence number (index). Similarly, the name “Themba” is associated with the age 25 and the sequence number 2. In a computer system, these data might be stored in two one dimensional arrays (e.g. Name and Age) such that Data item name Name(1) Name(2) Age(1) Age(2) Data item value George Themba 22 25 The values “George” and 22 are associated by their common index value (1) in the two arrays Name and Age. The values “Themba” and 25 are associated with one another in the same way. Note that the association between the arrays Name and Age (i.e. between each element of the array Name and the corresponding element of the array Age) is implicit. Nevertheless, this association serves to associate, and hence structure, the data contained in the two arrays. Alternatively, the above associations can be represented by forming sequences (sometimes called lists or vectors) of the values. For example, the above data collection could be written as a list of lists: ( (George 22) (Themba 25) ) Here each person is represented by a list of data values (his or her name and age). The lists for the individual persons are then put into a sequence to form a higher level list. Alternatively, these associations between data values could be represented in the structure ( (George Themba) (22 25) ) in which a list of names is followed by a list of ages. Each name is associated with the number (age) appearing in the corresponding position in the other list. Still another alternative for structuring this data is in the form of two separate lists (George Themba) (22 25) which are not explicitly associated with each other. They are instead only implicitly associated by the way they are interpreted by the designers and users of the system in which they are embedded, as in the case of the two arrays above. This implicit association will be reflected in the way the two lists are referred to in the program and its documentation. Presumably, this data will be interpreted to mean that some person named George is 22 years old and that some other person named Themba is 25 years old. This interpretation of the various data values and their associations is, however, external to the system and exists only in the mind of the reader. Note that this interpretation is based on several implicit associations not expressed in any way in the tables above: DDS Lecture Notes (draft) -6- Robert L. Baber, 1999 July the usual meanings of the sequences of symbols “Name” and “Age” in the English language the sequences of symbols “George” and “Themba” are common names of persons the sequences of symbols “22” and “25” are usually interpreted as decimal numbers ages of persons are usually measured in years and the numbers appearing above are plausible ages of human beings To someone with no knowledge of the English language and having lived only in a society in which the names George and Themba are unknown, this interpretation would be neither obvious nor natural. To people in ancient Rome, not even the numbers 22 and 25 would have been recognizable as such; only the sequences of symbols “XXII” and “XXV” would have conveyed these meanings to them. A data structure is, in short, a collection of values and associations among them. We will study specific types of data structures during this course. In summary, the following terms were introduced in this section. data item data value, or simply value array, list, vector (sequence) meaning representation data type association (between values) interpretation (both of values and of associations) data structure The student should review these terms, their meanings and the relationships between them. These terms will recur frequently in this and subsequent topics. DDS Lecture Notes (draft) -7- Robert L. Baber, 1999 July 3. Representing numerical data values 3.1. Classical systems for representing numerical values In early human societies numbers were often represented (if at all) using systems in which each symbol represented one specific number. A symbol was repeated and written together with other symbols as necessary to represent any particular number. The numerical value of a string of such symbols is determined by adding the values of the individual symbols or, in some systems and in some cases, subtracting or even multiplying the symbols’ values. For example, in the Roman number system, XIV = 10–1+5 = 14, and in the Attic system used in Greece around 450 BC, = 10+5+1+1+1 = 18, but = 5*10 = 50 and = 5*100 = 500 [Kline]. In such systems only a limited set of symbols is available. If a limit is placed on how many times any one symbol may be repeated, there is, in turn, a limit to the numbers that can be represented; larger numbers simply cannot be written in the system. Today the Roman number system is perhaps the best known number system of these types. In a sense it is a simple system, but certain operations are difficult to perform in it. Adding is not too difficult (add, for example, XVI (16) and VI (6) by writing them together to obtain XVVII which is normalized to XXII), but try multiplying these numbers together. Dividing is even more complicated. To overcome these limitations and shortcomings, e.g. to make it possible to represent any number, no matter how large, and to simplify arithmetic operations, the Arabic positional number system was introduced. In such a system the value associated with any symbol in a string is the basic value of the symbol multiplied by a factor depending upon its position in the number. That factor is usually an integral power of some number, called the radix of the system in question. For example, in the decimal (radix ten) positional system currently in widespread use throughout the world, the symbol 3 means either three (3*100), thirty (3*101), three hundred (3*102), three thousand (3*103), etc., depending upon the position in which it appears. Some older number systems included both non-positional and positional characteristics. The Akkadian system used in Babylonia around 2000 BC, for example, employed several symbols which were used to write numbers up to 59 in a non-positional scheme. Positional notation was then used to represent larger numbers, so that the system amounted to a positional system with radix 60. Initially, this system had no symbol for zero, so strings of symbols could be ambiguous, depending upon the position intended. E.g. the symbol for 1 followed by the symbol for 20 could mean either 80 (1*601+20) or 3620 (1*602+20). Spacing was often used to indicate no symbol in a particular position, but this did not eliminate the possiblity of misinterpreting the string of symbols. Much later a symbol indicating a space (i.e. the lack of a numerical symbol in the position in question) was introduced. [Kline] Today we would recognize such a separation symbol as the symbol for the numerical value zero. A system in use in Greece from about 300 BC used a similar mixture of non-positional and positional notation to represent numbers. A symbol-additive scheme was used to indicate numbers up to 999. Larger numbers were written in a positional notation. The system was, in effect, a positional system with radix 1000, but, like the Akkadian system outlined above, with a sequences of symbols in each position, rather than only a single symbol in each position as in our current system. [Kline] DDS Lecture Notes (draft) -8- Robert L. Baber, 1999 July 3.2. Positional notation The system for writing numbers which has crystallized out of the experience gained with using various systems over several thousand years has the following key characteristics: A limited set of symbols is used. Each symbol has its own inherent numerical value. These values are zero, one, two, etc. A number is represented by a sequence of symbols. Each sequence is bounded in length (“finite”) but no upper limit is placed on that length. A positional value is associated with each position in the sequence. This value is the radix of the number system raised to the power corresponding to the position in the sequence. These positions are counted from right to left, beginning with zero. The number of symbols used is the same as the radix of the system. The numerical value represented by any particular symbol is the product of its own inherent value and the value associated with its position in the number string. The value represented by the sequence of symbols is the sum of the numerical values represented by the individual symbols. The radix of the system in question is a positive integer often represented by R. Thus, a particular positional number system is characterized by its radix R and a set of R symbols representing the values 0, 1, 2, … R-1. The value of the number represented by the sequence of symbols sn sn–1 ... s1 s0 is defined to be n i=0 val(si) * Ri where the function val maps each symbol to its own inherent numerical value. Usually each symbol is interpreted to be that value itself, so the above can also be written as n i=0 si * Ri Such a positional notational system can represent any non-negative integer and only nonnegative integers. Negative integers are usually represented by appending another special symbol, e.g. “–”, to the sequence of symbols. Usually the minus sign is written before the sequence of symbols representing the number, but sometimes after. The above system can be extended easily to non-integer values by allowing the sequence to continue to the right and by placing another special symbol (e.g. “.” or, in some countries, “,” or “-”, etc.) to separate the integer part from the fractional part of the sequence. For example, the value of the number represented by the sequence of symbols sn sn–1 ... s1 s0 . s–1 s–2 ... s–(m–1) s–m is defined to be n i=–m si * Ri The system in common use today in the world is the positional notational system with radix ten, called the decimal system. It is often said that ten was chosen because human beings DDS Lecture Notes (draft) -9- Robert L. Baber, 1999 July normally have ten fingers. However, with ten fingers one can represent 11 different symbols (in a non-positional scheme) by using no fingers for zero, so one might claim that if this had been the reason for choosing a radix, 11 would or should have been chosen instead of 10. Probably the presumed reason is correct but it was not applied completely consequently. Note that the above description of positional notation defines not one system, but a family of systems. Each system in the family is characterized completely by its radix R. Note also that if R=1, only one symbol may be used and its own inherent numerical value must be zero. Thus, in the positional number system with radix 1, only the value zero can be represented. This system is, therefore, of no practical interest and will not be considered further. Only systems with radix 2 or greater are of significance. Below we will deal only with these. The values of the symbols used are 0, 1, ... R–1. Given a number and a radix, the representation of that number in that radix system is unique except for leading left zeroes before and trailing right zeroes after a “.” separating symbol. I.e., one and only one sequence of symbols not beginning with a zero and not ending with a zero to the right of a “.” symbol exists which represents the given number for the given radix. Sample exercise: Consider the sequence of symbols 3245. If this is written in the octal (R=8) system, what decimal number does it mean? If it is written in the system of radix 6? In the hexadecimal (R=16) system? In the system of radix 4? Sample exercise: Consider the number 110101 written in the binary (R=2) system. What decimal number does it mean? Sample exercise: Consider the decimal number 34. How would it be written in the binary (R=2) system? 3.3. Converting integers between positional number systems with different radices We will begin by considering the problem of converting integers between two different radix systems. The algorithms can be extended to include fractional numbers. Consider first some examples of such conversions. Example: (binary to decimal, computing in decimal) Let 10110 be the binary representation (R=2) of some number. We can calculate the value, and hence the decimal representation, of this number by applying the definition above of the value represented by a sequence of symbols. We must only take care to perform all our calculations in the decimal system, the system in which we want the answer. Applying the definition n i=0 si * Ri of the value to the specific binary number 10110, we have n i=0 si * Ri = 0*20 + 1*21 + 1*22 + 0*23 + 1*24 = 0*1 + 1*2 + 1*4 + 0*8 + 1*16 = DDS Lecture Notes (draft) - 10 - Robert L. Baber, 1999 July 0 + 2 + 4 + 0 + 16 = 2 + 4 + 16 = 22 This is the value of the original number expressed in decimal notation. The same computation can be organized differently and the individual steps performed in a different order. This alternative is in many cases computationally more efficient because it effectively generates all the various powers of R in one single process while at the same time multiplying them by the corresponding si. It is based on the observation that n i=0 si * Ri = ... ((sn*R + sn–1)*R + sn–2) ... + s0 e.g. when n=4 4 i=0 si * Ri = s4 * R4 + s3 * R3 + s2 * R2 + s1 * R1 + s0 * R0 = (((s4*R + s3)*R + s2)*R + s1)*R + s0 Using this approach, the above conversion of 10110 to decimal would be done as follows. We begin with the first binary symbol on the left (s4, which is 1), multiply it by R (2) to obtain 2 and add the next symbol (s3, which is 0) to obtain 2 as the result of the first iterative step. We then multiply this 2 by R, obtaining 4, and add the next symbol (s2, which is 1) to obtain 5 as the result of this second step in the computation. We then multiply this 5 by R, obtaining 10, and add the next symbol (s1, which is 1) to obtain 11 as the result of this third step in the computation. We then multiply this 11 by R, obtaining 22, and add the next symbol (s 0, which is 0) to obtain 22 as the result of this fourth step in the computation. Having reached and added s0, the last symbol, into the accumulated result, we are finished. The answer is 22. Example: (decimal to binary, computing in binary) Consider the decimal number 22. We can convert it to its binary representation in exactly the same way used above if we perform all calculations in binary, the target system. We must first convert each decimal symbol (digit) into its binary representation. We must also write ten, the radix of the decimal system, in its binary form. The decimal digit 2 is, in binary, written as 10. The binary representation of the number ten is 1010. Proceeding in the same way outlined above, we begin with the left most digit 2, multiply its binary representation (10) by R (ten, or 1010 in binary) to obtain 10100 and add the next symbol (2, or 10 in binary) to obtain 10110. Having reached and added the right most digit into the accumulated result, we are finished. The answer is 10110. DDS Lecture Notes (draft) - 11 - Robert L. Baber, 1999 July Example: (decimal to binary, computing in decimal) Again consider the decimal number 22. We wish to convert it to binary, but this time performing the calculations in decimal (the source system), not binary (the target system). We wish to find a sequence of binary symbols tk tk–1 ... t1 t0 such that k i=0 ti * 2i = 22 where the symbol 2 and the number 22 are understood to be in decimal notation. We can take the 0th term out of the series and rewrite k i=0 ti * 2 i as k i=1 ti * 2i + t0 * 20 = k i=1 ti * 2i + t0 Notice that each term still remaining in the series contains at least one factor of 2. The entire series is, therefore, divisible by 2. The last term, t0, is a binary symbol which must be less than 2 and is, therefore, not divisible by 2. In other words, if we divide the original number by 2, the remainder will be the right most binary symbol in the representation of the number. Dividing 22 by 2 gives 11 with a remainder of 0. Therefore, t0 = 0. We must find the remaining binary symbols such that k i=1 ti * 2 i–1 = 11 where 11 is the quotient resulting above from dividing 22 by 2. This remaining task is basically the same as the original one. Dividing 11 by 2 results in a quotient of 5 and a remainder of 1. Therefore, t1 = 1. The same process is repeated: 5 divided by 2 yields 2 and a remainder of 1, so t2 = 1. The process is repeated again: 2 divided by 2 yields 1 with a remainder of 0, so t3 = 0. The process is repeated again: 1 divided by 2 yields 0 with a remainder of 1, so t4 = 1. The remaining quotient is zero, so repeating the above process will give rise to only leading zeroes, which can be omitted. In other words, we are finished. We have converted the decimal number 22 into its binary representation 10110 (t4 t3 t2 t1 t0). [end of example] From these examples we generalize and conclude that to convert a number from radix system Rs to radix system Rt, one can either DDS Lecture Notes (draft) - 12 - Robert L. Baber, 1999 July perform all calculations in the target radix system Rt and, by applying the definition of the numerical value of a sequence of symbols, repetitively multiply by Rs and add the next symbol or perform all calculations in the source radix system Rs and repetitively divide by Rt, the remainders becoming the symbols in the target system, from right to left. The following table summarizes these processes. Calculations performed in Process for converting an integer target system Rt repetitively multiply by Rs and add the next symbol, final sum is result source system Rs repetitively divide by Rt, remainders generate result Table: Converting integers from one radix system to another Sample exercise: Convert the octal (R=8) number 377 to decimal notation. To binary. To hexadecimal (R=16), using the symbols A, B, C, D, E and F to represent the values ten, eleven, twelve, thirteen, fourteen and fifteen respectively. Perform each computation in both the source and in the target system. Sample exercise: Convert the decimal number 134 to binary, octal and hexadecimal notation. Questions: What similarities do you notice between the binary, octal and hexadecimal representations of a number? Why do these similarities arise? How can this observation be used to facilitate the conversion of numbers between these radix systems? 3.4. Converting fractions between positional number systems with different radices One obvious way to convert a fraction or a number with a fractional part such as sn sn–1 ... s1 s0 . s–1 s–2 ... s–(m–1) s–m is to rewrite this as sn sn–1 ... s1 s0 s–1 s–2 ... s–(m–1) s–m * Rs–m and convert the sequence of symbols, which now represents an integer, as described above. The resulting representation in the target radix system is then multiplied by Rs –m, the multiplication being performed in the target radix system Rt, of course. This method is obviously applicable only to numbers whose original representation is of bounded length. However, because in practice all computation is limited to bounded representations, the above method can generally be applied in practice after truncating the original sequence of symbols at an appropriate place on the right. It is possible, of course, to convert the fractional part of a number from one radix system to another directly, e.g. in ways comparable to those illustrated in section 3.3 above. Example: (binary to decimal, computing in decimal) Let 0.1011 be the binary representation (R=2) of some number. By performing the calculations in decimal arithmetic DDS Lecture Notes (draft) - 13 - Robert L. Baber, 1999 July we can calculate the value, and hence the decimal representation, of this number by applying the definition in section 3.2 above of the value represented by a sequence of symbols. The appropriate form of the definition is that the value represented by the sequence of symbols 0 . s–1 s–2 ... s–(m–1) s–m is –1 i=–m si * Ri which is equivalent to m i=1 s–i * R–i Observing that m i=1 s–i * R–i = ( ... ((s–m/R + s–(m–1))/R + s–(m–2)) ... + s–1)/R suggests an algorithm structurally like one encountered earlier: repetitively divide the already accumulated value by R and add the next symbol (here proceeding from right to left). This is basically the reverse of that earlier procedure for converting an integer to a different radix system; here we divide instead of multiply and proceed from right to left instead of left to right. E.g. when m=4, the above equation becomes 4 i=1 s–i * R–i = (((s–4/R + s–3)/R + s–2)/R + s–1)/R Using this approach, the above conversion of 0.1011 to decimal would be done as follows, where all arithmetic will be done in decimal. We begin with the first binary symbol on the right (s–4, which is 1) and divide it by R (2) to obtain 0.5 as the result of the first iterative step. We then add the next symbol (s–3, which is 1), obtaining 1.5, and divide by R to obtain 0.75 as the result of this second step in the computation. We then add the next symbol (s–2, which is 0), obtaining 0.75, and divide by R to obtain 0.375 as the result of this third step in the computation. We then add the next symbol (s–1, which is 1), obtaining 1.375, and divide by R to obtain 0.6875 as the final result. Example: (decimal to binary, computing in binary) Consider the decimal number 0.6875. We can convert it to its binary representation in exactly the same way used above if we perform all calculations in binary, the target system. We must first convert each decimal symbol (digit) into its binary representation. We must also write ten, the radix of the decimal system, in its binary form. The decimal digits 6, 8, 7 and 5 are, in binary, written as 110, 1000, 111 and 101 respectively. The binary representation of the number ten is 1010. DDS Lecture Notes (draft) - 14 - Robert L. Baber, 1999 July Proceeding in the same way outlined above, we begin with the right most digit 5 and divide its binary representation (101) by R (ten, or 1010 in binary) to obtain 0.1. We then add the next symbol (7, or 111 in binary), obtaining 111.1, and divide this 111.1 by R (1010 in binary) to obtain 0.11 as the result of this step. We then add the next symbol (8, or in binary 1000), obtaining 1000.11, and divide this 1000.11 by R (1010 in binary) to obtain 0.111. We then add the next symbol (6, or in binary 110), obtaining 110.111, and divide this 110.111 by R (1010 in binary) to obtain 0.1011 as the final result. Example: (decimal to binary, computing in decimal) Again consider the decimal number 0.6875. We wish to convert it to binary, but this time performing the calculations in decimal (the source system), not binary (the target system). We wish to find a binary representation 0 . t–1 t–2 ... t–(m–1) t–m such that m i=1 t–i * 2–i = 0.6875 where the symbol 2 and the number on the right hand side are understood to be in decimal notation. The first term in the series above is t–1 * 2–1. If we were to multiply the series by 2, this first term would become simply t–1, a non-negative integer. The rest of the series would still be a fraction. We could then deduce a value for t–1. This observation suggests multiplying both sides of the equation above by 2 to obtain m i=1 t–i * 2–(i–1) = 1.375 Taking the first term out of the series and writing the number on the right hand side as the sum of an integer and a fraction, we obtain m t–1 + i=2 t–i * 2–(i–1) = 1 + 0.375 = m–1 t–1 + j=1 t–(j+1) * 2–j = 1 + 0.375 This suggests requiring that t–1 be 1 and that m–1 j=1 t–(j+1) * 2–j = 0.375 This decomposition of a real number into an integer and a fractional part on each side of the equation is unique. Why? Finding the remaining t–(j+1) is basically the same task as our original one above. Thus we can find the binary representation of a fraction by multiplying the fraction to be converted by 2 (the target radix), taking the integer part of the product as the next bit (the target symbol) and DDS Lecture Notes (draft) - 15 - Robert L. Baber, 1999 July repeating this process on the remaining fraction. When the remaining fraction is zero, the procedure is finished. Question: Must this process terminate? Why or why not? Thus, to convert 0.6875 from decimal to binary, computing in decimal, we begin by multiplying 0.6875 by 2, obtaining 1.375. The first bit t–1 is, therefore, 1. The remaining fraction is 0.375. Next, we multiply 0.375 by 2, obtaining 0.75. The next bit t–2 is, therefore, 0. The remaining fraction is 0.75. Next, we multiply 0.75 by 2, obtaining 1.5. The next bit t–3 is, therefore, 1. The remaining fraction is 0.5. Next, we multiply 0.5 by 2, obtaining 1. The next bit t–4 is, therefore, 1. The remaining fraction is 0 and we are therefore finished. The result is 0.1011. [end of example of converting 0.6875 from decimal to binary, computing in decimal] Notice that the processes for converting integers and fractions from one radix system to another are, in a certain sense, the opposites of each other. Calculations performed in Process for converting an integer Process for converting a fraction target system Rt from left to right repetitively multiply by Rs and add the next symbol, final sum is result from right to left repetitively divide by Rs and add the next symbol, final sum divided by Rs is result source system Rs repetitively divide by Rt, remainders generate result from right to left repetitively multiply by Rt, integer parts of products generate result from left to right Table: Converting integers and fractions from one radix system to another Sample exercise: Convert the binary number 0.1011 to decimal, computing in binary. Sample exercise: What is the binary representation of one tenth? What are some of the implications of the binary representation of one tenth? Additional exercises: Convert various numbers, both integers and fractions, between binary, octal, decimal, hexadecimal and other number systems such as radix 3, radix 5, etc. systems. 3.5. Representing integers in a computer memory Typically integers are represented in a computer memory using the positional notation introduced in section 3.2 and a fixed number of symbols in the radix system chosen (e.g. decimal DDS Lecture Notes (draft) - 16 - Robert L. Baber, 1999 July digits, binary bits, etc.). The most straightforward interpretation of each possible combination of symbols is as an unsigned (non-negative) integer. Example: The following table illustrates such an interpretation for a binary system (R=2) with three bits. Bit pattern Numerical value in binary decimal 111 +111 +7 110 +110 +6 101 +101 +5 100 +100 +4 011 +011 +3 010 +010 +2 001 +001 +1 000 (+)00 (+)0 Table: Representing unsigned (non-negative) integers For many applications such a system is not adequate, as negative values are also required. Various schemes can be used to represent integer values which can be positive or negative. In the specific case of binary representation, three of the most obvious or common schemes are: One bit is interpreted as the sign (+ or ) and the remaining bits are interpreted as the magnitude of the number (signed magnitude method). One bit is interpreted as the sign (+ or ) and the remaining bits are interpreted as follows. If the sign is positive, the remaining bits represent the magnitude of the number directly. If the sign is negative, the one’s complement of the remaining bits represent the magnitude of the number (one’s complement method). One bit is interpreted as the sign (+ or ) and the remaining bits are interpreted as follows. If the sign is positive, the remaining bits represent the magnitude of the number directly. If the sign is negative, the two’s complement of the remaining bits represent the magnitude of the number (two’s complement method). The following table illustrates these three different interpretations for a binary system (R=2) with three bits. DDS Lecture Notes (draft) - 17 - Robert L. Baber, 1999 July Signed magnitude, meaning in One’s complement, meaning in Two’s complement, meaning in binary decimal binary decimal binary decimal 111 11 3 ()00 ()0 01 1 110 10 2 01 1 10 2 101 01 1 10 2 11 3 100 ()00 ()0 11 3 100 4 011 +11 +3 +11 +3 +11 +3 010 +10 +2 +10 +2 +10 +2 001 +01 +1 +01 +1 +01 +1 000 (+)00 (+)0 (+)00 (+)0 (+)00 (+)0 Bit pattern Table: Some different ways of interpreting signed integers Note that both the simple signed method and the one’s complement system have two representations for zero, shown above as (+)0 and ()0. Mathematically and numerically there is, of course, no difference; a sign associated with the value zero has no meaning or consequence. Thus, these systems waste one possibility of representing a number. The representational difference between +0 and 0 can also sometimes lead to confusion (e.g. what result should a test for equality yield?). For these and other reasons, these systems are no longer frequently implemented, although they can sometimes be found in practice. Note also the break between 011 and 100 in the sequence of the numbers represented. The one’s and the two’s complement systems form, in effect, cyclical systems with only one break in sequence, i.e. between 111 and 000. Because of their cyclical nature, arithmetic is logically somewhat simpler in these two systems than in the simple signed method. For the reasons mentioned above, the unsigned and the two’s complement methods are probably the ones most commonly implemented today. But it is important to note that they are not the only possible ones, and for a special application some other scheme may be better. E.g., a particular application may be best served by a representational scheme in which a much larger range of positive integers is required than of negative integers. A particular method of fixed length for representing integers in a computer memory can be specified by stating (1) the radix, (2) the number of symbols in the representation and (3) the function to be used for interpreting each sequence of symbols as an integer (e.g. whether one’s or two’s complement). Still another method of representing negative numbers should be mentioned here because it is often used within binary floating point systems. Structurally it is similar in some respects to the unsigned integer and the two’s complement methods described above. Using this method DDS Lecture Notes (draft) - 18 - Robert L. Baber, 1999 July a signed number sn is represented by the unsigned number un = sn + bias, where bias is a suitable non-negative integer. This method is called the bias or offset method. Example: The following table illustrates several different specific instances of such a bias or offset system for representing signed numbers using three bits. The values of the various bit patterns are shown in decimal notation. Bit pattern Value as integer unsigned signed, bias = 4 signed, bias = 3 signed, bias = 1 signed, bias = 0 111 +7 +3 +4 +6 +7 110 +6 +2 +3 +5 +6 101 +5 +1 +2 +4 +5 100 +4 0 +1 +3 +4 011 +3 1 0 +2 +3 010 +2 2 1 +1 +2 001 +1 3 2 0 +1 000 0 4 3 1 0 Table: The bias or offset method for representing signed integers Note that the unsigned integer representation is just a special case of the bias method with the bias equal to zero. Notice also the cyclical structure of biased systems for representing signed integers — just like that of the one’s and especially of the two’s complement systems. The major difference is the position of the break, here between the top and the bottom of the list, i.e. at the point in the cycle where all 1s change to all 0s. Exercise: Extend the above table for a system using 4 bits. 8 bits. Exercise: Given the biased representations bx and by for two numbers x and y, how would you calculate the biased representation bsum for their sum x+y? the biased representation bdiff for their difference xy? What conditions must the arithmetic system and/or intermediate results satisfy in order to ensure that the results of your calculations are correct? 3.6. Floating point systems Numbers in positional notation form as presented in section 3.2 above are represented most simply in sequences of fixed length, whereby both the integer part and the fractional part of the number are of predefined lengths. I.e., the position of the decimal (or binary, etc.) point is fixed; hence the term “fixed point”. This, of course, imposes limits on the range of numbers which can be represented. For many scientific and technical computations these limits can DDS Lecture Notes (draft) - 19 - Robert L. Baber, 1999 July have severe consequences. For such computations a more flexible scheme for representing numbers is required. A common and useful solution to this problem is floating point representation. This relies on the fact that any number x can be represented by three numbers s, m and e with the following relationship: x = (1)s * m * Re where R is the radix of the number system being used, e is an integer, s is either 0 or 1, and m is a non-negative number. The number m is called the “mantissa” or “significand”, s is the “sign”, and e is called the “exponent”. The exponent is typically a signed integer, i.e. an integer which can take on positive and negative values. The value of e effectively defines the position of the “decimal” point in the representation of x, hence the name “floating point”. Many such representations exist for any number x. If the value of m is restricted to the range 1m<R (except for the representation of 0), m can be represented in fixed point format without imposing any bound on the values of x which can be represented. If the range of values of e is also restricted, a corresponding limit is placed on the values of x which can be represented, but this limit is much less severe than if x itself were to be represented in fixed point format. If R=2, i.e. if the binary system is used for representing the numbers, then the restriction 1m<R implies that the integer part (the high order bit) of m is always 1. Since this is a constant, it need not be stored, saving one bit in memory. Perhaps more importantly, greater precision is achieved without increasing the number of bits which must be stored. Advantage has been taken of this possibility in some implemented systems. The restriction 1 m < R is arbitrary. Any corresponding restriction can be imposed with the same effect. A number of systems have been implemented with the restriction R–1 m < 1 i.e. m is a fraction and its first digit (or bit, etc.) is not zero. A floating point representation which fulfils a specified restriction such as R–1 m < 1 is said to be in normalized form. A floating point system (together with its corresponding arithmetic operations) is a very useful approximation to the real number system with its arithmetic operations, but floating point arithmetic does not share all of the important characteristics of the corresponding mathematical operations. This has important consequences when analyzing the behaviour of floating point arithmetic. Total accuracy is not generally achieved and should never be assumed. Example: Consider for the purposes of illustration a small floating point system with a two bit mantissa, a one bit sign and a three bit exponent, in which the exponent is represented using the bias method. The following table shows the non-negative values (in decimal DDS Lecture Notes (draft) - 20 - Robert L. Baber, 1999 July notation) which can be represented in this floating point system. The corresponding negative values are also representable but are not shown. exponent (biased representation, bias = 4) 000 001 010 011 100 101 110 111 mantissa decimal values 4 3 2 1 0 +1 +2 +3 1.1 1.5 0.09375 0.1875 0.375 0.75 1.5 3 6 12 1.0 1.0 0.0625 0.125 0.25 0.5 1 2 4 8 0.1 0.5 0.03125 0.0625 0.125 0.25 0.5 1 2 4 0.0 0 0 0 0 0 0 0 0 0 Table: Non-negative values representable in a small floating point system (two bit mantissa, three bit exponent) The values representable in this floating point system are: 0, 0.03125, 0.0625, 0.09375, 0.125 0.1875, 0.25 0.375, 0.5 0.75, 1 1.5, 2 3, 4 6, 8 12 and their negatives. Notice how the precision (the inverse of the difference between two neighboring values) decreases with increasing value. This is a consequence of the definition of the value (see formula above) and is, therefore, a characteristic of all floating point systems. The precision is determined by the value of the exponent, and as it increases, the precision decreases. Because the precision varies with value, the attainable accuracy of arithmetic operations varies. Floating point arithmetic is only an approximation to mathematical arithmetic. For example, when two representable values are added, the most accurate result possible is the true mathematical result rounded to the closest representable value. Sometimes floating point arithmetic (e.g. floating point addition ) is exact, for example 0.125 0.25 = 0.375, and 0.125 + 0.25 = 0.375 but sometimes it is not exact, for example 3 1.5 = 4, but 3 + 1.5 = 4.5 Floating point addition is sometimes associative, for example (0.75 1) 1.5 = 2 1.5 = 4 and DDS Lecture Notes (draft) - 21 - Robert L. Baber, 1999 July 0.75 (1 1.5) = 0.75 3 = 4 but sometimes it is not associative, for example (6 0.75) 0.75 = 6 0.75 = 6, but 6 (0.75 0.75) = 6 1.5 = 8 [end of example] In the design of a floating point system a number of detailed decisions must be made, e.g. regarding the length of the mantissa, the range of the exponent, rounding, representing results of arithmetic operations which are too large or too small to be represented, etc. Standards have been agreed upon for some of these aspects of floating point systems, e.g. [IEEE 1985, 1987]. Typically, at least 8 bits are used to represent the exponent and at least 4 bytes (32 bits) for the entire floating point number. The IEEE standard for binary floating point arithmetic, for example, specifies a 4 byte format for single precision and an 8 byte format for double precision floating point numbers. In these formats, the high order bit of the mantissa is not stored; its value is implied by the exponent. Also, this standard calls for using the bias method to represent the exponent. [IEEE 1985] 3.7. Precision and accuracy In section 3.6 above on floating point representation of numbers, it was pointed out that the precision and accuracy with which a number can be represented varies with the value being represented. It is often impossible to represent a desired numerical value precisely. Many such values can be represented only approximately. Complete precision is not achievable when one attempts to represent an arbitrary element of a continuous space (e.g. a real number or a rational number) by an element of a finite space, e.g. using a sequence (of bounded length) of symbols selected from a finite set. In some contexts exact (i.e. completely precise and accurate) representation is possible. This is typically the case when representing integers selected from a finite set. Within the limits set by the bounds of the set of integers in question, arithmetic will also typically be exact, i.e. will yield a mathematically correct and precisely represented result. Note that floating point representation and arithmetic cannot in general be assumed to give exact results. Correspondingly, one may not, in general, assume convenient mathematical properties such as associativity. Only if one restricts the set of values actually used in a computation (e.g. to integers within a certain range) can one sometimes ensure that all results of floating point operations are exact. In such cases, the burden of proof clearly lies on the designer of the program in question. Documentation on specific programming language systems typically contains information on the precision and accuracy of the various implemented methods for representing numbers. Integer types will normally be exact within the range of values represented. Floating point types will generally not give exact results. The documentation of the programming language Scheme [Kelsey] distinguishes explicitly between exact and inexact numbers, independently of the particular representation used. See section 6.2.2. Exactness in [Kelsey]. DDS Lecture Notes (draft) - 22 - Robert L. Baber, 1999 July 4. Representing non-numerical data values In chapter 3 above numbers were represented by sequences of symbols, each symbol representing an integer between 0 and R1 inclusive, where R is the radix of the number system in question, or separating the integer and fractional parts of the number. More generally, any data value can be represented by a sequence of symbols selected from some finite set. The sequence may be of any length, including 0 (the empty sequence). The need to represent text (sequences of letters, digits and other special symbols, e.g. punctuation marks) or other non-numerical data in a computer memory arises often and in many types of applications. Because computer hardware memories are typically organized as sequences of cells, each cell storing a byte consisting of eight bits, it has become usual to represent each character (symbol) in a body of text by a pattern of eight bits, i.e. in one byte (cell) in such a memory. Because 28 = 256, any one of 256 different characters can be represented in one byte of computer memory. 4.1. Standards for representing individual characters Earlier, 5 and 6 bit representations of such characters were standard, e.g. in teletype networks and in computer systems. With 6 bits, 26 = 64 characters can be represented, which is not enough for both lower and upper case letters in the English alphabet, the ten digits and a number of special characters (e.g. punctuation marks). The ASCII 7 bit encoding standard was, therefore, defined. The following is an extract of the ASCII standard code table: DDS Lecture Notes (draft) - 23 - Robert L. Baber, 1999 July Binary representation Hexadecimal representation Decimal representation Character(s) 000 0000 - 001 1111 00 - 1F 0 - 31 control codes such as carriage return, line feed, other teletype functions, etc. 010 0000 20 32 the blank character (space) 010 0001 - 010 1111 21 - 2F 33 - 47 special characters (e.g. *, +, -) 011 0000 30 48 digit 0 011 0001 - 011 1000 31 - 38 49 - 56 digits 1 - 8 011 1001 39 57 digit 9 011 1010 - 100 0000 3A - 40 58 - 64 special characters 100 0001 41 65 capital letter A 100 0010 - 101 1001 42 - 59 66 - 89 capital letters B - Y 101 1010 5A 90 capital letter Z 101 1011 - 110 0000 5B - 60 91- 96 special characters 110 0001 61 97 lower case letter a 110 0010 - 111 1001 62 - 79 98 - 121 lower case letters b - y 111 1010 7A 122 lower case letter z 111 1011 - 111 1111 7B - 7F 123 - 127 special characters Table: Extract of the ASCII standard encoding table Several eight bit extensions have been defined to the ASCII encoding table. They define codes for letters appearing in European languages other than English (e.g. á, ä, å, æ, ç, é, ð, ñ, ö, ù, ü, þ, etc.). 4.2. Representing a value as a sequence of characters or symbols As stated in section 4 above, a data value can be represented by a sequence of characters or symbols. Assuming that the starting position of the sequence in computer memory is indicated in some appropriate way, a way of indicating the final position of the data value in question is still needed. Two common methods are to use a special symbol (a terminating character) to delimit the end of the sequence and to indicate the length of the sequence in the form of a number, either stored at the beginning of the sequence or somewhere else, e.g. together with the indication of the starting position of the sequence. For example, the blank character can be used to mark the end of data value. In effect, this was the method used in section 3 above. Any other character not otherwise used in the sequence of symbols representing the data value in question can be used as an end marker. DDS Lecture Notes (draft) - 24 - Robert L. Baber, 1999 July Another common method uses a length indicator placed at the beginning of the sequence of symbols representing the data value. In many systems, a single 8 bit (one byte) integer is used. This limits the length of possible sequences to 255 (if 0 is used to represent an empty sequence). If this poses an unreasonable limitation, two or more bytes can be used for the length. Schemes for representing such a length without imposing any upper limit exist. One possibility involves prefixing the sequence by its length in some radix system, prefixing that length by the length of the length, etc., until a predefined length (e.g. 1) is reached [Baber]. These and still other ways for representing such sequences in a computer memory utilize data structures of more general applicability, bringing us to the subject of representing data structures in a computer memory. Exercise: Determine the binary representation for the letter “c”. the sequence of letters “cde”. the sequence “f2xyz”. In each case use the 7-bit ASCII representation defined in the above table. DDS Lecture Notes (draft) - 25 - Robert L. Baber, 1999 July 5. Arrays, lists, vectors, sequences As mentioned in chapter 2 above, an array, a list or a vector is a sequence of values or data items, grouped together logically and usually also physically in a computer memory. The main structural aspect of an array, a list or a vector is its linear sequence, often expressed by physical adjacency. This structure is the logical basis of most computer hardware memories. 5.1. Computer memory Physically a computer memory typically consists of a linear sequence of cells. Each cell is identified by a numerical address and can store a symbol selected from a fixed set of symbols. In most modern computing systems, a cell consists of 8 bits (also called a byte), but sometimes a multiple of 8 bits. The term word is sometimes used as a synonym for cell, especially in older computer systems and in systems with a cell size larger than 8 bits. The addresses associated with the cells of a computer memory are typically integers beginning with 0. The following diagram illustrates the structure of a typical computer memory. Contents of the cells are shown, interpreted as characters in a defined set of symbols. Address Contents 0 d 1 g 2 K 3 1 4 2 5 3 6 F 7 & 8 % 9 @ 10 ? 11 / 12 + ... ... The electronic hardware comprising the memory is constructed in such a way that the address is the access key to the contents of the corresponding cell. That is, if the address 10 is sent to the memory circuitry, the character ? (more precisely, the bit representation of the character DDS Lecture Notes (draft) - 26 - Robert L. Baber, 1999 July ?) will be returned. If we consider the entire memory to be an array which we call cmem, then the values of various elements of this array are: cmem(0) = “d” (the symbol or character d) cmem(3) = “1” (the symbol or character 1) cmem(10) = “?” (the symbol or character ?) cmem(12) = “+” (the symbol or character +) etc. The cells of the computer memory are structured in a linear sequence by the linear sequence inherent in their addresses, which are consecutive non-negative integers. This may or may not correspond to the physical locations in which the values are actually stored. Similarly, the individual elements of a one-dimensional array are structured in a linear sequence by the linear sequence inherent in their indices, which are typically integers (and usually consecutive integers). For example, our model above of a computer memory as an array called cmem is a one-dimensional array with indices selected from the set of consecutive integers beginning with 0. 5.2. One-dimensional arrays and their representation in a computer memory A one dimensional array is typically stored (represented, mapped) in a computer memory in a sequence corresponding to the sequence defined by the values of the indices of the various elements of the array. Example: Consider an array called Age with 3 elements. If the computer memory has free available space beginning at memory address 75 and if each element in the array Age is an unsigned integer stored in one cell (byte) of the computer memory, the array Age might be stored in the computer memory as follows: Addresses in computer memory Contents Element of array ... ... 75 22 Age(1) 76 25 Age(2) 77 75 Age(3) ... ... Array Age The address in computer memory of any value in the array Age can obviously be calculated in the following way: address in computer memory = index in array Age + 74 Expressed differently, the value of the array element Age(i) is stored in cmem(i+74), for i = 1, 2, 3. Consider another array called Name, also with 3 elements. If each element in the array Name is a sequence of up to ten characters, each of which is stored in one cell (byte) of the computer memory, the array Name might be stored in the computer memory as follows: DDS Lecture Notes (draft) - 27 - Robert L. Baber, 1999 July Addresses in computer memory Contents Element of array ... ... 75 22 Age(1) 76 25 Age(2) 77 75 Age(3) 78-87 George Name(1) 88-97 Themba Name(2) 98-107 Gogo Name(3) ... ... Array Age Name The address in computer memory of the first cell of any value in the array Name can be calculated in the following way: starting address in computer memory = (index in array Name - 1) * 10 + 78 Expressed differently, the value of the array element Name(i) is stored in cmem((i1)*10+78), cmem((i1)*10+79), ... cmem((i1)*10+87), for i = 1, 2, 3. If a third array called City is added to this data structure, it might be stored in the computer memory as follows: DDS Lecture Notes (draft) - 28 - Robert L. Baber, 1999 July Addresses in computer memory Contents Element of array ... ... 75 22 Age(1) 76 25 Age(2) 77 75 Age(3) 78-87 George Name(1) 88-97 Themba Name(2) 98-107 Gogo Name(3) 108-122 Johannesburg City(1) 123-137 Pretoria City(2) 138-152 Nelspruit City(3) ... ... Array Age Name City The address in computer memory of the first cell of any value in the array City can be calculated in the same way, but with different constants in the formula: starting address in computer memory = (index in array City - 1) * 15 + 108 The value of the array element City(i) is stored in cmem((i1)*15+108), cmem((i1)*15+109), ... cmem((i1)*15+122), for i = 1, 2, 3. A one dimensional array reflects the linear structure of an array and, therefore, lends itself well to representing such a data structure and manipulating it simply and straightforwardly. The following exercises deal with searching, inserting a new item in an already sorted sequence and sorting. In some programming languages (such as C) formulae for addressing elements in arrays must be explicitly written into expressions in program statements referring to array elements. Such formulae will be of essentially the same form as the ones above. In many other programming languages (such as Pascal and Basic) the mapping from the array index to the memory address is generated automatically by the compiler or interpreter. I.e., in the program the expression “City(i)” would appear. It would automatically be converted to “cmem((i1)*15+108)” as in the example above. In fact, the constant 108, and possibly also the constant 15, would be unknown (and unavailable) to the programmer. Logically, a list in a language like Scheme is structurally equivalent to an array as described above. The Scheme structure “vector” is even closer in intent to the concept of an array. The manner in which a list or vector is actually stored in a computer memory is often not speciDDS Lecture Notes (draft) - 29 - Robert L. Baber, 1999 July fied in the documentation of the language and is purposely hidden from the programmer. In languages based on Lisp concepts a list is typically actually stored in a tree structure. (Trees will be covered later in this course.) Even when lists are actually stored in a tree structure, program statements or procedures are usually provided for accessing elements of a list or vector in essentially the same ways as accessing array elements in the above example. For example, the expression (list-ref City i) or (vector-ref City i) in Scheme corresponds to the reference City(i) above to an array element. Exercise (searching 1): Data is stored in an array x with index values ranging from 0 to n inclusive. The values of the array elements x(0), ... x(n) are not necessarily sorted in any particular order. Design an algorithm (procedure) which will determine whether or not the value d exists in this array and if it does, the index of its first occurrence. Your algorithm will, presumably, involve a loop. What is your loop invariant, i.e. what condition is always true before and after each execution of the body of the loop? Use a diagram to describe your loop invariant. Does your algorithm require that the array is not empty, i.e. that n 0? Exercise (searching 2): Data is stored in an array x as stated in exercise 1 above. The values of the array elements x(0), ... x(n) are in ascending order with duplicates allowed. Design an algorithm (procedure) as specified above, but which takes advantage of the fact that the values in the array x are sorted. If the value d does not exist in the array, your algorithm should determine where in the array it would have been. Hint: decide upon a suitable loop invariant before designing your algorithm and use it to guide your thinking leading to your design. Exercise (inserting a value into a sorted array): Data is stored in an array x with index values ranging from 0 to n inclusive. The values of the array elements x(0), ... x(n) are in ascending order with duplicates allowed. Design an algorithm which will insert the value of array element x(n+1) into the correct place in the array so that the entire array from x(0) to x(n+1) inclusive is sorted into ascending order. What is your loop invariant? How many times will the body of the loop be executed in the best possible case? in the worst case? on average? Under what condition will the best case occur? the worst case? What did you assume in determining the average case? Exercise (sorting): Data is stored in an array x with index values ranging from 0 to n inclusive. The values of the array elements x(0), ... x(n) are not necessarily sorted. Design an algorithm (procedure) which will sort these values into ascending order. What is your loop invariant? 5.3. Records To access the values of Age(3), Name(3) and City(3) in the example in section 5.2 above, several multiplications and additions are required in order to calculate the relevant addresses in computer memory. Furthermore, if this data is stored on a disk or similar electromechanical device, accessing these three data values might involve significant mechanical movements with attendant time delays. Especially if each of these arrays is large, e.g. with thousands of elements each, the delay can become excessive. Storing these three arrays in a different way can reduce this potential problem considerably. In section 5.2 above, each array was stored in contiguous cells in computer memory, one array after the other. This was an arbitrary choice. We might have chosen to organize the storage of these arrays differently, for example in the following way. DDS Lecture Notes (draft) - 30 - Robert L. Baber, 1999 July Addresses in computer memory Contents Element of array ... ... 75 22 Age(1) 76-85 George Name(1) 86-100 Johannesburg City(1) 101 25 Age(2) 102-111 Themba Name(2) 112-126 Pretoria City(2) 127 75 Age(3) 128-137 Gogo Name(3) 138-152 Nelspruit City(3) ... ... Now, addresses can be calculated as follows. Array element begins in computer memory address ends in computer memory address Age(i) (i1) * 26 + 75 (i1) * 26 + 75 Name(i) (i1) * 26 + 76 (i1) * 26 + 85 City(i) (i1) * 26 + 86 (i1) * 26 + 100 Notice that the term (i1)*26 is common to all address calculations. Thus, only one multiplication is requred to calculate the addresses of all three array elements for a given index value. Since multiplication is a relatively time consuming operation, three associated array elements (e.g. Age(3), Name(3) and City(3)) can usually be accessed more quickly if the arrays are stored interwoven with each other as shown above. If the data is stored on a disk or similar electromechanical device, the values of these three array elements, being stored adjacent to one another, can usually be accessed much more quickly than if they had been stored in widely separate locations, e.g. if the several arrays had been stored one after the other as in the earlier example above. Thus, if corresponding elements of different arrays tend to be accessed together, e.g. because they are logically associated with one another in the context of the application, it is often advantageous to store the related array elements together and allow each array to become dispersed. When related array elements are grouped in this way, each group is often called a record. Sometimes the group is given a name; in the above example, the group or record might be named Person. DDS Lecture Notes (draft) - 31 - Robert L. Baber, 1999 July A component of a record is often called a field. I.e., the above record called Person consists of the fields Age, Name and City. When arrays are involved, a reference to either a record as a whole or to one of the component fields must be qualified by the index, e.g. the suffix “(i)” in Age(i). In programming languages implementing record structures explicitly, the individual components are not often explicitly called or thought of as arrays or array elements, but logically they are arrays or array elements nevertheless. In many programming language systems, all of the address calculations referred to above are performed automatically within the relevant compiler, run time routines belonging to the programming language system or object code generated by the system. As other data structures, arrays may be viewed as abstract data types. The term “abstract” expresses the idea that details of the implementation (e.g. organization of storage and the representation of the data in a computer memory) are abstracted out of consideration. One concentrates one’s attention instead on the essence of the data viewed at a higher, more aggregate level. In the case of the field or array Age, the only aspects of interest would be the fact that the values of the array elements are selected from the set of integers {0, 1, ... 255} and the operations defined on this set, if any. Also, one could view each element of the array Age as an ADT or one could view the entire array as a single ADT, in which case the set associated with the array Age would be, in the above example, {0, 1, ... 255}3. All of these different views are valid. For different analytical purposes different views may be appropriate. 5.4. Files The term file is used in computing and business data processing in several different ways. In its most generic sense, it means any collection of interrelated data. Probably its most common meaning is any data structure which is physically stored in an auxiliary or external storage device such as disk, diskette, tape, etc. Such storage devices are typically much slower than the central memory (e.g. RAM) of a computing system. Data in such auxiliary or external storage cannot normally be processed directly, but can only be accessed (read) and rewritten. Because of the speed difference and the fact that data in auxiliary or external storage cannot be processed directly, it is often useful to distinguish between data in central memory vs. data in auxiliary or external storage; hence the introduction of the term file. One of the more specific meanings of the term file is a collection of records (see section 5.3 above), especially when the fields of each record are stored in adjoining areas of memory. When records are stored in auxiliary or external storage devices, their fields are normally stored in adjoining areas of the memory device. Thus, the last two meanings mentioned above are related to one another. 5.5. Multidimensional arrays A one dimensional array, as presented above, is an array each element of which is identified by a single index value. More generally, a multidimensional array is an array each element of which is identified by more than one index values. A multidimensional array can be viewed as an array of an array ... etc. For example, a two dimensional array is, in effect, a (one dimensional) array of (one dimensional) arrays. A three DDS Lecture Notes (draft) - 32 - Robert L. Baber, 1999 July dimensional array is a (one dimensional) array of two dimensional arrays, i.e. a (one dimensional) array of (one dimensional) arrays of (one dimensional) arrays. Example: Data on the number of votes cast for each of 5 candidates in each of 20 regions may be structured in the form of a two dimensional array calles Votes. The individual elements of this array would be, in many systems, referenced in the form Votes(c, r), where c is the index (identification number) of the candidate in question and r, the index (identification number) of the region in question. If the five candidates are identified by the numbers 1, 2, 3, 4 and 5 and the 20 regions are identified by the numbers 11, 12, ... 30 (for whatever reason), then the value of the index c would range from 1 to 5 inclusive and the value of the index r, from 11 to 30 inclusive. This two dimensional array Votes could be mapped onto a one dimensional array Oda with index values ranging from, say, 1 to 100 in different ways. One possibility would be: Votes(c, r) Oda(i) Votes(1, 11) Oda(1) Votes(1, 12) Oda(2) ... ... Votes(1, 30) Oda(20) Votes(2, 11) Oda(21) Votes(2, 12) Oda(22) ... ... Votes(2, 30) Oda(40) ... ... Votes(5, 11) Oda(81) Votes(5, 12) Oda(82) ... ... Votes(5, 30) Oda(100) The index i in the array Oda can be calculated straightforwardly from the indices c and r in the array Votes by applying the following formula: i = 1 + (c1)*20 + r11 Questions: How can one derive this formula? How can this formula be generalized for any given starting and ending values for c and r (e.g. cs, ce, rs, re) and starting value for i (e.g. is)? What then is the value of the ending value for i (e.g. ie) in terms of the other values? DDS Lecture Notes (draft) - 33 - Robert L. Baber, 1999 July Another possibility for mapping the two dimensional array Votes to the one dimensional array Oda is: Votes(c, r) Oda(j) Votes(1, 11) Oda(1) Votes(2, 11) Oda(2) ... ... Votes(5, 11) Oda(5) Votes(1, 12) Oda(6) Votes(2, 12) Oda(7) ... ... Votes(5, 12) Oda(10) ... ... Votes(1, 30) Oda(96) Votes(2, 30) Oda(97) ... ... Votes(5, 30) Oda(100) In this case the index j in the array Oda can be calculated from the indices c and r in the array Votes by the formula: j = 1 + (c1) + (r11)*5 A two dimensional array is a natural structure for storing a matrix. The value in column c and row r of a matrix M is typically stored in the array element M(c, r), although the choice between this and M(r, c) is arbitrary. Whichever convention one chooses, one must be careful to be consistent. Exercise: Design an algorithm for multiplying two matrices A and B to form the product C = A*B. Matrix A has ac columns and ar rows. The range of every index begins with 0. Similarly, matrix B has bc columns and br rows. What condition must be met if the product A*B is to be defined? The above approach to defining two dimensional arrays and storing them in a computer memory (a linear sequence of cells) can be extended in the obvious way to arrays of any higher dimension. DDS Lecture Notes (draft) - 34 - Robert L. Baber, 1999 July 6. Association by links and pointers In the preceding sections an association between two or more data values has been realized either implicitly (i.e. in the minds of the designers and users only) or explicitly by physical proximity of the data values in the computer memory or by the value of an index to arrays or lists. These mechanisms, while convenient and simple, do not always suffice, e.g. when the need arises to express associations between one data value and several others not in corresponding positions of other arrays. A common and more flexible method for expressing such associations is with the help of “links” and “pointers”. Different types of programming languages handle the link or pointer mechanism differently. (1) Some make no particular provision at all for them, leaving pointers and related operations entirely up to the programmer to implement when needed. (2) Other programming languages provide data types especially for pointers and procedures for performing related operations. (3) Still others use such mechanisms extensively but entirely automatically, hiding pointers and related operations from the programmer. Among the examples of the first type are most dialects of Basic. Pascal is a typical example of the second type. Scheme falls into the third type. Because neither the first nor the third type of programming languages provides explicit pointer mechanisms and related operations, they may appear to be similar or the same to the casual programmer. However, the programmer concerned with efficiency of execution must be fully aware of the difference and of various characteristics of these mechanisms and operations, especially time complexity. This chapter will examine links, pointers and related operations primarily from the standpoint of the first and second types of programming languages outlined above. The reader should keep in mind, however, that implementations of programming languages of the third type (e.g. Scheme) utilize the mechanisms and operations examined in this chapter extensively, even if typically invisibly. Understanding these mechanisms is essential if the Scheme programmer, for example, is to understand why some procedures are easy to express in the language but may be inefficient in execution. This link or pointer method for expressing associations between data values will first be explained and illustrated using a linear linked list as an example. In later sections, it will be extended to branching structures, such as trees and graphs. 6.1. Linked lists In section 5.2 above, a linear sequence of data values was represented in a one dimensional array in which the sequence of the data values was determined by the natural order of the integers used as indices to the array. Using this method, the sequence of 6 integers {11, 13, 14, 14, 37, 42} might be represented in the array x as follows: DDS Lecture Notes (draft) - 35 - Robert L. Baber, 1999 July index i value of x(i) 1 11 2 13 3 14 4 14 5 37 6 42 Another way of representing this sequence of integers eliminates the structural dependency upon the indices to the array x and expresses the sequence in a second array, called next below. In addition, a single variable is needed to indicate the first value in the sequence, here the variable start, whose value is 4. index i x(i) next(i) 1 13 5 2 ... ... 3 ... ... 4 11 1 5 14 6 6 14 10 7 ... ... 8 42 -1 9 ... ... 10 37 8 11 ... ... Note several characteristics of such a linked list: The value of a particular variable (here start) is the index of the first element in the list (sequence). The list is stored in two (or more) arrays. Each element of the list contains one (or more) data values and the index of the following element in the list. One particular value, which is not a valid index to the arrays in question, indicates the end of the list; here the value -1 is used as this end marker. Not all elements of the arrays need be occupied (used). Adjacent elements in the list (sequence) may, but need not, be stored in adjacent elements of the arrays. DDS Lecture Notes (draft) - 36 - Robert L. Baber, 1999 July Variations and extensions of the above characteristics are sometimes desirable. For example, in some applications it may be desirable to maintain a pointer to the last element in a list, along with the pointer to the first element. Question: How might an empty list be represented? Clearly, representing a sequence as a list instead of in a single array requires more memory. Some operations on the list are more difficult or time consuming to perform, but others are easier and less time consuming. The list representation is more flexible in some (but not all) respects. Tradeoffs are involved in making a decision how to store a sequence of data values; for some purposes the linked list has definite advantages and for other purposes the simple array is better. Such a linked list may have more than one data value and may be simultaneously ordered in different ways. For example, consider the following set of arrays and the two variables startforward and startbackward whose values are 4 and 8 respectively. index i x(i) y(i) next(i) previous(i) 1 13 2 5 4 2 ... ... 3 ... ... 4 11 1 1 -1 5 14 8 6 1 6 14 3 10 5 7 ... 8 42 9 ... 10 37 11 ... ... 6 -1 10 ... 6 8 6 ... This list is structured both in ascending and in descending order simultaneously. The list beginning at index 4 is in ascending order and the list beginning at index 8 is in descending order. Both lists consist of the same data elements. Exercise: Add another set of links to the above data structure so that the new links sequence the data elements in ascending order on the values of y. Where does the new list begin (with what index value)? DDS Lecture Notes (draft) - 37 - Robert L. Baber, 1999 July 6.2. Pointers and links In such lists, the index values (stored in the arrays next and previous above) are often called pointers, as they “point to” the next element in the list. Note that a pointer is nothing other than an index to an array (or to several arrays). The only substantial difference between an index to an array and a pointer in such a list is that indices to an array are usually selected from a set of consecutive integers, while pointer values are typically selected from a set of integers but not necessarily consecutive integers. This difference is usually of no great significance and is often overemphasized. Many specific programming languages introduce a syntactical difference between an index to an array and a pointer to an element in a list. For example, in Pascal, the pointer notation i^.x is used as x(i) was used above. Semantically these two references are synonyms for each other: both are interpreted to mean the value of x identified by the value of the index (pointer) i. In these two cases, the identifier x would be declared in syntactically different ways, but semantically the effect is the same. Other programming languages follow still other conventions for writing references to variables indexed by pointers. The connection or association established by a pointer is called a link. The sequence of links along one path is often called a chain. There are two chains in the above data structure, one beginning at startforward and the other beginning at startbackward. The exercise above adds a third chain to the data structure. An element in a list is sometimes also called a node. Thus, a link connects two nodes. 6.3. Diagrams of linked lists The arrays in which a list is stored and, in particular, whether some parts of the arrays are occupied or not are usually not the essential characteristics of a list. The tabular representation used in section 6.1 above is, therefore, not a particularly appropriate form for describing such a list. Instead, diagrams which highlight the links are typically employed. The following is such a diagram for the above list: 11 1 13 2 14 8 14 3 37 6 42 6 end end startforward startbackward Note that the actual values of the pointers are not of interest and, hence, are not shown on this diagram. Both forward and backward pointer chains are often shown on one diagram. Adding other chains, such as the third one added in the exercise above, typically gives the diagram a confused appearance, with chains crossing over each other. Different chains structuring the data in different ways are usually best shown on separate diagrams. DDS Lecture Notes (draft) - 38 - Robert L. Baber, 1999 July 6.4. Searching a linked list Frequently one wants to search a linear sequence of data items, be it in the form of a one dimensional array or in the form of a linked list. This is usually accomplished by scanning over unwanted elements of the list until the desired element is found or it becomes clear that the desired element is not in the list. The latter condition applies if the end of the list is reached or, in the case of a list known to be sorted, an element later in the ordering sequence than the desired element is encountered. We will design two searching algorithms. When designing the first one, we will assume that we do not know whether the list is initially sorted or not. When designing the second one, we will assume that the list is initially sorted. An important question will be whether utilizing this knowledge improves the search in any way. Example (list not necessarily initially sorted): Consider the list in the preceding examples. We seek the first element in the list whose value of x is equal to the value of the variable key. When the search terminates, we require that the result variable f points to the desired element in the list, if it exists, otherwise to the “end” of the list. In order to ensure that the element found is the first one satisfying the search criterion, we also require that all elements in the list before the one to which f points not satisfy the search criterion. Somewhat more formally, we require that when the search terminates the following condition will be fulfilled: (f 1 and x(f) = key or f = 1) [desired element found] [end of list encountered; desired element therefore not in list] and every element in the list before the one to which f points has a value of x not equal to the value of key This formulation of the final condition does not presuppose that the list is initially sorted in any order. Duplicate values are permitted. Question: Why is it not necessary that the list be initially sorted? The above condition represents the goal of our search algorithm. It, together with the necessity of starting the search at the beginning of the list, leads to the following algorithm: f := startforward while f 1 and x(f) key do f := next(f) endwhile Question: Why does this program segment satisfy the required final condition above? Question: What condition is true before and after every execution of the body of the loop, i.e. what is the loop invariant? Example (list initially sorted): Consider the same list as in the above examples, but now assume that the list is in ascending order on the values of x. We seek the first element in the list whose value of x is equal to the value of the variable key. When the search terminates, we require that the result variable f points to the desired element in the list, if it exists, otherwise to the element beyond the place where the desired element would be, or to the “end” of the list if no such place in the list exists. In order to ensure that the element found is the first one satisfying the search criterion, we also require that all elements in the list before the one to DDS Lecture Notes (draft) - 39 - Robert L. Baber, 1999 July which f points not satisfy the search criterion. Somewhat more formally, we require that when the search terminates the following condition will be fulfilled: (f 1 and x(f) = key or f 1 and x(f) > key or f = 1) [desired element found] [higher element found; desired element therefore not in list] [end of list encountered; desired element therefore not in list] and every element in the list before the one to which f points has a value of x less than the value of key The above final condition can obviously be simplified to (f 1 and x(f) key or f = 1) [location found, whether or not desired element present] [end of list encountered; desired element therefore not in list] and every element in the list before the one to which f points has a value of x less than the value of key This formulation of the final condition clearly presupposes that the list is initially sorted in ascending sequence of x, whereby duplicate values are permitted. Question: Why is it necessary that the list be initially sorted? The above condition represents the goal of our search algorithm. It, together with the necessity of starting the search at the beginning of the list, leads to the following algorithm: f := startforward while f 1 and x(f) < key do f := next(f) endwhile Question: Why does this program segment satisfy the required final condition above? Question: Is the second algorithm (the one based on the assumption that the list is initially sorted) better than the first? If so, in what way? The two algorithms above deliver as their result the value of the variable f, which points to the first element in the list which satisfies the termination condition (x(f) key or x(f) = key, depending on the algorithm). In some applications a pointer to the preceding element in the list is needed as a result of the search algorithm. This requirement can be added to the above specifications in a fairly simple way. Firstly, we introduce an additional result variable prev and require that it point to the predecessor of the element to which f points. Viewed differently but equivalently, prev should point to the element in the list which in turn points to the same element to which f points. DDS Lecture Notes (draft) - 40 - Robert L. Baber, 1999 July ... ... ... ... prev f An obvious strategy for achieving this new goal is to continually keep track of the predecessor of the node to which f points by adding appropriate statements to the algorithms above. I.e. prev and f should continually (that is, at most points in the algorithm) satisfy the condition f = next(prev) This expression, of course, makes sense only if prev is not the end value, i.e. if prev 1. This raises the question what the relationship between f and prev should be otherwise. Thus, our condition becomes prev 1 and f = next(prev) or prev = 1 and ? Note that the initialization of the two algorithms establishes the truth of the condition f = startforward. But if f = startforward and f = next(prev), then startforward = next(prev), which can be satisfied only if the first element in the list has a predecessor, which is a contradiction. Thus, the subcondition f = startforward (or something which implies it) is needed somewhere in the condition we are constructing. The place marked ? above is a possibility which immediately presents itself and which appears to (and, in fact, does) suffice. We add, therefore, the condition prev 1 and f = next(prev) or prev = 1 and f = startforward to the postconditions and to the loop invariants of the two algorithms above. Wherever f is assigned a value in those algorithms, appropriate statements assigning a suitable value to prev must be added. Thus, the algorithms become: prev := 1; f := startforward while f 1 and x(f) key do prev := f; f := next(f) endwhile [algorithm not requiring the list to be sorted] [note: next(f) could be replaced by next(prev)] and prev := 1; f := startforward while f 1 and x(f) < key do prev := f; f := next(f) endwhile DDS Lecture Notes (draft) [algorithm requiring the list to be sorted] [note: next(f) could be replaced by next(prev)] - 41 - Robert L. Baber, 1999 July where the new statements are highlighted in italics. Note that after every execution of the body of the loop, prev will not be equal to 1 and f will be equal to next(prev). Therefore, if the algorithm terminates with prev = 1, then the body will not have been executed at all; either the list is empty or the first element in the list satisfied the termination condition. Note how a thorough analysis of the problem leads relatively simply and quickly to the necessary program statements. This observation applies often in program design. Such types of analyses can be systematized more extensively than has been done here, but that is a subject for another course. Question: The new body of the loop was written as prev := f; f := next(f) and not f := next(f); prev := f Why? (Hint: consider the new condition to be satisfied.) 6.5. Inserting an element into a linked list When processing data represented in a linked list, it is often necessary to insert a new data item into an existing list. At first glance, there appear to be four special cases: inserting a data item into (A) an empty list, (B) at the beginning of an existing list, (C) between two elements of an existing list and (D) at the end of an existing list. In fact, only two cases need be handled, because the structure of the sublist following the insertion point is not relevant for the insertion operation. Cases A and B reduce to one situation (called case 1 below) and cases C and D reduce to another situation (called case 2 below). A question for the student to ponder is, “How can the remaining two cases be reduced still further to only one?” 6.5.1. Case 1: inserting a new item at the beginning of a list In this case a new data item is to be inserted at the beginning of a list (which may or may not be empty). Let the variable start point to the beginning of a list and the variable pnew point to an item to be inserted at the beginning of the list. The following diagram illustrates the conditions prevailing before the insertion: pnew ... ... ... ... start DDS Lecture Notes (draft) - 42 - Robert L. Baber, 1999 July and the following diagram illustrates the conditions required after the insertion: pnew ... ... ... ... start By comparing the two diagrams above defining the initial and final situations, we can deduce the changes which must be made: pnew ... ... insert ... ... delete insert start From these diagrams, especially the last one above, it follows that this program segment will accomplish the insertion: next(pnew) := start start := pnew Questions: Why was this sequence of statements selected and not the reverse? The above program segment contains two statements, each of which inserts a link. How does it delete the link from start to the (originally) first element in the list? Will this program segment also correctly insert the element to which new points if the list is initially empty? Why or why not? Draw the appropriate diagrams to convince yourself and others that your answer is correct. DDS Lecture Notes (draft) - 43 - Robert L. Baber, 1999 July 6.5.2. Case 2: inserting a new item after an existing element of a list In this case a new data item is to be inserted after an existing element of a list, i.e. either between two elements of an existing list or at the end of an existing non-empty list. Let the variable p point to the list element immediately after which the new element is to be inserted. Let the variable pnew point to the item to be inserted. The following diagram illustrates the conditions prevailing before the insertion: pnew ... ... ... ... ... ... p DDS Lecture Notes (draft) - 44 - Robert L. Baber, 1999 July and the following diagram illustrates the conditions required after the insertion: pnew ... ... ... ... ... ... p By comparing the two diagrams above defining the initial and final situations, we can deduce the changes which must be made: pnew ... ... insert insert ... ... ... ... delete p From these diagrams, especially the last one above, it follows that this program segment will accomplish the insertion: DDS Lecture Notes (draft) - 45 - Robert L. Baber, 1999 July next(pnew) := next(p) next(p) := pnew Questions: Why was this sequence of statements selected and not the reverse? The above program segment contains two statements, each of which inserts a link. How does it delete the link from the element to which p points to the subsequent element in the list? Will this program segment also correctly insert the element to which pnew points if the element to which p points was initially the last element in the list? Why or why not? Draw the appropriate diagrams to convince yourself and others that your answer is correct. Question: Can the remaining two cases be reduced still further to only one? If so, how? If not, why not? 6.6. Deleting an element from a linked list When processing data represented in a linked list, it is also often necessary to delete a data item from an existing list. Again here, two different situations arise which must be handled by different program segments. They are (1) deleting the first element from the list and (2) deleting an element other than the first from the list. Question: Why do only these two cases arise? Why does the deletion of the last element of a list not require special consideration? 6.6.1. Case 1: deleting the first element from a list In this case, the first element of a (non-empty) list is to be deleted, i.e. removed, from the list. Let the variable start point to the beginning of the list in question. After deletion, the variable pdeleted must point to the item which was deleted from the list. The following diagram illustrates the conditions prevailing before the deletion: ... ... ... ... start DDS Lecture Notes (draft) - 46 - Robert L. Baber, 1999 July and the following diagram illustrates the conditions required after the deletion: pdeleted ... ... ... ... start By comparing the two diagrams above defining the initial and final situations, we can deduce the changes which must be made: pdeleted insert ... ... (delete) ... ... insert delete start From these diagrams, especially the last one above, it follows that this program segment will delete the first element from the list and record in pdeleted the location (index) of the element that was deleted from the list: pdeleted := start start := next(start) Questions: Why is the one link labelled “(delete)”, i.e. “delete” in parentheses? Is it really necessary to delete this link? Why or why not? What does “delete” mean in this specific context? Why was this sequence of statements selected and not the reverse? The above program segment contains two statements, each of which inserts a link. How does it delete the link from start to the (originally) first element in the list? DDS Lecture Notes (draft) - 47 - Robert L. Baber, 1999 July Will this program segment also correctly delete the first element in the list if it is also the last element in the list, i.e. if it is the only element in the list? Why or why not? Draw the appropriate diagrams to convince yourself and others that your answer is correct. If the above program segment also correctly deletes the only element in a list, why does it not matter whether the element to be deleted from the list is the last element in the list or not? 6.6.2. Case 2: deleting an element other than the first from a list In this case, an element other than the first is to be deleted from a list. This statement of the problem implies that the element to be deleted from the list has a predecessor in the list. Let the variable p point to that predecessor; i.e., next(p) points to the element to be deleted from the list. After deletion, the variable pdeleted must point to the item which was deleted from the list. The following diagram illustrates the conditions prevailing before the deletion: ... ... ... ... ... ... p DDS Lecture Notes (draft) - 48 - Robert L. Baber, 1999 July and the following diagram illustrates the conditions required after the deletion: pdeleted ... ... ... ... ... ... p By comparing the two diagrams above defining the initial and final situations, we can deduce the changes which must be made: pdeleted ... ... delete (delete) ... ... ... ... insert p From these diagrams, especially the last one above, it follows that the program segment pdeleted := next(p) next(p) := next(next(p)) DDS Lecture Notes (draft) - 49 - Robert L. Baber, 1999 July will 1. delete an element other than the first from the list, in particular, will delete the successor of the element to which p points, and will 2. record in pdeleted the location (index) of the element that was deleted from the list: Notice that “next(p)” appears here exactly where “start” appeared in the algorithm for case 1 above (see section 6.6.1 above). Questions: What preconditions must be met before the program segment above is applied to a list? How can one deduce these preconditions from the statement of this case of the problem? from the program segment above? E.g., what is the minimum permissible length of a list to which the above program segment is applied? What other conditions must be fulfilled before this program segment is executed? Express these conditions both in a verbal form related to the problem description and in the form of mathematical expressions. Why is the one link labelled “(delete)”, i.e. “delete” in parentheses? Is it really necessary to delete this link? Why or why not? What does “delete” mean in this specific context? Why was this sequence of statements selected and not the reverse? The above program segment contains two statements, each of which inserts a link. How does it delete the link from start to the (originally) first element in the list? Will this program segment also correctly delete an element in the list if that element is also the last element in the list? Why or why not? Draw the appropriate diagrams to convince yourself and others that your answer is correct. If the above program segment also correctly deletes the last element in a list, why does it not matter whether the element to be deleted from the list is the last element in the list or not? Question: Can the above two cases be reduced still further to only one? If so, how? If not, why not? 6.7. List of available elements The functions of inserting and deleting elements of a linked list as described in sections and above give rise to the questions where does the memory space for the new list elements come from and what happens with the space freed by deleting old list elements? Many programming language systems which support linked list structures maintain a list of available, i.e. unused, list elements, sometimes called the list of free elements or, more simply, the free list. Such a list can be implemented as any other list, using the techniques and mechanisms described earlier in this chapter. If newly freed list elements are inserted into the free list at its beginning and list elements required for new data items are taken from the beginning of the free list, the insertion and deletion operations involve only one case each to consider and are correspondingly simple. Example: The arrays implementing the list in section 6.1 above could be extended to include a free list. The following table illustrates one such possible extension: DDS Lecture Notes (draft) - 50 - Robert L. Baber, 1999 July index i x(i) next(i) nextfree(i) 1 13 5 ... 2 ... ... 7 3 ... ... 2 4 11 1 ... 5 14 6 ... 6 14 10 ... 7 ... ... 11 8 42 -1 ... 9 ... ... -1 10 37 8 ... 11 ... ... 9 Here, the free list begins at index 3. Therefore, if the variable startfree is used to indicate the beginning of the free list, its value in this example would be 3. Questions: How can one determine whether free list elements are available or not in the above table? In some systems the need for determining the number of free elements available arises frequently. One way of determining this number is, of course, to scan the list, counting the number of elements traversed until the end of the free list is reached. Describe briefly a way of determining this number more quickly. What invariant condition must then be maintained by any program segment operating on the free list? [end of questions] Other mechanisms for monitoring and controlling available memory exist and are used in practice. Some are variants of the basic linked list techniques and methods described above. Exercise: A number of data items no longer needed are organized in a list. The variable start points to the first element of this list and the variable last points to the last element of the list. All of the data items in this list are to be returned to the free list. Design an algorithm which performs this operation. Under what initial conditions will your algorithm function correctly? DDS Lecture Notes (draft) - 51 - Robert L. Baber, 1999 July 7. Queues In various types of computer systems the need arises often for storing a number of data values in such a way that they are retrieved in the same order as they were stored. Such a queue, or buffer, is often called a first-in-first-out (FIFO) queue. In computer science jargon it is often called a pipeline. Alternatively, one may want to retrieve the values in a queue in the opposite order than that in which they were stored. Such a queue, or buffer, is often called a last-in-first-out (LIFO) queue. In computer science jargon such a queue is often called a stack. A queue of either of these types can be realized in a computer system in various ways. The most obvious and natural of these are a one dimensional array and a linked list. The two primary operations on a queue (either a pipeline or a stack) are to place a data item into the queue (also called store, insert, enqueue, put, push, etc.) and to retrieve a data item from the queue (also called fetch, remove, dequeue, serve, get, pop, etc.). Viewed mathematically as functions, put maps a queue and a data item to a queue, while get maps a queue to a queue and a data item: put: Q D Q get: Q D Q where Q is the set of queues and D is the set of data items. A queue here is a sequence of data items. The data items are of an unspecified type (i.e. are elements of an unspecified set). In the case of a stack, the put and get operations defined above are usually called push and pop respectively. In addition, other operations and functions are typically defined to create a queue, to determine whether a queue is empty or not and, if the length of the queue is bounded (as it always is in practice), to determine whether the queue is full or not. Still other functions can be defined to satisfy special needs arising in particular applications (e.g. to determine the number of elements in a queue). 7.1. Pipelines The distinguishing characteristics of a pipeline (a first-in-first-out queue) are: the put operation applied to a queue q and a data item d yields as its result a queue q' in which d has been appended to the end of the queue q. I.e., q' = q & [d], where & stands for concatenation and the get operation applied to a queue q returns the data item d, which is the first element in the queue q, and a queue q' which is the rest of q after removing its first element. These are sometimes written d = first(q) and q' = tail(q). The relationship between q, d and q' is given by the equation q = [d] & q'. Example of a pipeline: If q = [a, b, 2, 5t5] and d = z, then the result of performing the put operation is the queue (sequence) [a, b, 2, 5t5, z]. If q = [a, b, 2, 5t5, z], then performing the get operation yields the pair of results d and q', where d = a and q' = [b, 2, 5t5, z]. [end of example] DDS Lecture Notes (draft) - 52 - Robert L. Baber, 1999 July Note that the results of the get operation are defined above only for a non-empty queue. I.e., the domain of the get function is the set of non-empty queues. A pipeline can be realized in a straightforward way with a one dimensional array. Beginning at, say, index 0, items are placed into the pipeline at consecutively higher positions. Correspondingly, they are retrieved, beginning at 0, from consecutively higher positions. This can be represented by the following diagram: out in index 0 1 2 3 4 5 6 7 8 9 10 11 12 ... queue ... ... C D E F G H I J ... ... ... ... The variables out and in keep track of the positions at which the next retrieval and insertion operations respectively are to be performed. This pipeline currently contains the queue [C, D, E, F, G, H, I, J]. The pipeline contains data from index out to index in1 inclusive. The number of data items in the pipeline is given by the expression inout. This number must always be zero or positive, i.e. the condition out in must always be met. If out = in, the pipeline is empty and no retrieval operation may be performed. The put operation (as defined above) can be performed by the program segment queue(in) := d; in := in + 1 [put] and the get operation as defined above, by d := queue(out); out := out + 1 [get] A precondition for invoking this get operation is the condition alluded to above, namely that the queue is not empty. More precisely, this precondition is out < in (and, of course, that the values of out and in must be integers). These operations are efficient in terms of time, but not in terms of memory occupation. Enough memory must be provided to store the entire sequence passed through the queue, not just the maximum number of items actually stored in the queue. Memory once used but no longer needed (i.e. the region to the left of out) is never reused. Another shortcoming is that, in practice, the length of the array is always bounded, and as soon as that amount of data has passed through the pipeline, no more data can be placed into it, even though it might be empty. These shortcomings can sometimes be overcome by resetting out and in to 0 if and when the pipeline becomes empty. Depending upon the nature of the application using the pipeline, this may occur often, seldom, or never. Thus, this approach may, but need not, overcome the shortcomings identified above. The shortcomings above can always be overcome by shifting the data in the pipeline to the left each time a data item is retrieved. Then, the array need only be as long as the longest sequence ever actually held in the queue. This would be comparatively efficient in terms of memory occupation, but the need to shift a possibly long sequence to the left each time a single data item is retrieved would result in unnecessarily long execution times. Here we see an example of time vs. memory tradeoff which arises in design tasks. DDS Lecture Notes (draft) - 53 - Robert L. Baber, 1999 July Another approach, which avoids the shortcomings of both of the above algorithms, is to realize the pipeline by a circular “array”, which is, in turn, realized in a one dimensional array. The array queue with indices ranging from 0 to N1 is considered to be a circular structure, with queue(0) being the successor of (i.e. following) queue(N1). More generally, the index of the array element following queue(i) is queue(i1), where stands for addition modulo N. Modulo addition gives the array the circular structure desired. Also here, as in the example above, the variables out and in indicate the positions at which the next retrieval and insertion operations respectively are to be performed. The following diagram illustrates this data structure for a pipeline. N1 0 circular array queue out in The contents of the pipeline are the data items in array elements queue(out) (inclusive) through queue(in) (exclusive). In other words, the pipeline contains all data items from and including queue(out) to but excluding queue(in), in that order. A certain ambiguity arises in the case that out = in, in particular, whether the pipeline is full (contains N data items) or is empty (contains 0 data items). There are several ways of resolving this design question. Normally, it is necessary that an empty pipeline be provided for, so all generally useful alternatives must foresee the possibility of an empty queue. Perhaps the simplest solution is to define out = in to mean that the pipeline is empty. This implies, in turn, that when in1 = out, no more data items may be placed into the pipeline, for doing so would result in increasing in to be equal to out, effectively emptying the pipeline and losing all N data items (the N1 items previously in the pipeline and the one being inserted). In other words, in this case the pipeline is effectively full when in1 = out, even though it is not physically full then. Thus, at most N1 data items would ever be stored in the pipeline at any one time, effectively wasting 1 element in the array. For large N, this is not really significant. Another solution would be to maintain an additional variable indicating whether the pipeline is empty or not (or whether the pipeline is full or not). If out = in, the value of the additional DDS Lecture Notes (draft) - 54 - Robert L. Baber, 1999 July variable will determine whether the pipeline is full or empty. When in becomes equal to out, one can always distinguish between a full pipeline or an empty pipeline, so the correct value of this additional variable can always be maintained. If in becomes equal to out while placing a data item into the pipeline (i.e. while incrementing in), the pipeline becomes full (not empty). If in becomes equal to out while removing a data item from the pipeline (i.e. while incrementing out), the pipeline becomes empty (not full). Still another solution is to indicate an empty pipeline by a special value of out and/or in, the special value being outside the range of allowed array index values. I.e., this special value would be less than 0 or greater than N1. One such suitable special value would be 1. If we choose the convention that out = in always means that the pipeline is empty, the following program segment will implement the put operation: queue(in) := d; in := in 1 [put] The get operation will be implemented by d := queue(out); out := out 1 [get] A precondition for invoking this put operation is the condition that the pipeline is not full, i.e. that in1 out. A precondition for invoking this get operation is the condition that the queue is not empty, i.e. that in out. Summarizing the full and empty conditions in the design convention selected here, the pipeline is full iff in1 = out and it is empty iff in = out. 7.2. Stacks As already stated on page 52 in section 7 above, in the case of a stack, the put and get operations defined above for pipelines are usually called push and pop respectively. The defining characteristics of a stack (a last-in-first-out queue) are: if a push operation is followed immediately by a pop operation, the stack is returned to its original condition and the data item retrieved by the pop operation is the same as the one inserted by the push operation. Viewed in another but equivalent way, the distinguishing characteristics of a stack are: the push operation applied to a queue (stack) q and a data item d yields as its result a queue q' in which d has been appended to the end (also called “top” in the context of a stack) of the queue q. I.e., q' = q & [d], where & stands for concatenation and the pop operation applied to a queue q returns the data item d, which is the last element in the queue q, and a queue q' which is the rest of q after removing its last element. Example of a stack: If q = [a, b, 2, 5t5] and d = z, then the result of performing the push operation is the queue (stack) [a, b, 2, 5t5, z]. If q = [a, b, 2, 5t5, z], then performing the pop operation yields the pair of results d and q', where d = z and q' = [a, b, 2, 5t5]. [end of example] Sometimes the definition of a stack appears in a different form in which the newly pushed item is prefixed to the sequence, in which case an item is popped from the beginning of the sequence. This aspect of the definition is immaterial; important is only that both operations DDS Lecture Notes (draft) - 55 - Robert L. Baber, 1999 July are performed on the same end of the queue representing the stack, ensuring that the last item pushed (stored) onto the stack will be the first (next) one to be popped (retrieved). Note that the results of the pop operation are defined above only for a non-empty stack. I.e., the domain of the pop function is the set of non-empty stacks. A stack can be realized in a straightforward way with a one dimensional array. Beginning at, say, index 0, items are placed onto the stack at consecutively higher positions. Correspondingly, they are retrieved, beginning at the top, from consecutively lower positions. This can be represented by the following diagram: top index 0 1 2 3 4 5 6 7 8 9 10 11 12 ... queue A B C D E F G H I J ... ... ... ... The variable top keeps track of the “top” of the stack, i.e. the position at which the last data item pushed onto the stack was inserted. This stack currently contains the queue [A, B, C, D, E, F, G, H, I, J]. The stack contains data from index 0 to index top inclusive. The number of data items in this stack is given by the expression top+1. This number must always be zero or positive, i.e. the condition 1 top must always be met. The stack is empty iff top = 1, in which case no pop (retrieval) operation may be performed. The push operation (as defined above) can be performed by the program segment top := top + 1; queue(top) := d [push] and the pop operation as defined above, by d := queue(top); top := top 1 [pop] A precondition for invoking this pop operation is the condition mentioned above, namely that the stack is not empty. More precisely, this precondition is 1 < top (or, equivalently, that 0 top), and, of course, that the value of top must be an integer. In practice, the amount of computer memory is always bounded and, therefore, each array, such as queue above, is limited in size. This must be taken into account when designing and/or using an implemented stack mechanism. Question: Assume that in the above illustration of an implementation of a stack, the array queue has been declared to have N elements, i.e. that the range of indices is from 0 to N1 inclusive. What additional condition(s) must be placed on invoking the push and/or pop operations? What other changes, if any, must be made in the above design of the stack system? What mathematical expression is equivalent to the statement “the stack is full”? How did you deduce or derive this expression and why is it valid? Exercise: Implement the push and pop operations as above, but using a linked linear list to represent the stack. DDS Lecture Notes (draft) - 56 - Robert L. Baber, 1999 July Stacks are used in computer systems for various purposes. One of the common uses is in the implementation of recursive subprograms. All of the data associated with each invocation (activation) of a recursive subprogram is typically stored in the stack: the return address (the location of the next instruction to be executed after the execution of the subprogram is complete), variables local to the subprogram, the arguments (input data) of the invocation in question and any other information defining the context of the computation. Example: Consider the following recursive procedure for calculating the factorial of a number: procedure factorial(n) if n=0 then return 1 as the result else return n*factorial(n-1) as the result endif end procedure Typically this subprogram would be compiled to machine code which satisfies the following (or a very similar) specification: If the top item in the stack is a non-negative integer n, [precondition] the second item in the stack is the label (location, address) of a program statement and control is transferred to the label “factorial”, then the top two items will be popped (removed) from the stack, [postcondition] the factorial of n will be pushed (placed) onto the stack and control will be transferred to the program statement identified by the label which was originally the second item in the stack. The values of the program variables n, f and next and the stack items mentioned above may have been modified by the execution of the procedure beginning at the label “factorial”, but no other changes will have been made to the data environment, in particular, other items lower in the stack are left unchanged. A program segment satisfying the above specification can be used to calculate the factorial of a number n by executing the following sequence of program statements: push “end” push n goto “factorial” end: stop Upon termination (reaching the stop command) the top item in the stack will be the factorial of n, the value pushed onto the stack by the statement “push n” above. DDS Lecture Notes (draft) - 57 - Robert L. Baber, 1999 July The above procedure “factorial” would typically be compiled to the following code or its equivalent: procedure factorial(n) if n=0 then return 1 as the result else return n*factorial(n-1) as the result factorial: pop n if n=0 then pop next push 1 goto next else push n push “continue” push n1 goto factorial continue: pop f pop n pop next push n*f goto next endif end procedure Notice how the program code on the right ensures that the specification is satisfied, in particular, regarding the state of the stack. Note also how the value of n, which must be multiplied by the value of the factorial of n1, as well as the return address are maintained over the internal recursive activation of the procedure by placing these values on the stack. Notice also how the specification of the procedure factorial is used in the design and verification of the program code on the right. This corresponds to the inductive step in a proof by induction. The then part of the if construct corresponds to the base step in a proof by induction. No limitation on the stack size has been considered above for didactical reasons. In practice, appropriate additions to the above program code would have to be made to ensure that such a limitation is not exceeded. This would require extending the specification appropriately. Exercise: Trace through the execution of the above stack oriented program for n = 0, 1, 2, etc. Other applications of stacks include traversing trees and converting between Polish and infix notation for mathematical expressions. These subjects are treated in later sections below. 7.3. Priority Queues Queues may also be defined in such a way that neither the first nor the last data item inserted into the queue is necessarily the next item which will be retrieved. Some other criterion can be defined to determine the retrieval priority. Such queues are sometimes called priority queues or highest-priority-in/first-out queues (HPIFO). Logically, a priority queue amounts to a linear queue coupled with a sorting mechanism. A particularly simple way to implement such a queue is to put (insert) each new data item into the queue in a position corresponding to its priority. A get operation then retrieves the data item at the high-priority end of the queue. DDS Lecture Notes (draft) - 58 - Robert L. Baber, 1999 July Question: How many data items must be examined when entering a new data item into a priority queue implemented in a one dimensional array or a linear linked list? (Give the minimum, maximum and average number and state any assumptions needed to determine these quantities.) Tree structures can enable priority queues to be implemented so that the insertion time is considerably less than that for a linear queue. Tree structures will be covered in the next section. DDS Lecture Notes (draft) - 59 - Robert L. Baber, 1999 July 8. An introduction to trees 8.1. Definitions and terminology A tree is defined as a non-empty collection of nodes and edges exhibiting the following characteristics: 1. A node (also called a vertex) can contain information (data item(s)). 2. An edge (also called an arc, a branch or a link) is a directed connection between two distinct (different) nodes. The direction of an edge defines a predecessor-successor (parentchild) relationship between the two nodes it connects. 3. There is a unique node, called the first or root node, which has no predecessor (parent). 4. Every node except the root node has exactly one predecessor (parent). 5. A node may have any number of successors (children). 6. For every node other than the root node, there exists a path (see definition below) from the root node to that other node. Additional useful terms and conventions relating to trees include: A node with no successors (children) is called a leaf (end, terminal) node. A node which is neither a root nor a leaf node is sometimes called an internal node. A path is a sequence of vertices in which every pair of adjacent vertices is connected by an edge in the tree, whereby all such pairs are related in the same parent-child direction. Alternatively, a path can be viewed as the sequence of edges connecting these vertices. The root node of a tree is usually drawn as a circle at the top of the diagram. Each edge is drawn in a downward direction from the predecessor to the successor. An edge is sometimes drawn with an arrowhead to indicate the direction of the predecessor-successor relationship. If the above mentioned convention of the downward direction is systematically followed, however, the arrowhead is superfluous (see point above) and, therefore, can be omitted. A descendant of a node N is any node which is a child of N or a child of a child of N, ... etc. Two or more nodes are siblings of each other if they have the same parent. Any node can be thought of as the root of a subtree consisting of itself and all of its descendants. Additional characteristics of a tree following from the above definition include: There is exactly one path from the root node to any given node in the tree. There are no cycles (loops) in the tree, i.e. there is no path from any node back to itself. A unique number can be associated with each node, that number being the number of edges along the path from the root to the node in question. This number defines the hierarchical level upon which the node in question is located. This hierarchical level of a node is sometimes called the depth of the node. The maximum level number over all nodes is called the height (depth) of the tree. DDS Lecture Notes (draft) - 60 - Robert L. Baber, 1999 July Example: Consider the following tree: A B E Level 0 C F D G K L H M I Level 1 J Level 2 Level 3 Node A is the root node. Its children are nodes B, C and D. Nodes K, L and M are children of node G. Node D is the parent of nodes H, I and J. The root node (here A) is always on level 0. Nodes B, C and D are on level 1. Nodes E, F, G, H, I and J are on level 2. Nodes K, L and M are on level 3. Nodes G, K, L and M are the descendants of node C. The tree is of height 3. The leaf nodes are E, F, K, L, M, H, I and J. Nodes B, C, D and G are internal nodes. Nodes H, I and J are siblings. The descendants of node A are the nodes B, C, D, E, ... M. [end of example] A binary tree is defined as a tree in which each node has at most two successors (children). More generally, an n-tree is defined as a tree in which each node has at most n successors (children). Trees are used to represent various types of data in quite different kinds of applications. They are used to represent mathematical expressions in compilers and other programs which manipulate such expressions. In computer systems used for supporting product design, development and manufacturing trees are used to represent the hierarchical composition of a product through its various subsystems. Such information is needed, for example, to produce bills of materials for production planning. 8.2. Traversing a tree A process involving inspecting nodes and processing the data associated with them is often called traversing a tree. The term traversing expresses the notion that many or all of the nodes must be “visited” or scanned in this process. The nodes of tree may be scanned and processed in different sequences; different sequences give rise to different ways of traversing the tree. Three common ways of traversing a tree are inorder, preorder and postorder traversals. They differ in the order in which the parent and children nodes are processed. In a preorder transversal of a (sub)tree, the root node of the (sub)tree in question is processed first, then its subtrees are traversed in order, beginning with the leftmost subtree and ending with the rightmost. DDS Lecture Notes (draft) - 61 - Robert L. Baber, 1999 July In a postorder transversal of a (sub)tree, first its subtrees are traversed in order, beginning with the leftmost subtree and ending with the rightmost subtree. Finally, the root node of the (sub)tree in question is processed. In an inorder transversal of a binary (sub)tree, its left subtree is transversed first, then the root node of the (sub)tree in question is processed, and finally, its right subtree is transversed. For trees other than binary trees, this procedure must be suitably generalized, depending upon the interpretation of the data structure represented by the tree. One possibility, which is often meaningful, is to process the root node of the (sub)tree in question between the traversals of its subtrees. Variants of the three traversal procedures are also found in practice. For example, an inorder traversal is sometimes supplemented by an initial and a final procedure. I.e., the initial procedure is first performed, then the sequence of subtree traversals and root node processing is executed as described above, and, lastly, the final procedure is performed. Example (mathematical expression): Consider the following tree: * x + * a c b Clearly this tree could be interpreted as representing the arithmetic expression (x*(a*b + c)). Question: Why? Consider the following variant of an inorder traversal algorithm. Its parameter is a pointer to the root node of a (sub)tree. DDS Lecture Notes (draft) - 62 - Robert L. Baber, 1999 July procedure outputexpression(treeroot) if node treeroot has children nodes then output “(“ outputexpression(leftsubtree(treeroot)) output the data (arithmetic operation) at node treeroot outputexpression(rightsubtree(treeroot)) output “)” else (node treeroot is a leaf node) output the data (variable name) at node treeroot endif endprocedure This procedure will output a fully parenthesized expression with the same arithmetic meaning as the tree structure beginning at the node to which the input parameter points. Applied to the tree in the preceding diagram, this procedure outputexpression will result in the following output string: (x*((a*b)+c)) The following procedure is a postorder traversal algorithm for such a tree: procedure outputpost(treeroot) if node treeroot has children nodes then outputpost(leftsubtree(treeroot)) outputpost(rightsubtree(treeroot)) output the data (arithmetic operation) at node treeroot else (node treeroot is a leaf node) output the data (variable name) at node treeroot endif endprocedure Applied to the tree in the preceding diagram, this procedure outputpost will output the following string: xab*c+* We will see in section 9 that an expression in this form (called Polish postfix notation) is another meaningful way of describing or formulating the computation represented by the tree and by the expression (x*((a*b)+c)). The following procedure is a preorder traversal algorithm for such a tree: procedure outputpre(treeroot) if node treeroot has children nodes then output the data (arithmetic operation) at node treeroot outputpre(leftsubtree(treeroot)) outputpre(rightsubtree(treeroot)) else (node treeroot is a leaf node) output the data (variable name) at node treeroot endif endprocedure DDS Lecture Notes (draft) - 63 - Robert L. Baber, 1999 July Applied to the tree in the preceding diagram, this procedure outputpre will output the following string: *x+*abc In section 9 we will see that also this expression is a meaningful way of describing the computation represented by the other expressions and the tree above. This form for an expression is called Polish prefix notation. Example (bill of materials): Consider a production planning activity concerned with making desks. In part of the planning process the total quantities of the various materials required to produce the desired number of desks must be calculated. This calculation is often called a “bill of materials explosion”. A tree structure for representing the data upon which a bill of materials explosion is based lends itself well to the hierarchical composition of such a product consisting of various subassemblies and components. In this simplified example we will assume that: Each desk consists of 2 pedestals and 1 top. Each pedestal consists of 1 pedestal frame, 2 feet and 3 drawers. Each pedestal frame is an end item (e.g. externally procured). Each foot is an end item. Each drawer consists of 1 side and bottom trough, 1 back panel and 1 front panel. Each side and bottom trough is an end item. Each back panel is an end item. Each front panel is an end item. DDS Lecture Notes (draft) - 64 - Robert L. Baber, 1999 July Each top is an end item. The above data could be represented by the following tree: desk 2 pedestal 1 2 3 pedestal frame top foot 1 trough 1 drawer 1 1 back panel front panel The following procedure will traverse such a tree and output a list of the quantities of the end items required to make a given number of desks. Its parameters are (1) the quantity of the product required and (2) a pointer to the root node of the (sub)tree for the product required. procedure listenditems(quantityrequired, treeroot) if node treeroot has children nodes then for each child of the node treeroot listenditems(quantityrequired*quantityperproduct(treeroot, child), pointertochild(treeroot, child)) endfor else (node treeroot is a leaf node) output quantityrequired and the data (name of the end item) at node treeroot endif endprocedure In this tree, each node contains the name of the product or subassembly in question and, for each child, (1) the quantity of the child (subsidiary component) required to make one unit of the product or subassembly in question and (2) a pointer to the subtree defining the child and its composition. If the above procedure were invoked with a value of 10 for the parameter quantityrequired and a pointer to the node for “desk” for the parameter treeroot, the following list would be produced: 20 pedestal frame 40 foot 60 back panel 60 front panel DDS Lecture Notes (draft) - 65 - Robert L. Baber, 1999 July 60 trough 10 top These end items are required in these quantities in order to make 10 desks. In an actual production planning application additional data would be contained in the various nodes, such as the cost of each component or subassembly, the cost of assembling each product (i.e. the cost of labor, energy, amortization of manufacturing equipment, etc.), production times, etc. The structure of the data collection would be the same, only more details would be provided and processed. Furthermore, the above example has been simplified; many parts such as fasteners, packaging materials, etc. have been omitted. Again, a realistic application system would exhibit the same structure, only more details of the same type as already included above would be added. Question: How could a drawing of the desk be represented by a data structure of the above type? What data would be required in each node? Question: Typical programming languages require that the number of data items (including pointers to other nodes) associated with each type of node be bounded by a fixed number. How can a data structure in which a node may have any number of children be represented in such a system? 8.3. Creating a tree Various algorithms can be designed to create a tree from equivalent expressions in different forms. An expression in the form corresponding to the output of a preorder scan is particularly convenient as a basis for generating a tree. When creating a tree, nodes with empty data fields must be made available. In programming language systems supporting tree structures, a system function is typically provided which, when called, will allocate memory for a new node and return a pointer to that node. Such a system function often removes the memory required from a list of free memory as described earlier in section 6.7 above. Example: The following algorithm (in pseudocode form) will generate the tree corresponding to an input expression in Polish prefix notation (e.g. *x+*abc, cf. the example of the mathematical expression in section 8.2 above). The result parameter pointer will point to the root node of the tree generated. The input expression must be a non-empty, syntactically correct expression in Polish prefix notation. Note that the example expression involves only binary operators (operators with exactly two arguments each). Thus its corresponding tree will be a binary tree. If operators with a larger number of arguments could appear in the expression, the corresponding tree would no longer be a binary tree. The basic idea underlying the following algorithm is that when it is invoked, a node is created for the first symbol in the expression (either an operator or a value or variable name). The rest of the expression consists of the expressions for the left subtree and the right subtree. These two expressions are separated and the subtree for each is generated by (recursively) calling the procedure for generating a tree. Links from the parent node to the subtrees are established. Question: When is such an expression syntactically correct and in Polish prefix notational form? Give a rule for determining whether a string of symbols meets this requirement. DDS Lecture Notes (draft) - 66 - Robert L. Baber, 1999 July procedure buildtree(expression, rootnode) Separate expression into firstsymbol, leftsubexpression and rightsubexpression. Allocate a new node and set rootnode to point to the new node. Copy the value of firstsymbol into the data field of node rootnode. if firstsymbol is an operator (e.g. +, *, etc.) then buildtree(leftsubexpression, subnode) In the node to which rootnode points, set the pointer to the left subtree to subnode. buildtree(rightsubexpression, subnode) In the node to which rootnode points, set the pointer to the right subtree to subnode. else {firstsymbol is a value or variable} In the node to which rootnode points, set the pointer to the left subtree to the end marker value. [see note below] In the node to which rootnode points, set the pointer to the right subtree to the end marker value. [see note below] endif endprocedure Note: Some systems will provide the newly created node with these pointer fields already initialized to the end marker value. In such cases, these statements (and hence the entire else part of the if statement) are superfluous and may be omitted. Question: Is termination of the above algorithm guaranteed? Why? Questions: How can a Polish prefix expression be separated into the first symbol and its two subexpressions (i.e. on the basis of what criteria)? How does this process relate to the criterion for a syntactically correct expression (cf. previous question above). How does this criterion relate to the structure of the algorithm above and, in particular, its correctness? Question: Separating an expression in Polish prefix notation into the first symbol and two subexpressions (cf. the first step in the above algorithm) will presumably involve scanning the expression, or at least a significant part of it, symbol by symbol. How can the algorithm above be reorganized to eliminate the need for this process and, thereby, reduce the time complexity of generating the tree? DDS Lecture Notes (draft) - 67 - Robert L. Baber, 1999 July 9. Polish notation 9.1. Expressions in infix, Polish prefix, Polish postfix and tree form In section 8.2 above it was mentioned that the tree * x + * a c b and the expressions *x+*abc [Polish prefix notation] xab*c+* [Polish postfix notation] (x*(a*b + c)) [infix notation] can all be interpreted to have the same meaning and are equivalent to each other in this sense. The first two expressions above are in a form called Polish notation. Notice that they are both free of parentheses, which gives rise to one of their advantages: the order of computation is specified by the sequence of symbols only. The more familiar and, by people at least, more commonly used notation is called infix notation to distinguish it from the two Polish notational forms. The term infix was suggested by the fact that the symbols for the mathematical operations are interspersed within the expression, between their operands. When writing an expression in infix notation, the sequence of symbols representing the variables and the operations is not sufficient to specify the intended mathematical function; parentheses are often also needed to specify the order of computation. The tree clearly suggests the following sequence of computations: 1. multiply the values of the variables a and b together 2. add the product obtained in step 1 above and the value of the variable c 3. multiply the value of the variable x and the sum obtained in step 2 above to give the final result The expression in infix form above also indicates the same sequence of computational steps. DDS Lecture Notes (draft) - 68 - Robert L. Baber, 1999 July An expression in Polish postfix notation is often interpreted with the help of a stack. The expression is scanned from left to right. Each variable or value encountered is pushed onto the stack. Whenever a symbol for an operation is encountered, the appropriate number (in the above example, 2) of variables is popped from the stack and the operation applied to them. The result is pushed onto the stack and the scan of the expression continued. After the expression has been completely scanned, the result is the only item left in the stack. Alternatively, instead of performing the calculation, machine language instructions to perform the calculation can be easily generated; this technique is often used in compilers and interpreters. 9.2. Converting expressions between infix, Polish notation and tree form We have already encountered several algorithms for transforming expressions between tree, infix, Polish prefix and Polish postfix forms; they are illustrated in the following diagram. The number on each arrow representing a conversion indicates the section in which the algorithm appears. tree 8.2 8.3 8.2 infix 8.2 Polish prefix Polish postfix In the sections below algorithms for other conversions between these forms will be given. They should be considered as examples, not the only or the best way of performing the conversion in question. 9.2.1. Converting from fully parenthesized infix to Polish postfix notation The following algorithm will convert an expression from fully parenthesized infix form to the equivalent expression in Polish postfix notation. The logic in this algorithm depends critically on the input expression being fully parenthesized, that is, a pair of parentheses encloses each operator together with its operands. Furthermore, only a single pair of parentheses may enclose each such group. Expressed differently, the number of left parentheses, the number of right parentheses and the number of operator symbols must all be equal to each other. The input to the algorithm is the expression in fully parenthesized infix form. The output is an initially empty string of symbols. An initially empty stack is used in the process of generating the postfix expression from the infix expression. DDS Lecture Notes (draft) - 69 - Robert L. Baber, 1999 July The input expression in fully parenthesized infix form is scanned from left to right. For each symbol scanned an action is performed as follows: If the symbol scanned is a value or variable an operator right parenthesis ) left parenthesis ( the action below is performed. the value or variable is appended to the output string the operator is pushed onto the stack the operator at the top of the stack is popped and appended to the output string no action (the left parenthesis is skipped over) The algorithm terminates after the last symbol in the input expression has been scanned and processed, i.e. when there is no more symbol in the input string. Basically, this algorithm moves each operator to the right of its right hand subexpression, immediately preceding its corresponding right parenthesis. In the resulting expression, the parentheses would be superfluous, so they are removed in the conversion process. Question: What is the time complexity of this algorithm? memory space complexity? Question (optional): How can an expression in infix form but not necessarily fully parenthesized be converted into Polish postfix notation? Consider operator precedence rules in the absence of corresponding parentheses. 9.2.2. Converting from fully parenthesized infix to Polish prefix notation An expression can be converted from fully parenthesized infix form to Polish prefix notation by an algorithm basically symmetric to the one in section 9.2.1 above. Again here, the logic in this algorithm depends critically on the input expression being fully parenthesized, that is, a single pair of parentheses encloses each operator together with its operands. The number of left parentheses, the number of right parentheses and the number of operator symbols must all be equal to each other. The input to the algorithm is the expression in fully parenthesized infix form. The output is an initially empty string of symbols. An initially empty stack is used in the process of generating the prefix expression from the infix expression. The input expression in fully parenthesized infix form is scanned from right to left. For each symbol scanned an action is performed as follows: If the symbol scanned is a value or variable an operator left parenthesis ( right parenthesis ) the action below is performed. the value or variable is prefixed to the output string the operator is pushed onto the stack the operator at the top of the stack is popped and prefixed to the output string no action (the right parenthesis is skipped over) The algorithm terminates after the leftmost symbol in the input expression has been scanned and processed, i.e. when there is no more symbol in the input string. Basically, this algorithm moves each operator to the left of its left hand subexpression, immediately to the right of its corresponding left parenthesis. In the resulting expression, the parentheses would be superfluous, so they are removed in the conversion process. Question: What is the time complexity of this algorithm? memory space complexity? DDS Lecture Notes (draft) - 70 - Robert L. Baber, 1999 July Question (optional): How can an expression in infix form but not necessarily fully parenthesized be converted into Polish prefix notation? Consider operator precedence rules in the absence of corresponding parentheses. 9.2.3. Converting from Polish postfix to fully parenthesized infix notation In this algorithm we will make use of a stack, each element of which is a string. Each element of the stack will be a fully parenthesized infix expression or a single value or variable. Because a single value or variable contains no operation and no parentheses, it can be viewed as a special case of a fully parenthesized infix expression. Thus, every element in the stack can be considered to be a fully parenthesized infix expression. The input to the algorithm is the expression in Polish postfix notation. The output is a string of symbols as the only element in the stack. An initially empty stack is used in the process of generating the fully parenthesized infix expression from the input expression in Polish postfix notation. The input expression in Polish postfix notation is scanned from left to right. For each symbol scanned an action is performed as follows: If the symbol scanned is a value or variable an operator the action below is performed. The value or variable is pushed onto the stack. The top of the stack is popped to the working variable rightexpression. The next element in the stack is popped to the working variable leftexpression. The string formed by concatenating a left parenthesis, leftexpression, the operator being scanned, rightexpression and a right parenthesis (in that order) is pushed onto the stack. This algorithm terminates after the last symbol in the input expression has been scanned and processed, i.e. when there is no more symbol in the input string. Upon termination, there will be exactly one element in the stack and it will contain the fully parenthesized infix expression equivalent to the input expression in Polish postfix notation. Basically, this algorithm scans the expression in Polish postfix notation and for each operator, forms the corresponding fully parenthesized infix expression, retaining it in the stack. The final contents of the stack will be the fully parenthesized infix expression for the last operator in the input expression, which is the desired expression in fully parenthesized infix notation. Question: What is the time complexity of this algorithm? memory space complexity? Having added the above three algorithms to our collection, we now have algorithms for the following conversions: DDS Lecture Notes (draft) - 71 - Robert L. Baber, 1999 July tree 8.2 8.2 infix 8.2 9.2.2 9.2.1 8.3 9.2.3 Polish prefix Polish postfix As before, the number on each arrow representing a conversion indicates the section in which the algorithm appears. This set of conversion algorithms forms a complete set in the sense that it enables us to convert an expression in any of these notational forms to an expression in any other, either directly or indirectly. Of course, algorithms for the other direct conversions can be designed. DDS Lecture Notes (draft) - 72 - Robert L. Baber, 1999 July 10. Other types of trees Basic and fundamental aspects of tree data structures were presented in chapter 8. One special type of tree was introduced there: a binary tree. There are, of course, many other special cases of trees, some of which will be examined in this chapter. By placing additional restrictions on tree structures, certain desirable characteristics and properties can be obtained, enabling special goals to be achieved and/or facilitating processing the tree. 10.1. Binary search trees A binary search tree is a binary tree (see the definition in section 8.1 above) which satisfies the following additional conditions: 1. Each node’s data value is an element of a linearly ordered set (i.e., an order relation is defined on the set of the nodes’ data values). 2. For every node N in the tree 2.1. the value of every node L in the left subtree of N is less than the value of N and 2.2. the value of every node R in the right subtree of N is greater than the value of N. Requirement 2.1 above can be modified by replacing the relation “less than” by “less than or equal to”. Similarly, requirement 2.2 above can be modified by replacing the relation “greater than” by “greater than or equal to”. Phrases such as “the data value of node A” or “the data value at node A” occur often and sometimes give rise to lengthy and even clumsy sentences. Such phrases are often shortened to “node A” or even just “A” when it is clear from the context precisely what is meant. I.e., the node identification alone often stands for the data value at (of) that node. Rephrased in this way the conditions above become: 1. Each node is an element of a linearly ordered set. 2. For every node N in the tree 2.1. every node L in the left subtree of N is less than (less than or equal to) N and 2.2. every node R in the right subtree of N is greater than (greater than or equal to) N. Example: 18 6 40 3 1 10 4 DDS Lecture Notes (draft) 7 25 13 19 - 73 - 60 28 42 85 Robert L. Baber, 1999 July Questions: How many nodes can a tree of height 2 contain? height 3? height h? Questions: What is the minimum height of a tree containing n nodes? the maximum height? Questions: What is the minimum number of nodes which must be examined to find a node containing a given value? the maximum number of nodes? the average number of nodes? Note that when the tree is balanced the maximum number of nodes which must be searched is significantly less than when the tree is unbalanced. Thus, the better balanced the tree is, the higher the searching efficiency. Questions: Given that a binary search tree contains 7 data elements (and hence nodes), what is the minimum possible height of the tree? the maximum possible height? if the tree contains n data elements? Note that a non-leaf node in a binary search tree need not have 2 children; it may have only 1. In fact, it may occur that every non-leaf node has only 1 child. The data in the tree in the above diagram could, for example, be organized differently so that no node has more than 1 child. Examples: 85 60 42 40 ... or DDS Lecture Notes (draft) - 74 - Robert L. Baber, 1999 July 85 1 60 42 ... Exercise: Design an algorithm which searches a binary search tree for a node containing a specific value. Assume that the variable top points to the root node of the binary tree and that the value of the variable searchkey is the value to be located in the tree. The algorithm should set the variable nodefound to point to the node containing the value sought. If that value is not present in the tree, the algorithm should set the variable nodefound to the end marker value (endmark). Exercise: Outline an algorithm which deletes a specific node in a binary search tree. What information must be available to such an algorithm? Exercise: Outline an algorithm which inserts a given value into a binary search tree. Questions: How many nodes must be examined or processed when searching a binary search tree and inserting and deleting nodes? Distinguish between the best, average and worst cases. 10.2. Heaps A heap is a binary tree (see the definition in section 8.1 above) which satisfies the following additional conditions: 1. Each node is an element of a linearly ordered set. I.e., an order relation is defined on the set of the nodes’ data values. 2. The tree is complete*, that is, every level has the maximum possible number of nodes except possibly the lowest (deepest) level, and the nodes at the lowest level are in their left most positions. 3. Each node in the tree is less than every one of its children. (This requirement can be modified by replacing the relation “less than” by “less than or equal to”, “greater than” or “greater than or equal to”). * Note: The mathematical and computing science literature contains different definitions of the term “complete tree” which are inconsistent with one another. There is no one single universally accepted meaning of this term. The above definition will be used throughout this document. Example: The following diagram shows a data collection arranged as a heap. DDS Lecture Notes (draft) - 75 - Robert L. Baber, 1999 July 4 6 [1] [2] 40 [4] 30 [3] 49 [5] 33 [6] 55 43 50 52 39 [8] [9] [10] [11] [12] 80 [7] The numbers in the circles representing the nodes are the data values. The numbers in square brackets next to or below the circles representing the nodes are sequence numbers. Note that these sequence numbers form a set of consecutive integers, each of which identifies the exact location of its associated node, i.e. both the level number and the position within that level. Note further that de(i) < de(2i) and de(i) < de(2i+1) for all applicable values of i. Question: If the heap contains n nodes (data elements), what are the applicable values of the sequence numbers i? Question: Why is it possible to assign sequence numbers with the above properties to the nodes of a heap? Why not to other types of binary trees? Because the sequence numbers form a set of consecutive integers, a heap can be implemented efficiently in an array. No array cells will be left unused. Furthermore, pointers from one node to another are superfluous and can be eliminated, saving storage space. Question: For what function is the heap particularly suited? Which restriction in the definition of a heap gives rise to its special advantage for this function? Question: How can a given data value be found in a heap? a new data value inserted into a heap? deleted from a heap? (Outline the algorithms.) Exercise: Compare the binary search tree and the heap with regard to the efficiency (speed and memory requirements) of the following operations: searching for a node with a given data value, inserting a new data element, deleting a specific data value and sorting. DDS Lecture Notes (draft) - 76 - Robert L. Baber, 1999 July 10.3. B-Trees In the tree structures described in the preceding sections of this chapter, each node contained one data element (value) of significance to the structure of the tree. In the case of a B-tree this restriction is relaxed; more than one such value may be present in any node. The number of values in any one node is, however, bounded to facilitate implementation of the B-tree. Let d be a positive integer. A B-tree of order d is a tree satisfying the following conditions: 1. The root node contains between 0 and 2d (inclusive) data elements. 2. If the root node contains 0 data elements, then it has no children and the B-tree is empty. 3. Each node other than the root node contains between d and 2d (inclusive) data elements. 4. The number of children of any node is either zero or one more than the number of data elements it contains. That is, if a node contains k data elements, then that node has either 0 or k+1 children. 5. Each data element (value) in a node is an element of a linearly ordered set. I.e., an order relation is defined on the set of the nodes’ data elements. 6. The data elements within each node are sorted in ascending order. I.e., if a node contains k data elements called de(1), de(2), ... de(k), then de(1) de(2) ... de(k). 7. If node N contains k data elements (called de(1), ... de(k) below) and k 1, then 7.1. every element in the first (left most) subtree of node N is less than or equal to de(1), 7.2. every element in the jth subtree of node N is between de(j-1) and de(j) inclusive, for 1 < j k, and 7.3. every element in the k+1st (last, right most) subtree of node N is greater than or equal to de(k). In basic concept the B-tree is a generalization of a binary search tree with additional constraints imposed which tend to keep the tree balanced (but do not guarantee it). This, in turn, tends to maintain efficiency, e.g. of searching. Exercise: Compare the B-tree with the binary search tree and the heap with regard to the efficiency (speed and memory requirements) of the following operations: searching for a node with a given data value, inserting a new data element, deleting a specific data value and sorting. DDS Lecture Notes (draft) - 77 - Robert L. Baber, 1999 July 11. Comparison of data structures and their algorithms The data structures with their attendant algorithms can be compared with regard to a number of criteria. Two of the most important criteria for assessing algorithms in general are time and memory complexity, i.e. the asymptotic behaviour of the time or memory required as a function of the “size” of the problem, e.g. the number of data elements in the collection to be stored and processed. Most of the algorithms discussed in this document have the same or similar memory complexity, so we will not consider this aspect of these algorithms further. With regard to time complexity, however, the algorithms associated with the data structures introduced in this document vary considerably with regard to time complexity. The following table indicates the time complexity of various functions for several data structures. Where different complexities are given, they are for different common algorithms or for best, average and worst cases. Also, in some cases of algorithms for operating on trees, the time complexity depends upon how well balanced the tree is. Data structure search insert delete sort linear array (not sorted) O(n) constant O(n) O(n*log(n)) O(n1.5) O(n2) linear array (sorted) O(log(n)) O(n) O(n) O(n) n.a. linked list (not sorted) O(n) constant constant O(n2) linked list (sorted) O(n) constant constant n.a. pipeline, stack n.a. constant constant n.a. binary search tree O(log(n)) O(n) constant O(log(n)) O(n) n.a. heap O(n) O(log(n)) O(log(n)) n.a. B-tree O(log(n)) O(n) constant O(log(n)) O(n) constant O(log(n)) O(n) n.a. In the cases of inserting and deleting an item above, it is assumed that the location in question is known. If this is not the case, the time for a search must be considered also. DDS Lecture Notes (draft) - 78 - Robert L. Baber, 1999 July Exercise: The suitability of the several data structures for various processing functions also varies significantly. Assess the several data structures with regard to the functions (e.g. searching, inserting, deleting, sorting, buffering a data stream, etc.) for which they are particularly suited or not suited and fill in the table below correspondingly. Data structure Well suited for Suited or adequate for Not suited for linear array (not sorted) linear array (sorted) linked list (not sorted) linked list (sorted) pipeline, stack binary search tree heap B-tree tree (general) multi-dimensional array DDS Lecture Notes (draft) - 79 - Robert L. Baber, 1999 July 12. Selected other data structures A number of other data structures can be defined and are useful for various applications. Most are variants, generalizations, extensions, etc. of the basic types described in the sections above. The following sections briefly introduce some of these other data structures. A more detailed treatment of these data structures is beyond the scope of this course. Extensive literature on these and other data structures exists. 12.1. Graphs A graph is a collection of nodes and edges (cf. trees, section 8.1) but without most of the restrictions imposed upon a tree. In contrast to a tree, a graph does not necessarily have a root node, may have cycles (loops) and need not be connected (there may be two nodes not connected by any path). An edge connects two nodes, but need not have a direction. A graph is frequently represented in a computer system by nodes and links of the same type as used for linked lists and trees as described in several sections above. For example, the following data structure is a graph, but obviously does not satisfy the definition of a tree. X B P E T R L U F P D Question: Which conditions in the definition of a tree are violated by this graph? (Cf. section 8.1.) 12.2. Hash tables In all of the data structures described above, the procedure for locating a data item with a given value requires, in general, examining a number of data items. Because such searches can take some time and are performed often in many computerized systems, it would be desirable to have a data structure in which a data item with any given value can be directly found and accessed. The hash table provides such access, at least under certain conditions and within certain limits. In its basic form, a hash table is an array of records, each containing one data item or data group. When a data item or group is to be stored in the array, an index is calculated based on the value of one or more of the fields in the record to be stored. In the simple case, the record is then stored in that element of the array referenced by the index value calculated. The function which maps data values to index values is called the hashing function. DDS Lecture Notes (draft) - 80 - Robert L. Baber, 1999 July Early implementations based on this idea calculated the index of a record by adding together the individual characters in the field or fields in question, each viewed as an integer, and either discarding high order carries or wrapping them around. The total derived in this way was called a hash total or hash sum, hence the term hash table for a data structure based on such a mapping function. If a data item is to be stored in an already occupied array element, a collision is said to have occurred. The new record is then stored in some other place; determining which is a design feature of the system which can have a significant effect on the performance of the system, especially with regard to its speed. If a data item is to be located, the hash function is first calculated to give the index of the array element in which the data item would normally be stored. If that element is empty or contains some other element, other locations must possibly be examined, depending upon the insertion algorithm used when collisions arise and also upon the algorithm employed for deleting items. The two main design criteria of such a data storage system are usually resources required (memory space) and performance (time). An ideal hash table system consists, therefore, of an array just large enough to contain the data to be stored and a hashing function which distributes the actual data items uniformly over the available index space without collisions. Usually a design tradeoff is involved. The more densely occupied the array is, the higher the collision rate will be. I.e. memory space can be reduced at the cost of increasing the collision rate and, hence, processing time. An optimum balance between memory cost and processing time is the design goal. 12.3. Permutation arrays Linear sequences of data items arise in many applications. It is often desirable to sort the sequence into various different orders. In order to avoid repeated sorting into different orders and then back again while avoiding the obvious alternative of maintaining a number of copies of the data in different sequences, the data may be effectively sorted by employing one or more permutation arrays. A permutation array is an array of pointers to the original data collection with the property that it permutes the original sequence (which need not be in any particular order). A data collection stored in an array or in a group of arrays may have many permutation arrays associated with it. In this way, the data collection can be sorted in several different orders at one time. Because the data collection is, in effect, stored in a sorted array, the binary search algorithm can be used to find a data element with a given value. DDS Lecture Notes (draft) - 81 - Robert L. Baber, 1999 July For example, consider the following related data arrays Name and Age. The permutation array P1 effectively orders the data alphabetically by name. The permutation array P2 orders the data by age. Index i P1(i) P2(i) Name(i) Age(i) 1 4 1 George 22 2 1 5 Gogo 75 3 2 4 Pieter 52 4 3 3 Christa 33 5 5 2 Themba 25 Note how P1 sorts the arrays Name and Age into alphabetical order on the names: Index i P1(i) Name(P1(i)) Age(P1(i)) 1 4 Christa 33 2 1 George 22 3 2 Gogo 75 4 3 Pieter 52 5 5 Themba 25 and how, at the same time, P2 sorts the arrays Name and Age by age: Index i P2(i) Name(P2(i)) Age(P2(i)) 1 1 George 22 2 5 Themba 25 3 4 Christa 33 4 3 Pieter 52 5 2 Gogo 75 while the original arrays Name and Age remain unchanged and are in no particular order. The arrays P1 and P2 effectively permute the data values in the arrays Name and Age into the desired sequences. Viewed more mathematically, P1 and P2 are one-to-one functions from the set of index values onto itself — the general property characteristic of a permutation. DDS Lecture Notes (draft) - 82 - Robert L. Baber, 1999 July Thus, with the help of permutation arrays, a single collection of data can be in different orders at one and the same time. Questions: How can such a data structure be searched for an entry (record) with a particular name? with a particular age? for an entry satisfying some other criterion? Questions: How can a new data record be added to such a data structure? a record deleted from such a data structure? DDS Lecture Notes (draft) - 83 - Robert L. Baber, 1999 July 13. Summary and conclusions Typical computer hardware provides for only one type of data structure — a sequence of data values, each consisting of a small group of bits (currently most commonly 8 bits, sometimes somewhat larger groups). We have seen in this course that it is possible to realize a wide variety of data structures with only this simple basis to build upon. In your future work you will encounter many different types of data structures, some of which have not yet been developed. In this course, you have been introduced to most of the fundamental data structures in common use today. Many of the others not covered here can be viewed as variants, extensions and specializations of the data structures presented in this course. No one data structure is optimum, even suitable, for all applications. Each data structure has its advantages and disadvantages, its strengths and weaknesses for the various applications. In particular, we have seen that the time complexity for any one type of operation varies from one data structure to another significantly. The designer’s task is to determine the data structure or the combination thereof which results in the best overall performance in the application in question, taking into account the usage patterns which can be expected. DDS Lecture Notes (draft) - 84 - Robert L. Baber, 1999 July 14. References Baber, Robert L.; “A method for representing data items of unlimited length in a computer memory”, IEEE Transactions on Software Engineering, Vol. SE-7, No. 6, Nov. 1981, p. 590-593. IEEE; An American National Standard, IEEE Standard for Binary Floating-Point Arithmetic, ANSI/IEEE Std 754-1985, IEEE, New York, 1985. IEEE; An American National Standard, IEEE Standard for Radix-Independent FloatingPoint Arithmetic, ANSI/IEEE Std 854-1987, IEEE, New York, 1987. Kelsey, Richard; Clinger, William; Rees, Jonathan (eds.); Revised5 Report on the Algorithmic Language Scheme, http://www.cena.dgac.fr/~decrock/langages/scheme/r5rs/r5rshtml/r5rs_toc.html, 20 February 1998. Kline, Morris; Mathematical Thought from Ancient to Modern Times, Oxford University Press, New York, 1972. Recommended reading The following is a small selection of the many books which contain sections dealing with subjects covered in this course: Abelson, Harold; Sussman, Gerald Jay; Structure and Interpretation of Computer Programs, MIT Press, Cambridge, Massachusetts, 1985. Amsbury, Wayne; Data Structures from Arrays to Priority Queues, Wadsworth Publishing Co., Belmont, CA, 1985. Beidler, John; An Introduction to Data Structures, Allyn and Bacon, Inc., Boston, 1982. Brunskill, David; Turner, John; Understanding Algorithms and Data Structures, McGraw-Hill, London, 1996. Kruse, Robert L.; Data Structures and Program Design, Prentice-Hall, Englewood Cliffs, NJ, 1984. Manber, Udi; Introduction to Algorithms: A Creative Approach, Addison-Wesley, Reading, MA, 1989. Manis, Vincent S.; Little, James J.; The Schematics of Computation, Prentice Hall, Englewood Cliffs, New Jersey, 1995. Patterson, David A.; Hennessy, John L.; Computer Organization & Design: the Hardware/Software Interface, Morgan Kaufmann, San Francisco, 1994. Stubbs, Daniel F.; Webre, Neil W.; Data Structures with Abstract Data Types and Pascal, Brooks/Cole Publishing Co., Monterey, CA, 1985. Tenenbaum, Aaron M.; Augenstein, Moshe J.; Data Structures Using Pascal, Prentice-Hall, Englewood Cliffs, NJ, 1986. Warford, J. Stanley; Computer Science, Volumes 1 and 2, D. C. Heath, Lexington, MA, 1991. Weiss, Mark Allen; Data Structures and Algorithm Analysis, Benjamin/Cummings, Redwood City, CA, 1992. DDS Lecture Notes (draft) - 85 - Robert L. Baber, 1999 July