Download **** 1 - Postech

CSED233: Data Structures (2014F)
Lecture14: Hashing
Bohyung Han
CSE, POSTECH
[email protected]
Maps
• A map models a searchable collection of key‐value entries
• Characteristics
 The main operations: search, inserting, and deleting items
 Multiple entries with the same key are not allowed.
• Applications
 Address book
 Student‐record database
key
value
value
Customer ID
Name
Address
010‐5323‐3221
Kim, XXX
Seoul
011‐4221‐1949
Lee, YYY
Pohang
010‐9492‐0988
Park, ZZZ
Daegu
010‐8383‐3221
Kim, XXX
Seoul
2
entry
CSED233: Data Structures
by Prof. Bohyung Han, Fall 2014
The Map ADT
• Methods for map M
 get(key): if M has an entry with key, return its associated value;
otherwise, return null
 put(key,value): insert entry (key,value) into the map M; if key is
not already in M, then return null; else, return old value associated
with the key
 remove(key): if the map M has an entry with key, remove it from M
and return its associated value; else, return null
 size()
 isEmpty()
 entrySet(): return an iterable collection of the entries in M
 keySet(): return an iterable collection of the keys in M
 values(): return an iterator of the values in M
3
CSED233: Data Structures
by Prof. Bohyung Han, Fall 2014
Example
Operation
isEmpty()
put(5,A)
put(7,B)
put(2,C)
put(8,D)
put(2,E)
get(7)
get(4)
get(2)
size()
remove(5)
remove(2)
get(2)
isEmpty()
Output
true
null
null
null
null
C
B
null
E
4
A
E
null
false
Map
Ø
(5,A)
(5,A),(7,B)
(5,A),(7,B),(2,C)
(5,A),(7,B),(2,C),(8,D)
(5,A),(7,B),(2,E),(8,D)
(5,A),(7,B),(2,E),(8,D)
(5,A),(7,B),(2,E),(8,D)
(5,A),(7,B),(2,E),(8,D)
(5,A),(7,B),(2,E),(8,D)
(7,B),(2,E),(8,D)
(7,B),(8,D)
(7,B),(8,D)
(7,B),(8,D)
4
CSED233: Data Structures
by Prof. Bohyung Han, Fall 2014
A Simple List-Based Map
• We can efficiently implement a map using an unsorted list
• We store the items of the map in a list S (based on a
doubly-linked list), in arbitrary order
nodes/positions
header
9 c
5 c
6 c
trailer
8 c
entries
5
CSED233: Data Structures
by Prof. Bohyung Han, Fall 2014
Methods for Map
• get(k) algorithm
Algorithm get(k)
B  S.positions()
// B is an iterator of the positions in S
while B.hasNext() do
p  B.next()
// the next position in B
if p.element().getKey() = k then
return p.element().getValue()
return null
// There is no entry with key equal to k
6
CSED233: Data Structures
by Prof. Bohyung Han, Fall 2014
Methods for Map
• put(k,v) algorithm
Algorithm put(k,v):
B  S.positions()
while B.hasNext() do
p  B.next()
if p.element().getKey() = k then
t  p.element().getValue()
S.set(p,(k,v))
return t
// Return the old value
S.addLast((k,v))
nn+1
// increment variable storing number of entries
return null
// There was no entry with key equal to k
7
CSED233: Data Structures
by Prof. Bohyung Han, Fall 2014
Methods for Map
• remove(v) algorithm
Algorithm remove(k):
B  S.positions()
while B.hasNext() do
p  B.next()
if p.element().getKey() = k then
t  p.element().getValue()
S.remove(p)
nn–1
// decrement number of entries
return t
// return the removed value
return null
// there is no entry with key equal to k
8
CSED233: Data Structures
by Prof. Bohyung Han, Fall 2014
Performance of a List-Based Map
• Performance:
 The put operation takes 𝑂 1 time since we can insert the new item
at the beginning or at the end of the sequence.
 The get and remove operations take 𝑂(𝑛) time since in the worst
case (the item is not found) we traverse the entire sequence to look
for an item with the given key
• The unsorted list implementation is effective only
 for maps of small size or
 for maps in which puts are the most common operations, while
searches and removals are rarely performed
• e.g., historical record of logins to a workstation
 This is not true in our implementation, either; even put operation is
as slow as get and remove.
9
CSED233: Data Structures
by Prof. Bohyung Han, Fall 2014
Problems and Solution
• Problems
 We have a lot of data to store.
 We desire efficient performance, 𝑂 1 , for insertion, deletion, and
searching.
 Too much (wasted) memory is required if we use an array indexed by
data’s key.
• Solution
 Hashing
10
CSED233: Data Structures
by Prof. Bohyung Han, Fall 2014
Hashing
• Hash function
 A mathematical function ℎ that, given a key 𝑘, provides an index ℎ 𝑘
into hash table (an array), where the item with key 𝑘 can be found.
• Case
 Integer key in the range 1, 32000
 Array indices are 0, 255
 No more than 256 keys are used altogether.
• Compression map
 Trivial hash function ℎ 𝑘 = 𝑘 is not plausible.
 We require a hash function that will scale the key down to the
allowable range 0, 255 .
 Try ℎ 𝑘 = 𝑘 mod 256.
11
CSED233: Data Structures
by Prof. Bohyung Han, Fall 2014
Hash Functions and Hash Tables
• A hash function ℎ maps
 keys of a given type to integers in a fixed interval 0, 𝑁 − 1
 Example:
• ℎ 𝑥 = 𝑥 mod 𝑁
• A hash function for integer keys
• Hash value
 The integer ℎ 𝑥 is called the hash value of key 𝑥.
• A hash table for a given key type consists of
 Hash function ℎ
 Array (called table) of size 𝑁
 When implementing a map with a hash table, the goal is to store item
(key, value) at index 𝑖 = ℎ 𝑘 .
12
CSED233: Data Structures
by Prof. Bohyung Han, Fall 2014
Hash Tables
• Use keys to store and access entries (in constant time)
13
CSED233: Data Structures
by Prof. Bohyung Han, Fall 2014
Example
• Hash table for SSN (Social Security Number)





Storing entries (SSN, Name)
Key: SSN, a nine-digit positive integer
Value: name of a person
Index: last four digits of SSN
Hash function:
ℎ 𝑥 ≡ last four digits of 𝑥
0 
1
2
3 
4
025-612-0001
981-101-0002
451-229-0004
…
= 𝑥 mod 10000
9997 
9998
9999 
 Hash table size: 10,000
14
200-751-9998
CSED233: Data Structures
by Prof. Bohyung Han, Fall 2014
Hash Functions
• Typically specified as the composition of two functions:
 Hash code ℎ1 : key → integer
 Compression function ℎ2 : integer → 0, 𝑁 − 1
• Hash function
 The hash code is applied first.
 The compression function is applied next on the result.
 Example:ℎ 𝑥 = ℎ2 ℎ1 𝑥
• The goal of the hash function
 “Disperse” the keys in an apparently random way
15
CSED233: Data Structures
by Prof. Bohyung Han, Fall 2014
Hash Codes
• Memory address
 We reinterpret the memory address of the key object as an integer
(default hash code of all Java objects)
 Good in general, except for numeric and string keys
• Integer cast
 We reinterpret the bits of the key as an integer
 Suitable for keys of length less than or equal to the number of bits of
the integer type (e.g., byte, short, int and float in Java)
• Component sum
 We partition the bits of the key into components of fixed length (e.g.,
16 or 32 bits) and we sum the components (ignoring overflows).
 Suitable for numeric keys of fixed length greater than or equal to the
number of bits of the integer type (e.g., long and double in Java)
16
CSED233: Data Structures
by Prof. Bohyung Han, Fall 2014
Hash Codes (cont.)
• Polynomial accumulation
 We partition the bits of the key into a sequence of components of
fixed length (e.g., 8, 16 or 32 bits) as 𝑎0 𝑎1 … 𝑎𝑛−1
 We evaluate the polynomial
𝑝 𝑧 = 𝑎0 + 𝑎1 𝑧 + 𝑎2 𝑧 2 + ⋯ + 𝑎𝑛−1 𝑧 𝑛−1
at a fixed value 𝑧, ignoring overflows
 Polynomial 𝑝 𝑧 can be evaluated 𝑂 𝑛 time using Horner’s rule:
𝑝0 𝑧 = 𝑎𝑛−1
𝑝𝑖 𝑧 = 𝑎𝑛−𝑖−1 + 𝑧𝑝𝑖−1 𝑧 ,
𝑖 = 1, 2, … , 𝑛 − 1
𝑝 𝑧 = 𝑝𝑛−1 𝑧
 Especially suitable for strings (e.g., the choice 𝑧 = 33 gives at most 6
collisions on a set of 50,000 English words)
17
CSED233: Data Structures
by Prof. Bohyung Han, Fall 2014
Compression Functions
• Division:
 ℎ2 𝑦 = 𝑦 mod 𝑁
 The size 𝑁 of the hash table is usually chosen to be a prime to
minimize collision.
• Multiply, Add and Divide (MAD):
 ℎ2 𝑦 = 𝑎𝑦 + 𝑏 mod 𝑁
 𝑎 and 𝑏 are nonnegative integers such that 𝑎 mod 𝑁 ≠ 0
 Otherwise, every integer would map to the same value 𝑏.
18
CSED233: Data Structures
by Prof. Bohyung Han, Fall 2014
Collision Handling
• Collisions occur when different elements are mapped to the
same cell
• Separate Chaining
 Let each cell in the table point to a linked list of entries that map there.
 Simple, but requires additional memory outside the table
0 
1
2 
3 
4
025-612-0001
451-229-0004
19
981-101-0004
CSED233: Data Structures
by Prof. Bohyung Han, Fall 2014
Map with Separate Chaining
Algorithm get(k)
return A[h(k)].get(k)
Algorithm put(k,v):
t = A[h(k)].put(k,v)
if t = null then // k is a new key
n=n+1
return t
Algorithm remove(k)
t = A[h(k)].remove(k)
if t ≠ null then
// k was found
n=n-1
return t
20
CSED233: Data Structures
by Prof. Bohyung Han, Fall 2014
Linear Probing
• Open addressing
 The colliding item is placed in a different cell of the table
• Linear probing
 Handles collisions by placing the colliding item in the next (circularly)
available table cell
 Each table cell inspected is referred to as a “probe”
 Colliding items lump together, causing future collisions to cause a
longer sequence of probes
• Example
 ℎ 𝑥 = 𝑥 mod 13
 Insert keys in the following order
18, 41, 22, 44, 59, 32, 31, 73
0 1 2 3 4 5 6 7 8 9 10 11 12
41
18 44 59 32 22 31 73
0 1 2 3 4 5 6 7 8 9 10 11 12
21
CSED233: Data Structures
by Prof. Bohyung Han, Fall 2014
Search with Linear Probing
• get(k) method
 Consider a hash table A that uses
linear probing
 We start at cell ℎ 𝑥
 We probe consecutive locations
until one of the following occurs
• An item with key is found, or
• An empty cell is found, or
• All cells have been unsuccessfully
probed
22
Algorithm get(k)
i  h(k)
p0
repeat
c  A[i]
if c = 
return null
else if c.getKey () = k
return c.getValue()
else
i  (i + 1) mod N
pp+1
until p = N
return null
CSED233: Data Structures
by Prof. Bohyung Han, Fall 2014
Updates with Linear Probing
• If collided item is removed
 The items with the same key value may not be accessible.
 To handle insertions and deletions, we introduce a special object,
called AVAILABLE, which replaces deleted elements.
• remove(k)
 We search for an entry with key k
 If such an entry (key,value) is found, we replace it with the special
item AVAILABLE and we return element value
 Otherwise, we return null.
23
CSED233: Data Structures
by Prof. Bohyung Han, Fall 2014
Updates with Linear Probing
• put(key,value) method
 We throw an exception if the table is full.
 We start at cell ℎ 𝑥 .
 We probe consecutive cells until one of the following occurs
• A cell is found that is either empty or stores AVAILABLE, or
• N cells have been unsuccessfully probed (no free space)
 We store (key,value) in the founded empty or AVAILABLE cell.
24
CSED233: Data Structures
by Prof. Bohyung Han, Fall 2014
Double Hashing
• Double hashing
 Uses a secondary hash function 𝑑 𝑘
 Handles collisions by placing an item in the first available cell of the
series ℎ 𝑘 + 𝑗 ⋅ 𝑑 𝑘 mod 𝑁 for 𝑗 = 0,1, … , 𝑁 − 1.
 The secondary hash function 𝑑 𝑘 cannot have zero values.
 The table size 𝑁 must be a prime to allow probing of all the cells.
• Common choice of compression function for the secondary
hash function:
 𝑑 𝑘 = 𝑞 − 𝑘 mod 𝑞 , where 𝑞 < 𝑁 and 𝑞 is a prime.
 The possible values for 𝑑 𝑘 are 𝑖 = 1,2, … , 𝑞.
25
CSED233: Data Structures
by Prof. Bohyung Han, Fall 2014
Example of Double Hashing
• Consider a hash table storing
integer keys that handles
collision with double hashing
 𝑁 = 13
 ℎ 𝑘 = 𝑘 mod 13
 𝑑 𝑘 = 7 − 𝑘 mod 7
• Example
k
18
41
22
44
59
32
31
73
h (k ) d (k ) Probes
5
2
9
5
7
6
5
8
3
1
6
5
4
3
4
4
5
2
9
5
7
6
5
8
10
9
0
 Insert keys in the following order
18, 41, 22, 44, 59, 32, 31, 73.
0 1 2 3 4 5 6 7 8 9 10 11 12
31
41
18 32 59 73 22 44
0 1 2 3 4 5 6 7 8 9 10 11 12
26
CSED233: Data Structures
by Prof. Bohyung Han, Fall 2014
Performance of Hashing
• The worst case time complexity on hash table
 Insertion: 𝑂 𝑛
 Deletion: 𝑂 𝑛
 The worst case occurs when all the keys inserted into the map collide.
• Our expectation
 The load factor 𝑎 =
𝑛
𝑁
affects the performance of a hash table.
 Assuming that the hash values are like random numbers, it can be
shown that the expected number of probes for an insertion with
1
open addressing is
.
1−𝑎
• The expected running time
 All operations in a hash table: 𝑂 1
 In practice, hashing is very fast provided the load factor is not close to
100%.
27
CSED233: Data Structures
by Prof. Bohyung Han, Fall 2014
28