Download No Slide Title

Data Mining:
Concepts and Techniques
— Slides for Textbook —
— Chapter 6 —
©Jiawei Han and Micheline Kamber
Department of Computer Science
University of Illinois at Urbana-Champaign
www.cs.sfu.ca, www.cs.uiuc.edu/~hanj
May 22, 2017
Data Mining: Concepts and Techniques
1
Chapter 6: Mining Association
Rules in Large Databases




Association rule mining
Mining single-dimensional Boolean association rules
from transactional databases
Mining multilevel association rules from transactional
databases
Mining multidimensional association rules from
transactional databases and data warehouse

From association mining to correlation analysis

Constraint-based association mining

Summary
May 22, 2017
Data Mining: Concepts and Techniques
2
What Is Association Mining?
Association rule mining:
 Finding frequent patterns, associations, correlations, or
causal structures among sets of items or objects in
transaction databases, relational databases, and other
information repositories.
 Applications:
 Basket data analysis, cross-marketing, catalog design,
loss-leader analysis, clustering, classification, etc.
 Examples.
 Rule form: “Body ead [support, confidence]”.
 buys(x, “diapers”)  buys(x, “beers”) [0.5%, 60%]
 major(x, “CS”) ^ takes(x, “DB”) grade(x, “A”) [1%,
75%]

May 22, 2017
Data Mining: Concepts and Techniques
3
Association Rule: Basic Concepts


Given: (1) database of transactions, (2) each transaction is
a list of items (purchased by a customer in a visit)
Find: all rules that correlate the presence of one set of
items with that of another set of items
 E.g., 98% of people who purchase tires and auto
accessories also get automotive services done

Applications




*  Maintenance Agreement (What the store should
do to boost Maintenance Agreement sales)
Home Electronics  * (What other products should
the store stocks up?)
Attached mailing in direct marketing
Detecting “ping-pong”ing of patients, faulty “collisions”
May 22, 2017
Data Mining: Concepts and Techniques
4
Rule Measures: Support and
Confidence
Customer
buys both
Find all the rules X & Y  Z with
minimum confidence and support
 support, s, probability that a
transaction contains {X  Y 
Z}
 confidence, c, conditional
Customer
buys beer
probability that a transaction
having {X  Y} also contains Z
Transaction ID Items Bought Let minimum support 50%, and
minimum confidence 50%,
2000
A,B,C
we have
1000
A,C
 A  C (50%, 66.6%)
4000
A,D
5000
B,E,F
 C  A (50%, 100%)
May 22, 2017
Customer 
buys diaper
Data Mining: Concepts and Techniques
5
Association Rule Mining: A Road Map




Boolean vs. quantitative associations (Based on the types of values
handled)
 buys(x, “SQLServer”) ^ buys(x, “DMBook”) buys(x, “DBMiner”)
[0.2%, 60%]
 age(x, “30..39”) ^ income(x, “42..48K”) buys(x, “PC”) [1%, 75%]
Single dimension vs. multiple dimensional associations (see ex. Above)
Single level vs. multiple-level analysis
 What brands of beers are associated with what brands of diapers?
Various extensions
 Correlation, causality analysis



Association does not necessarily imply correlation or causality
Maxpatterns and closed itemsets
Constraints enforced

May 22, 2017
E.g., small sales (sum < 100) trigger big buys (sum > 1,000)?
Data Mining: Concepts and Techniques
6
Chapter 6: Mining Association
Rules in Large Databases




Association rule mining
Mining single-dimensional Boolean association rules
from transactional databases
Mining multilevel association rules from transactional
databases
Mining multidimensional association rules from
transactional databases and data warehouse

From association mining to correlation analysis

Constraint-based association mining

Summary
May 22, 2017
Data Mining: Concepts and Techniques
7
Mining Association Rules—An Example
Transaction ID
2000
1000
4000
5000
Items Bought
A,B,C
A,C
A,D
B,E,F
Min. support 50%
Min. confidence 50%
Frequent Itemset Support
{A}
75%
{B}
50%
{C}
50%
{A,C}
50%
For rule A  C:
support = support({A C}) = 50%
confidence = support({A C})/support({A}) = 66.6%
The Apriori principle:
Any subset of a frequent itemset must be frequent
May 22, 2017
Data Mining: Concepts and Techniques
8
Mining Frequent Itemsets: the
Key Step

Find the frequent itemsets: the sets of items that have
minimum support

A subset of a frequent itemset must also be a
frequent itemset



i.e., if {AB} is a frequent itemset, both {A} and {B} should
be a frequent itemset
Iteratively find frequent itemsets with cardinality
from 1 to k (k-itemset)
Use the frequent itemsets to generate association
rules.
May 22, 2017
Data Mining: Concepts and Techniques
9
The Apriori Algorithm


Join Step: Ck is generated by joining Lk-1with itself
Prune Step: Any (k-1)-itemset that is not frequent cannot be
a subset of a frequent k-itemset

Pseudo-code:
Ck: Candidate itemset of size k
Lk : frequent itemset of size k
L1 = {frequent items};
for (k = 1; Lk !=; k++) do begin
Ck+1 = candidates generated from Lk;
for each transaction t in database do
increment the count of all candidates in Ck+1
that are contained in t
Lk+1 = candidates in Ck+1 with min_support
end
return k Lk;
May 22, 2017
Data Mining: Concepts and Techniques
10
The Apriori Algorithm — Example
Database D
TID
100
200
300
400
itemset sup.
C1
{1}
2
{2}
3
Scan D
{3}
3
{4}
1
{5}
3
Items
134
235
1235
25
C2 itemset sup
L2 itemset sup
2
2
3
2
{1
{1
{1
{2
{2
{3
C3 itemset
{2 3 5}
Scan D
{1 3}
{2 3}
{2 5}
{3 5}
May 22, 2017
2}
3}
5}
3}
5}
5}
1
2
1
2
3
2
L1 itemset sup.
{1}
{2}
{3}
{5}
2
3
3
3
C2 itemset
{1 2}
Scan D
{1
{1
{2
{2
{3
3}
5}
3}
5}
5}
L3 itemset sup
{2 3 5} 2
Data Mining: Concepts and Techniques
11
How to Generate Candidates?

Suppose the items in Lk-1 are listed in an order

Step 1: self-joining Lk-1
insert into Ck
select p.item1, p.item2, …, p.itemk-1, q.itemk-1
from Lk-1 p, Lk-1 q
where p.item1=q.item1, …, p.itemk-2=q.itemk-2, p.itemk-1 <
q.itemk-1

Step 2: pruning
forall itemsets c in Ck do
forall (k-1)-subsets s of c do
if (s is not in Lk-1) then delete c from Ck
May 22, 2017
Data Mining: Concepts and Techniques
12
How to Count Supports of
Candidates?

Why counting supports of candidates a problem?



The total number of candidates can be very huge
One transaction may contain many candidates
Method:

Candidate itemsets are stored in a hash-tree

Leaf node of hash-tree contains a list of itemsets
and counts


Interior node contains a hash table
Subset function: finds all the candidates contained in
a transaction
May 22, 2017
Data Mining: Concepts and Techniques
13
Example of Generating Candidates

L3={abc, abd, acd, ace, bcd}

Self-joining: L3*L3


abcd from abc and abd

acde from acd and ace
Pruning:


acde is removed because ade is not in L3
C4={abcd}
May 22, 2017
Data Mining: Concepts and Techniques
14
Methods to Improve Apriori’s Efficiency

Hash-based itemset counting: A k-itemset whose corresponding
hashing bucket count is below the threshold cannot be frequent

Transaction reduction: A transaction that does not contain any
frequent k-itemset is useless in subsequent scans

Partitioning: Any itemset that is potentially frequent in DB must be
frequent in at least one of the partitions of DB

Sampling: mining on a subset of given data, lower support
threshold + a method to determine the completeness

Dynamic itemset counting: add new candidate itemsets only when
all of their subsets are estimated to be frequent
May 22, 2017
Data Mining: Concepts and Techniques
15
Is Apriori Fast Enough? — Performance
Bottlenecks

The core of the Apriori algorithm:



Use frequent (k – 1)-itemsets to generate candidate frequent kitemsets
Use database scan and pattern matching to collect counts for the
candidate itemsets
The bottleneck of Apriori: candidate generation
 Huge candidate sets:



104 frequent 1-itemset will generate 107 candidate 2-itemsets
To discover a frequent pattern of size 100, e.g., {a1, a2, …,
a100}, one needs to generate 2100  1030 candidates.
Multiple scans of database:

May 22, 2017
Needs (n +1 ) scans, n is the length of the longest pattern
Data Mining: Concepts and Techniques
16
Mining Frequent Patterns Without
Candidate Generation

Compress a large database into a compact, FrequentPattern tree (FP-tree) structure



highly condensed, but complete for frequent pattern
mining
avoid costly database scans
Develop an efficient, FP-tree-based frequent pattern
mining method


May 22, 2017
A divide-and-conquer methodology: decompose mining
tasks into smaller ones
Avoid candidate generation: sub-database test only!
Data Mining: Concepts and Techniques
17
Construct FP-tree from a
Transaction DB
TID
100
200
300
400
500
Items bought
(ordered) frequent items
{f, a, c, d, g, i, m, p}
{f, c, a, m, p}
{a, b, c, f, l, m, o}
{f, c, a, b, m}
{b, f, h, j, o}
{f, b}
{b, c, k, s, p}
{c, b, p}
{a, f, c, e, l, p, m, n}
{f, c, a, m, p}
Steps:
{}
Header Table
1. Scan DB once, find frequent
1-itemset (single item
pattern)
2. Order frequent items in
frequency descending order
3. Scan DB again, construct
FP-tree
May 22, 2017
min_support = 0.5
Item frequency head
f
4
c
4
a
3
b
3
m
3
p
3
Data Mining: Concepts and Techniques
f:4
c:3
c:1
b:1
a:3
b:1
p:1
m:2
b:1
p:2
m:1
18
Benefits of the FP-tree Structure


Completeness:
 never breaks a long pattern of any transaction
 preserves complete information for frequent pattern
mining
Compactness
 reduce irrelevant information—infrequent items are gone
 frequency descending ordering: more frequent items are
more likely to be shared
 never be larger than the original database (if not count
node-links and counts)
 Example: For Connect-4 DB, compression ratio could be
over 100
May 22, 2017
Data Mining: Concepts and Techniques
19
Mining Frequent Patterns Using FP-tree


General idea (divide-and-conquer)
 Recursively grow frequent pattern path using the FPtree
Method
 For each item, construct its conditional pattern-base,
and then its conditional FP-tree
 Repeat the process on each newly created conditional
FP-tree
 Until the resulting FP-tree is empty, or it contains only
one path (single path will generate all the combinations of its
sub-paths, each of which is a frequent pattern)
May 22, 2017
Data Mining: Concepts and Techniques
20
Major Steps to Mine FP-tree
1)
Construct conditional pattern base for each node in the
FP-tree
2)
Construct conditional FP-tree from each conditional
pattern-base
3)
Recursively mine conditional FP-trees and grow
frequent patterns obtained so far

May 22, 2017
If the conditional FP-tree contains a single path,
simply enumerate all the patterns
Data Mining: Concepts and Techniques
21
Step 1: From FP-tree to Conditional
Pattern Base



Starting at the frequent header table in the FP-tree
Traverse the FP-tree by following the link of each frequent item
Accumulate all of transformed prefix paths of that item to form a
conditional pattern base
Header Table
Item frequency head
f
4
c
4
a
3
b
3
m
3
p
3
May 22, 2017
{}
Conditional pattern bases
f:4
c:3
c:1
b:1
a:3
b:1
p:1
item
cond. pattern base
c
f:3
a
fc:3
b
fca:1, f:1, c:1
m:2
b:1
m
fca:2, fcab:1
p:2
m:1
p
fcam:2, cb:1
Data Mining: Concepts and Techniques
22
Properties of FP-tree for Conditional
Pattern Base Construction

Node-link property


For any frequent item ai, all the possible frequent
patterns that contain ai can be obtained by following
ai's node-links, starting from ai's head in the FP-tree
header
Prefix path property

May 22, 2017
To calculate the frequent patterns for a node ai in a
path P, only the prefix sub-path of ai in P need to be
accumulated, and its frequency count should carry the
same count as node ai.
Data Mining: Concepts and Techniques
23
Step 2: Construct Conditional FP-tree

For each pattern-base
 Accumulate the count for each item in the base
 Construct the FP-tree for the frequent items of the
pattern base
Header Table
Item frequency head
f
4
c
4
a
3
b
3
m
3
p
3
{}
f:4
c:3
c:1
b:1
a:3
b:1
p:1
m-conditional pattern
base:
fca:2, fcab:1
{}

f:3 
m:2
b:1
c:3
p:2
m:1
a:3
All frequent patterns
concerning m
m,
fm, cm, am,
fcm, fam, cam,
fcam
m-conditional FP-tree
May 22, 2017
Data Mining: Concepts and Techniques
24
Mining Frequent Patterns by Creating
Conditional Pattern-Bases
Item
Conditional pattern-base
Conditional FP-tree
p
{(fcam:2), (cb:1)}
{(c:3)}|p
m
{(fca:2), (fcab:1)}
{(f:3, c:3, a:3)}|m
b
{(fca:1), (f:1), (c:1)}
Empty
a
{(fc:3)}
{(f:3, c:3)}|a
c
{(f:3)}
{(f:3)}|c
f
Empty
Empty
May 22, 2017
Data Mining: Concepts and Techniques
25
Step 3: Recursively mine the
conditional FP-tree
{}
{}
Cond. pattern base of “am”: (fc:3)
f:3
c:3
f:3
am-conditional FP-tree
c:3
{}
Cond. pattern base of “cm”: (f:3)
a:3
f:3
m-conditional FP-tree
cm-conditional FP-tree
{}
Cond. pattern base of “cam”: (f:3)
f:3
cam-conditional FP-tree
May 22, 2017
Data Mining: Concepts and Techniques
26
A Special Case: Single Prefix Path in FP-tree


{}
a1:n1
a2:n2
Suppose a (conditional) FP-tree T has a shared single prefixpath P
Mining can be decomposed into two parts


Reduction of the single prefix path into one node
Concatenation of the mining results of the two
parts
a3:n3
b1:m1
C2:k2
May 22, 2017
r1
{}
C1:k1
C3:k3

r1
=
a1:n1
a2:n2
+
a3:n3
Data Mining: Concepts and Techniques
b1:m1
C2:k2
C1:k1
C3:k3
28
Principles of Frequent Pattern
Growth

Pattern growth property


Let  be a frequent itemset in DB, B be 's
conditional pattern base, and  be an itemset in B.
Then    is a frequent itemset in DB iff  is
frequent in B.
“abcdef ” is a frequent pattern, if and only if


May 22, 2017
“abcde ” is a frequent pattern, and
“f ” is frequent in the set of transactions containing
“abcde ”
Data Mining: Concepts and Techniques
29
Why Is Frequent Pattern Growth
Fast?

Our performance study shows

FP-growth is an order of magnitude faster than
Apriori, and is also faster than tree-projection

Reasoning
May 22, 2017

No candidate generation, no candidate test

Use compact data structure

Eliminate repeated database scan

Basic operation is counting and FP-tree building
Data Mining: Concepts and Techniques
30
FP-growth vs. Apriori: Scalability With
the Support Threshold
Data set T25I20D10K
100
D1 FP-grow th runtime
90
D1 Apriori runtime
80
Run time(sec.)
70
60
50
40
30
20
10
0
0
May 22, 2017
0.5
1
1.5
2
Support threshold(%)
Data Mining: Concepts and Techniques
2.5
3
31
FP-growth vs. Tree-Projection: Scalability
with Support Threshold
Data set T25I20D100K
140
D2 FP-growth
Runtime (sec.)
120
D2 TreeProjection
100
80
60
40
20
0
0
May 22, 2017
0.5
1
Support threshold (%)
Data Mining: Concepts and Techniques
1.5
2
32
Presentation of Association Rules
(Table Form )
May 22, 2017
Data Mining: Concepts and Techniques
33
Visualization of Association Rule Using Plane Graph
May 22, 2017
Data Mining: Concepts and Techniques
34
Visualization of Association Rule Using Rule Graph
May 22, 2017
Data Mining: Concepts and Techniques
35
Iceberg Queries


Icerberg query: Compute aggregates over one or a set of
attributes only for those whose aggregate values is above
certain threshold
Example:
select P.custID, P.itemID, sum(P.qty)
from purchase P
group by P.custID, P.itemID
having sum(P.qty) >= 10

Compute iceberg queries efficiently by Apriori:
 First compute lower dimensions
 Then compute higher dimensions only when all the
lower ones are above the threshold
May 22, 2017
Data Mining: Concepts and Techniques
36
Chapter 6: Mining Association
Rules in Large Databases




Association rule mining
Mining single-dimensional Boolean association rules
from transactional databases
Mining multilevel association rules from transactional
databases
Mining multidimensional association rules from
transactional databases and data warehouse

From association mining to correlation analysis

Constraint-based association mining

Summary
May 22, 2017
Data Mining: Concepts and Techniques
37
Multiple-Level Association Rules
Food





Items often form hierarchy.
Items at the lower level are
expected to have lower
support.
Rules regarding itemsets at
appropriate levels could be
quite useful.
Transaction database can be
encoded based on
dimensions and levels
We can explore shared multilevel mining
May 22, 2017
bread
milk
skim
Fraser
TID
T1
T2
T3
T4
T5
2%
wheat
white
Sunset
Items
{111, 121, 211, 221}
{111, 211, 222, 323}
{112, 122, 221, 411}
{111, 121}
{111, 122, 211, 221, 413}
Data Mining: Concepts and Techniques
38
Mining Multi-Level Associations

A top_down, progressive deepening approach:
 First find high-level strong rules:
milk  bread [20%, 60%].

Then find their lower-level “weaker” rules:
2% milk  wheat bread [6%, 50%].

Variations at mining multiple-level association rules.
 Level-crossed association rules:
2% milk

Association rules with multiple, alternative
hierarchies:
2% milk
May 22, 2017
 Wonder wheat bread
 Wonder bread
Data Mining: Concepts and Techniques
39
Multi-level Association: Uniform
Support vs. Reduced Support

Uniform Support: the same minimum support for all levels
 + One minimum support threshold.
No need to examine itemsets
containing any item whose ancestors do not have minimum
support.

– Lower level items do not occur as frequently. If support threshold
too high  miss low level associations
 too low  generate too many high level associations
Reduced Support: reduced minimum support at lower levels
 There are 4 search strategies:






May 22, 2017
Level-by-level independent
Level-cross filtering by k-itemset
Level-cross filtering by single item
Controlled level-cross filtering by single item
Data Mining: Concepts and Techniques
40
Uniform Support
Multi-level mining with uniform support
Level 1
min_sup = 5%
Level 2
min_sup = 5%
Milk
[support = 10%]
2% Milk
Skim Milk
[support = 6%]
[support = 4%]
Back
May 22, 2017
Data Mining: Concepts and Techniques
41
Reduced Support
Multi-level mining with reduced support
Level 1
min_sup = 5%
Level 2
min_sup = 3%
Milk
[support = 10%]
2% Milk
Skim Milk
[support = 6%]
[support = 4%]
Back
May 22, 2017
Data Mining: Concepts and Techniques
42
Multi-level Association: Redundancy
Filtering




Some rules may be redundant due to “ancestor”
relationships between items.
Example
 milk  wheat bread
[support = 8%, confidence = 70%]
 2% milk  wheat bread [support = 2%, confidence = 72%]
We say the first rule is an ancestor of the second rule.
A rule is redundant if its support is close to the “expected”
value, based on the rule’s ancestor.
May 22, 2017
Data Mining: Concepts and Techniques
43
Multi-Level Mining: Progressive
Deepening

A top-down, progressive deepening approach:
 First mine high-level frequent items:
milk (15%), bread (10%)

Then mine their lower-level “weaker” frequent
itemsets:
2% milk (5%), wheat bread (4%)

Different min_support threshold across multi-levels
lead to different algorithms:
 If adopting the same min_support across multilevels
then toss t if any of t’s ancestors is infrequent.

If adopting reduced min_support at lower levels
then examine only those descendents whose ancestor’s
support is frequent/non-negligible.
May 22, 2017
Data Mining: Concepts and Techniques
44
Progressive Refinement of Data
Mining Quality

Why progressive refinement?



Trade speed with quality: step-by-step refinement.
Superset coverage property:


Mining operator can be expensive or cheap, fine or
rough
Preserve all the positive answers—allow a positive false
test but not a false negative test.
Two- or multi-step mining:


First apply rough/cheap operator (superset coverage)
Then apply expensive algorithm on a substantially
reduced candidate set (Koperski & Han, SSD’95).
May 22, 2017
Data Mining: Concepts and Techniques
45
Progressive Refinement Mining of
Spatial Association Rules


Hierarchy of spatial relationship:
 “g_close_to”: near_by, touch, intersect, contain, etc.
 First search for rough relationship and then refine it.
Two-step mining of spatial association:
 Step 1: rough spatial computation (as a filter)


Step2: Detailed spatial algorithm (as refinement)

May 22, 2017
Using MBR or R-tree for rough estimation.
Apply only to those objects which have passed the rough
spatial association test (no less than min_support)
Data Mining: Concepts and Techniques
46
Chapter 6: Mining Association
Rules in Large Databases




Association rule mining
Mining single-dimensional Boolean association rules
from transactional databases
Mining multilevel association rules from transactional
databases
Mining multidimensional association rules from
transactional databases and data warehouse

From association mining to correlation analysis

Constraint-based association mining

Summary
May 22, 2017
Data Mining: Concepts and Techniques
47
Multi-Dimensional Association:
Concepts

Single-dimensional rules:
buys(X, “milk”)  buys(X, “bread”)

Multi-dimensional rules:  2 dimensions or predicates
 Inter-dimension association rules (no repeated
predicates)
age(X,”19-25”)  occupation(X,“student”)  buys(X,“coke”)

hybrid-dimension association rules (repeated predicates)
age(X,”19-25”)  buys(X, “popcorn”)  buys(X, “coke”)


Categorical Attributes
 finite number of possible values, no ordering among
values
Quantitative Attributes
 numeric, implicit ordering among values
May 22, 2017
Data Mining: Concepts and Techniques
48
Techniques for Mining MD
Associations
Search for frequent k-predicate set:
 Example: {age, occupation, buys} is a 3-predicate
set.
 Techniques can be categorized by how age are
treated.
1. Using static discretization of quantitative attributes
 Quantitative attributes are statically discretized by
using predefined concept hierarchies.
2. Quantitative association rules
 Quantitative attributes are dynamically discretized
into “bins”based on the distribution of the data.
3. Distance-based association rules
 This is a dynamic discretization process that
considers the distance between data points.

May 22, 2017
Data Mining: Concepts and Techniques
49
Static Discretization of Quantitative
Attributes

Discretized prior to mining using concept hierarchy.

Numeric values are replaced by ranges.

In relational database, finding all frequent k-predicate sets
will require k or k+1 table scans.

Data cube is well suited for mining.

The cells of an n-dimensional
()
(age)
(income)
(buys)
cuboid correspond to the
predicate sets.

(age, income)
Mining from data cubes
can be much faster.
May 22, 2017
(age,buys) (income,buys)
(age,income,buys)
Data Mining: Concepts and Techniques
50
Quantitative Association Rules




Numeric attributes are dynamically discretized
 Such that the confidence or compactness of the rules
mined is maximized.
2-D quantitative association rules: Aquan1  Aquan2  Acat
Cluster “adjacent”
association rules
to form general
rules using a 2-D
grid.
Example:
age(X,”30-34”)  income(X,”24K 48K”)
 buys(X,”high resolution TV”)
May 22, 2017
Data Mining: Concepts and Techniques
51
ARCS (Association Rule Clustering
System)
How does ARCS work?
1. Binning
2. Find frequent
predicateset
3. Clustering
4. Optimize
May 22, 2017
Data Mining: Concepts and Techniques
52
Limitations of ARCS

Only quantitative attributes on LHS of rules.

Only 2 attributes on LHS. (2D limitation)

An alternative to ARCS

Non-grid-based

equi-depth binning


clustering based on a measure of partial
completeness.
“Mining Quantitative Association Rules in
Large Relational Tables” by R. Srikant and R.
Agrawal.
May 22, 2017
Data Mining: Concepts and Techniques
53
Mining Distance-based Association Rules


Binning methods do not capture the semantics of interval
data
Price($)
Equi-width
(width $10)
Equi-depth
(depth 2)
Distancebased
7
20
22
50
51
53
[0,10]
[11,20]
[21,30]
[31,40]
[41,50]
[51,60]
[7,20]
[22,50]
[51,53]
[7,7]
[20,22]
[50,53]
Distance-based partitioning, more meaningful discretization
considering:
 density/number of points in an interval
 “closeness” of points in an interval
May 22, 2017
Data Mining: Concepts and Techniques
54
Chapter 6: Mining Association
Rules in Large Databases




Association rule mining
Mining single-dimensional Boolean association rules
from transactional databases
Mining multilevel association rules from transactional
databases
Mining multidimensional association rules from
transactional databases and data warehouse

From association mining to correlation analysis

Constraint-based association mining

Summary
May 22, 2017
Data Mining: Concepts and Techniques
57
Interestingness Measurements

Objective measures
Two popular measurements:
 support; and


May 22, 2017
confidence
Subjective measures (Silberschatz & Tuzhilin,
KDD95)
A rule (pattern) is interesting if
 it is unexpected (surprising to the user);
and/or
 actionable (the user can do something with it)
Data Mining: Concepts and Techniques
58
Criticism to Support and Confidence

Example 1: (Aggarwal & Yu, PODS98)
 Among 5000 students
 3000 play basketball
 3750 eat cereal
 2000 both play basket ball and eat cereal
 play basketball  eat cereal [40%, 66.7%] is misleading
because the overall percentage of students eating cereal is 75%
which is higher than 66.7%.
 play basketball  not eat cereal [20%, 33.3%] is far more
accurate, although with lower support and confidence
basketball not basketball sum(row)
cereal
2000
1750
3750
not cereal
1000
250
1250
sum(col.)
3000
2000
5000
May 22, 2017
Data Mining: Concepts and Techniques
59
Criticism to Support and Confidence
(Cont.)


Example 2:
 X and Y: positively correlated,
 X and Z, negatively related
 support and confidence of
X=>Z dominates
We need a measure of dependent
or correlated events
corrA, B

P( A B)

P( A) P( B)
X 1 1 1 1 0 0 0 0
Y 1 1 0 0 0 0 0 0
Z 0 1 1 1 1 1 1 1
Rule Support Confidence
X=>Y 25%
50%
X=>Z 37.50%
75%
P(B|A)/P(B) is also called the lift
of rule A => B
May 22, 2017
Data Mining: Concepts and Techniques
60
Other Interestingness Measures: Interest

Interest (correlation, lift)
P( A  B)
P( A) P( B)

taking both P(A) and P(B) in consideration

P(A^B)=P(B)*P(A), if A and B are independent events

A and B negatively correlated, if the value is less than 1;
otherwise A and B positively correlated
X 1 1 1 1 0 0 0 0
Y 1 1 0 0 0 0 0 0
Z 0 1 1 1 1 1 1 1
May 22, 2017
Itemset
Support
Interest
X,Y
X,Z
Y,Z
25%
37.50%
12.50%
2
0.9
0.57
Data Mining: Concepts and Techniques
61
Chapter 6: Mining Association
Rules in Large Databases




Association rule mining
Mining single-dimensional Boolean association rules
from transactional databases
Mining multilevel association rules from transactional
databases
Mining multidimensional association rules from
transactional databases and data warehouse

From association mining to correlation analysis

Constraint-based association mining

Summary
May 22, 2017
Data Mining: Concepts and Techniques
62
Constraint-Based Mining


Interactive, exploratory mining giga-bytes of data?
 Could it be real? — Making good use of constraints!
What kinds of constraints can be used in mining?
 Knowledge type constraint: classification, association,
etc.
 Data constraint: SQL-like queries


Dimension/level constraints:


small sales (price < $10) triggers big sales (sum > $200).
Interestingness constraints:

May 22, 2017
in relevance to region, price, brand, customer category.
Rule constraints


Find product pairs sold together in Vancouver in Dec.’98.
strong rules (min_support  3%, min_confidence  60%).
Data Mining: Concepts and Techniques
63
Rule Constraints in Association Mining

Two kind of rule constraints:
 Rule form constraints: meta-rule guided mining.


Rule (content) constraint: constraint-based query
optimization (Ng, et al., SIGMOD’98).


P(x, y) ^ Q(x, w) takes(x, “database systems”).
sum(LHS) < 100 ^ min(LHS) > 20 ^ count(LHS) > 3 ^ sum(RHS) >
1000
1-variable vs. 2-variable constraints (Lakshmanan, et al.
SIGMOD’99):


1-var: A constraint confining only one side (L/R) of the
rule, e.g., as shown above.
2-var: A constraint confining both sides (L and R).

May 22, 2017
sum(LHS) < min(RHS) ^ max(RHS) < 5* sum(LHS)
Data Mining: Concepts and Techniques
64
Constrain-Based Association Query

Database: (1) trans (TID, Itemset ), (2) itemInfo (Item, Type, Price)
A constrained asso. query (CAQ) is in the form of {(S1, S2 )|C },

where C is a set of constraints on S1, S2 including frequency
constraint
A classification of (single-variable) constraints:




Class constraint: S  A. e.g. S  Item
Domain constraint:




S v,   { , , , , ,  }. e.g. S.Price < 100
v S,  is  or . e.g. snacks  S.Type
V S, or S V,   { , , , ,  }
 e.g. {snacks, sodas }  S.Type
Aggregation constraint: agg(S)  v, where agg is in {min,
max, sum, count, avg}, and   { , , , , ,  }.

May 22, 2017
e.g. count(S1.Type)  1 , avg(S2.Price)  100
Data Mining: Concepts and Techniques
65
Constrained Association Query
Optimization Problem




Basic idea: Reduce mining cost by pruning with constraints
Main challenge: results must be sound and complete
 sound: It only finds frequent sets that satisfy the given
constraints C
 complete: All frequent sets satisfy the given constraints
C are found
Question: when and where is the safe point to throw out
invalid data items, or stop checking constraints?
Approach:
 analyzing the properties of constraints
 push them as deeply as possible inside the frequent set
computation.
May 22, 2017
Data Mining: Concepts and Techniques
66
Anti-monotone and Monotone
Constraints

A constraint Ca is anti-monotone iff. for any
pattern S not satisfying Ca, none of the super-
patterns of S can satisfy Ca

A constraint Cm is monotone iff. for any pattern
S satisfying Cm, every super-pattern of S also
satisfies it
May 22, 2017
Data Mining: Concepts and Techniques
67
Property of Constraints:
Anti-Monotone

Anti-monotonicity: If a set S violates the constraint, any
superset of S violates the constraint.


Examples:

sum(S.Price)  v is anti-monotone

sum(S.Price)  v is not anti-monotone

sum(S.Price) = v is partly anti-monotone
Application:

Push “sum(S.price)  1000” deeply into iterative
frequent set computation.
May 22, 2017
Data Mining: Concepts and Techniques
68
Characterization of
Anti-Monotonicity Constraints
S  v,   { , ,  }
vS
SV
SV
SV
min(S)  v
min(S)  v
min(S)  v
max(S)  v
max(S)  v
max(S)  v
count(S)  v
count(S)  v
count(S)  v
sum(S)  v
sum(S)  v
sum(S)  v
avg(S)  v,   { , ,  }
(frequent constraint)
May 22, 2017
yes
no
no
yes
partly
no
yes
partly
yes
no
partly
yes
no
partly
yes
no
partly
convertible
(yes)
Data Mining: Concepts and Techniques
69
Succinct Constraint



A subset of item Is is a succinct set, if it can be
expressed as p(I) for some selection predicate
p, where  is a selection operator
SP2I is a succinct power set, if there is a fixed
number of succinct set I1, …, Ik I, s.t. SP can
be expressed in terms of the strict power sets
of I1, …, Ik using union and minus
A constraint Cs is succinct provided SATCs(I) is a
succinct power set
May 22, 2017
Data Mining: Concepts and Techniques
70
Property of Constraints: Succinctness



Succinctness:
 For any set S1 and S2 satisfying C, S1  S2 satisfies C
 Given A1 is the sets of size 1 satisfying C, then any set S
satisfying C are based on A1 , i.e., it contains a subset
belongs to A1 ,
Example :
 sum(S.Price )  v is not succinct
 min(S.Price )  v is succinct
Optimization:
 If C is succinct, then C is pre-counting prunable. The
satisfaction of the constraint alone is not affected by the
iterative support counting.
May 22, 2017
Data Mining: Concepts and Techniques
71
Characterization of Constraints by
Succinctness
S  v,   { , ,  }
vS
S V
SV
SV
min(S)  v
min(S)  v
min(S)  v
max(S)  v
max(S)  v
max(S)  v
count(S)  v
count(S)  v
count(S)  v
sum(S)  v
sum(S)  v
sum(S)  v
avg(S)  v,   { , ,  }
(frequent constraint)
May 22, 2017
Yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes
weakly
weakly
weakly
no
no
no
no
(no)
Data Mining: Concepts and Techniques
72
The Apriori Algorithm — Example
Database D
TID
100
200
300
400
itemset sup.
C1
{1}
2
{2}
3
Scan D
{3}
3
{4}
1
{5}
3
Items
134
235
1235
25
C2 itemset sup
L2 itemset sup
2
2
3
2
{1
{1
{1
{2
{2
{3
C3 itemset
{2 3 5}
Scan D
{1 3}
{2 3}
{2 5}
{3 5}
May 22, 2017
2}
3}
5}
3}
5}
5}
1
2
1
2
3
2
L1 itemset sup.
{1}
{2}
{3}
{5}
2
3
3
3
C2 itemset
{1 2}
Scan D
{1
{1
{2
{2
{3
3}
5}
3}
5}
5}
L3 itemset sup
{2 3 5} 2
Data Mining: Concepts and Techniques
73
Naïve Algorithm: Apriori + Constraint
Database D
TID
100
200
300
400
itemset sup.
C1
{1}
2
{2}
3
Scan D
{3}
3
{4}
1
{5}
3
Items
134
235
1235
25
C2 itemset sup
L2 itemset sup
2
2
3
2
{1
{1
{1
{2
{2
{3
C3 itemset
{2 3 5}
Scan D
{1 3}
{2 3}
{2 5}
{3 5}
May 22, 2017
2}
3}
5}
3}
5}
5}
1
2
1
2
3
2
L1 itemset sup.
{1}
{2}
{3}
{5}
2
3
3
3
C2 itemset
{1 2}
Scan D
L3 itemset sup
{2 3 5} 2
Data Mining: Concepts and Techniques
{1
{1
{2
{2
{3
3}
5}
3}
5}
5}
Constraint:
Sum{S.price < 5}
74
The Constrained Apriori Algorithm: Push
an Anti-monotone Constraint Deep
Database D
TID
100
200
300
400
itemset sup.
C1
{1}
2
{2}
3
Scan D
{3}
3
{4}
1
{5}
3
Items
134
235
1235
25
C2 itemset sup
L2 itemset sup
2
2
3
2
{1
{1
{1
{2
{2
{3
C3 itemset
{2 3 5}
Scan D
{1 3}
{2 3}
{2 5}
{3 5}
May 22, 2017
2}
3}
5}
3}
5}
5}
1
2
1
2
3
2
L1 itemset sup.
{1}
{2}
{3}
{5}
2
3
3
3
C2 itemset
{1 2}
Scan D
L3 itemset sup
{2 3 5} 2
Data Mining: Concepts and Techniques
{1
{1
{2
{2
{3
3}
5}
3}
5}
5}
Constraint:
Sum{S.price < 5}
75
The Constrained Apriori Algorithm:
Push a Succinct Constraint Deep
Database D
TID
100
200
300
400
itemset sup.
C1
{1}
2
{2}
3
Scan D
{3}
3
{4}
1
{5}
3
Items
134
235
1235
25
C2 itemset sup
L2 itemset sup
2
2
3
2
{1
{1
{1
{2
{2
{3
C3 itemset
{2 3 5}
Scan D
{1 3}
{2 3}
{2 5}
{3 5}
May 22, 2017
2}
3}
5}
3}
5}
5}
1
2
1
2
3
2
L1 itemset sup.
{1}
{2}
{3}
{5}
2
3
3
3
C2 itemset
{1 2}
Scan D
L3 itemset sup
{2 3 5} 2
Data Mining: Concepts and Techniques
{1
{1
{2
{2
{3
3}
5}
3}
5}
5}
Constraint:
min{S.price <= 1 }
76
Convertible Constraint


Difficult constraint: avg(S.price) <= $100
Make it anti-monotone or monotone [Pei et al.
ICDE 2001]:



May 22, 2017
Suppose all items in patterns are listed in a total
order R
A constraint C is convertible anti-monotone iff a
pattern S satisfying the constraint implies that each
suffix of S w.r.t. R also satisfies C
A constraint C is convertible monotone iff a pattern S
satisfying the constraint implies that each pattern of
which S is a suffix w.r.t. R also satisfies C
Data Mining: Concepts and Techniques
77
Example of Convertible Constraints:
Avg(S)  V


Let R be the value descending order over the
set of items
 E.g. I={9, 8, 6, 4, 3, 1}
Avg(S)  v is convertible anti-monotone w.r.t. R
 Avg(S)  7
 {9, 8, 6} passes, {9, 8, 6, 4} fails
 No need to check {9, 8, 6, 4, 3}
May 22, 2017
Data Mining: Concepts and Techniques
78
Strongly Convertible Constraints


Some constraints can be made anti-monotone
or monotone, by using different R.
 Avg(S)  v is monotone w.r.t. value
ascending order
 Monotone constraints can reduce overhead
of checking constraints
Which order to use?
 Decide by the selectivity of the constraint

May 22, 2017
Avg(S)  $500 vs. avg(S)  $2
Data Mining: Concepts and Techniques
79
What Constraints Are Convertible?
Constraint
Convertible
anti-monotone
Convertible
monotone
Strongly
convertible
avg(S)  ,  v
Yes
Yes
Yes
median(S)  ,  v
Yes
Yes
Yes
sum(S)  v (items could be of any
value, v  0)
Yes
No
No
sum(S)  v (items could be of any
value, v  0)
No
Yes
No
sum(S)  v (items could be of any
value, v  0)
No
Yes
No
sum(S)  v (items could be of any
value, v  0)
Yes
No
No
……
May 22, 2017
Data Mining: Concepts and Techniques
80
Classification of Constraints
Monotone
Antimonotone
Succinct
Strongly
convertible
Convertible
anti-monotone
Convertible
monotone
Inconvertible
May 22, 2017
Data Mining: Concepts and Techniques
81
Discussion—Handling Multiple
Constraints



Different constraints may require different or even
conflicting item-ordering
If there exists an order R s.t. both C1 and C2 are
convertible w.r.t. R, then there is no conflict between
the two convertible constraints
If there exists conflict on order of items


May 22, 2017
Try to satisfy one constraint first
Then using the order for the other constraint to
mine frequent itemsets in the corresponding
projected database
Data Mining: Concepts and Techniques
82
Chapter 6: Mining Association
Rules in Large Databases




Association rule mining
Mining single-dimensional Boolean association rules
from transactional databases
Mining multilevel association rules from transactional
databases
Mining multidimensional association rules from
transactional databases and data warehouse

From association mining to correlation analysis

Constraint-based association mining

Mining Sequential Patterns: A Pattern-Growth Approach
May 22, 2017
Data Mining: Concepts and Techniques
83
Sequence Databases and Sequential
Pattern Analysis

Transaction databases, time-series databases vs. sequence
databases

Frequent patterns vs. (frequent) sequential patterns

Applications of sequential pattern mining

Customer shopping sequences:


First buy computer, then CD-ROM, and then digital camera,
within 3 months.
Medical treatment, natural disasters (e.g., earthquakes),
science & engineering processes, stocks and markets, etc.

Telephone calling patterns, Weblog click streams

DNA sequences and gene structures
May 22, 2017
Data Mining: Concepts and Techniques
84
What Is Sequential Pattern Mining?

Given a set of sequences, find the complete set
of frequent subsequences
A
sequence
: < (ef) (ab) (df) c b >
A sequence database
SID
sequence
10
<a(abc)(ac)d(cf)>
20
<(ad)c(bc)(ae)>
30
<(ef)(ab)(df)cb>
40
<eg(af)cbc>
An element may contain a set of items.
Items within an element are unordered
and we list them alphabetically.
<a(bc)dc> is a subsequence
of <a(abc)(ac)d(cf)>
Given support threshold min_sup =2, <(ab)c> is a
sequential pattern
May 22, 2017
Data Mining: Concepts and Techniques
85
Challenges on Sequential Pattern
Mining


A huge number of possible sequential patterns are
hidden in databases
A mining algorithm should



find the complete set of patterns, when possible,
satisfying the minimum support (frequency) threshold
be highly efficient, scalable, involving only a small
number of database scans
be able to incorporate various kinds of user-specific
constraints
May 22, 2017
Data Mining: Concepts and Techniques
86
Previous Studies on Sequential Pattern
Mining




Concept introduction and an initial Apriori-like algorithm
 R. Agrawal & R. Srikant. “Mining sequential patterns,” ICDE’95
GSP—An Apriori-based, influential mining method (developed at IBM
Almaden)
 R. Srikant & R. Agrawal. “Mining sequential patterns:
Generalizations and performance improvements,” EDBT’96
From sequential patterns to episodes (Apriori-like + constraints)
 H. Mannila, H. Toivonen & A.I. Verkamo. “Discovery of frequent
episodes in event sequences,” Data Mining and Knowledge
Discovery, 1997
Mining sequential patterns with constraints

Rastogi, et al. “SPIRIT: Mining sequential patterns with regular
expression constraints,” VLDB’99
May 22, 2017
Data Mining: Concepts and Techniques
87
A Basic Property of Sequential Patterns:
Apriori

A basic property: Apriori (Agrawal & Sirkant’94)
 If a sequence S is not frequent
 Then none of the super-sequences of S is frequent
 E.g, <hb> is infrequent  so do <hab> and <(ah)b>
Seq. ID
Sequence
10
<(bd)cb(ac)>
20
<(bf)(ce)b(fg)>
30
<(ah)(bf)abf>
40
<(be)(ce)d>
50
<a(bd)bcb(ade)>
May 22, 2017
Given support threshold
min_sup =2
Data Mining: Concepts and Techniques
88
GSP—A Generalized Sequential Pattern
Mining Algorithm

GSP (Generalized Sequential Pattern) mining algorithm


Outline of the method




proposed by Agrawal and Srikant, EDBT’96
Initially, every item in DB is a candidate of length-1
for each level (i.e., sequences of length-k) do
 scan database to collect support count for each
candidate sequence
 generate candidate length-(k+1) sequences from
length-k frequent sequences using Apriori
repeat until no frequent sequence or no candidate
can be found
Major strength: Candidate pruning by Apriori
May 22, 2017
Data Mining: Concepts and Techniques
89
Finding Length-1 Sequential Patterns


Examine GSP using an example
Initial candidates: all singleton sequences


<a>, <b>, <c>, <d>, <e>, <f>,
<g>, <h>
Scan database once, count support for
candidates
min_sup =2
May 22, 2017
Cand
Sup
<a>
3
<b>
5
<c>
4
<d>
3
<e>
3
<f>
2
Seq. ID
Sequence
10
<(bd)cb(ac)>
<g>
1
20
<(bf)(ce)b(fg)>
<h>
1
30
<(ah)(bf)abf>
40
<(be)(ce)d>
50
<a(bd)bcb(ade)>
Data Mining: Concepts and Techniques
90
Generating Length-2 Candidates
51 length-2
Candidates
<a>
<a>
<b>
<c>
<d>
<e>
<f>
May 22, 2017
<a>
<b>
<c>
<d>
<e>
<f>
<a>
<aa>
<ab>
<ac>
<ad>
<ae>
<af>
<b>
<ba>
<bb>
<bc>
<bd>
<be>
<bf>
<c>
<ca>
<cb>
<cc>
<cd>
<ce>
<cf>
<d>
<da>
<db>
<dc>
<dd>
<de>
<df>
<e>
<ea>
<eb>
<ec>
<ed>
<ee>
<ef>
<f>
<fa>
<fb>
<fc>
<fd>
<fe>
<ff>
<b>
<c>
<d>
<e>
<f>
<(ab)>
<(ac)>
<(ad)>
<(ae)>
<(af)>
<(bc)>
<(bd)>
<(be)>
<(bf)>
<(cd)>
<(ce)>
<(cf)>
<(de)>
<(df)>
Without Apriori
property,
8*8+8*7/2=92
candidates
<(ef)>
Apriori prunes
44.57% candidates
Data Mining: Concepts and Techniques
91
Generating Length-3 Candidates and Finding
Length-3 Patterns

Generate Length-3 Candidates



Self-join length-2 sequential patterns
 Based on the Apriori property
 <ab>, <aa> and <ba> are all length-2 sequential
patterns  <aba> is a length-3 candidate
 <(bd)>, <bb> and <db> are all length-2 sequential
patterns  <(bd)b> is a length-3 candidate
46 candidates are generated
Find Length-3 Sequential Patterns


Scan database once more, collect support counts for
candidates
19 out of 46 candidates pass support threshold
May 22, 2017
Data Mining: Concepts and Techniques
92
The GSP Mining Process
5th scan: 1 cand. 1 length-5 seq.
pat.
Cand. cannot pass
sup. threshold
<(bd)cba>
Cand. not in DB at all
4th scan: 8 cand. 6 length-4 seq. <abba> <(bd)bc> …
pat.
3rd scan: 46 cand. 19 length-3 seq. <abb> <aab> <aba> <baa> <bab> …
pat. 20 cand. not in DB at all
2nd scan: 51 cand. 19 length-2 seq.
<aa> <ab> … <af> <ba> <bb> … <ff> <(ab)> … <(ef)>
pat. 10 cand. not in DB at all
1st scan: 8 cand. 6 length-1 seq.
<a> <b> <c> <d> <e> <f> <g> <h>
pat.
min_sup =2
May 22, 2017
Seq. ID
Sequence
10
<(bd)cb(ac)>
20
<(bf)(ce)b(fg)>
30
<(ah)(bf)abf>
40
<(be)(ce)d>
50
<a(bd)bcb(ade)>
Data Mining: Concepts and Techniques
93
Is GSP Efficient Enough?—Bottlenecks

A huge set of candidates could be generated
1,000 frequent length-1 sequences generate
1000  999
1000 1000 
 1,499,500 length-2 candidates!


2
Multiple of scans of database in mining

Real challenge: mining long sequential patterns


An exponential number of short candidates
A length-100 sequential pattern needs 1030
candidate sequences!
100  100
30



2

1

10

 
i 1  i 
100
May 22, 2017
Data Mining: Concepts and Techniques
95
Can We Do Better?—The Evolution Path
of Our Approach

Initial try: Using FP-tree and FP-growth?



Based on our SIGMOD’00 on frequent-pattern mining
Development of a sequential pattern growth approach

FreeSpan (KDD’00)

PrefixSpan (ICDE’01)
Extensions to PrefixSpan

Closed- and max- sequential patterns

Constraint-based sequential pattern growth

From sequential patterns to structured patterns
May 22, 2017
Data Mining: Concepts and Techniques
96
Using FP-Tree and FP-Growth?

FP-tree and FP-growth: A successful frequent-pattern mining
approach




J. Han, J. Pei, and Y. Yin: “Mining frequent patterns without candidate
generation”. In Proc. ACM-SIGMOD’2000, pp. 1-12, Dallas, TX, May
2000.
FP-tree: A compressed transaction database for frequent-pattern
mining
FP-growth: A frequent-pattern growth approach
 No candidate generation and test
 Growth of frequent-patterns in projected databases
Can we extend FP-growth to sequential pattern mining?
 Sequential-pattern tree?— Unfortunately, little sharing can be
explored to compress sequence database!
 However, pattern-growth is still valuable
May 22, 2017
Data Mining: Concepts and Techniques
97
Why Does FP-growth Win in Frequent-Pattern
Mining ?

Divide-and-conquer:



Other factors





decompose both the mining task and DB according to
the frequent patterns obtained so far
leads to focused search of smaller databases
no candidate generation, no candidate test
compressed database: FP-tree structure
no repeated scan of entire database
basic ops—counting and FP-tree building, not pattern
search and matching
But no compressed trees in sequential pattern mining!
May 22, 2017
Data Mining: Concepts and Techniques
98
Our First Try: FreeSpan: Frequent Pattern-Projected
Sequential Pattern Mining


J. Han J. Pei, B. Mortazavi-Asi, Q. Chen, U. Dayal, M.C. Hsu, FreeSpan:
Frequent pattern-projected sequential pattern mining, KDD’00.
A divide-and-conquer approach


Recursively project a sequence database into a set of smaller
databases based on the current set of frequent patterns
Mine each projected database to find its patterns
Sequence Database SDB
< (bd) c b (ac) >
< (bf) (ce) b (fg) >
< (ah) (bf) a b f >
< (be) (ce) d >
< a (bd) b c b (ade) >
May 22, 2017
f_list: b:5, c:4, a:3, d:3, e:3, f:2
All seq. pat. can be divided into 6 subsets:
•Seq. pat. containing item f
•Those containing e but no f
•Those containing d but no e nor f
•Those containing a but no d, e or f
•Those containing c but no a, d, e or f
•Those containing only item b
Data Mining: Concepts and Techniques
99
From FreeSpan to PrefixSpan: Why?

Freespan:



Projection-based: No candidate sequence needs to
be generated
But, projection can be performed at any point in the
sequence, and the projected sequences do will not
shrink much
PrefixSpan


May 22, 2017
Projection-based
But only prefix-based projection: less projections and
quickly shrinking sequences
Data Mining: Concepts and Techniques
100
Prefix and Suffix (Projection)


<a>, <aa>, <a(ab)> and <a(abc)> are
prefixes of sequence <a(abc)(ac)d(cf)>
Given sequence <a(abc)(ac)d(cf)>
May 22, 2017
Prefix
Suffix (Prefix-Based Projection)
<a>
<(abc)(ac)d(cf)>
<aa>
<ab>
<(_bc)(ac)d(cf)>
<(_c)(ac)d(cf)>
Data Mining: Concepts and Techniques
101
Mining Sequential Patterns by Prefix
Projections


Step 1: find length-1 sequential patterns
 <a>, <b>, <c>, <d>, <e>, <f>
Step 2: divide search space. The complete set of seq. pat.
can be partitioned into 6 subsets:
 The ones having prefix <a>;
 The ones having prefix <b>;
SID
sequence
 …
10
<a(abc)(ac)d(cf)>
 The ones having prefix <f>
20
<(ad)c(bc)(ae)>
May 22, 2017
30
<(ef)(ab)(df)cb>
40
<eg(af)cbc>
Data Mining: Concepts and Techniques
102
Finding Seq. Patterns with Prefix <a>

Only need to consider projections w.r.t. <a>


<a>-projected database: <(abc)(ac)d(cf)>,
<(_d)c(bc)(ae)>, <(_b)(df)cb>, <(_f)cbc>
Find all the length-2 seq. pat. Having prefix <a>: <aa>,
<ab>, <(ab)>, <ac>, <ad>, <af>

Further partition into 6 subsets
sequence

Having prefix <aa>;
10
<a(abc)(ac)d(cf)>
<(ad)c(bc)(ae)>

…
20
30
<(ef)(ab)(df)cb>
40
<eg(af)cbc>

May 22, 2017
SID
Having prefix <af>
Data Mining: Concepts and Techniques
103
Completeness of PrefixSpan
SDB
Having prefix <a>
<a>-projected database
<(abc)(ac)d(cf)>
<(_d)c(bc)(ae)>
<(_b)(df)cb>
<(_f)cbc>
SID
sequence
10
<a(abc)(ac)d(cf)>
20
<(ad)c(bc)(ae)>
30
<(ef)(ab)(df)cb>
40
<eg(af)cbc>
Length-1 sequential patterns
<a>, <b>, <c>, <d>, <e>, <f>
Having prefix <c>, …, <f>
Having prefix <b>
<b>-projected database
Length-2 sequential
patterns
<aa>, <ab>, <(ab)>,
<ac>, <ad>, <af>
…
……
Having prefix <aa> Having prefix <af>
<aa>-proj. db
May 22, 2017
…
<af>-proj. db
Data Mining: Concepts and Techniques
104
Efficiency of PrefixSpan

No candidate sequence needs to be generated

Projected databases keep shrinking

Major cost of PrefixSpan: constructing
projected databases

May 22, 2017
Can be improved by bi-level projections
Data Mining: Concepts and Techniques
105
Optimization Techniques in PrefixSpan

Single-level vs. bi-level projection

Bi-level projection with 3-way checking may reduce the
number and size of projected databases

Physical projection vs. pseudo-projection

Pseudo-projection may reduce the effort of projection
when the projected database fits in main memory

Parallel projection vs. partition projection

Partition projection may avoid the blowup of disk space
May 22, 2017
Data Mining: Concepts and Techniques
106
Scaling-up by Bi-level Projection

Partition search space based on length-2
sequential patterns

Only form projected databases and pursue
recursive mining over bi-level projected
databases
May 22, 2017
Data Mining: Concepts and Techniques
107
Pair-wise Checking Using S-matrix
SDB
SID
sequence
10
<a(abc)(ac)d(cf)>
20
<(ad)c(bc)(ae)>
30
<(ef)(ab)(df)cb>
40
<eg(af)cbc>
Length-1 sequential patterns
<a>, <b>, <c>, <d>, <e>, <f>
<aa> happens twice
<(ac)> happens once
<ac> happens
4 times
<ca> happens
twice
a
2
b
(4, 2, 2)
1
c
(4, 2, 1)
(3, 3, 2)
3
d
(2, 1, 1)
(2, 2, 0)
(1, 3, 0)
0
e
(1, 2, 1)
(1, 2, 0)
(1, 2, 0)
(1, 1, 0)
0
f
(2, 1, 1)
(2, 2, 0)
(1, 2, 1)
(1, 1, 1)
(2, 0, 1)
1
a
b
c
d
e
f
S-matrix
All length-2 sequential patterns are found in S-matrix
May 22, 2017
Data Mining: Concepts and Techniques
108
Mining <ab>-projected Database
SDB
Length-1 sequential patterns
<a>, <b>, <c>, <d>, <e>, <f>
SID
sequence
10
<a(abc)(ac)d(cf)>
a
20
<(ad)c(bc)(ae)>
b
30
<(ef)(ab)(df)cb>
40
<eg(af)cbc>
2
4
( , 2, 2)
c
(4, 2, 1)
(3, 3, 2)
3
d
(2, 1, 1)
(2, 2, 0)
(1, 3, 0)
0
e
(1, 2, 1)
(1, 2, 0)
(1, 2, 0)
(1, 1, 0)
0
f
(2, 1, 1)
(2, 2, 0)
(1, 2, 1)
(1, 1, 1)
(2, 0, 1)
1
a
b
c
d
e
f
<ab>-projected database
<(_c)(ac(cf)>
Lead to pattern
<(_c)a>
<a(bc)a>
<c>
Local length-1
sequential patterns:
<a>, <c>, <(_c)>
May 22, 2017
S-matrix
1
No hope to form (_ac), so no
need to count it.
S-matrix
a
0
c
(1, 0, 1)
1
(_c)
(, 2, )
(, 1, )

a
c
(_c)
Data Mining: Concepts and Techniques
109
Benefits of Bi-level Projection

More patterns are found in each shoot

Much less projections

In the example, there are 53 patterns.

53 level-by-level projections

22 bi-level projections
May 22, 2017
Data Mining: Concepts and Techniques
110
Three-Way Apriori Checking

Using Apriori heuristic to prune items in
projected databases

Absorb goodness of Apriori-like algorithms
<acd> cannot be a pattern!
Exclude d from <ac>-projected database
a
2
b
(4, 2, 2)
1
c
(4, 2, 1)
(3, 3, 2)
3
d
(2, 1, 1)
(2, 2, 0)
(1, 3, 0)
0
e
(1, 2, 1)
(1, 2, 0)
(1, 2, 0)
(1, 1, 0)
0
f
(2, 1, 1)
(2, 2, 0)
(1, 2, 1)
(1, 1, 1)
(2, 0, 1)
1
a
b
c
d
e
f
May 22, 2017
Data Mining: Concepts and Techniques
111
Speed-up by Pseudo-projection

Major cost of PrefixSpan: projection


Postfixes of sequences often appear
repeatedly in recursive projected
databases
When (projected) database can be held in main
memory, use pointers to form projections

Pointer to the sequence

Offset of the postfix
s=<a(abc)(ac)d(cf)>
<a>
s|<a>: ( , 2) <(abc)(ac)d(cf)>
<ab>
s|<ab>: ( , 4) <(_c)(ac)d(cf)>
May 22, 2017
Data Mining: Concepts and Techniques
112
Pseudo-Projection vs. Physical Projection



May 22, 2017
Pseudo-projection avoids physically copying
postfixes
 Efficient in running time and space when
database can be held in main memory
However, it is not efficient when database cannot fit
in main memory
 Disk-based random accessing is very costly
Suggested Approach:
 Integration of physical and pseudo-projection
 Swapping to pseudo-projection when the data
set fits in memory
Data Mining: Concepts and Techniques
113
Experiments and Performance Analysis

Comparing PrefixSpan with GSP and FreeSpan in large
databases


GSP (IBM Almaden, Srikant & Agrawal EDBT’96)
FreeSpan (J. Han J. Pei, B. Mortazavi-Asi, Q. Chen, U.
Dayal, M.C. Hsu, KDD’00)

Prefix-Span-1 (single-level projection)

Prefix-Scan-2 (bi-level projection)

Comparing effects of pseudo-projection

Comparing I/O cost and scalability
May 22, 2017
Data Mining: Concepts and Techniques
114
Runtime (second)
PrefixSpan Is Faster Than GSP and
FreeSpan
400
PrefixSpan-1
350
PrefixSpan-2
300
FreeSpan
250
GSP
200
150
100
50
0
0.00
0.50
1.00
1.50
2.00
2.50
3.00
Support threshold (%)
May 22, 2017
Data Mining: Concepts and Techniques
115
Effect of Pseudo-Projection
PrefixSpan-1
200
Runtime (second)
PrefixSpan-2
PrefixSpan-1 (Pseudo)
160
PrefixSpan-2 (Pseudo)
120
80
40
0
0.20
0.30
0.40
0.50
0.60
Support threshold (%)
May 22, 2017
Data Mining: Concepts and Techniques
116
I/O Cost: When It Cannot Fit in
Memory
1.E+10
PrefixSpan-1
PrefixSpan-1 (pseudo)
PrefixSpan-2
PrefixSpan-2 (pseudo)
I/O Cost
8.E+09
6.E+09
4.E+09
2.E+09
0.E+00
0.0
1.0
2.0
3.0
Support threshold (% )
May 22, 2017
Data Mining: Concepts and Techniques
117
Scalability (When DB Is Large)
Runtime (thousand
second)
30
25
20
15
10
PrefixSpan-1
5
PrefixSpan-2
0
0
100
200
300
400
500
# of sequences (thousand)
May 22, 2017
Data Mining: Concepts and Techniques
118
Major Features of PrefixSpan




Both PrefixSpan and FreeSpan are patterngrowth methods
 Searches are more focused and thus efficient
Prefix-projected pattern growth (PrefixSpan) is
simpler and more elegant than frequent patternguided projection (FreeSpan)
The Apriori heuristic is integrated into bi-level
projection PrefixSpan
Pseudo-projection substantially enhances the
performance of memory-based processing
May 22, 2017
Data Mining: Concepts and Techniques
119
Further Evolution of PrefixSpan

Closed- and max- sequential patterns

Finding only the most meaningful (longest)
sequential patterns

Constraint-based sequential pattern growth


Adding user-specific constraints
From sequential patterns to structured patterns

Beyond sequential patterns, mining structured
patterns in XML documents
May 22, 2017
Data Mining: Concepts and Techniques
120
Closed- and Max- Sequential Patterns

A closed- sequential pattern is a frequent sequence s where there is
no proper super-sequence of s sharing the same support count with s

A max- sequential pattern is a sequential pattern p s.t. any proper
super-pattern of p is not frequent

Benefit of the notion of closed sequential patterns



{<a1 a2 … a50>, <a1 a2 … a100>}, with min_sup = 1
There are 2100 sequential patterns, but only 2 are
closed
Similar benefits for the notion of max- sequential-patterns
May 22, 2017
Data Mining: Concepts and Techniques
121
Methods for Mining Closed- and MaxSequential Patterns

PrefixSpan or FreeSpan can be viewed as projection-guided depth-first
search

For mining max- sequential patterns, any sequence which does not
contain anything beyond the already discovered ones will be removed
from the projected DB



{<a1 a2 … a50>, <a1 a2 … a100>}, with min_sup = 1
If we have found a max-sequential pattern <a1 a2 …
a100>, nothing will be projected in any projected DB
Similar ideas can be applied for mining closed- sequential-patterns
May 22, 2017
Data Mining: Concepts and Techniques
122
Constraint-Based Sequential Pattern Mining


Constraint-based sequential pattern mining

Constraints: User-specified, for focused mining of desired patterns

How to explore efficient mining with constraints? — Optimization
Classification of constraints

Anti-monotone: E.g., value_sum(S) < 150, min(S) > 10

Monotone: E.g., count (S) > 5, S  {PC, digital_camera}

Succinct: E.g., length(S)  10, S  {Pentium, MS/Office, MS/Money}


Convertible: E.g., value_avg(S) < 25, profit_sum (S) > 160,
max(S)/avg(S) < 2, median(S) – min(S) > 5
Inconvertible: E.g., avg(S) – median(S) = 0
May 22, 2017
Data Mining: Concepts and Techniques
123
Sequential Pattern Growth for ConstraintBased Mining


Efficient mining with convertible constraints

Not solvable by candidate generation-and-test methodology

Easily push-able into the sequential pattern growth framework
Example: push avg(S) < 25 in frequent pattern growth

project items in value (price/profit depending on mining
semantics) ascending/descending order for sequential pattern
growth

Grow each pattern by sequential pattern growth

If avg(current_pattern)  25, toss the current_pattern


May 22, 2017
Why?—future growths always make it bigger
But why not candidate generation?—no structure or ordering in
growth
Data Mining: Concepts and Techniques
124
From Sequential Patterns to Structured
Patterns

Sets, sequences, trees and other structures

Transaction DB: Sets of items


Seq. DB: Sequences of sets:


{{<i1, i2>, …, <im, in, ik>}, …}
Sets of trees (each element being a tree):


{<{i1, i2}, …, {im, in, ik}>, …}
Sets of Sequences:


{{i1, i2, …, im}, …}
{t1, t2, …, tn}
Applications: Mining structured patterns in XML documents
May 22, 2017
Data Mining: Concepts and Techniques
125
Conclusions



Sequential pattern mining—an important task in data
mining
Previous sequential pattern mining methodology:
candidate generation-and-test
Our methodology: sequential pattern growth (mining
sequential patterns without candidate generation)


Project on prefixes by exploring level-by-level & bilevel projections as well as pseudo-projection
Extensions:

May 22, 2017
Mining max-/closed sequential/structured patterns
with constraints
Data Mining: Concepts and Techniques
126
References










R. Agarwal, C. Aggarwal, and V. V. V. Prasad. A tree projection algorithm for generation of frequent
itemsets. In Journal of Parallel and Distributed Computing (Special Issue on High Performance Data
Mining), 2000.
R. Agrawal, T. Imielinski, and A. Swami. Mining association rules between sets of items in large
databases. SIGMOD'93, 207-216, Washington, D.C.
R. Agrawal and R. Srikant. Fast algorithms for mining association rules. VLDB'94 487-499, Santiago,
Chile.
R. Agrawal and R. Srikant. Mining sequential patterns. ICDE'95, 3-14, Taipei, Taiwan.
R. J. Bayardo. Efficiently mining long patterns from databases. SIGMOD'98, 85-93, Seattle,
Washington.
S. Brin, R. Motwani, and C. Silverstein. Beyond market basket: Generalizing association rules to
correlations. SIGMOD'97, 265-276, Tucson, Arizona.
S. Brin, R. Motwani, J. D. Ullman, and S. Tsur. Dynamic itemset counting and implication rules for
market basket analysis. SIGMOD'97, 255-264, Tucson, Arizona, May 1997.
K. Beyer and R. Ramakrishnan. Bottom-up computation of sparse and iceberg cubes. SIGMOD'99, 359370, Philadelphia, PA, June 1999.
D.W. Cheung, J. Han, V. Ng, and C.Y. Wong. Maintenance of discovered association rules in large
databases: An incremental updating technique. ICDE'96, 106-114, New Orleans, LA.
M. Fang, N. Shivakumar, H. Garcia-Molina, R. Motwani, and J. D. Ullman. Computing iceberg queries
efficiently. VLDB'98, 299-310, New York, NY, Aug. 1998.
May 22, 2017
Data Mining: Concepts and Techniques
127
References (2)










G. Grahne, L. Lakshmanan, and X. Wang. Efficient mining of constrained correlated sets. ICDE'00, 512521, San Diego, CA, Feb. 2000.
Y. Fu and J. Han. Meta-rule-guided mining of association rules in relational databases. KDOOD'95, 3946, Singapore, Dec. 1995.
T. Fukuda, Y. Morimoto, S. Morishita, and T. Tokuyama. Data mining using two-dimensional optimized
association rules: Scheme, algorithms, and visualization. SIGMOD'96, 13-23, Montreal, Canada.
E.-H. Han, G. Karypis, and V. Kumar. Scalable parallel data mining for association rules. SIGMOD'97,
277-288, Tucson, Arizona.
J. Han, G. Dong, and Y. Yin. Efficient mining of partial periodic patterns in time series database.
ICDE'99, Sydney, Australia.
J. Han and Y. Fu. Discovery of multiple-level association rules from large databases. VLDB'95, 420-431,
Zurich, Switzerland.
J. Han, J. Pei, and Y. Yin. Mining frequent patterns without candidate generation. SIGMOD'00, 1-12,
Dallas, TX, May 2000.
T. Imielinski and H. Mannila. A database perspective on knowledge discovery. Communications of ACM,
39:58-64, 1996.
M. Kamber, J. Han, and J. Y. Chiang. Metarule-guided mining of multi-dimensional association rules
using data cubes. KDD'97, 207-210, Newport Beach, California.
M. Klemettinen, H. Mannila, P. Ronkainen, H. Toivonen, and A.I. Verkamo. Finding interesting rules
from large sets of discovered association rules. CIKM'94, 401-408, Gaithersburg, Maryland.
May 22, 2017
Data Mining: Concepts and Techniques
128
References (3)









F. Korn, A. Labrinidis, Y. Kotidis, and C. Faloutsos. Ratio rules: A new paradigm for fast, quantifiable
data mining. VLDB'98, 582-593, New York, NY.
B. Lent, A. Swami, and J. Widom. Clustering association rules. ICDE'97, 220-231, Birmingham,
England.
H. Lu, J. Han, and L. Feng. Stock movement and n-dimensional inter-transaction association rules.
SIGMOD Workshop on Research Issues on Data Mining and Knowledge Discovery (DMKD'98), 12:112:7, Seattle, Washington.
H. Mannila, H. Toivonen, and A. I. Verkamo. Efficient algorithms for discovering association rules.
KDD'94, 181-192, Seattle, WA, July 1994.
H. Mannila, H Toivonen, and A. I. Verkamo. Discovery of frequent episodes in event sequences. Data
Mining and Knowledge Discovery, 1:259-289, 1997.
R. Meo, G. Psaila, and S. Ceri. A new SQL-like operator for mining association rules. VLDB'96, 122133, Bombay, India.
R.J. Miller and Y. Yang. Association rules over interval data. SIGMOD'97, 452-461, Tucson, Arizona.
R. Ng, L. V. S. Lakshmanan, J. Han, and A. Pang. Exploratory mining and pruning optimizations of
constrained associations rules. SIGMOD'98, 13-24, Seattle, Washington.
N. Pasquier, Y. Bastide, R. Taouil, and L. Lakhal. Discovering frequent closed itemsets for association
rules. ICDT'99, 398-416, Jerusalem, Israel, Jan. 1999.
May 22, 2017
Data Mining: Concepts and Techniques
129
References (4)










J.S. Park, M.S. Chen, and P.S. Yu. An effective hash-based algorithm for mining association rules.
SIGMOD'95, 175-186, San Jose, CA, May 1995.
J. Pei, J. Han, and R. Mao. CLOSET: An Efficient Algorithm for Mining Frequent Closed Itemsets.
DMKD'00, Dallas, TX, 11-20, May 2000.
J. Pei and J. Han. Can We Push More Constraints into Frequent Pattern Mining? KDD'00. Boston,
MA. Aug. 2000.
G. Piatetsky-Shapiro. Discovery, analysis, and presentation of strong rules. In G. Piatetsky-Shapiro
and W. J. Frawley, editors, Knowledge Discovery in Databases, 229-238. AAAI/MIT Press, 1991.
B. Ozden, S. Ramaswamy, and A. Silberschatz. Cyclic association rules. ICDE'98, 412-421, Orlando,
FL.
J.S. Park, M.S. Chen, and P.S. Yu. An effective hash-based algorithm for mining association rules.
SIGMOD'95, 175-186, San Jose, CA.
S. Ramaswamy, S. Mahajan, and A. Silberschatz. On the discovery of interesting patterns in
association rules. VLDB'98, 368-379, New York, NY..
S. Sarawagi, S. Thomas, and R. Agrawal. Integrating association rule mining with relational database
systems: Alternatives and implications. SIGMOD'98, 343-354, Seattle, WA.
A. Savasere, E. Omiecinski, and S. Navathe. An efficient algorithm for mining association rules in
large databases. VLDB'95, 432-443, Zurich, Switzerland.
A. Savasere, E. Omiecinski, and S. Navathe. Mining for strong negative associations in a large
database of customer transactions. ICDE'98, 494-502, Orlando, FL, Feb. 1998.
May 22, 2017
Data Mining: Concepts and Techniques
130
References (5)










C. Silverstein, S. Brin, R. Motwani, and J. Ullman. Scalable techniques for mining causal
structures. VLDB'98, 594-605, New York, NY.
R. Srikant and R. Agrawal. Mining generalized association rules. VLDB'95, 407-419, Zurich,
Switzerland, Sept. 1995.
R. Srikant and R. Agrawal. Mining quantitative association rules in large relational tables.
SIGMOD'96, 1-12, Montreal, Canada.
R. Srikant, Q. Vu, and R. Agrawal. Mining association rules with item constraints. KDD'97, 67-73,
Newport Beach, California.
H. Toivonen. Sampling large databases for association rules. VLDB'96, 134-145, Bombay, India,
Sept. 1996.
D. Tsur, J. D. Ullman, S. Abitboul, C. Clifton, R. Motwani, and S. Nestorov. Query flocks: A
generalization of association-rule mining. SIGMOD'98, 1-12, Seattle, Washington.
K. Yoda, T. Fukuda, Y. Morimoto, S. Morishita, and T. Tokuyama. Computing optimized rectilinear
regions for association rules. KDD'97, 96-103, Newport Beach, CA, Aug. 1997.
M. J. Zaki, S. Parthasarathy, M. Ogihara, and W. Li. Parallel algorithm for discovery of association
rules. Data Mining and Knowledge Discovery, 1:343-374, 1997.
M. Zaki. Generating Non-Redundant Association Rules. KDD'00. Boston, MA. Aug. 2000.
O. R. Zaiane, J. Han, and H. Zhu. Mining Recurrent Items in Multimedia with Progressive
Resolution Refinement. ICDE'00, 461-470, San Diego, CA, Feb. 2000.
May 22, 2017
Data Mining: Concepts and Techniques
131
References (6)






J. Pei, J. Han, H. Pinto, Q. Chen, U. Dayal, and M.-C. Hsu, "PrefixSpan: Mining Sequential Patterns
Efficiently by Prefix-Projected Pattern Growth", Proc. 2001 Int. Conf. on Data Engineering (ICDE'01),
Heidelberg, Germany, April 2001. (U.S. Patent Pending)
J. Han, J. Pei, B. Mortazavi-Asl, Q. Chen, U. Dayal, M.-C. Hsu, "FreeSpan: Frequent Pattern-Projected
Sequential Pattern Mining", Proc. 2000 Int. Conf. on Knowledge Discovery and Data Mining (KDD'00),
Boston, MA, August 2000.
J. Han, J. Pei, and Y. Yin: “Mining frequent patterns without candidate generation”. In Proc. ACMSIGMOD’2000, pp. 1-12, Dallas, TX, May 2000. (U.S. Patent Pending)
J. Pei, J. Han, and L. V. S. Lakshmanan, "Mining Frequent Itemsets with Convertible Constraints", Proc.
2001 Int. Conf. on Data Engineering (ICDE'01), Heidelberg, Germany, April 2001.
J. Pei, J. Han, and R. Mao, "CLOSET: An Efficient Algorithm for Mining Frequent Closed Itemsets", Proc.
2000 ACM-SIGMOD Int. Workshop on Data Mining and Knowledge Discovery (DMKD'00), Dallas, TX,
May 2000.
J. Han, J. Pei, G. Dong, and K. Wang, “Computing Iceberg Data Cubes with Complex Measures”, Proc.
ACM-SIGMOD’2001, Santa Barbara, CA, May 2001. (U.S. Patent Pending)
May 22, 2017
Data Mining: Concepts and Techniques
132
Other Recent Studies: Constraint-Base Data Mining

Constraint-Based Clustering in Large Databases (ICDT'01)


Spatial Clustering in the Presence of Obstacles (ICDE'01)


A. K. H. Tung, J. Hou, and J. Han, "Spatial Clustering in the Presence of Obstacles", Proc. 2001 Int.
Conf. on Data Engineering (ICDE'01), Heidelberg, Germany, April 2001.
Mining Frequent Itemsets with Convertible Constraints (ICDE'01, KDD'00)



A. K. H. Tung, J. Han, L. V. S. Lakshmanan, and R. T. Ng, "Constraint-Based Clustering in Large
Databases", Proc. 2001 Int. Conf. on Database Theory (ICDT'01), London, U.K., Jan. 2001.
J. Pei, J. Han, and L. V. S. Lakshmanan, "Mining Frequent Itemsets with Convertible Constraints",
Proc. 2001 Int. Conf. on Data Engineering (ICDE'01), April 2001.
J. Pei and J. Han "Can We Push More Constraints into Frequent Pattern Mining?", Proc. 2000 Int.
Conf. on Knowledge Discovery and Data Mining (KDD'00), Boston, MA, August 2000.
Constraint-Based Association Mining (SIGMOD’98, SIGMOD’99, COMPUTER’99)



J. Han, L. V. S. Lakshmanan, and R. T. Ng, "Constraint-Based, Multidimensional Data Mining",
COMPUTER (special issues on Data Mining), 32(8): 46-50, 1999.
R. Ng, L.V.S. Lakshmanan, J. Han & A. Pang. “Exploratory mining and pruning optimizations of
constrained association rules.” SIGMOD’98
L. V. S. Lakshmanan, R. Ng, J. Han and A. Pang, "Optimization of Constrained Frequent Set Queries
with 2-Variable Constraints", SIGMOD’99
May 22, 2017
Data Mining: Concepts and Techniques
133
http://www.cs.uiuc.edu/~hanj
Thank you !!!
May 22, 2017
Data Mining: Concepts and Techniques
134