Download data quality

第四部分
数据库新应用与新技术
60年代中后期至90年代末，发展速度之快，应用范围之广，
是许多其他技术远不能及的。
发展的动力

计算环境的发展

与其他相关技术发展相结合

受应用需求推动

. . .
数据库技术发展和研究方向











并行数据库
分布式数据库
移动式、嵌入式数据库
面向对象数据库，对象-关系数据库
演绎数据库，知识库，主动数据库
多媒体数据库
空间数据库
数据仓库和OLAP，数据挖掘
信息集成
Web环境中的数据库技术
...
数据库技术发展和研究方向











并行数据库
分布式数据库
移动式、嵌入式数据库
面向对象数据库，对象-关系数据库
演绎数据库，知识库，主动数据库
多媒体数据库
空间数据库
数据仓库和OLAP，数据挖掘
信息集成
Web环境中的数据库技术
...
第十一章 数据库中的知识发现
Slides are basically from：
Data Mining:
Concepts and Techniques
— Slides for Textbook —
Jiawei Han and Micheline Kamber
Simon Fraser University, Canada
http://www.cs.sfu.ca

Motivation: Why Knowledge discovery in
databases?

Data Warehouse and OLAP

data mining
Motivation: “Necessity is the Mother
of Invention”

Data explosion problem

Automated data collection tools and mature database
technology lead to tremendous amounts of data
stored in databases, data warehouses and other
information repositories

We are drowning in data, but starving for knowledge!

Solution: Data warehousing and data mining


Data warehousing and on-line analytical processing
Extraction of interesting knowledge (rules, regularities,
patterns, constraints) from data in large databases
Evolution of Database Technology




1960s:
 Data collection, database creation, IMS and network
DBMS
1970s:
 Relational data model, relational DBMS
implementation
1980s:
 RDBMS, advanced data models (extended-relational,
OO, deductive, etc.) and application-oriented DBMS
(spatial, scientific, engineering, etc.)
1990s—2000s:
 Data mining and data warehousing, multimedia
databases, and Web databases

Motivation: Why Knowledge discovery in databases?

Data Warehouse and OLAP


What is a data warehouse?

A multi-dimensional data model

Data warehouse architecture

Data warehouse application
Data mining
What is Data Warehouse?



Defined in many different ways, but not rigorously.
 A decision support database that is maintained
separately from the organization’s operational
database
 Support information processing by providing a solid
platform of consolidated, historical data for analysis.
“A data warehouse is a subject-oriented, integrated,
time-variant, and nonvolatile collection of data in support
of management’s decision-making process.”—W. H.
Inmon
Data warehousing:
 The process of constructing and using data
warehouses
Data Warehouse—Subject-Oriented

Organized around major subjects, such as customer,
product, sales.

Focusing on the modeling and analysis of data for
decision makers, not on daily operations or transaction
processing.

Provide a simple and concise view around particular
subject issues by excluding data that are not useful in
the decision support process.
Data Warehouse—Integrated


Constructed by integrating multiple, heterogeneous
data sources
 relational databases, flat files, on-line transaction
records
Data cleaning and data integration techniques are
applied.
 Ensure consistency in naming conventions, encoding
structures, attribute measures, etc. among different
data sources


E.g., Hotel price: currency, tax, breakfast covered, etc.
When data is moved to the warehouse, it is
converted.
Data Warehouse—Time Variant

The time horizon for the data warehouse is significantly
longer than that of operational systems.



Operational database: current value data.
Data warehouse data: provide information from a
historical perspective (e.g., past 5-10 years)
Every key structure in the data warehouse


Contains an element of time, explicitly or implicitly
But the key of operational data may or may not
contain “time element”.
Data Warehouse—Non-Volatile

A physically separate store of data transformed from the
operational environment.

Operational update of data does not occur in the data
warehouse environment.

Does not require transaction processing, recovery,
and concurrency control mechanisms

Requires only two operations in data accessing:

initial loading of data and access of data.
Data Warehouse vs. Heterogeneous
DBMS

Traditional heterogeneous DB integration:


Build wrappers/mediators on top of heterogeneous
databases
Query driven approach:



When a query is posed to a client site, a metadictionary is used to translate the query into queries
appropriate for individual heterogeneous sites
involved, and the results are integrated into a global
answer set.
Complex information filtering, compete for sources
Data warehouse: update-driven, high performance.
Data Warehouse vs. Operational DBMS

OLTP (on-line transaction processing)




Major task of traditional relational DBMS
Day-to-day operations: purchasing, inventory, banking,
manufacturing, payroll, registration, accounting, etc.
OLAP (on-line analytical processing)

Major task of data warehouse system

Data analysis and decision making
Distinct features (OLTP vs. OLAP):

User and system orientation: customer vs. market

Data contents: current, detailed vs. historical, consolidated

Database design: ER + application vs. star + subject

View: current, local vs. evolutionary, integrated

Access patterns: update vs. read-only but complex queries
OLTP vs. OLAP
OLTP
OLAP
users
clerk, IT professional
knowledge worker
function
day to day operations
decision support
DB design
application-oriented
subject-oriented
data
current, up-to-date
detailed, flat relational
isolated
repetitive
historical,
summarized, multidimensional
integrated, consolidated
ad-hoc
lots of scans
unit of work
read/write
index/hash on prim. key
short, simple transaction
# records accessed
tens
millions
#users
thousands
hundreds
DB size
100MB-GB
100GB-TB
metric
transaction throughput
query throughput, response
usage
access
complex query
Why Separate Data Warehouse?


High performance for both systems
 DBMS— tuned for OLTP: access methods, indexing,
concurrency control, recovery
 Warehouse—tuned for OLAP: complex OLAP queries,
multidimensional view, consolidation.
Different functions and different data:
 missing data: Decision support requires historical data
which operational DBs do not typically maintain
 data consolidation: DS requires consolidation
(aggregation, summarization) of data from
heterogeneous sources
 data quality: different sources typically use inconsistent
data representations, codes and formats which have to
be reconciled

Motivation: Why Knowledge discovery in databases?

Data Warehouse and OLAP


What is a data warehouse?

A multi-dimensional data model

Data warehouse architecture

Data warehouse application
Data mining
Multidimensional Data Model

A data warehouse is based on a multidimensional data
model which views data in the form of a data cube



Measures attributes: attributes which measure some
value and can be aggregated upon. such as
dollars_sold.
Dimension attributes: attributes on which measure
attributes and summaries of measure attributes are
viewed. such as item (item_name, brand, type), or
time(day, week, month, quarter, year).
Data cube: multidimensional view of data. such as
sales.
A concept hierarchy: dimension (location)
all
all
Europe
region
country
city
office
Germany
Frankfurt
...
...
...
Spain
North_America
Canada
Vancouver ...
L. Chan
...
...
Mexico
Toronto
M. Young
Multidimensional Data

Sales volume as a function of product, month,
and region
Dimensions: Product, Location, Time
Hierarchical summarization paths
Industry Region
Year
Product
Category Country Quarter
Product
City
Office
Month
Month Week
Day
A Sample Data Cube
2Qtr
3Qtr
4Qtr
sum
U.S.A
Canada
Mexico
sum
Country
TV
PC
VCR
sum
1Qtr
Date
Total annual sales
of TV in U.S.A.
cuboid and Data Cubes

In data warehousing literature, an n-D base cube is
called a base cuboid. The top most 0-D cuboid, which
holds the highest-level of summarization, is called the
apex cuboid. The lattice of cuboids forms a data cube.
Cuboids Corresponding to the Cube
all
0-D(apex) cuboid
product
product,date
date
country
product,country
1-D cuboids
date, country
2-D cuboids
3-D(base) cuboid
product, date, country
More Example of Cube and Cuboids
all
time
time,item
0-D(apex) cuboid
item
time,location
location
item,location
time,supplier
time,item,location
supplier
1-D cuboids
location,supplier
2-D cuboids
item,supplier
time,location,supplier
3-D cuboids
time,item,supplier
item,location,supplier
4-D(base) cuboid
time, item, location, supplier
Conceptual Modeling of
Data Warehouses

Modeling data warehouses: dimensions & measures

Star schema: A fact table in the middle connected to a
set of dimension tables

Snowflake schema: A refinement of star schema
where some dimensional hierarchy is normalized into a
set of smaller dimension tables, forming a shape
similar to snowflake

Fact constellations: Multiple fact tables share
dimension tables, viewed as a collection of stars,
therefore called galaxy schema or fact constellation
Example of Star Schema
time
item
time_key
day
day_of_the_week
month
quarter
year
Sales Fact Table
time_key
item_key
branch_key
branch
location_key
branch_key
branch_name
branch_type
units_sold
dollars_sold
avg_sales
Measures
item_key
item_name
brand
type
supplier_type
location
location_key
street
city
province_or_state
country
Example of Snowflake Schema
time
time_key
day
day_of_the_week
month
quarter
year
item
Sales Fact Table
time_key
item_key
branch_key
branch
location_key
branch_key
branch_name
branch_type
units_sold
dollars_sold
avg_sales
Measures
item_key
item_name
brand
type
supplier_key
supplier
supplier_key
supplier_type
location
location_key
street
city_key
city
city_key
city
province_or_state
country
Example of Fact Constellation
time
time_key
day
day_of_the_week
month
quarter
year
item
Sales Fact Table
time_key
item_key
item_name
brand
type
supplier_type
item_key
location_key
branch_key
branch_name
branch_type
units_sold
dollars_sold
avg_sales
Measures
time_key
item_key
shipper_key
from_location
branch_key
branch
Shipping Fact Table
location
to_location
location_key
street
city
province_or_state
country
dollars_cost
units_shipped
shipper
shipper_key
shipper_name
location_key
shipper_type
Typical OLAP Operations

Roll up (drill-up): summarize data


Drill down (roll down): reverse of roll-up


project and select
Pivot (rotate):


from higher level summary to lower level summary or detailed
data, or introducing new dimensions
Slice and dice:


by climbing up hierarchy or by dimension reduction
reorient the cube, visualization, 3D to series of 2D planes.
Other operations


drill across: involving (across) more than one fact table
drill through: through the bottom level of the cube to its backend relational tables (using SQL)

Motivation: Why Knowledge discovery in databases?

Data Warehouse and OLAP


What is a data warehouse?

A multi-dimensional data model

Data warehouse architecture

Data warehouse application
Data mining
The Process of Data Warehouse
Design


Top-down, bottom-up approaches or a combination of
both
 Top-down: Starts with overall design and planning
(mature)
 Bottom-up: Starts with experiments and prototypes
(rapid)
From software engineering point of view
 Waterfall: structured and systematic analysis at each
step before proceeding to the next.
 Spiral: rapid generation of increasingly functional
systems, short turn around time, quick turn around.
Multi-Tiered Architecture
other
Metadata
sources
Operational
DBs
Extract
Transform
Load
Refresh
Monitor
&
Integrator
Data
Warehouse
OLAP Server
Serve
Analysis
Query
Reports
Data mining
Data Marts
Data Sources
Data Storage
OLAP Engine Front-End Tools
Three Data Warehouse Models


Enterprise warehouse
 collects all of the information about subjects spanning
the entire organization
Data Mart
 a subset of corporate-wide data that is of value to a
specific groups of users. Its scope is confined to specific,
selected groups, such as marketing data mart


Independent vs. dependent (directly from warehouse) data mart
Virtual warehouse
 A set of views over operational databases
 Only some of the possible summary views may be
materialized
Data Warehouse Development:
A Recommended Approach
Multi-Tier Data
Warehouse
Distributed
Data Marts
Data
Mart
Data
Mart
Model refinement
Enterprise
Data
Warehouse
Model refinement
Define a high-level corporate data model
OLAP Server Architectures




Relational OLAP (ROLAP)
 Use relational or extended-relational DBMS to store and manage
warehouse data and OLAP middle ware to support missing pieces
 Include optimization of DBMS backend, implementation of
aggregation navigation logic, and additional tools and services
 greater scalability
Multidimensional OLAP (MOLAP)
 Array-based multidimensional storage engine (sparse matrix
techniques)
 fast indexing to pre-computed summarized data
Hybrid OLAP (HOLAP)
 User flexibility, e.g., low level: relational, high-level: array
Specialized SQL servers
 specialized support for SQL queries over star/snowflake schemas

Motivation: Why Knowledge discovery in databases?

Data Warehouse and OLAP


What is a data warehouse?

A multi-dimensional data model

Data warehouse architecture

Data warehouse application
Data mining
Data Warehouse Usage

Three kinds of data warehouse applications

Information processing



Analytical processing

multidimensional analysis of data warehouse data

supports basic OLAP operations, slice-dice, drilling, pivoting
Data mining



supports querying, basic statistical analysis, and reporting
using crosstabs, tables, charts and graphs
knowledge discovery from hidden patterns
supports associations, constructing analytical models,
performing classification and prediction, and presenting the
mining results using visualization tools.
Differences among the three tasks
From On-Line Analytical Processing to On
Line Analytical Mining

Why online analytical mining?

High quality of data in data warehouses


Available information processing structure
surrounding data warehouses


mining with drilling, dicing, pivoting, etc.
On-line selection of data mining functions
integration and swapping of multiple mining functions,
algorithms, and tasks.
Architecture of OLAM


ODBC, OLEDB, Web accessing, service facilities, reporting
and OLAP tools
OLAP-based exploratory data analysis


DW contains integrated, consistent, cleaned data
An OLAM Architecture
Mining query
Mining result
Layer4
User Interface
User GUI API
OLAM
Engine
OLAP
Engine
Layer3
OLAP/OLAM
Data Cube API
Layer2
MDDB
MDDB
Meta Data
Filtering&Integration
Database API
Filtering
Layer1
Data cleaning
Databases
Data
Data integration Warehouse
Data
Repository

Motivation: Why Knowledge discovery in databases?

Data Warehouse and OLAP

data mining

Introduction to data mining

Characterization

Association rule mining

Classification

Cluster Analysis
What Is Data Mining?


Data mining (knowledge discovery in databases):
 Extraction of interesting (non-trivial, implicit,
previously unknown and potentially useful)
information or patterns from data in large
databases
Alternative names and their “inside stories”:
 Data mining: a misnomer?
 Knowledge discovery(mining) in databases
(KDD), knowledge extraction, data/pattern
analysis, data archaeology, data dredging,
information harvesting, business intelligence, etc.
Why Data Mining? — Potential
Applications

Database analysis and decision support

Market analysis and management


Risk analysis and management



target marketing, customer relation management, market
basket analysis, cross selling, market segmentation
Forecasting, customer retention, improved underwriting,
quality control, competitive analysis
Fraud detection and management
Other Applications

Text mining (news group, email, documents) and Web analysis.

Intelligent query answering
Data Mining: A KDD Process
Pattern Evaluation

Data mining: the core of
knowledge discovery
Data Mining
process.
Task-relevant Data
Data Warehouse
Data Cleaning
Data Integration
Databases
Selection
Architecture of a Typical Data Mining
System
Graphical user interface
Pattern evaluation
Data mining engine
Knowledge-base
Database or data
warehouse server
Data cleaning & data integration
Databases
Filtering
Data
Warehouse
Data Mining: On What Kind of Data?




Relational databases
Data warehouses
Transactional databases
Advanced DB and information repositories
 Object-oriented and object-relational databases
 Spatial databases
 Time-series data and temporal data
 Text databases and multimedia databases
 Heterogeneous and legacy databases
 WWW
Data Mining Functionalities (1)

Concept description: Characterization and discrimination


Generalize, summarize, and contrast data
characteristics, e.g., dry vs. wet regions
Association



contains(x, “computer”)  contains(x, “software”)
[support = 1%, confidence = 75%]
age(X, “20..29”) ^ income(X, “20..29K”)  buys(X,
“PC”) [2%, 60%]
Multi-dimensional vs. single-dimensional association
Data Mining Functionalities (2)

Classification and Prediction



Finding models (functions) that describe and distinguish classes
or concepts for future prediction
E.g., classify countries based on climate, or classify cars based
on gas mileage

Presentation: decision-tree, classification rule, neural network

Prediction: Predict some unknown or missing numerical values
Cluster analysis


Class label is unknown: Group data to form new classes, e.g.,
cluster houses to find distribution patterns
Clustering based on the principle: maximizing the intra-class
similarity and minimizing the interclass similarity
Data Mining Functionalities (3)

Outlier analysis

Outlier: a data object that does not comply with the general
behavior of the data

It can be considered as noise or exception but is quite useful in
fraud detection, rare events analysis


Trend and evolution analysis

Trend and deviation: regression analysis

Sequential pattern mining, periodicity analysis

Similarity-based analysis
Other pattern-directed or statistical analyses
Are All the “Discovered” Patterns
Interesting?

A data mining system/query may generate thousands of patterns,
not all of them are interesting.



Suggested approach: Human-centered, query-based, focused
mining
Interestingness measures: A pattern is interesting if it is easily
understood by humans, valid on new or test data with some degree
of certainty, potentially useful, novel, or validates some hypothesis
that a user seeks to confirm
Objective vs. subjective interestingness measures:


Objective: based on statistics and structures of patterns, e.g.,
support, confidence, etc.
Subjective: based on user’s belief in the data, e.g.,
unexpectedness, novelty, actionability, etc.
Data Mining: Confluence of Multiple
Disciplines
Database
Technology
Machine
Learning
Information
Science
Statistics
Data Mining
Visualization
Other
Disciplines

Motivation: Why Knowledge discovery in databases?

Data Warehouse and OLAP

data mining

Introduction to data mining

Characterization

Association rule mining

Classification

Cluster Analysis
Data Generalization and Summarizationbased Characterization

Data generalization
 A process which abstracts a large set of task-relevant
data in a database from a low conceptual levels to
higher ones.
1
2
3
4
Conceptual levels
5

Approaches:
 Data cube approach(OLAP approach)
 Attribute-oriented induction approach
Characterization: Data Cube Approach
(without using AO-Induction)

Perform computations and store results in data cubes

Strength

An efficient implementation of data generalization

Computation of various kinds of measures



e.g., count( ), sum( ), average( ), max( )
Generalization and specialization can be performed on a data
cube by roll-up and drill-down
Limitations


handle only dimensions of simple nonnumeric data and
measures of simple aggregated numeric values.
Lack of intelligent analysis, can’t tell which dimensions should
be used and what levels should the generalization reach
Attribute-Oriented Induction



Proposed in 1989 (KDD ‘89 workshop)
Not confined to categorical data nor particular measures.
How it is done?
 Collect the task-relevant data( initial relation) using a
relational database query
 Perform generalization by attribute removal or attribute
generalization.
 Apply aggregation by merging identical, generalized
tuples and accumulating their respective counts.
 Interactive presentation with users.
Basic Principles of AttributeOriented Induction





Data focusing: task-relevant data, including dimensions,
and the result is the initial relation.
Attribute-removal: remove attribute A if there is a large set
of distinct values for A but (1) there is no generalization
operator on A, or (2) A’s higher level concepts are
expressed in terms of other attributes.
Attribute-generalization: If there is a large set of distinct
values for A, and there exists a set of generalization
operators on A, then select an operator and generalize A.
Attribute-threshold control: typical 2-8, specified/default.
Generalized relation threshold control: control the final
relation/rule size. see example
Class Characterization: An Example
Name
Gender
Jim
Initial
Woodman
Relation Scott
M
Major
M
F
…
Removed
Retained
Residence
Phone #
GPA
Vancouver,BC, 8-12-76
Canada
CS
Montreal, Que, 28-7-75
Canada
Physics Seattle, WA, USA 25-8-70
…
…
…
3511 Main St.,
Richmond
345 1st Ave.,
Richmond
687-4598
3.67
253-9106
3.70
125 Austin Ave.,
Burnaby
…
420-5232
…
3.83
…
Sci,Eng,
Bus
City
Removed
Excl,
VG,..
Gender Major
M
F
…
Birth_date
CS
Lachance
Laura Lee
…
Prime
Generalized
Relation
Birth-Place
Science
Science
…
Country
Age range
Birth_region
Age_range
Residence
GPA
Canada
Foreign
…
20-25
25-30
…
Richmond
Burnaby
…
Very-good
Excellent
…
Birth_Region
Canada
Foreign
Total
Gender
M
16
14
30
F
10
22
32
Total
26
36
62
Count
16
22
…
Presentation of Generalized Results

Generalized relation:


Cross tabulation:


Relations where some or all attributes are generalized, with counts
or other aggregation values accumulated.
Mapping results into cross tabulation form (similar to contingency
tables).

Visualization techniques:

Pie charts, bar charts, curves, cubes, and other visual forms.
Quantitative characteristic rules:

Mapping generalized result into characteristic rules with quantitative
information associated with it, e.g.,
grad( x)  male( x) 
birth_ region( x) "Canada"[t :53%] birth_ region( x) " foreign"[t : 47%].
Presentation—Generalized Relation
Presentation—Crosstab

Motivation: Why Knowledge discovery in databases?

Data Warehouse and OLAP

data mining

Introduction to data mining

Characterization

Association rule mining

Classification

Cluster Analysis
What Is Association Rule Mining?



Association rule mining:
 Finding frequent patterns, associations, correlations, or
causal structures among sets of items or objects in
transaction databases, relational databases, and other
information repositories.
Applications:
 Basket data analysis, cross-marketing, catalog design,
loss-leader analysis, clustering, classification, etc.
Examples.
 Rule form: “antecedent consequent [support,
confidence]”.
 buys(x, “diapers”)  buys(x, “beers”) [0.5%, 60%]
 major(x, “CS”) ^ takes(x, “DB”) grade(x, “A”) [1%,
75%]
Association Rules


Given
 A database of customer transactions
 Each transaction is a list of items (purchases by a
customer in a visit)
Find all rules that correlated the presence of one set of
items with another set of items
 Example: 98% of people who purchase tires and auto
accessories also get automotive services done


Any number of items in the consequent/antecedent of
rule
Possible to specify constraints on rules (e.g. find only
rules involving Home Laundry Appliances).
Characteristics
Number of items: Hundreds of thousands
Number of transactions: Millions
Data set sizes: High Gigabytes



Discovering all rules rather than verifying if a rule
holds
Performance
Scalability
Association Rules -- Problem Statement





I = {i1, i2 ,…, im} : a set of literals, called items.
Transaction T: a set of items such that T  I.
Database D : a set of transactions.
A transaction T contains X, a set of some items,
if X  T.
An association rule is an implication of the form
X  Y, where X,Y  I and X  Y = .
Association Rules –
Confidence and Support

confidence of rule X  Y is the percentage of
transactions in D containing X that also contain Y.
c = (number of transactions containing both X and Y) /
(total number of transactions containing X )

support of rule X  Y is the percentage of
transactions in D that contain both X and Y.
s = (number of transactions containing both X and Y) /
(total number of transactions in D)
Discover all rules that have minimum user-specified
confidence and support.
Mining Association Rules


Find the frequent itemsets : sets of items that
have minimum support .
 A subset of a frequent itemset must also be a
frequent itemset, i.e., if {AB} is a frequent
itemset, both {A} and {B} should be a frequent
itemset
 Iteratively find frequent itemsets with cardinality
from 1 to k (k-itemset)
Use the frequent itemsets to generate the desired
rules.
Mining Association Rules -- Example
Transaction ID
2000
1000
4000
5000
Items Bought
A,B,C
A,C
A,D
B,E,F
Min. support 50%
Min. confidence 50%
Frequent Itemset Support
{A}
75%
{B}
50%
{C}
50%
{A,C}
50%
For rule A  C:
support = support({A, C}) = 50%
confidence = support({A, C})/support({A}) = 66.6%
The Apriori Algorithm — Example
Database D
TID
100
200
300
400
itemset sup.
C1
{1}
2
{2}
3
Scan D
{3}
3
{4}
1
{5}
3
Items
134
235
1235
25
C2 itemset sup
L2 itemset sup
2
2
3
2
{1
{1
{1
{2
{2
{3
C3 itemset
{2 3 5}
Scan D
{1 3}
{2 3}
{2 5}
{3 5}
2}
3}
5}
3}
5}
5}
1
2
1
2
3
2
L1 itemset sup.
{1}
{2}
{3}
{5}
2
3
3
3
C2 itemset
{1 2}
Scan D
L3 itemset sup
{2 3 5} 2
{1
{1
{2
{2
{3
3}
5}
3}
5}
5}
Using frequent itemsets to generate rules
Association rules can be generated as follows:


For each frequent itemset l, generate all
nonempty subsets of l.
For each nonempty subset s of l, output the
rule “s  (l-s)” if
support_count(l)/support_count(s) >=min_conf,
where min_conf is the minimum confidence
threshold.
Summary

Association rule mining


probably the most significant contribution from the
database community in KDD
A large number of papers have been published

Many interesting issues have been explored

An interesting research direction

Association analysis in other types of data: spatial
data, multimedia data, time series data, etc.

Motivation: Why Knowledge discovery in databases?

Data Warehouse and OLAP

data mining

Introduction to data mining

Characterization

Association rule mining

Classification

Cluster Analysis
Classification


Classification:
 predicts categorical class labels
 classifies data (constructs a model) based on the
training set and the values (class labels) in a
classifying attribute and uses it in classifying new data
Typical Applications
 credit approval
 target marketing
 medical diagnosis
 treatment effectiveness analysis
Classification—A Two-Step Process


Model construction: describing a set of predetermined classes
 Each tuple/sample is assumed to belong to a predefined class,
as determined by the class label attribute
 The set of tuples used for model construction: training set
 The model is represented as classification rules, decision trees,
or mathematical formulae
Model usage: for classifying future or unknown objects
 Estimate accuracy of the model
 The known label of test sample is compared with the
classified result from the model
 Accuracy rate is the percentage of test set samples that are
correctly classified by the model
 Test set is independent of training set, otherwise over-fitting
will occur
Classification Process (1): Model
Construction
Training
Data
NAME
M ike
M ary
B ill
Jim
D ave
A nne
RANK
YEARS TENURED
A ssistant P rof
3
no
A ssistant P rof
7
yes
P rofessor
2
yes
A ssociate P rof
7
yes
A ssistant P rof
6
no
A ssociate P rof
3
no
Classification
Algorithms
Classifier
(Model)
IF rank = ‘professor’
OR years > 6
THEN tenured = ‘yes’
Classification Process (2): Use the
Model in Prediction
Classifier
Testing
Data
Unseen Data
(Jeff, Professor, 4)
NAME
T om
M erlisa
G eorge
Joseph
RANK
YEARS TENURED
A ssistant P rof
2
no
A ssociate P rof
7
no
P rofessor
5
yes
A ssistant P rof
7
yes
Tenured?
Supervised vs. Unsupervised
Learning

Supervised learning (classification)



Supervision: The training data (observations,
measurements, etc.) are accompanied by labels
indicating the class of the observations
New data is classified based on the training set
Unsupervised learning (clustering)


The class labels of training data is unknown
Given a set of measurements, observations, etc. with
the aim of establishing the existence of classes or
clusters in the data
Classification by Decision Tree
Induction



Decision tree
 A flow-chart-like tree structure
 Internal node denotes a test on an attribute
 Branch represents an outcome of the test
 Leaf nodes represent class labels or class distribution
Decision tree generation consists of two phases
 Tree construction
 At start, all the training examples are at the root
 Partition examples recursively based on selected attributes
 Tree pruning
 Identify and remove branches that reflect noise or outliers
Use of decision tree: Classifying an unknown sample
 Test the attribute values of the sample against the decision tree
Training Dataset
This
follows
an
example
from
Quinlan’s
ID3
age
<=30
<=30
30…40
>40
>40
>40
31…40
<=30
<=30
>40
<=30
31…40
31…40
>40
income student credit_rating
high
no fair
high
no excellent
high
no fair
medium
no fair
low
yes fair
low
yes excellent
low
yes excellent
medium
no fair
low
yes fair
medium
yes fair
medium
yes excellent
medium
no excellent
high
yes fair
medium
no excellent
buys_computer
no
no
yes
yes
yes
no
yes
no
yes
yes
yes
yes
yes
no
Output: A Decision Tree for “buys_computer”
age?
>40
<=30
30..40
student?
no
no
yes
yes
yes
credit rating?
excellent
no
fair
yes
Another Example

An
Example
from
Quinlan
O u tlo o k
su n n y
su n n y
o verc ast
rain
rain
rain
o verc ast
su n n y
su n n y
rain
su n n y
o verc ast
o verc ast
rain
T em p reatu re
hot
hot
hot
m ild
cool
cool
cool
m ild
cool
m ild
m ild
m ild
hot
m ild
H u m id ity
h ig h
h ig h
h ig h
h ig h
n o rm al
n o rm al
n o rm al
h ig h
n o rm al
n o rm al
n o rm al
h ig h
n o rm al
h ig h
W in d y
false
tru e
false
false
false
tru e
tru e
false
false
false
tru e
tru e
false
tru e
C lass
N
N
P
P
P
N
P
N
P
P
P
P
P
N
A Sample Decision Tree
Outlook
sunny overcast
humidity
rain
windy
P
high
normal
true
false
N
P
N
P
Algorithm for Decision Tree Induction


Basic algorithm (a greedy algorithm)
 Tree is constructed in a top-down recursive divide-and-conquer
manner
 At start, all the training examples are at the root
 Attributes are categorical (if continuous-valued, they are
discretized in advance)
 Examples are partitioned recursively based on selected attributes
 Test attributes are selected on the basis of a heuristic or statistical
measure (e.g., information gain)
Conditions for stopping partitioning
 All samples for a given node belong to the same class
 There are no remaining attributes for further partitioning –
majority voting is employed for classifying the leaf
 There are no samples left
Summary

Classification is an extensively studied problem (mainly in
statistics, machine learning & neural networks)

Classification is probably one of the most widely used
data mining techniques with a lot of extensions

Scalability is still an important issue for database
applications: thus combining classification with database
techniques should be a promising topic

Research directions: classification of non-relational data,
e.g., text, spatial, multimedia, etc..

Motivation: Why Knowledge discovery in databases?

Data Warehouse and OLAP

data mining

Introduction to data mining

Characterization

Association rule mining

Classification

Cluster Analysis
Examples of Clustering Applications





Marketing: Help marketers discover distinct groups in their
customer bases, and then use this knowledge to develop
targeted marketing programs
Land use: Identification of areas of similar land use in an
earth observation database
Insurance: Identifying groups of motor insurance policy
holders with a high average claim cost
City-planning: Identifying groups of houses according to
their house type, value, and geographical location
Earth-quake studies: Observed earth quake epicenters
should be clustered along continent faults
What Is Good Clustering?



A good clustering method will produce high quality
clusters with

high intra-class similarity

low inter-class similarity
The quality of a clustering result depends on both the
similarity measure used by the method and its
implementation.
The quality of a clustering method is also measured by its
ability to discover some or all of the hidden patterns.
Major Clustering Approaches

Partitioning algorithms: Construct various partitions and
then evaluate them by some criterion

Hierarchy algorithms: Create a hierarchical decomposition
of the set of data (or objects) using some criterion

Density-based: based on connectivity and density functions

Grid-based: based on a multiple-level granularity structure

Model-based: A model is hypothesized for each of the
clusters and the idea is to find the best fit of that model to
each other
The K-Means Clustering Method

Example
10
10
9
9
8
8
7
7
6
6
5
5
4
4
3
3
2
2
1
1
0
0
0
1
2
3
4
5
6
7
8
9
10
0
10
10
9
9
8
8
7
7
6
6
5
5
4
4
3
3
2
2
1
1
0
1
2
3
4
5
6
7
8
9
10
0
0
1
2
3
4
5
6
7
8
9
10
0
1
2
3
4
5
6
7
8
9
10
Hierarchical Clustering

Use distance matrix as clustering criteria. This method
does not require the number of clusters k as an input,
but needs a termination condition
Step 0
a
Step 1
Step 2 Step 3 Step 4
ab
b
abcde
c
cde
d
de
e
Step 4
agglomerative
(AGNES)
Step 3
Step 2 Step 1 Step 0
divisive
(DIANA)
AGNES (Agglomerative Nesting)

Introduced in Kaufmann and Rousseeuw (1990)

Implemented in statistical analysis packages, e.g., Splus

Use the Single-Link method and the dissimilarity matrix.

Merge nodes that have the least dissimilarity

Go on in a non-descending fashion

Eventually all nodes belong to the same cluster
10
10
10
9
9
9
8
8
8
7
7
7
6
6
6
5
5
5
4
4
4
3
3
3
2
2
2
1
1
1
0
0
0
1
2
3
4
5
6
7
8
9
10
0
0
1
2
3
4
5
6
7
8
9
10
0
1
2
3
4
5
6
7
8
9
10
DIANA (Divisive Analysis)

Introduced in Kaufmann and Rousseeuw (1990)

Implemented in statistical analysis packages, e.g., Splus

Inverse order of AGNES

Eventually each node forms a cluster on its own
10
10
10
9
9
9
8
8
8
7
7
7
6
6
6
5
5
5
4
4
4
3
3
3
2
2
2
1
1
1
0
0
0
0
1
2
3
4
5
6
7
8
9
10
0
1
2
3
4
5
6
7
8
9
10
0
1
2
3
4
5
6
7
8
9
10