Download Chapter 1: Introduction

Chapter 1: Introduction
Hankz Hankui Zhuo (卓汉逵)
Email: [email protected]
Homepage: http://www.zsusoft.com/~hankz
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
How to get good marks?
• Marks = regular (40%) + final exam (60%)
Roll calls
(20%)
Assignments
(20%)
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
2
Syllabus
•
•
•
•
•
•
•
•
•
•
Chapter 1: Introduction
Chapter 2: Introduction to the Relational Model
Chapter 3: Introduction to SQL
Chapter 6: Database Design and EntityRelationship Model
Chapter 7: Relational Database Design
Chapter 11: Storage and File Structure
Chapter 12: Indexing and Hashing.
Chapter 13: Query Processing
Chapter 15: Transactions
*Chapter 20: Data Warehousing and Mining
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
3
Database Management System (DBMS)
• Describe a particular enterprise:
– A collection of interrelated data
– Programs
– Environment
Environment
Programs
DBMS
data
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
4
Database Applications
• Banking
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
5
Database Applications
• Banking
• Airlines
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
6
Database Applications
• Banking
• Airlines
• Universities
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
7
Database Applications
•
•
•
•
Banking
Airlines
Universities
Sales
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
8
Database Applications
•
•
•
•
•
•
•
•
Banking
Airlines
Universities
Sales
Online retailers
Manufacturing
Human resources
…
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
9
Purpose
• In the early days, applications were built
directly on top of file systems
• Drawbacks
:
– Data redundancy and inconsistency
• Multiple file formats, duplication of information in
different files
– Difficulty in accessing data
• Need to write a new program to carry out each new
task
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
10
Purpose
– Integrity problems
• Integrity constraints (e.g. account balance > 0) become
“buried” in program code rather than being stated
explicitly
• Hard to add new constraints or change existing ones
– Atomicity of updates
• Failures may leave database in an inconsistent state
with partial updates carried out
• Example: Transfer of funds from one account to
another should either complete or not happen at all
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
11
Purpose
– Concurrent access by multiple users
• Concurrent accessed needed for performance
• Uncontrolled concurrent accesses can lead to
inconsistencies
– Example: Two people reading a balance and updating it at the
same time
– Security problems
• Hard to provide user access to some, but not all, data
• Purpose of database systems?
– offer solutions to all the above problems!!!!
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
12
Architecture of database systems
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
13
Architecture of database systems
• Physical level: describes how a record (e.g., customer)
is stored.
• Logical level: describes data stored in database, and
the relationships among the data.
type customer = record
customer_id : string;
customer_name : string;
customer_street : string;
customer_city : integer;
end;
• View level: application programs hide details of data
types. Views can also hide information (such as an
employee’s salary) for security purposes.
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
14
Schemas
• the logical structure of the database
– Example: the database consists of information
about a set of customers and accounts and the
relationship between them)
– Analogous to type information of a variable in a
program
– Physical schema: database design at the physical
level
– Logical schema: database design at the logical
level
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
15
Instances
• the actual content of the database at a
particular point in time
– Analogous to the value of a variable
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
16
Instances
• the actual content of the database at a
particular point in time
– Analogous to the value of a variable
Independence
between schemas and
instance?
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
17
Physical Data Independence
• The ability to modify the physical schema
without changing the logical schema
– Applications depend on the logical schema
– In general, the interfaces between the various
levels and components should be well defined so
that changes in some parts do not seriously
influence others.
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
18
Physical Data Independence
• The ability to modify the physical schema
without changing the logical schema
– Applications depend on the logical schema
– In general, the interfaces between the various
levels and components should be well defined so
that changes in some parts do not seriously
influence others.
Nice!
But, do we have any
tool to model the data?
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
19
Tools
• Relational model
• Entity-Relationship data model (mainly for
database design)
• Object-based data models (Object-oriented
and Object-relational)
• Semistructured data model (XML)
• Other older models:
– Network model
– Hierarchical model
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
20
Relational Model
As an example, Relational Model:
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
Attributes
21
A Sample Relational Database
Many tables
“stacked” together
form a database!
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
22
A Sample Relational Database
Many tables
“stacked” together
form a database!
Having “data” is
not enough. We
also need Data
Manipulation
Language (DML).
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
23
Data Manipulation Language (DML)
• Language for accessing and manipulating the data
organized by the appropriate data model
– DML also known as query language
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
24
Data Manipulation Language (DML)
• Language for accessing and manipulating the data
organized by the appropriate data model
– DML also known as query language
• Two classes of languages
– Procedural – user specifies what data is required and
how to get those data
– Declarative (nonprocedural) – user specifies what
data is required without specifying how to get those
data
• SQL is the widely used nonprocedural language
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
25
SQL
• SQL: widely used non-procedural language
– Example: Find the name of the customer with
customer-id 192-83-7465
select customer.customer_name
from
customer
where customer.customer_id = ‘192-83-7465’
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
26
SQL
• SQL: widely used non-procedural language
– Example: Find the name of the customer with
customer-id 192-83-7465
select customer.customer_name
from
customer
where customer.customer_id = ‘192-83-7465’
– Example: Find the balances of all accounts held by
the customer with customer-id 192-83-7465
select account.balance
from
depositor, account
where depositor.customer_id = ‘192-83-7465’ and
depositor.account_number = account.account_number
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
27
SQL
• SQL: widely used non-procedural language
– Example:
the name of the customer with
We now Find
know:
customer-id 192-83-7465
select customer.customer_name
Data
DML
from
customer
(database)
(SQL)
where customer.customer_id
= ‘192-83-7465’
+
– Example: Find the balances of all accounts held by
the customer with customer-id 192-83-7465
select account.balance
from
depositor, account
where depositor.customer_id = ‘192-83-7465’ and
depositor.account_number = account.account_number
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
28
SQL
• SQL: widely used non-procedural language
– Example:
the name of the customer with
We now Find
know:
customer-id 192-83-7465
select customer.customer_name
Data
DML
from
customer
(database)
(SQL)
where customer.customer_id
= ‘192-83-7465’
+
– Example: Find the balances of all accounts held by
Enough?
Something
missing?
the
customer
with customer-id
192-83-7465
select account.balance
from
depositor, account
where depositor.customer_id = ‘192-83-7465’ and
depositor.account_number = account.account_number
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
29
SQL
• SQL: widely used non-procedural language
– Example:
the name of the customer with
We now Find
know:
customer-id 192-83-7465
select customer.customer_name
Data
DML
from
customer
(database)
(SQL)
where customer.customer_id
= ‘192-83-7465’
+
– Example: Find the balances of all accounts held by
Enough?
Something
missing?
Yes!
the
customer
with customer-id
192-83-7465
select account.balance
from
depositor, account
where depositor.customer_id = ‘192-83-7465’ and
depositor.account_number = account.account_number
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
30
SQL
• SQL: widely used non-procedural language
– Example:
the name of the customer with
We now Find
know:
customer-id 192-83-7465
select customer.customer_name
Data
DML
from
customer
(database)
(SQL)
where customer.customer_id
= ‘192-83-7465’
+
– Example: Find the
balances
of all accounts held by
How
?
the customer with customer-id 192-83-7465
select account.balance
from
depositor,
account
Design
where depositor.customer_id = ‘192-83-7465’ and
depositor.account_number = account.account_number
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
31
Database Design
The process of designing the general structure
of the database:
– Logical Design – Deciding on the database
schema.
• Business decision – What attributes should we record in
the database?
• Computer Science decision – What relation schemas
should we have and how should the attributes be
distributed among the various relation schemas?
– Physical Design – Deciding on the physical layout
of the database
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
32
Database Design
The process of designing the general structure
of the database:
– Logical Design – Deciding on the database
schema.
An–example
method:should we record in
• Business decision
What attributes
the database? Entity-relationship model
• Computer Science decision – What relation schemas
should we have and how should the attributes be
distributed among the various relation schemas?
– Physical Design – Deciding on the physical layout
of the database
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
33
Database Design
The process of designing the general structure
of the database:
– Logical Design – Deciding on the database
schema.
An–example
method:should we record in
• Business decision
What attributes
the database? Entity-relationship model
• Computer Science decision – What relation schemas
should we have and how should the attributes be
distributed among the various relation schemas?
– Physical Design – Deciding on the physical layout
of the database
Storage Management
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
34
The Entity-Relationship Model
• Models an enterprise as a collection of entities and
relationships
– Entity: a “thing” or “object” in the enterprise that is
distinguishable from other objects
• Described by a set of attributes
– Relationship: an association among several entities
• Represented diagrammatically by an entityrelationship diagram:
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
35
Storage Management
• Storage manager is a program module that
provides the interface between the low-level data
stored in the database and the application
programs and queries submitted to the system.
• The storage manager is responsible to the
following tasks:
– Interaction with the file manager
– Efficient storing, retrieving and updating of data
• Issues:
– Storage access
– File organization
– Indexing
and
hashing
Hankz
Hankui
Zhuo: http://www.zsusoft.com/~hankz
36
SQL
• SQL: widely used non-procedural language
– Example:
the name of the customer with
We now Find
know:
customer-id 192-83-7465
select customer.customer_name
Data
DML
from
customer
(database)
(SQL)
where customer.customer_id
= ‘192-83-7465’
+
– Example: Find the balances of all accounts held by
the customer with customer-id 192-83-7465
select account.balance
from
depositor,
account
Design
where depositor.customer_id = ‘192-83-7465’ and
depositor.account_number = account.account_number
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
37
SQL
• SQL: widely used non-procedural language
– Example:
the name of the customer with
We now Find
know:
customer-id 192-83-7465
select customer.customer_name
Data
DML
from
customer
(database)
(SQL)
where customer.customer_id
= ‘192-83-7465’
+
– Example: Find the balances of all accounts held by
the customer with customer-id 192-83-7465
select account.balance
Query
from
depositor,
account
Design
Processing and
where depositor.customer_id = ‘192-83-7465’
depositor.account_number = account.account_number
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
38
Query Processing
1.Parsing and translation
2.Optimization
3.Evaluation
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
39
Query Processing
1.Parsing and translation
2.Optimization
3.Evaluation
In query processing, one of the most important
concepts : transaction
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
40
Transaction Management
• A transaction is a collection of operations
• Transaction-management component ensures
that the database remains in a consistent
(correct) state despite system failures (e.g.,
power failures and operating system crashes)
and transaction failures.
• Concurrency-control manager controls the
interaction among the concurrent transactions,
to ensure the consistency of the database.
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
41
Finally, let me say something
about the whole database
architecture!
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
42
Database Architecture
The architecture of a database systems is greatly
influenced by the underlying computer system
on which the database is running:
– Centralized
– Client-server
– Parallel (multi-processor)
– Distributed
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
43
Database Administrator
• Coordinates all the activities of the database
system.
• Database administrator's duties include:
–
–
–
–
–
–
–
Schema definition
Storage structure and access method definition
Schema and physical organization modification
Granting user authority
Specifying integrity constraints
Acting as liaison with users
Monitoring performance and responding to changes
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
44
History of Database Systems
• 1950s and early 1960s:
– Data processing using magnetic tapes for storage
• Tapes provide only sequential access
– Punched cards for input
• Late 1960s and 1970s:
– Hard disks allow direct access to data
– Network and hierarchical data models in widespread use
– Ted Codd defines the relational data model
• Would win the ACM Turing Award for this work
• IBM Research begins System R prototype
• UC Berkeley begins Ingres prototype
– High-performance (for the era) transaction processing
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
45
History of Database Systems
• 1980s:
– Research relational prototypes evolve into commercial systems
• SQL becomes industrial standard
– Parallel and distributed database systems
– Object-oriented database systems
• 1990s:
– Large decision support and data-mining applications
– Large multi-terabyte data warehouses
– Emergence of Web commerce
• 2000s:
– XML and XQuery standards
– Automated database administration
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
46
• For students interested in database research:
– Top conferences:
•
•
•
•
VLDB
Sigmod
Sigkdd
ICDE
You can find papers via google with
“<conference name>+year”, e.g.,
“VLDB 2011”.
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
47
• For students interested in database research:
– Top conferences:
•
•
•
•
VLDB
Sigmod
Sigkdd
ICDE
– Top journals:
•
•
•
•
ACM Transactions on Database Systems
VLDB Journal
ACM Transactions on Information and Systems
IEEE Transactions on Knowledge and Data Engineering
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
48
The End!
Hankz Hankui Zhuo: http://www.zsusoft.com/~hankz
49