Download - Courses - University of California, Berkeley

Physical Database Design
University of California, Berkeley
School of Information
I 257: Database Management
IS 257 – Fall 2009
2009-09-15 SLIDE 1
Lecture Outline
• Review
–Relational Algebra and Calculus
–Introduction to SQL
• Physical Database Design
• Access Methods
IS 257 – Fall 2009
2009-09-15 SLIDE 2
Lecture Outline
• Review
–Relational Algebra and Calculus
–Introduction to SQL
• Physical Database Design
• Access Methods
IS 257 – Fall 2009
2009-09-15 SLIDE 3
Relational Algebra Operations
•
•
•
•
•
•
•
•
Select
Project
Product
Union
Intersect
Difference
Join
Divide
IS 257 – Fall 2009
2009-09-15 SLIDE 4
Select
• Extracts specified tuples (rows) from a
specified relation (table).
IS 257 – Fall 2009
2009-09-15 SLIDE 5
Project
• Extracts specified attributes(columns) from
a specified relation.
IS 257 – Fall 2009
2009-09-15 SLIDE 6
Join
• Builds a relation from two specified
relations consisting of all possible
concatenated pairs, one from each of the
two relations, such that in each pair the
two tuples satisfy some condition. (E.g.,
equal values in a given col.)
A1 B1
A2 B1
A3 B2
IS 257 – Fall 2009
B1 C1
B2 C2
B3 C3
(Natural
or Inner)
Join
A1 B1 C1
A2 B1 C1
A3 B2 C2
2009-09-15 SLIDE 7
Outer Join
• Outer Joins are similar to PRODUCT -- but
will leave NULLs for any row in the first
table with no corresponding rows in the
second.
Outer
Join
A1
A2
A3
A4
IS 257 – Fall 2009
B1
B1
B2
B7
B1 C1
B2 C2
B3 C3
A1 B1 C1
A2 B1 C1
A3 B2 C2
A4 * *
2009-09-15 SLIDE 8
Join Items
Part #
Invoice # Part #
Quantity
93774
3
10
84747
23
1
88367
75
2
88647
4
3
776879
22
5
65689
76
12
93774
23
10
88367
34
2
1
2
3
4
5
6
7
8
9
Cust #
COMPANY
STREET1
Integrated Standards
1 Ltd.
35 Broadway
IS 257 – Fall 2009
Rep #
3
4
5
9
2
6
1
1
2
1
2
2
STREET2
STATE
ZIPCODE
NY
02111
34 Bureaucracy Plaza Floors 1-172
3 Control Elevation
Cyber Assicates
Place
Center
Phildelphia
PA
03756
Cyberoid
NY
08645
35 Libra Plaza
Nashua
NH
09242
1 Broadway
Middletown
IN
32467
88 Oligopoly Place
3 Independence
Parkway
Sagrado
TX
78798
Rivendell
CA
93456
8 Little Mighty Micro
34 Last One Drive
Orinda
CA
94563
9 SportLine Ltd.
38 Champion Place
Compton
CA
95328
3 Cyber Associates
General
4 Consolidated
Consolidated
5 MultiCorp
Internet Behometh
6 Ltd.
Consolidated
7 Brands, Inc.
Floor 12
CITY
New York
2 MegaInt Inc.
Invoice # Cust #
93774
84747
88367
88647
776879
65689
Name
Price
Count
Big blue widget
3.76
2
Small blue Widget
7.35
4
Tiny red widget
5.25
7
large red widget
157.23
23
double widget rack
10.44
12
Small green Widget
30.45
58
Big yellow widget
7.96
1
Tiny orange widget
81.75
42
Big purple widget
55.99
9
Suite 882
2009-09-15 SLIDE 9
Relational Algebra
• What is the name of the customer who
ordered Large Red Widgets?
– Select “large Red Widgets” from Part as
temp1
– Join temp1 with Line-item on Part # as temp2
– Join temp2 with Invoice on Invoice # as temp3
– Join temp3 with customer on cust # as temp4
– Project Name from temp4
IS 257 – Fall 2009
2009-09-15 SLIDE 10
Relational Calculus
• Relational Algebra provides a set of
explicit operations (select, project, join,
etc) that can be used to build some
desired relation from the database.
• Relational Calculus provides a notation for
formulating the definition of that desired
relation in terms of the relations in the
database without explicitly stating the
operations to be performed
• SQL is based on the relational calculus.
IS 257 – Fall 2009
2009-09-15 SLIDE 11
SQL - History
• Structured Query Language
• SEQUEL from IBM San Jose
• ANSI 1992 Standard is the version used
by most DBMS today (SQL92)
• Basic language is standardized across
relational DBMSs. Each system may have
proprietary extensions to standard.
IS 257 – Fall 2009
2009-09-15 SLIDE 12
SQL Uses
• Database Definition and Querying
– Can be used as an interactive query language
– Can be imbedded in programs
• Relational Calculus combines Select,
Project and Join operations in a single
command: SELECT
IS 257 – Fall 2009
2009-09-15 SLIDE 13
SELECT
• Syntax:
– SELECT [DISTINCT] attr1, attr2,…, attr3
FROM rel1 r1, rel2 r2,… rel3 r3 WHERE
condition1 {AND | OR} condition2 ORDER
BY attr1 [DESC], attr3 [DESC]
IS 257 – Fall 2009
2009-09-15 SLIDE 14
SELECT
• Syntax:
– SELECT a.author, b.title FROM authors a,
bibfile b, au_bib c WHERE a.AU_ID =
c.AU_ID and c.accno = b.accno ORDER BY
a.author ;
• Examples in Access...
IS 257 – Fall 2009
2009-09-15 SLIDE 15
SELECT Conditions
•
•
•
•
•
•
= equal to a particular value
>= greater than or equal to a particular value
> greater than a particular value
<= less than or equal to a particular value
<> not equal to a particular value
LIKE “*term*” (may be other wild cards in other
systems)
• IN (“opt1”, “opt2”,…,”optn”)
• BETWEEN val1 AND val2
• IS NULL
IS 257 – Fall 2009
2009-09-15 SLIDE 16
Relational Algebra Selection using SELECT
• Syntax:
– SELECT * FROM rel1 WHERE condition1
{AND | OR} condition2;
IS 257 – Fall 2009
2009-09-15 SLIDE 17
Relational Algebra Projection using SELECT
• Syntax:
– SELECT [DISTINCT] attr1, attr2,…, attr3
FROM rel1 r1, rel2 r2,… rel3 r3;
IS 257 – Fall 2009
2009-09-15 SLIDE 18
Relational Algebra Join using SELECT
• Syntax:
– SELECT * FROM rel1 r1, rel2 r2 WHERE
r1.linkattr = r2.linkattr ;
IS 257 – Fall 2009
2009-09-15 SLIDE 19
Sorting
• SELECT BIOLIFE.[Common Name],
BIOLIFE.[Length (cm)]
FROM BIOLIFE
ORDER BY BIOLIFE.[Length (cm)] DESC;
Note: the square brackets are not part of the standard,
But are used in Access for names with embedded blanks
IS 257 – Fall 2009
2009-09-15 SLIDE 20
Subqueries
• SELECT SITES.[Site Name],
SITES.[Destination no]
FROM SITES
WHERE sites.[Destination no] IN
(SELECT [Destination no] from DEST
where [avg temp (f)] >= 78);
• Can be used as a form of JOIN.
IS 257 – Fall 2009
2009-09-15 SLIDE 21
Aggregate Functions
•
•
•
•
•
•
Count
Avg
SUM
MAX
MIN
Others may be available in different
systems
IS 257 – Fall 2009
2009-09-15 SLIDE 22
Using Aggregate functions
• SELECT attr1, Sum(attr2) AS name
FROM tab1, tab2 ...
GROUP BY attr1, attr3 HAVING condition;
IS 257 – Fall 2009
2009-09-15 SLIDE 23
Using an Aggregate Function
• SELECT DIVECUST.Name, Sum([Price]*[qty]) AS Total
FROM (DIVECUST INNER JOIN DIVEORDS ON
DIVECUST.[Customer No] = DIVEORDS.[Customer No])
INNER JOIN DIVEITEM ON DIVEORDS.[Order No] =
DIVEITEM.[Order No]
GROUP BY DIVECUST.Name
HAVING (((DIVECUST.Name) Like "*Jazdzewski"));
IS 257 – Fall 2009
2009-09-15 SLIDE 24
GROUP BY
• SELECT DEST.[Destination Name],
Count(*) AS Expr1
FROM DEST INNER JOIN DIVEORDS
ON DEST.[Destination Name] =
DIVEORDS.Destination
GROUP BY DEST.[Destination Name]
HAVING ((Count(*))>1);
• Provides a list of Destinations with the
number of orders going to that destination
IS 257 – Fall 2009
2009-09-15 SLIDE 25
SQL Commands
• Data Definition Statements
– For creation of relations/tables…
IS 257 – Fall 2009
2009-09-15 SLIDE 26
CREATE Table
• CREATE TABLE table-name (col_name1
col_definition1 [PRIMARY KEY],
col_name2 col_definition2,…,col_nameN
col_definitionN);
• Adds a new table with the specified
attributes (and types) to the database.
IS 257 – Fall 2009
2009-09-15 SLIDE 27
Database Design Process
Application 1
External
Model
Application 2
Application 3
Application 4
External
Model
External
Model
External
Model
Application 1
Conceptual
requirements
Application 2
Conceptual
requirements
Application 3
Conceptual
requirements
Conceptual
Model
Logical
Model
Internal
Model
Application 4
Conceptual
requirements
IS 257 – Fall 2009
Physical
Design
2009-09-15 SLIDE 28
Physical Database Design
• Many physical database design decisions
are implicit in the technology adopted
– Also, organizations may have standards or an
“information architecture” that specifies
operating systems, DBMS, and data access
languages -- thus constraining the range of
possible physical implementations.
• We will be concerned with some of the
possible physical implementation issues
IS 257 – Fall 2009
2009-09-15 SLIDE 29
Physical Database Design
• The primary goal of physical database
design is data processing efficiency
• We will concentrate on choices often
available to optimize performance of
database services
• Physical Database Design requires
information gathered during earlier stages
of the design process
IS 257 – Fall 2009
2009-09-15 SLIDE 30
Physical Design Information
• Information needed for physical file and
database design includes:
– Normalized relations plus size estimates for them
– Definitions of each attribute
– Descriptions of where and when data are used
• entered, retrieved, deleted, updated, and how often
– Expectations and requirements for response time,
and data security, backup, recovery, retention and
integrity
– Descriptions of the technologies used to implement
the database
IS 257 – Fall 2009
2009-09-15 SLIDE 31
Physical Design Decisions
• There are several critical decisions that
will affect the integrity and performance of
the system
– Storage Format
– Physical record composition
– Data arrangement
– Indexes
– Query optimization and performance tuning
IS 257 – Fall 2009
2009-09-15 SLIDE 32
Storage Format
• Choosing the storage format of each field
(attribute). The DBMS provides some set
of data types that can be used for the
physical storage of fields in the database
• Data Type (format) is chosen to minimize
storage space and maximize data integrity
IS 257 – Fall 2009
2009-09-15 SLIDE 33
Objectives of data type selection
•
•
•
•
•
Minimize storage space
Represent all possible values
Improve data integrity
Support all data manipulations
The correct data type should, in minimal
space, represent every possible value (but
eliminate illegal values) for the associated
attribute and can support the required data
manipulations (e.g. numerical or string
operations)
IS 257 – Fall 2009
2009-09-15 SLIDE 34
Access Data Types (Not MySQL)
•
•
•
•
•
•
•
•
•
Numeric (1, 2, 4, 8 bytes, fixed or float)
Text (255 max)
Memo (64000 max)
Date/Time (8 bytes)
Currency (8 bytes, 15 digits + 4 digits decimal)
Autonumber (4 bytes)
Yes/No (1 bit)
OLE (limited only by disk space)
Hyperlinks (up to 64000 chars)
IS 257 – Fall 2009
2009-09-15 SLIDE 35
Access Numeric types
• Byte
– Stores numbers from 0 to 255 (no fractions). 1 byte
• Integer
– Stores numbers from –32,768 to 32,767 (no fractions) 2
bytes
• Long Integer (Default)
– Stores numbers from –2,147,483,648 to 2,147,483,647 (no
fractions). 4 bytes
• Single
– Stores numbers from -3.402823E38 to –1.401298E–45 for
negative values and from 1.401298E–45 to 3.402823E38
for positive values.
4 bytes
• Double
– Stores numbers from –1.79769313486231E308 to –
4.94065645841247E–324 for negative values and from
1.79769313486231E308 to 4.94065645841247E–324 for
positive values.
15
8 bytes
• Replication ID
– Globally unique identifier (GUID)
IS 257 – Fall 2009
N/A
16 bytes
2009-09-15 SLIDE 36
MySQL Data Types
• MySQL supports all of the standard SQL numeric data
types. These types include the exact numeric data types
(INTEGER, SMALLINT, DECIMAL, and NUMERIC), as
well as the approximate numeric data types (FLOAT,
REAL, and DOUBLE PRECISION). The keyword INT is
a synonym for INTEGER, and the keyword DEC is a
synonym for DECIMAL
• Numeric (can also be declared as UNSIGNED)
–
–
–
–
–
–
–
–
TINYINT (1 byte)
SMALLINT (2 bytes)
MEDIUMINT (3 bytes)
INT (4 bytes)
BIGINT (8 bytes)
NUMERIC or DECIMAL
FLOAT
DOUBLE (or DOUBLE PRECISION)
IS 257 – Fall 2009
2009-09-15 SLIDE 37
MySQL Data Types
• The date and time types for representing
temporal values are DATETIME, DATE,
TIMESTAMP, TIME, and YEAR. Each temporal
type has a range of legal values, as well as a
“zero” value that is used when you specify an
illegal value that MySQL cannot represent
–
–
–
–
–
–
DATETIME
'0000-00-00 00:00:00'
DATE '0000-00-00'
TIMESTAMP (4.1 and up) '0000-00-00 00:00:00'
TIMESTAMP (before 4.1) 00000000000000
TIME '00:00:00'
YEAR 0000
IS 257 – Fall 2009
2009-09-15 SLIDE 38
MySQL Data Types
• The string types are CHAR, VARCHAR,
BINARY, VARBINARY, BLOB, TEXT, ENUM,
and SET
• Maximum length for CHAR is 255 and for
VARCHAR is 65,535
Value
""
"ab"
"abcd"
"abcdefg"
CHAR(4) Storage VARCHAR(4) Storage
" "
4 ""
1
"ab "
4 "ab"
3
"abcd"
4 "abcd"
5
"abcd"
4 "abcd"
5
• VARCHAR uses 1 or 2 bytes for the length
• For longer things there is BLOB and TEXT
IS 257 – Fall 2009
2009-09-15 SLIDE 39
MySQL Data Types
• A BLOB is a binary large object that can hold a
variable amount of data.
• The four BLOB types are TINYBLOB, BLOB,
MEDIUMBLOB, and LONGBLOB. These differ
only in the maximum length of the values they
can hold
• The four TEXT types are TINYTEXT, TEXT,
MEDIUMTEXT, and LONGTEXT. These
correspond to the four BLOB types and have the
same maximum lengths and storage
requirements
• TINY=1byte, BLOB and TEXT=2bytes,
MEDIUM=3bytes, LONG=4bytes
IS 257 – Fall 2009
2009-09-15 SLIDE 40
MySQL Data Types
• BINARY and VARBINARY are like CHAR and VARCHAR
but are intended for binary data of 255 bytes or less
• ENUM is a list of values that are stored as their
addresses in the list
– For example, a column specified as ENUM('one', 'two', 'three')
can have any of the values shown here. The index of each value
is also shown:
•
•
•
•
•
•
Value = Index
NULL = NULL
‘’
=
0
'one’ =
1
‘two’ =
2
‘three’ = 3
– An enumeration can have a maximum of 65,535 elements.
IS 257 – Fall 2009
2009-09-15 SLIDE 41
MySQL Data Types
• The final string type (for this version) is a SET
• A SET is a string object that can have zero or more
values, each of which must be chosen from a list of
allowed values specified when the table is created.
• SET column values that consist of multiple set members
are specified with members separated by commas (‘,’)
• For example, a column specified as SET('one', 'two')
NOT NULL can have any of these values:
–
–
–
–
''
'one'
'two'
'one,two‘
• A set can have up to 64 member values and is stored as
an 8byte number
IS 257 – Fall 2009
2009-09-15 SLIDE 42
Controlling Data Integrity
•
•
•
•
•
Default values
Range control
Null value control
Referential integrity (next time)
Handling missing data
IS 257 – Fall 2009
2009-09-15 SLIDE 43
Designing Physical Records
• A physical record is a group of fields
stored in adjacent memory locations and
retrieved together as a unit
• Fixed Length and variable fields
IS 257 – Fall 2009
2009-09-15 SLIDE 44
Designing Physical/Internal Model
• Overview
• terminology
• Access methods
IS 257 – Fall 2009
2009-09-15 SLIDE 45
Physical Design
• Internal Model/Physical Model
User request
Interface 1
External Model
DBMS
Internal Model
Access Methods
Interface 2
Operating
System
Access Methods
Interface 3
Data
Base
IS 257 – Fall 2009
2009-09-15 SLIDE 46
Physical Design
• Interface 1: User request to the DBMS.
The user presents a query, the DBMS
determines which physical DBs are
needed to resolve the query
• Interface 2: The DBMS uses an internal
model access method to access the data
stored in a logical database.
• Interface 3: The internal model access
methods and OS access methods access
the physical records of the database.
IS 257 – Fall 2009
2009-09-15 SLIDE 47
Physical File Design
• A Physical file is a portion of secondary storage
(disk space) allocated for the purpose of storing
physical records
• Pointers - a field of data that can be used to
locate a related field or record of data
• Access Methods - An operating system
algorithm for storing and locating data in
secondary storage
• Pages - The amount of data read or written in
one disk input or output operation
IS 257 – Fall 2009
2009-09-15 SLIDE 48
Lecture Outline
• Review
–Relational Algebra and Calculus
–Introduction to SQL
• Physical Database Design
• Access Methods
IS 257 – Fall 2009
2009-09-15 SLIDE 49
Internal Model Access Methods
• Many types of access methods:
– Physical Sequential
– Indexed Sequential
– Indexed Random
– Inverted
– Direct
– Hashed
• Differences in
– Access Efficiency
– Storage Efficiency
IS 257 – Fall 2009
2009-09-15 SLIDE 50
Physical Sequential
• Key values of the physical records are in
logical sequence
• Main use is for “dump” and “restore”
• Access method may be used for storage
as well as retrieval
• Storage Efficiency is near 100%
• Access Efficiency is poor (unless fixed
size physical records)
IS 257 – Fall 2009
2009-09-15 SLIDE 51
Indexed Sequential
• Key values of the physical records are in logical
sequence
• Access method may be used for storage and
retrieval
• Index of key values is maintained with entries for
the highest key values per block(s)
• Access Efficiency depends on the levels of
index, storage allocated for index, number of
database records, and amount of overflow
• Storage Efficiency depends on size of index and
volatility of database
IS 257 – Fall 2009
2009-09-15 SLIDE 52
Index Sequential
Data File
Actual
Value
IS 257 – Fall 2009
Address
Block
Number
Dumpling
1
Harty
2
Texaci
3
...
…
Adams
Becker
Dumpling
Block 1
Getta
Harty
Block 2
Mobile
Sunoci
Texaci
Block 3
2009-09-15 SLIDE 53
Indexed Sequential: Two Levels
Key
Value
Key
Value
Address
150
1
385
2
001
003
.
.
150
Address
385
7
678
8
805
9
…
Key
Value
Address
536
3
678
4
Key
Value
251
.
.
385
455
480
.
.
536
605
610
.
.
678
Address
785
5
805
6
791
.
.
805
IS 257 – Fall 2009
705
710
.
.
785
2009-09-15 SLIDE 54
Indexed Random
• Key values of the physical records are not
necessarily in logical sequence
• Index may be stored and accessed with Indexed
Sequential Access Method
• Index has an entry for every data base record.
These are in ascending order. The index keys
are in logical sequence. Database records are
not necessarily in ascending sequence.
• Access method may be used for storage and
retrieval
IS 257 – Fall 2009
2009-09-15 SLIDE 55
Indexed Random
Becker
Harty
Actual
Value
Address
Block
Number
Adams
2
Becker
1
Dumpling
3
Getta
2
Harty
1
Adams
Getta
Dumpling
IS 257 – Fall 2009
2009-09-15 SLIDE 56
Btree
F
B
||
D || F|
||
P || Z|
H ||
L || P|
R ||
S || Z|
Devils
Aces
Boilers
Cars
IS 257 – Fall 2009
Flyers
Hawkeyes
Hoosiers
Minors
Panthers
Seminoles
2009-09-15 SLIDE 57
Inverted
• Key values of the physical records are not
necessarily in logical sequence
• Access Method is better used for retrieval
• An index for every field to be inverted may
be built
• Access efficiency depends on number of
database records, levels of index, and
storage allocated for index
IS 257 – Fall 2009
2009-09-15 SLIDE 58
Inverted
Student
name
Course
Number
CH 145
101, 103,104
Actual
Value
Address
Block
Number
CH 145
1
CS 201
2
CS 623
3
PH 345
…
Adams
CH145
Becker
cs201
Dumpling ch145
CS 201
102
Getta
ch145
Harty
cs623
Mobile
cs623
CS 623
105, 106
IS 257 – Fall 2009
2009-09-15 SLIDE 59
Direct
• Key values of the physical records are not
necessarily in logical sequence
• There is a one-to-one correspondence
between a record key and the physical
address of the record
• May be used for storage and retrieval
• Access efficiency always 1
• Storage efficiency depends on density of
keys
• No duplicate keys permitted
IS 257 – Fall 2009
2009-09-15 SLIDE 60
Hashing
• Key values of the physical records are not
necessarily in logical sequence
• Many key values may share the same physical
address (block)
• May be used for storage and retrieval
• Access efficiency depends on distribution of
keys, algorithm for key transformation and space
allocated
• Storage efficiency depends on distibution of
keys and algorithm used for key transformation
IS 257 – Fall 2009
2009-09-15 SLIDE 61
Comparative Access Methods
Factor
Storage space
Sequential
retrieval on
primary key
Random Retr.
Multiple Key
Retr.
Deleting records
Sequential
No wasted space
Indexed
Hashed
No wasted
space for data
but extra space for index
more space needed for
addition and deletion of
records after initial load
Very fast
Moderately Fast
Impractical
Moderately Fast
Very fast with
multiple indexes
OK if dynamic
Very fast
OK if dynamic
very easy
Easy but requires
Maintenance of
indexes
very easy
Impractical
Possible but needs
a full scan
can create wasted
space
Adding records requires rewriting
file
Updating records usually requires
rewriting file
IS 257 – Fall 2009
Not possible
very easy
2009-09-15 SLIDE 62
Database Creation in Access
• Simplest to use a design view
– wizards are available, but less flexible
• Need to watch the default values
• Helps to know what the primary key is, or
if one is to be created automatically
– Automatic creation is more complex in other
RDBMS and ORDBMS
• Need to make decision about the physical
storage of the data
IS 257 – Fall 2009
2009-09-15 SLIDE 63
Database Creation in Access
• Some Simple Examples
IS 257 – Fall 2009
2009-09-15 SLIDE 64
Next Time
• Tuesday -- Kevin Heard on MySQL
• Thursday -- no class
IS 257 – Fall 2009
2009-09-15 SLIDE 65