Download Informationslogistik Unit 10: OLTP, OLAP, SAP, Data Warehouse

SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
Informationslogistik
Unit 10: OLTP, OLAP, SAP, Data Warehouse, and
Object-relational Databases
Ronald Ortner
20. V. 2014
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
Outline
1
SQL: Subtleties for COUNT and JOINs
2
OLTP, OLAP, SAP, and Data Warehouse
OLTP and OLAP
SAP
Data Warehouse & OLAP
3
Objectrelational Databases
An Example
Objectrelational Databases
4
Ranking and Data Mining
Ranking
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
SQL-Lesson
Today:
Extensions I:
counting with 0
Extensions II:
when to put conditions in the ON / WHERE part
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
OLTP and OLAP
Outline
1
SQL: Subtleties for COUNT and JOINs
2
OLTP, OLAP, SAP, and Data Warehouse
OLTP and OLAP
SAP
Data Warehouse & OLAP
3
Objectrelational Databases
An Example
Objectrelational Databases
4
Ranking and Data Mining
Ranking
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
OLTP and OLAP
OLTP vs. OLAP
OLTP: online transaction processing
Database applications for ongoing work
Examples: orders, bookings, etc.
current data is important
→ many updates and changes in database
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
OLTP and OLAP
OLTP vs. OLAP
OLTP: online transaction processing
Database applications for ongoing work
Examples: orders, bookings, etc.
current data is important
→ many updates and changes in database
OLAP: online analytical processing
Database applications for analysis and decision support
Example: analysis of trends
historical data is important
→ lots of data, need information in aggregated form
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
SAP
Outline
1
SQL: Subtleties for COUNT and JOINs
2
OLTP, OLAP, SAP, and Data Warehouse
OLTP and OLAP
SAP
Data Warehouse & OLAP
3
Objectrelational Databases
An Example
Objectrelational Databases
4
Ranking and Data Mining
Ranking
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
SAP
SAP
SAP: software system, mainly for OLTP
SAP has three levels:
big relational database system in the background
applications that work on the database system
graphical user interface
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
SAP
SAP
SAP: software system, mainly for OLTP
SAP has three levels:
big relational database system in the background
applications that work on the database system
graphical user interface
Access to underlying database system:
Some tables can be accessed also outside SAP (using SQL).
Usually only read access is sensible.
Some other tables can be accessed only via SAP.
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
SAP
SAP
SAP: software system, mainly for OLTP
SAP has three levels:
big relational database system in the background
applications that work on the database system
graphical user interface
Writing applications with ABAP/4 – access to databases with
Native SQL (using special interface)
Open SQL (direct access to databases)
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
Data Warehouse & OLAP
Outline
1
SQL: Subtleties for COUNT and JOINs
2
OLTP, OLAP, SAP, and Data Warehouse
OLTP and OLAP
SAP
Data Warehouse & OLAP
3
Objectrelational Databases
An Example
Objectrelational Databases
4
Ranking and Data Mining
Ranking
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
Data Warehouse & OLAP
OLTP vs. OLAP
OLTP: online transaction processing
Database applications for ongoing work
Examples: orders, bookings, etc.
current data is important
→ many updates and changes in database
OLAP: online analytical processing
Database applications for analysis and decision support
Example: analysis of trends
historical data is important
→ lots of data, need information in aggregated form
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
Data Warehouse & OLAP
OLTP vs. OLAP
OLTP: online transaction processing
Database applications for ongoing work
Examples: orders, bookings, etc.
current data is important
→ many updates and changes in database
OLAP: online analytical processing
Database applications for analysis and decision support
Example: analysis of trends
historical data is important
→ lots of data, need information in aggregated form
→ no good idea to do OLTP and OLAP on the same database system
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
Data Warehouse & OLAP
Data Warehouse
Idea of Data Warehouse:
Do OLTP on operational databases
Store information from operational databases regularly
(but not online!) in data warehouse
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
Data Warehouse & OLAP
Data Warehouse
Idea of Data Warehouse:
Do OLTP on operational databases
Store information from operational databases regularly
(but not online!) in data warehouse
Database Scheme for Data Warehouse:
Star Scheme:
Central ‘fact’ table
other tables not normalized
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
Data Warehouse & OLAP
Data Warehouse
Idea of Data Warehouse:
Do OLTP on operational databases
Store information from operational databases regularly
(but not online!) in data warehouse
Database Scheme for Data Warehouse:
Star Scheme:
Central ‘fact’ table
other tables not normalized
Snowflake Scheme:
Central ‘fact’ table
other tables normalized (→ more joins necessary)
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
Data Warehouse & OLAP
Roll Up and Drill Down
Queries on Data Warehouse for analysis usually aggregate data
(→ GROUP BY)
Drill down: more attributes in GROUP BY
Roll up: fewer attributes in GROUP BY
Data can be summarized in a cross table (data cube)
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
Data Warehouse & OLAP
Relations for Aggregation & the Cube Operator
Creating the data cube:
expensive to execute all queries for creating cube
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
Data Warehouse & OLAP
Relations for Aggregation & the Cube Operator
Creating the data cube:
expensive to execute all queries for creating cube
can store relation for data cube
(using NULL values where aggregated)
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
Data Warehouse & OLAP
Relations for Aggregation & the Cube Operator
Creating the data cube:
expensive to execute all queries for creating cube
can store relation for data cube
(using NULL values where aggregated)
still elaborate and uncomfortable
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
Data Warehouse & OLAP
Relations for Aggregation & the Cube Operator
Creating the data cube:
expensive to execute all queries for creating cube
can store relation for data cube
(using NULL values where aggregated)
still elaborate and uncomfortable
→ idea: new SQL operator CUBE
Usage: GROUP BY CUBE( attr1, attr2, . . . )
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
Data Warehouse & OLAP
Relations for Aggregation & the Cube Operator
Creating the data cube:
expensive to execute all queries for creating cube
can store relation for data cube
(using NULL values where aggregated)
still elaborate and uncomfortable
→ idea: new SQL operator CUBE
Usage: GROUP BY CUBE( attr1, attr2, . . . )
Other possibility:
storing maximally drilled-down table
aggregate this table
(cheaper than doing each aggregation from scratch)
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
Data Warehouse & OLAP
Row Store vs. Column Store
Usually, tables are stored row-wise.
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
Data Warehouse & OLAP
Row Store vs. Column Store
Usually, tables are stored row-wise.
When there are many columns, it may be better to store
column-wise:
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
Data Warehouse & OLAP
Row Store vs. Column Store
Usually, tables are stored row-wise.
When there are many columns, it may be better to store
column-wise:
Most queries consider only few columns.
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
Data Warehouse & OLAP
Row Store vs. Column Store
Usually, tables are stored row-wise.
When there are many columns, it may be better to store
column-wise:
Most queries consider only few columns.
Column values can be better compressed.
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
Data Warehouse & OLAP
Row Store vs. Column Store
Usually, tables are stored row-wise.
When there are many columns, it may be better to store
column-wise:
Most queries consider only few columns.
Column values can be better compressed.
Use e.g. dictionary table.
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
An Example
Outline
1
SQL: Subtleties for COUNT and JOINs
2
OLTP, OLAP, SAP, and Data Warehouse
OLTP and OLAP
SAP
Data Warehouse & OLAP
3
Objectrelational Databases
An Example
Objectrelational Databases
4
Ranking and Data Mining
Ranking
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
An Example
An Example: Storing Books
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
An Example
An Example: Storing Books
We introduce IDs for authors, books and key words.
Then:
One table for each entity type (with IDs being the primary key):
authors: {[a_id, name]}
books: {[b_id, title, publisher]}
key words: {[k_id, keyword]}
One table for each relation:
writes: {[a_id, b_id]}
has_keyword: {[b_id, k_id, weight]}
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
An Example
An Example: Storing Books
We introduce IDs for authors, books and key words.
Then:
One table for each entity type (with IDs being the primary key):
authors: {[a_id, name]}
books: {[b_id, title, publisher]}
key words: {[k_id, keyword]}
One table for each relation:
writes: {[a_id, b_id]}
has_keyword: {[b_id, k_id, weight]}
→ need five (!) tables for storing books (actually one entity)
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
Objectrelational Databases
Outline
1
SQL: Subtleties for COUNT and JOINs
2
OLTP, OLAP, SAP, and Data Warehouse
OLTP and OLAP
SAP
Data Warehouse & OLAP
3
Objectrelational Databases
An Example
Objectrelational Databases
4
Ranking and Data Mining
Ranking
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
Objectrelational Databases
Object-oriented Extensions
→ extend relational database theory with more object-oriented ideas:
Possible approaches:
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
Objectrelational Databases
Object-oriented Extensions
→ extend relational database theory with more object-oriented ideas:
Possible approaches:
include structure into database system:
→ implicit joins (no join condition necessary)
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
Objectrelational Databases
Object-oriented Extensions
→ extend relational database theory with more object-oriented ideas:
Possible approaches:
include structure into database system:
→ implicit joins (no join condition necessary)
give up first NF and allow structured information:
→ can store several authors/keywords in set
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
Objectrelational Databases
Objectrelational Databases
In a proper objectrelational database system one also has
possibility to define objects with other objects
inheritance
object methods
→ cf. object-oriented programming languages like Java
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
Ranking
Outline
1
SQL: Subtleties for COUNT and JOINs
2
OLTP, OLAP, SAP, and Data Warehouse
OLTP and OLAP
SAP
Data Warehouse & OLAP
3
Objectrelational Databases
An Example
Objectrelational Databases
4
Ranking and Data Mining
Ranking
Ronald Ortner
SQL: Subtleties for COUNT and JOINs OLTP, OLAP, SAP, and Data Warehouse Objectrelational Databases Ranking and Data Min
Ranking
Ranking
Task:
We want to rank objects with respect to more than one criterion.
We are only interested in “top” objects.
Naive solution:
Compute value of objective function for all objects and sort.
However, there are better ways to do this:
Threshold algorithm
No Random Access algorithm
Ronald Ortner