Download Oracle for Developer

Oracle Datenbank 12c
für Entwickler
Michael Künzner
Leitender Systemberater
Oracle Server Technologies Customer Center Süd DB
09. Oktober 2014
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
3
Agenda – Oracle Database 12c für Entwickler
1
Übersicht
2
SQL und PL/SQL
3
Big Data SQL
4
JSON und REST Data Services
5
In-Memory Technologien
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
4
Oracle Datenbank 12c
Übersicht
08.10.2014
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
5
Oracle Database 12c
Application Development
Big Data & Analytics
Consolidation
Data Optimization
Big Data Appliance
Data Warehousing
High Availability
In-Memory
Exadata
Recovery Appliance
Performance & Scalability
Security & Compliance
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Oracle Database 12c for the Developer
Supporting all major development environments and API’s
Ruby
JSON
Oracle ADF
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Oracle APEX
Oracle RDS
7
Oracle Database 12c
SQL und PL/SQL
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
8
SQL Evolution
15+ Years of Innovation
•
•
•
•
Introduction of
Window functions
•
•
•
•
•
Statistical functions
SQL model clause
Partition Outer Join
Data mining I
4
• Enhanced Window functions (percentile,etc)
• Rollup, grouping sets, cube
•
•
•
•
Data mining II
SQL Pivot
Recursive WITH
ListAgg, Nth value window
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Pattern matching
Top N clause
Identity Columns
Column Defaults
Data Mining III
5
Pattern Matching in Sequences of Rows
The Challenge – a real-world business problem
“
… detect if a phone card went from phone A to phone B to phone C... and
back to phone A within ‘N‘ hours... ”
“… and detect if pattern above occurs at least ‘N’ times within 7 days …”
• Currently pattern recognition in SQL is difficult
– Use multiple self joins (not good for *)
• T1.handset_id <> T2.handset_id <>T3.handset_id AND…. T1.sim_id=‘X’ AND T2.time BETWEEN
T1.time and T1.time+2….
– Use recursive query for * (WITH clause, CONNECT BY)
– Use Window Functions (likely with multiple query blocks)
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
10
Pattern Matching in Sequences of Rows
Objective
“Find one or more event A followed by one B
Provide native SQL
language construct
followed by one or more C in a 1 minute interval”
Apply expressions across
rows
Soon to be in ANSI SQL
Standard
> 1 min
Align with well-known
regular expression
declaration (PERL)
A+ B C+ (perl)
EVENT
TIME
LOCATION
A
1
SFO
A
1
SFO
A
2
ATL
A
2
ATL
A
2
LAX
A
2
LAX
B
2
SFO
C
2
LAX
B
2
SFO
C
3
LAS
C
2
LAX
A
3
SFO
B
3
NYC
C
4
NYC
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
11
Pattern Recognition In Sequences of Rows
SQL Pattern Matching - Concept
• Recognize patterns in sequences of events using SQL
– Sequence is a stream of rows
– Event equals a row in a stream
• New SQL construct MATCH_RECOGNIZE
– Logically partition and order the data
• ORDER BY mandatory (optional PARTITION BY)
– Pattern defined using regular expression using variables
– Regular expression is matched against a sequence of rows
– Each pattern variable is defined using conditions on rows and aggregates
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
12
SQL Pattern Matching in action
Example: Find a double bottom pattern (W-shape) in ticker stream
Find a W-shape pattern in a
ticker stream:
Stock price
Output the beginning and
ending date of the pattern
Calculate average price
each the W-shape
Find only patterns that
lasted less than a week
days
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
13
next = lineNext.getQuantity();
}
if (!q.isEmpty() && (prev.isEmpty() || (eq(q, prev) && gt(q, next)))) {
state = "S";
return state;
}
Pattern Matching
if (gt(q, prev) && gt(q, next)) {
state = "T";
return state;
}
Finding Double Bottom (W)
if (lt(q, prev) && lt(q, next)) {
state = "B";
return state;
}
if (!q.isEmpty() && (next.isEmpty() || (gt(q, prev) && eq(q, next)))) {
state = "E";
return state;
}
if (q.isEmpty() || eq(q, prev)) {
state = "F";
return state;
}
SELECT first_x, last_z
FROM ticker MATCH_RECOGNIZE (
PARTITION BY name ORDER BY time
MEASURES FIRST(x.time) AS first_x,
LAST(z.time) AS last_z
ONE ROW PER MATCH
PATTERN (X+ Y+ W+ Z+)
DEFINE X AS (price < PREV(price)),
Y AS (price > PREV(price)),
W AS (price < PREV(price)),
Z AS (price > PREV(price) AND
z.time - FIRST(x.time) <= 7 ))
return state;
}
private boolean eq(String a, String b) {
if (a.isEmpty() || b.isEmpty()) {
return false;
}
return a.equals(b);
}
private boolean gt(String a, String b) {
if (a.isEmpty() || b.isEmpty()) {
return false;
}
return Double.parseDouble(a) > Double.parseDouble(b);
}
private boolean lt(String a, String b) {
if (a.isEmpty() || b.isEmpty()) {
return false;
}
return Double.parseDouble(a) < Double.parseDouble(b);
}
public String getState() {
return this.state;
}
}
BagFactory bagFactory = BagFactory.getInstance();
@Override
public Tuple exec(Tuple input) throws IOException {
long c = 0;
String line = "";
String pbkey = "";
V0Line nextLine;
V0Line thisLine;
V0Line processLine;
V0Line evalLine = null;
V0Line prevLine;
boolean noMoreValues = false;
String matchList = "";
ArrayList<V0Line> lineFifo = new ArrayList<V0Line>();
boolean finished = false;
250+ Lines of Java and PIG
DataBag output = bagFactory.newDefaultBag();
if (input == null) {
return null;
}
if (input.size() == 0) {
return null;
}
Object o = input.get(0);
if (o == null) {
return null;
}
//Object o = input.get(0);
if (!(o instanceof DataBag)) {
int errCode = 2114;
12 Lines of SQL
20x less code, 5x faster
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
New SQL Functionality
• PL/SQL in SQL via the “with” clause
– Useful for read only databases
– Potentially faster for some operations
• IDENTITY based columns
– Auto incrementing columns
– Supports ANSI standard
• Increased size for VARCHAR2
– Increase from 4000 to 32K
• New Row limiting clause
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
15
PL/SQL Function embedded in WITH clause
Example
WITH
FUNCTION get_domain(url VARCHAR2) RETURN VARCHAR2 IS
pos BINARY_INTEGER;
len BINARY_INTEGER;
BEGIN
pos := INSTR(url, 'www.');
len := INSTR(SUBSTR(url, pos + 4), '.') - 1;
RETURN SUBSTR(url, pos + 4, len);
END;
SELECT
DISTINCT get_domain(catalog_url)
FROM orders;
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
16
IDENTITY based columns
Example
• Create a table where the id column is always populated by Oracle
CREATE TABLE t1
( id
NUMBER GENERATED AS IDENTITY
, first_name VARCHAR2(30)
);
• Create a table where the id column is populated by Oracle when not
provided
CREATE TABLE t2
( id
NUMBER GENERATED BY DEFAULT AS IDENTITY
(START WITH 100 INCREMENT BY 10)
, first_name VARCHAR2(30)
);
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
17
32K VARCHAR2/NVARCHAR2
Example
• Enable 32k support in the Oracle Database 12c
ALTER SYSTEM SET max_string_size = EXTENDED SCOPE = SPFILE
• Create table with 32k VARCHAR2
CREATE TABLE applicants
( id
NUMBER GENERATED AS IDENTITY
, first_name VARCHAR2(30)
, last_name
VARCHAR2(30)
, application DATE
, cv
VARCHAR2(32767)
);
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
18
New Row limiting clause
Example
• Select only the first 5 rows
SELECT employee_id, last_name
FROM employees
ORDER BY employee_id
FETCH FIRST 5 ROWS ONLY;
• Select the first 5% of rows and those whose salary “ties” with the lowest of
the 5%
SELECT employee_id, last_name, salary
FROM employees
ORDER BY salary
FETCH FIRST 5 PERCENT ROWS WITH TIES;
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
19
New Row limiting clause
“Who are the 5 less money makers in my enterprise?”
Natively identify top N in
SQL
Simplifies code
development
ANSI SQL:2008
SELECT empno, ename, deptno
FROM (SELECT empno, ename, deptno, sal, comm,
row_number() OVER (ORDER BY sal,comm) rn
FROM emp
)
WHERE rn <=5
ORDER BY sal, comm;
versus
SELECT empno, ename, deptno
FROM emp
ORDER BY sal, comm FETCH FIRST 5 ROWS ONLY;
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
20
Oracle Database 12c
Big Data SQL
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
21
Oracle Support for Any Data Management System
Hadoop
NoSQL
Change the Business
Scale the Business




Scale-out, low cost store
Collect any data
Map-reduce, SQL
Analytic applications




Relational
Run the Business
Scale-out, low cost store
Collect key-value data
Find data by key
Web applications
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |




Scale-out and scale-up
Collect any data
SQL
Transactional and analytic
applications for the enterprise
 Secure and highly available
22
Oracle Big Data SQL
One fast SQL query , on all your data.
Oracle SQL on Hadoop and beyond
• With a Smart Scan service inspired by Exadata
• With native SQL operators
• With the security and certainty of Oracle Database
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
23
Accessing Big Data
Select w.sess_id,
w.cust_id,
w.pag e_id
From web_logs w
W h e r e w . s o u r c e _ c o u n t r y = ‘B r a z i l ’
A n d w . c a t e g o r y = ‘T V ’
A n d w . c h a n n e l = ‘M o b i l e ’
Without SQL Push Down
Request for Data
100’s of Terabytes of Data
WEB_LOGS
Hadoop Cluster
Low utilization of
available resources
All columns and rows from the
table are returned
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
High load on
database server
24
Big Data SQL Push Down
Select w.sess_id,
w.cust_id,
w.pag e_id
From web_logs w
W h e r e w . s o u r c e _ c o u n t r y = ‘B r a z i l ’
A n d w . c a t e g o r y = ‘T V ’
A n d w . c h a n n e l = ‘M o b i l e ’
With SQL Push Down
Big Data SQL
SQL shipped to BDA
10’s of Gigabytes of Data
WEB_LOGS
Hadoop Cluster
Only columns and rows needed to
answer query are returned
Good utilization of
available resources.
SQL executed on
Hadoop cluster
Lower load on Server,
Faster response
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
25
Big Data SQL Push Down
S e l e c t w.ses s_id ,
w.cust_id,
w.page_id,
c.name
From web_logs w, customers c
W h e r e w . s o u r c e _ c o u n t r y = ‘B r a z i l ’
A n d w . c a t e g o r y = ‘T V ’
A n d w . c h a n n e l = ‘M o b i l e ’
And c.customer_id = w.cust_id
With SQL Push Down
Big Data SQL
SQL shipped to BDA
10’s of Gigabytes of Data
WEB_LOGS
CUSTOMERS
Hadoop Cluster
Only columns and rows needed to
answer query are returned
Good utilization of
available resources.
SQL executed on
Hadoop cluster
Data joined between
CUSTOMERS and
WEB_LOGS on server
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
26
Big Data SQL Push Down
S e l e c t w.ses s_id ,
w.cust_id,
w.page_id,
c.name
From web_logs w, customers c
W h e r e w . s o u r c e _ c o u n t r y = ‘B r a z i l ’
A n d w . c a t e g o r y = ‘T V ’
A n d w . c h a n n e l = ‘M o b i l e ’
And c.customer_id = w.cust_id
With SQL Push Down
Big Data SQL
WEB_LOGS
SQL Push Downs supported by Big Data SQL
•
•
•
•
•
SQL shipped to BDA
Hadoop scans (InputFormat, SerDe)
JSON 10’s
parsing
of Gigabytes of Data
WHERE clause evaluation
Column projection
Bloom filters for faster join
Hadoop Cluster
CUSTOMERS
Only columns and rows needed to
answer query are returned
Good utilization of
available resources.
SQL executed on
Hadoop cluster
Data joined between
CUSTOMERS and
WEB_LOGS on server
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
27
New Data Sources for Oracle External Tables
• New set of properties
CREATE TABLE web_logs
(click VARCHAR2(4000))
ORGANIZATION EXTERNAL
(
TYPE ORACLE_HIVE
DEFAULT DIRECTORY Dir1
ACCESS PARAMETERS
(
com.oracle.bigdata.tablename logs
com.oracle.bigdata.cluster mycluster)
)
REJECT LIMIT UNLIMITED
– ORACLE_HIVE and ORACLE_HDFS access drivers
– Identify a Hadoop cluster, data source, column
mapping, error handling, overflow handling,
logging
• New table metadata passed from Oracle
DDL to Hadoop readers at query execution
• Architected for extensibility
– StorageHandler capability enables future
support for other data sources
– Examples: MongoDB, HBase, Oracle NoSQL DB
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
28
Advanced Query & Analysis
Full Power of SQL and Advanced Analytics
Leverages All Your Data
Relational, Hadoop and NoSQL
Secure
Unified Governance on All Data
Fastest Performance
Utilize SQL Processing Across the Platform
Transparent to Applications
No Changes to Application Code
Big Data Management System
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
29
Oracle Database 12c
JSON und REST Data Services
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
30
Oracle REST Data Services
New-generation RESTful access to Oracle Database
• Retrieve and store data via standard HTTP URI RESTful GETs and POSTs
– Build applications without writing SQL and without specialized drivers
• JavaScript framework friendly
– Results returned in JSON format
• Ships with Oracle Database 12c Release 1 (12.1.0.2)
– Formerly known as Oracle APEX Listener
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
31
Oracle REST Data Services
Oracle REST Data
Services
Use REST to get JSON
from any data source
JSON Collection API
Pass Back
URI
{JSON}
HTTP(S) client
Map & Bind
Transform to JSON
NoSQL API
PassBack
Auto Generated SQL
{JSON}
SQL
SQL Result Set
Data held in JSON
Document Store
Data held in
Relational Table
Key Value Lookup
{JSON}
Data held in Oracle
NoSQL
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
32
JSON Support in Oracle Database
Flexible Application Development + Powerful SQL Analytics
Oracle Database 12c
JSON
Data accessed via
RESTful service or
native API’s
SQL
Data persisted in database
In JSON
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Data analyzed via SQL
33
JSON Support in Oracle Database
• Store and manage JSON documents in Oracle Database
– JSON stored as text (VARCHAR2 or CLOB)
– Full text index (inverted index) on entire JSON documents
– Functional indexes on specific attributes in JSON documents
• Access JSON via developer-friendly API’s
– REST services
– Document-store API for Java (additional languages planned)
• SQL query capabilities over JSON documents
• Powerful reporting and analysis directly on JSON
• Simple integration with relational data
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
34
Storing and Querying JSON
• Application developers
– Store JSON using RESTful API
PUT /my_database/my_schema/customers HTTP/1.0
Content-Type: application/json
Body:
{
"firstName": "John",
“lastName”: "Smith",
"age": 25,
"address": {
"streetAddress": "21 2nd Street",
"city": "New York",
"state": "NY",
"postalCode": "10021“,
"isBusiness" : false },
"phoneNumbers": [
{"type": "home",
"number": "212 555-1234“ },
{"type": "fax",
"number": "646 555-4567“ } ]
}
• Analytical tools and business users
– Query JSON using SQL
select
c.document.firstName,
c.document.lastName,
c.document.address.city
from customers c;
firstName
----------“John”
lastName
----------“Smith”
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
address.city
-------------“New York”
35
Oracle Database 12c for the Developer
Supporting all major development environments and API’s
Java
.NET
Node.js
REST (ORDS)
Ruby
Python
PHP
R
Perl
SQL API
Document Store API









H2 2014
Use REST API
H2 2014
DB 12.1.0.2
Use REST API
Use REST API
Use REST API
Use new
document-store
API’s to build
applications
without writing
SQL
Use REST API
Use REST API
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
36
Oracle Database 12c
In-Memory Technologien
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
37
Result
Cache
Database Buffer
Default Cache
Keep Cache
Recycle Cache
LRU-Algorithmus
Exadata X4
In-Memory
Hybrid Columnar Compression
Smart Scan
Storage Index
Smart Flash Cache (Read/Write)
Smart Flash Logging
(Redo-Write)
Shared Pool oder Client
Transparent für Anwendungen
Technik der
Flash
Cache
Datenbank
Data Buffer Erweiterung
2nd level Cache
Auch für Datafiles
Memory optimierende
Techniken
Compression
Partitioning
Parallelisierung
OLAP
In-Memory Option 12c
In-Memory
Technik der
TimesTen IMDB
Middle-Tier
IMDB Database Cache Option
TimesTen for
Exalytics
Row und Column In-Memory
Formate auf der selben Tabelle
Simultan und transaktionsKonsistent
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Coherence
Data Grid
Object Cache / Data Grid
für verteilte
Anwendungen
38
38
Optimizing Transaction and Query Performance
Row Format Databases versus Column Format Databases
SALES
– Example: Query or Insert a sales order
– Fast processing few rows, many columns
Row
SALES
Column
• Transactions run faster on row format
• Analytics run faster on column format
– Example : Report on sales totals by region
– Fast accessing few columns, many rows
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Row Store vs. Column Store
• OLTP-style query
Row Store
Column Store
SELECT * FROM salesorders
WHERE document_number = 95779216
• OLAP-style query
SELECT sum(value) FROM salesorders
WHERE document_date > 2013-08-28
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
40
Breakthrough: Dual Format Database
Memory
Memory
SALES
SALES
Row
Format
Column
Format
• BOTH row and column
formats for same table
• Simultaneously active and
transactionally consistent
• Analytics & reporting use
new in-memory Column
format
• OLTP uses proven row format
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
41
Oracle In-Memory Design Goals
Target Specification
• Integration in existing Infrastructure
– High Availability
– Backup & Recovery and Desaster Recovery
• Applications should profit without any change
– Transparent Usage
– SQL & Optimizer aware
– OLTP & Analytics
– No Data Migration
Memory
OLTP
Row
Format
Memory
Analytics
Column
Format
• Ease of Use
– Simple Definition through SQL
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
42
Complex OLTP is Slowed by Analytic Indexes
Table
1–3
OLTP
Indexes
10 – 20
Analytic
Indexes
• Most Indexes in complex OLTP (e.g.
ERP) databases are only used for
analytic queries
• Inserting one row into a table
requires updating 10-20 analytic
indexes: Slow!
• Indexes only speed up predictable
queries & reports
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
43
Column Store Replaces Analytic Indexes
Table
1–3
OLTP
Indexes
In-Memory
Column Store
• In-Memory Column Store replaces
analytic indexes and removes their
overhead for tables that fit in memory
• Fast analytics on any columns
– Better for unpredictable analytics
– Less tuning & administration
• Column Store not persistent so update
cost is much lower
Less
tuning & admin
• Both predefined and ad-hoc analytic
queries run faster
• OLTP & batch run faster
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
44
Weitere Informationen
http://search.oracle.com
database
Technologie Überblick
 auf: http://www.oracle.com/database
 Whitepapers und Webinars ansehen
Technische Information, Demos, Software
 auf OTN: otn.oracle.com  products  database
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
45
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
46
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
47
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
48
Oracle Database 12c
SQL und PL/SQL
Oracle Database 12c
SQL Language
Miscellaneous
Enhancements
Data Types
SQL row limiting
clause
Increased length
limits of the Data
Types
Invisible column
ANYDATA
Enhancements
Making DDLs
ONLINE
SQL column
defaulting on
explicit NULL
Enhancements to
Partitioning
concepts
Gateways
Enhancements
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
SQL IDENTITY
Column
SQL column
defaulting on
Sequence