* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Oracle for Developer
Microsoft Jet Database Engine wikipedia , lookup
Entity–attribute–value model wikipedia , lookup
Extensible Storage Engine wikipedia , lookup
Functional Database Model wikipedia , lookup
Navitaire Inc v Easyjet Airline Co. and BulletProof Technologies, Inc. wikipedia , lookup
Clusterpoint wikipedia , lookup
Microsoft SQL Server wikipedia , lookup
Open Database Connectivity wikipedia , lookup
Database model wikipedia , lookup
Relational model wikipedia , lookup
Oracle Datenbank 12c für Entwickler Michael Künzner Leitender Systemberater Oracle Server Technologies Customer Center Süd DB 09. Oktober 2014 Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 3 Agenda – Oracle Database 12c für Entwickler 1 Übersicht 2 SQL und PL/SQL 3 Big Data SQL 4 JSON und REST Data Services 5 In-Memory Technologien Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 4 Oracle Datenbank 12c Übersicht 08.10.2014 Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 5 Oracle Database 12c Application Development Big Data & Analytics Consolidation Data Optimization Big Data Appliance Data Warehousing High Availability In-Memory Exadata Recovery Appliance Performance & Scalability Security & Compliance Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle Database 12c for the Developer Supporting all major development environments and API’s Ruby JSON Oracle ADF Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle APEX Oracle RDS 7 Oracle Database 12c SQL und PL/SQL Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 8 SQL Evolution 15+ Years of Innovation • • • • Introduction of Window functions • • • • • Statistical functions SQL model clause Partition Outer Join Data mining I 4 • Enhanced Window functions (percentile,etc) • Rollup, grouping sets, cube • • • • Data mining II SQL Pivot Recursive WITH ListAgg, Nth value window Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Pattern matching Top N clause Identity Columns Column Defaults Data Mining III 5 Pattern Matching in Sequences of Rows The Challenge – a real-world business problem “ … detect if a phone card went from phone A to phone B to phone C... and back to phone A within ‘N‘ hours... ” “… and detect if pattern above occurs at least ‘N’ times within 7 days …” • Currently pattern recognition in SQL is difficult – Use multiple self joins (not good for *) • T1.handset_id <> T2.handset_id <>T3.handset_id AND…. T1.sim_id=‘X’ AND T2.time BETWEEN T1.time and T1.time+2…. – Use recursive query for * (WITH clause, CONNECT BY) – Use Window Functions (likely with multiple query blocks) Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 10 Pattern Matching in Sequences of Rows Objective “Find one or more event A followed by one B Provide native SQL language construct followed by one or more C in a 1 minute interval” Apply expressions across rows Soon to be in ANSI SQL Standard > 1 min Align with well-known regular expression declaration (PERL) A+ B C+ (perl) EVENT TIME LOCATION A 1 SFO A 1 SFO A 2 ATL A 2 ATL A 2 LAX A 2 LAX B 2 SFO C 2 LAX B 2 SFO C 3 LAS C 2 LAX A 3 SFO B 3 NYC C 4 NYC Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 11 Pattern Recognition In Sequences of Rows SQL Pattern Matching - Concept • Recognize patterns in sequences of events using SQL – Sequence is a stream of rows – Event equals a row in a stream • New SQL construct MATCH_RECOGNIZE – Logically partition and order the data • ORDER BY mandatory (optional PARTITION BY) – Pattern defined using regular expression using variables – Regular expression is matched against a sequence of rows – Each pattern variable is defined using conditions on rows and aggregates Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 12 SQL Pattern Matching in action Example: Find a double bottom pattern (W-shape) in ticker stream Find a W-shape pattern in a ticker stream: Stock price Output the beginning and ending date of the pattern Calculate average price each the W-shape Find only patterns that lasted less than a week days Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 13 next = lineNext.getQuantity(); } if (!q.isEmpty() && (prev.isEmpty() || (eq(q, prev) && gt(q, next)))) { state = "S"; return state; } Pattern Matching if (gt(q, prev) && gt(q, next)) { state = "T"; return state; } Finding Double Bottom (W) if (lt(q, prev) && lt(q, next)) { state = "B"; return state; } if (!q.isEmpty() && (next.isEmpty() || (gt(q, prev) && eq(q, next)))) { state = "E"; return state; } if (q.isEmpty() || eq(q, prev)) { state = "F"; return state; } SELECT first_x, last_z FROM ticker MATCH_RECOGNIZE ( PARTITION BY name ORDER BY time MEASURES FIRST(x.time) AS first_x, LAST(z.time) AS last_z ONE ROW PER MATCH PATTERN (X+ Y+ W+ Z+) DEFINE X AS (price < PREV(price)), Y AS (price > PREV(price)), W AS (price < PREV(price)), Z AS (price > PREV(price) AND z.time - FIRST(x.time) <= 7 )) return state; } private boolean eq(String a, String b) { if (a.isEmpty() || b.isEmpty()) { return false; } return a.equals(b); } private boolean gt(String a, String b) { if (a.isEmpty() || b.isEmpty()) { return false; } return Double.parseDouble(a) > Double.parseDouble(b); } private boolean lt(String a, String b) { if (a.isEmpty() || b.isEmpty()) { return false; } return Double.parseDouble(a) < Double.parseDouble(b); } public String getState() { return this.state; } } BagFactory bagFactory = BagFactory.getInstance(); @Override public Tuple exec(Tuple input) throws IOException { long c = 0; String line = ""; String pbkey = ""; V0Line nextLine; V0Line thisLine; V0Line processLine; V0Line evalLine = null; V0Line prevLine; boolean noMoreValues = false; String matchList = ""; ArrayList<V0Line> lineFifo = new ArrayList<V0Line>(); boolean finished = false; 250+ Lines of Java and PIG DataBag output = bagFactory.newDefaultBag(); if (input == null) { return null; } if (input.size() == 0) { return null; } Object o = input.get(0); if (o == null) { return null; } //Object o = input.get(0); if (!(o instanceof DataBag)) { int errCode = 2114; 12 Lines of SQL 20x less code, 5x faster Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | New SQL Functionality • PL/SQL in SQL via the “with” clause – Useful for read only databases – Potentially faster for some operations • IDENTITY based columns – Auto incrementing columns – Supports ANSI standard • Increased size for VARCHAR2 – Increase from 4000 to 32K • New Row limiting clause Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 15 PL/SQL Function embedded in WITH clause Example WITH FUNCTION get_domain(url VARCHAR2) RETURN VARCHAR2 IS pos BINARY_INTEGER; len BINARY_INTEGER; BEGIN pos := INSTR(url, 'www.'); len := INSTR(SUBSTR(url, pos + 4), '.') - 1; RETURN SUBSTR(url, pos + 4, len); END; SELECT DISTINCT get_domain(catalog_url) FROM orders; Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 16 IDENTITY based columns Example • Create a table where the id column is always populated by Oracle CREATE TABLE t1 ( id NUMBER GENERATED AS IDENTITY , first_name VARCHAR2(30) ); • Create a table where the id column is populated by Oracle when not provided CREATE TABLE t2 ( id NUMBER GENERATED BY DEFAULT AS IDENTITY (START WITH 100 INCREMENT BY 10) , first_name VARCHAR2(30) ); Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 17 32K VARCHAR2/NVARCHAR2 Example • Enable 32k support in the Oracle Database 12c ALTER SYSTEM SET max_string_size = EXTENDED SCOPE = SPFILE • Create table with 32k VARCHAR2 CREATE TABLE applicants ( id NUMBER GENERATED AS IDENTITY , first_name VARCHAR2(30) , last_name VARCHAR2(30) , application DATE , cv VARCHAR2(32767) ); Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 18 New Row limiting clause Example • Select only the first 5 rows SELECT employee_id, last_name FROM employees ORDER BY employee_id FETCH FIRST 5 ROWS ONLY; • Select the first 5% of rows and those whose salary “ties” with the lowest of the 5% SELECT employee_id, last_name, salary FROM employees ORDER BY salary FETCH FIRST 5 PERCENT ROWS WITH TIES; Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 19 New Row limiting clause “Who are the 5 less money makers in my enterprise?” Natively identify top N in SQL Simplifies code development ANSI SQL:2008 SELECT empno, ename, deptno FROM (SELECT empno, ename, deptno, sal, comm, row_number() OVER (ORDER BY sal,comm) rn FROM emp ) WHERE rn <=5 ORDER BY sal, comm; versus SELECT empno, ename, deptno FROM emp ORDER BY sal, comm FETCH FIRST 5 ROWS ONLY; Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 20 Oracle Database 12c Big Data SQL Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 21 Oracle Support for Any Data Management System Hadoop NoSQL Change the Business Scale the Business Scale-out, low cost store Collect any data Map-reduce, SQL Analytic applications Relational Run the Business Scale-out, low cost store Collect key-value data Find data by key Web applications Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Scale-out and scale-up Collect any data SQL Transactional and analytic applications for the enterprise Secure and highly available 22 Oracle Big Data SQL One fast SQL query , on all your data. Oracle SQL on Hadoop and beyond • With a Smart Scan service inspired by Exadata • With native SQL operators • With the security and certainty of Oracle Database Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 23 Accessing Big Data Select w.sess_id, w.cust_id, w.pag e_id From web_logs w W h e r e w . s o u r c e _ c o u n t r y = ‘B r a z i l ’ A n d w . c a t e g o r y = ‘T V ’ A n d w . c h a n n e l = ‘M o b i l e ’ Without SQL Push Down Request for Data 100’s of Terabytes of Data WEB_LOGS Hadoop Cluster Low utilization of available resources All columns and rows from the table are returned Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | High load on database server 24 Big Data SQL Push Down Select w.sess_id, w.cust_id, w.pag e_id From web_logs w W h e r e w . s o u r c e _ c o u n t r y = ‘B r a z i l ’ A n d w . c a t e g o r y = ‘T V ’ A n d w . c h a n n e l = ‘M o b i l e ’ With SQL Push Down Big Data SQL SQL shipped to BDA 10’s of Gigabytes of Data WEB_LOGS Hadoop Cluster Only columns and rows needed to answer query are returned Good utilization of available resources. SQL executed on Hadoop cluster Lower load on Server, Faster response Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 25 Big Data SQL Push Down S e l e c t w.ses s_id , w.cust_id, w.page_id, c.name From web_logs w, customers c W h e r e w . s o u r c e _ c o u n t r y = ‘B r a z i l ’ A n d w . c a t e g o r y = ‘T V ’ A n d w . c h a n n e l = ‘M o b i l e ’ And c.customer_id = w.cust_id With SQL Push Down Big Data SQL SQL shipped to BDA 10’s of Gigabytes of Data WEB_LOGS CUSTOMERS Hadoop Cluster Only columns and rows needed to answer query are returned Good utilization of available resources. SQL executed on Hadoop cluster Data joined between CUSTOMERS and WEB_LOGS on server Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 26 Big Data SQL Push Down S e l e c t w.ses s_id , w.cust_id, w.page_id, c.name From web_logs w, customers c W h e r e w . s o u r c e _ c o u n t r y = ‘B r a z i l ’ A n d w . c a t e g o r y = ‘T V ’ A n d w . c h a n n e l = ‘M o b i l e ’ And c.customer_id = w.cust_id With SQL Push Down Big Data SQL WEB_LOGS SQL Push Downs supported by Big Data SQL • • • • • SQL shipped to BDA Hadoop scans (InputFormat, SerDe) JSON 10’s parsing of Gigabytes of Data WHERE clause evaluation Column projection Bloom filters for faster join Hadoop Cluster CUSTOMERS Only columns and rows needed to answer query are returned Good utilization of available resources. SQL executed on Hadoop cluster Data joined between CUSTOMERS and WEB_LOGS on server Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 27 New Data Sources for Oracle External Tables • New set of properties CREATE TABLE web_logs (click VARCHAR2(4000)) ORGANIZATION EXTERNAL ( TYPE ORACLE_HIVE DEFAULT DIRECTORY Dir1 ACCESS PARAMETERS ( com.oracle.bigdata.tablename logs com.oracle.bigdata.cluster mycluster) ) REJECT LIMIT UNLIMITED – ORACLE_HIVE and ORACLE_HDFS access drivers – Identify a Hadoop cluster, data source, column mapping, error handling, overflow handling, logging • New table metadata passed from Oracle DDL to Hadoop readers at query execution • Architected for extensibility – StorageHandler capability enables future support for other data sources – Examples: MongoDB, HBase, Oracle NoSQL DB Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 28 Advanced Query & Analysis Full Power of SQL and Advanced Analytics Leverages All Your Data Relational, Hadoop and NoSQL Secure Unified Governance on All Data Fastest Performance Utilize SQL Processing Across the Platform Transparent to Applications No Changes to Application Code Big Data Management System Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 29 Oracle Database 12c JSON und REST Data Services Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 30 Oracle REST Data Services New-generation RESTful access to Oracle Database • Retrieve and store data via standard HTTP URI RESTful GETs and POSTs – Build applications without writing SQL and without specialized drivers • JavaScript framework friendly – Results returned in JSON format • Ships with Oracle Database 12c Release 1 (12.1.0.2) – Formerly known as Oracle APEX Listener Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 31 Oracle REST Data Services Oracle REST Data Services Use REST to get JSON from any data source JSON Collection API Pass Back URI {JSON} HTTP(S) client Map & Bind Transform to JSON NoSQL API PassBack Auto Generated SQL {JSON} SQL SQL Result Set Data held in JSON Document Store Data held in Relational Table Key Value Lookup {JSON} Data held in Oracle NoSQL Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 32 JSON Support in Oracle Database Flexible Application Development + Powerful SQL Analytics Oracle Database 12c JSON Data accessed via RESTful service or native API’s SQL Data persisted in database In JSON Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Data analyzed via SQL 33 JSON Support in Oracle Database • Store and manage JSON documents in Oracle Database – JSON stored as text (VARCHAR2 or CLOB) – Full text index (inverted index) on entire JSON documents – Functional indexes on specific attributes in JSON documents • Access JSON via developer-friendly API’s – REST services – Document-store API for Java (additional languages planned) • SQL query capabilities over JSON documents • Powerful reporting and analysis directly on JSON • Simple integration with relational data Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 34 Storing and Querying JSON • Application developers – Store JSON using RESTful API PUT /my_database/my_schema/customers HTTP/1.0 Content-Type: application/json Body: { "firstName": "John", “lastName”: "Smith", "age": 25, "address": { "streetAddress": "21 2nd Street", "city": "New York", "state": "NY", "postalCode": "10021“, "isBusiness" : false }, "phoneNumbers": [ {"type": "home", "number": "212 555-1234“ }, {"type": "fax", "number": "646 555-4567“ } ] } • Analytical tools and business users – Query JSON using SQL select c.document.firstName, c.document.lastName, c.document.address.city from customers c; firstName ----------“John” lastName ----------“Smith” Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | address.city -------------“New York” 35 Oracle Database 12c for the Developer Supporting all major development environments and API’s Java .NET Node.js REST (ORDS) Ruby Python PHP R Perl SQL API Document Store API H2 2014 Use REST API H2 2014 DB 12.1.0.2 Use REST API Use REST API Use REST API Use new document-store API’s to build applications without writing SQL Use REST API Use REST API Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 36 Oracle Database 12c In-Memory Technologien Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 37 Result Cache Database Buffer Default Cache Keep Cache Recycle Cache LRU-Algorithmus Exadata X4 In-Memory Hybrid Columnar Compression Smart Scan Storage Index Smart Flash Cache (Read/Write) Smart Flash Logging (Redo-Write) Shared Pool oder Client Transparent für Anwendungen Technik der Flash Cache Datenbank Data Buffer Erweiterung 2nd level Cache Auch für Datafiles Memory optimierende Techniken Compression Partitioning Parallelisierung OLAP In-Memory Option 12c In-Memory Technik der TimesTen IMDB Middle-Tier IMDB Database Cache Option TimesTen for Exalytics Row und Column In-Memory Formate auf der selben Tabelle Simultan und transaktionsKonsistent Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Coherence Data Grid Object Cache / Data Grid für verteilte Anwendungen 38 38 Optimizing Transaction and Query Performance Row Format Databases versus Column Format Databases SALES – Example: Query or Insert a sales order – Fast processing few rows, many columns Row SALES Column • Transactions run faster on row format • Analytics run faster on column format – Example : Report on sales totals by region – Fast accessing few columns, many rows Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Row Store vs. Column Store • OLTP-style query Row Store Column Store SELECT * FROM salesorders WHERE document_number = 95779216 • OLAP-style query SELECT sum(value) FROM salesorders WHERE document_date > 2013-08-28 Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 40 Breakthrough: Dual Format Database Memory Memory SALES SALES Row Format Column Format • BOTH row and column formats for same table • Simultaneously active and transactionally consistent • Analytics & reporting use new in-memory Column format • OLTP uses proven row format Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 41 Oracle In-Memory Design Goals Target Specification • Integration in existing Infrastructure – High Availability – Backup & Recovery and Desaster Recovery • Applications should profit without any change – Transparent Usage – SQL & Optimizer aware – OLTP & Analytics – No Data Migration Memory OLTP Row Format Memory Analytics Column Format • Ease of Use – Simple Definition through SQL Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 42 Complex OLTP is Slowed by Analytic Indexes Table 1–3 OLTP Indexes 10 – 20 Analytic Indexes • Most Indexes in complex OLTP (e.g. ERP) databases are only used for analytic queries • Inserting one row into a table requires updating 10-20 analytic indexes: Slow! • Indexes only speed up predictable queries & reports Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 43 Column Store Replaces Analytic Indexes Table 1–3 OLTP Indexes In-Memory Column Store • In-Memory Column Store replaces analytic indexes and removes their overhead for tables that fit in memory • Fast analytics on any columns – Better for unpredictable analytics – Less tuning & administration • Column Store not persistent so update cost is much lower Less tuning & admin • Both predefined and ad-hoc analytic queries run faster • OLTP & batch run faster Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 44 Weitere Informationen http://search.oracle.com database Technologie Überblick auf: http://www.oracle.com/database Whitepapers und Webinars ansehen Technische Information, Demos, Software auf OTN: otn.oracle.com products database Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 45 Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 46 Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 47 Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 48 Oracle Database 12c SQL und PL/SQL Oracle Database 12c SQL Language Miscellaneous Enhancements Data Types SQL row limiting clause Increased length limits of the Data Types Invisible column ANYDATA Enhancements Making DDLs ONLINE SQL column defaulting on explicit NULL Enhancements to Partitioning concepts Gateways Enhancements Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | SQL IDENTITY Column SQL column defaulting on Sequence