* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Powering the Cloud with Oracle WebLogic
Survey
Document related concepts
Concurrency control wikipedia , lookup
Oracle Database wikipedia , lookup
Microsoft Jet Database Engine wikipedia , lookup
Ingres (database) wikipedia , lookup
Functional Database Model wikipedia , lookup
Entity–attribute–value model wikipedia , lookup
Microsoft SQL Server wikipedia , lookup
Open Database Connectivity wikipedia , lookup
Clusterpoint wikipedia , lookup
Extensible Storage Engine wikipedia , lookup
Transcript
Lucas Jellema (AMIS, The Netherlands) THE VERY, VERY LATEST IN ORACLE DATABASE DEVELOPMENT Oracle Open World 2012, San Francisco Public Expertezed Session – Thursday 29th November 2012 THE VERY VERY VERY LATEST… <Secret Code> TOM KYTE TO THE RESCUE… THE DATABASE IN MODERN ARCHITECTURES Oracle Open World 2012, San Francisco NO SQL THE TOP-3 EARNING EMPLOYEES • What can you say about the result of this query with respect to the question: “Who are our top three earning employees?” A. B. C. D. Correct Answer Sometimes correct Correct if there are never duplicate salaries Not Correct IN-LINE VIEWS TOP-N QUERIES IN 12C • Last part of a query to be evaluated – to fetch only selected rows from the result set: select * from emp order by sal desc FETCH FIRST 3 ROWS ONLY; – To select the next set of rows: select from order by OFFSET * emp sal desc 3 FETCH NEXT 4 ROWS ONLY; TOP-N% QUERYING • To query for a percentage of the result set (rather than an absolute number of rows) select * from emp order by sal desc FETCH FIRST 30 PERCENT ROWS ONLY; • And the next batch select * from emp order by sal desc OFFSET (0.3*(select count(*) from emp)) ROWS FETCH NEXT (0.3*(select count(*) from emp)) ROWS ONLY; BOTTOM-N QUERY IN 12C • Return only the last three rows in the ordered result set (in the proper order) select * from emp order by sal desc OFFSET ((select count(*) from emp)-3) ROWS FETCH NEXT 3 ROWS ONLY – or: select * from ( select * from emp order by sal asc FETCH FIRST 3 ROWS ONLY ) order by sal desc; IN-LINE PL/SQL FUNCTIONS AND PROCEDURES • Procedures are also allowed in-line • In-Line Functions and Procedures can invoke each other WITH procedure increment( operand in out number , incsize in number) is begin operand:= operand + incsize; end; FUNCTION inc(value number) RETURN number IS l_value number(10):= value; BEGIN increment(l_value, 100); RETURN l_value; end; SELECT inc(sal) from emp SPECIAL ‘BUSINESS RULE’: DEFAULT VALUE • The default values is the value that should be inserted for a column when the client has ignored the column – not provided a value nor indicated NULL • The default value is applied prior to the execution of the Before Row trigger – So :new.<column_value> has the value that will be inserted – The Before Row trigger has no built in way to telling whether the value was provided by the client or supplied as default by the database • Default value is typically used for auditing purposes – Note: default values for columns exposed in UI should be set in the client COLUMN DEFAULT • Columns can have default values – Static or literals – SQL expressions evaluating to a static – Pseudo-columns like USER and CURRENT_DATE • DO NOT USE SYSDATE! DO NOT USE USER! – References to Application Context parameters • sys_context(‘USERENV’, ‘IP_ADDRESS’).. – Some funny value to let the before row trigger know that the real (complex) default must be calculated create table citizens ( name varchar2(100) default 'John Doe' , birthdate date default current_date - 1 , city varchar2(50) default sys_context('KANE_CTX', 'DEFAULT_CITY' ) , zipcode varchar2(8) default 'XYXYXYXYXQQ' ) NEW OPTIONS WITH DEFAULT VALUE alter table emp modify (sal number(10,2) DEFAULT ON NULL 1000 ) alter table emp modify (empno number(5) NOT NULL DEFAULT ON NULL EMPNO_SEQ.NEXTVAL ) create table emp ( empno NUMBER GENERATED BY DEFAULT AS IDENTITY (START WITH 100 INCREMENT BY 10) , ... ) APPLICATION CONTEXT • Memory area that enables application developers to define, set, and access key/value pairs Application • Rapid access in SQL and PL/SQL Context Attribute Attribute Value Value Attribute Value Pairs select sys_context('USERENV', 'SESSION_USER') from dual l_user:= sys_context('USERENV', 'SESSION_USER') • Two Application Contexts are always around: – CLIENTCONTEXT and USERENV APPLICATION CONTEXT APPEARANCES • Per session (default) – Stored in UGA, just like package state • Globally Accessible (shared across all sessions) – Stored in SGA • Associated with a Client Identifier – Attributes in a Globally Accessible Application Context can explicitly be tied to the Client Identifier – And are only accessible to sessions with that Client Identifier TYPICAL WEB ARCHITECTURE USING CONNECTION POOL JDBC Connection Pool Session 1 Package A globals Session 2 Package B Session 3 Package C Session 4 PACKAGE STATE IS TIED TO DATABASE SESSION JDBC Connection Pool Session 1 Session 2 Session 3 globals Package A globals Package B Package C Session 4 PACKAGE STATE IS TIED TO DATABASE SESSION – NOT WEB SESSION JDBC Connection Pool Session 1 Session 2 Session 3 globals Package A globals Package B Package C Session 4 APPLICATION CONTEXT TO RETAIN STATE FOR LIGHT WEIGHT END USERS JDBC Connection Pool Session 1 Package A globals Session 2 Session 3 globals ? Package B Package C Session 4 APPLICATION CONTEXT TO RETAIN STATE FOR LIGHT WEIGHT END USERS JDBC Connection Pool Session 1 Session 2 Session 3 USERENV Package A globals USERENV Global Context globals globals Session 4 Package C APPLICATION CONTEXT TO RETAIN STATE FOR LIGHT WEIGHT END USERS JDBC Connection Pool Session 1 USERENV Package A globals Session 2 Session 3 USERENV USERENV Global Context globals globals Session 4 Package C PACKAGE GLOBALS: THE STATE OF THE PACKAGE IN A SESSION • This state is lost when the package is recompiled – That is undesirable in a highly available environment Package PACKAGE GLOBALS CAN BE REPLACED BY APPLICATION CONTEXT • The Application Context is untouched by recompilation of the package – All ‘globals’ in the application context retain their values Package Application Context EBR TO KILL PLANNED DOWNTIME (BECAUSE OF APPLICATION UPGRADE) Application X VERSION 1 Application X VERSION 2 TIME TRAVELLING FLASHBACK • • • • Introduced in 9i Based on UNDO Initially only for recovery As of 11g – Total Recall option with Flashback Data Archive – Controlled history keeping • Look back into history – Query trends (version history) – Difference reporting – Audit trails (Replace journaling tables) • Require trick for transaction history: WHO? • Also: when is the start of history? OOW 2012 SESSION COMES TO THE RESCUE • CON8511 - Temporal Database Capabilities with the Latest Generation of Database Technology TOTAL RECALL - FLASHBACK DATA ARCHIVE IMPROVEMENTS • Complete schema evolution support: all table definition, partitioning, and space management DDLs are supported on FDA-enabled tables. • The metadata information for tracking transactions including the user context is now tracked. – This could mean that journaling tables are now officially deprecated • And the current contents of journaling tables can even be migrated to Flashback Data Archive • Introduction of SQL 2011 Valid Time Temporal Modeling TOTAL RECALL • Import and export of history – Support for import and export using Data Pump for FDA-enabled tables. Data Pump can now be used to export and import an FDA-enabled base table along with its schema-evolution metadata and historical row versions. • Construct and manipulate the Flashback Data Archive – import user-generated history • Restore points: Support for the use of named restore points in AS OF and versions queries has been added. • Total Recall will (in all likelihood) be part of every edition of the database – including SE VALID TIME TEMPORAL MODELING • Validity (or effectivity) of facts recorded in a database is frequently specified through dates or timestamps – For example begin date and [derived] end date of a price, membership, allocation, certificate, agreement • This valid time can differ from the transaction time at which a record is entered into the database • Multiple entries with different, non-overlapping validtime periods can exist for a single entity • In 12c the notion of Valid Time is introduced into the Oracle Database – The valid-time dimension consists of two date-time columns specified in the table definition (create or alter) – These Valid Time columns specify the period during which a record is valid – A table can have multiple valid_time markers CREATING A TABLE WITH VALID TIME DIMENSION • Table with explicit valid time columns: CREATE TABLE EMP ( employee_number NUMBER , salary NUMBER , department_id NUMBER , name VARCHAR2(30) , hiredate TIMESTAMP , firedate TIMESTAMP , PERIOD FOR user_time (hiredate, firedate) ); • Table with valid time dimension and implicit columns: CREATE TABLE EMP ( employee_number NUMBER , salary NUMBER , department_id NUMBER , name VARCHAR2(30) , PERIOD FOR contract_time ); columns valid_time_start and valid_time_end (TIMESTAMP) are added implicitly VALID TIME AWARE FLASHBACK QUERIES • Select all employees who were employed at a certain moment in time SELECT * FROM EMP AS OF PERIOD FOR user_time TO_TIMESTAMP('01-JUN-2012 12.00.01 PM') • Perform all queries for records that are valid at a certain point in time EXECUTE DBMS_FLASHBACK_ARCHIVE.enable_at_valid_time ( 'ASOF' , TO_TIMESTAMP('29-JUL-12 12.00.01 PM') ); • Return all records currently (session time) valid EXECUTE DBMS_FLASHBACK_ARCHIVE.enable_at_valid_time('CURRENT'); • Return all records (default) EXECUTE DBMS_FLASHBACK_ARCHIVE.enable_at_valid_time('ALL'); DATABASE IN MODERN ARCHITECTURE WS Mobile Business Tier Enterprise Service Bus Cache/Grid (L1, L2, L3) Services Database Database Standard Application s Legacy Application s MULTI TIER ARCHITECTURE WS Mobile Business Tier Enterprise Service Bus HTTP REST HTTP SOAP FTP/WEBDAV JDBC JPA (H/EL) Services Monitor, Trace, Audit Database Authentication & Fine Grained Authorization Cache/Grid (L1, L2, L3) DB QRCN HTTP JMX, JMX Stored Procedures Encapsulation Decoupling Caching Business Logic APPLICATION ARCHITECTURE: DRIVE APPLICATION FROM META DATA • Agility • Design Time at Run Time • Define part of the application behavior and appearance through meta-data (outside the base source code) – The default settings are defined by developers and deployed along with the application – Read and interpreted at run time – Manipulated and re-read and re-interpreted at run time Application • Note: very similar to the way the database operates: – Data Dictionary is the meta-data driving the meta behavior of the database SEPARATE BASE DATA AND CUSTOMIZED DATA • If a value is changed during site-level implementation – Or run time customization • It should be kept apart from the base ‘meta-data’ – To prevent overwriting customized data when the new release arrives – To allow for (temporarily) reverting to base data • A simple solution: the Complex View with two underlying tables approach – Note: Select… For Update Of ORIGINAL_NAME is not allowed IO trg Customized Values New release BaseValues REPLACE THE ORIGINAL SINGLE TABLE WITH A TWO-TABLE BASE/CUSTOM SPLIT • rename <original> to <base> • create table <customizations> as select * from base where rownum = 0 • create or replace view <original> as select * from <customizations> union all select * from <base> b left outer join <customizations> c on (b.id = c.id) where c.rowid is null REPLACE THE ORIGINAL SINGLE TABLE WITH A TWO-TABLE BASE/CUSTOM SPLIT (2) • create or replace trigger handle_insert_trg instead of insert on original for each row begin insert into <customizations> (col, col2,…) values(:new.col, :new.col2,…); end; • create or replace trigger handle_update_trg instead of update on original for each row begin update <customizations> set col = :new.col, … where id = :new.id ; if sql%rowcount = 0 then insert into <customizations> (id, col, col2,…) (select id, :new.col, :new.col2 from base where id = :new.id); end if; end; VERY SIMILAR TO THE ARCHITECTURE OF PLUGGABLE DATABASES New release of Oracle Database ROOT PDB APPLICATION ARCHITECTURE: NO SQL • NO SQL – Complex SQL is hidden away inside the database – Cache to not have to query all the time from the database – … and to not take the overhead of a commit for not so important data NO – Process first – in memory, on middle tier (BigData and CEP) and only persist what is useful Web Browser SQL JEE Application Server RDBMS SQL QUERY RESULT CHANGE NOTIFICATION • Continuous Query Notification: – Send an event when the result set for a query changes – Background process calls PL/SQL Handler or Java Listener or OCI client when the Java commit has occurred Listener – Event contains rowid of changed rows • Used for: – Refreshing specific data caches (middle tier, global context) – (custom) Replication PL/SQL CONTINUOUS PROCESSING OF DATA STREAMS USING CQL • Aggregation, Spot deviation, Match on complex patterns WHO IS AFRAID OF RED, YELLOW AND BLUE • Table Events – Column Seq number(5) – Column Payload varchar2(200) SOLUTION USING LEAD • With LEAD it is easy to compare a row with its successor(s) – As long as the pattern is fixed, LEAD will suffice with look_ahead_events as ( SELECT e.* , lead(payload) over (order by seq) next_color , lead(payload,2) over (order by seq) second_next_color FROM events e ) select seq from look_ahead_events where payload ='red' and next_color ='yellow' and second_next_color='blue' FIND THE PATTERN RED, YELLOW AND BLUE • Using the new 12c Match Recognize operator for finding patterns in relational data SELECT * FROM events MATCH_RECOGNIZE ( ORDER BY seq MEASURES RED.seq AS redseq , MATCH_NUMBER() AS match_num ALL ROWS PER MATCH PATTERN (RED YELLOW BLUE) DEFINE RED AS RED.payload ='red', YELLOW AS YELLOW.payload ='yellow', BLUE AS BLUE.payload ='blue' ) MR ORDER BY MR.redseq , MR.seq; MATCH_RECOGNIZE FOR FINDING PATTERNS IN RELATIONAL DATA • The expression MATCH_RECOGNIZE provides native SQL support to find patterns in sequences of rows Table Source & Where Match_ Recognize Process and Filter Select & Order By • Match_recognize returns Measures for selected (pattern matched) rows – Similar to MODEL clause • Match Conditions are expressed in columns from the Table Source, aggregate functions and pattern functions FIRST, PREV, NEXT, LAST • Patterns are regular expressions using match conditions to express a special sequence of rows satisfying the conditions DID WE EVER CONSECUTIVELY HIRE THREE EMPLOYEES IN THE SAME JOB? • Find a string of three subsequent hires where each hire has the same job • Order by hiredate, pattern is two records that each have the same job as their predecessor SELECT * FROM EMP MATCH_RECOGNIZE ( ORDER BY hiredate MEASURES SAME_JOB.hiredate AS hireday , MATCH_NUMBER() AS match_num ALL ROWS PER MATCH PATTERN (SAME_JOB{3}) DEFINE SAME_JOB AS SAME_JOB.job = FIRST(SAME_JOB.job) ) MR THE SHOPPING ALGORITHM THE SHOPPING ALGORITHM • shopForItem Item ( String itemName) { driveToShop; Item item = buyItemAtShop ( itemName); driveHomeFromShop; return item; } GET THIS WEEK’S GROCERIES getGroceries Item[] ( String[] shoppingList) { Item[] items = new Item[ shoppingList.length]; for (int i=0; i < shoppingList.length; i++) { items[i] = shopForItem (shoppingList[i]); } return items; } PENSION FUND – SEPTEMBER 2012 Employer Participants Job & Benefits < > FETCHING THE DATA OF THE PENSION FUND FOR THE WEB APPLICATION < > select * from employers where id = < 324> 1 record 100s records select * from participants where employer_id = < 324> 10s records select * from benefits where participant_id = <#> REPORTING ON MANY EMPLOYERS select * from employers 100s records 1 query 10k records select * 100s queries from participants where employer_id = <#> 100k records select * from benefits 10k queries where participant_id = <#> APPLICATION ARCHITECTURE – BULK RETRIEVE • Have the database bulk up the data retrieval • Return Ref Cursor, Types and Collections or JSON/XML Benefits Package select * from employers where id in <some set> select * from participants where employer_id in <some set> select b.* from benefits b join participants p on (p.id = b.participant_id) where p.employer_id in <some set> APPLICATION ARCHITECTURE – SERVICE ENABLING WebLogic Server Database Native DB WebService HTTP EPG Java/JEE PL/SQL package SOA Suite JDBC AQ View Oracle Service Bus Other (Email, FTP/File, XMPP/Chat) XML DB Chat/IM XMPP Server File/FTP Server Email Server Table XML/JSON Relational/Oracle Type JEE Server Database 11g Native DB WebService HTTP 10g EPG http Java App SDO WS Oracle Service Bus DB Adapters SOA Suite ADF BC /SDO WS XML XML & XSD JAX-WS JPublisher WS Ref Cursor JDBC AQ 8i AQ JMS JMS Queue JMS EJB File FTP JSON/ CSV EJB/JPA Pojo UMS Other (Email, FTP/File, XMPP/Chat) Chat/IM XMPP Server File/FTP Server Email Server XML PL/SQL package Types & Coll Types utl_file, BFILE, URITYPE 9i XML DB View Table BUSINESS RULES • Data Oriented Rules or Data Constraints • Declarative support in database – For referential integrity • Order must be for a Customer – For attribute and tuple rules • Salary must be numeric, • Hiredate may not be in the future, • End date must come after begin date • No declarative support for complex data rules – across multiple records and tables – A department in France may not have less then 20% female employees – Order items of type weapon may not be part of an order that ships around Christmas BUSINESS RULES – WHERE AND HOW TO IMPLEMENT • Criteria: – Safe – Well performant – Reusable and maintainable – Productive to implement • Options – Client side • JavaScript – Middle-tier • Java, Enterprise Service Bus – Database • Constraints and triggers are statement level – i/o transaction level RDBMS NOT ALWAYS EXCLUSIVELY ACCESSED THROUGH ONE LAYER SOA, ESB, WebServices Batch Bulk Processes Standard Applications Database Data Replication & Synchronization Legacy Applications 11G VIRTUAL COLUMNS • Add columns to a table based on an expression – Using ‘real’ columns, SQL Function and User Defined Functions – No data is stored for Virtual Columns, only meta-data VIRTUAL – Virtual Columns can be indexed alter table emp ADD (income AS (sal + nvl(comm,0))) UNIQUENESS RULES USING VIRTUAL COLUMNS • Business Rule: – Not more than one manager per department alter table emp add constraint only_one_mgr_in_dept_uk unique (one_mgr_flag) alter table emp ADD ( one_mgr_flag as ( case when job ='MANAGER' then deptno end ) ) CHALLENGE: ORDERS BELONG TO A CUSTOMER IN ONE OF TWO TABLES • The Orders table contains Order records for customers – either Dutch or Australian customers • These customers are stored in two different tables • Can we implement referential integrity to ensure that the order’s customer exists? OZ_CUSTOMER ORDER Country Customer_Id …. ? Id Name DUTCH_CUSTOMER Id Name USING VIRTUAL COLUMNS IN FOREIGN KEY RELATIONS • A foreign key can be created on a Virtual Column – That means for example we can have a single column with some id – And two virtual columns with CASE expressions that produce NULL or the ID value – With Foreign Keys on the Virtual Columns ORDER Country Customer_Id Dutch_id (VC) Australian_id (VC) OZ_CUSTOMER Id Name DUTCH_CUSTOMER Id Name USING VIRTUAL COLUMNS IN FOREIGN KEY RELATIONS alter table orders add (australian_ctr_id as (case country when 'OZ' then customer_id end)) ORDER Country Customer_Id Dutch_id (VC) Australian_id (VC) alter table orders add (dutch_ctr_id as (case country when 'NL' then customer_id end)) alter table orders add constraint odr_ocr_fk foreign key (australian_ctr_id) references oz_customer (id) OZ_CUSTOMER Id Name DUTCH_CUSTOMER Id Name alter table orders add constraint odr_dcr_fk foreign key (dutch_ctr_id) references dutch_customer (id) FOREIGN KEY SHOULD ONLY REFER TO CERTAIN RECORDS USING VC • Foreign Key can reference a UK based on a Virtual Column • That allows a ‘conditional foreign key’ or a foreign key that can only reference specific records in the referenced table – Only refer to Women in the PEOPLE table for the Mother Foreign Key – Only refer to Values in the Domain Values table for the Domain Name == ‘COLORS’ RESTRICTED FOREIGN KEYS USING VIRTUAL COLUMNS alter table domain_values add (color_value as (case domain_name when 'COLORS' then domain_value end)) CARS ID Make Type Color Year alter table domain_values add (country_value as alter table domain_values (case domain_name add (country_value as whendomain_name 'COUNTRIES' (case then'COUNTRIES' domain_value when end)) then domain_value end)) DOMAIN_VALUES Id Domain_Name Domain_Value Color_Value Gender_Value OrderStatus_Value Country_Value ShipmentMethod_Value alter table cars add constraint car_clr_fk foreign key (color) references domain_values (color_value) LACK OF WATERTIGHTNESS IN TRIGGER BASED RULE VALIDATOIN VALIDATION • Statement time validation means: DML in different session DML validation • • • More DML Commit validation To prevent leakage we should validate at commit time – Logically correct as transaction is the logical unit – Effects from other sessions between statement and commit are taken into account However: Oracle unfortunately does not provide us with a pre-commit or on-commit trigger Workarounds: – Dummy Table with Materialized View On Commit Refresh and Trigger on Materialized View – Do a soft-commit by calling a package to do the actual commit – that will first do transaction level checks • Supported by a deferred check constraint that is violated by each operation that potentially violates a business rule SAFE SOLUTION: USE CUSTOM LOCKS • Prior to validating a certain business rule for a specific record – acquire a custom lock – That identifies both Rule and Record – Using dbms_lock DML in different session DML validation More DML Commit validation • When a record is being validated for a certain rule, other sessions have to wait • The commit (or rollback) releases all locks • Validation in a different session will include all committed data SUMMARY • • • • • • • • • Inline Views Defaulting Application Context Flashback and the time dimension NoSQL means smart SQL – Cache refresh driven by change notification – Streaming analysis before persisting Decoupling galore – Bulk retrieval – Service enabling Business Rules EBR 12c promises even more