Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
<Insert Picture Here> Oracle Warehouse Builder: Overview Jean-Pierre Dijcks and Michelle Bird Warehouse Builder Product Management Agenda • Integrated Solution • Market Position • Roadmap – Merging the ODI and OWB products – Common pricing bundle • Product Overview – New Features in OWB 11.1 – Preview of OWB 11.2 • Pricing and Packaging • For More Information ... <Insert Picture Here> <Insert Picture Here> Integrated Solution Distributed Data Marts and Servers Expensive data warehouse architecture Data Marts Online Analytics ETL Data Mining Typical ETL Multiple tools require separate training, install, maintenance Generic Repositories Metadata Exchange Classic Data Integration Tools Classic Data Quality Consolidated Data Warehouse Single source of truth on low cost servers & storage Data Marts Online Analytics OWB ETL Data Mining Oracle Database 11g with integrated ETL, Analytics & Data Mining OWB Integrated in the Oracle Database Centralized metadata repository • hosted on Oracle Database secure, scalable • import metadata • manage & propagate metadata changes Profile and cleanse data • profile applications, tables, files, … • derive or write custom rules Transform cleansed data • rich library of transformations and operators • leverage Oracle partitioning Monitor incoming data • automatic notifications if data quality falls below threshold Oracle Warehouse Builder OWB is optimized for Oracle environments • Benefit from your Oracle investments – Better performance through Oracle optimizations – Skills, people and technology • Improved productivity – Use the latest Oracle features without rewriting your code – Advanced task automation features • Shorten your project times while improving data quality – Discover problems up-front with data profiling – Improve data quality in a declarative way In-Database ETL Higher Productivity In-house ETL Oracle InDatabase ETL Tune and Administer Test and Update Create Data Quality Routines Tune and Administer Test and Update Create Data Transformations Years to Months Create DQ Routines Create Data Transformations Months to weeks Advanced features made easy Any SQL or PL/SQL expression Parallel Loading with DML Error Handling Heterogeneous Source Access Fuzzy Matching and Merging Impact Analysis and Lineage Partition Exchange Loading <Insert Picture Here> Mark Rittman Co-founder, Rittman Mead Consulting “"Rittman Mead recommends Oracle Warehouse Builder because it is the most effective way to leverage advanced capabilities of the Oracle Database for ETL development, while also providing metadata management and data quality features required for full life-cycle deployments.” <Insert Picture Here> Market Position Market Position Recognized as a leader since OWB 10.2 • 10,000+ companies rely on OWB • Optimized for Oracle environments 2005 <Insert Picture Here> Roadmap Product Highlights An abbreviated history of OWB 2000 2003 2004 2006 2007 Release 2.0.4 Release 9.2 Release 10.1 Release 10.2 Release 11.1 Basic ETL functionality Oracle RAC certified Oracle 10g certified Data profiling & auditing Simplified installation: installed w/ Oracle 11g DB Mapping Editor Set and row based code generation Mapping Debugger Interactive LIA Name & Address Match/Merge 3,000+ companies Release 9.0.4 Compete against Informatica and Ascential Metadata Security Process Flow Editor Scripting Language Multi-table insert Pivot and Unpivot Recognized as a leader in Gartner Magic Quadrant Scheduling Improved connector for SAP R/3 Data Watch and Repair (for Oracle MDM) Siebel Adapter EBS and PeopleSoft connectors OLAP and relational modeling Pluggable mappings Non-Oracle targets Recognized as a leader in Forrester Enterprise ETLWave 10,000+ companies High-Level Roadmap Unified platform that is a superset of OWB and ODI 10gR1 10gR2 11gR1 • OWB/ODI investments are 11gR2 fully protected • No forced migrations OWB • Natural upgrade path Unified dev team “ CY 2012” Common pricing ODI 11gR2 10gR3 11gR1 • No regressions in functionality • Training and support to continue for both product releases Common Pricing Includes ODI product and OWB Enterprise ETL option DATA INTEGRATOR ENTERPRISE EDITION DATA INTEGRATOR WAREHOUSE BUILDER ENTERPRISE ETL ODI Flagship Data Integration Product Heterogeneous, HighPerformance Data Integration Modern Java/SOA Architecture Flexible Knowledge Module framework OWB EETL Flagship ETL Product Extensible Metadata Management Wizards for Slowly Changing Dimensions Interactive Lineage and Impact Analysis Oracle’s Roadmap for its ETL tools • Long Term Direction: Deliver an integrated data integration product • Short Term: Deliver stand-alone releases with first integration steps – Shared knowledge module framework (OWB 11.2) – Runtime interoperability (future releases) – Same user interface framework (future releases) • Protect investments for the entire customer base http://www.oracle.com/technology/products/warehouse/pdf/owb_roadmap.pdf <Insert Picture Here> Product Details Oracle Warehouse Builder The ETL and DQ tool shipped with Oracle Database • Oracle Database as transformation engine • Oracle Database as metadata repository Oracle Database 11g Oracle Warehouse Builder 11.1 Heterogeneous In-line Data Integration Data Quality Unified Unified Metadata Metadata Smart Data Modeling for Oracle OWB 11.1 Oracle-centric data integration platform BI Tool Integration OBI EE (Enterprise Edition) : design & leverage Oracle objects Discoverer (Standard Edition): design & deploy EUL Databases Databases Non-Oracle BI : design & leverage Oracle objects Oracle Database 11g Applications Applications Connectivity Connectivity Oracle Warehouse Builder 11.1 Odbc / Gateways Files Connectors CDC: Queues / Streams/ Third Party Messages Odbc / Gateways Heterogeneous In-line Data Integration Data Quality SQL*Loader/ Ex Table Analytics Smart Data Modeling for Oracle Connectors Files CDC: Queues / Streams/ Third Party Messages Analytics Unified Unified Metadata Metadata Semi-Structured Semi-Structured Web Services Web Services Architecture Design Runtime Deploy Sources & Targets Data Sources • Oracle – Tables, Views, MViews, Queues, External Tables, Sqlloader, Transportable Tablespaces, Data Pump… • DB2, Sybase, SQLServer, Informix, Mainframes, … (Oracle Transparent Gateways) • ODBC • Flat Files • XML • Applications – Oracle eBusiness Suite – PeopleSoft – SAP R/3 – Siebel – Oracle MDM (UCM, CDH, & PIM) Data Targets* • Oracle • DB2, Sybase, SQLServer, Informix, Mainframes, … (Oracle Transparent Gateways) • ODBC • Flat Files • XML • Applications – Oracle eBusiness Suite – PeopleSoft – Siebel and UCM * OWB can target many data stores. However, its metadata repository and therefore DWH target is Oracle Database. ETL: Mappings • • • • • • Declarative modeling of Data Flows Map from Source to Target Integrated Data Quality – N&A standardization – Match/Merge – Profiling Generates SQL & PL/SQL – Merge, transportable tablespaces, data pump, sqlloader, xml data types, BLOBS/CLOBS, … Leverage custom data transformations Pluggable maps for reusability of logic ETL: Process flows • Declarative modeling of Process/work Flows • Co-ordinate execution of Maps and other activities • Create complex transitions • Send email, FTP source/target files, call any external process, SQL Plus, Notifications • Generates Oracle Workflow, Oracle Scheduler & XPDL Data Quality in OWB • Data Quality functionalities are integrated into ETL processes – Disciplined approach to Data Quality, not an afterthought – Data Quality is modeled, executed and audited just like any other transformation • Consists of – Data Profiling – Name and Address Cleansing – Match-Merge Data Profiling Data Cleansing • • • • Parsing Standardization Correction Augmentation Input Data Name: Josephine Random Title: Senior Manager Company Name: Oracle Address: 500 oracle parkay Line1: redwood, az 94065 Line2: Phone: E-mail: USA 506 7000 [email protected] Name: Title: Company Name: Address: City: State: Postal Code: Country: Phone: E-mail: Gender: Corrected Data Ms. Josephine Random Sr. Mgr. Oracle Corp. 500 Oracle Pkwy Redwood City CA 94065-1675 USA 650 506 7000 [email protected] F Match-Merge • Matching is the process of determining, through business rules, which records refer to the same logical data • Merging is the business rules driven consolidation of the data from the matched set into a single record • Supports multiple algorithms • Uses of Match-Merge are – De-duplication – House-holding – Record Linking Integrated Data Object Design • One editor for: – Creation of objects – Configuration for multiple databases – Validation – DDL Code generation – Deployment to multiple databases – Data viewing Integrated Data Object Design • One editor for: – Relational modeling • Tables, views, materialized views • All storage and database properties – Dimensional modeling • Star and snow flake schemas • Multi-dimensional (OLAP) structures for Oracle OLAP • Support for Skip-Level, calculated measures, value based hierarchies • Slowly changing dimensions (including hierarchy versioning) – Support for business definitions • Oracle BI SE Data Object Editor Metadata Services • Dependency Management – Data Lineage at attribute level – Impact Analysis at attribute level • Metadata Snapshots • Change Management (diff, merge and reconcile) • Security (roles) • Multi-language Support (MLS via XLIFF) • Extensibility - user defined properties, objects, and associations • Reporting (browser) • APIs (Scripting, SQL, PL/SQL) • Exchange (import/export) Impact Analysis Runtime Management • Both client and web based administration GUI • Smart deployment • Start, stop, expedite jobs from a web client • Extensive auditing for: – Deployments – Executions – Error scenarios • Full scripting support • Full public view support User-Defined Objects & Icons • User-Definable: – Objects – Associations – Properties ALL in the OWB repository! • Custom icons for easy recognition • Access full metadata services of OWB e.g. impact analysis Experts • Re-use OWB Components to build your OWN application! – Define best practices & directed guidance • Run Standalone or within OWB Design-time • Declarative Definition & Guided execution <Insert Picture Here> New Features in OWB 11.1 Some New Features for OWB 11.1 • New Connectivity Options – 11g Database support – Siebel Connector – Gateway 11g support • Performance Features – Embedded explain plan support – Embedded statistics gathering – Embedded SQL/View tuning advisor • Simplified Installation – Embedded in the 11g database install – Includes Oracle Workflow in OWB install Some New Features for OWB 11.1 (cont.) • Enhanced SQL Generation – Advanced aggregation support – DML error logging in mappings • More Dimensional Support – Hierarchy versioning (supports multiple hierarchies) – Advanced binding options • Better support for federation – Graphical creation of views/MVs – Full impact analysis and lineage for federated structures <Insert Picture Here> Preview of OWB 11.2 The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. OWB 11gR2 “Plays well with others” BI Tool Integration OBI EE (Enterprise Edition) : design & deploy UDML Discoverer (Standard Edition): design & deploy EUL Databases Databases Non-Oracle BI : design & leverage Oracle objects Oracle Database 11g Applications Applications Connectivity Connectivity Oracle Warehouse Builder 11.2 Odbc / Gateways Files Connectors CDC: Queues / Streams/ Third Party Messages Odbc / Gateways Heterogeneous In-line Data Integration Data Quality Smart Data Modeling Files CDC: Queues / Streams/ Third Party for Oracle Messages SQL*Loader/ Ex Table JDBC JDBC Analytics Connectors Native Loaders Native Un-Loaders Knowledge Module Framework Unified Unified Metadata Metadata Native Loaders Analytics Native Un-Loaders Semi-Structured Semi-Structured Service Oriented Architectures Web Services Web Services for data integration and data quality Web Services Leveraging the Oracle Database New Database Features exposed in OWB 11.2 • • • • • • Advanced Compression Virtual columns New partitioning methods Data Pump Cube MVs for summaries And more… Easy upgrade to OWB 11.2 1. 2. 3. Import existing OWB design to OWB 11.2 Select new feature; e.g., Cube MVs Regenerate code Integrating with OBI EE How it’s done in OWB 11.2 1. In OWB, derive the model from the dimensional model 2. In OWB, generate the metadata for all 3 layers of OBI EE 3. In Oracle Answers, browse the data Flexible access to non-Oracle sources Enabled by knowledge module framework • Knowledge modules are snippets of code you can run, compile code in non-Oracle environments • Choose from a library of existing knowledge modules (KMs) • Write your own or edit existing KMs • Originally developed by Sunopsis, now Oracle Data Integrator (ODI) OWB + knowledge modules = 1st big step in merging OWB and ODI Knowledge Modules for flexible access Limited only by the code you write or find in the library • Access any new source type: relational, non-relational, message based and content sources • Rich support of ERP, CRM and MDM on any database platform • Rich library of Knowledge Modules Viewing and Editing the KM code Increased Flexibility 1. 2. 3. Modify template Regenerate the code Reuse the template Using KMs in mappings Easily change the connectivity methods for a single logical design 1. Complete the logical design (or import an existing mapping) Logical view Execution view 2. 3. Define “execution units” for each type of source ( 3 shown here) Select the best connectivity method for each execution unit (connectivity can be through ODBC, Gateways, Knowledge Modules, etc.) <Insert Picture Here> Pricing and Packaging In-Database ETL Included in many popular Oracle licenses • Database: SE, SE1, EE, Oracle Database Machine • Business Intelligence: SE, SE1 • e-Business Suite Intelligence products: Oracle HR Intelligence, Oracle Financial Data Manager Warehouse Builder Packaging Options rely on In-Database ETL Enterprise ETL/ ODI EE Data Profiling & Quality Profile tables, files, applications Detect or create data rules Audit incoming data Correct data In-Database ETL Design ETL mappings View HTML Metadata reports Perform data cleansing Use GUI or scripting language More… Data Watch & Repair Integrate with Oracle MDM: Siebel Universal Customer Master Product Information Management Data Hub Customer Data Hub Create new metadata types Promote from Dev>QA>PROD Navigate lineage & impact Propagate metadata changes Schedule jobs using UI Advanced process flow logic Create nested mappings Slowly changing dimensions Design in RAC environment More… Application Adapters Access to data and metadata in: SAP R/3 eBusiness Suite Peoplesoft Siebel Options licensed independently of each other, except for Data Watch & Repair which requires Data Profiling & Quality. Warehouse Builder Packaging Options rely on In-Database ETL Anticipated for OWB 11.2 (subject to change) Enterprise ETL/ ODI EE Data Profiling & Quality Profile tables, files, applications Detect or create data rules Audit incoming data Correct data In-Database ETL Design ETL mappings View HTML Metadata reports Perform data cleansing Use GUI or scripting language New & improved GUI Integration with OBI EE More… Data Watch & Repair Integrate with Oracle MDM: Siebel Universal Customer Master Product Information Management Data Hub Customer Data Hub Create new metadata types Promote from Dev>QA>PROD Navigate lineage & impact Propagate metadata changes Schedule jobs using UI Advanced process flow logic Create nested mappings Slowly changing dimensions Design in RAC environment Enhanced SOA support Flexible access to non-Oracle sources Orphan Management More… Application Adapters Access to data and metadata in: SAP R/3 eBusiness Suite Peoplesoft Siebel Options licensed independently of each other, except for Data Watch & Repair which requires Data Profiling & Quality. Common Pricing Includes ODI product and OWB Enterprise ETL option DATA INTEGRATOR ENTERPRISE EDITION DATA INTEGRATOR WAREHOUSE BUILDER ENTERPRISE ETL ODI Flagship Data Integration Product Heterogeneous, HighPerformance Data Integration Modern Java/SOA Architecture Flexible Knowledge Module framework OWB EETL • One license at $23k/cpu – – Flagship ETL Product Extensible Metadata Management Wizards for Slowly Changing Dimensions Interactive Lineage and Impact Analysis “ODI” and “OWB EETL” option are removed from pricelist Both are only available in ODI EE • Two separate installations • Implement one or both OWB Product Offerings This pricing applies to OWB 10.2 and 11.1 releases ORACLE DATABASE DATA INTEGRATOR WAREHOUSE BUILDER DATA PROFILING & QUALITY ENTERPRISE EDITION ENTERPRISE ETL OWB EETL OWB in DB Mapping Editor & Debugger OWB in EETL ODI EE APPLICATION ADAPTERS FOR WAREHOUSE BUILDER $2,300/cpu DBEE+$17.5k/cpu $23k/cpu ODPQ ADAPT Data quality algorithms run in Oracle Database Extensible Extensible Metadata Metadata Framework Framework Reveals business views of application data Basic dependency management with Process Flows Out of the box support for Slowly Changing Dimensions Profile file, table, application data in place or stage the data. Natively access SAP R/3 hosted on any database Robust scripting language & APIs Interactive Lineage and Impact Analysis Auto- detect data rules. Audit, auto-correct incoming data. Read from and write to eBusiness Suite, PeopleSoft, or Siebel Fuzzy match/ merge and Name&Address cleansing DATA WATCH & REPAIR ODPQ+$23k DWR Data quality for Oracle MDM: Siebel UCM, PIM, CDH <Insert Picture Here> For More Information ... For More Information… search.oracle.com Warehouse Builder or oracle.com/technology/products/warehouse/index.html • see “Training”“Oracle By Example” • go to “Product Discussion Forum” to contact us