Download Slide 1 - Oracle

<Insert Picture Here>
Oracle Warehouse Builder: Overview
Jean-Pierre Dijcks and Michelle Bird
Warehouse Builder Product Management
Agenda
• Integrated Solution
• Market Position
• Roadmap
– Merging the ODI and OWB products
– Common pricing bundle
• Product Overview
– New Features in OWB 11.1
– Preview of OWB 11.2
• Pricing and Packaging
• For More Information ...
<Insert Picture Here>
<Insert Picture Here>
Integrated Solution
Distributed Data Marts and Servers
Expensive data warehouse architecture
Data
Marts
Online
Analytics
ETL
Data Mining
Typical ETL
Multiple tools require separate training, install, maintenance
Generic
Repositories
Metadata
Exchange
Classic Data
Integration Tools
Classic Data
Quality
Consolidated Data Warehouse
Single source of truth on low cost servers & storage
Data
Marts
Online
Analytics
OWB
ETL
Data Mining
Oracle Database 11g
with integrated ETL, Analytics
& Data Mining
OWB Integrated in the Oracle Database
Centralized metadata repository
• hosted on Oracle Database  secure, scalable
• import metadata
• manage & propagate metadata changes
Profile and cleanse data
• profile applications, tables, files, …
• derive or write custom rules
Transform cleansed data
• rich library of transformations and operators
• leverage Oracle partitioning
Monitor incoming data
• automatic notifications if data quality falls below threshold
Oracle Warehouse Builder
OWB is optimized for Oracle environments
• Benefit from your Oracle investments
– Better performance through Oracle optimizations
– Skills, people and technology
• Improved productivity
– Use the latest Oracle features without rewriting your code
– Advanced task automation features
• Shorten your project times while improving data
quality
– Discover problems up-front with data profiling
– Improve data quality in a declarative way
In-Database ETL
Higher Productivity
In-house ETL
Oracle InDatabase ETL
Tune and Administer
Test and Update
Create
Data Quality
Routines
Tune and Administer
Test and Update
Create Data
Transformations
Years to Months
Create DQ Routines
Create Data
Transformations
Months to weeks
Advanced features made easy
Any SQL or
PL/SQL expression
Parallel Loading with
DML Error Handling
Heterogeneous
Source Access
Fuzzy Matching
and Merging
Impact Analysis and Lineage
Partition Exchange
Loading
<Insert Picture Here>
Mark Rittman
Co-founder, Rittman Mead
Consulting
“"Rittman Mead recommends Oracle
Warehouse Builder because it is the most
effective way to leverage advanced
capabilities of the Oracle Database for ETL
development, while also providing metadata
management and data quality features
required for full life-cycle deployments.”
<Insert Picture Here>
Market Position
Market Position
Recognized as a leader since OWB 10.2
• 10,000+ companies rely on OWB
• Optimized for Oracle environments
2005
<Insert Picture Here>
Roadmap
Product Highlights
An abbreviated history of OWB
2000
2003
2004
2006
2007
Release 2.0.4
Release 9.2
Release 10.1
Release 10.2
Release 11.1
Basic ETL
functionality
Oracle RAC certified
Oracle 10g certified
Data profiling &
auditing
Simplified
installation: installed
w/ Oracle 11g DB
Mapping Editor
Set and row based
code generation
Mapping Debugger
Interactive LIA
Name & Address
Match/Merge
3,000+ companies
Release 9.0.4
Compete against
Informatica and
Ascential
Metadata Security
Process Flow Editor
Scripting Language
Multi-table insert
Pivot and Unpivot
Recognized as a
leader in Gartner
Magic Quadrant
Scheduling
Improved connector
for SAP R/3
Data Watch and
Repair (for Oracle
MDM)
Siebel Adapter
EBS and PeopleSoft
connectors
OLAP and relational
modeling
Pluggable mappings
Non-Oracle targets
Recognized as a
leader in Forrester
Enterprise ETLWave
10,000+ companies
High-Level Roadmap
Unified platform that is a superset of OWB and ODI
10gR1
10gR2
11gR1
• OWB/ODI investments are
11gR2
fully protected
• No forced migrations
OWB
• Natural upgrade path
Unified dev team
“ CY 2012”
Common pricing
ODI
11gR2
10gR3
11gR1
• No regressions in
functionality
• Training and support to
continue for both product
releases
Common Pricing
Includes ODI product and OWB Enterprise ETL option
DATA INTEGRATOR
ENTERPRISE EDITION
DATA INTEGRATOR
WAREHOUSE BUILDER
ENTERPRISE ETL
ODI
Flagship
Data
Integration
Product
Heterogeneous,
HighPerformance
Data Integration
Modern
Java/SOA
Architecture
Flexible
Knowledge
Module
framework
OWB
EETL
Flagship
ETL Product
Extensible
Metadata
Management
Wizards for Slowly
Changing
Dimensions
Interactive
Lineage and
Impact Analysis
Oracle’s Roadmap for its ETL tools
• Long Term Direction:
Deliver an integrated data integration product
• Short Term:
Deliver stand-alone releases with first integration
steps
– Shared knowledge module framework (OWB 11.2)
– Runtime interoperability (future releases)
– Same user interface framework (future releases)
• Protect investments for the entire customer base
http://www.oracle.com/technology/products/warehouse/pdf/owb_roadmap.pdf
<Insert Picture Here>
Product Details
Oracle Warehouse Builder
The ETL and DQ tool shipped with Oracle Database
• Oracle Database as transformation engine
• Oracle Database as metadata repository
Oracle Database 11g
Oracle Warehouse Builder 11.1
Heterogeneous
In-line
Data
Integration
Data
Quality
Unified
Unified Metadata
Metadata
Smart
Data
Modeling
for Oracle
OWB 11.1
Oracle-centric data integration platform
BI Tool Integration
OBI EE (Enterprise Edition) : design & leverage Oracle objects
Discoverer (Standard Edition): design & deploy EUL
Databases
Databases
Non-Oracle BI : design & leverage Oracle objects
Oracle Database 11g
Applications
Applications
Connectivity
Connectivity
Oracle Warehouse Builder 11.1
Odbc / Gateways
Files
Connectors
CDC: Queues /
Streams/ Third Party
Messages
Odbc / Gateways
Heterogeneous
In-line
Data
Integration
Data
Quality
SQL*Loader/ Ex Table
Analytics
Smart
Data
Modeling
for Oracle
Connectors
Files
CDC: Queues /
Streams/ Third Party
Messages
Analytics
Unified
Unified Metadata
Metadata
Semi-Structured
Semi-Structured
Web Services
Web Services
Architecture
Design
Runtime
Deploy
Sources & Targets
Data Sources
• Oracle
– Tables, Views, MViews, Queues,
External Tables, Sqlloader,
Transportable Tablespaces, Data
Pump…
• DB2, Sybase, SQLServer, Informix,
Mainframes, … (Oracle Transparent
Gateways)
• ODBC
• Flat Files
• XML
• Applications
– Oracle eBusiness Suite
– PeopleSoft
– SAP R/3
– Siebel
– Oracle MDM (UCM, CDH, & PIM)
Data Targets*
• Oracle
• DB2, Sybase, SQLServer, Informix,
Mainframes, … (Oracle Transparent
Gateways)
• ODBC
• Flat Files
• XML
• Applications
– Oracle eBusiness Suite
– PeopleSoft
– Siebel and UCM
* OWB can target many data stores. However, its
metadata repository and therefore DWH target is Oracle
Database.
ETL: Mappings
•
•
•
•
•
•
Declarative modeling of Data
Flows
Map from Source to Target
Integrated Data Quality
– N&A standardization
– Match/Merge
– Profiling
Generates SQL & PL/SQL
– Merge, transportable
tablespaces, data pump,
sqlloader, xml data types,
BLOBS/CLOBS, …
Leverage custom data
transformations
Pluggable maps for reusability
of logic
ETL: Process flows
• Declarative modeling of
Process/work Flows
• Co-ordinate execution of
Maps and other activities
• Create complex transitions
• Send email, FTP
source/target files, call any
external process, SQL Plus,
Notifications
• Generates Oracle Workflow,
Oracle Scheduler & XPDL
Data Quality in OWB
• Data Quality functionalities are integrated into ETL
processes
– Disciplined approach to Data Quality, not an afterthought
– Data Quality is modeled, executed and audited just like any
other transformation
• Consists of
– Data Profiling
– Name and Address Cleansing
– Match-Merge
Data Profiling
Data Cleansing
•
•
•
•
Parsing
Standardization
Correction
Augmentation
Input Data
Name:
Josephine Random
Title:
Senior Manager
Company Name: Oracle
Address:
500 oracle parkay
Line1:
redwood, az 94065
Line2:
Phone:
E-mail:
USA
506 7000
[email protected]
Name:
Title:
Company Name:
Address:
City:
State:
Postal Code:
Country:
Phone:
E-mail:
Gender:
Corrected Data
Ms. Josephine Random
Sr. Mgr.
Oracle Corp.
500 Oracle Pkwy
Redwood City
CA
94065-1675
USA
650 506 7000
[email protected]
F
Match-Merge
• Matching is the process of determining, through
business rules, which records refer to the same logical
data
• Merging is the business rules driven consolidation of
the data from the matched set into a single record
• Supports multiple algorithms
• Uses of Match-Merge are
– De-duplication
– House-holding
– Record Linking
Integrated Data Object Design
• One editor for:
– Creation of objects
– Configuration for multiple databases
– Validation
– DDL Code generation
– Deployment to multiple databases
– Data viewing
Integrated Data Object Design
• One editor for:
– Relational modeling
• Tables, views, materialized views
• All storage and database properties
– Dimensional modeling
• Star and snow flake schemas
• Multi-dimensional (OLAP) structures for Oracle OLAP
• Support for Skip-Level, calculated measures, value based
hierarchies
• Slowly changing dimensions (including hierarchy versioning)
– Support for business definitions
• Oracle BI SE
Data Object Editor
Metadata Services
• Dependency Management
– Data Lineage at attribute level
– Impact Analysis at attribute level
• Metadata Snapshots
• Change Management (diff, merge and reconcile)
• Security (roles)
• Multi-language Support (MLS via XLIFF)
• Extensibility - user defined properties, objects, and associations
• Reporting (browser)
• APIs (Scripting, SQL, PL/SQL)
• Exchange (import/export)
Impact Analysis
Runtime Management
• Both client and web based
administration GUI
• Smart deployment
• Start, stop, expedite jobs
from a web client
• Extensive auditing for:
– Deployments
– Executions
– Error scenarios
• Full scripting support
• Full public view support
User-Defined Objects & Icons
• User-Definable:
– Objects
– Associations
– Properties
ALL in the OWB repository!
• Custom icons for easy
recognition
• Access full metadata services
of OWB e.g. impact analysis
Experts
• Re-use OWB Components
to build your OWN
application!
– Define best practices &
directed guidance
• Run Standalone or within
OWB Design-time
• Declarative Definition &
Guided execution
<Insert Picture Here>
New Features in OWB 11.1
Some New Features for OWB 11.1
• New Connectivity Options
– 11g Database support
– Siebel Connector
– Gateway 11g support
• Performance Features
– Embedded explain plan support
– Embedded statistics gathering
– Embedded SQL/View tuning advisor
• Simplified Installation
– Embedded in the 11g database install
– Includes Oracle Workflow in OWB install
Some New Features for OWB 11.1 (cont.)
• Enhanced SQL Generation
– Advanced aggregation support
– DML error logging in mappings
• More Dimensional Support
– Hierarchy versioning (supports multiple hierarchies)
– Advanced binding options
• Better support for federation
– Graphical creation of views/MVs
– Full impact analysis and lineage for federated structures
<Insert Picture Here>
Preview of OWB 11.2
The following is intended to outline our general
product direction. It is intended for information
purposes only, and may not be incorporated into any
contract. It is not a commitment to deliver any
material, code, or functionality, and should not be
relied upon in making purchasing decisions.
The development, release, and timing of any
features or functionality described for Oracle’s
products remains at the sole discretion of Oracle.
OWB 11gR2
“Plays well with others”
BI Tool Integration
OBI EE (Enterprise Edition) : design & deploy UDML
Discoverer (Standard Edition): design & deploy EUL
Databases
Databases
Non-Oracle BI : design & leverage Oracle objects
Oracle Database 11g
Applications
Applications
Connectivity
Connectivity
Oracle Warehouse Builder 11.2
Odbc / Gateways
Files
Connectors
CDC: Queues /
Streams/ Third Party
Messages
Odbc / Gateways
Heterogeneous
In-line
Data
Integration
Data
Quality
Smart
Data
Modeling
Files
CDC: Queues /
Streams/ Third Party
for Oracle
Messages
SQL*Loader/ Ex Table
JDBC
JDBC
Analytics
Connectors
Native Loaders
Native Un-Loaders
Knowledge Module Framework
Unified
Unified Metadata
Metadata
Native Loaders
Analytics
Native Un-Loaders
Semi-Structured
Semi-Structured
Service Oriented Architectures
Web Services
Web Services for data integration and data quality
Web Services
Leveraging the Oracle Database
New Database Features exposed in OWB 11.2
•
•
•
•
•
•
Advanced Compression
Virtual columns
New partitioning methods
Data Pump
Cube MVs for summaries
And more…
Easy upgrade to OWB 11.2
1.
2.
3.
Import existing OWB design to OWB 11.2
Select new feature; e.g., Cube MVs
Regenerate code
Integrating with OBI EE
How it’s done in OWB 11.2
1.
In OWB, derive the
model from the
dimensional model
2.
In OWB, generate the
metadata for all 3
layers of OBI EE
3.
In Oracle Answers,
browse the data
Flexible access to non-Oracle sources
Enabled by knowledge module framework
• Knowledge modules are snippets of code you can run, compile code
in non-Oracle environments
• Choose from a library of existing knowledge modules (KMs)
• Write your own or edit existing KMs
• Originally developed by Sunopsis, now Oracle Data Integrator (ODI)
OWB + knowledge modules
= 1st big step in merging OWB and ODI
Knowledge Modules for flexible access
Limited only by the code you write or find in the library
•
Access any new source type:
relational, non-relational, message
based and content sources
•
Rich support of ERP, CRM and
MDM on any database platform
•
Rich library of Knowledge Modules
Viewing and Editing the KM code
Increased Flexibility
1.
2.
3.
Modify template
Regenerate the code
Reuse the template
Using KMs in mappings
Easily change the connectivity methods for a single logical design
1.
Complete the logical
design (or import an
existing mapping)
Logical view
Execution view
2.
3.
Define “execution units”
for each type of source (
3 shown here)
Select the best
connectivity method for
each execution unit
(connectivity can be
through ODBC,
Gateways, Knowledge
Modules, etc.)
<Insert Picture Here>
Pricing and Packaging
In-Database ETL
Included in many popular Oracle licenses
• Database: SE, SE1, EE, Oracle Database Machine
• Business Intelligence: SE, SE1
• e-Business Suite Intelligence products: Oracle HR
Intelligence, Oracle Financial Data Manager
Warehouse Builder Packaging
Options rely on In-Database ETL
Enterprise ETL/ ODI EE
Data Profiling & Quality
Profile tables, files, applications
Detect or create data rules
Audit incoming data
Correct data
In-Database ETL
Design ETL mappings
View HTML Metadata reports
Perform data cleansing
Use GUI or scripting language
More…
Data Watch & Repair
Integrate with Oracle MDM:
Siebel Universal Customer Master
Product Information Management Data Hub
Customer Data Hub
Create new metadata types
Promote from Dev>QA>PROD
Navigate lineage & impact
Propagate metadata changes
Schedule jobs using UI
Advanced process flow logic
Create nested mappings
Slowly changing dimensions
Design in RAC environment
More…
Application Adapters
Access to data and
metadata in:
SAP R/3
eBusiness Suite
Peoplesoft
Siebel
Options licensed independently of each other, except for Data Watch & Repair which requires
Data Profiling & Quality.
Warehouse Builder Packaging
Options rely on In-Database ETL
Anticipated for
OWB 11.2
(subject to change)
Enterprise ETL/ ODI EE
Data Profiling & Quality
Profile tables, files, applications
Detect or create data rules
Audit incoming data
Correct data
In-Database ETL
Design ETL mappings
View HTML Metadata reports
Perform data cleansing
Use GUI or scripting language
New & improved GUI
Integration with OBI EE
More…
Data Watch & Repair
Integrate with Oracle MDM:
Siebel Universal Customer Master
Product Information Management Data Hub
Customer Data Hub
Create new metadata types
Promote from Dev>QA>PROD
Navigate lineage & impact
Propagate metadata changes
Schedule jobs using UI
Advanced process flow logic
Create nested mappings
Slowly changing dimensions
Design in RAC environment
Enhanced SOA support
Flexible access to non-Oracle sources
Orphan Management
More…
Application Adapters
Access to data and
metadata in:
SAP R/3
eBusiness Suite
Peoplesoft
Siebel
Options licensed independently of each other, except for Data Watch & Repair which requires
Data Profiling & Quality.
Common Pricing
Includes ODI product and OWB Enterprise ETL option
DATA INTEGRATOR
ENTERPRISE EDITION
DATA INTEGRATOR
WAREHOUSE BUILDER
ENTERPRISE ETL
ODI
Flagship
Data
Integration
Product
Heterogeneous,
HighPerformance
Data Integration
Modern
Java/SOA
Architecture
Flexible
Knowledge
Module
framework
OWB
EETL
• One license at $23k/cpu
–
–
Flagship
ETL Product
Extensible
Metadata
Management
Wizards for Slowly
Changing
Dimensions
Interactive
Lineage and
Impact Analysis
“ODI” and “OWB EETL” option are
removed from pricelist
Both are only available in ODI EE
• Two separate installations
• Implement one or both
OWB Product Offerings
This pricing applies to OWB 10.2 and 11.1 releases
ORACLE DATABASE
DATA INTEGRATOR
WAREHOUSE
BUILDER
DATA PROFILING & QUALITY
ENTERPRISE
EDITION
ENTERPRISE
ETL
OWB EETL
OWB
in
DB
Mapping Editor
& Debugger
OWB in
EETL
ODI
EE
APPLICATION ADAPTERS
FOR WAREHOUSE BUILDER
$2,300/cpu
DBEE+$17.5k/cpu
$23k/cpu
ODPQ
ADAPT
Data quality
algorithms
run in Oracle
Database
Extensible
Extensible
Metadata
Metadata
Framework
Framework
Reveals
business views of
application data
Basic dependency
management
with Process
Flows
Out of the box
support for Slowly
Changing
Dimensions
Profile file,
table, application
data in place or
stage the data.
Natively access
SAP R/3 hosted
on any database
Robust scripting
language & APIs
Interactive Lineage
and Impact
Analysis
Auto- detect
data rules.
Audit, auto-correct
incoming data.
Read from and
write to eBusiness
Suite, PeopleSoft,
or Siebel
Fuzzy match/
merge and
Name&Address
cleansing
DATA WATCH & REPAIR
ODPQ+$23k
DWR
Data quality
for Oracle MDM:
Siebel UCM,
PIM, CDH
<Insert Picture Here>
For More Information ...
For More Information…
search.oracle.com
Warehouse Builder
or
oracle.com/technology/products/warehouse/index.html
• see “Training”“Oracle By Example”
• go to “Product Discussion Forum” to contact us