Download What’s all the fuss about “Big Data”? Doug Cackett Oracle Enterprise Architecture

What’s all the fuss about “Big Data”?
Doug Cackett
Oracle Enterprise Architecture
[email protected]
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Adapted from Tom Davenport
Delivering business benefits
Analytics 3.0
Analytics 2.0
• Platform for monetisation
Analytics 1.0
• “Competing on Analytics”
• Basic reporting
• Limited range of tabular data
• Batch oriented analysis
• Analysis bolted onto limited
set of business processes
Data as a business cost
• Extended analytics to larger and less
structured datasets
• Fact and exception based
management doctrine
• Recognition of Data Science
•
•
•
•
Deeper analysis on more data
Faster test-do-learn
wider business process coverage
Analysts focus on discovery and
driving business value
Data as a business benefit
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Momentum around Information Management
Data scope & operational linkage
Changing technology footprint
• Blurring of operational and
analytical boundaries
• Data volumes driving costs
and surfacing performance issues
• Externally sourced data mashups
driven by business users
• Polyglot persistence model &
Hadoop market momentum
• Rationalising existing investments
that often date back many years
• Expansion in range of possible
architecture patterns
Delivery and alignment strategy
Governance & organisation
• Recognition of Bi-Modal
delivery pattern for alignment
• Shift in power and budgets towards Business
• Shift from “truth” to “trust” and
the implications on data quality
• The critical need for rational
Analytical governance
• Efficient, everyday, curated
• Rapid discovery of business insights
• Agile development approach
that links the modes together
Gartner 2015 BI Summit
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Discovery is a key capability
Business
Value
• Architecture, approach
and governance aim to:
Commercial
Exploitation
–Accelerate time to discovery
with lower manual effort
–Reduce the size of the
Governance step
–Eliminate growth in technical
and architectural debt
Governance
Understanding
of the data
Discovery phase
Time / Effort
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Innovation through Data Discovery process
It’s about “Data Discovery” not IT development
What IT thinks it should be
Requirement
Analysis
High Level
Design
Low Level
Design
Coding
Testing
Acceptance
Testing
Local Access
Database
Shared Access
Server
SQL Server
Database
Oracle
Datawarehouse
What normally happens
Excel
Spreadsheet
Shared linked
spreadsheets
What we need to achieve in the Discovery process
Discovery & Profile
Model
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Exploit
Conceptual architecture for Information Management
“At the Internet age,
we can not confront
the challenges of the
future by doing the
things as was thought
20 years ago”
CaixaBank executive statement for
Inspiring the “data pool” and User
Experience Initiatives
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Initial customer deployment patterns
Discovery Lab
Re-conceptualise IM
1
Operationalise Insights
5
2
Budget
Use-cases
Sponsorship
Timeline
Skills
4
3
Big Data Application
Big Data technology pilot
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Oracle’s Information Management Reference Architecture
Full lifecycle that also balances ingestion and interpretation costs
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Balancing solution demands and the persistence model
Identifying the right technology and design pattern
Straight Through
Processing (STP)
Tooling maturity
5
4
Availability & Business
Continuity
3
Ingestion rate
2
1
ACID transactional
requirement
In-Memory
Set or row
loading into
database or
memory
Relational
Hadoop
0
Cost effectively store low
value data
Hadoop
Relational
Security
Ingestion (coding)
simplicity
Variety of data formats
Data sparsity
API or utility
loading into
Non-relational
stores
Access
NoSQL
Other stores…
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
9
Big Data SQL – an aggregate approach
ETL of aggregate data moved to RDBMS
10’s of Gigabytes of Data
WEB_LOGS
B B B
CUSTOMERS
Hadoop Cluster
Oracle Database
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
10
Big Data SQL – a federated approach
All the data copied to RDBMS for query processing
10’s of Gigabytes of Data
WEB_LOGS
B B B
CUSTOMERS
Hadoop Cluster
Oracle Database
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
11
Big Data SQL – a franchised approach
SELECT
w.sess_id, c.name
FROM web_logs w, customers c
WHERE
w.source_country = ‘Brazil’
AND
w.cust_id = c.customer_id;
Relevant SQL runs on BDA nodes
Big Data SQL
10’s of Gigabytes of Data
WEB_LOGS
B B B
Only columns and rows needed to
answer query are returned
Hadoop Cluster
CUSTOMERS
Oracle Database
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
12
Apache Zoo Families & Species (Solution Classes & Products)
Keeping pace with the rapidly evolving Apache Zoo
Non-Persistent
Data
Stores
Data Capture
Data
&
Integration
Delivery
Data
Processing
Persistent
Data
Stores
Data
Discovery
Search
Workflow
&
Orchestration
Advanced
Analytics
Data
Query
Other
Data
Movement
(Relational)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Finding out more about the Reference Architecture
• Download the architecture white paper here
• Attend an Information Masterclass event
–Full day workshop with other customers or Partners
–Delivered by architect for architect
–Whiteboard not PPT
–Product agnostic
Customer event details here (Oct 15th)
Partner event details here ( Nov 24th)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
14