Download Accelerating Business Intelligence with Oracle TimesTen In

Accelerating Business Intelligence with Oracle
TimesTen In-Memory Database
Jonathan Bar-Gil, ISV/OEM Principal Sales Consultant
1
The Quest for Response Time
Why is Response Time Important?
• Failure to meet service level agreements (SLA)
– Due to long response time
• Customer churns
– Competitive products offer better response time
• Decreased user productivity and decreased customer
satisfaction
– Unhappy customers
• Loss of revenue and decreased profitability
– Effect of lost business and opportunities
2
Improving Application Response Time
Understanding Latency Requirements
L
Slower
A
T
E
N
C
Y
Faster
• Define business requirements for latency
• Evaluate solutions
3
TimesTen In-Memory Database
Over a Decade of Innovation 2011
2010
2009
2007
2005
LOBs
Cache Advisor
In-Memory Analytics
Columnar Compression
Parallel Replication
ODP.NET Support
Cache Grid for Scale Out
Oracle Clusterware Integration
PL/SQL Support
Oracle Call Interface Support
OEM & SQLDeveloper Integration
Oracle Database Data Types Support
2000
National Language Support
Integration with Oracle RAC
1998
Mid-tier Cache for Oracle DB
Online Upgrades
High Availability
1st Commercial In-Memory RDBMS
4
Enabling The Real-Time World
Authorization,
Online Charging,
Location-Based
Services
Market Data,
Market Events,
Order Matching,
Trading
Real-Time Analytics Interactive Dashboard
Data Mart, Scorecard
eCommerce,
Personalization,
Real-Time Ad
Serving
Real-Time Applications
Instantly Responsive / Highly Scalable / Always-On
Mainstream 64-bit
Processors
Fast
Networks
Key Enabling Technology
5
Large Capacity
RAM
Agenda
TimesTen In-Memory Database
• Quick Overview
– Oracle TimesTen In-Memory Database and
– Oracle In-Memory Database Cache
• Proven technology for OLTP workloads
• Expansion into Business Intelligence
• Summary
7
Agenda
TimesTen In-Memory Database
• Quick Overview
– Oracle TimesTen In-Memory Database and
– Oracle In-Memory Database Cache
• Proven technology for OLTP workloads
• Expansion into Business Intelligence
• Summary
8
Oracle TimesTen In-Memory Database
Memory-Optimized Standalone Database
Telco Services
Financial Services
CRM, Portal, SaaS,Intelligence,
Customer-facing Defense, RealApplications
Time
BAM & BI
• Persistent, memory-optimized relational
database
• Delivers extreme performance for real-time
applications
• Highly available
• Deployed in the middle tier, close to the
application
Application
Application
Application
• Operates as a standalone database
• Or …
9
Oracle In-Memory Database Cache
Low-Cost Extreme Performance in the Middle-tier
• Accelerates Oracle Database applications
• Utilizes middle-tier resources
Business
Applicatio
ns
Cached
tables
Business
Applicatio
ns
Cached
tables
• Data cached in Oracle TimesTen InMemory Database
– Database tables
In-Memory Database Cache
– Subsets of rows and columns
• Fast, consistent response times
– High transaction throughput
Real Application Cluster
• Standard SQL interface
– Synchronized with Oracle Database
Automatic Storage
Management
10
Oracle TimesTen In-Memory Database
Optimized In-Memory Database Technology
• In-memory RDBMS
– Entire database in memory
– Persistent database
– Optimized memory layout and algorithms
• Exceptional performance
– Instantaneous response time
– High throughput
– Embeddable
• Persistent and durable
– Transactions with ACID properties
– Flexible options for data persistence to disk
Client-Server
Application
TimesTen Client
Lib
Client/
Server
Directly-Linked
Application
TimesTen
Libraries
JDBC / ODBC / OCI / Pro*c
Fast
data
access
Checkpoint
Files
• Standard SQL and PL/SQL with
– JDBC, ODBC, OCI, Pro*C
– Data types compatible with Oracle Database
11
Log
Files
Memory-Resident
Database
Lightning Fast Response Time
Average Response Time
TimesTen In-Memory Database
Microseconds
8
7
Millionths of
a second
4
1.78
0
Millionths of
a second
Read Transaction
Update Transaction
One Exalogic
Compute Node
Oracle TimesTen In-Memory Database 11.2.2.0 - Intel Xeon 5670 2.93Ghz, 2 CPUs, 6 cores/CPU - Oracle Linux 5.6
12
Throughput Scalability for OLTP Workloads
100% Read Workload on Exalogic
Throughput on One Compute Node
• Memory-optimized data
layout
• No locks acquired on
reads
Transactions Per Second
• Memory-optimized
access methods
4,500,000
4,144,896
4,000,000
3,553,895
3,500,000
Over 4.14M
2,890,719
Reads per Sec
3,000,000
2,500,000
2,155,411
2,000,000
1,550,766
1,500,000
1,000,000
792,858
560,212
500,000
0
1
2
4
6
8
Number of Concurrent Processes
13
10
12
Throughput Scalability for OLTP Workloads
Read-Update-Insert-Delete Workload on Exalogic
Throughput on One Compute Node
• Memory-optimized data
layout
• No locks acquired on
reads
• No waits of reads on
updates (versioned
concurrency control)
• Multi-threaded logging
14
2,000,000
Transactions per Second
• Memory-optimized
access methods
2,132,053
2,200,000
1,710,622
1,800,000
Over1,550,077
2.13M
TPS
1,600,000
1,400,000
1,200,000
1,015,445
856,235
1,000,000
800,000
458,555
600,000
400,000
287,461
200,000
0
1
2
4
6
8
Number of Concurrent Processes
10
12
What If .. you have an existing Oracle Database?
How to enable your application to use in-memory
technology?
Use TimesTen as an In-Memory Database Cache
 Oracle In-Memory Database Cache
 A product option for Oracle Database Enterprise Edition
15
TimesTen In-Memory Database Cache
Accelerates Oracle Database Applications
Application
Transactions
Application
Application
Application
IMDB Cache Grid
Automatic
Synchronization
16
Application
Transactions
•
•
•
•
•
•
•
•
•
Runs in the middle-tier
Caches subset of Oracle database
Full featured in-memory RDBMS
Accelerates applications
Scale up, Scale out
Read/write caching, local or global
Cache pre-loaded/loaded-on-demand
Automatic synchronization
Built-in high availability
Scale Out with TimesTen In-Memory Cache Grid
Grid Member
• On-demand addition/
removal of grid
members
Standby
Active
• Built-in HA
• Integration with
Oracle Clusterware
for automated failover
and recovery
17
Standby
Active
Standby
Active
Standby
Active
Standby
Application
Active
What does TimesTen offer for high availability?
TimesTen Replication provides real-time replication of
transaction data between TimesTen databases
18
Real-Time Replication
High Availability, Low-Latency Durability, On-Line Upgrades
• Real-time transactional replication
• High performance
–
–
–
–
–
Synchronous / asynchronous
Memory-to-memory replication of log records
Parallel send of log streams
Parallel apply of changes on Standby
Data compression
Application
Transactions
Application
Reads on Hot
Standby
Active
Standby
• Multiple topologies, including
– Active Standby Pair + Read-only Subscribers
with parallel apply of changes on Subscribers
– Active Active – Split Workload
• LAN and WAN configurations
• Online upgrades – no downtime
19
Subscriber
Subscriber
Subscriber
Read-only subscribers
• Reader Farm
• Disaster Recovery
Read-write and Read-only Caching
High Availability
Application
Transactions
Active
Hot Standby
available for reads
Standby
• Read-write caching
– Transactions applied to Active
– Parallel replication of committed
transactions to Standby
– Parallel writethrough of committed
transactions to Oracle database
• Read-only caching
Automatic
Synchronization
– Multi-stream refresh of committed
transactions from Oracle database
– Parallel replication of refresh transactions
to Standby
• Application continues operating even if
connection to Oracle database is down
20
Agenda
TimesTen In-Memory Database
• Quick Overview
– Oracle TimesTen In-Memory Database and
– Oracle In-Memory Database Cache
• Proven technology for OLTP workloads
• Expansion into Business Intelligence
• Summary
21
Oracle TimesTen with Exalogic and Exadata
Extreme Performance on Oracle Exa Stack
In-Memory Database Cache Grid
• TimesTen is deployed
in the middle-tier
• TimesTen can be a
cache to Oracle
Database, including
Exadata
• TimesTen is certified
on Exalogic
Application
Tier
22
Database
Tier
TimesTen In-Memory Database Cache Grid
Scalability on Exalogic – Mobile Prepaid Subscriber Application
• Java/JDBC application
• 5 different transactions
–
–
–
–
–
Authenticate user/account
Check account balance
Update account profile
Credit or debit account
Search for accounts that meet certain criteria
• Transactions executed in TimesTen and
propagated asynchronously to Oracle DB
• TimesTen In-Memory Cache Grid on Exalogic
– 8 compute nodes (One Quarter Rack)
– 4 grid members/compute node, 32 grid members total
– 40 TB shared storage
23
Exalogic Cloud
TimesTen on Exalogic – Extreme Performance
Scalability and Peak Throughput on Exalogic
8.7 Million
TPS
24
Agenda
TimesTen In-Memory Database
• Quick Overview
– Oracle TimesTen In-Memory Database and
– Oracle In-Memory Database Cache
• Proven technology for OLTP workloads
• Expansion into Business Intelligence
• Summary
25
How Is TimesTen Used in Exalytics?
• TimesTen is used as an in-memory cache for BI aggregates
• Summary Advisor determines what data to load using ETL tool
within OBI
Monitor and
Analyze User
Queries
Scripts persisted for
cache update
26
Update TimesTen
In-Memory Cache
Runs constantly with near
zero impact
Determine
Optimal
Datamart
Optimizes for maximum
impact. Administrator
override available
In-Memory Analytics
• Adaptive In-Memory Cache
1 TB RAM
– Decides what gets stored in-memory
– Adapts to change in analytic workload
• TimesTen In-Memory Database
– Enhanced with columnar compression
– Enhanced with analytic functions
27
Adaptive
In-Memory
Cache
TimesTen
In-Memory
Database
In-Memory Columnar Compression
• Columnar compression applied to
one or more columns of a table
• Optimized for in-memory storage
• Suitable of OLTP and BI workloads
• Compressed data remains available
for online access
• Space reduction of over 5x on
sample customer data
28
Space Utilization
6
5
4
3
2
1
0
Without
With
Compression Compression
In-Memory Columnar Compression – How?
• One or more columns in a table
can be compressed called a
compressed column group
• Dictionary table is created for
each compressed column group
• The compressed column group
contains a pointer to row in the
dictionary
• Width of pointer can be 1, 2, or 4
29
In-Memory Columnar Compression -- Example
30
OLAP Reporting Features
• Enhanced SQL aggregation
– GROUPING SETS
– CUBE
– ROLLUP
• Analytic functions
– Ranking functions
– Statistical functions
– With/without partitions and with/without sliding windows
• Query optimization and query processing optimized for in-memory data
management
31
GROUPING SETS – Example
The example specifies the grouping sets of (region_name, country_name), state_province, and
grand totals
32
Analytic functions – RANK Example
Use the RANK function to rank the first 10 employees in department 80 based on their salary and
commission. Identical salary values receive the same rank and cause nonconsecutive ranks.
35
Agenda
TimesTen In-Memory Database
• Quick Overview
– Oracle TimesTen In-Memory Database and
– Oracle In-Memory Database Cache
• Proven technology for OLTP workloads
• Expansion into Business Intelligence
• Summary
36
Summary
• TimesTen enables the realtime enterprise
• Optimized for OLTP and
Business Intelligence
workloads
• Complements Exadata with
microsecond response time
for real-time applications
• Certified on Exalogic
• Optimized for Exalytics
37
Questions
38