Download 12.2_InMemory_new_features

Oracle Database In-Memory
New Features 12c Release 2
Gavin Soorma
Agenda
 What benefits can In-Memory provide
 In-Memory architecture and components
 How do we configure In-Memory in the database
 What’s new in Oracle 12c Release 2





Dynamically increase In-Memory area on the fly
In-Memory support for Active Data Guard
Optimize join performance via Join Groups
In-Memory Expressions
In-Memory Fast Start
The In-Memory Option
 In-Memory option included in Oracle Database Enterprise Edition
12.1.0.2
 Separate licensable option
 Transparently accelerate analytic queries by orders of magnitude
 Businesses benefit from better decisions made in real-time, resulting in
lower costs, improved productivity, and increased competitiveness
 No application change required
 Entire database need not be in memory as case with some competition
Best of both worlds
 Even mainly OLTP databases occasionally will have analytical type
workload
 Single database can now efficiently support both OLTP as well as
Data Warehouse type workloads
 Dual-format architecture
 Enables data to be maintained in the existing Oracle row format
ideal for OLTP operations (Buffer Cache)
 Also enables data to be maintained in a new purely in-memory
columnar format, optimized for analytical processing (In-Memory
Column Store)
Dual Row Format Architecture
OLTP
Few rows
Many columns
Data Warehouse
Millions of rows
Few columns
In-Memory boosts OLTP performance
OLTP is Slowed by Analytic Indexes
Column Store Replaces Analytic Indexes
Configure the In-Memory Area
 In-Memory Area is an optional SGA component that contains the IM
column store
 IM column store does not replace the buffer cache, but acts as a
supplement
 It is not a cache – objects do not age out!
 Data can now be stored in memory in both row and column format
 In-Memory Area is controlled by the INMEMORY_SIZE initialization
parameter
 By default, the size of the In-Memory Area is 0, which means the IM
column store is disabled
Configure the In-Memory Area
 In-Memory Area is subtracted from the SGA_TARGET initialization
parameter setting
 SGA_TARGET =10 GB and INMEMORY_SIZE=4 GB
 40% of the SGA_TARGET setting is allocated to the In-Memory Area
 The In-Memory area is sub-divided into two pools:
 1MB pool used to store the actual column data populated into memory
 64K pool used to store metadata about the objects that are populated into the IM
column store
In-Memory Architecture
In-Memory Area
Columnar Data
IMEU
IMCU
Background
Processes
IMCO
Metadata
SMU
Populate
Repopulate
w000
IMCU
w001
IMEU
Populate
SMU
w002
Repopulate
1 MB Pool
64 KB Pool
In-Memory Compression Units (IMCU)
 IMCU is a compressed, read-only storage unit that contains data for
one or more columns
 An IMCU is analogous to a tablespace extent
 An IMCU has two parts:
 A set of Column Compression Units (CUs)
 A header that contains metadata such as the IM storage index
 An IM storage index stores the minimum and maximum value for all
columns within the IMCU
Snapshot Metadata Units (SMUs)
 A Snapshot Metadata Unit (SMU) contains metadata and
transactional information for an associated IMCU
 Contained in the 64KB Pool in-memory area
 Each IMCU maps to a Snapshot Metadata Unit or SMU in the 64KB
pool that holds the metadata about the IMCU
 IMCU is a read only structure, SMU is modified as it contains the
Transaction Journal
Transaction Journal
 Every SMU contains a transaction journal which tracks row
modifications
 The database uses the transaction journal to keep the IMCU
transactionally consistent
 The database uses the buffer cache to process DML just as when the IM
column store is not enabled
 UPDATE statement might modify a row in an IMCU
 ROWID for the modified row is added to the transaction journal and
marked as stale in the IMCU as of the SCN of the DML statement
 If a query needs to accesses the new version of the row, then the
database obtains the row from the database buffer cache
Transaction Journal
 If a query accesses the columnar data, and discovers modified
rows, then it can obtain the corresponding rowids from the
transaction journal, and then retrieve the modified rows from the
buffer cache
In-Memory Expression Units (IMEUs)
 An In-Memory Expression Unit (IMEU) is a storage container for
materialized In-Memory Expressions and user-defined virtual
columns
 Every IMEU maps to exactly one IMCU, mapping to the same row
set
 The IMEU contains expression results for the data contained in its
associated IMCU
 When the IMCU is populated, the associated IMEU is also populated
Expression Statistics Store
 The Expression Statistics Store (ESS) is a repository maintained
by the optimizer to store statistics about expression evaluation
 The ESS resides in the SGA and also persists on disk in the
SYSAUX tablespace
 The database uses the ESS to determine whether an expression is
“hot” (frequently accessed), and thus a candidate for an IM
expression
 IM expressions are exposed as system-generated virtual columns,
prefixed by the string SYS_IME
Populating the In-Memory Column Store
 Population is a streaming mechanism, converting row data into
columnar format, and then compressing it
 New INMEMORY attribute for tables and materialized views
 Only objects with the INMEMORY attribute are populated into the IM
column store
 The INMEMORY attribute can be specified on a tablespace, table,
partition, or materialized view
 Repopulation occurs based on threshold and new in 12.2 is trickle
repopulation
Populating the In-Memory Column Store
ALTER TABLESPACE ts_data DEFAULT INMEMORY;
ALTER TABLE sales INMEMORY;
ALTER TABLE sales INMEMORY NO INMEMORY(prod_id);
ALTER TABLE sales MODIFY PARTITION SALES_Q1_1998 NO INMEMORY;
 Objects are populated into the IM column store either in a prioritized list
immediately after the database is opened or after they are scanned
(queried) for the first time
ALTER TABLE customers INMEMORY PRIORITY CRITICAL;
 Oracle Database In-Memory Advisor (Doc ID 1965343.1)
In-Memory Background Processes
 In-Memory Coordinator Process (IMCO) initiates population and
repopulation of columnar data
 Space Management Worker Processes (Wnnn) populate or
repopulate data on behalf of IMCO
 During population, Wnnn processes are responsible for creating
IMCUs, SMUs, and IMEUs
 Space Management Worker Processes also wake up at frequent
intervals to initiate repopulation for objects with ‘stale’ columnar data
Repopulating the In-Memory Column Store
 As the number of modifications increase, so do the size of
Transaction Journals in the SMUs
 Leads to increase in the amount of data that must be fetched from
database buffer cache in the absence of current columnar data
available in the In-Memory store
 The more stale entries there are in an IMCU, the slower the scan of
the IMCU will become
 To avoid degrading query performance through journal access,
background processes repopulate modified objects back into the
column store
Repopulating the In-Memory Column Store
 Threshold-based repopulation and trickle repopulation
 Database will repopulate an IMCU when the number of stale entries
in the transaction journal reaches a staleness threshold
 Trickle repopulation supplements threshold-based repopulation by
periodically refreshing stale columnar data even when the threshold
has not been reached
 INMEMORY_TRICKLE_REPOPULATE_SERVERS_PERCENT
limits the maximum number of background populate servers used
for In-Memory Column Store repopulation
In-Memory Architecture
In-Memory Area
Columnar Data
IMEU
IMCU
Background
Processes
IMCO
Metadata
SMU
Populate
Repopulate
w000
IMCU
w001
IMEU
Populate
SMU
w002
Repopulate
1 MB Pool
64 KB Pool
Dynamically increase size of In-Memory
Column Store
 When more memory is required for the IM column store, it can be
increased in size dynamically
 Before 12.2 required instance restart
 The size of the IM column store cannot be decreased dynamically
 The new size of the IM column store must be at least 128 megabytes
greater than the current INMEMORY_SIZE setting
 There must be enough memory available in the SGA to increase the
size of the IM column store dynamically to the new value
ALTER SYSTEM SET INMEMORY_SIZE = 500M SCOPE=BOTH;
In-Memory Expressions
 Analytic queries often contain complex expressions in the select list
or where clause predicates that need to be evaluated for every row
processed by the query
 The evaluation of these complex expressions can be very resource
intensive and time consuming
 IM Expressions greatly improve the performance of analytic queries
that use computationally intensive expressions and access large
data sets
 IM expressions speed queries of large data sets by precomputing
computationally intensive expressions
In-Memory Expressions
 The Expression Statistics Store (ESS) automatically tracks the
results of frequently evaluated (“hot”) expressions
 Invoke the IME_CAPTURE_EXPRESSIONS procedure which is part
of the DBMS_INMEMORY_ADMIN package
 Queries the ESS, and identifies the 20 most frequently accessed
(“hottest”) expressions in the specified time range
 Expressions are stored as added hidden virtual columns to the table
 Under the direction of In-Memory Coordinator Process (IMCO),
Space Management Worker Processes (Wnnn) load IM expressions
into IMEUs during population/repopulation
In-Memory Expressions
SELECT
employee_id, last_name, salary, commission_pct,
ROUND(salary*12/52,2) as "weekly_sal",
12*(salary*NVL(commission_pct,0)+salary) as "ann_comp"
FROM employees
ORDER BY ann_comp;
In-Memory Fast Start
 In-Memory population is a CPU bound operation, involving
reformatting data into a columnar format and compressing that data
before placing it in memory
 With IM FastStart enabled IMCUs from the In-Memory column store
are checkpointed to Fast Start area ondisk at periodic intervals
 On subsequent database restarts, column store data is populated
via the IMCUs stored in FastStart area on disk rather than from
querying the base tables
 This relieves the CPU overhead of population at the cost of
additional disk space requirements
In-Memory Fast Start
 Specify a tablespace for the FastStart area using the
DBMS_INMEMORY_ADMIN.FASTSTART_ENABLE procedure
 The Space Management Worker Processes (Wnnn) write IMCUs to
the SecureFiles LOB named SYSDBinstance_name_LOBSEG$
 Transactional consistency checks involve comparing the System
Change Number (SCN) at which the IM FastStart checkpoint was
taken for the IMCU with the most recent SCN
 Depending on result of this check IMCU will be populated from:
 FastStart Area
 FastStart Area with some rows marked invalid (due to data modification
after the IMCU was written to the FastStart area)
 Completely discarded and populated from disk
In-Memory and Active Data Guard
 IM column store can be enabled in an Oracle Active Data Guard
standby database
 IM column store can be configured only on the primary database,
only on a standby database, or on both the primary and standby
databases
 Primary database can support the transactional workload while the
standby database supports the analytic workload
 INMEMORY_ADG_ENABLED initialization parameter is set to true
on the standby database instance
In-Memory and Active Data Guard
 INMEMORY attribute with the DISTRIBUTE FOR SERVICE clause
set on all objects to be populated in the IM column store in the
standby database
 Create service which connects to ADG Standby database
 An object is only populated in the column store on database
instances on which the service is active
 Set Primary Database INMEMORY_SIZE=0 and ADG Standby
INMEMORY_SIZE=50G
In-Memory and Active Data Guard
 The redo generated on the primary database for all DML statements
includes metadata indicating whether the change is to an
INMEMORY object
 If an INMEMORY object is modified, then the standby database
invalidates the modified rows just as it does on the primary database,
using the transaction journal and Snapshot Metadata Unit (SMU)
to track the changes
 The repopulation mechanism works the same way in a standby
database as it does in a primary database
ALTER TABLE sales INMEMORY DISTRIBUTE FOR SERVICE reporting_standby_svc;
(command issued on Primary)
In-Memory and Automatic Data Optimization
 Automatic Data Optimization (ADO) was introduced in Oracle
Database 12c Release 1 to enable the automation of Information
Lifecycle Management (ILM) tasks
 ADO supports both compression tiering and storage tiering using
policies defined at the row or segment level on tables and partitions
 New in Oracle 12.2 ADO also manages content of IM column store
 User-defined policies to move objects in and out of IM column store
In-Memory and Automatic Data Optimization
 User-defined policies to adjust the compression level of objects
within the IM column store
 SET INMEMORY – enables INMEMORY attribute
 MODIFY INMEMORY – changes columnar compression level of the object from
lower to higher level
 NO INMEMORY – removes object from IM column store
ALTER TABLE sales_mar ILM ADD POLICY SET INMEMORY AFTER 14
DAYS OF CREATION;
 Table initially subject to a lot of DML but later on subject to mainly
analytical type queries
Hash Joins In-Memory 12.1.0.2
1)
Hash table created in memory
HASH TABLE
4)
Join completed by probing
Hash Join table in memory
to find matching rows
PGA
NAME
VEHICLES
2)
Table is accessed in memory via full scan
Data is uncompressed
NAME
SALES
3)
Table is accessed in memory via full scan
Data is uncompressed
Hashing algorithm applied
Data sent to hash join
Local Dictionary
Each IMCU has a local dictionary which has a list of distinct values and their
corresponding dictionary codes or compression symbol
The IMCU stores only the dictionary code rather than the original value
Optimizing Joins with Join Groups
 Join Groups have been introduced to improve the performance of
HASH JOINS on tables in the IM column store
 A join group is a set of columns on which a set of tables is frequently
joined
CREATE INMEMORY JOIN GROUP jgroup_name (sales(name), vehicles(name));
 The Join Group tells the IM column store that the NAME column in
both the VEHICLES and SALES tables should share the same
compression dictionary or common dictionary
Common Dictionary
Local Dictionary in each IMCU now has references to the Common Dictionary
Common Dictionary created when Join Group is created
Optimizing Joins with Join Groups
 The database operates on compressed data as it is stored in the
columnar store
 The is no costly decompressing of data first before hash join is
performed
 In a hash join based on a join group, the database uses an array
instead of building a hash table in PGA which is faster and uses less
resources
 The database stores codes for each join column value in a common
dictionary
 The database joins on the codes rather than on the actual column
values
Optimizing Joins with Join Groups
 The database automatically creates a common dictionary in the IM
column store when a join group is defined on the underlying
columns
 The common dictionary enables the join columns to share the same
dictionary codes
 The local dictionary now stores references to the values stored in
the common dictionary
 No decompressing of data for HASH JOINS
 Reduce memory requirements for performing HASH JOINS
Optimizing Joins with Join Groups
CREATE INMEMORY JOIN GROUP sh.sales_products_jg
(sh.sales(prod_id), sh.products(prod_id));
ALTER TABLE sh.sales INMEMORY;
ALTER TABLE sh.products INMEMORY;
Thanks for attending!!
Please send your feedback and any questions to:
Gavin Soorma
[email protected]
GoldenGate 12c Advanced Workshop
Course topics include :




Advanced troubleshooting tips and techniques
Configure Downstream Capture and Replication
Configure GoldenGate on a clustered environment
Providing High Availability for GoldenGate using Grid Infrastructure
agents
 Minimal downtime database upgrades using GoldenGate
 Install and configure GoldenGate Monitor 12c
 Install and configure GoldenGate Veridata 12c