Download Overview of First4DB Monitoring

Oracle RAC with Data Guard
and Golden Gate
Examination of 2 X 3-Node Clusters across 2 Data Centers
Steve Recsky
Complete DBA Inc.
A Partner of First4 Database Partners Inc.
First4 Database Partners Inc.
Objectives

Review and discuss the following:










MAA and it’s components
Data Guard
RAC (Real Application Clusters)
Golden Gate
Data Guard set up and configuration
Oracle Cluster Resources and Services
Show a Data Guard Switchover
Show connection to Service
Discuss challenges for maintaining the environment
Questions
First4 Database Partners Inc.
Oracle MAA
Maximum Availability Architecture
 Address the following unplanned
downtime:


Site failure
Cluster wide failure
Computer failure
Storage failure
Data corruption
Human error
Lost writes

Hang or slowdown






First4 Database Partners Inc.
Oracle MAA
First4 Database Partners Inc.
Why RAC, Data Guard, Golden Gate?

RAC – Real Application Clusters
Highly scalable
 SCAN (Single Client Access Name)
support
 Removes single-point-of-failure for
database server (Except for disk
and/or site failure)

First4 Database Partners Inc.
RAC
Oracle® Real Application Clusters
Administration and Deployment Guide
11g Release 2 (11.2)
E41960-03
First4 Database Partners Inc.
Why RAC, Data Guard, Golden Gate?

Data Guard



Allows for the support of a Physical standby
(Redo Apply) and Logical standby (SQL
Apply) database that replicates the exact
contents of the Primary across the Oracle
Net network layer
If licensed, the standby database can be
opened in read-only mode to support the
Active Data Guard option
Purpose-built for disaster protection and
database rolling upgrades
First4 Database Partners Inc.
Why RAC, Data Guard, Golden Gate?

Data Guard Protection Levels
Maximum Protection (zero data loss,
synchronous replication)
 Maximum Availability (same as
Protection but does not shut down if a
fault prevents it from writing its redo
stream to the remote standby)
 Maximum Performance (asynchronous
replication)

First4 Database Partners Inc.
Data Guard
First4 Database Partners Inc.
Why RAC, Data Guard, Golden Gate?

Golden Gate



Supports subset replication to extract,
transform and aggregate data from
numerous data sources into a central data
store
Multi-master replication can be used to
synchronize several databases
Used for planned maintenance that Data
Guard can’t support (i.e. cross-endian
platform migration, application upgrades,
etc.)
First4 Database Partners Inc.
Golden Gate – single direction
First4 Database Partners Inc.
Golden Gate – bi-directional
FYI – Need to be aware of collisions with Sequences, Trigger processing
and Cascade Deletes
First4 Database Partners Inc.
Golden Gate Components
First4 Database Partners Inc.
Combination of all 3 solutions
First4 Database Partners Inc.
Environment
RAC
Node 1
Instance 1
RAC
Node 2
Instance 2
RW
Service
Primary
RAC
Node 3
Instance 3
Primary Data Center
(PDC)
First4 Database Partners Inc.
RO
Service
Data Guard
Secondary
RAC
Node 1
Instance 1
RAC
Node 2
Instance 2
RAC
Node 3
Instance 3
Secondary Data Center
(SDC)
Quick Steps to set up Data Guard


Make sure Primary database is in ARCHIVELOG mode
Prepare Primary for Data Guard








Configure Oracle Net
Configure Redo Transport



alter database force logging;
alter system set standby_file_management = 'AUTO';
alter database add standby logfile…
set db_unique_name parameter
alter system set fal_server =…
alter system set log_archive_config = 'dg_config=(…)’;
log_archive_dest_1
log_archive_dest_2
Prepare Standby




Copy password file from $ORACLE_HOME/dba/orapw…
Create standby spfile from the primary
alter sytem set db_file_name_convert=…
alter system set log_file_name_convert=…
First4 Database Partners Inc.
Quick Steps to set up Data Guard


startup the Standby database with nomount…
Create Standby Database


Start the Redo Apply


ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT LOGFILE
DISCONNECT FROM SESSION;
Verify Redo Apply is working





RMAN – duplicate target database … for standby from active database;
Primary and Standby: select DEST_ID, STATUS, DESTINATION, ERROR from V$ARCHIVE_DEST where
DEST_ID <=2;
Primary: select SEQUENCE#, FIRST_TIME, NEXT_TIME, APPLIED, ARCHIVED from
V$ARCHIVED_LOG where name = ‘…' order by FIRST_TIME;
select STATUS, GAP_STATUS from V$ARCHIVE_DEST_STATUS where DEST_ID = 2;
Look for errors here

select * from V$DATAGUARD_STATUS order by TIMESTAMP;
First4 Database Partners Inc.
Oracle Cluster Services and RAC
PDC
pdcoradb1[DBATEST1](oracle):/home/oracle> srvctl status database -d DBATEST_PDC
Instance DBATEST1 is running on node pdcoradb1
Instance DBATEST2 is running on node pdcoradb2
Instance DBATEST3 is running on node pdcoradb3
SDC
sdcoradb1[DBATEST1](oracle):/home/oracle> srvctl status database -d DBATEST_SDC
Instance DBATEST1 is running on node sdcoradb1
Instance DBATEST2 is running on node sdcoradb2
Instance DBATEST3 is running on node sdcoradb3
First4 Database Partners Inc.
Oracle Cluster Services and RAC
PDC
pdcoradb1[DBATEST1](oracle):/home/oracle> srvctl config database -d DBATEST_PDC
Database unique name: DBATEST_PDC
Database name: DBATEST
Oracle home: /usr/oracle/product/11.2.0.2
Oracle user: oracle
Spfile: /oradata/pdcoraclu/admin/dbs/spfileDBATEST.ora
Domain: PDC
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Server pools: DBATEST_PDC
Database instances: DBATEST1,DBATEST2,DBATEST3
Disk Groups:
Mount point paths:
Services: DBATEST_RO.CORP,DBATEST_RW.CORP
Type: RAC
Database is administrator managed
First4 Database Partners Inc.
Oracle Cluster Services and RAC
PDC
sdcoradb1[DBATEST1](oracle):/home/oracle> srvctl config database -d DBATEST_SDC
Database unique name: DBATEST_SDC
Database name: DBATEST
Oracle home: /usr/oracle/product/11.2.0.2
Oracle user: oracle
Spfile: /oradata/sdcoraclu/admin/dbs/spfileDBATEST.ora
Domain: SDC
Start options: open
Stop options: immediate
Database role: PHYSICAL_STANDBY
Management policy: AUTOMATIC
Server pools: DBATEST_SDC
Database instances: DBATEST1,DBATEST2,DBATEST3
Disk Groups:
Mount point paths:
Services: DBATEST_RO.CORP,DBATEST_RW.CORP
Type: RAC
Database is administrator managed
First4 Database Partners Inc.
Data Guard Broker
PDC/SDC
DGMGRL> show database verbose "DBATEST_PDC"
Database - DBATEST_PDC
Role:
Intended State:
Instance(s):
DBATEST1
DBATEST2
DBATEST3
PRIMARY
TRANSPORT-ON
Properties:
DGConnectIdentifier
= 'DBATEST.PDC'
ObserverConnectIdentifier
= ''
<...snipped...>
StandbyFileManagement
= 'AUTO'
ArchiveLagTarget
= '1800'
LogArchiveMaxProcesses
= '4'
LogArchiveMinSucceedDest
= '1'
DbFileNameConvert
= ''
LogFileNameConvert
= '/oradata/DBATEST/redo/redoA, /oradata/DBATEST/redo/redoA,
/oradata/DBATEST/redo/redoB, /oradata/DBATEST/redo/redoB'
FastStartFailoverTarget
= ''
<..snipped..>
Database Status:
SUCCESS
ArchiveLagTarget limits the amount of data that can be lost and effectively increases the
availability of the standby database by forcing a log switch after the specified amount of time
elapses.
First4 Database Partners Inc.
Data Guard Broker
PDC/SDC
DGMGRL>
show instance verbose "DBATEST1" on database "DBATEST_PDC"
Instance 'DBATEST1' of database 'DBATEST_PDC'
Host Name: pdcoradb1.CORP.ca
PFILE:
Properties:
SidName
= 'DBATEST1'
StaticConnectIdentifier
=
'(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=10.96.44.38)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=DBATEST.PDC)(INSTANCE
_NAME=DBATEST1)(SERVER=DEDICATED)))'
StandbyArchiveLocation
= '/oradata/DBATEST/arch'
AlternateLocation
= ''
LogArchiveTrace
= '0'
LogArchiveFormat
= 'DBATEST_%t_%s_%r.arc'
TopWaitEvents
= '(monitor)'
Instance Status:
SUCCESS
Set StaticConnectIdentifier so as to not have to define a TNS
Names entry called service_name_DGMGRL
First4 Database Partners Inc.
Services
pdcoradb1[DBATEST1](oracle):/home/oracle> srvctl config service -s DBATEST_RW.CORP -d DBATEST_PDC
Service name: DBATEST_RW.CORP
Service is enabled
Server pool: DBATEST_PDC_DBATEST_RW.CORP
Cardinality: 3
Disconnect: false
Service role: PRIMARY
Management policy: AUTOMATIC
DTP transaction: false
AQ HA notifications: true
Failover type: SELECT
Failover method: BASIC
TAF failover retries: 0
TAF failover delay: 0
Connection Load Balancing Goal: LONG
Runtime Load Balancing Goal: NONE
TAF policy specification: NONE
Edition:
Preferred instances: DBATEST1,DBATEST2,DBATEST3
Available instances:
First4 Database Partners Inc.
Services
sdcoradb1[DBATEST1](oracle):/home/oracle> srvctl config service -s DBATEST_RO.CORP -d DBATEST_SDC
Service name: DBATEST_RO.CORP
Service is enabled
Server pool: DBATEST_SDC_DBATEST_RO.CORP
Cardinality: 3
Disconnect: false
Service role: PHYSICAL_STANDBY
Management policy: AUTOMATIC
DTP transaction: false
AQ HA notifications: true
Failover type: SELECT
Failover method: BASIC
TAF failover retries: 0
TAF failover delay: 0
Connection Load Balancing Goal: LONG
Runtime Load Balancing Goal: NONE
TAF policy specification: NONE
Edition:
Preferred instances: DBATEST1,DBATEST2,DBATEST3
Available instances:
First4 Database Partners Inc.
Services
sdcoradb1[DBATEST1](oracle):/home/oracle> sqlplus srecsky@dbatest_rw.CORP
SQL*Plus: Release 11.2.0.2.0 Production on Mon Dec 9 13:10:52 2013
Copyright (c) 1982, 2010, Oracle.
All rights reserved.
Enter password:
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options
13:10:57 DBATEST1 SRECSKY>select host_name from v$instance;
HOST_NAME
---------------------------------------------------------------pdcoradb1.CORP.ca
DBATEST_RW.CORP =
(DESCRIPTION =
(ADDRESS_LIST=
(ADDRESS = (PROTOCOL = TCP)(HOST = sdcoraclu-scan.CORP.ca)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = pdcoraclu-scan.CORP.ca)(PORT = 1521))
)
(CONNECT_DATA = (SERVICE_NAME = DBATEST_RW.CORP)
(FAILOVER_MODE=(TYPE=SELECT)(METHOD=BASIC)(RETRIES=30)(DELAY=5))
)
)
First4 Database Partners Inc.
Data Guard Switchover
pdcoradb1[DBATEST1](oracle):/home/oracle> dgmgrl
DGMGRL for Linux: Version 11.2.0.2.0 - 64bit Production
Copyright (c) 2000, 2009, Oracle. All rights reserved.
Welcome to DGMGRL, type "help" for information.
DGMGRL> connect sys/****************@dbatest.corppdc
Connected.
DGMGRL> switchover to 'DBATEST_CORPSDC';
Performing switchover NOW, please wait...
New primary database "DBATEST_CORPSDC" is opening...
Operation requires shutdown of instance "DBATEST3" on database "DBATEST_CORPPDC"
Shutting down instance "DBATEST3"...
ORACLE instance shut down.
Operation requires startup of instance "DBATEST3" on database "DBATEST_CORPPDC"
Starting instance "DBATEST3"...
ORACLE instance started.
Database mounted.
Database opened.
Switchover succeeded, new primary is "DBATEST_CORPSDC"
DGMGRL>
First4 Database Partners Inc.
Data Guard – RW Connection
H:\>sqlplus srecsky@dbatest_rw.corp
SQL*Plus: Release 11.2.0.1.0 Production on Tue Dec 10 14:37:47 2013
Copyright (c) 1982, 2010, Oracle.
All rights reserved.
Enter password:
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options
SQL> select host_name from v$instance;
HOST_NAME
---------------------------------------------------------------sdcoradb2.corp.ca
SQL>
First4 Database Partners Inc.
Data Guard – RO Connection
H:\>sqlplus srecsky@dbatest_ro.corp
SQL*Plus: Release 11.2.0.1.0 Production on Tue Dec 10 14:51:07 2013
Copyright (c) 1982, 2010, Oracle.
All rights reserved.
Enter password:
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options
SQL> select host_name from v$instance;
HOST_NAME
---------------------------------------------------------------pdcoradb3.corp.ca
SQL>
First4 Database Partners Inc.
Data Guard - Configuration
DGMGRL> show configuration
Configuration - DBATEST
Protection Mode: MaxPerformance
Databases:
DBATEST_CORPSDC - Primary database
DBATEST_CORPPDC - Physical standby database
Fast-Start Failover: DISABLED
Configuration Status:
SUCCESS
DGMGRL>
sdcoradb1[DBATEST1](oracle):/home/oracle> srvctl status service -s DBATEST_RW.CORP -d
DBATEST_CORPSDC
Service DBATEST_RW.CORP is running on instance(s) DBATEST1,DBATEST2,DBATEST3
First4 Database Partners Inc.
Golden Gate Challenge
Oracle
Database
Oracle
Database
Primary
G
o
l
d
e
n
G
a
t
e
Data Guard
Secondary
G
o
l
d
e
n
G
a
t
e
GG
Checkpoin
t/Trail Files
Primary
RAC
RAC
Node
1
RAC
Node
Instance 12
Node23
Instance
Instance 3
Primary Data Center
First4 Database Partners Inc.
Data Guard
DBFS
Primary
Primary
Secondary
GG
Checkpoin
t/Trail Files
RAC
RAC
Node
1
Node
RAC
Instance
12
Instance
Node 23
Instance 3
Secondary Data Center
DBFS
Oracle Database File System





Creates a standard file system interface on top of files
and directories that are stored in database tables
Similar to NFS in that it provides a shared network file
system that looks like a local file system
Files are stored as SecureFiles LOBs in a database table
A set of PL/SQL procedures implement the file system
access primitives such as create, open, read, write, and
list directory
Oracle states that there can be overhead on the primary
database for the DBFS being part of it
First4 Database Partners Inc.
DBFS
Oracle Database File System
Oracle® Database SecureFiles and Large
Objects Developer's Guide
11g Release 2 (11.2)
E18294-04
First4 Database Partners Inc.
Doc ID 1436913.1




Best Practice - Oracle GoldenGate 11gr2 integrated
extract and Oracle Data Guard - Switchover/Fail-over
Operations
Integrate GoldenGate integrated extract with Data Guard
so that GoldenGate processes can also be switchedover/failed-over automatically with no manual
intervention as part of Data Guard switchover/failover
Switchover/failover is actually done by a UNIX shell
program which is triggered by the database system event
"DB_ROLE_CHANGE" when switchover/failover occurs
Database must be 11.2.0.3 or higher to support
Integrated Extract
First4 Database Partners Inc.
Some Parting Thoughts




Look at tools available to you as part of the MAA
Follow best practices
Build, test and document
Be cognizant of the license requirements
First4 Database Partners Inc.
The End
[email protected]
First4 Database Partners Inc.