* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download DataGuard_for_RAC_migrations_v2 - Indico
Survey
Document related concepts
Tandem Computers wikipedia , lookup
Entity–attribute–value model wikipedia , lookup
Serializability wikipedia , lookup
Microsoft Access wikipedia , lookup
Extensible Storage Engine wikipedia , lookup
Microsoft SQL Server wikipedia , lookup
Open Database Connectivity wikipedia , lookup
Functional Database Model wikipedia , lookup
Ingres (database) wikipedia , lookup
Oracle Database wikipedia , lookup
Microsoft Jet Database Engine wikipedia , lookup
Concurrency control wikipedia , lookup
Relational model wikipedia , lookup
Database model wikipedia , lookup
Transcript
Oracle Data Guard for RAC migrations WLCG Service Reliability Workshop CERN, November 30th, 2007 Jacek Wojcieszuk, CERN IT CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it LCG Outline • • • • • • Problem description Possible approaches Oracle Data Guard Migration Procedure Possible Variations Summary CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it RAC MIgration – WLCG Service Reliability Workshop, Nov 2007 - 2 Problem Description • More and more data centers run Oracle databases on commodity hardware relying on: – Software solutions for high availability (RAC, ASM) – Hardware redundancy • Using commodity hardware may impose relatively frequent hardware changes due to: – Short hardware lifetime – Short support period Replacing database hardware without significantly compromising service availability, becomes a challenge as database systems grow larger and larger. CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it RAC MIgration – WLCG Service Reliability Workshop, Nov 2007 - 3 Possible approaches – copy • Copy over the database with OS tools – Procedure • • • • Setup the new system (hardware and software) Stop the database Copy over datafiles, redo logs and control files and the spfile Open the database on the new hardware – Advantages: • Simple concept • Does not require knowing any extra tools (scp is enough) – Disadvantages • Difficult if ASM or RAW devices in use • Imposes long database downtime scp CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it datafiles, control files, redo logs spfile RAC MIgration – WLCG Service Reliability Workshop, Nov 2007 - 4 Possible approaches – export/import • Export/Import of the whole database – Procedure • Setup new system • Lock the original database for users • Copy over the data using exp/imp or expdp/impdp programs – Advantages: • Simplicity – Disadvantages: • Long database downtime proportional to the database size • Requires a lot of testing (export/import sometimes throw unexpected errors) export import import over DB link CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it RAC MIgration – WLCG Service Reliability Workshop, Nov 2007 - 5 Possible approaches - RMAN • Backup and Recovery with RMAN – Procedure: • • • • Setup the new system Stop and backup the old database Duplicate/recover the database to the new hardware Open the database with resetlogs – Advantages: • Faster then export/import approach • Old RMAN backups stay valid • Simple and reliable – Disadvantages: • Long downtime proportional to the database size CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it backup restore/ duplication RAC MIgration – WLCG Service Reliability Workshop, Nov 2007 - 6 Possible approaches – incremental replacement • Iterative hardware replacement – Procedure: • Remove a piece of old hardware from the cluster • Add a piece of new hardware to replace removed one • Repeat till all the hardware gets replaced – Advantages: • In theory no downtime • No extra space needed in the computing center – Disadvantages • Complicated and error-prone • Labor intensive • Requires a lot of communication between DBAs, Sysadmins and technicians CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it RAC MIgration – WLCG Service Reliability Workshop, Nov 2007 - 7 Possible approaches – Data Guard • Data Guard – Procedure • • • • Install new system Configure it as a standby database with Oracle Data Guard Perform switchover Redirect all users to the new system – Advantages • Very short downtime which lenght does not depend on the database size – Disadvantages • At first glance the procedure seems to be more complicated than at least some of procedures described before 1. Setup dataguard 2. Perform switchover CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it RAC MIgration – WLCG Service Reliability Workshop, Nov 2007 - 8 Oracle Data Guard • Widely used and mature feature of Oracle database software – Available since version 8i – Previously known as Standby Server • Helps to create and keep synchronized 1 or more standby databases • Well integrated with other HA features of Oracle software • Supports 2 types of standby database – Physical – Logical CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it RAC MIgration – WLCG Service Reliability Workshop, Nov 2007 - 9 Physical Standby Database Transactions Redo Transport Redo Apply Redo Stream Primary Database Physical Standby Database CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it RAC MIgration – WLCG Service Reliability Workshop, Nov 2007 - 10 Migration Procedure – step 1 • New system: – Hardware assembly (architecture must be the same as in case of the old system e.g. X86_64) – OS installation – Clusterware installation – RAC software installation: • Version must match the version on the old system • Use of clonning procedure highly recommended – Listener configuration • Preferebly using netca tool – Shared storage configuration • Using the same configuration as on the source system, although not mandatory, simplifies the migration • If you plan to use ASM configure and start ASM instances CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it RAC MIgration – WLCG Service Reliability Workshop, Nov 2007 - 11 Migration Procedure - step 2 • New system: – Configuration of naming methods on all nodes of the new cluster • There should be an entry pointing to the old system OLD_DB = (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = oldnode1-vip)(PORT = 1521)) (ADDRESS = (PROTOCOL = TCP)(HOST = oldnode2-vip)(PORT = 1521)) (LOAD_BALANCE = yes) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = orcl.cern.ch) ) ) – Creation of password files – Configuration of backup • At least one node should have access to the backups of the database being migrated CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it RAC MIgration – WLCG Service Reliability Workshop, Nov 2007 - 12 Migration Procedure – step 3 • On the old system: – Enabling forced logging • To ensure that all data changes will go to redo logs SQL> alter database force logging; – Performing a full backup (or at least level 1 backup) RMAN> backup database; – Performing control file backup for standby RMAN> backup current controlfile for standby; – Defining a TNS entry pointing to the new system CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it NEW_DB = (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = newnode1-vip)(PORT = 1521)) (ADDRESS = (PROTOCOL = TCP)(HOST = newnode2-vip)(PORT = 1521)) (LOAD_BALANCE = yes) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = orcl.cern.ch) ) ) RAC MIgration – WLCG Service Reliability Workshop, Nov 2007 - 13 Migration Procedure – step 4 • New system: – Preparation of parameter file: • The easiest is to reuse parameter file from the old system • Parameters to be added: log_archive_dest_2, standby_file_management, fal_server, fal_client, service_names • Parameters to be modified: db_recovery_file_dest, db_recovery_file_dest_size , db_create_file_dest, dump destinations • Parameters to be removed: control_files, autotuned memory allocation parameters CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it *.log_archive_dest_2='service=OLD_DB valid_for=(online_logfiles,primar y_role)' *.standby_file_management=auto *.fal_server='OLD_DB' *.fal_client='NEW_DB' *.service_names='orcl.cern.ch' # *.db_file_name_convert=... # *.log_file_name_convert=... # # # # # # Modify other parameters if needed: *.db_recovery_file_dest=... *.db_recovery_file_dest_size=... *.db_create_file_dest=... *.background_core_dump=... ... # Delete control_files parameter # Delete shared memory allocation parameters (parameters with names starting with double underscore) – Creation of the spfile RAC MIgration – WLCG Service Reliability Workshop, Nov 2007 - 14 Migration Procedure – step 5 • New System – Database duplication using RMAN rman target SYS@OLD_DB auxiliary / nocatalog RMAN> startup nomount RMAN> DUPLICATE TARGET DATABASE FOR STANDBY; – Enabling redo apply SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT FROM SESSION; – Update of the cluster registry: • Defining database and DB instance targets • Defining dependencies between ASM and DB instances • Defining custom services CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it srvctl srvctl srvctl srvctl srvctl srvctl srvctl BASIC add database -d orcl -o $ORACLE_HOME add instance -d orcl -i orcl1 -n newnode1 modify instance -d orcl -i orcl1 -s +ASM1 add instance -d orcl -i orcl2 -n newnode2 modify instance -d orcl -i orcl2 -s +ASM2 add service –d orcl –s orcl_loadbalanced –r orcl1,orcl2 –P BASIC add service –d orcl –s orcl_noloadbalanced –r orcl1 –a orcl2 –P RAC MIgration – WLCG Service Reliability Workshop, Nov 2007 - 15 Migration Procedure – step 6 • Old System: – Starting the synchronization: SQL> alter system set log_archive_dest_2='service=NEW_DB valid_for=(online_logfiles,primary_role)' scope=both sid='*'; SQL> alter system set standby_file_management=auto scope=both sid='*'; – Switch over to the standby role: • All services should be stopped • All DB instances but one should be stopped SQL> ALTER DATABASE COMMIT TO SWITCHOVER TO PHYSICAL STANDBY WITH SESSION SHUTDOWN; SQL> SHUTDOWN IMMEDIATE SQL> STARTUP MOUNT CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it RAC MIgration – WLCG Service Reliability Workshop, Nov 2007 - 16 Migration Procedure – step 7 • On the new system: – Switch over to the primary role: SQL> ALTER DATABASE COMMIT TO SWITCHOVER TO PRIMARY; SQL> ALTER DATABASE OPEN; – Startup other database instances and services – Redirect user to the new system CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it RAC MIgration – WLCG Service Reliability Workshop, Nov 2007 - 17 Possible Variations • Described procedure can be easily customized to allow performing extra actions: – OS version change – Migration to a bigger/smaller cluster – Changer of the storage management layer • With an extra intermediate step the procedure can be also used to migrate the database from 32 to 64 bit platform 3. Stop 1. Setup DataGuard database 32bit 32bit 2. Perform switchover 32bit 4. Start database in migrate mode and run migration script 64bit 64bit CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it RAC MIgration – WLCG Service Reliability Workshop, Nov 2007 - 18 Summary • The Data Guard based migration procedure has been used this year at CERN: – we migrated all production and validation databases ~15 systems in total – we moved from RHEL 3 to RHEL 4 at the same time – we also enlarged all production clusters – downtime associated with the migration did not exceed 1 hour per database CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it RAC MIgration – WLCG Service Reliability Workshop, Nov 2007 - 19