Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download g EMC VSPEX FOR VIRTUALIZED ORACLE DATABASE 11 OLTP
Document related concepts
Entity–attribute–value model wikipedia , lookup
Microsoft SQL Server wikipedia , lookup
Open Database Connectivity wikipedia , lookup
Extensible Storage Engine wikipedia , lookup
Functional Database Model wikipedia , lookup
Microsoft Jet Database Engine wikipedia , lookup
Ingres (database) wikipedia , lookup
Relational model wikipedia , lookup
Concurrency control wikipedia , lookup
Database model wikipedia , lookup
Transcript
IMPLEMENTATION GUIDE
EMC VSPEX FOR VIRTUALIZED ORACLE
DATABASE 11g OLTP
EMC VSPEX
Abstract
This Implementation Guide describes the high-level steps and best practices
required to implement the VSPEX Proven Infrastructure for Virtualized Oracle
Databases on a VSPEX Private Cloud with VMware vSphere enabled by VNX and
EMC backup and recovery systems.
April 2013
Copyright © 2013 EMC Corporation. All rights reserved. Published in the USA.
EMC believes the information in this publication is accurate of its publication date.
The information is subject to change without notice.
The information in this publication is provided as is. EMC Corporation makes no
representations or warranties of any kind with respect to the information in this
publication, and specifically disclaims implied warranties of merchantability or
fitness for a particular purpose. Use, copying, and distribution of any EMC software
described in this publication requires an applicable software license.
EMC2, EMC, and the EMC logo are registered trademarks or trademarks of EMC
Corporation in the United States and other countries. All other trademarks used
herein are the property of their respective owners.
For the most up-to-date regulatory document for your product line, go to the technical
documentation and advisories section on the EMC online support website.
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Part Number H11445
2
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Contents
Chapter 1
Introduction ........................................................................ 11
Purpose of this guide .................................................................................... 12
Business value.............................................................................................. 12
Scope............................................................................................................ 13
Audience....................................................................................................... 13
Terminology .................................................................................................. 14
Chapter 2
Before You Start .................................................................. 15
Overview ....................................................................................................... 16
Pre-deployment tasks .......................................................................................... 16
Documentation workflow .............................................................................. 17
Deployment prerequisites ............................................................................. 17
Planning and sizing Oracle Database 11gR2 ................................................. 19
Support resources......................................................................................... 19
Design guide ........................................................................................................ 19
Solution overviews............................................................................................... 19
VSPEX Proven Infrastructure ................................................................................. 19
VMware documentation ....................................................................................... 19
Chapter 3
Solution Overview ............................................................... 21
Overview ....................................................................................................... 22
Solution architecture............................................................................................ 22
Key components ........................................................................................... 24
Introduction ......................................................................................................... 24
EMC VSPEX .......................................................................................................... 24
Oracle Database 11g............................................................................................ 26
VMware vSphere 5.1 ............................................................................................ 27
VMware vSphere HA ............................................................................................. 27
VMware vSphere Distributed Resource Scheduler ................................................ 27
VMware vSphere PowerCLI ................................................................................... 28
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
3
Contents
Red Hat Enterprise Linux 6.3 ................................................................................ 28
EMC VNX series .................................................................................................... 28
EMC Virtual Storage Integrator ............................................................................. 29
EMC Unisphere .................................................................................................... 29
EMC Avamar......................................................................................................... 29
EMC Data Domain ................................................................................................ 30
Chapter 4
Solution Implementation..................................................... 31
Overview ....................................................................................................... 32
Setting up the physical components ............................................................. 32
Implementing the network ............................................................................ 33
Implementing the storage array .................................................................... 35
Overview .............................................................................................................. 35
Setting up initial VNX configuration ...................................................................... 35
Provisioning storage for the NFS datastore .......................................................... 36
Configuring Unified Storage Management ............................................................ 36
Configuring FAST Cache (optional) ....................................................................... 39
Example storage layout ........................................................................................ 41
Implementing ESXi and vCenter .................................................................... 42
Preparing the Oracle Database 11gR2 virtual machine template................... 43
Overview .............................................................................................................. 43
Creating the Oracle Database 11gR2 virtual machine ........................................... 43
Completing the Oracle install pre-requisites......................................................... 44
Installing the Oracle Database 11gR2 binaries and patches ................................. 46
Preparing the Oracle Database 11gR2 virtual machine template .......................... 48
Customizing the VMware template ....................................................................... 48
Deploying the Oracle Database 11gR2 virtual machine ................................. 49
Overview .............................................................................................................. 49
Deploying new virtual machines........................................................................... 49
Cloning the Oracle binaries ona new virtual machine ........................................... 50
Determining the database prerequisites .............................................................. 51
Configuring the database file layout for NFS ......................................................... 52
Configuring the initialization parameters ............................................................. 53
Enabling the HugePages setting ........................................................................... 54
Creating the database using DBCA ....................................................................... 55
Performing the database post creation activities .................................................. 55
Implementing backup and recovery .............................................................. 58
Overview .............................................................................................................. 58
Minimum backup software and hardware requirements ....................................... 59
Deploying the backup software ............................................................................ 59
4
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Contents
Installing the backup software ............................................................................. 60
Configuring the backup software .......................................................................... 61
Using the Data Domain system ............................................................................ 66
Chapter 5
Solution Verification ........................................................... 69
Verifying the baseline infrastructure ............................................................. 70
Overview .............................................................................................................. 70
Verifying the ESXi functionality............................................................................. 70
Verifying the solution component redundancy ..................................................... 70
Verifying the Oracle Database configuration......................................................... 71
Using the health monitoring tools ........................................................................ 71
Verifying the Oracle Database ....................................................................... 72
Overview .............................................................................................................. 72
Understanding the key metrics............................................................................. 72
Reviewing the Oracle dataset ............................................................................... 74
Backup and recovery verification methodology ............................................. 74
Overview .............................................................................................................. 74
Backing up an Oracle database ............................................................................ 75
Changing the database contents ......................................................................... 75
Restoring the Oracle database ............................................................................. 75
Chapter 6
Reference Documentation ................................................... 77
White papers................................................................................................. 78
Oracle .................................................................................................................. 78
EMC ..................................................................................................................... 78
VMware................................................................................................................ 78
Product documentation ................................................................................ 79
EMC ..................................................................................................................... 79
Oracle .................................................................................................................. 79
VMWare ............................................................................................................... 79
Other documentation .................................................................................... 80
Links ............................................................................................................. 80
Appendix A
Configuration Worksheet ..................................................... 81
Configuration worksheet for Oracle Database 11gR2 ................................... 82
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
5
Contents
6
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Figures
Figure 1.
Figure 2.
Figure 3.
Figure 4.
Figure 5.
Figure 6.
Figure 7.
Figure 8.
Figure 9.
Figure 10.
Figure 11.
Figure 12.
Figure 13.
Figure 14.
Figure 15.
Figure 16.
Figure 17.
Figure 18.
Figure 19.
Figure 20.
Figure 21.
Figure 22.
Figure 23.
Figure 24.
Figure 25.
Figure 26.
Figure 27.
Figure 28.
Figure 29.
Figure 30.
Figure 31.
Figure 32.
Figure 33.
Figure 34.
Architecture of the validated infrastructure ......................................... 23
VSPEX Proven Infrastructure ............................................................... 25
Storage network configuration............................................................ 34
Oracle Database 11gR2 storage elements .......................................... 35
Configuring EMC Unified Storage Management in vSphere Client........ 36
Configuring NFS Storage on VNX for Oracle VMs ................................. 37
Configuring VNX storage pools ........................................................... 37
Unisphere Create File System Wizard.................................................. 38
EMC Unisphere Create NFS Export Wizard ........................................... 39
Enabling FAST Cache in the Storage Pool Properties dialog ................. 40
Example storage layout for VNX .......................................................... 41
Oracle Universal Installer – Install choose database software only ..... 46
Enable Oracle Direct NFS client ........................................................... 47
Script root_clone.sh to be created and owned by root ........................ 47
Script cloneHome.sh to be created and owned by Oracle ................... 47
Details for template, vspx-Orac11203EE-2vcpu-8GB-tmplt ................. 48
Details custom specification, Oracle_2vCPU_8GB_11203_EE_spec ... 48
Example PowerCLI script to deploy a virtual machine from template ... 50
Sample PowerCLI script to clone and reattach the ORACLE_HOME ...... 51
Oracle Net Configuration Assistant utility ........................................... 51
Sample /etc/mtab NFS mount opitons ............................................... 52
Oranfstab example ............................................................................. 53
Oracle Database Configuration Assistant setup screen ....................... 55
Oracle startup/shutdown script /etc/init.d/dbora .............................. 57
Avamar server and Avamar Plug-in for Oracle...................................... 58
Link to 'Documents and Downloads' below for Avamar documentation
and server/client agents and plug-ins. ............................................... 60
Avamar Administrator authentication and server (Grid) selection ....... 61
Avamar dataset for Oracle RMAN (LINUX) ............................................ 62
Specifying the Avamar Oracle dataset information ............................. 63
Avamar Schedule named 'BRS' in force for RMAN dataset named
‘Oracle’ in this case. ........................................................................... 64
Avamar Retention policy named 'BRSORACLE' in force for this dataset
and earlier set schedule. .................................................................... 65
Avamar Policy tab highlighting Group Name of 'BRS' for the 'Oracle'
dataset with prior Schedules and Retention Periods in force per the
created policy. .................................................................................... 66
Avamar's Server panel ........................................................................ 67
Avamar’s 'Add Data Domain System' panel ........................................ 68
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
7
Figures
Figure 35.
Figure 36.
Figure 37.
Figure 38.
Figure 39.
8
Swingbench interface load panel screenshot ..................................... 73
Swingbench order entry screen for workload simulation ..................... 73
Status of Completed Avamar backup job. ........................................... 75
Avamar Administrator Backup and Restore tab ................................... 76
Avamar's Activity panel showing a Completed job .............................. 76
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Tables
Table 1.
Table 2.
Table 3.
Table 4.
Table 5.
Table 6.
Table 7.
Table 8.
Table 9.
Table 10.
Table 11.
Table 12.
Table 13.
Table 14.
Table 15.
Table 16.
Table 17.
Table 18.
Table 19.
Table 20.
Table 21.
Table 22.
Table 23.
Table 24.
Table 25.
Table 26.
Table 27.
Table 28.
Table 29.
Table 30.
Terminology........................................................................................ 14
Tasks for pre-deployment ................................................................... 16
Solution deployment process ............................................................. 17
Deployment prerequisites checklist .................................................... 18
Reference virtual machine characteristics ........................................... 25
Mapping the Oracle sizing model to the
VSPEX reference virtual machine ........................................................ 26
Tasks for physical setup ..................................................................... 32
Tasks for switch and network configuration ........................................ 33
Tasks for storage configuration........................................................... 35
VNX storage layouts for Oracle Database ............................................ 36
Steps for installing and configuring ESXi and vCenter ......................... 42
Virtual machine template specification............................................... 43
Oracle install prerequisites ................................................................. 44
Red Hat Enterprise Linux 6 RPM packages for Oracle Database 11gR2 45
Installing the Oracle Database binaries and patches .......................... 46
Oracle Database 11gR2 reference virtual machine model ................... 49
Database file layout for NFS ................................................................ 52
Hardware requirements for Avamar Grid ............................................. 59
Install Avamar Admin and configure Oracle Server .............................. 59
Tasks for verifying the VSPEX Proven Infrastructure installation ......... 70
Tools for monitoring the solution ........................................................ 71
General operations ............................................................................. 72
Schema tables and indexes ................................................................ 74
Avamar backup terminology ............................................................... 74
Common server information ............................................................... 82
ESXi server information ...................................................................... 82
Array information................................................................................ 83
Network infrastructure information ..................................................... 83
VLAN information ............................................................................... 84
Service accounts ................................................................................ 84
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
9
Tables
10
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Chapter 1
Introduction
This chapter presents the following topics:
Purpose of this guide ................................................................................. 12
Business value .......................................................................................... 12
Scope ........................................................................................................ 13
Audience ................................................................................................... 13
Terminology............................................................................................... 14
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
11
Introduction
Purpose of this guide
EMC® VSPEX™ Proven Infrastructures are optimized for virtualizing business-critical
applications. VSPEX provides modular solutions built with technologies that enable
faster deployment, more simplicity, greater choice, higher efficiency, and lower risk.
VSPEX provides partners with the ability to design and implement the virtual assets
required to support a fully integrated virtualized solution for Oracle relational
database management systems on a VSPEX private cloud infrastructure.
The VSPEX for virtualized Oracle infrastructure provides customers with a modern
system, capable of hosting a virtualized database solution that is scalable and
delivers a constant performance level. This solution uses VMware vSphere™ to
provide virtualization, backed by EMC VNX® providing storage, and EMC Avamar® and
EMC Data Domain® for backup. The compute and network components, while vendordefinable, are designed to provide redundancy and sufficient power to handle the
processing and data needs of the virtual machine environment.
This Implementation Guide describes how to implement, using best practices, the
VSPEX Proven Infrastructure for virtualized Oracle Databases on a VSPEX Private
Cloud with VMware vSphere.
Business value
Database management systems software continues to be the dominant software
used to manage data in nearly all commercial segments. This growth is expected to
continue despite the increasing market share of other data management tools. This
growth is expected to accelerate as customers continue to diversify their
infrastructures and supporting technologies and drive towards more hardware and
software appliances and configurations.
This VSPEX Proven Infrastructure is focused on helping EMC partners understand the
value that the VNX series, EMC backup and recovery systems, and Oracle bring to
customers who often have growing, isolated IT environments running server-centric
applications and who face increasing Oracle backup and recovery issues.
This VSPEX solution is designed to meet the customer’s Oracle database challenges
while enabling customers to grow in performance, scalability, reliability, and
automation. By consolidating their database applications on EMC VNX, they can
consolidate onto a single centralized storage platform that enables them to more
effectively manage the exploding growth in data that is challenging businesses today.
This solution has been sized and proven with EMC backup and recovery systems to:
12
•
Reduce the customer’s backup storage requirements and costs
•
Meet backup windows
•
Enable fast disk-based recovery
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Introduction
Scope
This Implementation Guide describes the high-level steps required to deploy the
VSPEX virtualization solution for Oracle RDBMS and Online Transaction Processing
(OLTP) systems. This VSPEX application solution is based on the current VSPEX
Proven Infrastructure: VMware Private Cloud and has been modified to support Oracle
Database requirements. This solution uses the VSPEX standard or “reference virtual
machine” building blocks of vCPU, memory, and storage/ I/O configurations, and
describes the implementation based on three (small, medium, and large) different
database workload thresholds.
This Implementation Guide applies to and assumes that a VSPEX Proven
Infrastructure with VMware vSphere with VNX and EMC backup and recovery systems
already exists in the customer’s environment.
Audience
This guide is intended for internal EMC personnel and qualified EMC VSPEX partners.
This guide assumes the VSPEX partners who intend to deploy this solution are:
•
Qualified by EMC to sell, install, and configure the EMC VNX family of storage
systems
•
Qualified to sell, install, and configure the network and server products
required for VSPEX Proven Infrastructures
•
Certified for selling VSPEX Proven Infrastructure
Partners implementing this solution should also have the necessary technical
training and background to install and configure:
•
VMware vSphere
•
Redhat Enterprises Linux 6.3
•
Oracle Database 11gR2 or above
•
EMC next-generation backup, which includes EMC Avamar and EMC Data
Domain
This document provides external references where applicable. EMC recommends that
readers are familiar with these documents. For details, refer to the section entitled
Support resources.
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
13
Introduction
Terminology
Table 1 lists the terminology used in the guide.
Table 1.
14
Terminology
Term
Definition
AWR
Automatic Workload Repository
DNFS
Oracle Direct NFS client
DNS
Domain name system
FAST VP
Fully Automated Storage Tiering for Virtual Pools
FQDN
Fully Qualified Domain Name
IOPS
Input/output operations per second
FRA
Fast Recovery Area (Oracle)
NFS
Network File System
NL-SAS
Near-line serial-attached SCSI
ODM
Oracle Disk Manager
OLTP
Online transaction processing
Oracle EE
Oracle Enterprise Edition
Oracle SE
Oracle Standard Edition
PowerCLI
A Windows PowerShell interface to the VMware vSphere and vCloud
APIs
Reference virtual
machine
Represents a unit of measure for a single virtual machine to qualify
the compute resources in a VSPEX Proven Infrastructure
SGA
System global area
Statspack
Oracle database monitoring and reporting utilities
TPS
Transactions per second
VMDK
VMware virtual machine disk
VMFS
VMware virtual machine file system
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Chapter 2
Before You Start
This chapter presents the following topics:
Overview ................................................................................................... 16
Documentation workflow ........................................................................... 17
Deployment prerequisites .......................................................................... 17
Planning and sizing Oracle Database 11gR2 ............................................... 19
Support resources ..................................................................................... 19
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
15
Before You Start
Overview
Before you layer the Oracle OLTP on a VSPEX private cloud, EMC recommends you
complete the pre-deployment tasks listed in Table 2.
Pre-deployment
tasks
Pre-deployment tasks include procedures that do not directly relate to environment
installation and configuration, but whose results are needed at the time of
installation. Examples of pre-deployment tasks include the collection of hostnames,
IP addresses, VLAN IDs, license keys, installation media, and so on. Perform these
tasks before visiting customer to decrease the time required on site.
Table 2.
16
Tasks for pre-deployment
Task
Description
Reference
Gather documents
Gather the related documents listed in
Support resources. These are used
throughout the text of this document to
provide details on setup procedures and
deployment best practices for the various
components of the solution.
Support resources
Gather tools
Gather the required and optional tools for
the deployment. Use Table 4 to confirm that
all equipment, software, and appropriate
licenses are available before the
deployment process.
Deployment
prerequisites
Gather data
Collect the customer-specific configuration
data for networking, naming, and required
accounts. Enter this information into the
Customer Oracle Database 11gR2
configuration worksheet for reference during
the deployment process.
Appendix A
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Before You Start
Documentation workflow
EMC recommends that you follow the process flow in Table 3 to design and
implement your VSPEX Proven Infrastructure for virtualized Oracle Database 11gR2
solution.
Table 3.
Solution deployment process
Step
Action
1
Use the VSPEX qualification worksheet to collect user requirements. The qualification
worksheet is in Appendix A of the companion Design Guide.
2
Use the VSPEX Sizing Tool to determine the recommended VSPEX Proven
Infrastructure for a virtualized Oracle Database 11gR2 solution.
For more information about the VSPEX Sizing Tool, refer to the VSPEX Sizing Tool on
the EMC Business Value Portal.
Note You need to register the first time you access the tool. If the VSPEX Sizing Tool is
not available, you can manually size the application using the sizing guidelines in
Appendix B of the companion Design Guide.
3
To determine the final design for the VSPEX Proven Infrastructure for virtualized
Oracle Database 11g, refer to the companion Design Guide.
Note Ensure that all application requirements are considered, not just this
application.
4
To select and order the right solution, refer to the appropriate VSPEX Proven
Infrastructure document in Support resources.
5
To deploy and test your VSPEX solution, refer to this guide.
Deployment prerequisites
Table 4 itemizes the hardware and software requirements to configure the solution.
For additional information, refer to the hardware and software tables in the relevant
document in the Support resources section.
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
17
Before You Start
Table 4.
Requirement
Deployment prerequisites checklist
Description
Version
Physical servers: sufficient physical server
capacity to host the required number of virtual
machines as recommended by the companion
Design Guide and VSPEX Sizing Tool.
Hardware
Reference notes
Reference Architecture:
EMC VSPEX Private
Cloud: VMware vSphere
5.1 for up to 500 Virtual
Machines
VMware vSphere 5.1 servers to the host virtual
infrastructure servers. This requirement may be
covered in the existing infrastructure.
EMC Avamar 6.1
hardware and licenses
Networking: switch port capacity and capabilities
as required by the virtual server infrastructure.
EMC Data Domain
hardware and licenses
EMC VNX: multiprotocol storage array with the
required disk layout.
EMC Backup: Avamar 6.1, Data Domain
Software
VNX OE for file
Release
7.1.47-5
VNX OE for block
Release
05.32.000.5
.008
EMC VSI for VMware vSphere: Unified Storage
Management
5.4
EMC VSI for VMware vSphere: Storage Viewer
5.4
VMware ESXi™ installation media
5.1
VMware vCenter Server installation media
5.1
EMC Avamar
6.1 in server
and client
versions
Red Hat Enterprise Linux 6.3
Red Hat
Enterprise
Linux 6.3
media
www.redhat.com
VMware vSphere license keys. This may be
covered in the VSPEX Proven Infrastructure.
5.1
www.vmware.com
Red Hat Enterprise Linux
6.3
www.redhat.com
Oracle Database 11gR2
11.2.0.x
www.oracle.com
EMC online support
Licenses
18
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Before You Start
Planning and sizing Oracle Database 11gR2
To plan and size your Oracle Database 11gR2 deployment on the VSPEX
infrastructure, you should follow the recommendations in the companion Design
Guide. Size the storage accordingly to information obtained from the VSPEX for
Virtualized Oracle 11g Qualification worksheet as detailed in Appendix A & B of the
companion Design Guide.
Support resources
Before implementing the solution described in this document, EMC recommends that
you read the following documents, available from the VSPEX space in the EMC
Community Network or from EMC.com or the VSPEX Proven Infrastructure partner
portal. .
Design guide
•
EMC VSPEX for Virtualized Oracle Database 11g OLTP Design Guide
Solution overviews
•
EMC VSPEX Server Virtualization for Midmarket Businesses
•
EMC VSPEX Server Virtualization for Small and Medium Businesses
•
Reference Architecture: EMC VSPEX Private Cloud: VMware vSphere 5.1 for up
to 500 Virtual Machines
•
White Paper: EMC Avamar Backup for Oracle Environments
•
White Paper: EMC Avamar Backup with Data Domain
VSPEX Proven
Infrastructure
VMware
documentation
VMware VSphere Product Documentation
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
19
Before You Start
20
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Chapter 3
Solution Overview
This chapter presents the following topics:
Overview ................................................................................................... 22
Key components ........................................................................................ 24
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
21
Solution Overview
Overview
This chapter provides an overview of the VSPEX Proven Infrastructure for Oracle
Database 11gR2 and the key technologies used in this solution. This Implementation
Guide documents a virtualized Oracle solution built upon on a VSPEX private cloud
that supports up to 500 virtual machines.
This VSPEX Proven Infrastructure for virtualized Oracle Database solution was
validated using VNX and VMware virtualized Linux platforms to consolidate storage
and server hardware.
The solution described in this guide includes servers, storage, network, backup and
recovery, and Oracle Database components that are focused on midrange
environments. The solution enables customers to quickly and consistently deploy a
medium virtualized Oracle Database in a VSPEX Proven Infrastructure.
The VNX storage arrays are multiprotocol platforms that can support the iSCSI, NFS,
and CIFS protocols depending on the customer’s specific needs.
Solution
architecture
Figure 1 shows the architecture that characterizes the infrastructure validated for an
Oracle Database 11gR2 overlay on a VSPEX infrastructure. To validate this solution,
we1:
•
Deployed all Oracle Database 11gR2 servers as virtual machines on VMware
vSphere 5.1
•
Used the VSPEX sizing tool for Oracle Database 11gR2 to determine the
number of, and the detailed compute resources for, each Oracle Database
11gR2 server role. Figure 1 displays the three sizing options (small, medium,
and large) for this solution. Use the sizing tools provided with this solution to
size your customer’s environment and choose the option that best suits your
customer.
•
Determined the recommended storage layout for Oracle Database 11gR2 and
the virtual infrastructure pool in the VNX series storage arrays (using the
VSPEX sizing tool).
1
In this Implementation Guide, "we" refers to the EMC Solutions engineering team that
validated the solution.
22
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Solution Overview
Figure 1.
Architecture of the validated infrastructure
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
23
Solution Overview
Key components
Introduction
EMC VSPEX
This section provides an overview of the key technologies used in this solution,
including:
•
EMC VSPEX
•
Oracle Database 11g
•
VMware vSphere 5.1
•
VMware vSphere HA
•
vSphere Distributed Resource Scheduler
•
VMware vSphere PowerCLI
•
Red Hat Enterprise Linux 6.3
•
Reference virtual machine
•
Oracle Small/Medium/Large Model
•
EMC VNX® series
•
Unisphere
•
EMC Virtual Storage Integrator (VSI)
•
EMC Avamar 6.1
•
EMC DataDomain Deduplication Storage
EMC has joined forces with the industry’s leading providers of IT infrastructure to
create a complete virtualization solution that accelerates the deployment of private
cloud technologies. Built with best-of-breed technologies, VSPEX enables faster
deployment, more simplicity, greater choice, higher efficiency, and lower risk.
VSPEX provides a virtual infrastructure for customers looking to gain simplicity that is
characteristic of truly converged infrastructures while at the same time gaining more
choice in individual components of the stack. VSPEX solutions are proven by EMC and
packaged and sold exclusively by EMC channel partners. VSPEX provides channel
partners more opportunity, faster sales cycle, and end-to-end enablement. By
working even more closely together, EMC and its channel partners can now deliver an
infrastructure that accelerates the journey to the cloud for even more customers.
As shown in Figure 2, a VSPEX Proven Infrastructure is a modular, virtualized system
validated by EMC and delivered by EMC partners. VSPEX includes a virtualization
layer, server, network, and storage, designed by EMC to deliver reliable and
predictable performance.
VSPEX provides the flexibility to choose the best network, server, and virtualization
technologies that fit a customer’s environment to create a complete virtualization
solution.
24
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Solution Overview
Figure 2.
VSPEX Proven Infrastructure
Reference virtual machine
To simplify the virtual infrastructure discussion, the VSPEX solution has defined a
representative customer reference workload.
For VSPEX solutions, the reference virtual machine is defined as a measure unit of a
single virtual machine to qualify the compute resources in the VSPEX virtual
infrastructure. Table 5 lists the characteristics of this virtual machine.
Table 5.
Reference virtual machine characteristics
Characteristic
Value
Virtual processors per virtual machine
1
RAM per virtual machine
2 GB
Available storage capacity per virtual machine
100 GB
I/O operations per second (IOPS)
per virtual machine
25
I/O pattern
Random
I/O read/write ratio
2:1
VSPEX for virtualized Oracle 11g sizing model
Scale-up testing formed part of the validation process. Using a standard computesizing model for Oracle simplified and standardized the validation testing. During the
testing, we defined three general computer model thresholds based on general
concurrent user loads. This enabled us to identify the configuration required to run a
TPC-C like OLTP workload with a 60:40 read/write ratio yielding acceptable response
times for those thresholds.
Table 6 shows how we mapped the Oracle sizing model to the VSPEX reference virtual
machine.
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
25
Solution Overview
Table 6.
Mapping the Oracle sizing model to the VSPEX reference virtual machine
Oracle model
Resources
Equivalent
reference
virtual
machine
Small—virtual machine for
up to 150 users
Compute requirement
4
•
2 vCPU
•
8 GB memory
Storage Requirement (OS & Oracle binaries)
• 100 GB
•
Medium—virtual machine
for up to 250 users
25 IOPS
Compute requirement
• 4 vCPU
•
8
16 GB of Memory
Storage requirement (OS & Oracle binaries)
Large—virtual machine for
more than 250 Users
•
100 GB
•
25 IOPS
Compute requirement
• 8 vCPU
•
16
32 GB of memory
Storage requirement (OS & Oracle binaries)
• 100 GB
•
25 IOPS
We calculated the database storage I/O thresholds and capacity separately from
those required by the VSPEX reference virtual machine.
Oracle Database
11g
Oracle Database 11gR2 is available in a variety of editions tailored to meet the
business and IT needs of an organization. In this solution we will be considering:
•
Oracle Database 11g Release 2 Standard Edition (SE)
•
Oracle Database 11g Release 2 Enterprise Edition (EE)
Oracle SE is an affordable, full-featured data management solution that is ideal for all
companies. It is available on single or clustered servers, has a maximum capacity of
four processor sockets, and is licensed by the socket regardless of core count. The SE
license includes Oracle Real Application Clusters (RAC) as a standard feature with no
additional cost.
26
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Solution Overview
Oracle Database 11g R2 EE delivers industry-leading performance, scalability,
security, and reliability on a choice of clustered or single servers running Windows,
Linux, or UNIX. It supports advanced features, either included or as extra-cost
options, that are not available with Oracle Database 11gR2 SE. These include security
features such as Virtual Private Database, and data warehousing options such as
Partitioning and Advanced Analytics. Oracle Database 11g Release 2 EE extends the
processor-licensing model for multi-core processors and is priced using the following
formula:
(number of processors) x (number of cores) x (Oracle processor core factor)
For example, two 10-core Intel® Xeon® Processors E7-2870 (Oracle processor core
factor of 0.5) are licensed as follows
•
Oracle Database 11g Release 2 SE: Two processor socket SE licenses
•
Oracle Database 11g Release 2 EE: 2 x 10 x 0.5 = 10 EE licenses
The Oracle Database 11g R2 edition can affect the licensing cost and the size and
number of VMware ESXi clusters you can configure. This affects how you place and
manage the virtual machines. Refer to the document entitled Design Guide for EMC
VSPEX for Virtualized Oracle Database 11g OLTP, in the DRS Host Affinity and Oracle
processor licensing section, for more information on virtualization and Oracle
processor licensing.
VMware vSphere
5.1
VMware vSphere 5.1 abstracts applications and information from the complexity of
underlying infrastructure through comprehensive virtualization of server, storage, and
networking hardware. This transformation creates fully functional virtual machines
that run isolated and encapsulated operating systems and applications just like
physical computers. This virtualization of hardware resources enables efficiencies
through consolidation of multiple applications on fewer physical servers.
VMware vSphere
HA
VMware vSphere High Availability (HA) provides easy-to-use, cost-effective high
availability for applications running in virtual machines. In the event of physical
server failure, affected virtual machines are automatically restarted on other
production servers with spare capacity.
HA enables you to create a cluster from multiple ESXi servers, allowing you to protect
virtual machines so that if of one of the hosts in the cluster fails, the impacted virtual
machines are automatically restarted on other ESXi hosts within that same VMware
vSphere cluster.
VMware vSphere
Distributed
Resource
Scheduler
VMware vSphere Distributed Resource Scheduler (DRS) is an infrastructure service
run by VMware vCenter Server (vCenter). DRS aggregates ESXi host resources into
clusters and automatically distributes these resources to virtual machines by
monitoring utilization and continuously optimizing virtual machine distribution
across ESXi hosts. DRS can also use vMotion and storage vMotion (SvMotion) to
ensure that virtual machines have access to rebalance resource capacity and make
room for larger virtual machines. VMware recommends enabling DRS to achieve
higher consolidation ratios.
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
27
Solution Overview
VMware vSphere
PowerCLI
VMware vSphere PowerCLI provides a Windows PowerShell interface for the users of
vSphere 5.x and above and VMware Infrastructure 4.x and above. VMware vSphere
PowerCLI is a powerful command-line tool that lets you automate all aspects of
vSphere management, including network, storage, virtual machine, guest OS, and
more. PowerCLI is distributed as a Windows PowerShell snapin, and includes 330
PowerShell cmdlets for managing and automating vSphere and vCloud, along with
documentation and samples.
Red Hat Enterprise
Linux 6.3
Red Hat Enterprise Linux 6.3 includes enhancements and new capabilities that
provide rich functionality, especially the developer tools, virtualization features,
security, scalability, file systems, and storage. Red Hat Enterprise Linux is a versatile
platform for x86 and x86-64 that can be deployed on physical systems, as a guest on
the major hypervisors, or in the cloud. It supports all leading hardware architectures
with compatibility across releases.
EMC VNX series
The EMC VNX family is optimized for virtual applications delivering industry-leading
innovation and enterprise capabilities for file, block, and object storage in a scalable,
easy-to-use solution. This next-generation storage platform combines powerful and
flexible hardware with advanced efficiency, management, and protection software to
meet the demanding needs of today’s enterprises.
The VNX Series is powered by the Intel® Xeon® processor for intelligent storage that
automatically and efficiently scales in performance, while ensuring data integrity and
security.
VNX customer benefits
•
Next-generation unified storage, optimized for virtualized applications
•
Capacity optimization features including compression, deduplication, thin
provisioning, and application-centric copies
•
High availability, designed to deliver five 9s availability
•
Automated tiering with FAST™ VP (Fully Automated Storage Tiering for Virtual
Pools) and FAST Cache that can be optimized for the highest system
performance and lowest storage cost simultaneously
•
Simplified management with EMC Unisphere® for a single management
interface for all NAS, SAN, and replication needs
•
Up to three times improvement in performance with the latest Intel Xeon
multicore processor technology, optimized for Flash
VNX Software suites available
28
•
FAST Suite: Automatically optimizes for the highest system performance and
the lowest storage cost simultaneously.
•
Local Protection Suite: Practices safe data protection and repurposing.
•
Remote Protection Suite: Protects data against localized failures, outages,
and disasters.
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Solution Overview
•
Application Protection Suite: Automates application copies and proves
compliance.
•
Security and Compliance Suite: Keeps data safe from changes, deletions, and
malicious activity.
Software packs available
EMC Virtual
Storage Integrator
•
Total Efficiency Pack: Includes all five software suites.
•
Total Protection Pack: Includes local, remote, and application protection
suites.
The EMC VSI for VMware vSphere Unified Storage Management feature can provision
Network File System (NFS) datastores on NAS storage and Virtual Machine File System
(VMFS) datastores and Raw Device Mapping (RDM) volumes on block storage, and
perform array-based compression and array-based cloning of virtual machines in NFS
datastores. The cloning functions include Full Clones (copies) and Fast Clones
(snaps) of Virtual Machine Disk (VMDK) files. VMware administrators can use the
feature to manage NAS and block storage in VMware environments by using the
existing vSphere Client user interface.
VMware administrators can use the tool to:
•
Provision new NFS, VMFS, and RDM storage
•
Extend existing NFS and VMFS storage
•
Compress virtual machines in NFS datastores
•
Clone virtual machines in NFS datastores
•
Provide Fast Clones support – Limited to the same file system
EMC Unisphere
EMC Unisphere is the central management platform for the VNX series, providing a
single, combined view of file and block systems, with all features and functions
available through a common interface. Unisphere is optimized for virtual applications
and provides industry-leading VMware integration, automatically discovering virtual
machines and ESX servers and providing end-to-end, virtual-to-physical mapping.
Unisphere also simplifies configuration of FAST Cache and FAST VP on VNX platforms.
EMC Avamar
If you decide to implement a backup solution, EMC recommends Avamar. Avamar
deduplication backup software and system performs variable-length deduplication at
the client, so that backup data is reduced before it moves across networks (LAN or
WAN). Avamar identifies duplicate data segments and sends only unique segments
across the network to the backup appliance. This means shorter backup windows,
less backup storage consumed, and maximum leverage of available bandwidth.
Avamar provides:
•
Flexible deployment options. Avamar offers flexibility in solution
deployments, depending on the specific use case and recovery requirements.
Avamar is a turnkey backup and recovery solution that integrates with EMCcertified hardware for streamlined deployment.
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
29
Solution Overview
EMC Data Domain
30
•
Scalability, high availability, and reliability. Avamar uses a scalable grid
architecture, enabling linear performance and storage increases by simply
adding storage nodes.
•
Manageability and support. You can securely access Avamar systems through
existing network links and integrate them with management frameworks to
use SNMP for remote access.
EMC Data Domain deduplication storage system deduplicates data inline so that the
data lands on disk already deduplicated, which requires less disk space than the
original dataset. With Data Domain, you can retain backup and archive data on site
longer to quickly and reliably restore data from disk. The Data Domain software suite
includes the following options:
•
EMC Data Domain Replication
•
Virtual Tape Library (VTL)
•
Data Domain Boost
•
Retention Lock
•
Encryption
•
Extended Retention
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Chapter 4
Solution
Implementation
This chapter presents the following topics:
Overview ................................................................................................... 32
Setting up the physical components........................................................... 32
Implementing the network ......................................................................... 33
Implementing the storage array .................................................................. 35
Implementing ESXi and vCenter.................................................................. 42
Preparing the Oracle Database 11gR2 virtual machine template .................. 43
Deploying the Oracle Database 11gR2 virtual machine ............................... 49
Implementing backup and recovery ............................................................ 58
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
31
Solution Implementation
Overview
This chapter discusses the steps required to install, configure, and implement the
VSPEX Proven Infrastructure environment, which involves:
•
Preparing the components and their physical setup.
•
Implementing the network, including the configuration steps for physical and
virtual switches in line with vendor guidelines and The vSphere Networking
Guide.
•
Configuring the VNX storage array and optional FASTCache feature.
•
Building the VMware vSphere environment with ESXi server and vCenter
implementation.
•
Preparing a virtual machine template for rapid deployment of Oracle Database
11gR2 pre-requisites.
•
Quickly and simply deploying an Oracle Database 11gR2 virtual machine
using PowerCLI.
•
Implementing backup and recovery.
Setting up the physical components
This section includes information about the how to prepare the solution’s physical
components. After you complete the steps listed in Table 7, the new hardware
components will be racked, cabled, powered, and ready for network connection.
Note
If you already have a VSPEX Proven Infrastructure environment, you can skip
this section.
Table 7.
Tasks for physical setup
Task
Description
Reference
Prepare network
switches
Install switches in the rack
and connect them to power.
Your vendor’s installation
guide
Prepare servers
Install the servers in the rack
and connect them to power.
Your vendor’s installation
guide
Prepare VNX
Install the VNX in the rack and
connect it to power.
EMC VNX System Installation
Guide
For details of the physical setup, refer to Reference Architecture: EMC VSPEX Private
Cloud: VMware vSphere 5.1 for up to 500 Virtual Machines
32
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Solution Implementation
Implementing the network
This section documents requirements for the network infrastructure needed to
support this architecture. Table 8 provides a summary of the tasks for switch and
network configuration and references for further information.
Note
If you already have a VSPEX Proven Infrastructure environment, you can skip
this section.
Table 8.
Tasks for switch and network configuration
Task
Description
Reference
Network
Configuration
Configure storage network and host
infrastructure networking as
specified in the solution reference
architecture.
Refer to the appropriate
document in Support
resources.
Complete
network cabling
Connect:
N/A
Switch interconnect ports
VNX ports
ESXi NiC/CNA ports
Configure VLAN
Configure private and public VLANs
as required.
Vendor’s switch
configuration guide
Configure
VMare vSphere
networking
Create virtual switches for
separation of application,
management, and storage networks.
vSphere Networking Guide
For details of network implementation, refer to the appropriate document in Support
resources.
Figure 3 shows the configured vSphere network for this implementation. As
highlighted, redundant 10GbE network adapters have been dedicated for storage
network traffic.
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
33
Solution Implementation
Figure 3.
34
Storage network configuration
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Solution Implementation
Implementing the storage array
Overview
This section describes how to configure the VNX storage array. In this solution, the
VNX provides Network File System (NFS) or Virtual Machine File System (VMFS) data
storage for the VMware host.
Table 9.
Tasks for storage configuration
Task
Description
Set up initial VNX
configuration
Configure the IP address
information and other key
parameters on the VNX.
Provision storage
for NFS mounts
and NFS
datastores
Create NFS file systems that
will be mounted as NFS
datastores
Reference
•
VNX System Installation
Guide
•
VNX Series
Configuration
Worksheet
Figure 4 depicts the high-level architecture of this solution.
Figure 4.
Setting up initial
VNX configuration
Oracle Database 11gR2 storage elements
Ensure that network interfaces, IP address information, and other key parameters
such as DNS and NTP are configured on the VNX before provisioning the storage.
For more information on how to configure the VNX platform, refer to the appropriate
document in Support resources.
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
35
Solution Implementation
Before you provision the storage for the NFS datastore, follow the recommendations
Provisioning
storage for the NFS and VSPEX Sizing Tool proposals introduced in the companion Design Guide.
datastore
Before you start, refer to the Reference Architecture titled EMC VSPEX Private Cloud:
VMware vSphere 5.1 for up to 500 Virtual Machines to configure NFS file system and
provision storage for virtual machine operating system on VNX.
Table 10 shows an example of how to lay out the storage for the Oracle Database
11gR2 data in VNX. For more information about the storage layout recommendations
and design, refer to the Design Guide.
Table 10.
VNX storage layouts for Oracle Database
Storage pool name
RAID type
Disk type
No. of disks
Oracle Data pool
RAID 5 (4+1)
15,000 rpm SAS disks
30
Oracle FRA pool
RAID 6 (6+2)
7,200 rpm SAS disks
8
Oracle Redo pool
RAID 5 (4+1)
15,000 rpm SAS disks
5
Present the NFS filesystem for the virtual machine datastore to all ESXi hosts, using
either EMC Unisphere or the VSI plugin for VMware vSphere client. Figure 6 displays
how to configure the NFS datastores that are used to store virtual servers on the VNX
array using VSI.
Configuring
Unified Storage
Management
The EMC Unified Storage Management plugin for VMware vSphere provides an
interface for creating and managing VMFS datastores for VMware ESXi host directly
from the vSphere Client interface. Figure 5 shows the configuration screen accessed
through Home >Solutions and applications >EMC from the vSphere Client, which is
accessible after running the installer on your management host.
Figure 5.
Configuring EMC Unified Storage Management in vSphere Client
Unified storage management enables the vSphere administrator to provision NFS
datastores and mount them directly to the ESXi cluster nodes from vSphere Client, as
shown in Figure 6.
36
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Solution Implementation
Figure 6.
Configuring NFS Storage on VNX for Oracle VMs
To configure NFS network settings, storage pools for file, file systems, and NFS
exports on the VNX array in EMC Unisphere, perform the following steps:
1.
In Unisphere, select the VNX array for this solution.
2.
Select Settings Network Settings for File
3.
Configure the IP address for network ports used for NFS. For detailed steps
refer to EMC Procedure Generator for VNX.
4.
Select Storage Storage Configuration
Figure 7.
5.
Configuring VNX storage pools
Select the Storage Pools for File tab and create the additional storage pools
in the VNX for Oracle Database files, as shown in Figure 7. Refer to Table 10
for detailed information.
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
37
Solution Implementation
6.
Figure 8.
7.
38
Create the required file systems. Figure 8 shows the Create File System
Wizard from Unisphere.
Unisphere Create File System Wizard
Create the NFS exports for the database components that the Oracle nfs
client will mount on the database virtual machines. Figure 9 shows the
Unisphere Create NFS Export Wizard. Root and Access permissions are
granted to the NIC.
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Solution Implementation
Figure 9.
EMC Unisphere Create NFS Export Wizard
Complete the following steps in EMC Unisphere to configure the NFS file systems that
are used to store virtual servers on the VNX array:
Configuring FAST
Cache (optional)
1.
Create the storage pools in VNX. Refer to Table 10.
2.
If you are using VNX, provision the LUNs in these pools. For detailed best
practices on the number of the LUNs to be created in the storage pool for
the NFS datastore, refer to the EMC VSPEX for Virtualized Oracle Database
11g OLTP Design Guide.
3.
Create the file system from the storage pool to present to the ESXi servers as
the NFS datastore in the VNX array.
4.
Export the file systems using NFS and give root access to the ESXi servers.
Enabling FAST Cache is a transparent operation to Oracle Database 11gR2 and no
reconfiguration or downtime is necessary. To make the best use of either of the FAST
technologies, EMC recommends that you first enable FAST Cache on the Oracle
Database 11gR2 storage pool. For more details, refer to the EMC VSPEX for Virtualized
Oracle Database 11g OLTP Design Guide.
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
39
Solution Implementation
To create and configure FAST Cache, use the following steps:
1.
Refer to the Reference Architecture paper entitled EMC VSPEX Private Cloud:
VMware vSphere 5.1 for up to 500 Virtual Machines for detailed steps about
how to create FAST Cache.
2.
In Unisphere, after creating the FAST Cache, click the Storage tab and select
Storage Pool. Select Data Pool, and click Properties.
3.
Select the Advanced tab in Storage Pool Properties and click Enabled to
enable FAST Cache, as shown in Figure 10.
4.
Click OK to complete the configuration.
Note
The FAST Cache feature on the VNX series array does not cause an instant
performance improvement. The system must collect data about access
patterns and promote frequently used information into the cache. This
process can take a few hours during which the performance of the array
steadily improves.
Figure 10.
40
Enabling FAST Cache in the Storage Pool Properties dialog
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Solution Implementation
Example storage
layout
Figure 11 shows an example storage layout for VNX with FAST Cache enabled on the
Oracle Database 11gR2 pool.
Figure 11.
Example storage layout for VNX
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
41
Solution Implementation
Implementing ESXi and vCenter
This section documents the requirements for the installation and configuration of the
ESXi hosts, vCenter configuration, and infrastructure servers required to support the
architecture. Table 11 describes the tasks that must be completed.
Table 11.
Steps for installing and configuring ESXi and vCenter
Task
Description
Reference documents
Install ESXi
Install the ESXi 5.1 hypervisor on the
physical servers that are being deployed for
the solution.
vSphere Installation and Setup
Configure and deploy
vCenter Server
Install VMware vCenter Server 5.1
vCenter Server and Host
Management Guide
Installing vCenter Server 5.1 best
practices
Configure ESXi
networking
Configure ESXi networking including NIC
trunking, VMkernel port, virtual machine port
groups, and Jumbo Frames.
vSphere Networking
Connect VMware
datastores
Connect the VMware datastore to the ESXi
hosts deployed for the solution.
vSphere Storage
Create a virtual data
center
Create a virtual data center.
vCenter Server and Host
Management
Apply vSphere license
keys
Type the vSphere license keys in the vCenter
licensing menu.
vSphere Installation and Setup
Add ESXi hosts
Connect vCenter to ESXi hosts.
vCenter Server and Host
Management
Configure vSphere
clustering
Create a vSphere cluster
vSphere Resource Management
Perform array ESXi
host discovery
Perform ESXi host discovery within the
Unisphere console.
Using EMC VNX Storage with
VMware vSphere–TechBook
Enable VMware High
Availability (HA), DRS,
and vMotion
functionality
Once you enable DRS, you may need to
consider the use of VMware vSphere DRS
Affinity and Anti-Affinity must rules for
specific groups of virtual machines, for
example, Oracle VMs may be restricted to a
particular set of VMs due to licensing
Automating High Availability (HA)
Services with VMware HA –VMware
Technical note
Install EMC VNX USM
Install the EMC VNX USM on the
administration console.
EMC VSI for VMware vSphere:
Unified Storage Management—
Product Guide
Install the EMC VSI
plug-in
Install the EMC VSI plug-in on the
administration console.
EMC VSI for VMware vSphere:
Unified Storage Management—
Product Guide
For details of ESXi and vCenter, refer to the Reference Architecture titled EMC VSPEX
Private Cloud: VMware vSphere 5.1 for up to 500 Virtual Machines.
42
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Solution Implementation
Preparing the Oracle Database 11gR2 virtual machine template
Overview
VMware provides the ability to create a template that you can use as a master copy to
quickly create and provision virtual machines. By using a template, you can install a
guest OS and apply it to a virtual machine with application users and software
configured and ready for use with minimal administrative intervention. This
minimizes deployment time and avoids repetitive installation and configuration tasks
for each virtual machine that is required.
Customization specifications, maintained in vCenter, further simplify the rollout of
virtual machines. A deployment wizard, automation tool, or script can use these
templates to automatically pre-create or amend server settings (such as server name,
time zone, and network configuration) prior to building the new virtual machine.
This section documents the preparation of a VMware virtual machine template to
enable rapid deployment of new Oracle Database 11gR2 environments.
To configure the template with the requirements and prerequisites for the Oracle
software install, we will complete the following steps:
1.
Creating the
Oracle Database
11gR2 virtual
machine
Create a virtual machine with the following resources:
2 vCPUs
8 GB of vRAM
100 GB virtual disk
2.
Install the operating system and RPM packages
3.
Configure the system and kernel parameters
4.
Create the OS groups and users
5.
Install and detach the Oracle binaries ready for clone
6.
Convert the virtual machine to a VMware template
Create a virtual machine on the ESXi server with the customer’s guest OS
configuration. Table 12 lists an example specification used when building the virtual
machine template.
Table 12.
Virtual machine template specification
Part
Description
CPU
2 vCPUs
Memory
8 GB
Virtual Disk
100GB
•
8GB swap partition
•
92GB bootable root partition
Operating system
Redhat Enterprise Linux Server release 6.3
Kernel
2.6.32
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
43
Solution Implementation
Part
Description
Network interfaces
Eth0: public/management IP network
Eth1: Storage Network
Completing the
Oracle install prerequisites
OS users
Username: oracle
OS groups
Group: oinstall, dba
Oracle 11gR2 Binaries
11.2.0.4 (Oracle 11.2.0.3 installed and patched)
RPM packages installed
Refer to the Oracle Installation Guide for prerequisites
System configuration
See the Oracle Installation Guide for prerequisites.
This section describes how to install and configure the operating system and Oracle
prerequisites in the virtual machine prior to converting it to a template.
Table 13.
Oracle install prerequisites
Task
Description
Reference
Install the guest OS
Install the Red Hat Enterprise Linux 6.3
64-bit operating system and configure
the network interfaces.
Red Hat Enterprise Linux 6
Installation Guide
Install the required
RPM packages
Install or update required RPM packages
as listed in Table 14.
Oracle Database Installation
Guide 11g Release 2 (11.2) for
Linux
Set the kernel
parameters
Configure the kernel with Oracle-specific
settings for semaphores, virtual and
shared memory, network, and
asynchronous I/O.
Oracle Database Installation
Guide 11g Release 2 (11.2) for
Linux
Set the resource
limits for the Oracle
user
Check and adjust the resource limits for
the Oracle software installation users
Oracle Database Installation
Guide 11g Release 2 (11.2) for
Linux
Create the required
software directories
Create the required directories for the
Oracle software
Oracle Database Installation
Guide 11g Release 2 (11.2) for
Linux
Configure Huge
Pages
HugePages reduce the number of
memory pages and reduces virtual
memory management overhead.
Oracle Database 11g Release
2 on Red Hat Enterprise Linux
6 Deployment
Recommendations
Table 14 lists the Red Hat Enterprise Linux 6 RPM packages required to install Oracle
database 11gR2.
44
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Solution Implementation
Table 14.
Red Hat Enterprise Linux 6 RPM packages for Oracle Database 11gR2
binutils-2.20.51.0.2-5.11.el6 (x86_64)
libstdc++-4.4.4-13.el6.i686
compat-libcap1-1.10-1 (x86_64)
libstdc++-devel-4.4.4-13.el6 (x86_64)
compat-libstdc++-33-3.2.3-69.el6 (x86_64)
libstdc++-devel-4.4.4-13.el6.i686
compat-libstdc++-33-3.2.3-69.el6.i686
libaio-0.3.107-10.el6 (x86_64)
gcc-4.4.4-13.el6 (x86_64)
libaio-0.3.107-10.el6.i686
gcc-c++-4.4.4-13.el6 (x86_64)
libaio-devel-0.3.107-10.el6 (x86_64)
glibc-2.12-1.7.el6 (i686)
libaio-devel-0.3.107-10.el6.i686
glibc-2.12-1.7.el6 (x86_64)
make-3.81-19.el6
glibc-devel-2.12-1.7.el6 (x86_64)10
sysstat-9.0.4-11.el6 (x86_64)
glibc-devel-2.12-1.7.el6.i686
On Oracle Linux 6, and Red Hat Enterprise Linux
6:
ksh
unixODBC-2.2.14-11.el6 (x86_64) or later
libgcc-4.4.4-13.el6 (i686)
unixODBC-2.2.14-11.el6.i686 or later
libgcc-4.4.4-13.el6 (x86_64)
unixODBC-devel-2.2.14-11.el6 (x86_64) or later
libstdc++-4.4.4-13.el6 (x86_64)
unixODBC-devel-2.2.14-11.el6.i686 or later
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
45
Solution Implementation
Installing the
Oracle Database
11gR2 binaries
and patches
As advised in the Design Guide, decide whether to install Oracle SE or Oracle EE
Database 11gR2 software. Both editions use the same installation files.
Table 15 includes the steps required to install the Oracle Database binaries and
patches.
Table 15.
Installing the Oracle Database binaries and patches
Task
Description
Reference
Install the Oracle
binaries
Run the Oracle Universal installer and
choose to “Install choose database
software only” as shown in Figure 12.
Oracle Database Installation
Guide 11g Release 2 (11.2) for
Linux
Install the latest
PSU and CPU
Install the latest database Patch Set
Update and instructions available from
the My Oracle Support website. A login
ID and password as well as a support
contract are required.
See Critical Patch Updates,
Security Alerts and Third Party
Bulletin
Enable DNFS
Oracle Database is not shipped with
Direct NFS enabled by default. To enable
Direct NFS, refer to Error! Reference
source not found..
Oracle Database Installation
Guide 11g Release 2 (11.2) for
Linux
Detach the Oracle
Home directory?
To remove the Oracle Database 11gR2
software from the Oracle Universal
Installer inventory run the script
$ORACLE_HOME/oui/bin/detachhome.sh
Oracle Universal Installer and
OPatch User's Guide 11g
Release 2 (11.2) for Windows
and UNIX Chapter 2
Create the
ORACLE_HOME
cloning scripts
To automate the cloning Oracle software
process, create the scripts
/root/root_clone.sh shown in Figure 24
And $OHOME/oui/bin/cloneHome.sh
shown in 0
Oracle Universal Installer and
OPatch User's Guide 11g
Release 2 (11.2) for Windows
and UNIX Chapter 6
As well as the official installation guide, Oracle Database Installation Guide 11g
Release 2 (11.2) for Linux, Red Hat has produced a helpful whitepaper titled Oracle
Database 11g Release 2 on Red Hat Enterprise Linux 6 Deployment
Recommendations which offers guidance specific to the Red Hat operating system.
Figure 12.
46
Oracle Universal Installer – Install choose database software only
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Solution Implementation
Oracle Database is not shipped with Direct NFS enabled by default. To enable Direct NFS,
complete the following steps:
Change the directory to $ORACLE_HOME/rdbms/lib.
cp libodm11.so libodm11.so_stub
ln -s libnfsodm11.so libodm11.so
Enter the following command:
make -f ins_rdbms.mk dnfs_on
Figure 13.
Enable Oracle Direct NFS client
[root@vspex-ora02 ~]# cat /root/root_clone.sh
#!/bin/bash
OHOME=/u01/app/oracle/11.2.0.3
echo "Cloning Oracle Binaries as Oracle at $(date +%Y%m%d%H%M)"
su - oracle -c "$OHOME/oui/bin/cloneHome.sh"
echo "Clone Finished at $(date +%Y%m%d%H%M)"
echo "Running root.sh at $(date +%Y%m%d%H%M)"
$OHOME/root.sh
echo "Install Finished at $(date +%Y%m%d%H%M)"
Figure 14.
Script root_clone.sh to be created and owned by root
[root@vspex-ora02 ~]# cat $OHOME/oui/bin/cloneHome.sh
#!/bin/sh
OHOME=/u01/app/oracle/11.2.0.3
OHOMENAME=OraDb11g_home1
OBASE=/u01/app/oracle
CUR_DIR=`pwd`
cd $OHOME/oui/bin
${OHOME}/clone/bin/clone.pl ORACLE_HOME="${OHOME}"
ORACLE_HOME_NAME="${OHOMENAME}" ORACLE_BASE="${OBASE}" -ignoreSysPrereqs
cd $CUR_DIR
Figure 15.
Script cloneHome.sh to be created and owned by Oracle
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
47
Solution Implementation
Preparing the
Oracle Database
11gR2 virtual
machine template
Convert the virtual machine to a template called vspx-Orac11203EE-2vcpu-8GB-tmplt
as shown in Figure 16.
Figure 16.
Customizing the
VMware template
Create a custom specification, Oracle_2vCPU_8GB_11203_EE_spec, using the
vCenter client. This requests the new virtual machine name and sets place holder IP
addresses for the network interfaces as shown in Figure 17.
Figure 17.
48
Details for template, vspx-Orac11203EE-2vcpu-8GB-tmplt
Details custom specification, Oracle_2vCPU_8GB_11203_EE_spec
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Solution Implementation
Deploying the Oracle Database 11gR2 virtual machine
Overview
Table 16 shows the mapping of the compute requirements for the Oracle Database
11gR2 virtual machines.
Table 16.
Oracle Database 11gR2 reference virtual machine model
Virtual Machine Compute Requirement
for Oracle Database 11gr2
Deploying new
virtual machines
Memory in GB
Total reference virtual
machines
Small
8
4
Medium
16
8
Large
32
16
There are several methods available to deploy a virtual machine from a template
using custom specifications:
•
Manually in vCenter by choosing Deploy Virtual Machine from this Template
•
Choosing Customize using an existing customization specification
•
By creating a workflow with an orchestration tool such as vCenter
Orchestrator
•
By creating a custom PowerCLI script that can be edited or parameterized
The example PowerCLI script in Figure 18:
•
Assigns variables for: template name, custom specification name, virtual
machine name, ESXi host name, NFS or VMFS datastore name, virtual
machine public IP address, virtual machine storage IP address, and the
number of vCPUs, and vRAM size in MB.
•
Assigns the virtual machine public and storage IP addresses to the two vNICs
included in the custom specification in Figure 17.
•
Creates the new virtual machine using the template and custom specification.
•
Changes the CPU and VRAM assignment for the virtual machine.
•
Starts the virtual machine.
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
49
Solution Implementation
# Connect to Vcenter
Connect-VIServer -server 192.168.28.33 -user Administrator -password <Password>
{
#Assign Variables for template, specification, VM, ESXi Host , Datastore
# IP addresses for the Public and Storage network interfaces
# and the vCPU and vRAM assigned to the Virtual Machine
$Template="vspx-Orac11203EE-2vcpu-8GB-tmplt"
$Customization="Oracle_2vCPU_8GB_11203_EE_spec"
$VMName="<Virtual Machine Name>"
$VMHost=" <ESXi Host Name>"
$Datastore="<Datastore Name>"
$IPAdressPublic=”< Public IP Address>”
$IPAdressStorage=”<Storage IP Address>”
$NumVCPU=”4 ”
$vRAMMB=”16384”
# Assign IP addresses to the two vNICs included in the custom specification
Get-OSCustomizationSpec $Customization | Get-OSCustomizationNicMapping | where {
$_.Position -eq '1'} | Set-OSCustomizationNicMapping -IpMode UseStaticIP -IpAddress
$IPAdressPublic -Subnetmask 255.255.255.0 -DefaultGateway 192.168.28.1
Get-OSCustomizationSpec $Customization | Get-OSCustomizationNicMapping | where {
$_.Position -eq '2'} | Set-OSCustomizationNicMapping -IpMode UseStaticIP -IpAddress
$IPAdressStorage -Subnetmask 255.255.255.0 -DefaultGateway 192.168.40.1
# Create the new VM using the Template and custom spec
New-VM -Name $VMName -OSCustomizationSpec $Customization -Template $Template VMHost $VMHost -Datastore $Datastore
# Change the CPU and memory assignment for the VM
Set-VM $VMName -NumCPU $NumVCPU -MemoryMB $vRAMMB -Confirm:$false
# Start up the VM
Start-VM -VM $VMName -Confirm:$false
}
Figure 18.
Example PowerCLI script to deploy a virtual machine from template
Edit and run the example PowerCLI script, or edit it to read a parameter file and
deploy multiple virtual machines.
Cloning the Oracle
binaries ona new
virtual machine
50
Run the clone process and reattach the Oracle home directory to edit and run a
second PowerCLI to complete the virtual machine deployment. This script, shown in
Figure 19, calls the root_clone.sh from Figure 14 which does this.
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Solution Implementation
{
$script="/root/root_clone.sh"
$huser="root"
$hpwd="<ROOT Password>"
$VMName="<Virtual Machine Name>"
Invoke-VMScript -ScriptText $script -VM $VMName -GuestUser $huser -GuestPassword
$hpwd -ScriptType bash
}
Figure 19.
Determining the
database
prerequisites
Sample PowerCLI script to clone and reattach the ORACLE_HOME
Use the VSPEX Oracle Qualification Worksheet and the Design Guide to size the
virtualization infrastructure and the overall NFS filesystems in which to store the
Oracle database. These documents also provide guidance on the Oracle initialization
parameters required by the database.
Table 16 shows the sizing of the virtual machine.
Determine the size of the individual datafiles based on the schema design and
tablespace placement of tables and indexes. Datafile placement is covered in the
section Configuring the database file layout for NFS.
For guidance on configuring the initialization parameters, see the Design Guide and
the Configuring the initialization parameters section of this document.
Before creating the database, you must create a listener process to receive the
incoming client connection requests using the Oracle Net Configuration Assistant
(NETCA) utility, as shown in Figure 20.
Figure 20.
Oracle Net Configuration Assistant utility
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
51
Solution Implementation
Consider these items before creating an Oracle 11gR2 database:
Configuring the
database file
layout for NFS
•
Select the required database character set to meet your localization and
globalization needs.
•
Consider the time zones your database must support.
•
Ensure the SYSTEM and SYSAUX tablespace are created as Locally Managed
Tablespaces, appropriately sized, and have AUTOEXTEND enabled.
•
Plan to use an undo tablespace to manage your undo data.
The components of the Oracle Database: datafiles, online redo log files, temp files,
control files, and FRA files reside on NFS file systems. These file systems are designed
(in terms of the RAID level and number of disks used) to be appropriate for each type
of file.
Table 17 lists each file type and indicates where they reside.
Table 17.
Database file layout for NFS
Content
Location
Datafiles
/u02/app/oracle/oradata/${ORACLE_SID}
Temp files
/u04/app/oracle/oradata/${ORACLE_SID}
Online redo logs, control files
/u03/app/oracle/oradata/${ORACLE_SID}
FRA files
/u05/app/oracle/oradata/${ORACLE_SID}
This solution uses the Oracle Direct NFS client, which offers improved performance
over the standard NFS client supplied with the operating system.
By default, Direct NFS serves the mount entries found in the file /etc/mtab, which
lists all currently mounted filesystems along with their initialization options. This
should match the system configuration file /etc/fstab where the mount options are
defined. Figure 21 provides a sample /etc/mtab with NFS mount options:
NFSServer1:/vspex_ora2_data /u02/app/oracle/oradata/VSPEX1 nfs
rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,actimeo=0,vers=3,t
imeo=600,addr=192.168.40.11 0 0
NFSServer2:/vspex_ora2_redo /u03/app/oracle/oradata/VSPEX1 nfs
rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,actimeo=0,vers=3,t
imeo=600,addr=192.168.40.10 0 0
NFSServer1:/vspex_ora2_temp /u04/app/oracle/oradata/VSPEX1 nfs
rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,actimeo=0,vers=3,t
imeo=600,addr=192.168.40.11 0 0
NFSServer1:/vspex_ora2_fra /u05/app/oracle/oradata/VSPEX1 nfs
rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,actimeo=0,vers=3,t
imeo=600,addr=192.168.40.11 0 0
Figure 21.
Sample /etc/mtab NFS mount options
Oracle recommends that you set the NFS buffer size parameters rsize and wsize to
32768 in the system configuration file /etc/fstab. However, you can use the file
oranfstab to specify additional Oracle Database specific options for Direct NFS.
52
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Solution Implementation
The oranfstab is located in one of the two following locations:
•
/etc/oranfstab where the options are global to all Oracle databases on the
server
•
$ORACLE_HOME/dbs/oranfstab where the options are specific to a single
database
The Oracle Database requires that mount points be mounted by the kernel NFS
system even when served through Direct NFS.
To enable Direct NFS, you can optionally create an oranfstab file with the following
attributes for each NFS server to be accessed using Direct NFS:
•
Server: The NFS server name.
•
Path: Up to four network paths to the NFS server, specified either by IP
address, or by name, as displayed using the ifconfig command on the filer.
•
Local: Up to four local paths on the database host, specified by IP address or
by name, as displayed using the ifconfig command run on the database host.
•
Export: The exported path from the NFS server.
•
Mount: The corresponding local mount point for the exported volume.
•
Dontroute: Specifies that outgoing messages should not be routed by the
operating system, but instead sent using the IP address they are bound to.
Please note that this attribute does not work on Linux with multiple paths in
the same subnet.
•
mnt_timeout: Specifies (in seconds) how long Direct NFS client should wait
for a successful mount before timing out. This parameter is optional and the
default timeout is 10 minutes.
Figure 22 provides an example of an oranfstab file with two NFS server entries:
server: NFSServer1
local: vspex_ora2
path: NFSServer1
export: /vspex_ora2_data mount: /u02/app/oracle/oradata/vspex1
export: /vspex_ora2_temp mount: /u04/app/oracle/oradata/vspex1
export: /vspex_ora2_fra mount: /u05/app/oracle/oradata/vspex1
server: NFSServer2
local: vspex_ora2
path: NFSServer2
export: /vspex_ora2_redo mount: /u03/app/oracle/oradata/vspex1
Figure 22.
Configuring the
initialization
parameters
Oranfstab example
To configure the initialization parameters, you can use one of the following methods.
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
53
Solution Implementation
Automatic Shared Memory Management
To automate the management of shared memory structures, implement Automatic
Shared Memory Management (ASMM). The following shared memory structures will
be automatically managed:
•
DB_CACHE_SIZE
•
SHARED_POOL_SIZE
•
LARGE_POOL_SIZE
•
JAVA_POOL_SIZE
•
STREAMS_POOL_SIZE
When implemented, ASMM enables the Oracle database to distribute memory
between these components based on workload requirements. Hence, these
components are considered autotuned parameters. The background process, Memory
Manager (MMAN), coordinates the sizing of these components and moves memory to
where it is needed most.
To implement this feature, the set the following initialization parameters:
•
SGA_TARGET set to a nonzero value
•
STATISTICS_LEVEL=TYPICAL (or ALL)
Automatic Memory Management
Do not use Oracle Automatic Memory Management (AMM) since AMM is incompatible
with HugePages. If you want to use HugePages make sure that both MEMORY_TARGET
/ MEMORY_MAX_TARGET initialization parameters are not set.
For more information, see My Oracle Support Note ID 749851.1.
I/O operations for file system files
To enable direct I/O and async I/O for file system files, set
FILESYSTEMIO_OPTIONS=SETALL
With async I/O, processing continues while the I/O request is submitted and
processed. This allows async I/O to bypass some of the performance bottlenecks
associated with I/O operations. Direct I/O is a feature available in the filesystem that
delivers data directly to the application without caching in the file system buffer
cache.
Direct NFS does not depend on the value of FILESYSTEMIO_OPTIONS. Direct NFS
always issues async and direct I/O as it does not depend on OS support. However, so
that there is a backup to the OS NFS client in case of mis-configuration, set
filesystemio_options to SETALL if the OS supports it.
HugePages was enabled in the Linux Kernel as part of the Template creation process
Enabling the
HugePages setting described in Table 13.
The database initialization parameter, USE_LARGE_PAGES, was from 11.2.0.2 to
manage HugePages for use by the database. From version 11.2.0.3, setting
54
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Solution Implementation
USE_LARGE_PAGES to AUTO causes the oradism process to try to reconfigure the
Linux kernel to increase the number of HugePages to match the database
requirements.
See My Oracle Support article USE_LARGE_PAGES To Enable HugePages In 11.2 [ID
1392497.1] for more details.
Creating the
database using
DBCA
The Database Configuration Assistant (DBCA) is the preferred way to create a
database because it is a more automated approach and the database is ready to use
when DBCA completes. Launch DBCA as a standalone tool to create an Oracle
Database.
To start DBCA on UNIX, Linux, or at a Windows command-line prompt, enter the
following command:
dbca
Note: The dbca utility is typically located in the ORACLE_HOME/bin directory.
Figure 23.
Oracle Database Configuration Assistant setup screen
Oracle DBCA runs in interactive mode. Using the screens in Oracle DBCA, you can
either modify one of the preconfigured database types or customize the database.
General Purpose/Transaction Processing is a preconfigured database template that
you can create and customize during the installation. This involves amending the
Performing the
database post
creation activities
•
Initialization parameters
•
Size and placement of the database files to use the NFS mount points
After you create a database as described in Creating the database using DBCA, the
instance is open and available for normal database use. However, the Oracle
database will not start up or shut down automatically with the virtual machine.
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
55
Solution Implementation
With Oracle Database 11gR2, there are two methods to automate the Oracle
Database startup and shutdown process:
•
Use an initialization script and the /etc/oratab file as defined by My Oracle
Support Document How to Automate Startup/Shutdown of Oracle Database
on Linux [ID 222813.1]
•
Install the additional 11.2 Grid Infrastructure software using the single
instance option. This will use a modified Cluster Ready Services (CRS)
component to automate the startup of the database, listener, and Oracle
Automatic Storage Management (a grid Infrastructure component) if required.
This solution uses the initialization script shown in Figure 24. Follow these steps to
set up automated startup and shutdown of the database and listener.
1.
Update the /etc/oratab file with an entry in the following format:
<ORACLE_SID>:<ORACLE_HOME>:Y
Where:
ORACLE_SID is the name of the database
ORACLE_HOME is the directory where the Oracle 11gR2 database binaries
are installed
Y states that the database can be started up and shutdown automatically
For example: VPLEXM1: /u01/app/oracle/11.2.0.3:Y
2.
56
As root, create the service script /etc/init.d/dbora as shown in Figure 24.
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Solution Implementation
#!/bin/bash
#
# chkconfig run levels and script numbering
# chkconfig: 35 99 10
# description: Starts and stops Oracle processes
#
# Set ORA_HOME to be equivalent to the $ORACLE_HOME
# from which you wish to execute dbstart and dbshut;
# Set ORA_OWNER to the user id of the owner of the
# Oracle database in ORA_HOME.
#
ORA_HOME=/u01/app/oracle/11.2.0.3
ORA_OWNER=oracle
case "$1" in
'start')
# Start the TNS Listener
su - $ORA_OWNER -c "$ORA_HOME/bin/lsnrctl start"
# Start the Oracle databases:
su - $ORA_OWNER -c "$ORA_HOME/bin/dbstart"&
touch /var/lock/subsys/dbora
;;
'stop')
# Stop the TNS Listener
su - $ORA_OWNER -c "$ORA_HOME/bin/lsnrctl stop"
# Stop the Oracle databases:
su - $ORA_OWNER -c "$ORA_HOME/bin/dbshut"
rm -f /var/lock/subsys/dbora
;;
esac
# End of script dbora
Figure 24.
3.
Oracle startup/shutdown script /etc/init.d/dbora
Execute the script:
chmod 755 /etc/init.d/dbora
4.
Register the service to the Linux service mechanism for automatic startup at
run level 3 and 5 by issuing the following command.
/sbin/chkconfig --add dbora
With the infrastructure build tasks complete, the database is now ready for use.
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
57
Solution Implementation
Implementing backup and recovery
Overview
The Avamar Plug-in for Oracle works with Oracle and Oracle Recovery Manager
(RMAN) to backup an Oracle database, tablespace, or datafiles to an Avamar server or
a Data Domain system. The Avamar Plug-in for Oracle serves as a backup module and
the Avamar server and/or Data Domain system as a storage device.
RMAN initiates the backup and recovery operations. The Avamar plug-in for Oracle
interprets RMAN backup and recovery commands, and then routes the commands to
the Avamar server. The Avamar server sends commands to the Avamar Plug-in for
Oracle to perform the backup and restore activity.
Figure 25 illustrates how the Avamar server and the Avamar Plug-in for Oracle
interface with an Oracle database and RMAN.
Figure 25.
Note
58
Avamar server and Avamar Plug-in for Oracle
You can also use EMC Data Domain as the backup target for Avamar as an
option. The Avamar client and plug-ins are installed in the same way as when
using Avamar as the backup target.
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Solution Implementation
Minimum backup
software and
hardware
requirements
Table 18 below lists the hardware requirements for the Avamar administrator server
on Windows or LINUX.
Table 18.
Hardware requirements for Avamar Grid
Description
Requirement
vCPU
2
RAM
2 GB
Storage
20 GB
Network
1 Gbps
OS platform
Windows or Linux server
Required account privileges
You must have domain-level administrator access for account privileges. Typically,
the Administrator account you used when installing and configuring the Oracle server
is also a member of the Administrators group on each server.
Deploying the
backup software
Table 19 below lists the steps to install the Avamar Administrator management
console and to configure the Oracle server. For more detailed information, refer to
Avamar 6.1 for Oracle User Guide.
Table 19.
Install Avamar Admin and configure Oracle Server
Task
Description
Reference
Install the
Avamar Admin
console
Launch a web browser
eg: http://avamarserver
Download the Avamar administrator
software
Avamar 6.1 for Oracle
User Guide
Install the Avamar administrator on a
Windows server
Avamar 6.1 for Oracle
User Guide
Download the Avamar agent from the
Avamar server
Avamar 6.1 for Oracle
User Guide
Download the Avamar RMAN plug-in
from the Avamar server
Avamar 6.1 for Oracle
User Guide
Install the Avamar client on the
Oracle server
Avamar 6.1 for Oracle
User Guide
Install the Avamar RMAN plug-in on
the Oracle server
Avamar 6.1 for Oracle
User Guide
Install the
Avamar agent
on the Oracle
server
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
59
Solution Implementation
Installing the
backup software
Task
Description
Reference
Register the
Oracle server
agent to the
Avamar server
Use the scripts provided by Avamar.
Avamar 6.1 for Oracle
User Guide
Create a
backup user on
the Oracle
server
Use Oracle SQL commands to create
a backup user with sysdba privileges
Avamar 6.1 for Oracle
User Guide
To install the backup software, follow these steps:
1.
Install MMC on a Windows server. Launch a web browser and point to the
Avamar server name as shown in Figure 26.
Figure 26.
2.
Link to 'Documents and Downloads' below for Avamar documentation
and server/client agents and plug-ins.
Install the Avamar agent client package on the Oracle server using the
following command:
rpm –ivh AvamarClient-linux-rhel5-x86_64.VERSION.rpm
3.
Register the Avamar client (Oracle server) to the Avamar server using the
following command:
/usr/local/avamar/bin/avregister
60
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Solution Implementation
4.
Install the Avamar RMAN plug-in packages on the Oracle server using the
following command:
rpm –ivh AvamarRMAN-linux-rhel5-x86_64.VERSION.rpm
5.
Create the backup user with sysdba privileges using the following
commands:
SQL> create user backupuser identified by manager;
SQL> grant sysdba to backupuser;
Configuring the
backup software
To configure the backup software, follow these steps:
1.
Launch Avamar Administrator (shown in Figure 27) and configure Avamar to
back up the entire Oracle Database 11gR2.
Figure 27.
Avamar Administrator authentication and server (Grid) selection
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
61
Solution Implementation
2.
From Avamar Administrator, select Tools -> Manage Create datasets -> New.
The New Dataset dialog box displays as shown in Figure 28.
Figure 28.
62
Avamar dataset for Oracle RMAN (LINUX)
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Solution Implementation
3.
Click the Options tab to display the Edit Dataset options as shown in Figure
29. Enter Oracle instance name, backup user name and password, the
number of channels, and the database backup options.
Figure 29.
Specifying the Avamar Oracle dataset information
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
63
Solution Implementation
4.
Create a schedule from Avamar Administrator by selecting Tools -> Manage
schedules -> New. The Manage All Schedules screen displays as shown in
Figure 30.
Figure 30.
5.
64
Avamar Schedule named 'BRS' in force for RMAN dataset named ‘Oracle’
in this case.
Create retention policy. From Avamar Administrator, select Tools -> Manage
retention policies -> New. The Manage all Retention Policies dialog box
displays as shown in Figure 31.
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Solution Implementation
Figure 31.
Avamar Retention policy named 'BRSORACLE' in force for this dataset
and earlier set schedule.
6.
Create backup group. From Avamar Administrator, select Tools -> action ->
group -> new group.
7.
On the New Group screen, pecify a group name, select the datasets, a
schedule, the retention policy, and the client (which is Oracle server here).
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
65
Solution Implementation
Figure 32.
8.
Using the Data
Domain system
Avamar Policy tab highlighting Group Name of 'BRS' for the 'Oracle'
dataset with prior Schedules and Retention Periods in force per the
created policy.
With the backup configuration complete, you can to perform backup jobs.
Configuring the Data Domain system
To integrate a Data Domain system to the Avamar configuration, prepare the Data
Domain system by enabling DD Boost and the creating a DD Boost user account so
that Avamar can access the Data Domain system for backups, restores, and
replication, if applicable. Perform the following steps:
1.
Enable DD Boost on the Data Domain system by issuing the following
command:
ddboost enable
2.
Create a DD Boost account with admin privileges by issuing the following
command:
user add USER priv admin
3.
Set the new account as the DD Boost user by issuing the following
command:
ddboost set user-name USER
4.
Disable and enable DD Boost to allow the changes by issuing the following
commands”
ddboost disable
ddboost enable
66
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Solution Implementation
Adding a Data Domain system
This section describes how to add the Data Domain system. To add Data Domain to
the backup configuration, perform the following steps:
1.
In Avamar Administrator, select Server to launch the Avamar Server window
as shown in Figure 33.
Figure 33.
Avamar's Server panel
2.
Select Actions -> Add Data Domain System. The Add Data Domain System
screen displays as shown in Figure 34.
3.
Specify the Data Domain system name, DDBoost user name, and password,
maximum number of streams, indicate if you wish to use the system default
replication storage, and then click Get Info.
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
67
Solution Implementation
Figure 34.
4.
Avamar’s 'Add Data Domain System' panel
Click the SNMP tab and verify that the SNMP configuration is correct and
click OK.
This completes the process to add a Data Domain system to an Avamar server.
68
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Chapter 5
Solution Verification
This chapter presents the following topics:
Verifying the baseline infrastructure ........................................................... 70
Verifying the Oracle Database .................................................................... 72
Backup and recovery verification methodology ........................................... 74
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
69
Solution Verification
Verifying the baseline infrastructure
Overview
This section provides a list of items that you need to review after configuring the
solution. The goal of this section is to verify the configuration and functionality of the
solution, and to ensure that the configuration supports the core availability
requirements. Table 20 describes the tasks that must be completed.
Table 20.
Tasks for verifying the VSPEX Proven Infrastructure installation
Task
Description
Reference
Verify ESXi functionality
Verify the basic ESXi
functionality of the solution with
a post-installation checklist.
Using EMC VNX storage with
VMware vSphere
http://www.vmware.com
Verify redundancy of the
solution components
Verify the redundancy of the
solution of the components:
Vendor documentation
Storage
ESXi host
Network switch
Verify the Oracle Database
configuration in the VSPEX
Proven Infrastructure
Verify the Oracle Database
configuration.
http://support.oracle.com
Monitor the health of the VSPEX
Proven Infrastructure for
virtualized Oracle Database
Use the tool to monitor the
VSPEX Proven Infrastructure’s
health
View alert logs (Oracle Database
11gR2)
View storage status from Unisphere
management console
View VM logs from vSphere 5.1
client
Verifying the ESXi
functionality
EMC recommends that you verify the ESXi configurations before deploying the system
to production on each ESXi server.
For more detailed information, refer to appropriate documents in VSPEX Proven
Infrastructure.
Verifying the
solution
component
redundancy
To ensure that the various components of the solution maintain availability
requirements, it is important to test specific scenarios related to maintenance or
hardware failure. EMC recommends that you verify redundancy of the solution
components including storage, ESXi hosts, and network switches.
For detailed steps, refer to the appropriate documents in VSPEX Proven Infrastructure.
70
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Verifying the
Oracle Database
configuration
Using the health
monitoring tools
To ensure that Oracle Database 11gR2 is running smoothly in the VSPEX
infrastructure, use these steps to verify and monitor the Oracle Database
configuration:
1.
Check the installed Oracle Database contents and directory location. Make
sure Oracle Database has been installed successfully.
2.
Verify that the Oracle Database that you plan to use is accessible. Use the
TNSPING utility to test connectivity from the client with the following
command: tnsping net_service_name count.
Health monitoring is a simplified measurement that reflects the reliability, stability,
and performance of the entire VSPEX Proven Infrastructure for virtualized Oracle
Database. Table 21 lists some tools for you to monitor and troubleshoot your VSPEX
solution.
Table 21.
Tools for monitoring the solution
Tools
Description
esxtop
The esxtop tool provides a real-time view (updated every five
seconds, by default) of ESXi Hypervisor performance metrics, such
as CPU. It is important that you understand the performance for
hypervisor in order measure or troubleshoot the Oracle Database
performance-related issues.
Unisphere Analyzer
Unisphere Analyzer software is preloaded on all VNX storage
systems with the FLARE/block bundles (or later). It is an easy to
monitor, collect, and analyze the all aspects of storage related data
from it.
Oracle Enterprise
Manager Database
Control
Oracle Enterprise Manager Database Control (Database Control) is
the primary tool for managing your Oracle database. It is installed
with the database.
Using Database Control, you can perform administrative tasks such
as creating schema objects (tables, views, indexes, and so on),
managing user security, managing database memory and storage,
backing up and recovering your database, and importing and
exporting data. You can also view performance and status
information about your database.
Automatic
Workload
Repository (AWR)
The Automatic Workload Repository (AWR) collects, processes, and
maintains performance statistics for problem detection and selftuning purposes. This data is both in memory and stored in the
database. The gathered data can be displayed in both reports and
views.
EMC recommends that you use the above tools to monitor the performance of the
entire Oracle Database environment. For details about performance verification and
test methodology, refer to the companion Design Guide.
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
71
Solution Verification
Verifying the Oracle Database
Overview
This section provides a verification example of an Oracle OLTP Database in the VSPEX
Proven Infrastructure for virtualized Oracle Database solution. The purpose is to
measure the performance of the Oracle Database to ensure that the VSPEX Proven
Infrastructure meets the customer’s business requirements.
Before you verify your own VSPEX Proven Infrastructure, EMC recommends that you
read the Application validation methodology section in the companion Design Guide
to understand the test methodology.
Understanding the
key metrics
After you identify the business requirements, you need to know the key metrics for
the Oracle Database testing when running the Oracle Database validation tests. The
key metrics are identified in the Design Guide in the Application validation
methodology section. My Oracle Support Note 1275596.1 provides a list of typical
acceptable I/O response times.
Loading the test configuration
The Swingbench Order Entry – PL/SQL (SOE) schema delivers the OLTP workloads
required by the solution. The Order Entry schema introduces heavy contention on a
small number of tables and is designed to stress CPU and memory. To establish an
optimal configuration, we tested different users accessing the order entry schema.
Before running an Oracle Database 11gR2 test, it is important to understand the test
scenarios according to the business requirements. You can easily create your own
test scenarios using Swingbench. By editing the transaction list, you can define the
different operations that end users perform in the Oracle Database. Table 22 lists
some of the general operations we considered in the Oracle Database 11gR2 VSPEX
solution.
Table 22.
General operations
Workload
Operation
Weight
Swingbench Order
Entry
Customer Registration
30
Browse Products
60
Order Products
50
Process Orders
30
Browse Orders
10
Swingbench allows you to select a variable “think time” (delay) between transactions.
Figure 35 shows the load panel on the Swingbench interface, with the appropriate
think time parameters set.
72
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Figure 35.
Swingbench interface load panel screenshot
We simulated the workload using Swingbench against the database as shown in
Figure 36.
Figure 36.
Swingbench order entry screen for workload simulation
Once the test is complete, you can use the Automatic Workload Repository (AWR)
report to check the results and verify that you have achieved the key metrics in your
test environment. Since the metrics can vary with different kinds of Oracle Database
configuration and VSPEX offerings for VMware, refer to the VSPEX Sizing Tool for the
detailed metrics.
After enabling FAST Cache on the existing database, we observed a reduction in I/O
access latency (in milliseconds).
If the test results are not ideal, it is easy to identify the bottleneck from the AWR
report. For detailed information about the AWR report, refer to the My Oracle Support
Note ID 1359094.1.
After identifying potential bottlenecks, refer to the Implementing the storage section
to make sure that you have the correct configuration of VNX, switches, hypervisor, or
load balancer for the VSPEX for virtualized Oracle solution. System bottlenecks
represent a point of contention where there are insufficient resources to service user
transaction requests. Consider adding more resources into the solution to meet the
requirement. Often, the reason for the bottleneck is inefficient custom code or thirdparty solutions due to customization.
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
73
Solution Verification
Reviewing the
Oracle dataset
In this VSPEX validation test, we used the Swingbench Order Entry - PL/SQL (SOE)
schema to deliver the OLTP workloads. This schema models a traditional OLTP
database. Tables and indexes reside in separate tablespaces. The SOE schema used
in this solution contains the tables and indexes shown in Table 23.
Table 23.
Schema tables and indexes
Table name
Index
CUSTOMERS
CUSTOMERS_PK (UNIQUE),
CUST_ACCOUNT_MANAGER_IX,
CUST_EMAIL_IX, CUST_LNAME_IX,
CUST_UPPER_NAME_IX
INVENTORIES
INVENTORY_PK (UNIQUE),
INV_PRODUCT_IX,
INV_WAREHOUSE_IX
ORDERS
ORDER_PK (UNIQUE),
ORD_CUSTOMER_IX,
ORD_ORDER_DATE_IX,
ORD_SALES_REP_IX, ORD_STATUS_IX
ORDER_ITEMS
ORDER_ITEMS_PK (UNIQUE),
ITEM_ORDER_IX, ITEM_PRODUCT_IX
PRODUCT_DESCRIPTIONS
PRD_DESC_PK (UNIQUE),
PROD_NAME_IX
PRODUCT_INFORMATION
PRODUCT_INFORMATION_PK
(UNIQUE), PROD_SUPPLIER_IX
WAREHOUSES
WAREHOUSES_PK (UNIQUE)
LOGON
–
Backup and recovery verification methodology
Overview
This section provides a verification example of an Oracle OLTP Database in the VSPEX
Proven Infrastructure for virtualized Oracle Database solution. The purpose is to
perform backup and recovery on an Oracle Database using Avamar backup software
to ensure the database is fully protected and meets your business requirements.
Before testing backup and recovery of your Oracle 11gR2 database, you need to
understand Avamar backup terminology:
Table 24.
74
Avamar backup terminology
Term
Description
Datasets
Avamar datasets are lists of directories and files to back up from a
client. Assigning a dataset to a client or group enables you to save
backup selections. Each dataset defines a source data list, an
exclusion list, an inclusion list, and plug-in options.
Schedules
Schedules are reusable objects that control system backup
activity at one of the supported intervals. The intervals include
daily, weekly, monthly, and on-demand.
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Term
Description
Retention policies
Backup retention policies enable you to specify how long to retain
a backup in the system. The typical retention settings include a
retention period, a specific end date, or no end date.
Backup groups
Avamar uses groups to implement various policies that automate
backups and enforce consistent rules and system behavior across
an entire grid or most likely, a group.
Group members are client machines that have been added to a
particular group to perform the same policy-based scheduled
backups.
Backing up an
Oracle database
To run a backup job, from the Avamar Administrator interface, click backup to start
the backup process. While the backup job is running, use the Activity Monitor screen
to monitor the backup process.
After a backup job completes, the job status indicates Completed. On the Activity
Monitor screen, select a completed backup job, review the number of datasets that
were backed up, and then check the deduplication ratio of the backup dataset.
Figure 37 shows the status of the backup job as Completed on the Avamar
Administrator Activity Monitor screen.
Figure 37.
Status of Completed Avamar backup job.
Changing the
database contents
To change the database contents, rerun the Swingbench tool against the Oracle
database for a few minutes and then use SQL statements or Oracle Enterprise
Manager (OEM) to ensure that the size of the database has been changed.
Restoring the
Oracle database
After the Oracle database is backed up successfully, you can change the database to
suit the test plan. To simulate recovering an entire database to its last full database
backup, use Avamar Administrator’s Select for Restore option in the Backup and
Restore panel shown in Figure 38.
Select the Oracle client and then select the date to be recovered followed by how
much of ‘Contents of Backup’ is to be recovered. Choose the entire 11g database (but
if you want to back up to a more granular level, you can click on the ‘+’ sign to the left
to reveal tablespaces and datafiles).
From the Action menu, select Restore now to restore the database.
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
75
Solution Verification
Figure 38.
Avamar Administrator Backup and Restore tab
Once the restore job is complete, the job status changes to Completed as shown on
the Avamar administrator activity screen displayed in Figure 39.
Figure 39.
Avamar's Activity panel showing a Completed job
Use Oracle Enterprise Manager GUI or SQL statements to verify that the datasets are
successfully recovered.
76
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Chapter 6
Reference
Documentation
This chapter presents the following topics:
White papers ............................................................................................. 78
Product documentation ............................................................................. 79
Other documentation ................................................................................. 80
Links ......................................................................................................... 80
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
77
Configuration Worksheet
White papers
Oracle
•
Oracle Edition Comparisons
http://www.oracle.com/us/products/database/enterpriseedition/comparisons/index.html
•
Oracle Software Investment Guide
http://www.oracle.com/us/corporate/pricing/sig-070616.pdf
•
Oracle Database Licensing
http://www.oracle.com/us/corporate/pricing/databaselicensing-070584.pdf
•
Oracle Processor Core Factor Table
http://www.oracle.com/us/corporate/contracts/processor-core-factor-table070634.pdf
•
Installing and Using Standby Statspack in 11gR2 [ID 454848.1]
•
How to Tell if the IO of the Database is Slow
•
HugePages on Linux: What It Is... and What It Is Not... [ID 361323.1]
[Article ID 1275596.1]
https://support.oracle.com (Requires Login)
EMC
•
Deploying Oracle Database on EMC VNX Unified Storage
http://www.emc.com/collateral/hardware/white-papers/h8242-deployingoracle-vnx-wp.pdf
VMware
•
Understanding Oracle Certification Support and Licensing in Vmware Environments
http://www.vmware.com/files/pdf/techpaper/vmw-understanding-oraclecertification-supportlicensing-environments.pdf
•
Oracle Databases on VMware Best Practices Guide
http://www.vmware.com/files/pdf/partners/oracle/Oracle_Databases_on_VM
ware_-_Best_Practices_Guide.pdf
•
Best Practices for running VMware vSphere on NFS.
http://www.vmware.com/files/pdf/VMware_NFS_BestPractices_WP_EN.pdf
•
Performance Best Practices for VMware vSphere™ 5.1
http://www.vmware.com/pdf/Perf_Best_Practices_vSphere5.1.pdf
78
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Product documentation
All of the links included in this section were working correctly at the time of
publication.
EMC
•
EMC VSPEX Private Cloud: VMware vSphere 5.1 for up to 500 Virtual
Machines – Reference Architecture
•
VNX System Installation Guide
•
VNX Series Configuration Worksheet
•
Using EMC VNX Storage with VMware vSphere–TechBook
•
EMC VSI for VMware vSphere: Unified Storage Management—Product Guide
•
EMC VSI for VMware vSphere: Unified Storage Management—Product Guide
•
VNX family
http://www.emc.com/storage/vnx/vnx-family.htm
•
VNX series documentation on EMC online support site
https://support.emc.com/products/12781_VNX-Series/Documentation/
Oracle
•
Oracle Database 11gR2 Documentation Library 11g Release 2 (11.2)
http://www.oracle.com/pls/db112/homepage
http://support.oracle.com
•
•
VMWare
Oracle Database 11g Release 2 documentation:
http://docs.oracle.com/cd/E11882_01/install.112/e24321.pdf
http://www.oracle.com/technetwork/indexes/documentation/index.html
Oracle Database Release Notes for Linux
•
My Oracle Support (https://support.oracle.com/ latest OPatch and latest
PSU
•
My Oracle Support Note ID 749851.1
•
My Oracle Support Note ID 1392497.1
•
My Oracle Support Note ID 361468.1
•
My Oracle Support Note ID 1359094.1
•
Sphere Installation and Setup
•
vCenter Server and Host Management Guide
•
Installing vCenter Server 5.1 best practices
•
vSphere Networking
•
vSphere Storage
•
vCenter Server and Host Management
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
79
Configuration Worksheet
•
vSphere Installation and Setup
vCenter Server and Host Management
•
vSphere Resource Management
•
Automating High Availability (HA) Services with VMware HA –VMware
Technical note
•
General documentation:
http://www.vmware.com
•
VMware vSphere Documentation
http://www.vmware.com/support/pubs/vsphere-esxi-vcenter-server-pubs.html
•
vSphere PowerCLI Documentation
http://www.vmware.com/support/developer/PowerCLI/
Other documentation
•
Oracle Database 11g Release 2 on Red Hat Enterprise Linux 6 Deployment
Recommendations
http://www.redhat.com/resourcelibrary/reference-architectures/deployingoracle-11gr2-on-rhel-6
Links
•
Everything Oracle at EMC
https://community.emc.com/community/connect/everything_oracle
•
My Oracle Support: Oracle Support Website
https://support.oracle.com (user account required)
80
FAQ: How to Use AWR reports to Diagnose Database Performance Issues [ID
1359094.1]
Requirements for Installing Oracle 11gR2 RDBMS on RHEL6 or OL6 64-bit
(x86-64) [ID 1441282.1]
How to Tell if the IO of the Database is Slow [Article ID 1275596.1]
HugePages on Linux: What It Is... and What It Is Not... [ID 361323.1]
USE_LARGE_PAGES To Enable HugePages In 11.2 [ID 1392497.1]
HugePages on Oracle Linux 64-bit [ID 361468.1]
Installing and Using Standby Statspack in 11g [ID 454848.1]
Installing and Configuring StatsPack Package[Article ID 149113.1]
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Appendix A
Configuration
Worksheet
This appendix presents the following topic:
Configuration worksheet for Oracle Database 11gR2................................... 82
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
81
Configuration Worksheet
Configuration worksheet for Oracle Database 11gR2
Before configuring Oracle Database 11gR2 for this solution, you need to gather some
customer-specific configuration information such as IP addresses, hostnames, and so
on. The following tables provide a worksheet that you can use to record the
information. You can also print and use the worksheet as a customer “leave behind”
document for future reference.
To confirm the customer information, cross-reference with the relevant array
configuration worksheet: EMC VNX File and Unified Worksheets.
Table 25.
Common server information
Server Name
Purpose
Primary IP
Domain Controller
DNS Primary
DNS Secondary
DHCP
NTP
SMTP
SNMP
vCenter Server
Swingbench server
Database server 01
Database server 02
Database server 03
Table 26.
Server
Name
ESXi server information
Purpose
Primary
IP
Private Net (storage) addresses
ESXi
Host 1
ESXi
Host 2
…
82
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
Configuration Worksheet
Table 27.
Array information
Array name
Root password
NAS admin password
CS0 Primary hostname
CS1 Secondary hostname
(if applicable)
CS0 Primary IP address
CS1 Secondary IP address
(if applicable)
SPA IP address
SPB IP address
Virtual machine
infrastructure pool name
Virtual machine
infrastructure file system
name
Datafile storage pool
name
Datafile file system name
Redo log storage pool
name
Redo log file system name
Fra storage pool name
Fra file system name
Table 28.
Name
Network infrastructure information
Purpose
IP
Subnet
Mask
Default
Gateway
Ethernet Switch 1
Ethernet Switch 2
…
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
83
Configuration Worksheet
Table 29.
Name
VLAN information
Network Purpose
VLAN ID
Allowed Subnets
Virtual Machine
Networking
Management
NFS networking
vMotion
Table 30.
Account
Service accounts
Purpose
Password (optional, secure
appropriately)
vCenter Server administrator
Array administrator
Oracle Database administrator
84
EMC VSPEX for Virtualized Oracle Database 11g OLTP
Implementation Guide
