Download EMC VSPEX ORACLE COMPUTING Oracle Database Virtualization Enabled by EMC Data Protection

IMPLEMENTATION GUIDE
EMC VSPEX ORACLE COMPUTING
Oracle Database Virtualization
with VMware vSphere and EMC XtremIO
Enabled by EMC Data Protection
EMC VSPEX
Abstract
This Implementation Guide describes the high-level steps and best practices required
to implement the EMC® VSPEX® Proven Infrastructure for virtualized Oracle Database
11g or 12c on a VSPEX Private Cloud with VMware vSphere enabled by EMC
XtremIO™. Usage guidelines and major customer benefits are also included.
July 2015
Copyright © 2015 EMC Corporation. All rights reserved. Published in the USA.
Published July 2015
EMC believes the information in this publication is accurate as of its publication date.
The information is subject to change without notice.
The information in this publication is provided as is. EMC Corporation makes no
representations or warranties of any kind with respect to the information in this
publication, and specifically disclaims implied warranties of merchantability or
fitness for a particular purpose. Use, copying, and distribution of any EMC software
described in this publication requires an applicable software license.
EMC2, EMC, and the EMC logo are registered trademarks or trademarks of EMC
Corporation in the United States and other countries. All other trademarks used
herein are the property of their respective owners.
For the most up-to-date listing of EMC product names, see EMC Corporation
Trademarks on EMC.com.
EMC VSPEX Oracle Computing: Oracle Database Virtualization with VMware vSphere
and EMC XtremIO Implementation Guide
Part Number H14313
2
EMC VSPEX Oracle Computing: Oracle Database Virtualization with VMware vSphere
and EMC XtremIO Implementation Guide
Contents
Contents
Chapter 1
Introduction
6
Purpose of this guide.................................................................................................. 7
Business value ........................................................................................................... 7
Audience .................................................................................................................... 8
Terminology................................................................................................................ 9
Chapter 2
Before You Start
10
Overview .................................................................................................................. 11
Pre-deployment tasks ............................................................................................... 11
Deployment workflow ............................................................................................... 12
Deployment prerequisites ........................................................................................ 13
Essential reading ...................................................................................................... 14
Chapter 3
Solution Overview
15
Overview .................................................................................................................. 16
EMC VSPEX Proven Infrastructure.............................................................................. 16
Solution architecture ................................................................................................ 17
Summary of key components.................................................................................... 18
Chapter 4
Solution Implementation
19
Overview .................................................................................................................. 20
Implementing the network ........................................................................................ 21
Overview.............................................................................................................. 21
Configuring the infrastructure network ................................................................. 21
Configuring the VLANs ......................................................................................... 22
Configuring the storage network .......................................................................... 22
Completing the network cabling........................................................................... 23
Implementing the storage array ................................................................................ 24
Overview.............................................................................................................. 24
Preparing and installing XtremIO.......................................................................... 24
Creating XtremIO volumes for vSphere hosts ....................................................... 25
Implementing ESXi Server and vCenter ..................................................................... 25
Overview.............................................................................................................. 25
Installing vSphere ................................................................................................ 26
Configuring vSphere networking .......................................................................... 27
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
3
Tables
Provisioning storage for VMFS datastores ............................................................ 27
Optimizing vSphere for XtremIO ........................................................................... 29
Deploying an Oracle database .................................................................................. 29
Overview.............................................................................................................. 29
Preparing the Oracle virtual machine template..................................................... 30
Deploying and configuring a database ................................................................. 32
Performing the database post creation activities ................................................. 34
Implementing EMC Data Protection........................................................................... 35
Chapter 5
Solution Verification
36
Overview .................................................................................................................. 37
Using the health monitoring tools ........................................................................ 38
Chapter 6
Reference Documentation
39
Reference documentation......................................................................................... 40
EMC documents ................................................................................................... 40
Oracle documents................................................................................................ 40
VMware documents ............................................................................................. 40
Links.................................................................................................................... 41
Appendix A Configuration Worksheet
42
Customer Configuration Worksheet .......................................................................... 43
Figures
Figure 1.
VSPEX Proven Infrastructure ................................................................ 16
Figure 2.
Architecture of the validated infrastructure .......................................... 17
Figure 3.
Example redundant XtremIO network................................................... 22
Figure 4.
Sample FC storage network ................................................................. 23
Figure 5.
XtremIO volumes mapping to initiator groups ...................................... 28
Figure 6.
Oracle Database 12c Configuration Assistant setup ........................... 32
Figure 7.
Storage locations using ASM diskgroup............................................... 33
Tables
4
Table 1.
Terminology........................................................................................... 9
Table 2.
Tasks for pre-deployment .................................................................... 11
Table 3.
VSPEX workflow for virtualized Oracle database deployment ............... 12
Table 4.
Hardware and software requirements .................................................. 13
Table 5.
Solution components .......................................................................... 18
Table 6.
Implementation processes .................................................................. 20
Table 7.
Tasks for switch and network configuration ......................................... 21
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
Contents
Table 8.
Tasks for storage configuration............................................................ 24
Table 9.
XtremIO volumes for storing virtual Oracle servers ............................... 25
Table 10.
Tasks for server installation ................................................................. 25
Table 11.
Installation and configuration requirements ........................................ 26
Table 12.
Tasks for deploying an Oracle database .............................................. 30
Table 13.
Example of a virtual machine template specification ........................... 31
Table 14.
Storage design to ASM disk group mapping for Oracle database files .. 33
Table 15.
Tasks for verifying the VSPEX installation ............................................ 37
Table 16.
Tools for monitoring the solution ......................................................... 38
Table 17.
Common server information ................................................................ 43
Table 18.
ESXi server information ....................................................................... 43
Table 19.
XtremIO array information.................................................................... 44
Table 20.
Network infrastructure information ...................................................... 44
Table 21.
VLAN information ................................................................................ 44
Table 22.
Service accounts ................................................................................. 45
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
5
Chapter 1: Introduction
Chapter 1
Introduction
This chapter presents the following topics:
Purpose of this guide ................................................................................................. 7
Business value ........................................................................................................... 7
Audience .................................................................................................................... 8
Terminology ............................................................................................................... 9
6
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
Chapter 1: Introduction
Purpose of this guide
EMC® VSPEX® Proven Infrastructures are optimized for virtualizing business-critical
applications. VSPEX provides modular solutions built with technologies that enable
faster deployment, greater simplicity, more choices, higher efficiency, and lower risk.
VSPEX provides partners with the ability to design and implement the virtual assets
that are required to support a fully integrated virtualization solution for an Oracle
relational database management system (RDBMS) on a VSPEX private cloud
infrastructure.
The VSPEX for virtualized Oracle Database 11g or 12c1 infrastructure provides
customers with a system capable of hosting a virtualized database solution that is
scalable and delivers a constant performance level. This solution uses VMware
vSphere 5.5 to provide virtualization supported by the EMC XtremIO™ storage array.
The compute and network components, while vendor-definable, are designed to
provide redundancy and sufficient power to handle the processing and data needs of
the virtual machine environment.
This Implementation Guide describes how to implement the VSPEX Proven
Infrastructure for virtualized Oracle Database 11g or 12c on a VSPEX Private Cloud
with VMware vSphere. This Implementation Guide describes the high-level steps
required to deploy the VSPEX solution for Oracle Database 11g or 12c on a VSPEX
Private Cloud for VMware vSphere with EMC XtremIO storage. The infrastructure
servers required for this solution are designed to be layered on a VSPEX Private Cloud
for VMware vSphere Proven Infrastructure. The same principles and guidelines apply
to XtremIO that have been validated as part of the VSPEX program.
Business value
Database management systems software is used by many different types of
businesses. Despite the increasing market share of other data management tools,
growth in sales is expected to continue and accelerate as customers continue to
diversify their infrastructures and supporting technologies and use more appliances
and configurations.
This VSPEX Proven Infrastructure is focused on helping EMC partners understand the
value that the XtremIO series combined with Oracle bring to customers who often
have growing, isolated IT environments running server-centric applications.
The Oracle 11g version for this solution is Release 2 (11.2.0.4). We refer to this as 11g R2
throughout this document. The Oracle 12c version for this solution is Release 1 (12.1.0.2). We
refer to this as 12c R1 throughout this document.
1
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
7
Chapter 1: Introduction
This solution is designed to meet Oracle database challenges and enable customers
to increase performance, scalability, reliability, and automation. By consolidating
database applications on XtremIO, customers can consolidate a single centralized
storage platform to more effectively manage the exploding data growth that is
challenging businesses today. This solution has been sized and proven by EMC to:

Deploy systems faster, saving time and effort with EMC Proven Solutions

Increase performance and scalability out of the box

Minimize storage requirements and reduce costs
Audience
This guide is intended for internal EMC personnel and qualified EMC VSPEX partners.
This guide assumes that the VSPEX partners who intend to deploy this solution are:

Qualified by EMC to sell, install, and configure the XtremIO family of storage
systems

Qualified to sell, install, and configure the network and server products
required for VSPEX Proven Infrastructures

Certified to sell VSPEX Proven Infrastructures
Partners implementing this solution should also have the necessary technical
training and background to install and configure:

VMware vSphere 5.5 virtualization platform

Red Hat Enterprise Linux (RHEL) 6.5

Oracle Database 11g R2 or 12c R1
This guide provides external references where applicable. EMC recommends that
partners implementing this solution be familiar with these documents. For details,
refer to Essential reading and Reference documentation.
8
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
Chapter 1: Introduction
Terminology
Table 1 lists the key terms used in this guide.
Table 1.
Terminology
Term
Definition
System Global Area
(SGA)
SGA is a group of shared memory structures containing data and
control information from one Oracle database instance.
Process Global Area
(PGA)
PGA is memory dedicated to an operating process or thread that
is not shared by other processes or threads on the system.
Virtual Machine Disk
(VMDK)
VMware VMDK is an open file format that is a container for
virtual hard disk drives for virtual machines.
Virtual Machine File
System (VMFS)
VMware VMFS is a cluster file system that uses storage
virtualization for multiple installations of VMware ESX server.
XtremIO X-Brick
An X-Brick is a specialized configuration of the XtremIO All-Flash
Array that includes 25 SSD drives for this solution.
Storage Controller (SC)
Storage Controller is the computer component of the XtremIO
storage array. SCs are used for all aspects of data moving into,
out of, and between XtremIO arrays.
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
9
Chapter 2: Before You Start
Chapter 2
Before You Start
This chapter presents the following topics:
Overview ..................................................................................................................11
Pre-deployment tasks .............................................................................................. 11
Deployment workflow .............................................................................................. 12
Deployment prerequisites ........................................................................................ 13
Essential reading .....................................................................................................14
10
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
Chapter 2: Before You Start
Overview
This chapter provides an overview of important information you need to be aware of,
documents you need to be familiar with, and tasks you need to perform before you
start implementing the VSPEX Oracle Database 11g or 12c solution on XtremIO.
The Design Guide for this solution describes how to size and design the solution. The
deployment examples in this Implementation Guide are based on the
recommendations and examples in the Design Guide.
Before you deploy Oracle server on a VSPEX Proven Infrastructure, EMC recommends
that you complete the pre-deployment tasks as described in the following section.
Pre-deployment tasks
Pre-deployment tasks include those not directly related to the environment
installation and configuration, but that are prerequisites for installation. These tasks
include collecting hostnames, IP addresses, license keys, and installation media.
Perform the tasks in Table 2 to decrease the time required at the customer site.
Table 2.
Tasks for pre-deployment
Task
Description
Reference
Gather
documents
Get the related documents listed in Essential
reading. These are used throughout the text of this
document to provide details on setup procedures
and deployment best practices for the various
components of the solution.
Essential reading
Gather tools
Get the required and optional tools for the
deployment. Use the information in Table 4 to
confirm that all equipment, software, and
appropriate licenses are available before the
deployment process.
Deployment
prerequisites
Gather data
Collect the customer-specific configuration data for
networking, arrays, accounts, and so on. Enter this
information into the Customer Configuration
Worksheet for reference during the deployment
process.
 XtremIO Storage
In addition, for the most comprehensive arrayspecific information, complete the relevant
XtremIO pre-installation checklist and worksheet.
These documents are available on EMC Online
Support.
Array PreInstallation
Checklist
 Configuration
Worksheet
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
11
Chapter 2: Before You Start
Deployment workflow
Refer to the process in Table 3 to design and implement a VSPEX workflow for your
virtualized Oracle database solution.
Table 3.
VSPEX workflow for virtualized Oracle database deployment
Step
Action
1
Use the VSPEX qualification worksheet to collect user requirements. The qualification
worksheet is provided in Appendix A.
2
Use the EMC VSPEX Sizing Tool to determine the recommended VSPEX Proven
Infrastructure for your Oracle Database 11g or 12c solution, based on the user
requirements collected in Step 1.
For more information, refer to the VSPEX Sizing Tool on the EMC Business Value
Portal.
Note: You need to register the first time you access the tool.
3
Use the Design Guide to determine the final design for your VSPEX solution.
Note: Consider the requirements for all applications, not only the Oracle database.
4
Choose and order the VSPEX Proven Infrastructure components, including the XtremIO
array, server systems, and network switches. Refer to the appropriate VSPEX Proven
Infrastructure document in Essential reading for guidance.
5
Follow this guide to deploy and test your VSPEX solution.
Note: If you already have a VSPEX Proven Infrastructure environment, you can skip the
implementation steps that are already completed.
12
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
Chapter 2: Before You Start
Deployment prerequisites
Table 4 itemizes the hardware and software requirements to configure the solution.
For additional information, refer to the hardware and software tables in the
deployment prerequisites checklist.
Table 4.
Hardware and software requirements
Component
Description
Version
Hardware
Physical servers—Sufficient physical server
capacity to host the required number of virtual
machines, as recommended by the Design Guide
and VSPEX Sizing Tool
N/A
Virtual servers—VMware vSphere servers that host
the virtual infrastructure servers
VMware
vSphere 5.5
Networking—Switch port capacity and capabilities
as required by the virtual server infrastructure
N/A
Storage array—EMC XtremIO array with the required
configuration
EMC XtremIO 4.0
Note: These requirements might be covered in the
existing infrastructure.
Software
EMC XtremIO Management Server
N/A
EMC Virtual Storage Integrator (VSI) for VMware
vSphere Unified Storage Management
N/A
EMC VSI for VMware vSphere Storage Viewer
N/A
VMware vSphere installation media
5.5
VMware vCenter Server installation media
5.5
Oracle database Server
11.2.0.4 or
12.1.0.2
Red Hat Enterprise Linux
6.5
VMware vSphere license keys
5.5
(for infrastructure server hosts)
Licenses
VMware vCenter Server license key
5.5
Oracle database Server license key
11g R2 or 12c R1
Standard or
Enterprise Edition
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
13
Chapter 2: Before You Start
Essential reading
Before implementing the solution described in this guide, EMC recommends that you
read the following documents, which are available from the VSPEX space on the EMC
Community Network or from the VSPEX Proven Infrastructure area on the emc.com
website. If you do not have access to a document, contact your EMC representative.
14

EMC VSPEX Oracle Computing: Oracle Database Virtualization with VMware
vSphere and EMC XtremIO Design Guide

Server Virtualization with EMC XtremIO All-Flash Array and VMware
vSphere 5.5

EMC Backup and Recovery Options for VSPEX for Virtualized Oracle 11g R2
Design and Implementation Guide
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
Chapter 3: Solution Overview
Chapter 3
Solution Overview
This chapter presents the following topics:
Overview ..................................................................................................................16
EMC VSPEX Proven Infrastructure.............................................................................16
Solution architecture ............................................................................................... 17
Summary of key components ...................................................................................18
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
15
Chapter 3: Solution Overview
Overview
This chapter provides an overview of the VSPEX for virtualized Oracle Database 11g or
12c solution. This solution has been designed and proven by EMC to provide server
virtualization, network, storage, and backup resources. Refer to the VSPEX Proven
Infrastructure Guide listed in Essential reading for information on configuring the
required infrastructure components.
EMC VSPEX Proven Infrastructure
VSPEX Proven Infrastructures, as shown in Figure 1, are modular, virtualized
infrastructures validated by EMC and delivered by EMC VSPEX partners. They include
virtualization, server, network, storage, and data protection layers. Partners can
choose the virtualization, server, and network technologies that best fit a customer’s
environment, while the highly available XtremIO storage systems and EMC data
protection technologies provide the storage and data protection layers.
Figure 1.
VSPEX Proven Infrastructure
Refer to the VSPEX Proven Infrastructure Guide in Essential reading for information on
configuring the required infrastructure components.
16
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
Chapter 3: Solution Overview
Solution architecture
Figure 2 shows the architecture validated for an Oracle Database 11g or 12c server
overlay on a VSPEX infrastructure. The XtremIO storage array is an all-flash system
based on scale-out architecture. The system uses building blocks, called X-Bricks,
which can be clustered to grow performance and capacity as required. The solution
supports block storage for virtual Oracle databases using one X-Brick.
Figure 2.
Architecture of the validated infrastructure
To validate this solution, we2 completed the following tasks:

Deployed all Oracle Database 11g or 12c servers as virtual machines on
VMware vSphere 5.5

Deployed the XtremIO array in multiple configurations to support different
resources for both OLTP and DSS workloads on Oracle databases

Determined the recommended storage layout for an Oracle database and the
virtual infrastructure pool in the XtremIO series of storage arrays
The solution architecture includes the following:

2
Storage layer—One X-Brick with a total of 7.58 TB of usable physical capacity
In this guide, “we” refers to the EMC Solutions engineering team that validated the solution.
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
17
Chapter 3: Solution Overview

Oracle database layer— Oracle Database 11g or 12c server with multiple
differently-sized databases and snapshots

Network layer— SAN/IP switches that are designed to support virtualized
environments

Physical servers and virtualization layer—Multiple ESXi servers that enable a
high-performing and virtualized approach to deploy Oracle databases
Summary of key components
Table 5 summarizes the key components used in this solution. Refer to the Design
Guide for overviews of the individual components.
Table 5.
Solution components
Layer
Components
Application
Oracle Database 11g R2 or 12c R1
Virtualization
 VMware vSphere 5.5
 VMware vCenter Server 5.5
 VMware vSphere High Availability
 EMC Virtual Storage Integrator
Compute
VSPEX defines the minimum amount of compute layer resources
required but enables the customer to implement the requirements
using any server hardware that meets these requirements.
Network
Minimum number of network ports required, as defined by VSPEX
Note: VSPEX defines the minimum number of network ports required and
provides general guidance on network architecture, but enables you to
implement any network hardware that meets these requirements.
Storage
EMC XtremIO with:
 EMC XtremIO Management Server
 EMC Virtual Storage Integrator for VMware vSphere
 VMware vSphere Storage APIs for Array Integration
 EMC XtremIO Snapshots
Data protection
18
EMC Data Protection solutions
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
Chapter 4: Solution Implementation
Chapter 4
Solution Implementation
This chapter presents the following topics:
Overview ..................................................................................................................20
Implementing the network ....................................................................................... 21
Implementing the storage array ...............................................................................24
Implementing ESXi Server and vCenter ....................................................................25
Deploying an Oracle database .................................................................................29
Implementing EMC Data Protection ..........................................................................35
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
19
Chapter 4: Solution Implementation
Overview
This chapter describes the steps required to install, configure, and implement the
VSPEX for virtualized Oracle Database 11g or 12c solution. If you already have a
VSPEX Proven Infrastructure environment, you can skip the sections for the
implementation steps. Otherwise, refer to the VSPEX Proven Infrastructure Guide
listed in Essential reading for information on configuring the required infrastructure
components.
Note: This solution requires certain infrastructure services, as shown in Figure 2. These
services can be provided by existing infrastructure at the customer site, by a VSPEX Private
Cloud, or by deploying them as dedicated resources as part of this solution.
Table 6 lists the main stages in the solution implementation process.
Table 6.
20
Implementation processes
Task
Description
Reference
Prepare the physical
network.
Configure the switches and
connect to the network.
Implementing the
network
Prepare storage.
Install and configure the XtremIO
array.
Implementing the
storage array
Prepare the vSphere
hosts.
Install and configure the ESXi
Servers and create datastores
and virtual machines.
Implementing ESXi
Server and vCenter
Configure the virtual
network.
Install and configure VMware
vCenter and the virtual machine
network.
Implementing ESXi
Server and vCenter
Configure storage for
virtual machines.
Configure the datastores for
virtual machines.
Implementing the
storage array
Prepare the databases.
Set up the Oracle Servers and
create the databases.
Deploying an Oracle
database
Configure the backup.
Set up EMC Avamar® for
database backup.
Implementing EMC
Data Protection
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
Chapter 4: Solution Implementation
Implementing the network
Overview
This section describes requirements for the network infrastructure to support the
solution architecture. Table 7 provides a summary of the tasks for switch and network
configuration along with references for further information.
Table 7.
Tasks for switch and network configuration
Task
Description
Reference
Configure the
infrastructure
network
Configure storage network and host
infrastructure networking as
specified in the solution reference
architecture.
Configuring the
infrastructure network
Configure the
VLANs
Configure private and public VLANs
as required.
 Configuring the VLANs
Configure the
storage network
Connect Fibre Channel (FC) switch
ports and zoning for vSphere hosts
and the storage array.
 Configuring the storage
network
Connect the switch interconnect
ports and vSphere server ports.
Completing the network
cabling
Complete
network cabling
Configuring the
infrastructure
network
 Vendor switch
configuration guide
 Vendor switch
configuration guide
The infrastructure network requires redundant network links for each vSphere host,
the storage array, switch interconnect ports, and switch uplink ports. This
configuration provides both redundancy and additional network bandwidth.
This configuration is required regardless of whether the network infrastructure for the
solution already exists or is being deployed with other components of the solution.
Figure 3 shows an example of redundant network infrastructure for XtremIO storage. It
uses redundant switches and links to ensure that no single point of failure exists in
network connectivity.
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
21
Chapter 4: Solution Implementation
Figure 3.
Configuring the
VLANs
Configuring the
storage network
22
Example redundant XtremIO network
Ensure that there are adequate switch ports for the storage array and vSphere hosts.
EMC recommends that you configure the vSphere hosts with a minimum of two
VLANs:

Client access network—Virtual machine networking

Management network—vSphere management and VMware vMotion (private
network)
This solution used a dedicated FC network. The FC network required redundant FC
switches and links for each vSphere host and the storage array. This configuration
provided both redundancy and additional storage network bandwidth. We connected
each vSphere host to both FC switches, and each switch to each storage processor on
the array. We then placed each FC connection between the vSphere host and the
array in a separate FC zone.
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
Chapter 4: Solution Implementation
Figure 4 shows a sample FC storage network.
Figure 4.
Completing the
network cabling
Sample FC storage network
Ensure that all solution servers, storage arrays, switch interconnects, and switch
uplinks have redundant connections and are plugged into separate switching
infrastructures. Also, ensure that a complete connection is set up to the existing
customer network.
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
23
Chapter 4: Solution Implementation
Implementing the storage array
Overview
This section describes how to configure the XtremIO storage array. Table 8 provides a
summary of the tasks for storage configuration and references for further information.
Table 8.
Tasks for storage configuration
Task
Description
Reference
Prepare and install
XtremIO
 Install the XtremIO hardware
according to the product
documentation.
 EMC XtremIO
 Configure the IP address information
and other key parameters on the
XtremIO array.
 Configure XtremIO event handlers to
send notifications concerning array
physical space utilization.
Create XtremIO
volumes for
vSphere hosts
 Create XtremIO volumes that will be
presented to the vSphere servers as
VMFS datastores hosting the virtual
Oracle server and database files.
Storage Array
Hardware
Installation and
Upgrade Guide
 Preparing and
installing XtremIO
Creating XtremIO
volumes for vSphere
hosts
 Connect the VMware datastores to
the vSphere hosts deployed for the
solution.
Preparing and
installing XtremIO
This solution does not require any specific setup steps to install XtremIO. For
instructions on assembly, racking, cabling, and powering the XtremIO array, refer to
the following documents:

EMC XtremIO Storage Array Site Preparation Guide

EMC XtremIO Storage Array Pre-Installation Checklist

EMC XtremIO Storage Array Software Installation and Upgrade Guide

EMC XtremIO Storage Array Hardware Installation and Upgrade Guide
After preparing the XtremIO array, configure key information about the existing
environment so that the array can communicate with it. Configure the following
common items in accordance with your IT data center policies and existing
infrastructure information:
24

Domain Name System (DNS)

Network Time Protocol (NTP)

XMS network interface

Storage network interfaces

Storage network IP addresses
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
Chapter 4: Solution Implementation
Creating XtremIO
volumes for
vSphere hosts
With XtremIO, all drives are under XtremIO Data Protection (XDP). Random and
sequential I/Os that are typically generated by Oracle databases are treated equally
because data is randomized and evenly distributed within the array. Therefore,
consideration for different RAID groups is not required. As detailed in Table 9, we
configured the required volumes to present to the vSphere servers as VMFS
datastores for different types of Oracle files.
Table 9.
XtremIO volumes for storing virtual Oracle servers
XtremIO volume
Number of
volumes
Volume size(GB)
Purpose of volume
Datafile_vol
4
1024
Deploying Oracle data files
and control files
Logfile_vol
4
50
Deploying Oracle redo log files
Crsfile_vol
2
16
Deploying Oracle Clusterware
Ready Service(CRS) files and
Voting disk files
Frafile_vol
2
512
Deploying Oracle archived log
files and backup files
(optional)
Implementing ESXi Server and vCenter
Overview
This section provides information about installing and configuring the vSphere hosts
and infrastructure servers required to support the architecture. Table 10 describes the
tasks to be completed.
Table 10.
Tasks for server installation
Task
Description
Reference
Install vSphere
Install the vSphere hypervisor on
the physical servers deployed for
the solution.
 vSphere Installation and Setup
Configure
vSphere
networking
Configure vSphere networking
including network interface card
(NIC) trunking, VMkernel ports,
and virtual machine port groups.
 vSphere Networking
Provision
storage for
VMFS
datastores
Add the vSphere hosts to the
initiator groups created and
provision the storage for VMFS
datastores used by ESXi servers
Creating XtremIO volumes for
vSphere hosts
Optimize
vSphere
Perform the configuration
changes needed to ensure
optimal performance of the
XtremIO array.
 EMC XtremIO Storage Array User
Guide
 Installing vSphere
 Configuring vSphere networking
Guide
 Optimizing vSphere for XtremIO
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
25
Chapter 4: Solution Implementation
Installing vSphere
On initial power up of the servers being used for vSphere, confirm or enable the
hardware-assisted CPU virtualization setting and the hardware-assisted memory
management unit (MMU) virtualization setting in the server’s BIOS. If the servers are
equipped with a RAID controller, EMC recommends that you configure mirroring on
the local disks.
Start the vSphere installation media and install the hypervisor on each of the servers.
vSphere hostnames, IP addresses, and a root password are required for installation.
Table 11 describes the installation and configuration requirements to support the
architecture, including ESXi hosts, vCenter, and infrastructure servers.
Table 11.
Installation and configuration requirements
Task
Description
Reference
Install ESXi.
Install the ESXi 5.5 hypervisor on the
physical servers that are being deployed
for the solution.
vSphere Installation and Setup
Configure and deploy
vCenter Server.
Install VMware vCenter Server 5.5.
 vCenter Server and Host
Management
 Installing vCenter Server 5.5 best
practices
26
Configure ESXi
networking.
Configure ESXi networking including NIC
trunking, VMkernel port, virtual machine
port groups, and Jumbo Frames.
vSphere Networking
Create a virtual data
center.
Create a virtual data center.
vCenter Server and Host
Management
Apply vSphere license
keys.
Type the vSphere license keys in the
vCenter licensing menu.
vSphere Installation and Setup
Add ESXi hosts.
Connect vCenter to ESXi hosts.
vCenter Server and Host
Management
Configure vSphere
clustering.
Create a vSphere cluster.
vSphere Resource Management
Install the vCenter
Update Manager plug-in.
Install the vCenter Update Manager plugin from the administration console.
Installing and Administering VMware
vSphere Update Manager
Deploy EMC
PowerPath®/VE
(optional).
Use Update Manager to deploy the
PowerPath/VE plug-in to all vSphere
hosts.
EMC PowerPath/VE for VMware
vSphere Installation and
Administration Guide
Install the EMC VSI for
VMware vSphere plug-in.
Install the VSI for VMware vSphere plugin on the administration console.
EMC VSI for VMware vSphere: Unified
Storage Management— Product
Guide
Install EMC PowerPath
Viewer (optional).
Install PowerPath Viewer on the
administration console.
EMC PowerPath Viewer Installation
and Administration Guide
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
Chapter 4: Solution Implementation
Task
Description
Reference
Perform array ESXi host
discovery.
Perform ESXi host discovery in the EMC
Unisphere® console.
vSphere Resource Management
For details on ESXi and vCenter, refer to the relevant VSPEX Proven Infrastructure
Guide in Essential reading.
Configuring
vSphere
networking
The VMware vSphere Networking guide describes vSphere networking configuration,
including load balancing, link aggregation, and failover options. Select the
appropriate load-balancing option for the network infrastructure.
Network interface cards
During the installation of vSphere, a standard virtual switch (vSwitch) is created. By
default, vSphere chooses only one physical NIC as a vSwitch uplink. To maintain
redundancy and bandwidth requirements, configure an additional NIC, either by
using the vSphere console or by connecting to the vSphere host from the VMware
vSphere Client.
Each vSphere server should have multiple NICs for each virtual network to ensure
redundancy and provide for the use of network load balancing, link aggregation, and
network adapter failover.
VMkernel ports
Create the following VMkernel ports as required, based on the infrastructure
configuration:

VMkernel port for vMotion

Virtual Oracle Server port groups (used by the virtual Oracle servers to
communicate on the network)
The VMware vSphere Networking guide describes the procedure for configuring these
settings. Refer to the list of documents in Reference Documentation for more
information.
Provisioning
storage for VMFS
datastores
Provisioning volumes in XtremIO is simple and straightforward. Complete the
following steps in the EMC XtremIO Storage Management Application to configure
volumes on the XtremIO array for storing virtual Oracle servers:
1.
Click Configuration.
2.
Click Create Volume.
3.
In the Create Volumes window, click Add Multiple.
4.
In Number of Volumes, type the required number of datastores based on the
configuration.
5.
In Name, provide an appropriate name. In Size, type the size as required, and
then click OK.
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
27
Chapter 4: Solution Implementation
To enable vSphere servers to access the XtremIO volumes, configure XtremIO initiator
groups and add the appropriate vSphere hosts to each using the following steps in
the XtremIO Storage Management Application:
1.
Click Configuration.
2.
Under Initiator Groups, click Create Initiator Group.
3.
In the Create Initiator Group window, Click Add, and then type a name in
Initiator Name and Initiator Port Address.
4.
In the Create Initiator Group window, repeat step 3 as needed to add all
target initiator groups.
5.
In the Configuration menu, click Create/Modify Mapping, select the target
volumes, and then select the target initiator group, then click Map All. Click
Finish to complete the process and grant the vSphere servers access to the
selected volumes. Repeat as needed to grant the remaining initiator groups
access to the remaining volumes.
6.
Perform a Rescan for Datastores operation on the vSphere hosts so that they
can immediately discover the XtremIO volumes to which they have access.
Figure 5 shows the volumes presented for different types of Oracle files and the
initiator groups mapping.
Figure 5.
XtremIO volumes mapping to initiator groups
The VMware vSphere Storage guide provides instructions on how to format the
vSphere datastores when XtremIO initiator groups have been configured. Refer to the
list of documents in Reference Documentation for more information.
28
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
Chapter 4: Solution Implementation
Optimizing
vSphere for
XtremIO
Multiple changes are required to ensure optimal performance of the XtremIO array
when used with vSphere. These changes are outlined in the EMC XtremIO Storage
Array User Guide and include:

Change the vSphere Disk.SchedQuantum setting to 64 and the
Disk.DiskMaxIOSize setting to 4096. This can be done using the following
vSphere PowerCLI commands, replacing cluster with the name of the vSphere
cluster where the target vSphere hosts reside. Repeat as needed for each
cluster.
Get-VMhost -location <cluster> | SetVMhostAdvancedConfiguration Disk.SchedQuantum -Value 64
Get-VMhost -location <cluster> | SetVMhostAdvancedConfiguration Disk.DiskMaxIOSize -Value 4096

Change the vSphere Disk.SchedNumReqOutstanding setting to 256 for each
vSphere datastore. This can be done using the following vSphere PowerCLI
script. Repeat as needed for each vSphere cluster, updating the script with the
target cluster name.
$vmhosts = get-vmhost –location <cluster_name>
foreach ($vmhost in $vmhosts) {
$esxcli = get-esxcli -vmhost $vmhost
$AllLUNs = get-scsilun -vmhost $vmhost | where {$_.vendor -eq
"XtremIO"}
foreach ($lun in $AllLUNs) {
$CN = $lun.canonicalname
$EsxCli.storage.core.device.set($null,
$cn, $null, $null, $null, $null, $null, 256, $on)
}
}
Note: The settings described in this section apply only to vSphere hosts connected to EMC
XtremIO arrays, and should not be applied to block datastores hosted on other arrays,
including other EMC arrays.
Deploying an Oracle database
Overview
Table 12 describes the tasks for deploying the VSPEX for virtualized Oracle Database
11g or 12c solution. When the tasks are complete, Oracle Servers are set up on virtual
machines with all the database files—including data files, control files, online redo
files, voting disk files, Oracle CRS files, and other files—using the datastores
designed for this VSPEX infrastructure.
Note: EMC recommends that you put the operating system (OS) volume for the Oracle Server
virtual machine into the VSPEX private cloud pool, if available.
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
29
Chapter 4: Solution Implementation
Table 12.
Tasks for deploying an Oracle database
Task
Description
Reference
Prepare the virtual
machine template.
 Create a virtual machine template with the
requirements and prerequisites for the
Oracle Database 11g or 12c installation.
 vSphere Virtual Machine
Administration
 Enable the HugePages setting. The
HugePages feature enables the Linux kernel
to manage large pages of memory in
addition to the standard 4KB (on x86 and
x86_64) or 16KB (on IA64) page size.
 Preparing the Oracle virtual
machine template
 Install the Oracle 11g or 12c CRS and
database binaries and patches on the
virtual machine.
Deploy the database
and perform the
database postcreation activities.
Deploy and configure the database.
 Creating a database using
the Database Configuration
Assistant
 Configuring the database file
layout
 Configuring the initialization
parameters
 Performing the database post
creation activities
Preparing the
Oracle virtual
machine template
This section documents the preparation of a VMware virtual machine template to
enable rapid deployment of new Oracle Database 11g or 12c environments.
VMware provides the ability to create a template that you can use as a master copy to
quickly create and provision virtual machines. By using a template, you can install a
guest OS and apply it to a virtual machine with application users and software
configured and ready for use with minimal administrative intervention. This
minimizes deployment time and avoids repetitive installation and configuration tasks
for each virtual machine that is required.
To configure the template with the requirements and prerequisites for the Oracle
software installation, complete the following steps:
1.
30
Create a virtual machine with the following resources:

1 vCPU

4 GB of vRAM

100 GB virtual disk
2.
Install the OS and Red Hat Package Manager (RPM) packages.
3.
Configure the system and kernel parameters.
4.
Create the OS groups and users.
5.
Install the Oracle CRS and database binaries ready for clone.
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
Chapter 4: Solution Implementation
6.
Convert the virtual machine to a VMware template.
Table 13 provides an example of a virtual machine template specification.
Table 13.
Example of a virtual machine template specification
Component
Description
CPU
1 vCPU
Memory
4 GB
Virtual disk
100 GB with:
 8 GB for the swap partition
 92 GB for the bootable root partition
Operating system
Install the RHEL 6.5 64-bit OS and configure the NICs.
Kernel
2.6.32
RPM packages
 Install or update required RPM packages and configure the
kernel with Oracle-specific settings for semaphores, virtual and
shared memory, network, and asynchronous I/O.
 Check and adjust the resource limits for the Oracle software
installation users.
Refer to Oracle Database Quick Installation Guide for Linux x86-64
for more details.
Network interfaces
 Eth0: public/management IP network
 Eth1: Storage network
OS users
Username: oracle
OS groups
Group names: oinstall and dba
VMware tool
Install VMware tool using RPM
Oracle CRS and DB
binaries
11.2.0.4 or 12.1.0.2
HugePages
HugePages is crucial for faster Oracle database performance on
Linux if you have a large RAM and SGA. If your combined database
SGAs are large (more than 8 GB), you need to configure HugePages.
The size of the SGA is important.
The latest version of My Oracle Support Note ID 1392497.1:
USE_LARGE_PAGES To Enable HugePages on the Oracle website
provides more details.
You can deploy a virtual machine from a template with custom by manually selecting
Deploy Virtual Machine from the template in vCenter. Select Customize and use an
existing customization specification.
Use the VSPEX for Virtualized Oracle Qualification Worksheet in the Design Guide to
size the virtualization infrastructure for the Oracle database. Determine the size of the
individual data files based on the schema design and table space placement of
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
31
Chapter 4: Solution Implementation
tables and indexes. Refer to Creating XtremIO Volumes for vSphere Hosts for more
details.
Deploying and
configuring a
database
This solution used Oracle Automatic Storage Management (ASM) to store the relevant
database files, including data files, online redo log files, and CRS files.
Creating a database using the Database Configuration Assistant
The Oracle Database Configuration Assistant (DBCA) is the preferred way to create a
database because it is a more automated approach and the database is ready to use
when DBCA completes. Start DBCA as a standalone tool to create an Oracle database,
as shown in Figure 6.
Figure 6.
Oracle Database 12c Configuration Assistant setup
To start DBCA on UNIX, Linux, or at a Windows command-line prompt, type dbca and
press Enter.
Note: The dbca utility is typically located in the ORACLE_HOME/bin directory.
Oracle DBCA runs in interactive mode. Using the GUI in Oracle DBCA, you can either
modify one of the preconfigured database types or customize the database.
General Purpose/Transaction Processing is a preconfigured database template that
you can create and customize during the installation.
32
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
Chapter 4: Solution Implementation
Configuring the database file layout
After Oracle ASM and Oracle CRS were installed, we deployed an Oracle database
using the ASM diskgroup, which stored data on the previously created XtremIO
volumes.
Table 14 details the storage design on XtremIO array for an Oracle database.
Table 14.
Storage design to ASM disk group mapping for Oracle database files
XtremIO
volume
ASM diskgroup
Purpose
Datafile_vol
+DATA
Oracle ASM disks for data files
Logfile_vol
+REDO
Oracle ASM disks for redo log files
Crsfile_vol
+CRS
Oracle ASM disks for Cluster Registry
and Voting disk files
Frafile_vol
+FRA
Oracle ASM disks for archived log files
and backup files
(optional)
Figure 7 shows how we configured the database data file using the ASM diskgroup
+DATA.
Figure 7.
Storage locations using ASM diskgroup
For more information about the storage layout recommendations and design, refer to
the Design Guide.
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
33
Chapter 4: Solution Implementation
Configuring the initialization parameters
To configure the initialization parameters, you can use one of the following methods:

Automatic Shared Memory Management (ASMM)

Automatic Memory Management (AMM)

I/O operations for file system files
Automatic Shared Memory Management
Implement ASMM to automate the management of shared memory structures. The
following shared memory structures are automatically managed:

DB_CACHE_SIZE

SHARED_POOL_SIZE

LARGE_POOL_SIZE

JAVA_POOL_SIZE

STREAMS_POOL_SIZE
When implemented, ASMM enables the Oracle database to distribute memory
between these components based on workload requirements. Therefore, these
components are considered autotuned parameters. The background process, Memory
Manager (MMAN), coordinates the sizing of these components and moves memory to
where it is needed most.
To implement the ASMM feature, set the following initialization parameters:

SGA_TARGET—Set to a nonzero value.

STATISTICS_LEVEL—Set to TYPICAL (or ALL).
Automatic Memory Management
Do not use AMM because AMM is incompatible with HugePages. If you want to use
HugePages, ensure that both MEMORY_TARGET and MEMORY_MAX_TARGET
initialization parameters are not set.
For more information, refer to My Oracle Support Note ID 749851.1 on the Oracle
website.
I/O operations for file system files
Other recommended database parameter settings include:
Performing the
database post
creation activities
34

Set DISK_ASYNCH_IO= true. The default value for this asynchronous I/O in
both Oracle 11g and 12c is true.

Set FILESYSTEMIO_OPTIONS=SETALL. This setting enables both direct I/O and
asynchronous I/O. With asynchronous I/O, processing continues while the I/O
request is submitted and processed.
After you create a database as described in Creating a database using the Database
Configuration Assistant, the instance is open and available for normal database use.
However, the Oracle database will not start up or shut down automatically with the
virtual machine.
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
Chapter 4: Solution Implementation
To automate the Oracle database startup and shutdown process, refer toHow to
Automate Startup/Shutdown of Oracle Database on Linux (ID 222813.1) for more
details.
Implementing EMC Data Protection
All VSPEX solutions are sized and tested with EMC Data Protection products,
including EMC Avamar and EMC Data Domain®. If your solution includes backup
components, refer to EMC Backup and Recovery Options for VSPEX for Virtualized
Oracle Database 11g R2 Design and Implementation Guide for detailed information
on implementing these options into your VSPEX solution.
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
35
Chapter 5: Solution Verification
Chapter 5
Solution Verification
This chapter presents the following topics:
Overview ..................................................................................................................37
36
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
Chapter 5: Solution Verification
Overview
This section provides a list to review after configuring this solution. The goal of this
section is to verify the configuration and functionality of specific aspects of the
solution, and to ensure that the configuration supports core availability requirements.
Table 15 describes the required verification tasks.
Table 15.
Tasks for verifying the VSPEX installation
Task
Description
Reference
Verify installation
with a postinstallation checklist
Verify that adequate virtual ports
exist on each vSphere host virtual
switch.
 vSphere Storage Guide
 vSphere Networking
Verify that each vSphere host has
access to the required datastores
and VLANs.
Deploy and test a
single virtual Oracle
server
Verifying
redundancy of the
solution
components
Deploy a single virtual machine
from the vSphere interface by
using the customization
specification.
 vCenter Server and Host
 Restart each storage processor
in turn and ensure that
connectivity to the VMware
datastores is maintained.
 Vendor documentation
Management
 vSphere Virtual Machine
Management
 vCenter Server and Host
Management
 Disable each of the redundant
switches in turn and verify that
the vSphere host, virtual
machine, and storage array
connectivity remains intact.
 On a vSphere host that contains
at least one virtual machine,
enable maintenance mode and
verify that the virtual machine
can successfully migrate to an
alternate host.
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
37
Chapter 5: Solution Verification
Using the health
monitoring tools
Health monitoring is a simplified measurement that reflects the reliability, stability,
and performance of the entire VSPEX Proven Infrastructure for a virtualized Oracle
database. Table 16 lists some tools you can use to monitor and troubleshoot your
VSPEX solution.
Table 16.
Tools for monitoring the solution
Tool
Description
Esxtop
The esxtop tool provides a real-time view (updated every five
seconds, by default) of ESXi Hypervisor performance metrics, such
as CPU. These performance metrics enable you to measure and
troubleshoot Oracle database performance-related issues.
XtremIO Storage
Management
Application
XtremIO Storage Management Application is preloaded on XtremIO
storage systems. It is used to manage the XtremIO array and
monitor, collect, and analyze the all aspects of storage related data
from it.
Automatic
Workload
Repository
The Automatic Workload Repository collects, processes, and
maintains performance statistics for problem detection and selftuning purposes. This data is both in memory and stored in the
database. The gathered data can be displayed in both reports and
views.
EMC recommends that you use these tools to monitor the performance of the entire
Oracle database environment. For details about performance verification and test
methodology, refer to the Design Guide.
38
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
Chapter 6: Reference Documentation
Chapter 6
Reference Documentation
This chapter presents the following topics:
Reference documentation ........................................................................................ 40
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
39
Chapter 6: Reference Documentation
Reference documentation
EMC documents
Oracle documents
VMware
documents
The following documents, located on EMC Online Support provide additional and
relevant information. Access to these documents depends on your login credentials.
If you do not have access to a document, contact your EMC representative.

EMC XtremIO Storage Array User Guide

EMC XtremIO Storage Array Operations Guide

EMC XtremIO Storage Array Software Installation and Upgrade Guide

EMC XtremIO Storage Array Hardware Installation and Upgrade Guide

EMC XtremIO Storage Array Security Configuration Guide

EMC XtremIO Storage Array Pre-Installation Checklist

EMC XtremIO Storage Array Site Preparation Guide

EMC VSI for VMware vSphere: Storage Viewer Product Guide

EMC VSI for VMware vSphere: Unified Storage Management Product Guide

EMC PowerPath/VE for VMware vSphere Installation and Administration Guide
The following documents are available on the Oracle website:

Oracle Database Documentation Library 12c Release 1 (12.1)

Oracle Database Documentation Library 11g Release 2.11.2

Oracle Database Release Notes for Linux

See My Oracle Support for the latest O Patch and latest PSU.
The following documents are available on the VMware website:

VMware vSphere vCenter Server Installation and Setup documentation

Performance Best Practices for VMware vSphere 5.5

VMware vSphere Networking documentation

VMware vSphere Storage documentation

VMware vSphere Resource Management documentation

Automating High Availability (HA) Services with VMware HA Technical Note

Installing VMware Tools in a Linux virtual machine using Red Hat Package
Manager (RPM) is available in the VMware Knowledge Base (Article ID:
1018392).
40
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
Chapter 6: Reference Documentation
Links
The following web pages provide additional references for this solution:

Everything Oracle at EMC:
https://community.emc.com/community/connect/everything_oracle

My Oracle Support website: https://support.oracle.com (user account required)

General VMware documentation: http://www.vmware.com

vSphere documentation: http://www.vmware.com/support/pubs/vsphereesxi-vcenter-server-pubs.html

vSphere PowerCLI documentation:
http://www.vmware.com/support/developer/PowerCLI/
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
41
Appendix A: Configuration Worksheet
Appendix A
Configuration Worksheet
This appendix presents the following topic:
Customer Configuration Worksheet .........................................................................43
42
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
Appendix A: Configuration Worksheet
Customer Configuration Worksheet
Before configuring an Oracle database for this solution, you need to gather some
customer-specific configuration information, such as IP addresses, hostnames, and
so on. The following tables provide a worksheet that you can use to record the
information. You can also print and use the worksheet as a customer “leave behind”
document for future reference.
Table 17.
Common server information
Server name
Purpose
Primary IP address
Domain Controller
DNS Primary
DNS Secondary
DHCP
NTP
SMTP
SNMP
VMware vCenter Server
Oracle database server
Table 18.
ESXi server information
Server name
Purpose
Primary IP
address
Private net (storage) addresses
ESXi
Host 1
ESXi
Host 2
…
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
43
Appendix A: Configuration Worksheet
Table 19.
XtremIO array information
Array name
Array Name
XMS IP
Storage Controller IPs
Initiator group names
Datastore names
Virtual machine infrastructure
pool name
Virtual machine infrastructure
file system name
Data file storage pool name
Data file file system name
Redo log storage pool name
Redo log file system name
CRS storage pool name
CRS file system name
Table 20.
Name
Network infrastructure information
Purpose
IP address
Subnet mask
Default
gateway
Ethernet Switch 1
Ethernet Switch 2
…
Table 21.
Name
VLAN information
Network purpose
VLAN ID
Client access network
Storage network
Management network
44
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
Allowed subnets
Appendix A: Configuration Worksheet
Table 22.
Account
Service accounts
Purpose
Password (optional, secure
appropriately)
XtremIO array root account
ESXi server administrator
vCenter Server administrator
Linux Server administrator
Oracle database administrator
EMC VSPEX Oracle Computing: Oracle Database Virtualization
with VMware vSphere and EMC XtremIO Implementation Guide
45