Download High Availability and Disaster Recovery with the HP Enterprise

Microsoft® SQL Server® Technical Article
High Availability and Disaster Recovery with the HP
Enterprise Database Consolidation Appliance
Writers: Robert Zare (Microsoft), Dean Gjedde (Protingent)
Contributors: Ravi Chinthambureddy (Microsoft), Steven Wort
(Microsoft), Joe Sullivan (Hewlett Packard)
Published: April 2012
Applies to: SQL Server 2008 R2
Summary: This white paper details the built-in high availability and
disaster recovery capabilities of the HP Enterprise Database
Consolidation Appliance. In addition to this, the process and
recommended topology for backing up and restoring the appliance
is covered in detail.
Copyright
This document is provided “as-is”. Information and views expressed in this document, including URL and
other Internet Web site references, may change without notice; you bear the risk of using it.
Some examples depicted herein are provided for illustration only and are fictitious. No real association
or connection is intended or should be inferred.
This document does not provide you with any legal rights to any intellectual property in any Microsoft
product. You may copy and use this document for your internal, reference purposes.
© 2012 Microsoft. All rights reserved.
Contents
Introduction .................................................................................................................................................. 5
Data Protection Manager 2010 ................................................................................................................ 5
Overview ................................................................................................................................................... 5
Benefits ..................................................................................................................................................... 6
Configurations ........................................................................................................................................... 6
Management Stack ................................................................................................................................. 10
High Availability Features ........................................................................................................................... 10
Built-In Hardware Attributes for High Availability .................................................................................. 10
Disk Redundancy ................................................................................................................................. 10
Networking Redundancy ..................................................................................................................... 11
Power System Redundancy................................................................................................................. 13
Built-In Software Attributes for High Availability ................................................................................... 13
Host Clustering .................................................................................................................................... 13
Guest Clustering .................................................................................................................................. 14
Backup and Restore .................................................................................................................................... 14
Overview ................................................................................................................................................. 14
Getting Started........................................................................................................................................ 14
Typical Backup and Restore Durations ............................................................................................... 14
Backup Storage Recommendations .................................................................................................... 15
Prerequisites ....................................................................................................................................... 15
Configuring the Management Stack and Primary DPM Server ........................................................... 16
Configuring the Secondary DPM Server.............................................................................................. 18
Creating Protection Groups ................................................................................................................ 19
Performing On-Demand Backups ........................................................................................................... 21
Performing Restores ............................................................................................................................... 23
Clustered Guest Restore Considerations ............................................................................................ 23
Virtual Machine Restores .................................................................................................................... 24
SQL Server Recovery ........................................................................................................................... 29
Bare Metal Recovery ........................................................................................................................... 29
Full Appliance Recovery ...................................................................................................................... 33
Disaster Recovery........................................................................................................................................ 34
Appliance Requirements ......................................................................................................................... 34
Software Requirements .......................................................................................................................... 34
Data Protection Manager ....................................................................................................................... 34
SQL Server ............................................................................................................................................... 34
Database Mirroring ............................................................................................................................. 35
Log Shipping ........................................................................................................................................ 35
Conclusion ................................................................................................................................................... 35
Appendix ..................................................................................................................................................... 36
DBC General ............................................................................................................................................ 36
High Availability ...................................................................................................................................... 36
Data Protection Manager 2010 (DPM) ................................................................................................... 37
Introduction
The HP Enterprise Database Consolidation (DBC) Appliance Optimized for SQL Server is a highly scalable
combination of hardware and software that utilizes virtualization and large amounts of storage to
reduce data center and facilities operation costs, improve overall performance, and centralize the
support and management of business critical and standard instances of Microsoft SQL Server for an
organization into a single appliance.
There are myriad hardware and software-based features focused on high availability, which are
explained in detail in this white paper. Additionally, we discuss how to leverage the on-board backup
solution, Microsoft System Center 2010 - Data Protection Manager (DPM), to back up and restore all
software components of the appliance. Lastly, we address disaster recovery strategies in the context of
the appliance.
Data Protection Manager 2010
Data Protection Manager 2010 is preinstalled on the appliance and, as mentioned earlier, it provides the
core backup and restore functionality for the appliance. Because this paper assumes a working
knowledge of the DPM product, it often omits the ‘how’ piece of the story. If necessary, you can review
the following links to better understand DPM prior to implementing the procedures described here:




DPM Overview: http://www.microsoft.com/download/en/details.aspx?id=6287
Protecting SQL Server with DPM 2010:
http://www.microsoft.com/download/en/details.aspx?id=9744
How to Protect Hyper-V with DPM 2010:
http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=14575
Data Protection Manager 2010 Operations Guide:
http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=15193
Overview
The HP Enterprise Database Consolidation (DBC) Appliance Optimized for SQL Server helps to simplify
the deployment and management of a virtualized infrastructure. It is designed specifically for SQL Server
databases. The DBC Appliance enables the consolidation of hundreds of database workloads into a
single virtual environment. New SQL Server instances can be provisioned in minutes therefore drastically
reducing infrastructure stand-up time. The net result is an easy-to-manage, cost-efficient solution that
delivers agility to your business. SQL Server sprawl has become abundant in most enterprise
infrastructures; this appliance was designed to help eliminate underutilized servers resulting in a cost
savings for data center space, power utilization, and administration.
The integrated hardware and software offering provides a simple, turn-key experience. The DBC
Appliance comes with an integrated management software stack from HP and Microsoft and is designed
to provide compute, storage, and network resources inside a single rack. An easy-to-use centralized
management console is included to assist in provisioning, migration, and monitoring of your servers. The
management tools are preinstalled on the appliance, and licenses are sold separately by HP or Microsoft
to offer greater flexibility to customers.
Benefits
Built-in hardware and software based high availability



Reserved guest cluster capacity.
Hyper-V live migration, which enables you to seamlessly move virtual machines (VMs) from one
host to another.
Full hardware redundancy for switches, power, storage, and blades. Key server components
such as power supplies, fans, and storage are hot swappable.
Reduce database sprawl

Pooled system resources enable you to reduce up to 70 percent of resources like power,
cooling, and space by consolidating thousands of databases into a single appliance using pooled
system resources.
Simplify provisioning, management, and maintenance



Because this solution is built on Windows Server, Microsoft System Center, and Microsoft SQL
Server technologies, infrastructure management is simplified.
The Microsoft Assessment and Planning (MAP) toolkit makes it easier to discover, size, and place
SQL Server databases across your network onto the appliance.
One-Button-Update simplifies firmware and driver updates.
Accelerated time to value


Complete consolidation platform is virtualized and factory-built with integrated hardware and
management software to accelerate time to value.
Efficiently pools and allocates resources to run thousands of virtual machines (VMs) that host
SQL Server instances.
Enhanced business agility



The solution scales seamlessly as your business grows.
It is easy to provision new VMs or add storage as databases grow.
Microsoft Self Service portal automates provisioning of VM and SQL Server databases.
Coordinated HP and Microsoft support


All components are supported by both Microsoft and HP, with Microsoft being the first point of
contact.
Your support experience is cohesive.
Configurations
Configuration options span a single half-rack and scale to 10 full racks.
TYPICAL HALF AND FULL RACK CONFIGURATION SPECIFICATIONS
CORES
MEMORY
DRIVES
RAW DISK SPACE
FORMATTED DISK SPACE
ACHIEVABLE IOPS
HALF-RACK CONFIGURATION
96
1 TERABYTE
206
29 TERABYTES
13 TERABYTES
30,000 8KB @ 60/40 R/W
FULL-RACK CONFIGURATION
192
2 TERABYTES
412
57 TERABYTES
25 TERABYTES
60,000 8KB @ 60/40 R/W
Half-Rack Configuration
Full-Rack Configuration
Management Stack
The DBC Appliance is managed by the management stack in Microsoft System. Ten racks can be
managed by one System Center management stack as a single logical appliance. The components used
along with their uses are:





System Center Operations Manager
o Server/network alerting and monitoring
System Center Virtual Machine Manager (VMM)
o Hyper-V/Host management
System Center Virtual Machine Manager Self-Service Portal (VMMSSP)
o Pool, allocate, and manage resources to offer infrastructure as a service and to deliver
the foundation for a private cloud platform inside your data center.
System Center Data Protection Manager (DPM)
o Backup and restore services
System Center Configuration Manager
o Software distribution and patching
High Availability Features
Built-In Hardware Attributes for High Availability
The appliance hardware configuration is designed around a goal of eliminating any single point of
failure. Built-in hardware features include redundant disks, networking, power supplies, and fans. These
features are described in detail here.
Disk Redundancy
The DBC Appliance is built with the concept of a storage block. Each storage block consists of a P2000 G3
iSCSI array controller shelf and three attached D2700 drive shelves. In a half-rack appliance
configuration there are two storage blocks and in a full rack design there are four storage blocks.
In each storage block there are six RAID 10 VDisks carved out as shown in the following picture. Five of
these VDisks are utilized as independent cluster shared volumes (CSVs). The sixth VDisk is reserved as
space to be used for either shared disk for guest clustering or it can be used as additional space for more
virtual machines. Note that while the spare volume is available for custom use, we recommend that you
not use this volume for backups.
Networking Redundancy
The DBC Appliance comes with a fully redundant network stack right out of the box. In each rack (both
half and full rack configurations) there are two 10-GB/s switches and two 1-GB/s switches. Networking
redundancy between the switches and the rest of the appliance infrastructure is achieved through Link
Aggregation Protocol (LACP) support and with redundant HP Virtual Connect VC Flex-10 modules. At the
host level, the physical NICs are teamed together to provide maximum throughput and availability.
In addition to the physical network infrastructure having redundancy, there is also redundancy at the
virtual level. Each host has redundant Hyper-V Virtual Switches with each guest’s Hyper-V Virtual NICs
are connected to each Virtual Switch. This end-to-end redundancy ensures the maximum protection
from potential network-related issues. The following diagram shows an example of the mapping of
several of the appliance networks. Note, not all networks are displayed in the diagram for ease of
viewing.
Power System Redundancy
Along with the other components, the power infrastructure is also designed to support high availability.
With dual power supplies for all of the hardware components and redundant power distribution units
(PDUs), the DBC Appliance has two completely independent power paths, each capable of supporting a
full 100 percent of the system’s requirements should it be necessary. This represents true 2N
redundancy. Many off-the-shelf setups present the appearance of a redundant power system, but in
reality doesn’t provide full paths from each PDU. This consolidation appliance is configured with four
distinct PDUs, each connecting to a separate power circuit.
Built-In Software Attributes for High Availability
The HP Enterprise Database Consolidation Appliance features several software-based solutions to help
achieve high availability; they are built into the management stack.
Host Clustering
Host clustering in conjunction with Hyper-V is used to increase the availability of workloads as well as
the applications that they host. A half-rack configuration contains a 4-node host cluster, and a full-rack
configuration contains an 8-node host cluster. The cluster ensures that the loss of a single host will not
impact appliance availability.
The host cluster enables migration of VMs across these physical hosts via live migration. This capability
enables seamlessly moving VMs to alternate hosts to account for patching, upgrade, or load balancing
scenarios.
Guest Clustering
Clustering should be used when your application requires minimal downtime and automatic health
detection for services or applications. Setting up clustering is also a viable option if you want to keep
downtime to a minimum while you patch the guest nodes in the cluster, because you can fail over
resources and applications to a vacant node within the guest cluster while you perform maintenance.
By default, virtual machines in the appliance are not set up as clustered. Instead, they are all stand-alone
instances of Windows. We do not recommend having more than 15-20 percent of the guest VMs in the
appliance set up for clustering, based on the default amount of nonprovisioned storage. You should
consider using this guideline if you need to maximize consolidation density within the appliance. Our
suggested guideline is not a hard requirement, because you can reprovision existing storage capacity if
needed.
Backup and Restore
Overview
Included as part of the DBC management stack, System Center 2010 Data Protection Manager (DPM) is
the recommended tool for backing-up the appliance software stack. We do not recommend using the
storage blocks for backup purposes; rather we recommend that you present storage (for example, SAN,
NAS) from your local network to the appliance. If you have an existing enterprise backup solution in
place such as NetBackup, CommVault, or Tivoli, you can leverage those solutions with the Database
Consolidation Appliance. Lastly, there is a dedicated onboard backup network, which you should use
exclusively for backups in order to avoid impacting performance of running workloads.
Getting Started
You should back up the entire appliance as soon as possible after deployment is complete to ensure full
rollback capability to a known good state, should circumstances require this. It’s important to note that
all backups should be complete prior to any workloads being migrated unto the appliance.
We recommend that you provision a second, local DPM server, at a minimum, for the purposes of
backing-up the primary DPM server. If you need to account for a data center failure scenario, the
secondary DPM server should be located in a geographically separate location and configured to back up
the primary DPM server and all of its assets. After both primary and secondary backups are complete,
you can safely begin the process of moving workloads onto the appliance.
Typical Backup and Restore Durations
In our lab environment, we’ve observed the initial backups for both primary and secondary DPM servers
to take between four and six hours. A full appliance restoration takes between six and eight hours.
Restoring an individual management VM takes roughly fifteen minutes or approximately 2 GB per
minute.
Backup Storage Recommendations
Each instance of DPM that is backing up the appliance requires a dedicated storage pool per instance of
DPM. The recommended storage requirement is a minimum of 5 terabytes, but we recommend at least
10 terabytes of storage per DPM instance in order to accommodate backups of both the appliance hosts
and management VMs. This storage space accommodates a daily backup of all appliance-critical
resources with room for three to four backup sets for each resource. Note that this requirement does
not take into account storage required for any user-created VMs on the appliance. Additional storage is
needed to accommodate those backups as well.
Network (WAN)
Network Router/edge
Minimum 5 TB of disk space for backups (per
DPM)
Recommend at least 10 TB (daily of all / 3-4
backup sets)
“Initial sync”
Inconsistency (disk space usual culprit)
Appliance
Primary DPM
Network Router/edge
Minimum 5 TB of disk space for backups (per
DPM)
Recommend at least 10 TB (daily of all / 3-4
backup sets)
“Initial sync”
Inconsistency (disk space usual culprit)
Datacenter 2
Secondary DPM
(Physical or Virtual)
Datacenter 1
Prerequisites
Before you get started, it’s important to insure that the appliance has been fully deployed and that
you’ve identified and provisioned the necessary external resources for DPM:
1. Fully complete the deployment process outlined in the DBC Deployment Guide.
2. Identify dedicated physical or virtual hardware to host the secondary DPM server. This server
needs to be connected to both the DBC management and backup networks.
3. Identify dedicated storage for both primary and secondary DPM servers; this is likely to be some
type of network-based storage.
4. Provision static IP addresses for the backup network. You will need 14 IP addresses for a half
rack appliance and 22 for full rack.
Configuring the Management Stack and Primary DPM Server
The primary DPM virtual machine will be present on the appliance. Use the System Center Virtual
Machine Manager on the appliance control node to locate the DPM VM. Next, update and optimize the
DPM server configuration, and ready the management stack to make use of the built-in backup network.
This section walks you through that process using examples.
Shut down and configure Active Directory VMs
Starting with the Active Directory VM on the last host (for example, host 4 on a half rack), shut the
virtual machine down and add a new virtual NIC. Associate the virtual NIC with the Hyper-V virtual
switch Backup-Team, and then restart the Active Directory VM. After this is complete, follow the same
procedure for each host in descending order (in the half rack example, host 3).
Note: It’s very important to make these changes one virtual machine at a time.
Lastly, power the Active Directory VMs back up and assign the new NICs static IP addresses for the
backup network.
Shut down and configure SQL Server Guest Cluster VMs
Consider a configuration in which SQL Server database for System Center Virtual Machine Manager is
deployed as a two-node guest cluster. The host virtual machines are referred to as VDB1 and VDB2.
VDB1 is the primary virtual machine.
Start with VDB2. Shut down the virtual machine, add a new virtual NIC, and then associate the new NIC
with the Hyper-V virtual switch backup team. Restart the virtual machine, and then assign the new NIC a
static IP address for the backup network. When this process is complete for VDB2, repeat the same
process for VDB1.
Shut down and configure SDB1 and SDW1 VMs
Consider a configuration in which the System Center Operations Manager database and data warehouse
are hosted on dedicated virtual machines. SDB1 is the virtual machine that hosts the database, and
SDW1 is the virtual machine that hosts the data warehouse; the application server is SOM1.
To begin with, shut down the SOM1 VM in preparation for powering down the database VMs. Next, shut
down both SDB1 and SDW1, and then add new virtual NICs. Associate the virtual NICs with the Hyper-V
virtual switch Backup-Team. Restart the VMs, and then assign the new NIC a static IP address for the
backup network. Restart the SOM1 VM.
Validate Configuration Changes
As part of the deployment process the all of the hosts and DPM VM should have backup network static
IP addresses to those NICs. Check DNS inside the primary Active Directory VM and verify that new
addresses registered correctly. This may take as long as 30 minutes.
Configure Data Protection Manager
Shut down the DPM virtual machine and resize the VM as follows:
 VProcs = 4


Physical memory = 16GB
Pagefile = 36 GB
After this process is complete, restart the DPM virtual machine.
Next, open the DPM Management Shell window on the primary DPM server and run the following
command:
Add-BackupNetworkAddress -DpmServername <appliancename>-<racknumber>-dpm1.dbc01.local Address 173.16.0.0/16 -SequenceNumber 1”
Install the latest hotfix rollup package for DPM: http://support.microsoft.com/kb/2465832. When
installation is complete, open the DPM management console and configure the DPM storage pool. Note
that the underlying disks cannot be formatted or have a drive letter: they ought to show in disk manager
as being ‘unallocated’. We recommend 5 terabytes as the minimum initial size of the storage pool.
Lastly, configure DPM to publish alerts to Event Viewer so that the SCOM agent can harvest these
events. You do this in the Options dialog box, on the Alert Publishing tab. To verify that events are being
published, in Event Viewer, expand Applications and Services logs and look for DPM Server.
Configure DPM Agents
For each host server:


Install the Windows Server Backup feature and all subfeatures.
Enable AutoMount by running DiskPart and subsequently entering the following command:
automount [ { enable | disable } ] [scrub] [noerr
Next, install DPM agents:


One agent per host
One inside each of the following VMs:
o SCOM DB
o SCOM DW
o VMM SQL DB 1
o VMM SQL DB 2
DPM agents can typically be installed using the DPM administrator console via the Agents tab. In some
cases, you may need to install agents manually. For more information, see Installing Protection Agents
Manually (http://technet.microsoft.com/en-us/library/ff399459.aspx). Note that when you install the
agents manually, the fully qualified domain name (FQDN) of the DPM server must be used in the
command. To complete the agent installation process, restart all VMs and hosts.
Configuring the Secondary DPM Server
Configuring the secondary DPM server is similar to setting up the primary DPM, because settings and
configurations should be mirrored across the two. If the primary DPM server fails, or, in the worst case,
if a data center fails, the secondary DPM server takes on responsibilities for backup and restore.
Ensure that the latest hotfix rollup package is installed. Configure the DPM storage pool as you did
earlier. Next, assign static IP addresses to the NIC that will be attached to the backup network, and then
join the DPM server to the DBC Appliance domain. Install the secondary DPM server’s agent on the
primary DPM server.
Creating Protection Groups
Protection groups dictate the frequency and retention policy for one or more protected assets. Assets
within a protection group share a common protection configuration, including:




Data backup targets (disk or tape destinations).
A protection schedule that specifies how often you wish to synchronize the replica with live
data.
When to create recovery points on the replica.
Performance options you want to enable, such as compression.
When you plan your protection groups in Data Protection Manager, review the following considerations:




The business requirements of the organization:
o Service-level agreements (SLAs)
o Data retention needs
Network performance
Primary characteristics of the data:
o How often does the data change?
o How drastically does the data change? That is, are the changes small or large?
o Are there data size changes?
o How critical is the data?
o What is the appropriate retention period for the data?
o Which data sources will belong to the protection group?
What protection method will you use? Disk-based? Tape-based? Both?
While backup frequency and retention period can be modified as needed to suit business needs, we
recommend that you not deviate from the content of the protection groups described here because
these have been defined in a way that makes optimal usage of the underlying storage configuration.
Primary DPM Server Protection Groups
The following tables illustrate the recommended protection groups. You can modify the respective
names and backup frequencies as needed; we do not recommend modifying the contents of the
protection groups.
‘CSV3’ Protection Group
Protected Assets
Backup Frequency
‘CSV8’ Protection Group
Protected Assets
Backup Frequency
‘Active Directory’ Protection Group
Protected Assets
Control Node VM
Operations Manager Reporting VM
Internet Information Services (IIS) VM
Weekly
VMM SQL Server DB VM (VDB2)
Weekly
All Active Directory VMs
Backup Frequency
Daily
‘VMM SQL Server Data Protection Group
Protected Assets
Backup Frequency
VMM SQL Server Databases
30 minutes
‘SCOM Data’ Protection Group
Protected Assets
Backup Frequency
Operations Manager Data Warehouse Database
Operations Manager Application Database
30 minutes
‘Slowly Changing Management Components’ Protection Group
Protected Assets
Operations Manager Application VM
Operations Manager DB VM
Operations Manager DW VM
VMM VM
VMM SQL Server DB (VDB1)
Configuration Manager VM
DPM VM
Backup Frequency
Weekly
‘Bare Metal’ Protection Groups
Protected Assets
Host image
Backup Frequency
Weekly
Note: Each bare metal backup should be placed in its own protection group and offset in time to avoid
excessive CPU consumption. Additionally, the default Replica Volume Size property value must be
changed to 200 GB from within the DPM Protection Group Wizard for both primary and secondary DPM
servers.
Secondary DPM Server
Create two protection groups on the secondary DPM server in order to provide full protection for the
assets under the watch of the primary DPM server:
1. Bare Metal Backups
2. All other protected assets (includes primary DPM server)
As noted in the overview, the minimum recommended configuration is an onsite secondary DPM server
that protects the primary DPM server.
Optimizing Backup Performance

When creating protection groups, enable the options to co-locate data and perform autoconsistency checks.



If you enable caching of VMs, remember to clear cache when new VMs are added. For more
information, see Protecting Hyper-V Machines (http://technet.microsoft.com/enus/library/ff634200.aspx).
Enable per-LUN serialization. This serialization limits the number of virtual machine backups
happening on a single CSV LUN, which in turn prevents backups failing due to time-out settings
being exceeded. For more information, see Considerations for Backing Up Virtual Machines on CSV
with the System VSS Provider (http://technet.microsoft.com/en-us/library/ff634192.aspx).
o Create a DataSourceGroups.xml file by running the DSConfig.ps1 script on any one node of a
cluster that contains one or more Cluster Shared Volumes (CSVs).
o Place the file in the DPM server at %PROGRAMFILES%\Microsoft DPM\DPM\Config.
o Repeat the previous two steps for every cluster that is protected by a DPM server. In the context
of the DBC Appliance, any configuration beyond a single rack results in additional clusters (one
per rack).
If the appliance consists of two or more racks, additional clusters exist, dictating the need to merge
the DatasourceGroups.xml file for all CSVs protected by DPM server. To do this, copy the <Group>
tags from all the DataSourceGroup.xml files generated, and then add the text between the
<DataSourceGroup> tags. The DataSourceGroups.xml file now contains one <header> tag, one
<DataSourceGroup> tag, and <Group> tags from all CSV clusters.
Performing On-Demand Backups
A full set of backups should be taken immediately after the appliance is set up, and on a regular,
business-driven frequency thereafter. There are three types of software assets that you will protect on
the appliance: virtual machines, SQL Server, and hosts (bare metal). Backups typically happen
automatically, as defined in the associated protection group’s settings. However, you can also take ondemand backups as needed. The process for on-demand backups is the same for all DPM assets.
To perform an on-demand backup, open the DPM management console, and then click the Protection
tab. Expand the protection group that contains your chosen backup target. Right-click the backup target,
and then click Create Recovery Point.
In the Create Recovery Point dialog box, next to Create recovery point for, click Short term disk
protection.
You can view the progress of your backup from the Monitoring tab of the management console. After it
is complete, your new backup is available for use from your storage pool.
Performing Restores
This section details how to perform the various types of restore operations you will encounter in the
context of the appliance, including how to recover an entire appliance from backups in the unlikely
event that this becomes necessary. The three types of restore operations you will encounter on the
appliance are: Virtual Machine, SQL Server, and Bare Metal.
Clustered Guest Restore Considerations
The appliance ships with a single guest cluster preconfigured: the underlying database for System Center
Virtual Machine Manager. As previously noted, you can choose to cluster workloads that you bring unto
the appliance. In such cases, you may need to restore both the virtual machine environment and the
SQL Server databases in two separate passes of the Recovery Wizard, depending on the nature of the
failure.
When restoring a VM that is involved in a guest cluster configuration, you have the option to restore to
either the original or an alternate location. When restoring to the original location, you must first find
and delete the original VHD, and subsequently create a new, blank VHD in the same location with the
same name. After this is done, you can proceed to restore the VM. No further configuration is
necessary. For more information, see Recovering Hyper-V Data (http://technet.microsoft.com/enus/library/ff399776.aspx) and the blog post DPM 2010 - Restore a Clustered Virtual Machine
(http://blogs.technet.com/b/bill_mcconnells_system_center_bytes/archive/2011/06/20/dpm-2010restore-a-clustered-virtual-machine.aspx).
In the event that the original resource is unavailable, you can elect to restore the VM to an alternate
location. Note that virtual machines recovered in this fashion must be manually added to the cluster
using either Windows Failover Cluster Manager or System Center VMM.
Virtual Machine Restores
Restoring VMs is relatively straight forward process. To begin, navigate to the Restore tab in DPM and
select the asset you want to perform the restore for. In the window panel on the right-hand side, you
are shown a calendar widget and listing of all available recovery points for that VM. Right-click the
recovery point you want to use, and then click Recover.
The Recovery Wizard opens. The first screen shows you a quick overview of your recovery selection.
Review your chosen options and click Next.
Next you select your recovery type. Here you have several options, including restoring to the original
instance (that is, the host), restoring to another host in your Hyper-V host cluster, or restoring a
copy of the VM to a network folder.
Next, specify additional recovery options. If your storage pool is on a SAN that supports hardware
snapshots, you can enable that feature from this screen. Click Next to continue.
Finally you’re shown a summary screen of all your options. Review them for accuracy, and then click
Recover to initiate the recovery process.
You can subsequently monitor the progress of your restores via the Monitoring tab.
SQL Server Recovery
Recovering SQL Server is similar recovering a virtual machine. The primary differences are in the
recovery options, which are of course tuned to this particular application. The DPM recovery process
uses SQL Server functionality to recover a database such that all uncommitted transactions are rolled
back. The recovery process opens the transaction log to identify uncommitted transactions.
Uncommitted transactions are undone by being rolled back, unless they hold locks that prevent other
transactions from viewing transactionally inconsistent data. This step is called the undo, or roll back,
phase. In some circumstances, the SQL Server administrator might require the database to be restored
in a mode that allows log backups to be selectively played back. Using DPM, you can recover a database
and leave it in a restoring state in which additional log backups can be applied to the database.
Bare Metal Recovery
Bare metal recovery entails the restore of a host operating system to the factory state. This process
should only be needed in the event of a host failure at a scale that necessitates either full hardware
replacement or replacement of both operating system drives.
Before you get started, complete the following steps:
1. Identify an external server with at least 1 terabyte of free space and a 1GB NIC, running Windows
Server 2003 or later:
a. Connect the server to the DBC Management A Switch located in U39.
b. Assign this server an IP address of 172.16.<racknumber>.254, subnet 255.255.0.0 and DNS
of 172.16.<racknumber>.50 and 51.
c. Join the computer to the DBC domain using the FQDN of DBC<appliance number>.local
(note that there is a leading zero in the rack number, so if your rack number is 1, the FQDN
is DBC01.local).
d. Install a DPM agent this server from the DPM VM at 172.16.<racknumber>.104.
e. Install DHCP with a scope of 172.16.253.1 to 172.16.253.255.
f.
Create a share called ‘share’ and grant the DBC Appliance domain administrators full
control.
2. Create an ISO that includes NIC drivers for the blades:
a. Download the NIC drivers from
http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?lang=en&cc
=us&prodTypeId=3709945&prodSeriesId=4296125&prodNameId=4324854&swEnvOID=406
4&swLang=8&mode=2&taskId=135&swItem=MTX-12787b43fd3542cbbbd2fcb5c2
b. Extract the files from the .exe to a folder.
c. Create an ISO file that includes these files.
At this point you can begin the recovery process. If it is not already present, restore Bare Metal recovery
data to the external server's share using DPM. Next, connect to the blade you are recovering using iLO
through the Onboard Administrator website at ‘172.16.<racknumber>.10’. Mount the ISO file using the
iLO remote console and boot into the HP Factory Recovery environment as shown here.
Next, select your language, and enter your user name and password. Choose the System Image
Recovery option. In the Re-image Your Computer dialog box, click Cancel, and then click Next. On the
next screen, click Advanced.
Click Install a driver, and then click OK.
Browse to the CD drive and select the be2nd62.inf file. Next, select the HP NC551i Dual Port FlexFabric
10GB NIC and click Add drivers. Click Advanced again and then click Search for a system image on the
network; when a warning message appears, click Yes. Enter the UNC path the server share using the IP
address (for example, \\172.16.1.254\share). Note: This must be the complete path to the
WindowsImageBackup directory.
Log in using the DBC domain admin account. Select the backup that appears in the grid (there should
only be one), and then click Next. Once again, select the backup image from the grid (there should again
be only be one) and then click Next as shown here.
Click Next again, and then click Finish. When you see a message warning you that all your data will be
replaced with the bare metal restore, click Yes. The process completes without further input from you,
and your computer is restarted automatically. In order to validate the process, open cluster
administrator on another blade and verify the blade reappears and live migration works.
Full Appliance Recovery
In the event of a total site failure or a large scale failure local to the appliance, it’s possible to restore the
entire appliance infrastructure back to a working state from backups. This section outlines the specific
order of restore steps required to accomplish a full appliance restore.
1. Log on to the secondary DPM server. If you have not done so already, specify the secondary
DPM server as the new primary. For more information, see Switching Protection if the Primary
DPM Server Fails (http://technet.microsoft.com/en-us/library/bb808972.aspx).
2. Perform bare metal restore for all hosts as described earlier. After the restore process has
begun, select the option in the HP Onboard Administrator to not auto-power blades 2, 3, and 4
(also hosts 5-8 in full rack configuration).
3. Wait for host 1 to boot up and the guest VM that hosts Active Directory to completely boot up.
4. Restore all other hosts and wait for them to completely boot up.
5. Stop Active Directory services on all VMs on all hosts.
6. Initiate a restore operation for the host 1 Active Directory VM and wait for it to fully boot up.
7. Repeat step 6 for all other Active Directory VMs. The cluster service comes up for all the hosts.
After all these steps are completed, the cluster service should come up for all the hosts on its
own.
8. Restore the guest SQL Server cluster for System Center VMM (in the earlier example,
VDB1/VDB2).
a. Restore the SQL Server backups for the VMs.
9. Restore the virtual machines that host the database (in the earlier example, SDB1) (Operations
Manager) and the data warehouse (SDW1).
a. Restore the respective SQL Server backups for the VMs.
10. Begin restoring all other VMs the appliance hosts, with the exception of the primary DPM. The
order of restores at this point is not important.
11. After you restore all other management VMs, restore the primary DPM server and switch the
protection type on the secondary DPM server back to ‘secondary’ mode.
In our tests, restoring an appliance to a working state takes about five hours for a half-rack
configuration. For a full-rack, an additional one to two hours is needed due to the additional hosts and
Active Directory installations.
Disaster Recovery
The Database Consolidation Appliance is fully compatible with all existing SQL Server high availability
and disaster recovery technologies. This section covers requirements from an appliance perspective and
briefly touches upon the available options that are built in to SQL Server. It does not cover third party
solutions such as SAN-based replication, though you can incorporate them into your solution.
Appliance Requirements
The chief disaster recovery requirement is that a second appliance be located in a geographically
disparate location. The second appliance can be used solely for disaster recovery or, alternatively, both
the second appliance and the primary one can be designated as each other’s backup in addition to
running regular workloads. Appliances designated to handle additional workloads in the event of a
disaster should be sized relative to the expected workload transfer in the event of a disaster over and
above any steady state workload requirements.
It’s important to note that each appliance runs its own dedicated management software, and as such
there is no need to design or manage appliance-level failover or replication. With this mind, you are free
to focus on designing a workload-level disaster recovery plan, which in practice means focusing on SQL
Server.
Software Requirements
The key software components in the context of DBC disaster recovery are Data Protection Manager and
SQL Server. DPM provides basic disaster recovery through backup and restore for less critical workloads
(those with longer recovery time and point objectives). For more critical workloads, you can use the SQL
Server high availability features to provide significantly faster time to recovery and minimize potential
for data loss.
Data Protection Manager
A given DPM server can back up any other DPM server in addition to any other protection targets. In the
event of a disaster, the secondary DPM server is used to restore critical servers and workloads, and then
it is use to rebuild the new primary DPM server.
In a two-site topology, we recommend that you create a secondary DPM server at each site. Each
secondary DPM server handles backup for the local primary DPM server, as well as the remote primary
DPM server. Note that the only local asset backed up on the secondary DPM server is DPM itself. In a
three-site topology, a single DPM secondary server is used to back up each primary DPM server and all
protected assets.
If a failure occurs, an administrator can easily transfer active protection the secondary DPM server and
subsequently use the secondary DPM server to restore protected assets.
SQL Server
SQL Server workloads that run on the appliance can be set up for disaster recovery using any of the
proven, built-in high availability technologies. The key technologies to utilize are database mirroring and
log shipping. It’s important to note that there is nothing appliance specific to use these in the context of
DBC; you simply proceed as you would for any other SQL Server installation. A brief overview of these
technologies follows. For more detailed information, including links on these technologies in the context
of enabling high availability and disaster recovery with SQL Server, see the appendix.
Database Mirroring
Database mirroring provides a hot standby, and based on configuration, it can be set up for automatic or
manual failover. A read-only snapshot can be created on the mirror to facilitate reporting scenarios.
Mirroring can be established in two modes: High Performance mode (asynchronous) and High Safety
(synchronous). In High Performance mode, transactions are sent to the mirror and the principal does not
wait on acknowledgement from the mirror before committing the transaction. In High Safety mode,
each transaction is hardened and acknowledged by the mirror before being committed on the principal.
With high safety mode you can also have automatic failover with the use of a witness server used to
form a quorum. High Performance mode does not protect against data loss to the same degree, but it
requires less network capability, and it results in higher throughput, whereas High Safety mode helps
guarantee zero data loss.
Log Shipping
Log shipping is a technology built into SQL Server that automates the backup, copy, and restore of the
transaction logs on another database either on the same instance, a different instance, or a different
server altogether. The log can be shipped to multiple secondary servers. Log shipping provides a warm
standby, and if it is set up in standby mode, it can be used for read-only queries while the transaction
logs are being restored.
Conclusion
The HP Database Consolidation Appliance provides an industry-leading platform for maintaining a
complex SQL Server infrastructure without incurring the expanded hardware footprint required by
physical servers. The numerous configuration options makes it a good fit into most corporate budgets,
because it can scale from one half rack up to ten full racks.
The appliance design eliminates any one point of failure in the hardware stack, and it employs numerous
techniques to ensure high availability at the software level. Backup and recovery have been thoroughly
thought out, and recovery time has been dramatically reduced compared to the time to recover physical
servers.
Combining the technologies available in the appliance with proven disaster recovery techniques aligns
this solution well with corporate disaster recovery plans. Traditional SQL Server techniques such as
mirroring and log shipping can be used on the appliance in the same way as physical servers, ensuring
there is no extra effort or design work required to implement a disaster recovery strategy.
Appendix
DBC General
HP Enterprise Database Consolidation Appliance Whitepaper
http://download.microsoft.com/download/9/E/0/9E080BAC-4A39-4748-A6253EE2A4ACC883/HP_Enterprise_Database_Consolidation_Appliance_Whitepaper_10_3_11.docx
High Availability
Proven Architectures for High Availability and Disaster Recovery (SQLCAT)
http://sqlcat.com/sqlcat/b/whitepapers/archive/2010/06/07/proven-sql-server-architectures-for-highavailability-and-disaster-recovery.aspx
HA Demonstration of the DBC Appliance
http://blogs.technet.com/b/dataplatforminsider/archive/2011/10/25/a-closer-look-at-the-hadrcapabilities-of-the-hp-enterprise-database-consolidation-appliance.aspx
Making a Virtual Machine Highly Available
http://technet.microsoft.com/en-us/library/cc732181(WS.10).aspx#BKMK_HA
Database Mirroring Overview
http://msdn.microsoft.com/en-us/library/ms189852.aspx
Log Shipping Overview
http://msdn.microsoft.com/en-us/library/ms187103.aspx
SQL Server 2012 AlwaysOn Introduction
http://msdn.microsoft.com/en-us/sqlserver/gg490638
SQL Server 2012 AlwaysOn Availability Groups
http://msdn.microsoft.com/en-us/library/hh510230(v=SQL.110).aspx
SQL Server 2012 AlwaysOn Failover Cluster Instances
http://msdn.microsoft.com/en-us/library/ms189134(v=SQL.110).aspx
SQL Server 2012 AlwaysOn Solutions for High Availability and Disaster Recovery (SQLCAT)
http://sqlcat.com/sqlcat/b/whitepapers/archive/2012/02/25/microsoft-sql-server-alwayson-solutionsguide-for-high-availability-and-disaster-recovery.aspx
Hyper-V and Failover Clustering
http://technet.microsoft.com/en-us/library/cc732181(WS.10).aspx
Data Protection Manager (DPM) 2010
DPM Overview
http://www.microsoft.com/download/en/details.aspx?id=6287
Protecting SQL Server with DPM 2010
http://www.microsoft.com/download/en/details.aspx?id=9744
How to Protect Hyper-V with DPM 2010
http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=14575
Data Protection Manager 2010 Operations Guide
http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=15193
Latest Hotfix for DPM 2010:
http://support.microsoft.com/kb/2465832
Information on TCP Chimney Offload
http://support.microsoft.com/kb/951037
Manually Installing DPM Protection Agents
http://technet.microsoft.com/en-us/library/ff399459.aspx
Protecting Hyper-V Machines Using DPM 2010
http://technet.microsoft.com/en-us/library/ff634200.aspx
Considerations for Backing Up Virtual Machines on CSV with the System VSS Provider
http://technet.microsoft.com/en-us/library/ff634192.aspx
Recovering Hyper-V Data
http://technet.microsoft.com/en-us/library/ff399776.aspx
Switching Protection if the Primary DPM Server Fails
http://technet.microsoft.com/en-us/library/bb808972.aspx