Download Describe number and configuration of disks on

VALIDATED HARDWARE
CONFIGURATION
Partner Name
Reference Architecture
Table of Contents
DOCUMENT HISTORY ...................................................................................................................3
GLOSSARY .....................................................................................................................................4
TRADEMARKS ...............................................................................................................................5
OVERVIEW ......................................................................................................................................5
USE CASE SUMMARY ............................................................................................................................... 5
MIRANTIS OPENSTACK ............................................................................................................................ 5
HARDWARE OPTIONS ............................................................................................................................... 6
NETWORKING ........................................................................................................................................... 6
KEY BENEFITS .......................................................................................................................................... 6
MIRANTIS OPENSTACK ARCHITECTURE ..................................................................................7
MIRANTIS OPENSTACK OVERVIEW ......................................................................................................... 7
NODE TYPES ............................................................................................................................................ 8
Infrastructure Node ............................................................................................................................ 8
Controller Node................................................................................................................................... 8
Compute Node .................................................................................................................................... 8
Storage Node ...................................................................................................................................... 8
OPTIONAL COMPONENTS....................................................................................................................... 10
NETWORK ARCHITECTURE ....................................................................................................... 10
NETWORK DESIGN .................................................................................................................................. 11
CABLING.................................................................................................................................................. 13
Rack Physical Cabling Schema ..................................................................................................... 13
Server Interface Configuration ....................................................................................................... 14
Logical Networking Schema ........................................................................................................... 15
RACK HARDWARE SPECIFICATION ......................................................................................... 15
SERVERS ................................................................................................................................................ 16
RECOMMENDED MINIMUM SERVER CONFIGURATIONS ........................................................................ 16
SWITCHES ............................................................................................................................................... 16
DISK CONFIGURATION ............................................................................................................... 16
COMPUTE NODE ..................................................................................................................................... 17
CONTROLLER NODE ............................................................................................................................... 17
STORAGE NODE ...................................................................................................................................... 17
INFRASTRUCTURE NODE ........................................................................................................................ 17
SCALING ....................................................................................................................................... 18
SUGGESTED RACK CONFIGURATIONS .................................................................................................. 18
TESTING ........................................................................................................................................ 18
REFERENCES .............................................................................................................................. 18
Document History
Version
Revision Date
Description
0.1
DD-MM-YYYY
Initial Version
Glossary
Term
Description
BMC
Bonding
Ceph
DVR
The baseboard management controller
A virtual network interface consisting of aggregated physical links
A distributed block store, object store, and file system
Distributed Virtual Router – a way of configuring and running Neutron L3
agent on every Compute node in order to move load from Controller nodes.
Each Compute node serves East-West routing and North-South Floating IP
translation for local instances while Controller nodes serve only North-South
traffic for those instances which don’t have Floating IPs assigned.
Traffic within a cloud. I.e. VM to VM or tenant to tenant traffic.
A non-persistent disk or volume used by VM
An IP address that can be instantly moved from one VM to another.
Floating IP addresses enables dynamic IP address association with virtual
machine instances.
An open source project built to deploy OpenStack. Part of OpenStack Big
Tent.
A special package which extends Fuel’s functionality. For more details see
https://wiki.openstack.org/wiki/Fuel/Plugins
Intelligent Platform Management Interface
Link Aggregation Control Protocol - an international standard of network
interfaces bonding supported by almost all vendors and operating systems.
The current specification is IEEE 802.1AX-2008.
Mirantis OpenStack
A specific configuration of network switches when two or more independent
switches act as a single switch from the LACP standpoint.
Network address translation - a special mechanism that allows hosts from
one network to reach out to another network without having proper routing
established.
Traffic between cloud VMs and the rest of the network (anything outside
the cloud).
East-West traffic
Ephemeral storage
Floating IP
Fuel
Fuel plugin
IPMI
LACP
MOS
Multi-chassis Link
Aggregation
NAT
North-South traffic
OS
RA
Rally
Shaker
Tempest
TLS
Operating system
Reference Architecture
An open source tool to perform cloud verification, benchmarking & profiling.
A wrapper around popular network performance testing tools to measure
network performance of a cloud.
This is a set of integration tests to be run against a live OpenStack cloud.
Transport Layer Security protocol aimed to secure network connections.
Trademarks
<Partner-specific trademark notes>. Ubuntu is the registered trademark of Canonical Ltd. in the
U.S. and other countries.
DISCLAIMER: The OpenStack® Word Mark and OpenStack Logo are either registered
trademarks/ service marks or trademarks/service marks of the OpenStack Foundation, in the
United States and other countries, and are used with the OpenStack Foundation's permission.
Neither Mirantis nor <Partner> are affiliated with, endorsed or sponsored by the OpenStack
Foundation or the OpenStack community.
Other trademarks and trade names may be used in this publication to refer to either the entities
claiming the marks and names or their products. Mirantis, Inc., disclaims any proprietary interest
in trademarks and trade names other than its own.
Overview
This document describes the <RA name> — a fully-validated deployment of Mirantis OpenStack
on <Partner-Product> servers and <Network-equipment-provider-name> switches.
The <RA name> is engineered to facilitate construction of a <RA goal description>.
This Reference Architecture details all hardware and software components of the <RA name>;
describes their physical integration and logical interoperation; and discusses in depth certain
critical aspects of the design relevant to performance, high availability, data-loss prevention, and
scaling.
Use Case Summary
This Reference Architecture (RA) is designed for <RA workload type>. <Additional Description
Here>
Mirantis OpenStack
Mirantis is a number one contributor to OpenStack. Fuel - a core part of Mirantis OpenStack
distribution - has been taken under Big Tent.
Mirantis recommends the following configuration of Mirantis OpenStack:
● Mirantis OpenStack version 8.0 (OpenStack Liberty)
● Host OS: Ubuntu 14.04
● Core OpenStack services to be deployed:
○ Keystone
○ Nova
○ Glance
○ Cinder
○ Neutron
● Optional OpenStack services to be deployed:
○ Horizon
○ Heat
○ Murano
○ Ceilometer
● HA Configuration: Three OpenStack Controller nodes with:
○ HAproxy
○ Corosync/Pacemaker
○ MySQL Galera
○ RabbitMQ
○ MongoDB
● Storage: A Ceph cluster provides redundant backend for
○ Glance
○ Cinder
○ Nova Ephemeral
○ Swift-API
Cluster size = 3 (3 x replication) is utilized to protect data
● Secure public OpenStack API endpoints and Horizon with TLSv1.2.
Hardware Options
<Describe Partner Hardware Stack>.
<Describe Hardware Stack and Configuration Benefits>
Networking
<Describe Networking: what switches are used, their benefits, LACP/MLAG benefits>
Key Benefits
●
<Identify Key Benefits of the whole solution – Bullets>
Mirantis OpenStack Architecture
Mirantis OpenStack Overview
Mirantis OpenStack 8.0 is a Liberty based OpenStack distribution running on top of Ubuntu
14.04.
In Mirantis OpenStack all API services are put on a dedicated nodes - Controller nodes - for
better control and operability. Compute and Storage nodes serve virtualization and storage load
correspondingly. To avoid possible cross service influence those components are situated on
separate nodes.
All OpenStack services except Ceilometer are using local MySQL Galera Cluster as a database
backend. Ceilometer uses MongoDB.
All services use RabbitMQ as a message queueing service. Murano uses a separate instance of
RabbitMQ in order to separate security domains.
Ceph provides Swift API via RadosGW and also is used as a backend for Cinder, Glance, and
Nova Ephemeral storage.
To learn more, please, see the description of Mirantis OpenStack architecture at
https://docs.mirantis.com/openstack/fuel/fuel-8.0/mos-planning-guide.html#fuel-reference-architectureoverview.
Node Types
Servers used in this Reference Architecture will be deployed as one of these node types:
● Infrastructure
● Controller
● Compute
● Storage
Infrastructure Node
The Infrastructure node is an Ubuntu 14.04 based node which carries two virtual appliances:
● Fuel Master node - an OpenStack deployment tool.
● Cloud Validation node - a set of OpenStack post-deployment validation tools including
Tempest and Rally.
Controller Node
A Controller node is a control plane component of a cloud which incorporates all core
OpenStack infrastructure services: MySQL, RabbitMQ, HAproxy, OpenStack APIs (Keystone,
Nova, Neutron, Glance, Cinder, Heat, Murano, Ceilometer, RadosGW for Swift API), Horizon,
MongoDB. This node is not used to run virtual instances or store cloud data.
Compute Node
The Compute node is a KVM hypervisor component of a cloud that runs virtual instances. It has
Nova-compute and Neutron vSwitch and vRouter entities running.
Storage Node
Ceph is used as a storage back end. A storage node is a component of an OpenStack
environment that stores and replicates all user data stored in a cloud. High Availability
The High Availability model implemented in Mirantis OpenStack (MOS) is well described in the
official documentation here: https://docs.mirantis.com/openstack/fuel/fuel-8.0/mos-planningguide.html#fuel-ref-arch-ha-arch
Here is the short description of how Mirantis OpenStack HA works:
To protect core OpenStack services from node failure, all control plane components are
distributed across multiple Controller nodes. A minimum of 3 nodes is required to provide high
availability for the cloud control plane. It is possible to deploy an odd number of Controller nodes
greater than 3 or add more nodes in pairs after the initial deployment (keeping the total number
odd) to distribute load and increase the redundancy level. Pacemaker/Corosync is used to
manage critical infrastructure parts when failure occurs. Because of a quorum majority voting
algorithm used in Corosync, a cluster of ‘N’ Controller nodes may survive when ‘N/2-1’ nodes go
down. For 3 nodes it’s 1 node down, for 5 nodes it’s 2 nodes, and so on.
Almost all services are working in active-active mode and load is balanced by HAproxy listening
at a virtual IP address. Most OpenStack components support active/active mode natively, as do
RabbitMQ, MongoDB, and Galera Cluster, which is used to protect MySQL. The only exception
here is Cinder which is running in Active/Passive mode. That's aimed to be fixed in further
OpenStack releases.
From the networking perspective, high availability is maintained by redundant and bonded
connections to two ToR switches. That means if one cable gets cut or even the entire switch
goes down a cloud will survive with half its normal network bandwidth still available for the
affected nodes.
Another low-level hardware feature that provides redundancy is RAID1 for disks on
Infrastructure and Storage nodes to keep an operating system intact in case of a disk failure.
Also it’s highly recommended to use two independent hot-plug power supplies for each
hardware unit, ideally connected to separate power circuits. This protects a cloud from a power
failure.
Ceph provides redundancy for stored data. The recommended replication factor is 3 which
means at a given time there are 3 replicas of each object are distributed among Ceph nodes. It
is recommended to store not more than 10% of actual data (comparing to raw storage) on each
Storage node to avoid huge load to a network and Ceph cluster in case of Storage node failure.
Optional Components
There are two possible ways to use MongoDB with Ceilometer - install MongoDB locally or use
an external MongoDB database. Both options are available in Mirantis OpenStack. In the case
of installing MongoDB locally we recommend using at least 3 separate servers to form a
MongoDB cluster. Placing MongoDB onto the Controller node may cause resource starvation
for key services such as MySQL or RabbitMQ which may lead to severe issues with the cloud
as a whole. The best approach is to use dedicated nodes for MongoDB - either external nodes
or nodes deployed as part of MOS.
By default, Mirantis OpenStack protects public OpenStack APIs and Horizon with TLSv1.2 with
self-signed or user-provided certificates, but it’s possible to disable this feature if needed.
Mirantis OpenStack may be extended via Fuel plugins to provide two validated options to
monitor your cloud - Zabbix or LMA toolchain (Elasticsearch/Kibana, InfluxDB/Grafana and
Nagios).
Network Architecture
The underlying network comprises the OpenStack control plane, data plane, and BMC
Management. A pair of ToR 10G switches in each rack forms a MC-LAG group and serves as
OpenStack control and data plane traffic and uplinks. MC-LAG allows servers’ bonded NICs and
rack uplinks to be connected to both switches (opposite to traditional LACP), thus every
connection is secured against physical link or the entire switch failure.
A single 1G switch serves BMC Management network 1Gbps connections that include IPMI,
switch management traffic, and OpenStack Provisioning traffic.
Ceph IO and monitoring traffic is transported on the OpenStack Management network and is
protected against single path network failure with LACP.
It is recommended to transport Ceph replication traffic on a separate physical network and
protect it with LACP along with OpenStack Management network.
It is also recommended to transport tenant's traffic over a separate physical network and protect
it with LACP.
Network design
The networking model of Mirantis OpenStack requires the following networks to be set up
(VLAN numbers are only given for a reference):
Network
VLAN IDs
BMC Management networks:
Mirantis OpenStack Admin/PXE (OpenStack Provisioning) network
Out of Band IPMI network
OpenStack control plane networks:
OpenStack Management and Ceph Public network
OpenStack Public network
OpenStack Ceph Replication network
120
100
140
160
180
OpenStack data plane network:
OpenStack Private network
200-1000
To learn more about Mirantis OpenStack’s networking model see
https://docs.mirantis.com/openstack/fuel/fuel-8.0/mos-planning-guide.html#plan-the-network
We recommend deploying Mirantis OpenStack with Neutron DVR enabled. In this mode,
Neutron runs a standalone virtual router on every Compute node. This vRouter can route both
East-West and North-South traffic for Floating IPs of local instances, dramatically decreasing
load on Controller nodes (which serve only North-South NAT traffic when DVR is enabled). To
learn more about DVR see https://wiki.openstack.org/wiki/Neutron/DVR
The following subnets are used by Mirantis OpenStack by default.
Network
Subnet Details
Mirantis OpenStack PXE Network
10.20.0.0/24
This network may be fully isolated inside of a cloud
and does not need to be routed to a customer’s
network.
192.168.0.0/24
OpenStack Management and Ceph
Public Network
OpenStack Public Network
OpenStack Private Network
OpenStack Ceph Replication
Network
Out of Band IPMI and Switch
Management Network
Depends on the cloud’s intended use. Minimum /26
network is recommended. To calculate the required
size of the Public Network see
https://docs.mirantis.com/openstack/fuel/fuel8.0/mos-planning-guide.html#calculate-float-public-ip
This network is actually a set of VLANs where tenant
traffic is going. Since these networks are isolated
from the rest of data center any IP space may be
used here. For validation purposes, a single tenant
network is created during the deployment and given
the network address 192.168.122.0/24. This network
can be removed after validation is complete.
192.168.1.0/24
This network may be fully isolated inside of a cloud
and does not need to be routed to a customer’s
network
The network should be wide enough to accommodate
all IPMI and switch management IP addresses along
with a gateway for them.
Cabling
Rack Physical Cabling Schema
<Source of the image
https://drive.google.com/file/d/0Bw6txZ1qvn9CS2xFQXhaOUFIU1k/view?usp=sharing >
Server Interface Configuration
Next tables show how switch ports should be configured in regard to corresponded servers'
interfaces.
Controller Node
Interface
1st 1G
VLANs
120
(untagged)
Compute Node
Interface
1st 1G
VLANs
120
(untagged)
1st & 2nd 10G
bond0
Active/Active LACP
140, 160, 200-1000
(tagged)
IPMI
1st & 2nd 10G
bond0
Active/Active LACP
140, 160, 200-1000
(tagged)
IPMI
100 (untagged)
100 (untagged)
Note: Public network (VLAN 160) should be provided to Compute nodes only if Neutron DVR is
used.
Storage Node
Interface
VLANs
1st 1G
120
(untagged)
1st & 3nd 10G
bond0
Active/Active LACP
140, 200-1000
(tagged)
2rd & 4th 10G
bond1
Active/Active
LACP
180 (tagged)
IPMI
100
(untagged)
Note: Storage node has 2 network cards with 2 ports each. Each bonded interface consists of
two ports from different physical network cards to provide protection against network card
failure.
Logical Networking Schema
<Source of the image
https://drive.google.com/file/d/0Bw6txZ1qvn9COWpsaW1IQ19wckU/view?usp=sharing >
Rack hardware specification
Servers
<Servers used and brief descriptions>
Recommended Minimum Server Configurations
Controller
Node
Compute
Node
Storage Node
Infrastructure
Node
Server Model
Quantity
Sockets
Cores per
socket
RAM
Configuration
Storage
Configuration
3
-
-
-
1
1
6
-
-
-
32
2x 400GB
RAID1 SSDs
-
Networking
Configuration
1G NIC and
2x 10G NICs
2x 400GB
RAID1
SATA disks
1G NIC and
2x 10G
NICs
2x 400
GBRAID1
SATA disks
2 1G NICs
1G NIC and 2x 10G 2ports NICs
Switches
Switch Model
Purpose
Quantity
-
Provisioning and OOB traffic
Intracloud and external traffic
-
Disk Configuration
The following configuration is the recommended minimum in terms of storage requirements.
Node
RAID type
Disks number
Disk type
Purpose
Controller
RAID 1
1,2
SSD
Operating system
RAID 1
3,4
SSD
MongoDB
Compute
RAID 1
1,2
SATA
Operating system
Storage
RAID 1
1,2
SATA
Operating system
None
<Put actual
quantity of disks
<10k SAS or
SSD>
CEPH OSDs
None
Infrastructure
RAID1
per server
here>
<1 Journal disk
per 5 or 15
OSDs (see
below)>
1,2
SSD
CEPH Journal
SSD
Operating System
Compute Node
Each compute node requires at least one disk with average speed characteristics, used to
deploy the OS. We recommend using at least 400GB disks in RAID1 for this purpose.
Controller node
Each Controller node requires at least one disk with good speed characteristics for OS and
OpenStack services. We recommend using 400GB SSDs in RAID1 for these purposes.
You may use a separate set of disks to host MongoDB if the database is running on Controller
nodes. We recommend using 2 SSDs in RAID1 with at least 200GB each for a cloud with 30
Compute nodes. The actual MongoDB load depends on Ceilometer configuration. Please,
contact your Mirantis representative to get detailed information.
Storage node
Storage nodes require at least one disk with average speed characteristics to host the OS. We
recommend using at least 200GB disks in RAID1 for this purpose.
To speed up Ceph, we recommend putting Ceph journals on SSDs. The amount and size of
required SSDs is roughly calculated as follows: 40GB space on SSD for every 1-1.2TB SAS
disk and one SSD may be used with up to 5 SAS disks. For NVMe or PCIe disks the
recommendation is one journal disk for every 15 OSD drives. The above formula gets 2x 200GB
SSDs for 10 1.2TB SAS disks and so on.
<Describe recommended storage node disk calculation based on actual HW and
recommendations from above >
Infrastructure node
Infrastructure nodes require at least one disk with average speed characteristics for the OS. We
recommend using at least one 400GB disk for this purpose. RAID1 is optional but preferable.
Otherwise the Fuel Master node must be backed up.
<Describe number and configuration of disks on Infrastructure nodes>
Scaling
<Describe scaling test results, recommendations, and provide contact recommendation to users
who wish to inquire about scaling options>.
Suggested Rack Configurations
<Describe possible rack configurations. I.e. 10 computes + 4 storages, 20 computes + 8
storages, etc. How many average VMs fit into each configuration and what the power footprints
are.>
Testing
Post-deployment cloud validation is performed in a semi-automated way using tools such as
Tempest, Rally, Shaker, and others.
The testing approach is described in a separate Test Plan document.
References
●
Mirantis OpenStack 8.0 Documentation https://docs.mirantis.com/openstack/fuel/fuel8.0/