Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download the document - Support
Document related concepts
Transcript
Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper Issue 1.0 Date 2016-03-31 HUAWEI TECHNOLOGIES CO., LTD. Copyright © Huawei Technologies Co., Ltd. 2016. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Huawei Technologies Co., Ltd. Trademarks and Permissions and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd. All other trademarks and trade names mentioned in this document are the property of their respective holders. Notice The purchased products, services and features are stipulated by the contract made between Huawei and the customer. All or part of the products, services and features described in this document may not be within the purchase scope or the usage scope. Unless otherwise specified in the contract, all statements, information, and recommendations in this document are provided "AS IS" without warranties, guarantees or representations of any kind, either express or implied. The information in this document is subject to change without notice. Every effort has been made in the preparation of this document to ensure accuracy of the contents, but all statements, information, and recommendations in this document do not constitute a warranty of any kind, express or implied. Huawei Technologies Co., Ltd. Address: Huawei Industrial Base Bantian, Longgang Shenzhen 518129 People's Republic of China Website: http://www.huawei.com Email: [email protected] Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. i Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper Contents Contents 1 Overview......................................................................................................................................... 1 2 Huawei Solution ........................................................................................................................... 2 2.1 System Architecture .................................................................................................................................................. 2 2.1.1 Pre-integration......................................................................................................................................................... 4 2.1.2 Compatible with Mainstream Virtualization Platforms .................................................................................... 4 2.1.3 Convergence of Computing, Storage, and Networking Resources ................................................................. 5 2.1.4 Distributed Storage ................................................................................................................................................. 5 2.1.5 Hybrid Virtualization and Database Deployment ............................................................................................. 6 2.1.6 Automatic Deployment.......................................................................................................................................... 6 2.1.7 Unified O&M ........................................................................................................................................................... 6 2.2 FusionCube Networking .......................................................................................................................................... 7 2.2.1 Internal Networking ............................................................................................................................................... 7 2.2.2 External Networking .............................................................................................................................................. 8 2.3 FusionStorage Features............................................................................................................................................. 10 2.3.1 SCSI/iSCSI Block Interface .................................................................................................................................. 10 2.3.2 Thin Provisioning ................................................................................................................................................. 12 2.3.3 Snapshot ................................................................................................................................................................. 12 2.3.4 Linked Cloning...................................................................................................................................................... 13 2.3.5 Elastic Scaling ........................................................................................................................................................ 14 2.3.6 High Reliability ..................................................................................................................................................... 18 2.3.7 Multiple Resource Pools ...................................................................................................................................... 21 2.3.8 VMWare VAAI ...................................................................................................................................................... 22 3 Hardware Convergence .............................................................................................................. 25 3.1 FusionCube 9000 ...................................................................................................................................................... 25 3.1.1 E9000 Chassis ........................................................................................................................................................ 25 3.1.2 E9000 Blades .......................................................................................................................................................... 26 3.1.3 High-Performance Switch Modules ................................................................................................................... 29 3.2 FusionCube 6000 ...................................................................................................................................................... 30 3.2.1 X6800 Chassis ........................................................................................................................................................ 31 3.2.2 X6800 Server Nodes .............................................................................................................................................. 32 4 Typical Configurations .............................................................................................................. 34 Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. ii Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper Contents 4.1 FusionCube 6000 ...................................................................................................................................................... 34 4.2 FusionCube 9000 ...................................................................................................................................................... 36 Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. iii Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper Error! Use the Home tab to apply 标题 1 to the text that you want to appear here.Error! Use the Home tab to apply 标题 1 to the text that you want to appear here. 1 Overview Nowadays, a growing number of enterprises are using virtualization and cloud computing technologies to build their IT systems to improve resource utilization. The virtualization and cloud computing infrastructures must address the following challenges: Complex deployment and management of virtualization platforms and soaring operation and maintenance (O&M) costs High planning, deployment, and optimization skill requirements on operation personnel because hardware devices may be provided by different vendors Slow response to handling after-sales problems on operation portals of multiple vendors Huge maintenance system that involves hardware maintenance and virtualization platform management of multi-vendor products Obvious advantages only in large-scale data center virtualization scenarios Customers attach importance to cost control, rapid service provisioning, and risk management. They want resource-scalable, reliable, and high-performance IT systems with low total cost of ownership (TCO) and short service rollout time. The Huawei FusionCube Converged Infrastructure (FusionCube for short) solution integrates computing, storage, and network resources in an out-of-the-box packaging. It features pre-integration, automatic service deployment, and intelligent resource scaling. It provides high performance, reliability, and security, as well as unified O&M. FusionCube satisfies customers' long-term service development requirements and helps them win rigorous market competitions. This document describes the architecture, software and hardware, and configuration of FusionCube. It is intended for sales engineers, channel partners, senior service managers, and customers who intend to use FusionCube to deploy virtualized IT infrastructures. Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 1 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 2 Huawei Solution 2 Huawei Solution 2.1 System Architecture FusionCube is a flagship product of the Huawei IT product line. Designed based on an open architecture, FusionCube integrates blade servers, distributed storage, and network switches into a chassis. Therefore, no external storage device or switch is required. In addition, the distributed storage engine, virtualization platform, and management software are pre-integrated, allowing resource allocation on demand and linear expansion. Figure 2-1 shows the overall architecture of Huawei FusionCube hyper-converged virtualization infrastructure. Figure 2-1 Huawei FusionCube solution architecture Software FusionCube Center manages FusionCube virtualization and hardware resources and implements system monitoring and O&M. FusionCube Builder provides quick onsite installation and deployment of FusionCube system software and used to replace or update virtualization platform software. Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 2 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 2 Huawei Solution FusionStorage uses distributed storage technologies to provide high-performance, high-reliability block storage services by scheduling local hard disks on blade servers in an optimized manner. Hardware FusionCube uses E9000 blade servers or X6800 high-density servers. The servers support modular design of computing, storage, switch, and power modules, allowing on-demand configuration of computing and storage nodes in a chassis as required. FusionCube supports I/O expansion for components, such as graphics processing units (GPUs) and PCIe solid-state drive (SSD) cards, and various switch modules (including GE, 10GE, and IB) to meet different configuration requirements. Features FusionCube applies to the following typical service scenarios: Server virtualization: Provide integrated FusionCube virtualization infrastructures without other application software. Desktop cloud: Run Virtual Desktop Infrastructures (VDIs) or virtualization applications on FusionCube to provide desktop cloud services. Enterprise office automation (OA): Run Enterprise OA service applications such as Microsoft Exchange and SharePoint on FusionCube. Integrating and optimizing the Huawei-developed hardware platform, distributed storage and management software, FusionCube provides the following features: Integration FusionCube integrates computing, storage, and network resources. − Hardware integration: Computing, storage, and network resources are integrated into a linearly scalable system. − Management convergence: Centralized O&M significantly improves resource utilization and reduces operating expenses (OPEX). − Application convergence: Hardware and software are optimized based on application service models to improve system performance. Simplicity FusionCube implements system pre-integration, preinstallation, and pre-verification, automatic device discovery upon power-on, and unified O&M, greatly simplifying service delivery. − Simplified installation: FusionCube has hardware and software preinstalled before delivery. It can be used on-site out of the box. − Rapid deployment: After the system is powered on, devices are automatically discovered and parameters are automatically configured, which allows rapid service rollout. − Easy O&M: FusionCube provides a unified O&M interface and automatic fault locating, facilitating routine O&M. Optimization FusionCube uses industry-leading hardware and distributed storage software to ensure optimal user experience. − Issue 1.0 (2016-03-31) Storage optimization: FusionCube uses built-in distributed storage to provide storage services with high concurrency and throughput. Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 3 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper − 2 Huawei Solution Network optimization: FusionCube supports 56 Gbit/s InfiniBand to provide the fastest switching network in the industry. Openness FusionCube is an open hyper-converged infrastructure platform independent of specific applications. It provides computing, storage, and network resources for mainstream virtualization platforms and databases. − FusionCube supports mainstream virtualization platforms, such as FusionSphere and VMware vSphere. − A single system supports hybrid deployment of virtualization and database applications. In conclusion, FusionCube helps customers: Quicken service rollout, service adjustment, and capacity expansion and fasten business profit-making. Improve resourced utilization and reduce the costs on hardware, power, and maintenance to bring down IT costs. Implement unified O&M management, making IT delivery and maintenance easier and reducing labor costs. 2.1.1 Pre-integration FusionCube has all its components installed before delivery, simplifying onsite installation and commissioning. The commissioning duration is shortened from several weeks or even months to only hours. FusionCube pre-integration includes: Hardware preinstallation: Devices are installed in cabinets and cables are connected properly. Software preinstallation: FusionCube has the following software installed before delivery: customized basic input and output system (BIOS), virtualization software FusionCompute, platform management software FusionCube Center, and storage management software FusionStorage. Integrated system commissioning: System configuration integrity and connectivity are checked to ensure that the system configuration meets contract requirements and all internal components can communicate with each other. Cabinet shipment: The cabinet is assembled and shipped for delivery. After FusionCube is delivered to the site and powered on, users only need to perform a hardware check, network interconnection check, and service configuration and commissioning for the system. 2.1.2 Compatible with Mainstream Virtualization Platforms FusionCube supports mainstream virtualization platforms, such as FusionSphere and VMware vSphere. FusionCube provides unified computing, storage, and network resources for virtualization platforms. FusionCube supports preinstallation and automatic installation and deployment of virtualization platform software and FusionCube software, improving system deployment efficiency. FusionCube incorporates resource monitoring of the virtualization platform. It implements unified O&M through a management interface. Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 4 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 2 Huawei Solution 2.1.3 Convergence of Computing, Storage, and Networking Resources FusionCube integrates computing, storage, and network resources. FusionCube is prefabricated with computing, network, and storage devices in an out-of-the-box packaging. Users do not need to purchase extra storage or network devices. A distributed storage engine is deployed on compute nodes (server blades) to implement convergence of computing and storage resources, which reduces data access delay and improves overall access efficiency. FusionCube implements automatic network deployment and network resource configuration. In addition, the automatic network resource configuration is dynamically associated with computing and storage resource allocation. 2.1.4 Distributed Storage The build-in FusionStorage provides distributed storage services for FusionCube. FusionStorage uses an innovative cache algorithm and adaptive data distribution algorithm based on a unique parallel architecture, which eliminates high data concentration and improves system performance. FusionStorage also allows rapid automatic self-recovery and ensures high system availability and reliability. FusionStorage has the following features: Issue 1.0 (2016-03-31) Linear scalability and elasticity FusionStorage uses the distributed hash table (DHT) to distribute all metadata among multiple nodes. This mode prevents performance bottlenecks and allows linear expansion. FusionStorage leverages innovative data slicing technology and DHT-based data routing algorithm to evenly distribute volume data to fault domains of large resource pools. This allows load balancing on hardware devices and higher input/output operations per second (IOPS) and megabit per second (MBPS) performance of each volume. High performance FusionStorage uses a lock-free scheduled I/O software subsystem to prevent conflicts of distributed locks. The time delay is greatly reduced as there is no lock operations or metadata queries on I/O paths. Distributed stateless engines make hardware nodes to be fully utilized, greatly increasing the concurrent IOPS and MBPS of the system. In addition, the distributed SSD cache technology and large-capacity SATA disks (serving as the primary storage) ensure high performance and large storage capacity. High reliability FusionStorage supports multiple data redundancy and protection mechanisms. Two or three copies can be retained for each piece of data. FusionStorage supports configuration of flexible data reliability policies, allowing data copies to be stored on different servers. Data will not be lost even if a server is faulty. FusionStorage also protects valid data slices against loss. If a hard disk or server is faulty, valid data can be rebuilt concurrently. It takes less than 30 minutes to rebuild data of 1 TB. All these measures improve system reliability. Rich advanced storage functions FusionStorage provides a wide variety of advanced storage functions, such as thin provisioning, volume snapshot, and linked clone. Thin provisioning uses virtualization technology to give the appearance of having more physical resources than are actually available. The system allocates physical storage space only when data is written into volumes. Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 5 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 2 Huawei Solution Volume snapshot saves the state of the data on a logical volume at a certain point. The number of snapshots is not limited, and performance does not deteriorate. FusionStorage provides linked clone based on incremental snapshots. A snapshot can be used to create multiple cloned volumes. When a cloned volume is created, the data on the volume is the same as the snapshot. Subsequent modifications on the cloned volume do not affect the original snapshot and other cloned volumes. 2.1.5 Hybrid Virtualization and Database Deployment FusionCube supports co-existence of virtualization and physical deployment. The virtualization deployment improves resource utilization and implements cloud-based management. Moreover, deployment of database services on physical servers improves database performance. FusionCube provides computing, storage, and network resources for virtualization and database applications and implements unified O&M. FusionCube supports automatic installation and deployment of physical servers and provides wizard configuration of storage and network resources for the database through a graphical user interface. This feature greatly shortens the database service rollout time. 2.1.6 Automatic Deployment FusionCube supports automatic deployment, which simplifies operations on site and increases deployment quality and efficiency. FusionCube supports preinstallation, pre-integration, and pre-verification before the delivery, which simplifies onsite installation and deployment and reduces the deployment time. Devices are automatically discovered upon the system is powered on. Wizard system initialization configuration is provided to implement initialization of computing, storage, and network resources. An automatic deployment tool is provided to help users to rapidly change and upgrade their virtualization platforms. 2.1.7 Unified O&M FusionCube supports unified management of hardware devices (such as servers and switches) and resources (including computing, storage, and network resources). It can greatly improve O&M efficiency and quality of service (QoS). A unified management interface is provided for managing hardware devices and monitoring computing, storage, and network resources on a real-time basis. The IT resource usage and system operating status are automatically monitored. Alarms are reported for system faults and potential risks in real time, and alarm notifications can be sent to O&M personnel by email. FusionCube allows rapid, automatic capacity expansion. It supports automatic discovery of devices to be added and provides wizard-based configuration for system expansion. Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 6 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 2 Huawei Solution 2.2 FusionCube Networking The FusionCube networking logically consists of the service, storage, and management planes, which are isolated from each other, preventing the basic platform from being damaged. Storage plane: Storage devices on the server communicate with each other over layer 2 storage network. Storage devices provide storage resources for virtual machines (VMs) through the virtualization platform but do not communicate with VMs directly. Service plane: provides a channel for users to obtain services, for virtual NICs of VMs to communicate with each other, and for external applications to interact with the FusionCube system. VLANs can be configured to isolate access to different service planes. Management plane: supports system management, service deployment, and system loading. The BMC plane manages servers and is isolated from the management plane. FusionCube supports the service, management, and storage planes that are converged at a port and logically separated through VLANs or that are physically separated through independent ports. 2.2.1 Internal Networking The internal networking of FusionCube integrated with FusionSphere is different from that of FusionCube integrated with VMware vSphere. Figure 2-2 Networking diagram of FusionCube components (integrated with FusionSphere) Table 2-1 Nodes of FusionCube Node Description Deployment Rules Management Computing Node Agent (MCNA) A host that implements management, computing, and storage functions. FusionCube Center and FusionStorage VMs are deployed on the MCNA. Two MCNAs must be deployed. Storage Computing Node Agent (SCNA) A node that provides storage and computing functions One or more SCNAs can be deployed based on service requirements. Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 7 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 2 Huawei Solution Node Description Deployment Rules Computing Node Agent (CNA) A node that manages VMs and virtual volumes Zero to multiple CNAs can be deployed based on service requirements. Database Node (DBN) A host that provides database services in a database cluster Zero to multiple DBNs can be deployed based on service requirements. The management plane is the device management plane, and the BMC plane is the server hardware management plane. The management and BMC planes can share the same network segment or use different network segments. The storage plane refers to the FusionStorage internal communication plane. The service plane refers to service-related communication planes except for BMC, management, and storage planes. The communication planes can be divided into one or multiple networks based on service requirements. Figure 2-3 Networking diagram of FusionCube components (integrated with VMware vSphere) The management plane is the device management plane, and the BMC plane is the server hardware management plane. The management and BMC planes can share the same network segment or use different network segments. The storage plane includes the FusionStorage internal communication plane and the communication plane between the ESXi iSCSI target and the FusionStorage iSCSI interface. The two planes share the same network segment, but use different IP addresses. The service plane refers to service-related communication planes except for BMC, management, and storage planes. The communication planes can be divided into one or multiple networks based on service requirements. The interface between the CVM and the ESXi is used for internal communication, and the IP address of the interface is 169.254.1.1 or 169.254.1.2. 2.2.2 External Networking FusionCube connects to external networks through a layer 2 network in multiple interconnection modes. Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 8 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 2 Huawei Solution Physical Isolation Between Management and Service Planes Planes on external networks are physically isolated by different switches. For example, the management and service planes are physically isolated by different switches. Convergence of Management and Service Planes User switches work in active/standby mode using Virtual Router Redundancy Protocol (VRRP). User switches and FusionCube switching devices support Eth-Trunk (or port aggregation) and SmartLink. Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 9 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 2 Huawei Solution User switches or routers work in active/standby mode using VRRP. An independent VLAN is used to control VRRP heartbeats on the switches. User switches and the FusionCube switching devices support Eth-Trunk (or port aggregation). 2.3 FusionStorage Features The Huawei-developed built-in FusionStorage distributed storage system provides distributed storage services for FusionCube. FusionStorage uses an innovative cache algorithm and adaptive data distribution algorithm based on a unique parallel architecture, which eliminates high data concentration and improves system performance. FusionStorage also allows rapid automatic self-recovery and ensures high system availability and reliability. 2.3.1 SCSI/iSCSI Block Interface FusionStorage provides block interfaces using the virtual block store (VBS) over the Small Computer System Interface (SCSI) or Internet Small Computer Systems Interfaces (iSCSI) protocol. FusionSphere and the keyboard, video, and mouse (KVM) provide storage access and physical deployment capabilities over SCSI for the local server installed with the VBS. VMware and Microsoft SQL Server clusters provide the storage access capability over iSCSI for the VM and local server which are not installed with the VBS. Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 10 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 2 Huawei Solution SCSI provides SCSI-3 persistent reserved locks and non-persistent reserved locks. Persistent reserved locks are used for HANA clusters, and non-persistent reserved locks are used for MSCS clusters. Users access storage devices by connecting the local Initiator to the iSCSI target provided by the VBS. FusionStorage supports the following standards for secure access over iSCSI: FusionStorage supports Challenge Handshake Authentication Protocol (CHAP) identity verification for ensuring that client access is reliable and secure. CHAP periodically verifies the identity of a client by using a three-way handshake. This happens at the time of establishing the initial link, and may happen again at any time afterwards. CHAP provides protection against replay attack from the peer by using an incrementally changing identifier and of a variable challenge-value. FusionStorage authorizes hosts to access LUNs using LUN Masking. LUNs are local devices for SAN storage hosts. To maintain host data, users need to isolate LUNs for each host, to prevent host data from being damaged. LUNs and the world wide name (WWN) addresses of host bus adapters (HBAs) are bound using LUN Masking, to ensure that the LUNs can be accessed only by specified hosts or host clusters. The relationship between hosts and LUNs can be multiple-to-one or one-to-multiple. The one-to-multiple mapping meets storage requirements of small LUNs in virtualization scenarios, and the multiple-to-one mapping meets requirements for shared volumes of cluster systems such as Oracle RAC. LUN Masking is implemented through mappings among ports, hosts, host groups, and LUNs. Figure 2-4 LUN Masking component relationship diagram LUN Port Host Port LUN HostGroup Port Host LUN Port LUN Port Host LUN Issue 1.0 (2016-03-31) Port Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 11 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 2 Huawei Solution LUNs and storage device ports are bound using LUN mapping. Hosts are connected to ports using different LUNs. If hosts running different application systems are in different geographical addresses, a storage system provides the application systems with data storage services simultaneously using LUN mapping. 2.3.2 Thin Provisioning FusionStorage provides the thin provisioning function, which allows users to use much more storage space than that actually available on the physical storage device. This function significantly improves storage utilization when compared with thick provisioning. FusionStorage uses the DHT mechanism. No centralized metadata is required for recording thin provisioning information. Compared with traditional SAN storage devices, FusionStorage does not cause system performance deterioration. Figure 2-5 Mechanism of automatic thin provisioning 2.3.3 Snapshot FusionStorage provides the snapshot mechanism, which allows the system to capture the status of the data written into a logical volume at a specified time. The data snapshot can be exported and used for restoring the volume data when required. FusionStorage uses the redirect-on-write (ROW) technology when storing snapshot data. Snapshot creation does not deteriorate the performance of the original volume. Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 12 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 2 Huawei Solution Figure 2-6 FusionStorage snapshot mechanism 2.3.4 Linked Cloning FusionStorage provides the linked cloning mechanism so that multiple cloned volumes can be created for a volume snapshot. The data in the cloned volumes is the same as that in the snapshot. Subsequent modifications to a cloned volume do not affect the snapshot or other cloned volumes. FusionStorage supports a linked cloning ratio of 1:256, which significantly improve storage space utilization. A cloned volume inherits all functions of a base volume. You can create snapshots for a cloned volume, use the snapshots to restore the data in the cloned volume, and clone the data in the cloned volume. Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 13 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 2 Huawei Solution Figure 2-7 Linked cloning mechanism 2.3.5 Elastic Scaling The distributed architecture of FusionStorage supports elastic scaling without any performance deterioration. DHT Routing Algorithm FusionStorage employs the DHT routing algorithm to implement addressing and storage of the system data. Each storage node stores a small part of system data. Traditional storage devices use the centralized metadata management mechanism, which allows metadata to record the hard disk distribution of the logical unit number (LUN) data with different offsets. For example, the 4 KB data identified by an address starting with LUN1+LBA1 is stored on LBA2 of the 32nd hard disk. On a traditional storage device, each I/O operation initiates a query request to the metadata service. As the system scale grows, the metadata size also increases. However, the concurrent operation capability of the system is subject to the capability of the server running the metadata service. As a result, the metadata service eventually becomes a performance bottleneck of the system. Different from traditional storage devices, FusionStorage employs the DHT algorithm for data addressing. Figure 2-8 illustrates the mechanism of the DHT algorithm on FusionStorage. Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 14 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 2 Huawei Solution Figure 2-8 Mechanism of the DHT algorithm on FusionStorage FusionStorage sets the hash space to 232 and divides the hash space into N equal parts. Each part is a partition, and all these partitions are evenly allocated to hard disks in the system. For example, the system requires 3600 partitions by default. If the system has 36 hard disks, each hard disk is allocated 100 partitions. The partition-hard disk mapping has been configured during system initialization and will be flexibly adjusted when the number of hard disks in the system changes. The mapping table requires only small space, and FusionStorage nodes store the mapping table in the memory for rapid routing purposes. In this regard, the routing mechanism of FusionStorage is different from that of any conventional storage arrays. FusionStorage does not employ the centralized metadata management mechanism and therefore eliminates the performance bottleneck of the metadata service. An example is provided as follows: When an application needs to access the 4 KB data identified by an address starting with LUN1+LBA1, FusionStorage first constructs "key=LUN1+LBA1/1M", calculates the hash value for this key, performs modulo operation for the value N, gets the partition number, and then obtains the hard disk of the data based on the partition-hard disk mapping. The DHT algorithm has the following characteristics: Balance: Data is distributed to all nodes as evenly as possible, thereby balancing load among all nodes. Monotonicity: When new nodes are added to the system, system data is evenly distributed to all nodes again. However, data migration is implemented only on new nodes, and the data on the existing nodes is not significantly adjusted. Smooth Expansion FusionStorage uses the distributed architecture to support easy capacity expansion and ultra-large storage capacity. FusionStorage ensures rapid load balancing after capacity expansion and avoids migration of a large amount of data. FusionStorage supports flexible capacity expansion and allows both concurrent and separate expansion of computing nodes, hard disks, and storage nodes. When compute nodes are added, the storage capacity is also added. After capacity expansion, computing and storage resources are still integrated. FusionStorage evenly distributes caches and bandwidths of engines to all nodes, ensuring linear increase of the system IOPS, throughput, and cache with the number of nodes. Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 15 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 2 Huawei Solution Figure 2-9 Mechanism of smooth capacity expansion on FusionStorage Excellent Performance FusionStorage uses its distributed architecture to organize the dispersedly distributed, low-efficiency SATA or SAS disks into an efficient storage pool that provides higher I/O throughput than SAN devices, maximizing the storage performance. FusionStorage uses SSDs to replace HDDs as high-speed storage devices and the InfiniBand network to replace GE or 10GE networks to provide higher bandwidth. Therefore, FusionStorage is ideal for processing massive data in real time and meets the performance-demanding requirements. Distributed Engine FusionStorage uses distributed stateless engines. These engines are deployed on each server that requires access to FusionStorage, thereby preventing performance bottlenecks that may be caused by traditional, centrally deployed engines. Moreover, these distributed engines deployed on standalone servers consume much fewer CPU resources but provide much higher IOPS and throughput than centrally deployed engines do. An example is provided as follows: A system has 20 servers that need to access the storage resources provided by FusionStorage, and the bandwidth that each server provides for the storage plane is 2 x 10 Gbit/s. One VBS module (storage engine) is deployed on each server, so that the total throughput can reach 400 Gbit/s (20 x 2 x 10 Gbit/s). With the growth of the cluster scale, storage engines can be linearly added, thereby eliminating the performance bottlenecks that may be caused by centralized engines in traditional dual-controller or multi-controller storage systems. Distributed Cache FusionStorage integrates computing and storage resources and evenly distributes caches and bandwidths to all servers. Each disk on FusionStorage servers uses independent I/O bandwidths, preventing a large number of disks from competing for limited bandwidths between computing devices and storage devices in an independent storage system. FusionStorage can use certain server memory as the read cache and NVDIMMs or SSDs as the write cache. Caches are evenly distributed to all nodes. The total cache size on all servers Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 16 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 2 Huawei Solution is far greater than that provided by external storage devices. Even when using large-capacity, low-cost SATA disks, FusionStorage can still improve the I/O performance one to three times. FusionStorage can use SSDs for caching data. In addition to providing high capacity and the write cache function, the SSDs can collect statistics on and cache hotspot data, further improving system performance. Global Load Balancing The DHT mechanism of FusionStorage ensures that the I/O operations performed by upper-layer applications are evenly distributed on the hard disks of various servers to achieve global load balancing. The system automatically distributes data blocks on the hard disks of various servers. Data that is frequently or seldom used is evenly distributed on the servers, thereby preventing hotspots in the system. FusionStorage employs the data fragment distribution algorithm to ensure that active and standby copies are evenly distributed to different hard disks of the servers. In this way, each hard disk contains the same number of active and standby copies. When a node is added or deleted due to a failure, FusionStorage employs the data rebuilding algorithm to balance load among all nodes after system rebuild. Distributed SSD Storage Huawei FusionStorage supports distributed SSDs, which provide higher read and write performance than traditional SATA or SAS HDDs. FusionStorage virtualizes the PCIe SSD cards configured on storage nodes into a virtual storage resource pool to provide high-performance read and write for applications. FusionStorage supports Huawei-developed SSD cards and mainstream PCIe SSD cards. High-Speed InfiniBand Network FusionStorage supports the InfiniBand network designed for high-bandwidth, low-latency applications. The InfiniBand network has the following characteristics: Provides a data rate of 56 Gbit/s and allows high-speed connection setup. Uses standard multilayer fat-tree networking and allows smooth capacity expansion. Provides a communication network where congestion hardly occurs and avoids data switching bottlenecks. Provides minor communication delays within nanoseconds and transmits computing and storage information promptly. Provides lossless network QoS and ensures data integrity during transmission. Allows multi-path communication for active and standby ports to improve transmission reliability. Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 17 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 2 Huawei Solution 2.3.6 High Reliability Cluster Management FusionStorage manages the system in clusters to eliminate single point of failures. A faulty server or hard disk can be automatically isolated from the cluster, minimizing adverse impact on system services. Cluster management is as follows: ZooKeeper: provides primary arbitration to the MDC. A ZooKeeper also stores metadata generated during system initialization, including such data address information as the "partition-hard disk" mapping. The number of deployed ZooKeepers is an odd integer greater than 3. If the number of faulty ZooKeepers exceeds half of the total ZooKeeper count, ZooKeepers cannot work properly. Once the ZooKeepers are deployed, they cannot be expanded. MDC: works in clusters. Three MDC modules are deployed initially. When a resource pool is added, an MDC is automatically started or assigned for this resource pool. Multiple MDCs use a ZooKeeper to elect an active MDC that monitors other MDCs. If the active MDC detects an MDC fault, it restarts the MDC or appoints an MDC for the resource pool. When the active MDC is faulty, a new active MDC will be elected. FusionStorage Manager: works in active/standby mode. Two FusionStorage Manager nodes are deployed. OSD: works in active/standby mode. The MDC monitors the OSD status in real-time. When the active OSD where a specified partition resides is faulty, the storage services will automatically switch to the standby OSD in real-time, ensuring service continuity. Multiple Data Copies To ensure data reliability, FusionStorage stores one piece of data into two or three identical data copies. Before storing data, FusionStorage fragments the data to be stored on each volume in the system at the granule of 1 MB and then stores the data fragments on servers in the cluster based on the DHT algorithm. Figure 2-10 shows the multiple data copies that FusionStorage stores. As shown in Figure 2-10, P1' on disk 2 of server 2 is a copy of data block P1 on disk 1 of server 1. P1 and P1' are two data copies of the same data block. If disk 1 becomes faulty, P1' can take the place of P1 to provide storage services. Figure 2-10 Multiple data copies stored by FusionStorage Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 18 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 2 Huawei Solution Data Consistency When a user successfully writes application data into the storage system, the data copies in the storage system must be consistent, so that data read from any copy is the original data. FusionStorage uses multiple methods to ensure data consistency between data copies: Synchronous write of data copies When the VBS module sends a write request to the specified active OSD module, the OSD module synchronizes this write request to the standby OSD module while writing it to the local hard disk. This synchronization process is implemented based on the I/O number, thereby ensuring that the sequence of the I/O operations received by the active OSD module is the same as that synchronized to the standby OSD module. After the write request is processed on both the active and standby OSD modules, a success message is returned to the application. Figure 2-11 shows the process. Figure 2-11 Synchronous write of data copies Read repair FusionStorage supports the read repair mechanism. If failing to read data, FusionStorage automatically identifies the failure location. If the data cannot be read from a disk sector, FusionStorage retrieves the data from other copies of the data on another node and writes the data back into the original disk sector. This mechanism ensures the correct number of data copies and data consistency among data copies. Rapid Data Rebuild Each hard disk in the FusionStorage system stores multiple data blocks (partitions), whose data copies are scattered on other nodes in the system based on certain distribution rules. If detecting a hard disk or server fault, FusionStorage automatically repairs data in the background. The repair mechanism allows FusionStorage to simultaneously restore a minimal amount of data on different nodes because the data copies are stored on different storage nodes. This mechanism prevents performance deterioration caused by restoration of a large amount of data on a single node, and therefore minimizes adverse impacts on upper-layer services. Figure 2-12 shows the process for automatic data rebuild. Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 19 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 2 Huawei Solution Figure 2-12 Data rebuild process Fragments and stores data. A hardware fault occurs. Restores data concurrently from multiple nodes. Automatically detects a hardware fault. Automatically reconstructs data copies FusionStorage supports parallel and rapid troubleshooting and data rebuild: Data blocks (partitions) and their copies are scattered in a resource pool. If a hard disk is faulty, its data can be automatically rebuilt in the resource pool efficiently. Data is distributed to different servers so that data can be obtained or rebuilt even if a server is faulty. Load can be automatically balanced between existing nodes in the event of node failures or capacity expansion. You do not need to adjust application configuration to obtain larger capacity and higher performance. Power Failure Protection A server power failure may occur while the system is running. To prevent data loss in such cases, FusionStorage uses SSDs to store metadata and cached data. A running program writes metadata and cached data to SSDs. When a server has a power failure and restarts, the system automatically restores the metadata and cached data from the SSDs. Figure 2-13 shows the PCIe SSD card and SSD used by FusionStorage. Figure 2-13 PCIe SSD card, and SSD used by FusionStorage Hard Disk Reliability FusionStorage supports self-monitoring analysis and report technology (SMART), slowly rotating disk detection, hard disk SCSI fault handling, and hard disk scan. After performing these detections, the system can conduct read repair, remove a faulty disk, rebuild data, mark a bad block, scan valid data in disks, handle errors caused by the exceeded SMART threshold, and handle slowly rotating disks. Issue 1.0 (2016-03-31) Valid data disk scan Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 20 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 2 Huawei Solution The system periodically scans the valid data in a disk to prevent silent data corruption. If a bad sector is detected during a scan, the system performs read repair to repair the faulty sector. Bad sector table (BST) When the system detects a bad sector in a data scanning or read process, an access error is reported. FusionStorage attempts read repair first. If all the redundant copies of the data are unavailable, the system tags the bad sectors with BST and generates an alarm, instructing the application layer to repair the data. Hard disk health check FusionStorage monitors the SMART information and I/O processing of a hard disk, identifies a hard disk in the unhealthy state, automatically rebuilds data originally stored in this hard disk, and removes the hard disk from the cluster. Hard disk error detection During I/O processing, FusionStorage proactively identifies hard disk errors, such as WP, ABRT, and DF errors. If detecting such an error, FusionStorage automatically rebuilds data and removes the faulty hard disk from the cluster. 2.3.7 Multiple Resource Pools FusionStorage supports multiple resource pools to allow storage media of different performance and fault isolation. A set of FusionStorage Manager manages multiple resource pools, which share the same FusionStorage cluster, ZooKeepers, and active MDC. Each resource pool has a home MDC. When a resource pool is created, an MDC automatically starts as the home MDC of the resource pool. The maximum size of resource pools is 128, and that of MDCs is 96. If there are more than 96 resource pools, the existing MDCs will be appointed as the home MDC for the excessive resource pools. Each MDC manages a maximum of 2 resource pools. The home MDC of a resource pool is responsible for the initialization of the resource pool. The initialization divides storage resources into partitions and stores the views of the partitions and OSD in a ZooKeeper. If the home MDC of a resource pool is faulty, the active MDC appoints another MDC for the resource pool. FusionStorage supports offline volume migration between multiple resource pools. Multiple resource pools are planned according to the following guidelines: A resource pool supports a maximum of 96 hard disks for 2 copies, or 2048 hard disks for 3 copies. For more hard disks, a new resource pool must be planned. The hard disks in a resource pool are of the same type. Hard disks of different types are assigned to different resource pools. The hard disks in a resource pool are of the same capacity; where their capacities are different, they are used as if their capacities are of the smallest. The cache media in a resource pool are of the same type. Cache media of different types are assigned to different resource pools. The resource pools should be so created that the number of hard disks in each storage node should preferably be the same and by no means less than 33% of the maximum number of hard disks in a storage node; the allowed difference is 2. The resource pools should be so created that the number of hard disks in each storage node should preferably be the same and by no means less than 33% of the maximum number of hard disks in a storage node. The allowed difference is 2. Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 21 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 2 Huawei Solution The hard disks in a server can be of different types. The resource pools should be so created that the hard disks of different types in a server should be assigned to different resource pools. 2.3.8 VMWare VAAI To improve performance, VMWare provides the vStorage APIs for Array Integration (VAAI) that transfers storage-related operations to the storage devices. The APIs provided by FusionStorage have the following features: Copy offload Copy offload is also called Full Copy or Clone Blocks. When a VM clones or creates a VM according to a template, a lot of data blocks need to be copied. Copy offload enables offline copy on a storage node, sparing the ESXi server and reducing its overhead. This feature can be used in vMotion. Figure 2-14 illustrates the copy offload feature. Figure 2-14 Copy offload Block Zeroing After a VM is created, it is wise to clear the virtual disks immediately at no cost of future performance. Block zeroing greatly reduces the interaction between an ESX host and a storage node. Figure 2-15 illustrates the block zeroing feature. Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 22 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 2 Huawei Solution Figure 2-15 Block zeroing Automatic Test and Set (ATS) The virtual machine file system (VMFS) allows multiple hosts to concurrently access a shared logical volume. This function is a prerequisite for vMotion. The VMFS has a built-in security mechanism that prevents a VM from being accessed or modified concurrently by more than one host. The traditional file locking mechanism used by VSphere is the SCSI reservation that uses the RESERVE SCSI command to lock a logical volume when a storage-related instruction, such as when an incremental snapshot increases or occurs, is executed. This mechanism prevents conflicts at the cost of delayed storage because before a host can write data, it needs to wait for the RELEASE SCSI command that unlocks the logical volume. Atomic test and set (ATS) is a hardware-assisted locking mechanism that locks a storage array offline, specifically individual data blocks instead of a whole logical volume. ATS enables the remaining part of a logical volume to be accessed by a host when other part of the logical volume is locked, avoiding performance deterioration. When working with VMFS, ATS allows more hosts deployed in a cluster and more virtual hosts deployed in a logical volume. Figure 2-16 illustrates the ATS feature. Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 23 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 2 Huawei Solution Figure 2-16 ATS UNMAP/Reclaim command The lower layer of the VMFS is blocks. After a VM is deleted or migrated, the space for VMFS becomes larger, though the lower layer has not reclaimed the space. When the storage layer provides an API for the VMFS to invoke, the ESXi uses this interface to notify the lower layer to reclaim the space. When a VM is migrated or deleted from a DataDtore, the bottom layer invokes the VPI that is provided to it to reclaim the array space. Figure 2-17 illustrates the UNMAP feature. Figure 2-17 UNMAP Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 24 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 3 3 Hardware Convergence Hardware Convergence 3.1 FusionCube 9000 FusionCube 9000 uses Huawei proprietary E9000 blade servers as its hardware platform. 3.1.1 E9000 Chassis The E9000 server houses compute nodes and switch or pass-through modules in a 12 U chassis. The E9000 provides the following functions and features: An E9000 chassis can be configured with up to 8 full-width or 16 half-width compute nodes in the front. A half-width slot provides cooling capacity of up to 850 W, and a full-width slot provides cooling capacity of up to 1700 W. A half-width compute node supports up to 2 processors and 24 DIMMs, and a full-width compute node supports up to 4 processors and 48 DIMMs. A chassis supports a maximum of 32 processors and a maximum memory capacity of 12 TB. The midplane supports a maximum switching capacity of 5.76 Tbit/s. A chassis provides four (two pairs) slots for switch modules, which support Ethernet, Fibre Channel (FC), Fibre Channel over Ethernet (FCoE), and IB protocols and provide pass-through I/O ports. Figure 3-1 E9000 appearance FusionCube supports three E9000 chassis in a cabinet. Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 25 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 3 Hardware Convergence 3.1.2 E9000 Blades FusionCube supports the following blades (compute nodes): CH121 V3 2-socket compute node CH222 V3 2-socket compute and storage node CH220 V3 2-socket IO expansion node CH242 V3 4-socket compute node CH121 V3 Figure 3-2 CH121 V3 Category Specifications Form factor 2-socket half-width compute node CPU One or two Intel® Xeon® E5-2600 v3 series processors Memory Maximum number of DIMMs: 24 DDR4 DIMMs Maximum memory capacity: 1.5 TB Hard disk 2 x 2.5" SAS or SATA HDD or PCIe SSD RAID RAID 0 and RAID 1 Built-in flash Two MicroSD cards Two SATADOMs One USB 3.0 Two PCIe x16 mezzanine cards One standard full-height half-length (FHHL) PCIe x16 expansion card PCIe expansion Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 26 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 3 Hardware Convergence CH220 V3 Figure 3-3 CH220 V3 Category Specifications Form factor 2-socket full-width compute node CPU One or two Intel® Xeon® E5-2600 v3 series processors Memory Maximum number of DIMMs: 16 DDR4 DIMMs Maximum memory capacity: 1 TB Hard disk 2 x 2.5" SAS or SATA HDD or PCIe SSD RAID RAID 0 and RAID 1 Built-in flash Two MicroSD cards Two SATADOMs One USB 3.0 Four mezzanine cards (2 PCIe x16 + 2 PCIe x8) Six slots for PCIe3.0 x16 standard expansion cards in any of the following combinations: PCIe expansion – Six FHHL PCIe cards – One full height full length (FHFL) PCIe card (occupying two slots) + four FHHL PCIe cards – Two FHFL PCIe cards (each occupying two slots) Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 27 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 3 Hardware Convergence CH222 V3 Figure 3-4 CH222 V3 Category Specifications Form factor 2-socket full-width compute node CPU One or two Intel® Xeon® E5-2600 v3 series processors Memory Maximum number of DIMMs: 24 DDR4 DIMMs Maximum memory capacity: 1.5 TB Hard disk 15 x 2.5" SAS or SATA HDD or SSD RAID RAID 0 and RAID 1 Built-in flash Two MicroSD cards Two SATADOMs One USB 3.0 Two PCIe x16 mezzanine cards One standard FHHL PCIe x16 card PCIe expansion CH242 V3 Figure 3-5 CH242 V3 Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 28 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 3 Hardware Convergence Category Specifications Form factor Full-width 4-socket compute node CPU 2 or 4 Intel® Xeon® E7 v2 or v3 series processor, up to 18 cores 165 W When equipped with Intel® Xeon® E7 v2 processors, the compute node supports up to 32 DDR3 DIMMs and 1600 MHz. When equipped with Intel® Xeon® E7 v3 processors, the compute node supports up to 32 DDR4 DIMMs and 1866 MHz. Memory Hard disk 8 x SSD or SAS/SATA HDD RAID RAID 0 and RAID 1 Built-in flash Two MicroSD cards One USB 3.0 Four PCIe x16 mezzanine cards Two standard FHHL PCIe x16 cards PCIe expansion 3.1.3 High-Performance Switch Modules FusionCube uses CX310 and CX611 switch modules. CX310 FusionCube uses CX310 switch modules, which support 10GE network. Each chassis can be configured with two switch modules. Figure 3-6 shows a CX310 10GE switch module. Figure 3-6 CX310 10GE switch module Category Specifications Model CX310 10GE switch module Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 29 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 3 Hardware Convergence Category Specifications Network port Uplink ports: 16 x 10GE port Downlink ports: 32 x 10GE port L2: VLAN, MSTP, LACP, TRILL, Stack, and IGMP L3: RIP, OSPF, ISIS, BGP, VRRP, BFD, and PIM QoS: DCBX, PFC, ETS, ACL, CAR, and DiffServ Security: IPSG, MFF, DAI, FSB, and DHCP Snooping Network features Management port Two RS232 serial ports (1 service port + 1 management port) CX611 Figure 3-7 CX611 InfiniBand switch module Category Specifications Model CX611 InfiniBand switch module Network port Uplink ports: 18 x QDR or FDR QSFP+ port Downlink ports: 16 x 4X QDR or FDR port (one 4X QDR or FDR in each half-width slot) Network features QDR/FDR auto-negotiation, for applications demanding low delay and high bandwidth Management port Two RS232 serial ports (1 service port + 1 management port) 3.2 FusionCube 6000 FusionCube 6000 uses Huawei proprietary X6800 high-density servers as its hardware platform. Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 30 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 3 Hardware Convergence 3.2.1 X6800 Chassis The X6800 is a new-generation high-density server designed for Internet, high-performance computing (HPC), cloud computing, and data center applications. It has the following features: Supports server nodes of different types and widths in a 4 U chassis. Allows all server nodes in a chassis to share PSUs in 1+1 or 2+2 redundancy. Allows all server nodes in a chassis to share fan modules in N+1 redundancy. Supports network controller sideband interface (NC-SI) and provides service and management ports from the front and service ports from the rear of the server. The X6800 provides high density, reliability, and energy efficiency. It features easy maintenance and management as well as flexible expansion. The X6800 supports a wide range of server nodes to meet different service requirements. The X6800 has the following functions and features: High density Unified management and easy maintenance Shared architecture and high energy efficiency Redundancy design and high reliability High Density The X6800 provides higher density than a conventional rack server, which saves floor space in equipment rooms. The X6800 provides computing density twice that of a conventional 1 U rack server and four times that of a conventional 2 U rack server in the same rack space, which greatly improves space utilization in an equipment room. The X6800 provides storage density twice that of a conventional 1 U rack server in the same rack space. When fully configured with 4U4 server nodes, an X6800 cabinet supports up to 480 3.5" hard disks. An X6800 cabinet in maximum configuration supports up to eighty 4U8 server nodes, with up to 160 processors and 80 TB memory capacity. An X6800 chassis can hold four 4U4 server nodes or eight 4U8 server nodes in a 4 U chassis. A 4U4 server node occupies two slots, and a 4U8 server node occupies one slot. Unified Management and Easy Maintenance The X6800 leverages the blade server architecture to provide unified management and easy maintenance. The X6800 uses the Intelligent Baseboard Management Controller (iBMC) and Hyper Management Module (HMM) to implement unified server management. By incorporating advantages of rack and blade servers, the X6800 supports front maintenance and front and rear cabling. This feature meets deployment requirements of traditional equipment rooms (requiring rear cabling) and new equipment rooms (requiring front cabling). The X6800 also supports maintenance in the cold air area. The X6800 adopts a modular design and hot-swappable key components, greatly improving O&M efficiency. Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 31 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 3 Hardware Convergence Shared Architecture and High Energy Efficiency All server nodes in an X6800 chassis share the power supplies and heat dissipation system. The server nodes share four PSUs and five fan modules in an X6800 chassis, which simplifies deployment and increases PSU and fan module utilization. The X6000 uses the Dynamic Energy Management Technology (DEMT) to control system energy consumption and increase the energy efficiency. Redundancy Design and High Reliability The X6800 uses reliable system architecture to ensure stable, long-term operation. The X6800 uses a passive backplane to minimize impact from a single point of failure (SPOF). The X6800 supports redundant fan modules and PSUs as well as RAID configuration, preventing data loss and service interruption. The X6800 uses carrier-class components and manufacturing processes to ensure high stability and long life cycle. Figure 3-8 Front view Figure 3-9 Rear view 3.2.2 X6800 Server Nodes The XH628 V3 server node (XH628 V3 for short) is a 2-socket dual-slot storage node used in Huawei X6800 servers. It uses an innovative design to deliver high storage density and performance while breaking through power limits. It is also easy to manage and maintain. Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 32 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 3 Hardware Convergence Figure 3-10 XH628 V3 Category Specifications Form factor 2-socket dual-slot server node Number of processors 2 Processor model Intel® Xeon® E5-2600 v3 Memory 16 DDR4 DIMMs Hard disk 12 x 3.5" or 2.5" SAS/SATA HDD or SSD + (optional) 2 x 2.5" SATA HDD or SSD RAID RAID 0 and RAID 1 LOM Two or four GE and two 2 10GE onboard network ports PCIe expansion Five PCIe slots: Two front HHHL PCIe3.0 x8 cards Two rear HHHL PCIe3.0 x8 cards One one RAID controller card If front PCIe cards are configured, the two 2.5" hard disks cannot be configured due to space limitations. On-board storage medium Two Mini SSDs (SATA DOMs) and one USB flash drive Front port One universal connector port (UCP) The UCP connects to a multi-port cable for service, maintenance, and OS installation. The multiple-port cable provides one VGA port, three USB 2.0 ports, and one RJ45 serial port. Issue 1.0 (2016-03-31) One GE management port Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 33 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 4 4 Typical Configurations Typical Configurations Huawei provides typical configurations of FusionCube 6000 and FusionCube 9000 hyper-converged virtualization infrastructures. Huawei also provides customized solutions to meet requirements of small- and medium-sized enterprise data centers and large data centers. 4.1 FusionCube 6000 Table 4-1 Typical configuration of FusionCube 6000 for virtualization applications Category FusionCube 6000VH FusionCube 6000VS Application scenario Virtualization and cloud Virtualization and cloud Basic package 72 cores 48 cores 768 GB memory 384 GB memory 36 TB storage Expansion package Basic package configuration 60 TB storage 2400 GB SSD cache 1800 GB SSD cache 24 cores 16 cores 256 GB memory 128 GB memory 20 TB storage 12 TB storage 800 GB SSD cache 600 GB SSD cache 3 nodes, each of which has: 3 nodes, each of which has: – 2 x E5-2658A v3 processor with 12 cores – 2 x E5-2630 v3 processor with 8 cores – 8 x 32 GB DIMM – 8 x 16 GB DIMM – 10 x 2 TB SATA HDD – 6 x 2 TB SATA HDD – 800 GB SSDs – 600 GB SSDs – 2 x 10GE port Issue 1.0 (2016-03-31) Chassis fittings Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. – 2 x 10GE port Chassis fittings 34 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 4 Typical Configurations Category FusionCube 6000VH FusionCube 6000VS Expansion package configuration 1 node (2 x E5-2658A v3 processor, 8 x 32 GB DIMM, 10 x 2 TB SATA HDD, 800 GB SSDs, 2 x 10GE port) 1 node (2 x E5-2630 v3 processor, 8 x 16 GB DIMM, 6 x 2 TB SATA HDD, 600 GB SSDs, 2 x 10GE port) Network 10GE Maximum number of nodes 256 nodes/64 chassis (4 nodes per chassis) Storage FusionStorage 3.3 Virtualization platform FusionSphere 5.1 or later VMware vSphere5.5 6.0 Management software FusionCube Center 2.5 Cabinet height 4 U/19 inches (447 mm x 175 mm x 898 mm) Power consumption 60% load: 60% load: 1000 W (3 nodes) 740 W (3 nodes) 1300 W (4 nodes) 960 W (4 nodes) Basic package:100 VMs Basic package:40 VMs Expansion package: 40 VMs VM specifications: 1 x vCPU, 4 GB memory, 150 GB storage Expansion package: 20 VMs VM specifications: 1 x vCPU, 4 GB memory, 150 GB storage Remarks (1 core = 2 vCPUs) (1 core = 2 vCPUs) Table 4-2 Typical configuration of FusionCube 6000 for desktop cloud applications Category FusionCube 6000ES Application scenario Virtual desktop infrastructure (VDI) VDI user type Heavy-loaded OA users (2 vCPUs, 4 GB memory, 40 GB + 80 GB) Number of VDI users 100 to 20000 VDI users (basic package: 100 to 200 VDI users; expansion package: 80 VDI users) Maximum number of expansion nodes: 256 nodes/64 chassis (four nodes per chassis) 3 nodes, each of which has: Basic package 2 x E5-2658A v3 processor, 12 x 32 GB DIMM, 8 x 4 TB SATA HDD, 600 GB PCIe SSD, 2 x 10GE port) Issue 1.0 (2016-03-31) Chassis fittings Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 35 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper 4 Typical Configurations Category FusionCube 6000ES Expansion package 1 node (2 x E5-2658A v3 processor, 12 x 32 GB DIMM, 8 x 4 TB SATA HDD, 600 GB PCIe SSD, 2 x 10GE port) Network GE/10GE Storage FusionStorage 3.3 Management software FusionCube Center 2.5 VDI FusionAccess 5.1 and FusionSphere 5.1 or later Cabinet height 4 U/19 inches (447 mm x 175 mm x 898 mm) Power consumption 60% load: Remarks 900 W (3 nodes) 1170 W (4 nodes) One chassis supports four nodes. 4.2 FusionCube 9000 Table 4-3 Typical configuration of FusionCube 9000 Category Specifications Application scenario Virtualization, cloud, hybrid load, VDI Node specifications 2-socket compute node (2 x E5-2600 v3 processor) 4-socket computer node (4 x E7-4800 v2 or E7-4800/8800 v3 processor) 2-socket compute and storage converged node (2 x E5-2600 v3 processor, 2 to 6 x PCIe SSD delivering 2.4 TB/3.2 TB) 2-socket compute and storage node (2 x E5 -2600 v3 processor, 12 x SAS HDD) Network 10GE, IB 56G FDR switching (storage network) Storage FusionStorage distributed storage Maximum specifications 256 server nodes Management software FusionCube Center Virtualization software FusionSphere 5.1, VMWare vSphere5.5, or later Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 36 Huawei FusionCube 2.5 Hyper-converged Virtualization Infrastructure Technical White Paper Cabinet specifications 42 U (customizable) Power consumption 6000 W (maximum per chassis) 4 Typical Configurations Table 4-4 FusionCube typical configuration Node specifications Typical Configuration for Virtualization Typical Configuration for Hybrid Load Four to eight 2-socket compute and storage nodes (2 x E5-2600 v3 processor, 12 x SAS HDD) Four 2-socket compute and storage nodes (2 x E5-2600 v3 processor, 12 x SAS HDD) Two 4-socket computer nodes (4 x E7-4800 v2 or E7-4800/8800 v3 processor) Network 10GE 10GE, IB 56G FDR switching (storage network) Storage FusionStorage distributed storage FusionStorage distributed storage Management software FusionCube Center FusionCube Center Virtualization software FusionSphere 5.1, VMWare vSphere5.5, or later FusionSphere 5.1, VMWare vSphere5.5, or later Issue 1.0 (2016-03-31) Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 37
