Download Netvisor nvOS Configuration Guide

Netvisor nvOS Configuration Guide
Version 2.4
July 2016
pluribusnetworks.com
Table of Contents
Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i
nvOS Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Adding Switches to the Fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Displaying Fabric Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Displaying Information about Nodes in the Fabric . . . . . . . . . . . . . . . . . . . . . 16
Using the Fabric Transaction Commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Troubleshooting the Fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Using the Serial Console Port for Initial Configuration . . . . . . . . . . . . . . . . . . 22
Changing Other Switch Setup Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Adding License Keys to nvOS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Modifying and Upgrading Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Updating nvOS on the Server-Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Saving and Restoring Server-Switch Configurations . . . . . . . . . . . . . . . . . . . . 34
Changing the IP Port for vManage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Rebooting, Powering Off, and Resetting the Server-Switch. . . . . . . . . . . . . . 40
Installing the nvOS Linux API. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
Configuring Port Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .42
Configuring Rapid Spanning Tree Protocol (RSTP) . . . . . . . . . . . . . . . . . . . . . . . . 52
Configuring Link Aggregation Control Protocol (LACP) . . . . . . . . . . . . . . . . . . . . .56
Configuring Trunking for Link Aggregation (LAG) . . . . . . . . . . . . . . . . . . . . . . . . . 58
Configuring Layer 2 Multipathing for Virtual Chassis Link Aggregation . . . . 59
Configuring Active-Active VLAG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
Configuring Tagged and Untagged VLANs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .67
Displaying VLAN Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
Implementing Virtual Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .70
Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
Specifying the Type of VNET Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
Creating a Virtual Network (VNET). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Related Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Creating a Virtual Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Adding DHCP Service to a VNET . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
Verify Administrator User Creation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
Configuring Administration Login Using SSH . . . . . . . . . . . . . . . . . . . . . . . . . . 75
Adding a Default Gateway to the VNET . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
Adding Ports to the VNET . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
Configuring Virtual Resource Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
About Virtual Resource Group (VRG) Bandwidth Enforcement. . . . . . . . . . . 83
Configuring Network Services - DHCP and DNS . . . . . . . . . . . . . . . . . . . . . . . . . . .88
Overview of DHCP and DNS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
Configuring IP Pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
Configuring DHCP Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
Adding DHCP Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
i
Adding DHCP and DNS Records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
Removing DHCP and DNS Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
Configuring DNS Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
Adding a DNS Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
Overview of NAT and Hardware NAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .95
Hardware NAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
NAT and Hardware NAT Use Cases and Scenarios . . . . . . . . . . . . . . . . . . . . . 96
Configuring Network Address Translation Services . . . . . . . . . . . . . . . . . . . . . . . 99
Configuring Port Forwarding for NAT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
Configuring Static NAT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
Configuring Hardware-based Network Address Translation(NAT) . . . . . . . . . . .102
nvOS System Logging and SNMP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .104
Configuring System Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
Sending Log Messages to Syslog Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
Viewing Log Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Sending Log Messages to Syslog Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
Configuring SNMP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .114
SNMP Communities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
Users and SNMPv3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
Supported MIBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
High Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .123
Configuring a Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
Configuring Fabric-based Physical Storage Pools . . . . . . . . . . . . . . . . . . . . . . . . .126
Creating Virtual Storage for a Virtual Network (VNET) . . . . . . . . . . . . . . . . . . . . 129
Managing Host Operating Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
Provisioning Bare Metal Servers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
External Disk Drive Installation Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .135
Configuring High Availability for Storage Folders . . . . . . . . . . . . . . . . . . . . . . . . .137
Configuring a Linux Netvisor KVM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
Creating a Disk-based Netvisor KVM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
Creating a KVM by Importing an ISO Image . . . . . . . . . . . . . . . . . . . . . . . . . 143
Adding Virtual Machine (VM) Instances to the Server-Switch. . . . . . . . . . . 143
Managing Linux VM Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
Configuring and Implementing NetZones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .148
Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
Configuring a NetZone. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
Configuring vRouter Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .151
Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
Configuring Prefix Lists for BGP and OSPF . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
Configuring Packet Relay for DHCP Servers. . . . . . . . . . . . . . . . . . . . . . . . . . 152
Configuring Hardware Routing for a vRouter . . . . . . . . . . . . . . . . . . . . . . . . 152
Configuring BGP on a vRouter. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
Additional BGP Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
Configuring Open Shortest Path First (OSPF). . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
Adding Areas and Prefix Lists to OSPF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
Pluribus Networks Configuration Guide
ii
www.pluribusnetworks.com
Configuring Routing Information Protocol (RIP) . . . . . . . . . . . . . . . . . . . . . . . . . 164
Configuring Static Routes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
Adding IGMP Static Joins to a vRouter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
Configuring Virtual Router Redundancy Protocol . . . . . . . . . . . . . . . . . . . . . . . .167
Configuring Virtual Load Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .171
Configuring Virtual Load Balancing with Ubuntu 11.04 Servers and nvOS . 175
Adding Virtual Router Redundancy Protocol to VLB Interfaces. . . . . . . . . . 180
Configuring Roles and Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
Configuring TACACS+ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .186
About TACACS+ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
Configuring TACACS+. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
Creating and Implementing Access Control Lists (ACLs) . . . . . . . . . . . . . . . . . . .188
Using a Deny IP ACL to Block Network Traffic . . . . . . . . . . . . . . . . . . . . . . . . 188
Using IP ACLs to Allow Network Traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
Using MAC ACLs to Deny Network Traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
Using MAC ACLs to Allow Network Traffic. . . . . . . . . . . . . . . . . . . . . . . . . . . 191
Configuring IP ACLs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .193
Configuring an Internal Deny ACL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
Configuring an External Deny ACL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
Configuring an External Allow IP ACL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
Configuring a MAC ACL to Deny Network Traffic . . . . . . . . . . . . . . . . . . . . . 194
Configuring a MAC ACL to Allow Network Traffic . . . . . . . . . . . . . . . . . . . . . 195
Configuring vFlow for Analytics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .196
Using vFlows to Disable Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .200
Configuring Mirroring for vFlows and Ports . . . . . . . . . . . . . . . . . . . . . . . . . 202
Managing Traffic Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .204
Using Application Flows and Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .206
Displaying Standard Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
Understanding vFlow Statistics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
Example Use Cases for vFlows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .212
Configuring VXLANs and Tunnels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .215
Creating Tunnels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
Edge Virtual Bridging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .218
Understanding Edge Virtual Bridging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
Configuring Edge Virtual Bridging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
Implementing OpenFlow with FloodLight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
Configuring OpenFlow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .221
Enabling a Virtual Network for an OpenFlow Controller . . . . . . . . . . . . . . . 221
Creating OpenFlow Controllers with Multiple VLANs. . . . . . . . . . . . . . . . . . 223
Configuring the OpenFlow Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
Configuring Open Virtual Switch (OVS) for OpenFlow . . . . . . . . . . . . . . . . . 224
About sFlow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .225
Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
Configuring sFlow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
Configuring the sFlow Collector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
iii
Enabling sFlow on the Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
Adding Additional Ports to sFlow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
Internet Protocol Flow Information Export (IPFIX) . . . . . . . . . . . . . . . . . . . . . . . .231
Pluribus Networks Configuration Guide
iv
www.pluribusnetworks.com
THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE
WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED
TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS
MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.
THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH
IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY
THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY,
CONTACT YOUR PLURIBUS NETWORKS REPRESENTATIVE FOR A COPY.
NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE ARE
PROVIDED “AS IS” WITH ALL FAULTS. PLURIBUS NETWORKS DISCLAIMS ALL WARRANTIES, EXPRESS OR
IMPLIED, INCLUDING WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE
PRACTICE.
IN NO EVENT SHALL PLURIBUS NETWORKS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR
INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO
DATA, ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF PLURIBUS NETWORKS HAS
BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
Any Internet Protocol (IP) addresses used in this document are not intended to be actual addresses. Any
examples, command display output, and figures included in the document are shown for illustrative
purposes only. Any use of actual IP addresses in illustrative content is unintentional and coincidental.
© 2016 PLURIBUS NETWORKS, INC. ALL RIGHTS RESERVED.
www.pluribusnetworks.com
v
Preface
This preface includes the following sections:
 Audience
 Organization
 Conventions
 Related Documentation
 Obtaining Documentation and Submitting a Service Request
This preface describes the audience, organization, and conventions of this publication, and provides information
about obtaining related documentation.
Audience
This publication is for experienced network administrators responsible for configuring and maintaining Pluribus
Networks switches with some expertise in the following areas:
 Network administration
 Storage administration
 Server administration
 Application delivery administration
 Network security administration
Organization
This publication is organized as follows:
 Initial Switch Configuration
 Configuring Virtual Wire Functionality
 Configuring Visibility and Analytics Functionality
 Configuring Advanced Switch Functionality
Pluribus Networks Command Reference
1
www.pluribusnetworks.com
Conventions
Table 1: Organization of Content
Chapter
Title
Description
Chapter 1 -
Introduction
Provides information on the Pluribus Networks switch
CLI.
Chapter 2 - .
Initial Setup
Describes how to perform an initial switch setup and
creating a fabric.
Chapter 3 -
Basic Switch Functionality
Configuring switch port settings such as speed, jumbo
frames, tagged and untagged VLANs, STP, and MAC
and IP ACLs.
Chapter 4 -
VNETs and VRGs
Configuring virtual networks (VNETs) and virtual
routing groups (VRGs).
Chapter 5 -
DHCP and DNS Services
Creating and implementing IP address pools as well as
configuring DHCP and DNS services.
Chapter 6 -
Load Balancer, Router, and Configuring virtual load balancing, virtual routing and
NAT
network address translation (NAT) services.
Chapter 7 -
Storage
Configuring and implementing fabric-based virtual
storage.
Chapter 8 -
Edge Virtual Bridging
Overview of Edge Virtual Bridging (EVB) and VEPA
technology.
Chapter 9 -
Application Flows and
Statistics
Creating and displaying port and application flows and
traffic statistics.
Chapter 10 -
OpenFlow
Controlling a virtual network with either the built-in or
an external OpenFlow controller.
Chapter 11 -
Running your Own Code
Developing your own applications to run directly in the
network.
Chapter 12 -
High Availability and Link
Aggregation
Configuring and implementing clustering and network
resiliency.
Appendix A -
Acronyms
Defines the acronyms used in this software
configuration guide.
Appendix B -
Acknowledgments for
Open Source Software
Provides acknowledgments for open source software
used in nvOS®.
This document uses the following conventions:
Table 2: CLI Conventions
Convention
Indication
Bold font
Keywords, user interface elements, and user-entered text appear in bold
font.
Italic font
Document titles, new or emphasized terms, and variables that you
supply values are in italic font.
[]
Elements in square brackets are optional.
{x|y|z}
Required elements are grouped in curly braces and are separated by
vertical bars.
Pluribus Networks Command Reference
www.pluribusnetworks.com
2
Table 2: CLI Conventions
Convention
Indication
[x|y|z]
Optional parameters are grouped in brackets and separated by vertical
bars.
String
A non-quoted set of characters. Do not use quotation marks around the
string or the string includes the quotation marks.
courier font
Command Line Interface (CLI) commands and samples appear in
courier font.
<>
Nonprinting characters such as passwords are indicated by angle
brackets.
[]
Default responses to system prompts are in angle brackets.
CLI
network-admin@switch
>Indicates that you enter the following text at the command prompt.
Informational Note:
Indicates information of special interest.
Indicates a situation that could cause equipment failure or loss of data.
TIP!
TIP!Indicates information that can help you solve a problem.
Timesaver:
Indicates information that can help you save time.
Related Documentation
The Pluribus Networks switch nvOS documentation set includes the following publications:
 Pluribus Networks Hardware Installation Guide
 Pluribus Networks Virtual Wire Configuration Guide
 Pluribus Networks Virtual Wire Command Reference
Pluribus Networks Command Reference
3
www.pluribusnetworks.com
 Release Notes for Pluribus Networks nvOS
For a complete list of all Pluribus Networks documentation, see the Pluribus Networks support site at
www.plurisbusnetworks.com/support.
Additional documentation describing log messages and MIBs are also available for download at
www.plurisbusnetworks.com/support.
Documentation Feedback
To provide technical feedback on this document, or to report an error or omission, please send your comments to
[email protected]. We appreciate your feedback.
Obtaining Documentation and Submitting a Service Request
For information on obtaining documentation, submitting a service request, and gathering additional information,
please visit www.pluribusnetworks.com/support.
Pluribus Networks Command Reference
www.pluribusnetworks.com
4
nvOS Introduction
This chapter provides information for understanding and using the Pluribus Networks nvOS command
line interface (CLI) on a Pluribus Networks switch. Included in this chapter is the following information:
 Entering Commands and Getting Help
 Finding Command Options
 Specifying IP Address Netmasks
 Specifying Capacity, Throughput, and Scale
 Customizing Show Output Formats
 Specifying a Switch or Fabric for Command Scope
Entering Commands and Getting Help
Commands, options, and arguments are entered at the CLI prompt. A command name must be typed,
but included command-completion and help features contribute to the command entry process.
To display a list of command that you can use within a command mode, enter a question mark (?), or use
the tab key, or type help at the command prompt. You can also display keywords and arguments for
each command with this context-sensitive help feature. You can also use complete commands and
display keywords and arguments for each command using the tab key to assist with context-sensitive
command help and completion.
Table 3 lists the command that you can enter to get help specific to a command, keyword, or argument.
Table 3: Getting Help
abbreviated- command-entry?
Displays a list of commands that begin with a specific
character string. Do not leave a space between the string
and question mark.
abbreviated- command-entry <tab>
Completes a partial command name.
?
Lists all commands.
command ?
Lists all keywords for the command. Leave a space
between the command and the question mark.
command keyword ?
Lists all arguments for the keyword. Leave a space
between the command and the question mark.
Where a text string is used, such as name-string, the following characters are allowed as part of the
text string: a-z, A-Z, 0-9, _ (underscore), . (period), , (comma), : (colon), and - (dash).
Informational Note: If you enter a command that is invalid, then using the ? and tab key have no
effect and do not return any changes to the CLI.
www.pluribusnetworks.com
Informational Note: The CLI has an editing ability similar to UNIX and Linux functionality using
emacs keys. For example, ˄p steps backward through previous commands, ˄n moves to the next
command in the history, ˄a moves to the first character in the command and ˄e moves to the end of
the line, ˄u erases the current line, and ˄w erases the previous word.
Informational Note: Also you can use the up and down arrows on your keyboard to retrieve the last
command entered at the CLI.
Finding Command Options
The syntax can consist of optional or required keywords. To display keywords for a command, enter a
question mark (?) at the command prompt or after entering part of a command followed by a space.
nvOS® CLI displays a list of available keywords along with a brief description of the keywords. For
example, if you want to see all of the keywords for the command user, enter user ?.
Table , “Getting Help” displays examples of using the question mark (?) to assist you with entering
commands.
Table 4: Finding Command Options
CLI network-admin@switch > ?
All commands:
acl-ip-create
acl-ip-delete
...
Displays a list of commands that begin with a specific
character string. Do not leave a space between the string
and question mark.
Switch> user auth
User: <user>
Password: <password>
Completes a partial command name.
?
Lists all commands.
command ?
Lists all keywords for the command. Leave a space
between the command and the question mark.
command option ?
Lists all arguments for the option. Leave a space between
the command and the question mark.
Informational Note: Other useful options, especially for displaying statistics, include sort,
interval, duration, and show diff interval.
Additional Information on the Command Line Interface
For some commands, the parameter delete is used, and in other commands, the parameter remove
is used. This may appear as inconsistent usage, but the explanation is quite simple.
delete is used for top level commands, such as port-association-delete, or user-delete.
The following list is a sample of top level commands:
www.pluribusnetworks.com
 aaa-tacacs-delete
 user-delete
 user-delete
remove is used for commands with additional options, such as admin-syslog-match-remove
where the top level command is admin-syslog and the additional option match is added to the top
level command.
The same logic also applies to the usage of create and add. create is used for top level commands
and add is used with top level commands with additional options. For example, sflow-create and
sflow-port-add are two instances where this usage occurs in the CLI.
Alternate Command Format
The CLI has an alternate command format in that the commands start with a verb instead of a noun. This
format omits the hyphen in the command names. For example, connection-stats-show can also be
entered as show connection-stats. The command formats have the same features and can be
used interchangeably.
Understanding Role-based Access Control
Pluribus Networks nvOS® supports flexibly defined roles so that data centers can use the same best
practices for managing discrete servers, storage, and networks to operate a Pluribus Networks fabric.
You can create user roles with privileges that reflect user responsibilities in the data center. For example,
you can create the following types of roles:
 Fabric administrator roles with control over all fabric-wide tasks
 Cluster administrator roles with control over all cluster-wide tasks
 Switch-server administrator roles with control over single switch configuration tasks
 Virtual Network (VNET) administrator roles with control over one or multiple VNET configuration
tasks
 Virtual network services administrator with control over one or multiple network service(s)
configuration tasks.
Specifying IP Address Netmasks
Some commands call for the specification of an IP address netmask. Pluribus Networks nvOS supports
both CIDR and subnet notations.
For example, the range of IP addresses from 192.168.0.0 to 192.168.0.255 can be specified by either
entering 192.160.0.0 for the IP address input for a CLI command or either 24 or 255.255.255.0 for the
netmask.
Specifying Capacity, Throughput, and Scale
Many commands include input and output of capacity and throughput. Network values are always in bits
and storage values in bytes. Scale factors are allowed on input and displayed in output as well as shown
in Table 5, “Scale Numbers”.
Table 5: Scale Numbers
Scale Indicator
Meaning (Networking)
Meaning (Storage)
K or k
Kilobits
Kilobytes
M or m
Megabits
Megabytes
www.pluribusnetworks.com
Table 5: Scale Numbers
Scale Indicator
Meaning (Networking)
Meaning (Storage)
G or g
Gigabits
Gigabytes
T or t
Terabits
Terabytes
Customizing Show Output Formats
The output generated by the show commands can be customized by using the optional arguments
described in Table 6, “Show Output Formats”.
Table 6: Show Output Formats
format
<column_name1>,
<column_name2>,
<column_nameX>
Displays only the columns matching the list of column header names.
NOTE: The list of column names is comma-separated without spaces.
format all
Displays all available column headers. This output is also called verbose
mode.
By default, show commands output a terse set of the most commonly
useful column headers.
parsable-delim <separator> Displays the output of show command by separating columns by the
specified <separator> character(s).
For example, parsable-delim , produces a comma-separated output
(CSV).
NOTE: If the parsable-delim option is specified, the column header
names (titles) are suppressed from the output.
Specifying a Switch or Fabric for Command Scope
While a switch is the building block of a fabric, the goal of the Pluribus Networks design is that a fabric of
switches is easy to manage as a single switch. Because of this, the CLI can be used to run commands on
the local switch, a cluster of switches, other switches in the fabric, or the entire fabric. You don’t have to
log into each switch that you want to run commands.
By default, commands are run on the switch where you execute the command. For example, the
command port-config-modify port 5 disable disables port 5 on the switch where you
have executed the command.
To specify a different switch for a single command, use the switch prefix. For example, switch
pleiades23 port-config-modify port 28 enable enables port 28 on pleiades23, even if
the CLI is connected to a different switch in the fabric.
To specify a different switch for a series of commands, use the switch prefix with no command. For
example, type switch pleiades24 <return>. The CLI prompt changes to indicate that
pleiades24 is the switch you are executing commands. Additional commands are run on
pleiades24 rather than the switch that you’re physically connected.
network-admin@pleiades23>switch pleiades24
network-admin@pleiades24>
For most CLI show commands, the command displays results from all switches in the fabric by default.
For example, when the CLI command port-show is entered on the switch, it shows the ports of all
switches in the fabric.
www.pluribusnetworks.com
To specify that a CLI show command should apply to a specific switch, use the switch prefix to the CLI
command. For example, for the port-show command to only show the ports of the switch named
pleiades24, type the command switch pleiades24 port-show.
To execute a command on a local switch, use the switch-local command before entering the rest of
the commands:
CLI network-admin@switch > switch-local port-show port 25
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
www.pluribusnetworks.com
Introduction to nvOS Fabric
 Creating an Initial Fabric
 Adding Switches to the Fabric
 Fabric Over Management Interface
 Configuring a Fabric on the Control Plane Network
 Displaying Fabric Statistics
 Displaying Information about Nodes in the Fabric
 Using the Fabric Transaction Commands
 More Information About Undo Commands and Transactions
Overview
At Pluribus Networks, a fabric is defined as a distributed architecture based on a collection of compute clustering
techniques to present an open, standard-based Ethernet fabric as one logical switch. Every node shares the same
view of the fabric including MAC and IP addresses, connections, and application flows.
When you add switches to the fabric, all switches are under a single management domain which is highly available
through multiple link aggregation and load balancing between network resources.
The fabric performs a classic database 3-phase commit for configuration changes. All members of the fabric must
accept the configuration changes before the change is made in the fabric. Figure 1 Fabric Architecturedisplays the
fabric architecture of nvOS.
Figure 1: Fabric Architecture
Creating an Initial Fabric
After you complete the initial setup on the switch, you must create a new fabric for the switch or join an existing
fabric. When switches form a fabric, the fabric becomes one logical switch, and shares state information as well as
communicates commands so that any scope of a fabric- command is executed on each switch in the fabric. A
switch must be in a fabric in order to keep track of the fabric state. However, a switch can be a member of fabric, and
consist of a single switch. A switch leaving one fabric and joining another loses the fabric state of the first fabric and
learns the fabric state of the second fabric.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
10
1. To create a new fabric over Layer 2, use the following command:
CLI network-admin@switch > fabric-create name name-string
2. Create a name for the new fabric.
To require a password before joining the fabric, use the password option. Press the return key after typing the
password parameter:
CLI network-admin@switch > fabric-create name name-string <return>
password:*******
Re-enter password:*******
By default, the fabric is created on VLAN1. You can specify a different VLAN, but if you change the VLAN, you must
recreate the fabric.
To create a fabric over Layer 3, use the fabric-join command and the switch IP address. For example,
CLI network-admin@switch > fabric-join switch-ip 192.168.2.2 vlan 20
3. To show fabric details, use the fabric-show command:
CLI network-admin@switch > fabric-show
name
---------------info-dev
ursa-lyon
id
---------------a000030:5537b46c
6000210:566621ee
vlan
---3
0
fabric-network
-------------in-band
mgmt
control-network
--------------in-band
in-band
tid
---365
4928
You can also specify to send network traffic over the fabric network or the control plane network. To specify the
fabric network, use the fabric-network parameter, specify the in-band or management IP address.
Specifying the fabric-network parameter sends traffic over the data path for fabric administration, which includes
configuration changes and show commands.
To specify the control plane network, use the control-network parameter, and specify the in-band or
management IP address.
Using the control-network parameter specifies the data path for control plane traffic, which includes status updates,
vlag syncs, cluster syncs, and similar traffic.
Adding Switches to the Fabric
For this example, the switches are connected as in Figure 4:
Pluribus Networks Configuration Guide
11
www.pluribusnetworks.com
Figure 4: Directly Connected Switches in a Fabric
When you have more than one switch, you must add it to the fabric to take advantage of the features offered by the
fabric. To add the new switch, use the following command on one of the switches:
CLI network-admin@switch > fabric-join name pn-EBC4 fab1
You can join the fabric using either the fabric name or the switch IP address. If you use the Tab key to display the
available options, all fabrics on the network are displayed as options.
If you specify a password for the fabric, you must type it in twice. The password is used to encrypt communication
between the nodes in the fabric. When you join the fabric from a node, you must type in the password to join it.
You can specify a specific VLAN for the fabric when you create a new one, or by default, the fabric uses VLAN1.
However, you cannot change the fabric VLAN without recreating the fabric.
Informational Note: Avoid creating fabrics with the same name.
When the fabric is created, the switch begins sending multicast messages out on Layer 2 looking for other switches.
These messages are not propagated to other networks. This is how Switch B in Figure 4 learns about the fabric.
Once Switch B joins the fabric, the fabric configuration (commands with scope fabric) is downloaded on Switch B and
the switch reboots.
If you want to connect to a switch over Layer 3, you must specify the IP address for the switch in the fabric using the
following command:
CLI network-admin@switch > fabric-join switch-ip 192.168.11.1
Fabric Over Management Interface
You can now configure fabric communication run over either the management interface or the in-band interface.
Because fabric communication over the in-band interface can be disrupted due to STP, ports going up/down, and
other factors, fabric communication over management provides a more consistent configuration.
If you create a fabric with the management interface, any nodes joining the fabric inherit this setting. All nodes in a
single fabric all run on the same network type. You cannot run a mixed configuration of management and in-band
interfaces. Fabrics advertised on an incompatible network are not available for when you issue the fabric-join
command. This keeps a switch from joining an incompatible fabric.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
12
If the fabric is configured on the management interface, all fabric-communication is on the management network,
except for the following:
Cluster synchronization-related traffic such as VLAG synchronizations and forwarded STP packets.
Cluster keep-alive packets on the fabric
Fabric keep-alive packets and global-discovery packets because both run on mgmt and in-band interfaces.
Two options, network-type and control-network are added to the command, fabric-create:
CLI network-admin@switch > fabric-create
name name-string
any of the following options:
vlan 0..4095
password
fabric-network in-band|mgmt
control-network in-band|mgmt
delete-conflicts|abort-on-conflict
If not specified, the network defaults to in-band. Note the commands, fabric-join and fabric-unjoin,
remain unchanged.
Specifying the fabric-network parameter sets the data path for fabric administration, which includes
configuration changes and show commands.
Specifying the control-network parameter sets the data path for control plane traffic, which includes status
updates, VLAG syncs, cluster syncs, and other control plane traffic.
Two new states are added to the state field of fabric-node-show:
fabric-node-show ?
[state offline|online|in-band-only-online|mgmt-only-online|
fabric-joined|eula-required|setup-required|fabric-required| fresh-install]
Because there are now two networks for nvOS to monitor for connectivity, online means both management and
in-band are reachable; in-band-only-online means the switch is only reachable through the in-band
network; mgmt-only-online means it is only reachable through the management network; and offline
means the switch is not reachable on either network.
Monitoring and reporting are reported on both the management and in-band network connectivity.
Configuring a Fabric on the Control Plane Network
When you create a fabric, you can now specify the control plane network. Previous versions restricted control plane traffic to
in-band, but now it may be set to run over the management network. The network parameter has been renamed
to fabric-network and control-network.
fabric-network specifies the network to use for user-driven configuration traffic, including show
commands and configuration changes.
Pluribus Networks Configuration Guide
13
www.pluribusnetworks.com
control-network specifies the network to use for nvOS internal traffic, for example cluster and vport
coordination.
CLI (network-admin@f64-leaf01) > help fabric-create
fabric-create
create a fabric
name name-string
name of the fabric
any of the following options:
repeer-to-cluster-node cluster-repeer-node name Replace a dead cluster node by
restoring against the existing cluster node
vlan vlan-id
VLAN assigned to fabric
password
plain text password
fabric-network in-band|mgmt
fabric administration network
control-network in-band|mgmt
control plane network
delete-conflicts|abort-on-conflict
delete conflicts
Displaying Fabric Information
You can display information about the fabric using the fabric-info command:
CLI network-admin@switch > fabric-info format all layout vertical
name:
id:
vlan:
fabric-network:
control-network:
tid:
info-dev
a000030:5537b46c
3
in-band
in-band
365
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
14
Displaying Fabric Statistics
You can also display statistical information about fabric and node activity. Use the formatting options format all
and layout vertical to display the show output in an easy to read format. This output is from a switch,
corp-sw1, in a fabric with two other switches.
CLI network-admin@switch > fabric-stats-show format all layout vertical
switch:
id:
servers:
storage:
VM:
vxlan:
tcp-syn:
tcp-est:
tcp-completed:
tcp-bytes:
udp-bytes:
arp:
vlan:
switch:
id:
servers:
storage:
VM:
vxlan:
tcp-syn:
tcp-est:
tcp-completed:
tcp-bytes:
udp-bytes:
arp:
vlan:
switch:
id:
servers:
storage:
VM:
vxlan:
tcp-syn:
tcp-est:
tcp-completed:
tcp-bytes:
udp-bytes:
arp:
vlan:
corp-sw1
0
0
0
0
0
3
1
17
3.56M
0
0
0
corp-Leaf-1
0
0
0
0
0
42.5K
7.20K
1.99M
4.63T
0
0
0
corp-Spine1
0
0
0
0
0
115K
50.2K
106M
222T
0
0
0
Pluribus Networks Configuration Guide
15
www.pluribusnetworks.com
Displaying Information about Nodes in the Fabric
You can also display information about the nodes in the fabric. It is important to take note of the fab-tid value. If
the fab-tid values do not match for each node, you can use the commands transaction-rollback-to or
transaction-rollforward-to to resynchronize the fabric.
id:
name:
fab-name:
fab-id:
cluster-id:
fab-mcast-ip:
local-mac:
mgmt-nic:
mgmt-ip:
...
in-band-ip:
...
fab-tid:
out-port:
version:
state:
firmware_upgrade:
device_state:
ports:
id:
name:
fab-name:
fab-id:
cluster-id:
fab-mcast-ip:
local-mac:
mgmt-nic:
mgmt-ip:
...
in-band-ip:
...
fab-tid:
out-port:
version:
state:
firmware_upgrade:
device_state:
ports:
id:
name:
fab-name:
fab-id:
cluster-id:
fab-mcast-ip:
local-mac:
mgmt-nic:
mgmt-ip:
167772619
Leaf2
fab1
a0001c8:53e2601b
0:0
239.4.10.94
64:0e:94:28:06:f2
192.168.1.14/24
192.168.254.14/24
9
0
2.1.201015836,pn-nvOS-2.0.2-2000212196
online
not-required
ok
72
201326827
Leaf1
fab1
a0001c8:53e2601b
0:0
239.4.10.94
64:0e:94:30:03:97
192.168.1.11/24
192.168.254.11/24
9
129
2.1.201015836,pn-nvOS-2.0.2-2000212196
online
not-required
ok
72
167772618
Spine2
fab1
a0001c8:53e2601b
0:0
239.4.10.94
64:0e:94:28:06:ee
192.168.1.13/24
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
16
An example of a fabric that is out of sync for two nodes in the fabric:
CLI network-admin@switch > fabric-node-show format all layout vertical
id:
100663365
name:
CBF-switch
fab-name:
pn-CBF4
fab-id:
a0000c5:53ab701e
cluster-id:
0:0
fab-mcast-ip:
239.4.10.111
local-mac:
64:0e:94:18:01:03
mgmt-nic:
mgmt-ip:
192.168.1.61/24
...
in-band-ip:
192.168.77.61/24
...
fab-tid:
328
out-port:
128
version:
2.1.201005800,pn-nvOS-2.0.2-2000212196
state:
online
firmware_upgrade:
not-required
device_state:
ok
ports:
68
id:
201326771
name:
CBF-Leaf-1
fab-name:
corp-CBF4
fab-id:
a0000c5:53ab701e
cluster-id:
0:0
fab-mcast-ip:
239.4.10.111
local-mac:
64:0e:94:30:02:4d
mgmt-nic:
mgmt-ip:
192.168.1.53/24
...
in-band-ip:
192.168.77.53/24
...
fab-tid:
329
out-port:
128
version:
2.1.201005800,pn-nvOS-2.0.2-2000212196
state:
online
firmware_upgrade:
not-required
device_state:
ok
ports:
72
id:
167772357
name:
CBF-Spine1
fab-name:
pn-CBF4
fab-id:
a0000c5:53ab701e
cluster-id:
0:0
fab-mcast-ip:
239.4.10.111
local-mac:
64:0e:94:28:02:de
mgmt-nic:
mgmt-ip:
192.168.1.51/24
...
in-band-ip:
192.168.77.51/24
If you apply a configuration to the fabric, and a node does not respond to it, you can evict the node from the fabric,
and then troubleshoot the problem. To evict a node, use the following command:
CLI network-admin@switch > fabric-node-evict name CBF-Spine2
Pluribus Networks Configuration Guide
17
www.pluribusnetworks.com
or
CLI network-admin@switch > fabric-node-evict id b000021:52a1b620
Using the Fabric Transaction Commands
You can roll back the fabric to a specific fabric transaction number. If a failure occurs on the fabric, transactions on
nodes in the fabric can become out of sync. Once transactions are out of sync, no further transactions can be
executed across the scope of local, fabric, or cluster. Unjoining and rejoining the fabric causes the node to lose its
configuration.
As part of a single node transaction recovery, you can roll back the transaction number to a previous one. If multiple
nodes are out of sync, you must recover each node separately.
You can also roll the fabric transaction ID forward on a node if it is out of synch with the rest of the fabric.
In the previous example, the switch, CBF-Switch2, is out of synch with the rest of the fabric. The fabric transaction ID
is 327 and the rest of the nodes have a transaction ID of 328. In this case, you can roll the node, CBF-Switch2,
forward to transaction ID 328. Enter the following command on node CBF-Switch2:
CLI network-admin@switch > transaction-forward-to scope fabric tid 328
This command produces output when an error occurs during the transaction. If there is no output, the transaction is
successful.
To display transaction information for CBF-Switch2,use the transaction-show command:
CLI network-admin@switch > transaction-show format all layout vertical
start-time:
03-19,13:46:42
end-time:
03-19,13:46:43
scope:
fabric
tid:
33
state:
remote-commit
command:
--unrecoverable-- vlan-delete id 22
undo-command: --unrecoverable-- vlan-create id 22 nvid a000030:16 scope
fabric name vlan-22 active yes stats vrg 0:0 ports 1-72,128-129,255
untagged-ports none send-ports 31,41,47-48,51,65-66 active-edge-ports
none ports-specified false flags
---------------------------------------start-time:
09:36:09
end-time:
09:36:09
scope:
fabric
tid:
34
state:
remote-commit
command:
vlan-create id 35 scope fabric stats ports-specified true
The scope parameter indicates which set of transactions to display as each scope has an independent set of
transactions associated with it. The default scope is fabric unless another scope is specified.
You cannot copy and paste commands and undo-commands because they include information that cannot apply to
new commands. These fields are informational-only and allow you to see exactly what happens to the configuration
when you roll forward or roll back the transaction ID.
Once you decide which node you want to modify and the transaction that you want to roll forward or roll back, you
use the transaction-rollforward-to or transaction-rollback-to commands to re-run the
command (roll forward) or undo the command (rollback) on the node. This applies only to the local node.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
18
More Information About Undo Commands and Transactions
You may see output similar to this output:
start-time:
end-time:
scope:
tid:
state:
command:
undo-command:
21:54:53
21:54:53
local
3
commit
port-config-modify port 9 enable
port-config-modify port 9 enable
This output is actually correct. The undo information is taken from the current state on the fabric. So if the port is
currently enabled, and you try to enable it again, you see the undo-command in the output, since the previous
state is also enabled. If you actually disable the port first, and then enable it, you see the expected undo information
in the transaction log.
start-time:
10:05:22
end-time:
10:05:22
scope:
local
tid:
20
state:
commit
command:
port-config-modify port 12
undo-command: port-config-modify port 12
---------------------------------------start-time:
10:05:48
end-time:
10:05:48
scope:
local
tid:
21
state:
commit
command:
port-config-modify port 12
undo-command: port-config-modify port 12
disable
enable
enable
disable
So undo is not necessarily the opposite of the current command, but allows you to go back to the state before the
command was issued. This may be the exact same state as before.
Configuring Transaction Settings
Transactions are allowed to proceed if at least one node in the cluster is reachable. If a cluster node is offline when a
configuration change is requested the transaction proceeds even though one of the cluster members is offline.
Nodes that were ignored for transactions automatically try to recover the transactions. Auto-recovery is enabled by
default but may be disabled. You can also configure the length of time between retry attempts between the nodes.
This feature is enabled by default, but may be disabled.
The following is a sample CLI output with one cluster node offline:
CLI (network-admin@switch1) > vlan-create id 24 scope fabric
Warning: cluster node switch2 not reachable, continuing anyway
The following is a sample of CLI output with both cluster nodes offline:
CLI (network-admin@switch2) > vlan-create id 33 scope fabric
Warning: cluster node switch1 not reachable, continuing anyway
vlan-create: fabric error: switch1 unreachable, both cluster nodes offline
To configure transaction settings, use the transaction-settings-modify command and configure the
following options:
Pluribus Networks Configuration Guide
19
www.pluribusnetworks.com
 allow-offline-cluster-nodes — select this option to allow transactions to proceed on cluster
configurations even if the cluster is offline.
 auto-recover
— select this option to automatically recover missed transactions.
 auto-recover-retry-time — specify the duration of the retry time in days, hours, minutes, or seconds.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Troubleshooting the Fabric
There may be instances when you need to troubleshoot the fabric. The following is a list of helpful port numbers,
multicast information, and communication on the fabric.
 Internal Keepalive
Multicast IP: 239.4.9.7
UDP Destination Port: 23399
This packet is sent from the CPU to the internal port to ensure that the CPU path to the switch is working and
the internal port is up.
 Fabric Keepalive
UDP Destination Port: 23394
Point to point UDP fabric keepalive
If these messages don't get through, the fabric node may go to offline state.
 Global Discovery
Multicast IP: 239.4.9.3
UDP destination port: 23399
Each node periodically multicasts a message about the fabric. This enables fabric-show on L2-connected
nodes to show available packets and also enables fabric-join name name. It also enables you to join a
fabric over Layer 3 connectivity by specify an IP address.
 Proxy commands
TCP Destination Port: 23397 SSL
Used for nvOSd-to-nvOSd commands. Used for internal purposes and also to implement commands executed
on other switches from a local switch.
 Status propagation
TCP Destination Port: 23398 SSL
Port changes and vport changes propagated to other nodes in the fabric.
 TCP API clients
TCP Destination Port: 23396 SSL
C API clients connect to this port. Can be disabled using admin-service-modify if <mgmt/data>
no-net-api command.
 File System replication
TCP Destination Port: 23392 SSL
For ZFS send and ZFS receive messages when replicating file systems across the fabric.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
20
 L2 ARP/DMAC miss/Broadcast encapsulation
UDP Destination Port: 23389
These are VXLAN-encapsulated packets sent from CPU to CPU between two L2 connected switches.
 L3 ARP/DMAC miss/Broadcast encapsulation
UDP Destination Port: 23388
These are VXLAN-encapsulated packets sent from CPU to CPU between two L3 connected switches.
 vPORT status
Multicast IP: 239.4.9.4
UDP Destination Port: 23390
vPort updates from hypervisors or hosts in the fabric.
 vFlow CPU packets
UDP Destination Port: 23398
These packets are sent point-to-point for vflow-snoop of a fabric-scoped vFlow.
All of these messages need to be able to get through in order to keep an L2 fabric healthy. The multicast messages
don't propagate through routers so they aren't used for L3 fabrics.
fabric-node-show displays information about nvOS internal data structures for each node in the fabric. If no
keepalive or other messages are received from a fabric node for about 20 seconds, the node is marked as offline.
Anything that prevents keepalive or other kinds of messages from flowing freely between fabric nodes can cause
problems for fabric connectivity.
If the fabric transaction IDs become unsynchronized, use the transaction commands to either roll forward or back
the transaction IDs. See Using the Fabric Transaction Commands.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
21
www.pluribusnetworks.com
Configuring Basic Server-Switch Functionality
 Using the Serial Console Port for Initial Configuration
 Zero-Touch Provisioning Support (Phase 1)
 Transport Layer Security Protocol 1.2 Support
 Running Commands on a Local Switch
 Aggregation for Management Network Interface Card (NIC)
 Adding License Keys to nvOS
 Changing Other Switch Setup Parameters
 Confirming Connectivity on the Network
 Updating nvOS on the Server-Switch
 Implementing a Fabric Upgrade or a “Rolling” Fabric Upgrade
 Displaying and Managing Boot Environment Information
 Saving and Restoring Server-Switch Configurations
 Copying and Importing Configuration Files
 Rebooting, Powering Off, and Resetting the Server-Switch
 Layer 2 Enhancements
Overview
This section contains information about initial configuration of your switch as well as commands to manage,
upgrade, and restoring switch configurations.
Using the Serial Console Port for Initial Configuration
This procedure assumes that you have installed the server-switch in the desired location and it is powered on.
CAUTION! Do not connect any ports to the network until the server-switch is configured. You can accidentally
create loops or cause IP address conflicts on the network.
If you are going to cable host computers to the switch, there is an option to enable or disable host ports by default.
1. Connect the console port on the rear or front (depending on the model) of the server-switch to your laptop or terminal concentrator using a serial cable.
2. From the terminal emulator application on your computer, log into the switch with the username network-admin
and the default password admin.
3. . You can begin initial configuration using the setup questions displayed:
switch console login: network-admin
Password: admin
Last login: Fri Oct 3 12:23:04 on console
Pluribus Command Line Interface v1.2.2
System setup required:
System Name (switch): pleaides01 <return>
network-admin Password: password <return>
Re-enter Password:****** <return>
Enable mgmt link aggregation (no): yes
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
22
This might reset SSH connections after the
setup.Are you Sure? (no): yes
LACP mode of the mgmt LAG interface[active|passive|off]
(passive): invalid
Please answer "active", "passive", or "off"
LACP mode of the mgmt LAG interface[active|passive|off]
(passive): active
Mgmt IP/Netmask (10.9.19.107
Mgmt IP/Netmask: ip-address/netmask <return>
In-band IP/Netmask: ip-address/netmask
Gateway IP (0.0.0.0): 192.168.100.254 <return> or ip-address
Primary DNS IP (0.0.0.0): 192.168.100.253 <return> or ip-address
Secondary DNS IP (0.0.0.0): 192.168.200.253 <return> or ip-address
Domain name (pluribusnetworks.com): domain-name <return>
Automatically Upload Diagnostics (yes): <return>
Enable host ports by default (yes): no
nvOS system info:
serial number: 1245LC8500018
hostid: a000044
user auth cookie val = 152895552
Switch Setup:
Switch Name:
pleaides01
Switch Mgmt IP:
192.168.100.1/24
Switch In-band IP:
192.168.200.1/24
Switch Gateway:
192.168.100.254
Switch DNS Server:
192.168.100.254
Switch DNS2 Server:
192.168.100.253
Switch Domain Name:
pluribusnetworks.com
Switch NTP Server:
0.us.pool.ntp.org
Switch Timezone:
US/Pacific
Switch Date:
2013-10-03, 13:02:39
Upload Crash Reports:
yes
Fabric required. Please use fabric-create/join/show
Connected to Switch pluribus; nvOS Identifier:0x000044; Ver: 0.19.3398
Pluribus Networks Configuration Guide
23
www.pluribusnetworks.com
When you setup a switch for initial configuration, you can disable host-facing ports until you are ready to plug in
host cables to the switch. If no adjacency is detected on a port during the quickstart procedure, the ports
remain in the disabled state. To enable the ports after plugging in cables, use the port-config-modify port
port-number host-enable command. Host ports are enabled by default unless you specify no during the
quickstart procedure.
Netvisor OS Command Line Interface 2.3
By ANSWERING "YES" TO THIS PROMPT YOU ACKNOWLEDGE THAT YOU HAVE READ THE
TERMS OF THE PLURIBUS NETWORKS END USER LICENSE AGREEMENT (EULA) AND AGREE TO
THEM. [YES | NO | EULA]?: yes
Switch setup required:
Switch Name (e68-leaf-01):
network-admin Password:
Re-enter Password:
Mgmt IP/Netmask (10.13.25.225/16):
In-band IP/Netmask (192.168.97.2/24):
Gateway IP (10.42.42.1):
Primary DNS IP (10.42.44.1):
Secondary DNS IP:
Domain name (pluribusnetworks.com):
Automatically Upload Diagnostics (yes):
Enable host ports by default (yes): no
CLI (network-admin@e68-leaf-01) > port-show
switch
port status
config
------------ ---- ----------------e68-leaf-01 25
phy-up,host-disabled 10g
CLI (network-admin@e68-leaf-01) >port-config-modify port 25 host-enable
CLI (network-admin@e68-leaf-01) > port-show
switch
port status
config
------------ ---- -----------e68-leaf-01 25
up
-----10g
Zero-Touch Provisioning Support (Phase 1)
Zero Touch Provisioning (ZTP) is used to quickly bring up and deploy a configuration on a Pluribus switch with no
user interaction. It is typically used in large-scale data center deployments where the data center engineers simply
racks the equipment and connects it to the management network.
ZTP leverages an on-premise DHCP server where an administrator configures one or more DHCP options to
configure the switch.
The IP or MAC assignment allows a customer to rack a switch in a data center and have the switch get an IP address
via DHCP without needing to plug in a serial console or monitor to see what IP address the switch received which is
not possible in remote data centers.
The network administrator pre-programs the DHCP server with MAC-IP mappings. As soon as the switch is racked in
the data center and powered on, the IP is assigned based on this mapping and can then be remotely managed.
Pluribus Networks provides the MAC address of the MGMT port to the customer – either through the shipping label,
PN Cloud, or other means
Transport Layer Security Protocol 1.2 Support
The TLS protocol provides communications security over the Internet. The protocol allows client and server
applications to communicate in a way that is designed to prevent eavesdropping, tampering, or message forgery.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
24
Running Commands on a Local Switch
You can specify to run commands locally on a switch by using the switch-local parameter. For instance, using
switch-local port-stats-show displays output for the local switch ports only.
Aggregation for Management Network Interface Card (NIC)
Out of band management interfaces areaggregated to provide high availability (HA) and failover capabilities in nvOS
in the presence of two management NICs. You can configure nvOS to pool two management NICs into a single logical
management interface to increase bandwidth of the management link and add redundancy to the out of band
connection. By default, management link aggregation is disabled. When you configure link aggregation, a new
interface is created on the platform and a trunk link is also created. Physical management interfaces, MGMT0 and
MGMT1, are added to it. The IPv4 and IPv6 addresses are copied from MGMT0 if configured.
LACP is disabled by default, but can be enabled using the switch-setup-modify mgmt-lacp-mode command. The
default aggregation mode is active-active, and after configuring the link aggregation interface, nvOS waits for a short
interval to ensure that the interface is receiving packets. If no packets are seen on the second physical interface
configuration reverts back to the single management interface, and the appropriate error message is generated.
You are now ready to begin the rest of the configuration on the switch.
Informational Note:In order to use the “phone home” feature, you must open ports 8084 and 8443
on your firewall.
Changing the Default Timezone
The default timezone is US/Pacific Standard Time (PST). To change the timezone, use the
switch-setup-modify command:
CLI network-admin@switch > switch-setup-modify timezone timezone
Changing Other Switch Setup Parameters
You can also modify other switch parameters including the following:
 Switch name
 Management IPv4 and IPv6 addresses
 Management IPv4 and IPv6 netmasks
 Management IPv4 and IPv6 address assignments
 In-band IPv4 address
 In-band netmask
 Gateway IPv4 address
 Gateway IPv6 address
 Primary and secondary IPv4 addresses for DNS services
 Domain name
 NTP server
 End User License Agreement (EULA) acceptance and timestamp
 Password
 Date
Pluribus Networks Configuration Guide
25
www.pluribusnetworks.com
 Phone home for software updates
 Analytics store (storage type)
 Message of the Day (MOTD)
 Banner
CLI network-admin@switch > switch-setup-modify mgmt-ip6 2001::2/64 gateway-ip
10.10.10.1 gateway-ip6 2001::35 dns-ip 10.10.10.11 dns-secondary-ip 10.10.10.1
domain-name corpinfo.com ntp-server 0.us.pool.ntp.org timezone US/Pacific
<return>
To display the configured settings, use the switch-setup-show command:
CLI network-admin@switch > switch-setup-show
name:
mgmt-ip:
mgmt-ip6:
in-band-ip:
gateway-ip:
gateway-ip6:
dns-ip:
dns-secondary-ip:
domain-name:
ntp-server:
timezone:
date:
phone-home:
analytics-store:
pleiades01
10.10.10.79/16
2001::2/64
192.168.21.1/24
10.10.10.1
2001::35
10.10.9.1
10.10.10.1
corpinfo.com
0.us.pool.ntp.org
US/Pacific
2013-10-31, 16:00:00
yes
optimized
The analytics-store parameter refers to the storage location of nvOS analytics. The parameter, optimized,
indicates that a Fusion IO card is installed on the switch. You can now store statistics for connections, hosts, client
servers, and CPU package logs on the Fusion IO card. When you specify optimized, the statistics are stored on the
IO card with the highest amount of free space. If you select default, the statistics are stored on the nvOShard
drive.
Informational Note: Fusion IO cards are only available as an additional upgrade or when you purchase the
F64-F1LT model.
You can also configure a “Message of the Day” for users to see when logging into the switch. You may enter up to
511 characters including spaces. If you use spaces, enclose the MOTD in quotes. The MOTD can be used as a
temporary or short term message to display downtime or other activity. To add the message, “switch down 2-4pm
3/31/15” use the following syntax:
CLI network-admin@switch > switch-setup-modify motd “switch down 2-4pm 3/31/15”
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
26
When you log into the switch, the MOTD is displayed after the software version:
admin@pubdev03:~$ cli
Netvisor OS Command Line Interface 2.2
Please enter username and password:
Username (network-admin):
Password:
Connected to Switch pubdev03; nvOS Identifier:0xa0000e3; Ver: 2.2.202036795
pubdev03 down 2-4pm 3/31/15
You can also configure static banners to display switch information such as server identity.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Confirming Connectivity on the Network
After you’ve connected your server-switch, you may want to take the time to ensure that you have connectivity by
pinging an external IP address, and pinging a domain to ensure that you can resolve a domain name.
To ping the external network from the server-switch, use the ping command:
CLI network-admin@switch > ping 98.138.253.109
98.138.253.109 is alive.
To ping a domain, use the ping command again:
CLI network-admin@switch > ping yahoo.com
yahoo.com is alive.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Adding License Keys to nvOS
The license key for nvOS is bound to the serial number of the Pluribus Network switch and ships with the switch.
To install the license key, use the following syntax:
CLI network-admin@switch > software-license-install key license-key
The license key has the format of four words separated by commas. For example.
License Key:
rental,deer,sonic,solace
Pluribus Networks Configuration Guide
27
www.pluribusnetworks.com
Once the license key is installed, you can display information about the key using the following command:
CLI network-admin@switch > software-license-show format all layout vertical
switch:
Pleaides01
license-id:
F-ASDF-NVOS2.0
description: Freedom F-Line Advanced Software Defined Fabric License for
Netvisor 2.x
key:
rental,deer,sonic,solace
feature:
all
upgrade-from:
To display the status of the server-switch, use the switch-status-show command:
CLI (switch)>switch-status-show
switch
name
value units
state
-------- --------------- ----- --------- ----pluribus Switch Temp
41
degrees-C ok
pluribus CPU1 Temp
57
degrees-C ok
pluribus CPU2 Temp
49
degrees-C ok
pluribus System Temp
46
degrees-C ok
pluribus Peripheral Temp 30
degrees-C ok
pluribus PCH Temp
43
degrees-C ok
pluribus VTT
volts
ok
pluribus CPU1 Vcore
volts
ok
pluribus CPU2 Vcore
volts
ok
pluribus VDIMM AB
volts
ok
pluribus VDIMM CD
volts
ok
pluribus VDIMM EF
volts
ok
pluribus VDIMM GH
volts
ok
pluribus +1.1 V
volts
ok
pluribus +1.5 V
volts
ok
pluribus 3.3V
volts
ok
pluribus +3.3VSB
volts
ok
pluribus 5V
volts
ok
pluribus +5VSB
volts
ok
pluribus 12V
volts
ok
pluribus VBAT
volts
ok
pluribus switch-3.3v
volts
ok
pluribus switch-1.1v
volts
ok
pluribus switch-vcore
volts
ok
pluribus switch-5.0v
volts
ok
pluribus switch-2.5v
volts
ok
pluribus switch-0.95v
volts
ok
pluribus switch-1.8v
volts
ok
pluribus switch-1.2v
volts
ok
pluribus fan-1
3525 rpm
ok
pluribus fan-2
3760 rpm
ok
pluribus fan-3
3525 rpm
ok
pluribus fan-4
3760 rpm
ok
This command displays the physical status of the switch including fan speed, electrical voltage, temperature.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
28
To display additional physical information about the switch, use the switch-info-show command:
CLI network-admin@switch > switch-info-show
switch:
model:
chassis-serial:
cpu1-type:
cpu2-type:
system-mem:
switch-device:
switch-version:
polaris-device:
gandalf-version:
fan1-status:
fan2-status:
fan3-status:
fan4-status:
ps1-status:
ps2-status:
pluribus
F64-HWENT
1243PN8500014
Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
64.0G
ok
b2
ok
caff0044
ok
ok
ok
ok
ok
n/a
To display information about a specific switch, specify the name of the switch in the command:
CLI network-admin@switch > switch-info-show name name-string
If you don’t specify the name of the switch, all switches in the fabric are displayed.
To specify that a command is executed on the local switch only, use the following syntax to display port 5 on the
local switch only:
CLI network-admin@switch > switch-local port-show port 5
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Modifying and Upgrading Software
A switch can contact an upgrade server, either directly or through a proxy, to download and upgrade to a newer
version of nvOS. You can modify the upgrade process for the switch and add a proxy host.
Informational Note:This upgrade procedure applies to only one switch. To upgrade switches on the
fabric or to create a “rolling upgrade” on the fabric, see
What are Software Tracks?
Software tracks are a method for Pluribus Networks to manage different software releases available to customers.
The software track, release, is the default standard track, but other tracks, such as Beta, may be available for
download.
CLI network-admin@switch > software-modify phone-home
Pluribus Networks Configuration Guide
29
www.pluribusnetworks.com
Updating nvOS on the Server-Switch
Pluribus Networks switches can send “phone home” messages to the Pluribus Networks update servers to
determine if a new release of software is available for download.
1. To view the current version of nvOS on the switch, use the following command:
CLI network-admin@switch > software-show
version:
track:
upgrade-status:
version-available:
auto-upgrade:
use-proxy:
2.2.1-202016524
2.2-release
available
2.2.0-202006524 -> 2.2.1-202016554
disable
no
2. If the upgrade status indicates that a newer version of nvOS is available, request an update from the server:
CLI network-admin@switch > software-upgrade
upgrade successful. rebooting...
To check the status while the switch is upgrading, use the software-upgrade-status-show command.
3. To check the status of the switch after upgrading, reconnect to the switch, and enter the following command:
CLI network-admin@switch > software-show
version:
track:
upgrade-status:
auto-upgrade:
use-proxy:
2.2.1-202016554
2.2-release
up-to-date
disable
no
Informational Note: Allow plenty of time for the switch to download and install the new version
of software. Do not interrupt the operation while the upgrade is in progress. When the
upgrade is complete, the switch reboots and loads the latest version of the software.
If you encounter any problems with the new version of the software, a previous version can be
selected as the boot software. See “Topic Feedback” on page 2–31
Informational Note: Upgrading without an Internet connection - If the switch does not have direct
access to the Internet but can use a proxy server, enter the software-modify
use-proxy command to configure the proxy and then check for software upgrade
availability. If there is no access to the Internet from the switch, contact Pluribus Technical
Support for instructions on upgrading a switch offline.
To upgrade the current nvOS to a later release, use the software-upgrade command.
CLI network-admin@switch > software-upgrade package nvos-2.3.1-203018600.tgz
The parameter package allows you to specify the name of the upgrade file.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
30
To display information about the software upgrade path, you can use the software-track-show command.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Implementing a Fabric Upgrade or a “Rolling” Fabric Upgrade
You can now implement a fabric-wide upgrade and reboot the switches at the same time or in a sequential order. A
fabric upgrade requires downloading the new nvOS software package to each switch, and rolling upgrade downloads
the software packages from the update server and then copies the software to each switch as the upgrade proceeds.
The upgrade controller is the switch where the fabric-upgrade-start command is issued. All upgrade
commands should be executed from the upgrade controller.
The fabric upgrade feature has two phases:
 Upgrade — start the upgrade which creates and updates nvOS to new boot environments but does not reboot
the fabric.
 Reboot — reboots the entire fabric after all server-switches are upgraded to new boot environments. It is also
possible during this phase to abort the process and discard the new boot environments.
The fabric is locked during the entire process and you cannot change any configurations during the process.
Before You Begin the Fabric Upgrade
Before you begin, you may want to consider the following options for the fabric-upgrade-start command:
 auto-finish — you can specify to automatically reboot the entire fabric after the upgrade is complete.
 rolling — specify if you want to perform a rolling fabric upgrade. A rolling fabric upgrade performs the
upgrade procedure on a switch-by-switch basis and copies the software package from the controller to other
switches in the fabric. If you specify no-rolling, all switches are booted after the upgrade.
 abort-on-failure — specify if you want the upgrade to stop if there is a failure during the process.
 manual-reboot — specify if you want to manually reboot individual switches after the upgrade process. If
you specify no-manual-reboot, all switches reboot automatically after the upgrade is complete.
 prepare — specify if you want to perform setup steps prior to performing the upgrade. This step copies the
offline software package and then extracts and prepares it for the final upgrade process. Once you begin the
prepare process, you cannot add new switches to the fabric.
— specify to reboot switches in parallel if the switches are in a cluster configuration. Or,
you can reboot them one at time using the reboot-single option.
 reboot-parallel
 reboot-group — specify the number of switches to reboot as a group in parallel mode. The default is the
maximum number of switches in the fabric up to 100 switches.
Starting the Fabric Upgrade
1. Download the latest nvOS software from the update server onto a switch in the fabric.
2. Copy the nvOS software package to each switch in the fabric.
3. Select a switch in the fabric to act as the upgrade controller switch, and use the fabric-upgrade-start
command to begin the upgrade.
4. Depending on the options selected, the upgrade completes by reboot the fabric or rebooting all of the switches.
Pluribus Networks Configuration Guide
31
www.pluribusnetworks.com
Starting the Rolling Fabric Upgrade
If you opted for a rolling fabric upgrade, then the upgrade controller switch begins copying to software packages to
other switches in the fabric. Other than this step, the rolling fabric upgrade has the same behavior as a fabric
upgrade depending on the selected options.
You can check the status of the upgrade using the fabric-upgrade-status-show command:
CLI (network-admin@sw1) > fabric-upgrade-status-show
log
switch
state
----------------------------------------------- -------- -----------------(0:00:36)Upgrading software upgrade framework
sw3
Running
(0:00:08)Computing package update requirements. sw2
Running
(0:00:12)Agent needs restart
sw1* Agent restart wait
The first entry in the log is the duration of the upgrade process. It does not include waiting time. The switch with the
asterisk (*) is the controller server-switch where the fabric-upgrade-start command was issued.
Additional commands for the fabric upgrade feature:
— you can issue this command at any time during the fabric upgrade to reboot
all nodes in the fabric and complete the upgrade. Once the upgrade phase is complete, all server-switches
display the “Upgrade complete” message in the log field. You can then safely reboot the fabric.
 fabric-upgrade-finish
— aborts the software upgrade process. All changes to the server-switches are
cleaned up and the server-switches do not reboot. The configuration lock on the fabric is also released.
 fabric-upgrade-abort
If you issue the fabric-upgrade-abort command during the upgrade process, it may take some time before the
process stops because the upgrade has to reach a logical completion point before the changes are rolled back
on the fabric. This allows the proper cleanup of the changes.
 fabric-upgrade-prepare-cancel — cancels a fabric upgrade that was prepared earlier.
 fabric-upgrade-prepare-resume — resume a fabric upgrade that was prepared earlier.
 fabric-upgrade-prepare-show — displays the status of prepared upgrades on the fabric nodes.
Displaying and Managing Boot Environment Information
You can display information about the different boot environments on the switch. There are two boot environments:
the current boot environment, and the previous boot environment. To display boot environment information, use
the following command:
CLI network-admin@switch > bootenv-show
name
----------netvisor-22
netvisor-23
version
---------2.2.7-7356
2.3.1-8600
current
------no
yes
reboot
-----no
yes
space
----58.5M
27.4G
created
------------------2015-12-07,09:55:58
01-06,09:13:11
To reset the boot environment and reboot using the previous environment, use the following syntax:
CLI network-admin@switch > bootenv-activate-and-reboot name netvisor-22
To delete a boot environment, use the following syntax:
CLI network-admin@switch > bootenv-delete name netvisor-22
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
32
Enabling Administrative Services
There are many features of the Pluribus Networks fabric that require or can be enhanced using remote access. For
example, when packets are written to a log file, you may want to transfer that file from a switch to a different system
for analysis. Also, if you are creating a NetVM environment, an IOS image of the guest OS must be loaded on the
switch.
There are two file transfer methods:
 Secure File Transfer Protocol (SFTP)
 Network File System (NFS)
Both methods must be enabled before you can use them. Because SFTP relies on Secure Shell (SSH), you must
enable SSH before enabling SFTP.
1. To check the status of SFTP, use the following command:
CLI network-admin@switch > admin-service-show
switch
-------
nic
---
ssh
---
nfs
---
web
---
web-port
--------
snmp
----
net-api
-------
icmp
----
pleiades24
mgmt
off
on
off
80
off
off
off
2. To enable SSH, use the following command:
CLI network-admin@switch > admin-service-modify nic mgmt ssh
admin-sftp-modify enable
sftp password: <password>
confirm sftp password: <password>
The default SFTP username is sftp and the password can be change using the admin-sftp-modify
command:
CLI network-admin@switch > admin-sftp-modify
sftp password: <password>
confirm sftp password: <password>
CLI network-admin@switch > admin-service-show
switch
-----pleiades24
nic
--mgmt
ssh
--on
nfs
--on
web
--off
web-port
-------80
snmp
---off
net-api
------off
icmp
---off
CLI network-admin@switch > admin-sftp-show
switch:
sftp-user:
enable:
pleiades24
sftp
yes
Use SFTP from a host to the switch, and login with the username sftp and the password that you configured for SFTP.
Then you can download the available files or upload files to the switch.
Pluribus Networks Configuration Guide
33
www.pluribusnetworks.com
3. You can check the status of NFS service and enable it using the following command:
CLI network-admin@switch > admin-service-show
switch
nic
ssh
nfs
web
web-port
snmp
net-api
icmp
------
---
---
---
---
--------
----
-------
----
on
off
on
80
off
on
on
pleiades01 mgmt
To enable NFS, use the following command:
CLI network-admin@switch > admin-service-modify nic mgmt nfs
After you enable NFS, the directory /nvOS is mountable using NFS through the management IP addresses for access
to the files in that directory.
Saving and Restoring Server-Switch Configurations
A switch contains local configuration information such as port settings as well as fabric configuration information.
Fabric configurations are stored on every switch in the fabric and does not require that you save and restore before
replacing a switch. When a switch is replaced, removed, or otherwise disrupted, you can save and restore the local
configuration information.
The information that is saved and restored on the local switch includes the following:
 VNETs with VNET manager running on the switch
 Port VLAN associations
 Netvisor Zone configuration details, but not any modifications to NetZones such as installed applications
 Netvisor VMM configuration details, but not ISO images or disk images
 Netvisor KVM configuration details, but not ISO images or disk images
 Network services running on the switch
To display a full list of the current configuration details for a switch, use the running-config-show command.
SFTP and NFS can be used to transfer the configuration file, but you must enable the two features before using
them.
Caution! There is a potential for data loss when restoring a configuration. The configuration on the
switch is replaced by the configuration stored in the import file. Although ISO images and disk-library
images are not likely to disappear, you should only perform switch-config-import on a
switch that doesn’t have important data stored on it.
As a precaution, consider using the command switch-config-export to save the data on
the switch that you are importing the configuration file.
Also, copy the ISO images and disk images from the switch using the iso-image-library and
disk-library-image-export commands and copying the files from the switch.
1. To save the switch configuration to a file, use the following command:
CLI network-admin@switch > switch-config-export export-file pleiades24
Exported configuration to /nvOS/export/pleiades24.2013-11-04T22.33.31.tar.gz
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
34
2. To display the files available for import and export, use the following command:
CLI network-admin@switch > switch-config-show
switch
pleiades24
export-file
pleiades24.2013-11-04T22.33.31.tar.gz
You can now copy the configuration file to a different host using SFTP or NFS. For example, you can SFTP to the
switch-ip-address, and login using the SFTP password. Then use cd/nvOS/import, and use get to
download the configuration file.
The switch-config-export command is used to export the configuration of the local switch. The file that
is created is a tar file that includes a number of configuration files for the switch. The file is created under
/nvOS/export. This is the command used to export the current configuration on the local switch. vAlso, each
time you reset the switch using the command, switch-config-reset, a backup of the configuration is made
and places a file in the same location.
Once the switch configuration is exported, it becomes available to import on the same switch, by using the
switch-config-copy-to-import command. nvOS copies the configuration tar file from the
/nvOS/export to the /nvOS/import directory. Once in the /nvOS/import directory, it is possible to use
the switch-config-import command to import the switch configuration.
The switch-config-import command is used to import a configuration on the local switch. When using
that command, the intention is to import a switch configuration t previously exported by the same switch.
The switch-config-import command has a few parameters to it. The ignore-system-config and
the apply-system-config parameters are 2 parameters that allow the imported configuration of the switch
to override or not override the currently configured information found under the switch-setup-show
command. When you select the ignore-system-config parameter, the local configuration is saved to an archive. If
you select apply-system-config, the settings in the tar file are applied to the local switch.
When you import a configuration using the switch-config-import command, the current configuration on
the switch is overwritten by the imported configuration file.
The skip-fabric-join option imports the fabric configuration from the tar file. However, this information
may be out of date with respect to the fabric if transactions have occurred on the fabric since the file was
exported which causes the imported configuration to be out-of-sync with the current fabric. The alternative is to
specify do-fabric-join, which extracts the fabric name from the tar file, and attempts to join the fabric and
download the current fabric configuration, so that it is in sync with the rest of the fabric. The fabric configuration
in the tar file is ignored, but cluster and local configurations are imported from the tar file.
When a switch that was part of a cluster is replaced, the fabric-join repeer-to-cluster-node
command is used for the new switch to receive all required switch configuration, including the local configuration.
Pluribus Networks Configuration Guide
35
www.pluribusnetworks.com
To upload a configuration file to a switch and set the configuration for the switch using the configuration file, you
must transfer the configuration file to the target switch using the following sequence of commands:
sftp sftp@<switch-ip-address>
Connecting to switch-ip-address
Password: <password>
sftp> cd nvOS/import
sftp> put pleiades24.2013-11-04T22.33.31.tar.gz
Informational Note: The configuration file must use the *.tar.gz extension to be recognized by
nvOS.
CAUTION! Loading the configuration file causes nvOS to restart which results in a brief interruption to
switch traffic flow.
Now load the configuration file which replaces the current configuration on the switch with the information in the
file.
CLI network-admin@switch > switch-config-import import-file
pleiades24.2013-11-04T22.33.31.tar.gz
New configuration imported. Restarting nvOS...
Connected to Switch pleiades24; nvOS Identifier:0xb000011; Ver: 0.19.3747
There are many options available that allow you to control how the switch-config-import modifies the
switch, including the following:
 ignore-system-config
- ignore the current system configuration. The settings in the *.tar file are not
applied to the local switch.
— apply the system configuration in the imported file. The settings in the *.tar file
are applied to the local switch. You typically do not want to use this option as it changes the in-band IP address
and other settings.
 apply-system-config
— opt out of joining the fabric. This setting imports the fabric configuration from the
*.tar file, but this information may be out of date with respect to the fabric if additional transactions occur on
the fabric since the file was exported.
 skip-fabric-join
— join the current fabric. This setting extracts the fabric name from the *.tar file and
attempts to join the fabric. Then the switch contacts the current fabric to download the configuration so that
the switch is in sync with the rest of the fabric. Cluster and local configurations are imported from the *.tar file.
 do-fabric-join
 no-replace-switch
— do not replace the current switch.
— replace the current switch. This setting is used to replace a faulty switch and after
importing the file, has the same configuration as the replaced switch. This replaces all of the local, cluster, and
fabric configuration by downloading the configurations from peer switches. No configuration is necessary or
advised before running this command. However, you need to run the initial quickstart to obtain an in-band IP
address.
 replace-switch
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
36
By default, the initial switch system configuration, management IP addresses and other parameters, are not applied
if there is another switch in the fabric with the same settings. To apply the initial settings, use the
apply-system-config option. Also, by default, the imported configuration attempts to join the same fabric
that the original switch was a member. If that join fails, then the import fails. You can avoid this issue by using the
skip-fabric-join option. Finally, if the original switch is still on the network and you want to copy the
configuration to a new switch, but you want to prevent the new switch from taking ownership of any objects specific
to the original switch, such as VNET services, or VLAN port settings, you must use the no-replace-switch
option.
Copying and Importing Configuration Files
You can create a configuration file to import to another switch by using the
switch-config-copy-to-import command. To create a configuration file with the name config-092613 to
import on another switch, use the following syntax:
CLI network-admin@switch > switch-config-copy-to-import export-file
config-092613
After you create the configuration file, you can export it to /nvOS/export/ directory, and SFTP to it from the
target switch.
To review the available files for import and export, use the following syntax:
CLI network-admin@switch > switch-config-show
switch
pbg-nvos
export-file
config-092613.tar.gz
Depending on the available remote access services, you can now copy the configuration file to a different switch. For
example, you can SFTP to another switch using the IP address of the switch, login as SFTP with the password that
you previously set, cd /nvOS/import and get the configuration file.
To upload the configuration file to the target switch and set the configuration from the configuration file, transfer
the configuration file to the target switch with the IP address, 192.168.3.35.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Changing the IP Port for vManage
vManage is a Web-based service and it listens on an IP port to accept communications. By default, vManage listens
on port 80 on the management IP address that you set during the initial configuration, and can be reached using a
supported Web browser such as Safari, Firefox, or Chrome using the URL http://mgmt-ip. In some cases, you may
want to configure vManage to listen on a different port as in the case of a virtual load balancer sending traffic
arriving on port 80 of the management IP address to other systems. In this case, vManage cannot listen on port 80.
Use the admin-service command to change the listening port. Changing the port disrupts any current
connections to vManage.
1. To change the listening port to 8080 for vManage, use the following syntax:
CLI network-admin@switch > admin-service-modify nic mgmt web-port 8080
Pluribus Networks Configuration Guide
37
www.pluribusnetworks.com
2. To check the status of admin services, use the following command:
CLI network-admin@switch > admin-service-show
switch
nic
ssh
pleiades24 mgmt on
sftp
on
nfs
on
web
on
web-port
8080
snmp
off
net-api
on
icmp
on
Displaying System Statistics on a Server-Switch
You display system statistics on a server-switch using the system-stats-show command:
CLI network-admin@switch > system-stats-show layout vertical
switch:
uptime:
used-mem:
used-swap:
swap-scan:
cpu-user:
cpu-sys:
cpu-idle:
pleiades24
1h22m26s
27%
0%
0
0%
1%
98%
The swap-scan output displays the number of scans performed on the swap. A nonzero number indicates that
memory is paged from the physical memory (RAM) to virtual memory (disk or swap). A consistently high value
indicates that all memory, both physical and virtual, is exhausted and the system may stop responding.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
38
Displaying Connection Statistics
You can display information about the connection statistics for hosts connected to the switch:
CLI network-admin@switch > connection-stats-show ip 10.10.11.3
switch:
mac:
vlan:
ip:
port:
iconns:
oconns:
ibytes:
obytes:
total-bytes:
first-seen:
last-seen:
last-seen-ago:
switch:
mac:
vlan:
ip:
port:
iconns:
oconns:
ibytes:
obytes:
total-bytes:
first-seen:
last-seen:
last-seen-ago:
switch:
mac:
vlan:
ip:
port:
iconns:
oconns:
ibytes:
obytes:
total-bytes:
first-seen:
last-seen:
last-seen-ago:
switch:
mac:
vlan:
ip:
port:
iconns:
oconns:
ibytes:
obytes:
total-bytes:
first-seen:
last-seen:
pleiades24
66:0e:94:21:0e:7b
14
172.16.23.1
65
13
0
132K
375M
375M
06-16,08:15:24
06-16,08:19:11
31d30m19s
pleiades24
66:0e:94:21:f3:34
14
172.16.23.1
65
14
0
132K
375M
375M
06-16,11:54:12
06-16,11:58:25
30d20h51m5s
pleiades24
66:0e:94:21:67:e1
11
172.16.23.1
65
57
0
398K
1.10G
1.10G
06-20,15:05:39
07-02,09:44:05
14d23h5m25s
pleiades24
66:0e:94:21:78:2e
14
172.16.23.1
65
69
1
662K
1.83G
1.83G
06-16,14:58:42
06-17,11:12:48
Pluribus Networks Configuration Guide
39
www.pluribusnetworks.com
last-seen-ago: 29d21h36m42s
Rebooting, Powering Off, and Resetting the Server-Switch
There are two recommended ways to reboot a switch:
CLI command switch-reboot
Power button
To reboot the switch using the CLI, use the following command:
CLI network-admin@switch > switch-reboot
Informational Note: The switch-reboot command applies only to the switch where the command is
executed. You cannot reboot a remote switch using this command.
Alternatively, you can use the power button located on the front of the switch to power off.
To power off the switch, press and hold the front power button for approximately ten seconds until the power
button light changes from a rapid blink to a slow flashing cycle. The power button light tuns off and now the switch
is powered off.
You can also use the command, switch-poweroff, to turn off a switch.
To complete the process, switch the power toggle on the rear of the switch from 1 to 0. The system is now
completely powered off.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Installing the nvOS Linux API
nvOS is bundled with a Linux API that allows installation of nvOS on any Linux-based server. The API installs libraries
under /lib64, documents under /usr/share/java/doc/libnvOS/index.html, and sample code under
/usr/share/src/nvOS/samples.
Informational Note: You must physically connect the Linux host to the switch.
1. Modify the SFTP permissions on the switch using the admin-sftp-modify enable command.
To install the API on a Linux platform, use the following command:
CLI network-admin@switch > api-install linux-host name linux-host-string user
user-string
To run nvOS on the Linux host, use the following command:
cli --host switch-name ip
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
40
Layer 2 Enhancements
Two new commands allow you to verify Layer 2 table entries.
l2-check-fix
fix inconsistent L2 entries
any of the following options:
mac mac-address
MAC address assigned to vPort
vlan vlan-id
VLAN assigned to vPort
vxlan vxlan-id
VXLAN assigned to vPort
sw-port sw-port-number
Port in software
sw-state
active|static|vrrp|tunnel|software|needs-peer-status|port-mac|
hit
State in software
hw-port hw-port-number
Port in hardware
hw-state
active|static|vrrp|tunnel|software|needs-peer-status|port-mac|
hit
State in hardware
fix-action none|add-to-hardware|remove-from-hardware|
fix-port-in-hardware|FAILED-add-to-hardware| FAILED-remove-from-hardware|
FAILED-fix-port-in-hardware
Fix action
l2-check-show
show inconsistent L2 entries
any of the following options:
mac mac-address
MAC address assigned to vPort
vlan 0..4095
VLAN assigned to vPort
vxlan 0..16777215
VXLAN assigned to vPort
sw-port sw-port-number
Port in software
sw-state
active|static|vrrp|tunnel|software|needs-peer-status|port-mac|
hit
State in software
hw-port hw-port-number
Port in hardware
hw-state
active|static|vrrp|tunnel|software|needs-peer-status|port-mac|
hit
State in hardware
fix-action none|add-to-hardware|remove-from-hardware|
fix-port-in-hardware|FAILED-add-to-hardware| FAILED-remove-from-hardware|
FAILED-fix-port-in-hardware
Fix action
Pluribus Networks Configuration Guide
41
www.pluribusnetworks.com
Configuring Port Attributes
 Configuring Ports for Different Throughput
 Displaying Port Status
 Displaying Port Statistics
 Managing Control Plane Traffic Protection (CPTP)
 Display Physical Port Layer 2 Information
 Displaying Transceiver Information
 Configuring Port Storm Control
 Enabling Jumbo Frame Support
 Loop-Free Layer 2 Topology
Configuring Ports for Different Throughput
By default, ports on the switches are configured as 40GbE ports. You can also use them as 4 x 10GbE with
the right transceiver. To refer to the 40Gb port, use the last port number of the port group. For example,
the first 40Gb port, in the example above, is referred to as port 49 for 40GbE use and as ports 49, 50, 51,
and 52 for 4/10Gb use.
If you want to change the 40Gb port to 4x10Gb functionality, use the following command sequence:
CLI network-admin@switch > port-config-modify port 49-52 speed 10g
To change the port back to 40Gb operation, use the following command sequence:
CLI network-admin@switch > port-config-modify port 49 speed 40g
The default port speed is 10G and you can modify the parameters of a port:
 Speed - you can disable the port or set the speed to 10m, 100m, 1g, 2.5g, 10g, or 40g
 Egress rate — limit the egress rate or set to unlimited.
 Ethernet mode type — set the mode type to 1000base-x,
sgmii, autonegotiate
 Enable or disable a port
 LACP priority — between 1 and 65535
 Reflect — received frames are reflected for loopback testing.
 Edge-switch — Specify if the port connects to another ONVL device or is an uplink to a third-party
switch or host.
 Pause — pause traffic on the port.
 Description — description of the port
 Loopback — specify loopback
 Mirror-receive — receive mirrored traffic only.
 MAC address — specify a MAC address for the port.
 VLAG failover — specify if the port is used in VLAG failover.
 Sending port number — specify if the port number is sending traffic.
www.pluribusnetworks.com
Displaying Port Status
You can use the port-show command to display status information on all ports with active links.
Details for each port include the IP addresses and MAC addresses of hosts connected to that port. There
can be more than one host if a network device such as a switch is connected. The command also displays
the VLAN of the port, port status, and configuration details.
To display all port information for ports 1-6 on the switch, use the command, port-show port 1-6:
CLI network-admin@switch > port-show port 1-6
switch:
port:
bezel-port:
ip:
mac:
vlan:
vxlan:
hostname:
last-seen:
status:
loop-vlans:
lport:
config:
switch:
port:
bezel-port:
ip:
mac:
hostname:
status:
loop-vlans:
lport:
rport:
config:
trunk:
Spine1
0
0
192.168.1.3
2e:b2:a1:73:39:d1
1
0
Spine1
02-26,11:24:08
up,PN-internal,stp-edge-port
none
0
Spine1
3
3
192.168.1.5
a2:c6:9f:57:c4:0c
Leaf2
up,PN-fabric,LLDP,trunk,LACP-PDUs,vlag-active
none
3
3
fd,10g
4000-1_34
Displaying Port Statistics
You can also display statistics for all active ports on the switch. This information is useful for
understanding the traffic on the ports.
www.pluribusnetworks.com
Use the port-stats-show command to display the information:
CLI network-admin@switch > port-stats-show
port 5-6 format all layout vertical
switch:
s6000-2
time:
12:59:33
port:
5
counter:
0
ibytes:
1021K
iUpkts:
1.06K
iBpkts:
0
iMpkts:
864
iPauseFs:
0
iCongDrops: 0
idiscards:
764
ierrs:
0
obytes:
978K
oUpkts:
1.09K
oBpkts:
1
oMpkts:
837
oPauseFs:
0
oCongDrops: 0
odiscards:
0
oerrs:
0
port-speed: 40g
The output headers have the following meaning:
 switch — switch name
 time — the time that the command is issued
 port — port number
 counter — number of counters for the port
 ibytes — number of incoming bytes in K (Kilobytes), M (Megabytes), or G (Gigabytes)
 iUpkts — number of incoming unicast packets
 iBpkts — number of incoming broadcast packets
 iMpkts — number of incoming multicast packets
 iPauseFs — number of incoming paused fragmented packets
 iCongDrops — number of incoming packets dropped due to congestion
 idiscards — number of discarded incoming packets
 ierrs — number of incoming packets with errors
 obytes — number of outgoing bytes K (Kilobytes), M (Megabytes), or G (Gigabytes)
 oUpkts — number of outgoing unicast packets
 oBpkts — number of outgoing broadcast packets
 oMpkts — number of outgoing multicast packets
 oPauseFs — number of outgoing paused fragmented packets
 oCongDrops — number of outgoing packets dropped due to congestion
 odiscards — number of discarded outgoing packets
 oerrs — number of outgoing packets with errors
www.pluribusnetworks.com
Using Port Buffering
You can modify and display the port buffering settings for the switch ports. To display the port buffering
settings, use the port-buffer-settings-show command:
CLI network-admin@switch > port-buffer-settings-show
switch: Spine1
enable: yes
interval: 1m
disk-space: 50M
To modify port buffering settings, use the port-buffer-settings-modify command:
CLI network-admin@switch > port-buffer-settings-modify interval 2m
You can modify the buffer interval, duration, disk space, and enable or disable port buffering on the
switch.
To display the port buffer, use the port-buffer-show command:
CLI network-admin@switch > port-buffer-show
switch: Spine1
port: 0
ingress-used-buf: 0%
ingress-used-buf-val: 0
egress-used-buf: 0%
egress-used-buf-val: 0
switch: Spine1
port: 3
ingress-used-buf: 0%
ingress-used-buf-val: 0
egress-used-buf: 0%
egress-used-buf-val: 0
switch: Pleiades24
port: 57
ingress-used-buf: 0%
ingress-used-buf-val: 0
egress-used-buf: 0%
egress-used-buf-val: 0
switch: Spine1
port: 65
ingress-used-buf: 0%
ingress-used-buf-val: 0
egress-used-buf: 0%
egress-used-buf-val: 0
www.pluribusnetworks.com
switch: Spine2
port: 0
ingress-used-buf: 0%
ingress-used-buf-val: 0
egress-used-buf: 0%
egress-used-buf-val: 0
switch: Spine2
port: 1
ingress-used-buf: 0%
ingress-used-buf-val: 0
egress-used-buf: 0%
egress-used-buf-val: 0
Managing Control Plane Traffic Protection (CPTP)
Control Plane Traffic Protection (CPTP) applies to the internal control, data, and span ports which all
connect to the CPU, so the CPU resources are protected from large quantities of traffic arriving from
different sources such as control packets, cluster communication, fabric updates as well as the regular
flood traffic, learning packets and copy-to-cpu packets.
The purpose of CPTP is to classify the traffic on hardware to different Class of Service (CoS), and perform
priority scheduling between them, and also apply a rate limit for each of the CoS, to protect the CPU
resources and at the same time, provide a Service Level Agreement (SLA) for critical traffic.
CLI network-admin@switch > port-cos-rate-setting-show
switch port port-number cos0-rate cos1-rate cos2-rate cos3-rate cos4-rate cos5-rate
cos6-rate
--------- ----- ----------- --------- --------- --------- --------- --------- --------Spine1 pci-e 0 100 100 1000000 1000000 1000000 1000000 1000000 ...
Spine1 data 65 100 100 1000000 1000000 1000000 1000000 1000000 ...
Spine1 span 66 100 100 1000000 1000000 1000000 1000000 1000000 ...
You can modify the CoS rate settings using the port-cos-rate-setting-modify command. The
rate limits are set in packets per second.
CLI network-admin@switch > port-cos-stats-show
switch:
Spine1
port:
65
cos0-out:
977
cos0-drops: 0
cos1-out:
0
cos1-drops: 0
cos2-out:
0
cos2-drops: 0
cos3-out:
0
cos3-drops: 0
cos4-out:
0
cos4-drops: 0
cos5-out:
0
cos5-drops: 0
cos6-out:
124K
cos6-drops: 0
cos7-out:
25.2K
cos7-drops: 0
To clear the statistics for CoS on the ports, use the port-cos-stats-clear command.
www.pluribusnetworks.com
On the F64, the default CoS rate is 1,000,000 pps for pci-e port and 100,000 pps for data port and span
ports.
On E68 and E28Q, the default CoS rate is 100,000 pps for all internal ports.
Display Physical Port Layer 2 Information
You can display physical port information at Layer 2 using the port-phy-show command. This
command displays information about the default VLAN, link quality, maximum frame size, Ethernet
mode, speed, and status. You can also display the default VLAN for a port.
CLI network-admin@switch > port-phy-show
port state speed eth-mode
max-frame link-quality learning def-vlan
17
up
1000 1000base-x 1540
n/a
on
1
19
up
10000 10Gbase-cr 10232
n/a
on
1
Displaying Transceiver Information
You can display information about the transceivers connected to the switch using the
port-xcvr-show command:
CLI network-admin@switch-name > port-xcvr-show
switch
port vendor-name
part-number
serial-number
-------------- ---- ---------------- ------------------------------Spine1
3
3M
1410-P17-00-0.50
Spine1
4
3M
1410-P17-00-0.50
Spine1
53
FCI Electronics 10093084-2010LF 0015
Spine1
57
3M Company
9QA0-111-12-1.00 V10B9252
Spine1
65
3M Company
9QA0-111-12-1.00 V10B9614
Configuring Port Storm Control
Storm control prevents traffic on a LAN from being disrupted by a broadcast, multicast, or unicast storm
on a port. A LAN storm occurs when packets flood the LAN, creating excessive traffic and degrading
network performance.
Use the port-storm-control-modify to modify the percentage of total available bandwidth that can be
used by broadcast, multicast, or unicast traffic.
CLI network-admin@switch > port-storm-control-modify port 11
unknown-ucast-level 1.1
Use the port-storm-control-show command to display the configuration:
CLI network-admin@switch > port-storm-control-show
switch intf speed unknown-ucast-level unknown-mcast-level broadcast-level
------ --- ---- --------------------- ------------------- --------------pl-12
11 10g 1.1%
2.2%
3.3%
www.pluribusnetworks.com
Enabling Jumbo Frame Support
Jumbo frames are frames that are bigger than the standard Ethernet frame size, which is 1518 bytes
(including Layer 2 (L2) header and FCS). The definition of frame size is vendor-dependent, as these are
not part of the IEEE standard.
When the jumbo frame feature is enabled on a port, the port can switch large or jumbo frames. This
feature optimizes server-to-server performance. The default Maximum Transmission Unit (MTU) frame
size is 1548 bytes for all Ethernet ports. The MTU size is increased to 9216 bytes when the jumbo frame
feature is enabled on a port.
Jumbo frame support is disabled by default.
To enable jumbo frame support, add the jumbo parameter to the port-config-modify command:
CLI network-admin@switch > port-config-modify jumbo
Loop-Free Layer 2 Topology
Netvisor Loop Detection operates in conjunction with Rapid Spanning Tree Protocol (RSTP) . RSTP is used
to ensure loop free topology of the VLANs in the Layer 2 network as far as the networking equipment is
concerned.
RSTP prevents loops in the network caused by miscabled networking equipment, but does not address
misconfigured hosts. Netvisor Loop Detection goes beyond STP to protect the network from
misconfigured or miscabled hosts attached to the network.
Informational Note: This feature is not available on the F64 platform and is enabled by default on
E68-M and E28Q platforms.
Netvisor Control Plane — The Netvisor control plane includes information about every MAC address
attached to the Layer 2 network in a vPort database. The vport database is distributed throughout the
fabric so that each Netvisor switch has a copy of the vPort database for the entire fabric.
A MAC address is stored in a vPort, which includes the following information:
MAC address, VLAN ID, and VXLAN ID
owner-port and local-port
migration history including owner, time, and port
vPort state as active, static, moving, or loop-probe
Access to the Netvisor fabric goes through the nvOS software. Netvisor makes decisions on whether to
allow endpoints access to the network based on control plane data structures including the vPort
database.
Detecting Loops
Netvisor Loop Detection is implemented as part of Netvisor source MAC address miss handling.
Hardware learning of MAC addresses is disabled, so when a packet arrives with an unknown MAC
address, the switch sends the packet to nvOS rather than switching the packet normally. nvOS examines
the vPort table to determine if a packet with an unknown MAC is indicative of a loop.
nvOS uses two criteria to detect a loop on the network:
www.pluribusnetworks.com
A MAC address associated with an in-band NIC of a node in the fabric appears as the source
MAC on a packet that ingresses on a host port. Netvisor detects this situation by noting the
PN-internal status of a vPort that would otherwise migrate to a host port. Netvisor does not
allow the migration to take place and starts loop mitigation.
For the purposes of Netvisor Loop Detection, a host port is defined as a port not connected to another
Pluribus switch, not an internal port, and does not participate in STP with Netvisor which means that
Netvisor is not configured for STP or the device connected on the port is not configured for STP.
Packets with the same source MAC address arrive on multiple host ports in the fabric at
approximately the same time. In order to support VM and host migration, some rapid
movement of MAC addresses through the fabric is tolerated. When the same MAC address
moves rapidly back and forth between two ports, a loop is assumed and loop mitigation starts.
VRRP MAC addresses are not subject to Loop Detection and Mitigation, and can migrate freely.
Loops are detected on a port by port basis. A single loop typically involves two ports, either on the same
switch or on two different switches. When multiple loops are present, more than two ports are involved
and each port is still handled separately.
Loop Mitigation
When a loop is detected a message is logged to the system log indicating the host port and VLAN
involved in the loop. In addition the host port involved in the loop has the "loop" status added and the
VLAN is added to the host port's loop-vlans VLAN map, so that looping ports and VLANs can be seen in
"port-show" output.
At the start of loop mitigation Netvisor creates vports for use in sending loop probe packets. The vPorts
use the port MAC address for the in-band NIC port, have a status of PN-internal, and a state of
loop-probe. loop-probe vports are propagated throughout the fabric. A loop-probe vport is created for
each VLAN that is looping.
At the start of loop mitigation Netvisor removes all vports from the looping host port and VLAN. This
prevents the hardware from sending unicast packets to the looping port, and causes every packet that
arrives on the looping port to come to software as a source mac miss. During loop mitigation all packets
that arrive on the looping port are dropped.
During loop mitigation Netvisor sends loop probe packets on the looping VLANs every 3 seconds. As long
as the loop persists, Netvisor receives the probe packets as source mac miss notification on the looping
ports, so Netvisor can tell that the loop is still present. If 9 seconds go by with no received probe packets,
Netvisor detects that the loop is resolved and ends loop mitigation.
The end of loop mitigation involves logging a message to the system log, removing the loop-probe
vports, and removing the loop status and loop-vlans from the looping port.
To view affected ports, use the port-show command and add the parameter, status loop:
network-admin@switch-31>port-show status loop
switch
port hostname status
config
---------- ---- -------- --------------------- -----switch-31 9
up,stp-edge-port,loop fd,10g
switch-32 9
up,stp-edge-port,loop fd,10g
Note the new status, loop, in the status column.
www.pluribusnetworks.com
During loop mitigation, the MAC addresses for loop probes are displayed in the vPort table:
CLI (network-admin@switch-31) > vport-show state loop-probe
owner
mac
vlan ports state
hostname
status
---------- ----------------- ---- ----- ---------- -------------------switch-32 06:c0:00:16:f0:45 42
69
loop-probe leo-ext-32
PN-internal
switch-31 06:c0:00:19:c0:45 42
69
loop-probe leo-ext-31
PN-internal
Note the loop-probe state as well as the PN-internal state. The loop probes use the the format of the
port-mac addresses, and uses the internal port for the in-band NIC.
If you notice a disruption in the network, use the port-show command to find the looping ports, and
fix the loop. Fixing the loop typically involves correcting cabling issues, configuring virtual switches, or as
a stop-gap measure, using the port-config-modify command to change port properties for the
looping host ports. Once the loop is resolved, Netvisor no longer detects probes and leaves the loop
mitigation state, while logging a message:
2016-01-12,12:18:41.911799-07:00 leo-ext-31 nvOSd(25695)
system
host_port_loop_resolved(11381) : level=note : port=9 :
Traffic has stopped looping on host-port=9
At this point the loop status is removed from the port-show output for port 9 and the loop-probe
vPorts are removed.
Netvisor Loop Detection exposes loops using system log messages, port-show output, and vport-show
output. Netvisor Loop Detection is enabled or disabled by using the sys-flow-setting-modify
command:
network-admin@e68-leaf-01>sys-flow-setting-modify block-loops
network-admin@e68-leaf-01>sys-flow-setting-modify no-block-loops
The block-loops argument to sys-flow-setting-modify is not available on the F64 because
Netvisor Loop Detection does not work with the Alta chip.
When Netvisor detects an internal port MAC address on a host port, Netvisor prints a log message:
system
2016-01-19,15:36:40.570184-07:00 mac_move_denied
11379 note MOVE DENIED mac=64:0e:94:c0:03:b3 vlan=1 vxlan=0
from switch=leo-ext-31 port=69 to deny-switch=leo-ext-31
deny-port=9
reason=internal MAC of local switch not allowed to change ports
Netvisor starts Loop Mitigation by logging a message:
system
2016-01-19,15:36:40.570334-07:00 host_port_loop_detected
11380 warn Looping traffic detected on host-port=9
vlan=1. Traffic on this port/VLAN will be ignored until loop
resolved
During Loop Mitigation, Netvisor sends loop probes. When these probes, as well as any other packets,
are received on a looping host port, Netvisor logs a message:
system
2016-01-19,15:59:54.734277-07:00 mac_move_denied
11379 note MOVE DENIED mac=06:c0:00:19:c0:45 vlan=1 vxlan=0
from switch=leo-ext-31 port=69 to deny-switch=leo-ext-31
deny-port=9 reason=port is looping
www.pluribusnetworks.com
mac_move_denied messages are throttled down to one every 5 seconds for each vPort. This prevents
the system log from filling up with mac_move_denied messages during loop mitigation.
During loop mitigation the administrator can use the port-show command to see which ports are
involved in the loop:
CLI (network-admin@e68-leaf-01)> port-show status loop
switch
port hostname status
loop-vlans config
---------- ---- -------- --------------------- ---------- -----e68-leaf-01 9
up,stp-edge-port,loop 1
fd,10g
e68-leaf-01 9
up,stp-edge-port,loop 1
fd,10g
Note the loop status in the status column and the loop-vlans column.
During loop mitigation the MAC addresses for loop probes are displayed the vPort table:
CLI (network-admin@leo-ext-31) > vport-show state loop-probe,
owner
mac
vlan ports state
hostname status
---------- ----------------- ---- ----- ---------- -------- --------e68-leaf-01 06:c0:00:16:f0:45 42
e68-leaf-01 06:c0:00:19:c0:45 42
69
69
loop-probe leo-ext-32 PN-internal
loop-probe leo-ext-31 PN-internal
Note the loop-probe state as well as the PN-internal state. The format of the MAC address for
loop probes follows the format of port-mac addresses, and uses the internal port for the in-band NIC.
When you notice a disruption in the network, you use the port-show command to find the looping ports
and VLANs, and then fix the loop. Fixing the loop typically involves correcting cabling issues, configuring
virtual switches, or as a stop-gap measure using port-config-modify to change port properties for the
host ports that are looping. Once the loop has been resolved Netvisor detects no more probes received
and leaves loop mitigation, logging a message:
system
2016-01-19,15:39:37.601499-07:00
host_port_loop_resolved
11381 note Traffic has stopped looping on host-port=9 vlan=1
At this point the loop status is removed from port-show output for port 9 and the
loop-probe vPorts are removed.
www.pluribusnetworks.com
Configuring Rapid Spanning Tree Protocol (RSTP)
Spanning Tree Protocol (STP) is a standard inter-switch protocol to ensure that an ad hoc network topology is
loop-free at Layer 2, on a per-VLAN basis. If your network connections form loops and STP is disabled, packets
re-circulate between the switches, causing a degradation of network performance. If you are certain that your
network connections are loop-free, you do not need to enable STP.
To build a loop-free topology, switches (“bridges”) have to determine the root bridge and compute the port roles,
root, designated, or blocked. To do this, the bridges use special data frames called Bridge Protocol Data Units
(BPDUs) to exchange information about bridge IDs and root path costs. BPDUs are exchanged regularly, typically at
two second intervals, and enable switches to keep track of network topology changes and to start and stop
forwarding on ports as required. Hosts should not send BPDUs to their switch ports and to avoid malfunctioning or
malicious hosts from doing so, the switch can filter or block BPDUs. If you enable BPDU filtering on a port, BPDUs
received on that port are dropped but other traffic is forwarded as usual. If you enable BPDU blocking on a port,
BPDUs received on that port are dropped and the port is shut down.Pluribus Networks switches support the Per
VLAN Spanning Tree (PVST) variation of STP, and if a PVST BPDU is detected on a port, PVST is used on that port.
Rapid Spanning Tree Protocol is also supported by modifying an STP port and configuring it as an edge port.
Informational Note: RSTP is enabled on the switch by default.
Before you begin, view the status of STP on the switch by using the following command:
CLI network-admin@switch > stp-show
switch:
enable:
bridge-priority:
hello-time:
forwarding-delay:
max-age:
switch:
enable:
bridge-priority:
hello-time:
forwarding-delay:
max-age:
pleiades24
yes
32768
2
15
20
pleiades23
yes
32768
2
15
20
1. To disable STP, use the following command:
CLI network-admin@switch > stp-modify disable
Pluribus Networks nvOS Version 2.3.2
52
2. To display the STP state, use the following command:
CLI network-admin@switch > stp-state-show
switch:
vlan:
name:
bridge-id:
bridge-priority:
root-id:
root-priority:
root-port:
hello-time:
forwarding-delay:
max-age:
disabled:
learning:
forwarding:
discarding:
edge:
designated:
alternate:
backup:
vlag-mirror:
53
techpubs-aquila2
1
stg-default-stg
64:0e:94:18:00:8f
32769
64:0e:94:18:00:8f
32769
128
2
15
20
none
none
65-66,255
128
65-66,255
65-66,255
none
none
none
Pluribus Networks nvOS Version 2.3.2
To display information about STP on ports, use the stp-port-show command:
CLI network-admin@switch > stp-port-show
switch
-------pubdev03
pubdev03
pubdev03
pubdev03
pubdev03
pubdev03
pubdev03
pubdev03
pubdev03
pubdev02
pubdev02
pubdev02
pubdev02
pubdev02
pubdev02
pubdev02
pubdev02
pubdev01
pubdev01
pubdev01
pubdev01
pubdev01
pubdev01
pubdev01
pubdev01
pubdev01
port
---65
66
67
68
69
70
71
72
255
65
66
67
68
69
70
71
72
65
66
67
68
69
70
71
72
255
block
----off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
filter
-----off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
guard
----no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
3. To filter BPDUs on port 17, use the following command:
CLI network-admin@switch > stp-port-modify port 17 filter
4. To block BPDUs on port 17 and shut down the port if BPDUs are received on the port, use the following command:
CLI network-admin@switch > stp-port-modify port 17 block
5. To stop blocking BPDUs on port 17, use the following command:
CLI network-admin@switch > stp-port-modify port 17 no-block
6. You can disable STP on a port or a group of ports. If the devices connected to the switch ports are hosts and not
downstream switches, or you know that a loop is not possible, then disable STP and the port is enabled much faster
when the switch restarts.
7. To enable RSTP on port 35, use the following command:
CLI network-admin@switch > stp-port-modify port 35 edge
8. To enable STP, use the following command:
CLI network-admin@switch > stp-modify enable
Pluribus Networks nvOS Version 2.3.2
54
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
55
Pluribus Networks nvOS Version 2.3.2
Configuring Link Aggregation Control Protocol (LACP)
 Configuring Trunking for Link Aggregation (LAG)
 Configuring Layer 2 Multipathing for Virtual Chassis Link Aggregation
 Configuring Active-Active VLAG
 Active-Active VLAG over a Trunk with a Server-Switch and Host
Link Aggregation Control Protocol (LACP) is part of the IEEE specification 802.3ad that allows you to bundle several
physical ports to form a single logical channel. When you change the number of active bundled ports on a port
channel, traffic patterns reflect the rebalanced state of the port channel.
LACP supports the automatic creation of Gigabit Ethernet port trunks by exchanging LACP packets between ports. It
learns the capabilities of port groups and informs the other ports. Once LACP identifies correctly matched Ethernet
links, it facilitates grouping the links into a Gigabit Ethernet port trunks.
LACP packets are exchanged between ports in these modes:
 Active — Places a port into an active negotiating state, and the port initiates negotiations by sending LACP
packets.
 Passive — Places a port into a passive negotiating state where the port responds to LACP packets it receives but
does not initiate LACP negotiation. In this mode, the port channel group attaches the interface to the bundle.
 Off — LACP is not enabled on the switch port or trunk.
Active and passive modes allow LACP to negotiate between ports to determine if they can form a port channel
based on criteria such as port speed and trunking state.
To enable or disable LACP, or change the system priority, use the following command:
CLI network-admin@switch > lacp-modify enable system-priority 35000
The default system priority value is 32768 with a range from 0 to 65535.
LACP system priority can be configured on each switch running LACP. The configuration uses the default value or you
can use another value. LACP uses the system priority with the MAC address to form the system ID and also during
negotiation with other systems.
To create a trunk with LACP, use the following command:
CLI network-admin@switch > trunk-create name trunk23 port 20-36 lacp-mode
active
To modify a trunk with LACP, use the following command:
CLI network-admin@switch > trunk-modify name trunk23 lacp-mode passive
To modify a port configuration and add LACP priority to the port, use the following command:
CLI network-admin@switch > port-config-modify port 33 lacp-priority 34
LACP port priority is configured on each port using LACP. You can use the default value, 32768, or configure a specific
value from 0 to 65535. LACP uses the port priority with the port number to form the port identifier. The port priority
determines which ports should be in standby mode when there is a hardware limitation that prevents all compatible
ports from aggregating.
Pluribus Networks nvOS Version 2.3.2
56
LACP Enhancements
This feature enables ports in a static LACP trunk to operate as individual ports in the absence of proper LACP
negotiation with network peer. Once any port member hears a LACP PDU from the peer, all port members of the
trunk are bundled to operate as a trunk. This feature is useful for servers with multiple network interfaces that
would use PXE boot.
Informational Note: This feature is not supported on virtual link aggregation (vLAG) configurations.
With this configuration, nvOS creates the trunk in the switch, but does not add any of the port to the trunk. The
ports continue to operate individually until LACP PDUs are heard on any of the ports that constitute the trunk. Once
LACP PDUs are heard from the peer, then all ports of the trunk cease to operate individually and are added to the
trunk.
If no LACP PDUs are received for the number of seconds configured as the fallback timeout, nvOS LACP checks if
LACP negotiation has expired. If LACP negotiation has expired, the ports return to individual mode. If LACP
negotiation has not expired, another fallback timer is scheduled at a value equal to the fallback timeout.
Notes:
LACP fallback timeout is set to 50 seconds and LACP negotiation is set to default 90 seconds.
After 50 seconds, fallback timer is rescheduled because LACP negotiation has not expired.
After an additional 40 seconds (90 total) LACP negotiation expires and become inactive. Another 10 seconds passes
(100 seconds total) when the fallback timer expires and the ports fallback to individual.
57
Pluribus Networks nvOS Version 2.3.2
Configuring Trunking for Link Aggregation (LAG)
Informational Note: You must create unique names for each VLAG.
To configure a trunk for aggregating the links connected to ports 1, 2, 3, use the following steps:
1. Create a trunk called trunk-1 on ports 1, 2, 3, enter the following command:
CLI network-admin@switch > trunk-create name trunk-1 port 1,2,3
2. To verify the configuration, use the trunk-show command:
CLI network-admin@switch > trunk-show
name
trunk-1
port
1-3
speed
10g
autoneg
off
jumbo
off
3. Modify the trunk configuration by removing port 2:
CLI network-admin@switch > trunk-modify name trunk-1 port 1,3
4. Verify the updated trunk configuration.
CLI network-admin@switch > trunk-show
name
trunk-1
port
1,3
speed
10g
autoneg
off
jumbo
off
Notice that the ports have changed from 1-3 to 1,3 indicating that port 2 is no longer a member of the trunk
configuration.
5. Delete the trunk configuration from the switch:
CLI network-admin@switch > trunk-delete name trunk-1
Verify that the trunk configuration is removed by using the trunk-show command.
LACP Control Changes
This feature enables ports in a static LACP trunk to operate as individual ports in the absence of proper LACP
negotiation with network peer. Once any port member hears a LACP PDU from the peer, all port members of the
trunk are bundled to operate as a trunk. This feature is useful for servers with multiple network interfaces that
would use PXE boot.
Informational Note: This feature is not supported on virtual link aggregation (vLAG) configurations.
Pluribus Networks nvOS Version 2.3.2
58
With this configuration, nvOS creates the trunk in the switch, but does not add any of the port to the trunk. The
ports continue to operate individually until LACP PDUs are heard on any of the ports that constitute the trunk. Once
LACP PDUs are heard from the peer, then all ports of the trunk cease to operate individually and are added to the
trunk.
If no LACP PDUs are received for the number of seconds configured as the fallback timeout, nvOS LACP checks if
LACP negotiation has expired if LACP negotiation has expired, the ports return to individual mode. If LACP
negotiation has not expired, another fallback timer is scheduled at a value equal to the fallback timeout.
Notes
 LACP fallback timeout is set to 50 seconds and LACP negotiation is set to default 90 seconds.
 After 50 seconds, fallback timer is rescheduled because LACP negotiation has not expired.
 After an additional 40 seconds (90 total) LACP negotiation expires and become inactive. Another 10 seconds
passes (100 seconds total) when the fallback timer expires and the ports fallback to individual.
Configuring Layer 2 Multipathing for Virtual Chassis Link Aggregation
You can aggregate links between two switches by configuring Layer 2 multipathing and virtual chassis Link
Aggregation.
A virtual chassis Link Aggregation Group (VLAG) allows links that are physically connected to two different switches
to appear as a single Ethernet trunk to a third device. The third device can be a server, switch, or any other
networking device. A VLAG can create Layer 2 multipathing which allows you to create redundancy, enabling
multiple parallel paths between nodes.
A VLAG requires that a least one cross connection between the two switches, also called peers, where the VLAG links
terminate. The specific ports that connect the different switches, do not require explicit configuration before
creating a VLAG.
VLAGs can provide the following benefits:
 Allows a single device to use an Ethernet trunk across two access layer (top of rack) switches.
 Eliminates Spanning Tree Protocol (STP) blocked ports.
 Provides a loop-free topology
 Provides fast convergence if a link or device fails.
 Provides link-level resiliency.
 Helps ensure high availability.
59
Pluribus Networks nvOS Version 2.3.2
VLAG Topology Examples
Figure 1:L2 Design - Leaf and Spine with Active-Passive VLAG
Figure 2:L2 Design - Leaf and Spine with Active-Active VLAG
Pluribus Networks nvOS Version 2.3.2
60
Figure 3:L2 Design - Leaf and Third Party Spine without Multichassis LAG or VPC Mode
Figure 4:L2 Design - Leaf and Third Party Spine with Multichassis LAG, vPC and MLAG
To create a VLAG for aggregating links connected to ports 70 on the local switch and the peer called, eng-switch-b,
you must first create a cluster configuration between the two switches. Pluribus Networks switches must be
members of a cluster configuration before you can add VLAGs to them.
Third Party Interoperability with nvOS
61
Operating System
Host
PN Switch
SmartOS, OpenSolaris,
Illuminos, Oracle Solaris
Create aggr with
lacp-mode passive.
Create lacp-mode active
ad lacp-timeout fast.
Red Hat, Linux
Create bond with
mode 3.
Create lacp-mode off.
Pluribus Networks nvOS Version 2.3.2
Operating System
Host
PN Switch
CentOS
Create bond with
mode 4.
Create lacp-mode on.
Configuring Active-Active VLAG
Using the sample topology in Figure 5 Active-Active VLAG over a Trunk with a Server-Switch and Host, use the
following steps to configure Active-Active VLAG:
Informational Note: There must be a physical connection between PN-0 and PN-1 before you can configure VLAG.
Figure 5:Active-Active VLAG over a Trunk with a Server-Switch and Host
Three Pluribus Networks switches in a common fabric with the Spine switch as the RSTP root. It is important to note
that ports 19-22 on PN-0 and PN-1 are ports connected to PN-2 (Spine). Port 26 connects PN-0 to PN-1 for the
cluster configuration required for VLAG.
1. On PN-2, use the following command:
CLI network-admin@switch > stp-modify bridge-priority 4096
2. Create the fabric and add the switches:
Pluribus Networks nvOS Version 2.3.2
62
On PN-2, use the fabric-create command:
CLI network-admin@switch > fabric-create name fab-vlag
On PN-1, join the fabric:
CLI network-admin@switch > fabric-join name fab-vlag
On PN-0, join the fabric:
CLI network-admin@switch > fabric-join name fab-vlag
3. Create VLAN connectivity from the top switch to the bottom:
On PN-2, create the VLAN with scope fabric:
CLI network-admin@switch > vlan-create id 25 scope fabric
On PN-0, add the VLAN and untag the port connected to the host.
CLI network-admin@switch > vlan-port-add vlan-id 25 untagged ports 9
On PN-1, add the VLAN and untag the port connected to the host.
CLI network-admin@switch > vlan-port-add vlan-id 25 untagged ports 9
On PN-0, modify the host STP port to be an edge port.
CLI network-admin@switch > stp-port-modify port 9 edge
On PN-1, modify the host STP port to be an edge port.
CLI network-admin@switch > stp-port-modify port 9 edge
4. Create a cluster configuration between PN-1 and PN-0. This creates the cluster across port 26.
On PN-0, enter the cluster-create command:
CLI network-admin@switch > cluster-create name vlag cluster-node-1 PN-0
cluster-node-2 PN-1
5. You must disable ports between PN-2 and PN-0, and then create a static trunk between them:
On PN-0, modify the ports facing PN-2:
CLI network-admin@switch > port-config-modify port 19,20 disable
63
Pluribus Networks nvOS Version 2.3.2
Then create the trunk on PN-0:
CLI network-admin@switch > trunk-create name pn0-to-pn2 port 19,20 lacp-mode
off
CLI network-admin@switch > trunk-show format all layout vertical
switch:
intf:
name:
port:
speed:
autoneg:
jumbo:
enable:
lacp-mode:
lacp-priority:
lacp-timeout:
reflect:
edge-switch:
pause:
description:
loopback:
mirror-only:
unknown-ucast-level:
unknown-mcast-level:
broadcast-level:
lport:
rswitch-default-vlan:
port-mac-address:
status:
config:
send-port:
PN-0
128
pn0-to-pn2
19-20
10g
off
off
off
off
32768
slow
off
no
no
off
off
100%
100%
100%
0
0
06:60:00:02:10:80
0
From the above output, you can find the name of the trunk configuration, pn0-to-pn2. You need this information
to create the VLAG.
Then, on PN-1, repeat the same commands to create a trunk between PN-1 and PN-2.
6. You must disable ports between PN-2 and PN-1, and then create a static trunk between them:
On PN-1, modify the ports facing PN-2:
Pluribus Networks nvOS Version 2.3.2
64
port-config-modify port 21,22 disable
CLI network-admin@switch > trunk-create name pn1-to-pn2 port 21,22 lacp-mode
off
CLI network-admin@switch > trunk-show format all layout vertical
switch:
intf:
name:
port:
speed:
autoneg:
jumbo:
enable:
lacp-mode:
lacp-priority:
lacp-timeout:
reflect:
edge-switch:
pause:
description:
loopback:
mirror-only:
lport:
rswitch-default-vlan:
port-mac-address:
status:
config:
send-port:
0
PN-0
129
pn1-to-pn2
21-22
10g
off
off
off
off
32768
slow
off
no
no
off
off
0
0
06:60:00:02:10:80
7. Now create the VLAG from the bottom switches going upward and static trunk from the top down. Keep one side
of the VLAG disabled while you configure this step.
On PN-0, use the vlag-create command:
CLI network-admin@switch > vlag-create name to-spine port 128 peer-port 129
peer-switch PN-1 lacp-mode off mode active-active
On PN-2, create a trunk with the name trunk-pn:
CLI network-admin@switch > trunk-create name trunk-pn port 19,20,21,22
lacp-mode off
8. Now, you can enable ports on all switches:
On PN-2, enter the port-config-modify command:
CLI network-admin@switch > port-config-modify port 19,20,21,22 enable
On PN-0, enter the port-config-modify command:
CLI network-admin@switch > port-config-modify port 19,20 enable
On PN-1, enter the port-config-modify command:
CLI network-admin@switch > port-config-modify port 21,22 enable
65
Pluribus Networks nvOS Version 2.3.2
9. Create the server-facing VLAG:
On PN-0, enter the vlag-create command:
CLI network-admin@switch > vlag-create name to-spine port 9 peer-port 9
peer-switch PN-1 lacp-mode active mode active-active
Display the VLAG configuration information:
CLI network-admin@switch > vlag-show format all layout vertical
id:
name:
cluster:
mode:
switch:
port:
peer-switch:
peer-port:
failover-move-L2:
status:
local-state:
lacp-mode:
lacp-timeout:
lacp-key:
lacp-system-id:
a000024:0
to-spine
vlag
active-active
pubdev02
trunk2
pubdev01
129
no
normal
enabled,up
off
slow
26460
110013777969246
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks nvOS Version 2.3.2
66
Configuring Tagged and Untagged VLANs
Creating untagged VLANs is useful for connecting the switch to devices that do not support IEEE 802.1Q VLAN tags.
You can configure ports to map untagged packets to a VLAN.
Reserved VLANs and VLAN 0 and 1
The VLAN identifier is a 12-bit field in the header of each packet. Therefore, the maximum number of VLANs you can
define is 4096. Pluribus Networks switches reserve VLANs 0, 1, 4093, 4094, and 4095 for internal use.VLAN 0 is not a
standard VLAN in nvOS. It is used to represent all untagged or non-VLAN traffic. VLAN 1 is the default untagged
traffic VLAN. Untagged traffic can be mapped to any VLAN, but by default, it is mapped to VLAN 1.
It’s important to note that if you create a VLAN with scope fabric and untag all ports, you can cause problems
with the fabric communication.
Informational Note: The untagged VLAN feature is not the same as the default VLAN using the IEEE
802.1Q tag 1.
1. To create a VLAN on the current switch, with the identifier 595, use the following command:
CLI network-admin@switch > vlan-create name VLAN595 id 595 scope local
By default, all ports are trunked on the new VLAN. If you want to specify ports that are trunked, use the optional
parameter, ports, with a comma separated list of ports, or specify a range of ports.
In some cases, you may not want the VLAN created on all ports. You can specify none to apply the VLAN to
internal ports only.
CLI network-admin@switch > vlan-create id 35 scope fabric ports none
CLI network-admin@switch > vlan-show
switch:
id:
nvid:
scope:
name:
active:
stats:
vrg:
ports:
untagged-ports:
active-edge-ports:
switch:
pubdev01
35
a000030:23
fabric
vlan-35
yes
yes
0:0
65-72,255
none
none
pubdev02
To map ports on different switches into the scope fabric VLAN, use the following command:
CLI network-admin@switch > vlan-port-add switch switch-name ports
Pluribus Networks Configuration Guide
67
www.pluribusnetworks.com
To modify a VLAN name, use the vlan-modify command to modify VLAN 25 name from blue to red:
CLI network-admin@switch > vlan-modify id 25 name blue
To modify the port list, use the vlan-port-add and the vlan-port-remove commands.
2. To display the VLANs configured on the switch, use the vlan-show command.
CLI network-admin@switch > vlan-show format all layout vertical
switch:
id:
nvid:
scope:
name:
active:
stats:
vrg:
ports:
untagged-ports:
active-edge-ports:
active-edge-ports:
switch:
id:
nvid:
scope:
name:
active:
stats:
vrg:
ports:
untagged-ports:
pubdev01
1
a000030:1
local
default-1
yes
yes
0:0
1-72,128,255
1-72,128,255
31,45-46,66,128
65,128-129
pubdev02
1
a000024:1
local
default-1
yes
yes
0:0
1-72,128-129,255
1-72,128-129,255
3. To configure ports 17 and 18 to accept untagged packets and map them to VLAN 595, use the following command:
CLI network-admin@switch > vlan-port-add vlan-id 595 ports 17,18 untagged
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
68
Displaying VLAN Statistics
You can display network traffic statistics per VLAN using the vlan-stats-show command. This may be useful
when troubleshooting network issues.
CLI network-admin@switch > vlan-stats-show format all layout vertical
switch:
time:
vlan:
ibytes:
ipkts:
idrops-bytes:
idrops-pkts:
obytes:
opkts:
odrops-bytes:
odrops-pkts:
switch:
time:
vlan:
ibytes:
ipkts:
idrops-bytes:
idrops-pkts:
obytes:
opkts:
odrops-bytes:
odrops-pkts:
switch:
time:
vlan:
ibytes:
ipkts:
idrops-bytes:
idrops-pkts:
obytes:
opkts:
odrops-bytes:
odrops-pkts:
pubdev03
10:51:02
1
36.2T
89.0G
119M
313K
0
0
0
0
pubdev03
10:51:02
35
10.8K
154
0
0
0
0
0
0
pubdev02
10:51:02
1
34.9T
84.6G
3.03M
5.69K
0
0
0
0
The output displays the following information:
 switch
 time
 VLAN ID
 incoming and outgoing bytes
 incoming and outgoing packets
 incoming and outgoing dropped bytes
incoming and outgoing dropped packetsTopic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
69
www.pluribusnetworks.com
Implementing Virtual Networks
 Overview
 Using VNETs with nvOS
 Creating a Virtual Network
 Adding DHCP Service to a VNET
 Verify Administrator User Creation
 Configuring Administration Login Using SSH
 Adding a Default Gateway to the VNET
 Adding Ports to the VNET
 Configuring Virtual Resource Groups
Overview
A Virtual Network (VNET) is an abstract network resource realized across a fabric of Pluribus Networks switches.
Using VNETs, you can segregate a physical fabric into many logical networks, each with its own resources, network
services, and Quality of Service (QoS) guarantees. A VNET allows you to completely separate all traffic in one VNET
from the traffic of other VNETs.
Figure 1:Using VNETs with nvOS
Pluribus Networks nvOS Version 2.3.2
70
Each VNET has a single point of management. As the fabric administrator, you can create VNETs and assign
ownership of each VNET to individuals with responsibility for managing those resources. You can create separate
usernames and passwords for each VNET manager. Using the separate VNET administration credentials, the VNET
admin can use Secure Shell (SSH) to connect to the VNET manager and access a subset of the nvOS® CLI commands
to manage that VNET. This way, multiple tenants can share a fabric with each managing a VNET with security, traffic,
and resource protection from other VNETs.
VNETs are very flexible and can be used to create complex network architectures. For example, a Pluribus Networks
switch, or a fabric of switches, can be used to create multiple tenant environments in an OpenStack deployment. In
Figure 1 Using VNETs with nvOS, there are three VNETs, each with a management interface and a data interface.
Each VNET is assigned an IP address pool used for DHCP assignment of IP addresses to each node, server, or OS
component.
Underlying each VNET is the VNET manager. Each VNET manager runs in an OpenSolaris zone. When services are
created for a VNET they occupy the same zone on a server-switch. This is called a shared service and it is the default
when creating services. However, each zone can only support a single instance of a service. If a second service
instance is needed for a VNET, then it needs to occupy a separate zone. This is called a dedicated service. In most
cases, you can create services as shared unless you specifically want to create a dedicated service.
When a fabric is created, a VNET is automatically created and named fabric-name-global. This VNET owns all
resources within the fabric, and as new VNETs are created, resources are moved from the default VNET to the new
VNETs. Global services remain in the default VNET unless assigned specifically to a VNET. The software license for IPS
allows only the global VNET, but you can use it to create DHCP servers and other services for the entire switch.
Specifying the Type of VNET Interface
The mgmt, data, and span keywords used in different commands specify the path used to connect to the network
service. For example, to specify an out-of-band connection to a management interface of a VNET, the interface is
specified using the mgmt keyword. If in-band access to that management interface of the VNET is required, then the
data or span keywords are used in the specific command. The keywords, data and span, are essentially
equivalent but apply to two separate paths. To maximize throughput between the server and the switch
components, it is recommended to use both. The data keyword applies to port 65, and the span keyword applies
to port 66.
Each VNET can have one or more isolating zones and network services are applied to each zone. Network services
have their own zone or share the zone with the VNET manager which is the zone that the VNET user logs into to
manage the VNET. In shared zones, the network interfaces are available to all network services in the shared zones,
regardless of the service that created the network interface.
Informational Note: This is an important concept as you can use service commands such as vlb-interface-add to
add an interface or you can use vnet-manager-interface-add to add interfaces to a VNET. If you want the
service to be specific to a VNET as a dedicated service, then add the interfaces using the service-interface-add
commands.
71
Pluribus Networks nvOS Version 2.3.2
Creating a Virtual Network (VNET)
To separate resources, including switch ports, IP addresses, VLANs, and VXLANs, into separate management spaces,
create a VNET and place the resources in the VNET. Then configure a separate VNET admin to manage the network.
Informational Note: You cannot create another VNET inside of a VNET.
There is no performance impact when you send network traffic through a VNET. Packets are switched in the
hardware with full line-rate bandwidth and the same latency even if the packets are on a VNET or not. But, the VNET
allows you to provide different Service Level Agreements (SLAs) to each VNET when there are multiple VNETs on a
physical switch and there is resource contention based on traffic loads.
Related Tasks
 Creating a Virtual Network
 Configuring Virtual Resource Groups
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks nvOS Version 2.3.2
72
Creating a Virtual Network
To separate resources, including switch ports, IP addresses, VLANs, and VXLANs, into separate management spaces,
create a VNET and add those resources to the VNET. Then configure a separate administrator for that VNET.
To create a VNET named vnet1 with VLANs, 125 to 130, and a scope of fabric, use the following command:
CLI network-admin@switch > vnet-create name vnet1 scope fabric vlans 123-130
Vnet created.
To confirm that the VLAN is created, use the vnet-show command:
CLI network-admin@switch > vnet-show layout vertical
switch:
name:
scope:
vlans:
managed-ports:
admin:
vnet-mgr-name:
switch:
name:
scope:
vlans:
managed ports:
admin:
vnet-mgr-name:
antares10
vnet1
fabric
125-130
none
vnet1-admin
vnet1-mgr
antares15
vnet2
fabric
131-135
none
vnet2-admin
vnet2-mgr
When you add VLANs to a VNET, you can either assign a range of VLANs, such as 100-199, or a number of VLANs,
such as 5, which then assigns 5 VLANs from nvOS, starting with the lowest number of the available VLANs. You can
see the difference by using the num-vlans parameter to assign VLANs:
CLI network-admin@switch > vnet-create name tester-1 scope fabric num-vlans 3
CLI network-admin@switch > vnet-show name tester-1 layout vertical
switch:
antares10
name:
vnet1
scope:
fabric
vrg:
vnet1-vrg
num-vlans:
3
vlans:
5-7
managed-ports:
none
admin:
vnet1-admin
vnet-mgr-name:
vnet1-mgr
switch:
antares15
name:
vnet2
scope:
fabric
vlans:
123-130
managed ports:
none
admin:
vnet2-admin
vnet-mgr-name:
vnet2-mgr
All switches in the fabric are now in this VNET.
Pluribus Networks Configuration Guide
73
www.pluribusnetworks.com
Each VNET is associated with a VNET manager (VNM). The default VNM appends the suffix “mgr” to the name
created for the VNET. If you want to create a different name, use the vnet-mgr-option when creating a VNET.
The VNM represents the management interface to the VNET. You can log into the VNM in the same way you can log
into the management plane of the overall logical switch. In multi-tenant environments, access to the VNM is
typically provided to individual VNET administrators such as cloud tenants or application managers. This way the
VNET administrators can manage the configurations and properties of their VNETs. .
Informational Note: Command Execution Time
Some commands may take a few seconds to complete since there are multiple steps in the
commands.
Informational Note: Storage Pool Use
Use the vnet-create command option vnet-mgr-storage-pool to place the VNET
into a storage pool other than the default storage pool.
Adding Untagged VLANs to a VNET
To add untagged VLANs to a VNET, use the vlan-port-add command:
CLI network-admin@switch > vlan-port-add vlan-id 311 ports 15-20 untagged
Adding DHCP Service to a VNET
To add a pool of IP addresses used by a DHCP service, create the IP pool first. For example, you can create the IP
Pool, dhcp-pool, and addresses in the 172.16.23.0/24 network:
CLI network-admin@switch > ip-pool-create name dhcp-pool vnet vnet1 start-ip
172.16.23.0 end-ip 172.16.23.254 netmask 24
Then create the DHCP service:
CLI network-admin@switch > dhcp-create name dhcp-vnet1 vnet vnet1
initial-ip-pool dhcp-pool
The final step is creating the gateway for the DHCP service:
CLI network-admin@switch > dhcp-pool-modify dhcp-name dhcp-vnet1
dhcp-pool-name dhcp-pool gateway-ip 172.16.23.1
Now when you add Virtual Machines (VMs) such as Ubuntu 11.04 or CentOS 6.5, the interfaces receive IP addresses
from the DHCP service assigned to the VNET.
Informational Note: You can only run one instance of a DHCP service per VNET.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
74
Verify Administrator User Creation
When a VNET is created, an administrator for that VNET is automatically created in addition to the VNET manager. In
this example, the VNET, vnet1, is created, and the user vnet1-admin is created. The keyword, admin, is
appended to the name of the VNET. This is the default value, so if you want to create an administrator with a
different name, use the vnet-create admin option. vnet1-admin and the superuser, network-admin can
log into the VNET and manage it.
To confirm that the user was created, use the user-show command:
CLI network-admin@switch > user-show
name
vnet1-admin
scope
fabric
uid
20001
Use the user-modify command to change the password for the VNET administrator. The default password is the
same as the account name, vnet1-admin, in this example.
CLI network-admin@switch > user-modify name vnet1-admin
password:********
confirm password:*********
CAUTION!
It is not recommended to change the initial role for a VNET administrator. User roles have different implications and allow
access to the entire switch instead of just the VNET.
Configuring Administration Login Using SSH
In order for the vnet1-admin to login and administer the VNET using SSH, you must add an IP address on either the
switch data port or the mgmt interface. You cannot access the VNET through the management IP address of the
switch. To add the IP address, use the following command:
CLI network-admin@switch > vnet-manager-interface-add vnet-manager-name
vnet1-mgr if data ip 10.100.1.1/24
If you do not specify a VLAN, the interface is added, by default, to the lowest numbered VLAN in the VNET. To verify
that the interface was added, use the vnet-manager-interface-show command:
CLI network-admin@switch > vnet-manager-interface-show vnet-manager-name
vnet1-mgr layout vertical
vnet-manager-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
to_vnic_flow_name:
vnet1-mgr
vnet1.mgr.eth0
10.100.1.1/24
static
66:0e:94:4b:68:96
123
0
data
Pluribus Networks Configuration Guide
75
www.pluribusnetworks.com
Now you can SSH to the VNET, using the following syntax:
ssh [email protected]
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
76
Once you log into the VNET, you are placed directly into the CLI for nvOS. The following commands are available to a
VNET administrator:
acl-ip
acl-mac
client-server-stats
connection
connection-latency
connection-stats
dhcp
dhcp-lease
disk-library
dns
fabric
fabric-node
fabric-stats
igmp
igmp-static-group
igmp-static-source
interface-stats
ip-pool
iso-library
l2-history
l2-table
lldp
log-audit
log-event
log-system-counters
log-system
mcast
nat
netvisor-kvm
netvisor-vmm
netvisor-zone
openflow
openstack
openstack-plugin
ping
port-config
port
port-stats
port-vlan
role
running-config-show
sflow
software-license
software
ssh
ssh-known-hosts-delete
storage-folder
storage-pool
stp-port-event
stp-state
tech-support-show
user
vflow
vflow-share
Pluribus Networks Configuration Guide
77
www.pluribusnetworks.com
vflow-stats
vlan
vlan-stats
vlb
vnet-manager
vnet-service
vnet
vrouter
vrouter-cached-routes
pager
switch
help
quit
exit
Once you are logged into the VNET, you can add VMs or other features to it. For instance, you can install CentOS and
run applications on it or add Ubuntu servers to the VNET.
To remove an interface from the VNET manager, use the vnet-manager-interface-remove command.
Adding a Default Gateway to the VNET
Use the vnet-manager-modify command to add the gateway, 10.100.1.254 to the configuration.
CLI network-admin@switch > vnet-manager-modify name vnet1-mgr gateway
10.100.1.254
To verify the configuration, use the vnet-manager-show command:
CLI network-admin@switch > vnet-manager-show name vnet1-mgr layout vertical
name:
type:
scope:
vnet:
vnet-service:
state:
gateway:
vnet1mgr
vnet-mgr
fabric
vnet1
shared
enabled
10.100.1.254
Modifying and Displaying VNET Manager Services
You can modify the services on the VNET manager using the vnet-manager-service-modify command. If,
for example, you want to disable Web access to the interface, use the following syntax:
CLI network-admin@switch > vnet-manager-services-modify name pn-lab-vnet-mgr
if pn.lab.vnet.mgr.eth0 no-web
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
78
To display information about the VNET services, use the vnet-services-show command:
CLI (server-switch)>vnet-service-show layout vertical
name:
type:
scope:
vnet:
vnet-service:
state:
gateway:
name:
type:
scope:
vnet:
vnet-service:
state:
gateway:
pn-dhcp-dns
dhcp
fabric
pn-fab-global
shared
enabled
10.9.9.1
lab-dhcp
dhcp
fabric
pn-lab-vnet
shared
enabled
::
To display information about VNET Manager services, use the vnet-manager-service-show command:
CLI network-admin@switch > vnet-manager-service-show layout vertical
vnet-manager-name:
if:
ssh:
web:
web-ssl:
web-ssl-port:
web-port:
icmp:
vnet-manager-name:
if:
ssh:
web:
web-ssl:
web-ssl-port:
web-port:
icmp:
pn-lab-vnet-mgr
pn.lab.vnet.mgr.eth0
on
on
off
443
80
on
pn-lab-vnet-mgr
pn.lab.vnet.mgr.eth1
on
on
off
443
80
on
Adding Ports to the VNET
Ports can be managed by the VNET, but the VNET does not have absolute control over the port. Untagged traffic on
the port can be tagged to a VLAN that is assigned to the VNET. In most cases, it is not necessary to add a port to the
VNET.
Now, add ports, 5-8, 20-30, to the VNET on the local switch and a remote switch.
CLI network-admin@switch > vnet-port-add vnet-name vnet1 ports 5-8,20-30
CLI network-admin@switch > switch antares15 vnet-port-add vnet-name vnet1 ports
20-50
ports added.
Pluribus Networks Configuration Guide
79
www.pluribusnetworks.com
To verify the ports, use the vnet-show command:
CLI network-admin@switch > vnet-show name vnet1 layout vertical
switch:
name:
scope:
vlans:
managed-ports:
admin:
vnet-mgr-name:
switch:
name:
scope:
vlans:
managed-ports:
admin:
antares15
vnet1
fabric
123-130
5-8,20-30
vnet1-admin
vnet1-mgr
pleiades15
vnet1
fabric
123-130
5-8,20-30
vnet1-admin
Adding a vRouter to the VNET
If you have a VLAN 10 with a subnet 192.168.10.0/24 and a VLAN 12 with a subnet 192.168.12.0/24 on the same
VNET, net-resources, and you want to route traffic between the two VLANs, use the following steps:
1. Create the VNET.
CLI network-admin@switch > create-vnet name net-resources scope local vlans
10,12
2. Create VLAN 10.
CLI network-admin@switch > vlan-create id 10 scope local ports 10
untagged-ports 10
3. Create VLAN 12.
CLI network-admin@switch > vlan-create id 12 scope local ports 12
untagged-ports 12
4. Create the vRouter, subnets.
CLI network-admin@switch > vrouter-create name subnets vnet net-resources
enable
5. Add a vRouter interface for VLAN 10.
CLI network-admin@switch > vrouter-interface-add vrouter-name subnets ip
192.168.10.254 netmask 255.255.255.0 vlan 10
6. Add a vRouter interface for VLAN 12.
CLI network-admin@switch > vrouter-interface-add vrouter-name subnets ip
192.168.12.254 netmask 255.255.255.0 vlan 12
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
80
To view the configuration, use the vrouter-interface-show command:
CLI network-admin@switch > vrouter-interface-show layout vertical
switch:
vrouter-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
switch:
vrouter-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
pleiades24
subnets
net-resources.mgr.eth1
192.168.10.254/24
static
66:0e:94:24:34:31
10
0
data
pleiades24
subnets
net-resources.mgr.eth2
192.168.12.254/24
static
66:0e:94:24:f8:s9
12
0
data
Informational Note: Network Services Locations and Migration
All network services, such as VNET managers, DHCP servers, and virtual load balancers,
consume disk space, CPU, and memory on one of the switches in a fabric. There may be
instances when you need to move a service, for example, when a disk space shortage occurs,
or you replace a switch. The migrate commands, such as vnet-manager-migrate,
provide the ability to move the service to a different disk pool if you specify the
storage-pool option, or to a different switch within the fabric, if the location option is
specified.
You cannot migrate NetVMs and NetZones. Instead, you export and import them from the
configuration using the commands iso-image-library-export and
disk-library-image-export.
To complete the VNET configuration, you can assign a Virtual Resource Group (VRGs) to the VNET. VRGs allow you
allocate resources to each VNET so that a single VNET does not consume all of the resources on a switch. See
Configuring Virtual Resource Groups.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
81
www.pluribusnetworks.com
Configuring Virtual Resource Groups
After creating a VNET, a corresponding Virtual Resource Group (VRG) is created. You can configure VRGs to limit the
resources assigned to a VNET so that a single VNET cannot monopolize all of the resources of the fabric. The VRG can
be modified to limit the specific resources allocated to a VNET.
To create a VRG, use the following command:
CLI network-admin@switch > vrg-create name vnet1-vrg scope fabric num-vlans 8
vlans 123-150
To check the status of a VRG, use the vrg-show command:
CLI network-admin@switch > vrg-show name vnet1-vrg layout vertical
switch:
name:
scope:
num-vlans:
vlans:
ports:
num-flows:
rack-bw-limit (Mbps):
rack-bw(Mbps):
storage-bw(Mbps):
dc-bw(Mbps):
wan-bw(Mbps):
traffic-class:
priority:
restricted resources:
antares15
vnet1-vrg
fabric
8
123-130
0
0
0
0
0
0
0
0
If you want to limit the data bandwidth to 400 Mbps for the VNET, you can modify the VRG:
CLI network-admin@switch > vrg-modify name vnet1-vrg data-bw 400m
CLI network-admin@switch > vrg-show name vnet1-vrg layout vertical
switch:
name:
scope:
num-vlans:
vlans:
ports:
num-flows:
data-bw:
storage-bw:
service-bs:
restricted resources:
antares15
vnet1-vrg
fabric
8
123-130
None
0
400
0
0
data-bw
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
82
And finally, you want to assign the VRG to a VNET so the resource limitations apply to the VNET:
CLI network-admin@switch > vnet-modify name vnet1 vrg vnet1-vrg
vnet-show name vnet1 format all layout vertical
switch:
antares15
id:
a1634:0
name:
vnet1
scope:
fabric
vrg:
vnet1-vrg
num-vlans:
1
vlans:
150
managed-ports:
admin:
vnet1-admin
vnet-mgr-name:
vnet1-mgr
switch:
antares16
id:
a1635:0
name:
vnet1
scope:
fabric
vrg:
vnet1-vrg
num-vlans:
1
vlans:
150
managed-ports:
admin:
vnet1-admin
vnet-mgr-name:
vnet1-mgr
Timesaver: If the VRG is created before you assign it to a VNET, you can save a step by specifying
the VRG when the VNET is created.
About Virtual Resource Group (VRG) Bandwidth Enforcement
The resources available in a fabric of nvOS devices can be managed by allocating them to Virtual Resource Groups
(VRGs). Each VRG can include an allocation of VLANs and a guarantee of a minimum network bandwidth. VNETs are
then assigned to a VRG. The VNET can also include VLANs as well as other services and resources.
In this implementation, each VRG is assigned a Guaranteed Bandwidth (GBW) parameter specified in Mbps. To
enforce the GBW allocation, all network traffic associated with the VRG is sent to the Networking Processor Unit
(NPU). Flows running on VLANs associated with a VRG is assigned a portion of the GBW assigned to the VRG.
This version has the following limitations:
 Bandwidth guarantees for services and data are supported.
 Storage bandwidth guarantees are not supported.
 Available bandwidth is not enforced per VNET when there are multiple VNETs assigned to the same VRG. Only
VRGs and vFlows are allowed specified guaranteed bandwidth.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
83
www.pluribusnetworks.com
Understanding Virtual Ports
vPorts are basically Layer 2 (L2) entries that display a port that learns a MAC address on the switch. The hardware L2
table is limited by the switch chip memory and the nvOS software allows the tracking of a much larger L2 table that
displays vPort (L2) entries active on the switch. When a host tries to send a MAC address that isn’t in the hardware,
nvOS can forward the information to the hardware from the software table.
vPorts are also mirrored across the fabric so that every switch in the fabric is aware of every other L2 table entries.
The history of each vPort is tracked and using the vport-history-show displays how a vPort moved between
different ports over time. This is useful to track virtual machine (VM) migration. Also, the L2 table is the basis of all
L2 switching so you can analyze vPort information for any information about L2 switching activity.
Previously, nvOS tracked MAC addresses, IP addresses, and hostname for each endpoint attached to a port. This
information is not persistent and is displayed in the port-show output. Now, you can track this information using
vPort commands. vPorts has the scope of fabric meaning that vPorts are tracked fabric-wide.
vPort information is persistent and logged by nvOS so that you can query on active vPorts and attributes associated
with them during a specified time range. Each vPort has associated analytics including port counters. vPort
information is also stored in a log file and rotates when the log is full. Each log file starts with erasing the vPort table
so that the log file can be recovered after restart.
Information about active and inactive vPorts can be displayed using the vport-show command:
CLI network-admin@switch > vport-show format ip,mac,hostname,vlan,last-active
ip
---------192.168.1.3
192.168.1.6
192.168.1.9
mac
vlan
----------------- ---52:54:00:58:35:5f
7
12:5c:19:69:25:30 123
d6:f9:8a:29:25:44
42
hostname
-------db-serv1
db-serv2
db-serv1
last-active
----------2014-08-07,12:25:11
now
2014-08-07,12:25:11
An inactive port can be deleted:
CLI (server-switch)>vport-delete mac 52:54:00:58:35:5f vlan 7
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
84
To view historic information about hosts attached to the network, you can query the vPort history to display the
vPort state during a specified time interval. This provides assistance when troubleshooting a problem or auditing the
network.
CLI network-admin@switch > vport-history-show start-time 2014-08-17T12:35
end-time 2014-08-17T13:25
time:
log-type:
switch-id:
mac:
vlan:
ip:
local-intf:
ports:
state:
hostname:
status:
create-time:
last-seen:
hit:
time:
log-type:
switch-id:
mac:
vlan:
ip:
local-intf:
ports:
state:
hostname:
status:
create-time:
last-seen:
hit:
time:
log-type:
switch-id:
mac:
vlan:
local-intf:
ports:
create-time:
last-seen:
hit:
08-17,12:38:03
l2-modify
pleiades24
66:0e:94:23:38:64
121
182.18.0.200
65
65
active
pleiades24
PN-internal,dhcpsvr
08-12,15:38:33
08-17,12:38:03
1
08-17,12:38:03
l2-modify
pleiades24
66:0e:94:23:38:64
121
182.18.0.2
65
65
active
pleiades24
PN-internal,dhcpsvr
08-12,15:38:33
08-17,12:38:03
1
08-17,12:38:09
l2-modify
pleiades24
fa:16:3e:c5:83:56
110
56
56
08-08,01:02:53
08-17,12:29:07
1363
CLI network-admin@switch > vport-history-show format all layout vertical
CLI (network-admin@aquila13) > vport-history-show format all layout vertical
time:
log-type:
switch-id:
mac:
vlan:
ip:
intf:
ports:
07:28:27
l2-modify
pubdev03
64:0e:94:28:03:56
1
192.168.42.30
65
65
Pluribus Networks Configuration Guide
85
www.pluribusnetworks.com
state:
local-intf:
local-ports:
local-state:
hostname:
status:
create-time:
last-seen:
hit:
migrate:
drops:
time:
log-type:
active
128
47-48
active
pubdev03
PN-internal
01-21,11:58:34
07:28:27
47271
6892037
6407378
07:28:42
l2-modify
The output has the following meanings:
 time — the time of the event
 log-type — there several different log-types:
• cfg-create — the command vport-create issued.
• cdg-modify — the command vport-modify issued.
• cfg-delete — the command vport-delete issued.
• l2-modify — the vPort changed because of network activity such as MAC learning.
• k2-delete — the vPort is removed because of network activity such as the vPort table is full.
• save — vPort storage rotated so that all vPorts are saved in a new vPort storage file.
 switch-id — the ID assigned to the switch.
 mac — the MAC address assigned to the vPort.
 vlan — the VLAN assigned to the vPort.
 ip — IP address of the switch.
 intf — the interface of the vPort. If the local interface is a trunk, the trunk ports are displayed.
 ports — the ports assigned to the vPort.
 state — the current state of the vPort as active or inactive.
 local-intf — the local interface for vPorts.
 local-ports — the local ports assigned to vPorts.
 local
state — the local state of vPorts.
 hostname — hostname assigned to the vPorts.
 status — internal or external status.
 create-time — the time that the event was created.
 last-seen — the time that the event last occurred.
 hit — the number of times that the vPort has been activated in the MAC table.
 migrate — number of times that the vPort migrated on Layer 2.
 drops — number of times that the packets were dropped on Layer 2.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
86
Displaying Information about VMs and vPorts
You can display the following information about VMs and vPorts using vport-show command and VM limiters:
CLI network-admin@switch > vport-show format ip,vnet,vm-name,vm-flavor,cpu,
memory,disk
ip
------------192.168.3.57
192.168.3.58
vnet
---vn-db
vn-db
vm-name
------db1
db2
vm-flavor
--------m.xlarge
m.large
cpu
--8
4
memory
-----16G
8G
disk
---160G
80G
vPort Enhancements
vPort has the following enhancements:
L2 and L3 table changes logged to event log
L2 and L3 table changes sent as events to API clients that subscribe to vPort events
L2 and L3 table changes logged to nvOs log file consistently, including information about file, line, and
function where the change is made.
L2 and L3 logs include caller, reason, and a mask of what changed. This information is recorded in the
event.log, and sent to event clients.
vport-history-show displays historical information for both L2 and L3 entries from a fabric
perspective. Local information is no longer included in vport-history-show output
l2-history-show displays historical information for L2 entries.
l3-history-show
displays historical information for L3 entries.
In vport-show, and vport-history-show output, the parameter, switch-id, has changed to the
parameter, owner.
For the REST API, vport_switch_id is now vport_owner.
The vport-history-show output has new fields:
caller — indicates the module that made the vPort change.
reason — why the change was made to the entry.
Pluribus Networks Configuration Guide
87
www.pluribusnetworks.com
Configuring Network Services - DHCP and DNS
 Overview of DHCP and DNS
 Configuring IP Pools
 Configuring DHCP Services
 Adding DHCP Interfaces
 Adding DHCP and DNS Records
 Removing DHCP and DNS Services
 Configuring DNS Services
 Creating a DNS Server
 Configuring Network Address Translation Services
 Configuring Hardware-based Network Address Translation(NAT)
Overview of DHCP and DNS
In general, network services are associated with a VNET. When a fabric is created, a global VNET is also created and
should be used if the network service is available to all Server-Switches and all nodes on the network. Select a
specific VNET if the network service applies to a single VNET, limited to the VNETs resources, and is managed by the
VNET manager. Another option is to decide if the network service is applicable to the same logical zone as the VNET
(shared) or applicable to another separate zone (dedicated). For example, the zone on the VNET may already have a
service running, and another instance of the service is needed to avoid a conflict on the network. In the dedicated
instance, the VNET and the dedicated zone must be configured to see the same network traffic, for example, on the
same VLAN.
This topic describes configuring two virtual services, DNS and DHCP.
Related Tasks
 Configuring IP Pools
 Configuring DHCP Services
 Adding DHCP and DNS Records
 Removing DHCP and DNS Services
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
88
Configuring IP Pools
IP addresses are resources managed as pools. An IP address pool must be associated with a VNET, because a service
associated with the IP address pool, and the supported service must reside in a VNET. The VNET can be the default
fabric VNET created when the fabric is first created, and if this is the case, the IP address pool or pools are available
fabric-wide and have no resource limitations. If you want to assign restrictions to the IP pool, for example, assign it
to a VLAN or set of VLANs. Create a VNET, and then assign the IP address pool to the VNET.
A private IP address pool consists of private IPv4 addresses, which means that the addresses are not routable on the
Internet. However, you can later create and associate a virtual network address translation (vNAT) service between
the external network IP addresses and internal private IP addresses.
Create an IP address pool with the name dhcp-pool on VNET vnet1 using the IP address pool of 192.168.18.2
through 192.168.18.255 and specifying the optional VLAN group 124.
CLI network-admin@switch > ip-pool-create name dhcp-pool vnet vnet1 start-ip
192.168.18.2 end-ip 192.168.18.255 netmask 24 vlan 124
Pool created successfully.
CLI network-admin@switch > ip-pool-show layout vertical
name:
vnet:
scope:
vlan:
start-ip:
end-ip:
network:
dhcp-pool
vnet1
fabric
124
192.168.18.2
192.168.18.254
192.168.18.0/24
The IP address, 192.168.18.1, is excluded from this configuration because you need to configure it as the gateway IP
address of the DNS and DHCP services.
To modify an IP pool, use the ip-pool-modify command. You cannot modify the assigned VNET. If you decide
that you want to use the IP address pool on another VNET, you must delete the IP pool, and create a new one for the
new VNET.
To delete an IP pool, use the ip-pool-delete command.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
89
www.pluribusnetworks.com
Configuring DHCP Services
In this configuration, you use the IP address reserved from the IP address pool to create the DCHP service.
Informational Note:
Once you assign an IP address pool to a DHCP service that allocates dynamic IP addresses, you cannot
assign the same addresses as static IP addresses by other virtual network services.
Before you begin, see Configuring DNS Services to configure the DNS service shared by the DHCP.
1. Use the following command to create the DHCP service for VNET, vnet1. The DHCP server uses the assigned IP
address pool to allocate IP addresses to clients on the VNET.
CLI network-admin@switch > dhcp-create name vnet1-dhcp vnet vnet1
initial-ip-pool dhcp-pool
dhcp-show layout
name:
type:
scope:
vnet:
vnet-service:
state:
pxe-boot:
vertical
vnet1-dhcp
dhcp
fabric
vnet1
shared
enabled
disabled
2. Create the DHCP server for the VNET. Assign the IP pool configured earlier to the DHCP server which is used to distribute IP addresses.
CLI network-admin@switch > dhcp-create name vnet1-dhcp vnet vnet1
initial-ip-pool dhcp-pool
3. To display the configuration, use the dhcp-show command:
CLI network-admin@switch > dhcp-show layout vertical
dhcp-show
name:
type:
scope:
vnet:
vnet-service:
state:
pxe-boot:
vnet1-dhcp
dhcp
fabric
vnet1
shared
enabled
disabled
It is not necessary to add a network interface for the DHCP server since it is sharing the DNS service. In this case, the
vNIC is shared between DHCP and DNS.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
90
4. To display the vNIC information, use the dhcp-interface-show command:
CLI network-admin@switch > dhcp-interface-show
dhcp-name
nic
ip
mac
vlan if
--------- --------- -vnet1-dhcp vnet1.mgr.eth0 10.100.1.1/24
66:0e:94:4b:a3:e8 123 mgmt
vnet1-dhcp vnet1.mgr.eth1 192.168.18.1/24 66:0e:94:4b:af:75 124 data
5. Configure the options that the DHCP provides to DHCP clients. You can add the default route using the gateway IP
address, DNS domain name, and the IP address of the DNS server.
CLI network-admin@switch > dhcp-pool-modify dhcp-name vnet-dhcp name dhcp-pool
gateway-ip 192.168.18.1 ddns-domain pluribusnetworks.com dns-ip 192.168.18.1
Adding DHCP Interfaces
You can add DHCP services to an interface on the switch. To add DHCP to interface, dhcp-eng, with the IP address,
172.21.16.25, use the following command:
CLI network-admin@switch > dhcp-interface-create name dhcp-eng ip 172.21.16.25
netmask 32 assignment dhcp vlan 25
To modify the DHCP interface, use the dhcp-interface-modify command.
To remove the interface, use the dhcp-interface-remove command.
To display information about the DHCP interfaces, use the dhcp-interface-show command:
CLI network-admin@switch > dhcp-interface-show layout vertical
dhcp-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
to_vnic_flow_name:
dhcp-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
ext-50-dhcp
ext.50.mgr.eth0
10.111.1.1/24
static
66:0e:94:23:c4:7e
50
0
mgmt
www-51-dhcp
www.51.mgr.eth0
10.222.1.1/24
static
66:0e:94:23:bd:f6
51
0
data
Adding DHCP and DNS Records
The DHCP service adds hostname and IP address records dynamically to the DNS service if the DHCP client specifies
a hostname or if there is a static DHCP record for the client. You can also add hostname and IP address records
manually to the DHCP and DNS services.
Pluribus Networks Configuration Guide
91
www.pluribusnetworks.com
To manually add a static DHCP record, use the dhcp-host-add command:
CLI network-admin@switch > dhcp-host-add dhcp-name vnet1-dhcp hostname host1
fixed-ip 192.168.18.20 mac 10:0a:dd:ee:ff
When this DHCP client obtains a DHCP lease, the hostname and IP address pair are automatically added to the DNS
service.
To manually add a DNS record, use the dns-record-add command:
CLI network-admin@switch > dns-record-add dns-name vnet1-dns domain
pluribusnetworks.com host host2 ip 192.168.18.1
CLI network-admin@switch > dns-record-show
dns-name
vnet1-dns
vnet1-dns
ip
192.168.18.1
192.168.18.21
host
vnet-dns.pluribusnetworks.com
host2.plurisbusnetworks.com
Removing DHCP and DNS Services
To remove the configured DHCP and DNS services and the IP address pool, use the following commands:
CLI network-admin@switch > dhcp-delete name vnet1-dhcp
Deleted vnet1-dhcp
CLI network-admin@switch > dns-delete name vnet1-dns
Deleted vnet1-dns
CLI network-admin@switch > ip-pool-delete name dhcp-pool
Pool dhcp-pool deleted
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
92
Configuring DNS Services
In this topic, the necessary tasks required to configure DNS as a service to provide name translations for the IP
addresses assigned to the DHCP service.
Adding a DNS Server
Add a DNS server for the fabric-wide VNET, vnet1. The DNS and DHCP services are going to share the service zone
with the VNET manager.
1. To add the DNS server, use the following command:
CLI network-admin@switch > dns-create name vnet1-dns vnet vnet1
shared-vnet-service
2. The DNS service must communicate to hosts on the switch ports, so you must create a virtual NIC (vNIC) and add
an IP address. You have to specify the netmask and VLAN for the vNIC.
CLI network-admin@switch > dns-interface-add dns-name vnet1-dns if data ip
192.168.18.1/24 vlan 24
3. To display the configuration, use the dns-interface-show command:
CLI network-admin@switch > dns-interface-show layout vertical
dns-name:
nic:
ip:
assignment:
mac:
vlan:
if:
dns-name:
nic:
ip:
assignment:
mac:
vlan:
if:
vnet1-dns
vnet1.mgr.eth0
10.100.1.1/24
static
66:0e:94:4b:a3:e8
123
data
vnet1-dns
vnet1.mgr.eth1
192.168.18.1/24
static
66:0e:94:4b:af:75
124
data
This is a shared service, so in addition to the interface you just configured, the interface for the VNET manager is also
present.
Multiple domain names can be associated with an IP address. A reverse lookup is a query of the DNS for a domain
names when the IP address is known. This configuration requires that you define a reverse lookup pool IP addresses.
Pluribus Networks Configuration Guide
93
www.pluribusnetworks.com
4. Configure the DNS server for the domain and the reverse lookup pool for the DNS.
CLI network-admin@switch > dns-domain-add dns-name vnet1-dns domain
pluribusnetworks.com reverse-lookup-ip-pool dhcp-pool dns-ip 192.168.18.1
dns-domain-show layout vertical
dns-name:
domain:
type:
dns-ip:
reverse-lookup-ip-pool:
reverse-lookup-network:
forwarding:
forwarder:
vnet1-dns
pluribusnetworks.com
master
192.168.18.1
dpool
192.168.10.0/24
none
::
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
94
Overview of NAT and Hardware NAT
 Hardware NAT
 NAT and Hardware NAT Use Cases and Scenarios
  Static Mapping of Individual Private IP Addresses to Public IP Addresses
 Configuring Network Address Translation Services
 Configuring Hardware-based Network Address Translation(NAT)
Network Address Translation (NAT) substitutes the real address in a packet with a mapped address that is routable
on the destination network. NAT uses two steps: 1) translating a real address into mapped address, and 2) reversing
the process for returning traffic.
Just as you can assign DHCP and DNS services to a VNET, you can assign NAT services to a VNET. When you create the
NAT service, you can optionally configure it as a dedicated service, in a separate zone, or shared, in the same logical
zone, on a VNET, and assign a storage pool to it. You can also disable and enable the NAT service on the VNET.
Hardware NAT
Previously, NAT services were available only in nvOS software.Hardware-based NAT has the following functionality:
 HW-NAT only translates traffic that travels between different IP address realms and is configured for HW-NAT.
 The IP addresses inside of an internal domain can be re-used by other internal domains such as a VNET.
 A HW-NAT-enabled router, a vRouter, has an IP address translation table to translate addresses between realms.
 A HW-NAT-enabled router translates IP addresses in packets before forwarding the packets according to the
translation table lookup result.
 Endpoints are unaware of the NAT translation.
 If there is more than one exit point, for example, from internal to external realms, each NAT-enabled router
must have the same IP address translation table.
nvOS supports the following types of hardware-based NAT:
 Static basic NAT (Outbound NAT)
 Static basic NAT with subnet mask
 Dynamic NAT
 NAT-Protocol Translation (PT)
 1K bi-directional NAT sessions or subnets
 Only traditional NAT (outbound NAT) is supported. Two way NAT, bi-directional NAT and Twice NAT are not
supported.
 Applications with IP addresses in the payload, for example FTP, are supported with software NAT.
Pluribus Networks Configuration Guide
95
www.pluribusnetworks.com
NAT and Hardware NAT Use Cases and Scenarios
Figure 1: Static Mapping of Individual Private IP Addresses to Public IP Addresses
In Figure 1, a simple NAT diagram of mapping two internal IP addresses to a single external IP addresses.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
96
Figure 2: Dynamic NAT and NAT-PT
Figure 3: Static NAT
Pluribus Networks Configuration Guide
97
www.pluribusnetworks.com
Figure 4: NAT with Port Forwarding
Figure 5: NAT with Dynamic Mapping
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
98
Configuring Network Address Translation Services
To create a NAT service, vnet-nat1, on VNET, vnet-customer, as a dedicated service and enable it, use the following
command:
CLI network-admin@switch > nat-create name vnet-nat1 vnet vnet-customer
dedicated-vnet-service enable
Since this is a dedicated service, or if you have not created any network interfaces, then use the
nat-interface-add command to create the vNICs.
CLI network-admin@switch > nat-interface-add vnet-nat1 ip 10.100.1.1/24
assignment none vlan 123 if data
CLI network-admin@switch > nat-interface-add vnet-nat1 ip 192.168.18.1/24
assignment none vlan 124 if data
To modify the configuration, use the nat-interface-modify command. For instance, to change the VLAN
from 124 to 201, use the following syntax:
CLI network-admin@switch > nat-interface-modify vnet-nat1 ip 192.168.18.1/24
vlan 201
To display the configuration, use the nat-interface-show command:
CLI network-admin@switch > nat-interface-show nat-name vnet1-nat layout
vertical
nat-name: vnet1-nat
nic: vnet1.mgr.eth0
ip: 10.100.1.1/24
assignment: static
mac: 66:0e:94:4b:b8:0c
vlan: 123
vxlan: 0
if: data
nat-name: vnet1-nat
nic: vnet1.mgr.eth1
ip: 192.168.18.1/24
assignment: static
mac: 66:0e:94:4b:9d:cc
vlan: 201
vxlan: 0
if: data
To remove the NAT interfaces, use the nat-interface-remove command.
To delete the NAT service, use the nat-delete command. This command removes the entire NAT configuration
including the associated interfaces.
To modify the NAT service, use the nat-modify command.
Pluribus Networks Configuration Guide
99
www.pluribusnetworks.com
To enable dynamic NAT for internal IP addresses within the VNET, use the nat-map-add command. Traffic from
the interface is sent to the external IP address of the VNET.
CLI network-admin@switch > nat-map-add nat-name vnet1-nat name to-internal
ext-interface vnet1.mgr.eth0 network 192.168.18.2/24
To display the configuration, use the nat-map-show command:
CLI network-admin@switch > nat-map-show
nat-name
-------vnet-1-nat
name
---to-internal
ext-interface
------------vnet1.mgr.eth0
network
------192.168.18.2/24
The hosts on the VNET must have a default router with the internal IP address of the VNET manager. In this example,
the IP address is 192.168.18.1.
To remove the NAT mapping, use the nat-map-remove command.
Configuring Port Forwarding for NAT
Port forwarding or port mapping consists of configuring a gateway to send all packets received on a particular port
to a specific device on the internal network. For example, if the external network requires access a Web server with
port 80 and IP address 192.168.1.2, it is necessary to define a port forwarding rule on the gateway. The rule redirects
all TCP packets received on port 80 to machine 192.168.1.2.
To configure port forwarding from IP address 10.100.1.1:8888 to the internal IP address 192.168.18.4 and port 22,
use the following command:
CLI network-admin@switch > nat-port-forward-add nat-name vnet1-nat name vm1_ssh
ext-port 8888 int-ip 192.168.18.4 int-port 22
The NAT service now forwards from external address 10.100.1.1 port 8888 to the internal address 192.168.18.4 port
22 and permit Secure Shell connections on the well-known SSH port 22.
To remove the NAT port forwarding configuration, use the nat-port-forward-remove command.
To display NAT port forwarding information, use the nat-port-forward-show command.
Configuring Static NAT
Static NAT maps an unregistered IP address to a registered IP address on a one-to-one basis. This is useful when a
device needs to be accessible from outside the network. To configure a one-to-one mapping of the internal address
192.168.18.4 to the external IP address 10.100.1.1, use the following command:
CLI network-admin@switch > nat-static-nat-add nat-name gateway external-ip
10.100.1.1 internal-ip 192.168.18.4
To display the static NAT configuration, use the nat-static-nat-show command.
To remove the static NAT configuration, use the following syntax:
CLI network-admin@switch > nat-static-nat-remove nat-name gateway external-ip
10.100.1.1
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
100
Pluribus Networks Configuration Guide
101
www.pluribusnetworks.com
Configuring Hardware-based Network Address
Translation(NAT)
Before you can add the hardware-based NAT router, you must configure a fabric, VLAN, and vRouter interface. In this
example, we have the following configuration information:
 fabric-name — corp-fabric
 VLANs — VLAN 2 and VLAN 3
 ports — 53 and 55
 IP addresses — 2.2.2.1/24, 20.20.20.1/24, and 20.20.20.2/24
1. Create the fabric:
CLI network-admin@switch > fabric-create name corp-fabric
2. Create the vRouter:
CLI network-admin@switch > vrouter-create name hw-nat vnet global-default
router-type hardware
3. Add the VLANs to the configuration:
CLI network-admin@switch > vlan-create id 2 scope local ports all
untagged-ports 53
CLI network-admin@switch > vlan-create id 3 scope local ports all
untagged-ports 55
4. Add the vRouter interfaces:
CLI network-admin@switch > vrouter-interface-add vrouter-name hw-nat ip
2.2.2.1/24 vlan 2 if data
CLI network-admin@switch > vrouter-interface-add vrouter-name hw-nat ip
20.20.20.1/24 vlan 3 if data
CLI network-admin@switch > vrouter-interface-add vrouter-name hw-nat ip
20.20.20.2/24 alias-on hw.nat.eth1
5. Add the hardware-based NAT configuration:
CLI network-admin@switch > hw-nat-create name nat1 vrouter-name hw-router
Configuring Static NAT
To add a static NAT configuration to the hardware-NAT vRouter, add the following commands, and use the IP address
20.20.20.2 for an additional interface:
CLI network-admin@switch > hw-nat-static-nat-add hw-nat-name nat1 name
static-nat1 internal-ip 2.2.2.10 external-ip 20.20.20.1
CLI network-admin@switch > hw-nat-static-nat-add hw-nat-name nat1 name
static-nat2 internal-ip 2.2.2.20 external-ip 20.20.20.2
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
102
Configuring NAT with Port Forwarding
To add port forwarding from Host 1 using ports 1122 and 3344 to Host 2, add the following statements to the
configuration:
1. Remove the static NAT configuration from the previous example:
CLI network-admin@switch > hw-nat-static-nat-remove hw-nat-name nat1
CLI network-admin@switch > hw-nat-static-nat-remove hw-nat-name nat2
2. Add the port forwarding configuration:
CLI network-admin@switch > hw-nat-port-forward-add hw-nat-name nat1 name pf1
ext-ip 20.20.20.1 ext-port 80 int-ip 2.2.2.10 int-port 1122
CLI network-admin@switch > hw-nat-port-forward-add hw-nat-name nat1 name pf1
ext-ip 20.20.20.1 ext-port 80 int-ip 2.2.2.10 int-port 3344
Configuring Dynamic Mapping for NAT
To add dynamic mapping for hardware NAT, remove the port forwarding configuration and add the dynamic
mapping statements:
CLI network-admin@switch > hw-nat-port-forward-remove hw-nat-name nat1 name pf1
CLI network-admin@switch > hw-nat-port-forward-remove hw-nat-name nat1 name pf2
CLI network-admin@switch > hw-nat-map-add hw-nat-name nat1 name map1 network
2.2.2.1/24 ext-ip 20.20.20.1
To display the dynamic mapping, use the hw-nat-session-show:
CLI network-admin@switch > hw-nat-session-show
Pluribus Networks Configuration Guide
103
www.pluribusnetworks.com
nvOS System Logging and SNMP
 Configuring System Logging
 Displaying Log Counters Information
 Sending Log Messages to Syslog Servers
 Sending Log Messages to Syslog Servers
 Viewing Log Events
 Modifying and Displaying Log Event Settings
 Configuring SNMP
 SNMP Communities
 Users and SNMPv3
 Supported MIBs
Overview
nvOS logs all important activities that occur on the switch and fabrics created on them. Logging is enabled by default
and is viewable using the CLI. You can also configure system logging to send syslog-formatted messages to other
servers configured to receive them as part of centralized logging and monitoring.
Figure 1: nvOS Switch with Syslog Server
nvOS Switch
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
104
There are three types of activities logged by nvOS:
Table 1: Log Events
Type
Description
Event
Records action observed or performed by switches. Each Event type
can be enabled or disabled. Events are collected on a best effort
basis. If events occur too rapidly to be recorded, the event log is
annotated with the number of events lost. The following are
examples of event types:
•
•
•
•
Port state changes
TCP connections
STP port changes
PTP time corrections
Audit
When an administrative change to the configuration is made, an
audit log is recorded. An audit log consists of the command and
parameters along with the success or failure indication. When a
command fails, an error message is also recorded.
System
The system log records error conditions and conditions of interest.
There are four levels in the system log:
•
•
•
•
Perror
critical
error
warn
note
The perror log records messages on standard error output,
describing the last error encountered.
Each log message includes the following information:
 Category - event, audit, or system
 Timestamp within a microsecond
 Process name and process ID of the process producing the message
 Unique message name
 Unique five digit numerical message code
 Message: additional message-specific parameters and explanation
A log message may include optional parameters, including associated VLAN, VXLAN, or switch port.An audit log
message includes additional information:
 User
 Process ID
 Client IP of the remote computer issuing the command
An event log also includes the event type.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
105
www.pluribusnetworks.com
Configuring System Logging
To view event logs using the CLI, enter the following command:
CLI network-admin@switch > log-event-show
category
event
event
event
...
time
name
code event-type port message
2013-06-04,13:12:18.304740 port_up 62
port
62
up
2013-06-04,13:12:18.304740 port_up 62
port
50
up
2013-06-04,13:12:18.304740 port_up 62
port
10
up
To view audit log entries, enter the following command:
CLI network-admin@switch > log-audit-show
category time
name
code user
message
audit
2013-06-04,13:12:18.304740 command 1101 network-admin Command
create vnet id=b000011:! name=vnet1 scope=fabric vrg=b000011:0 vlans=100
vnet_mgr_id=b00001
audit
2013-06-04,13:12:18.304740 command 1101 network-admin Command
create vrouter id=b000011:! name=vnet1 scope=fabric vrg=b000011:0
vlans=100 vnet_mgr_id=b00001
To view system log entries, use the following command:
CLI network-admin@switch > log-system-show
time:
name:
level:
time:
name:
level:
time:
name:
level:
2015-09-17, 06:28:09.351514-07:00
11006
warn
2015-09-17, 11:28:09.351514-07:00
11006
warn
2015-09-17, 13:28:09.351514-07:00
11006
warn
Modifying and Displaying Log Event Settings
By default, only system and port events are logged. Other logging is possible, and you can add other events using the
log-event-settings-modify command. You can modify the way nvOS logs events by using the
log-event-settings-modify command to remove or add log events. For instance to remove logging of PTP
events, use the following command:
CLI network-admin@switch > log-event-settings-modify no-ptp
To display log event settings information, use the log-event-settings-show command.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
106
Displaying Log Counters Information
You can display information about the number of events that have occurred on the network by using the
log-system-counters-show command:
CLI network-admin@switch > log-system-counters-show layout vertical
switch:
critical:
error:
warn:
note:
pleiades24
0
0
1061
9
To reset the log counters, use the log-system-counters-reset command.
Formatting and Filtering of Logging Messages
There are many options for filtering and formatting of log messages returned by these commands. Use the <tab>
completion method and ? to explore them.
The log files are also available using SFTP, switch-ip:/sftp/nvOS/logs and NFS,
/net/switch-name/nvOS/logs if you have enabled the services.
Many systems support a syslog facility for sending or receiving log messages. Pluribus Networks infrastructure can
send messages to syslog servers using either RFC 5424 (Structure) or RFC 3164 (legacy) formats.
Sending Log Messages to Syslog Servers
To configure the switch to send all log messages to a syslog server with an IP address of 172.16.21.67, use the
following command:
CLI network-admin@switch > admin-syslog-create name log-all scope fabric host
172.16.21.76
To display the configuration use the admin-syslog-show command:
CLI network-admin@switch > admin-syslog-show
name
scope
log-all fabric
host
172.16.21.67
port
514
message-format
legacy
To specify sending the syslog messages in structured format, per RFC5424, add the message-format option to the
configuration.
CLI network-admin@switch > admin-syslog-modify name log-all message-format
structured
You can also modify the port that the service listens on to another port. More than one syslog listening service can
be configured and appropriate syslog messages are sent to each one.
By default, all log messages are forwarded to syslog servers. To filter the log messages, use the msg-level option
to specify the severity or other options:
CLI network-admin@switch > admin-syslog-match-add syslog-name log-all name
critical-msgs msg-level critical
Pluribus Networks Configuration Guide
107
www.pluribusnetworks.com
You can modify syslog matching using the admin-syslog-match-modify command, or remove matching
criteria using the admin-syslog-match-remove command.
To display the configuration, use the show command:
CLI network-admin@switch > admin-syslog-match-show
syslog-name
log-all
msg-level
critical
name
critical-msgs
Using Facility Codes with Log Messages
Log messages are labeled with a facility code indicating the area of the software that generated the log message.
nvOSuses the following facility codes by default:
 Log_Daemon
for events and system messages
 Log_AUDIT for audit messages
The following severities are used by default:
 Log_INFO for events and audit messages
 Log_Critical = critical
 Log_ERROR = error
 Log_WARNING = warn
 Log_NOTICE = note
You can override the default values by configuring matches for each syslog configuration which allows nvOSto
translate log messages into fields that the syslog servers understand.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
108
Viewing Log Events
For information about specific log events and their meaning, see the Pluribus Networks Log Message Reference
Guide.
A log message consists of common parameters separated by spaces and a colon (:), and optional parameters such as
key and value pairs, another colon, and then the log-specific message.
To view event logs using the CLI, enter the following command:
CLI network-admin@switch > log-event-show
category:
time:
switch:
program:
pid:
name:
code:
event-type:
vnet:
port:
vlan:
message:
category:
time:
switch:
program:
pid:
name:
code:
event-type:
vnet:
port:
vlan:
message:
category:
time:
event
2014-07-17,07:37:17.466173-07:00
pleiades24
nvOSd
6344
mac_ip_changed
11023
port
global-default
65
200
ip address change: mac=50:33:a5:e0:7f:fd ip=172.16.23.7
event
2014-07-17,07:37:50.109133-07:00
pleiades24
nvOSd
6344
mac_ip_changed
11023
port
vlb-web-svr
65
200
ip address change: mac=50:33:a5:e0:7f:fd ip=172.16.23.1
event
2014-07-17,07:42:17.418349-07:00...
Pluribus Networks Configuration Guide
109
www.pluribusnetworks.com
To view audit log entries, enter the following command:
CLI network-admin@switch > log-audit-show layout vertical
category:
time:
name:
code:
user:
message:
category:
time:
name:
code:
user:
message:
category:
time:
name:
code:
user:
message:
category:
time:
name:
code:
audit
2014-04-01,14:56:40.763626-07:00
user_command
11001
network-admin
Command "vlan-create id 25
audit
2014-04-01,14:56:40.765839-07:00
logout
11100
network-admin
logout
audit
2014-04-01,14:56:40.847912-07:00
login
11099
network-admin
login
audit
2014-04-01,14:56:40.888363-07:00
logout
11100
...
To view system log entries, use the following command:
CLI network-admin@switch > log-system-show
time:
name:
level:
time:
name:
level:
time:
name:
level:
2013-09-17, 06:28:09.351514-07:00
11006
warn
2013-09-17, 11:28:09.351514-07:00
11006
warn
2013-09-17, 13:28:09.351514-07:00
11006
warn
Modifying and Displaying Log Event Settings
By default, only system and port events are logged. Other logging is possible, and you can add other events using the
log-event-settings-modify command. You can modify the way nvOS logs events by using the
log-event-settings-modify command to remove or add log events. For instance to remove logging of PTP
events, use the following command:
CLI network-admin@switch > log-event-settings-modify no-ptp
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
110
To display log event settings information, use the log-event-settings-show command.
CLI network-admin@switch > log-event-settings-show
switch:
system:
port:
tcp:
stp:
igmp:
lldp:
lacp:
vdp:
ecp:
evb:
ptp:
openflow:
storage:
tacacs:
pleiades24
on
on
off
off
off
off
off
off
off
off
off
off
on
on
You can modify the log event settings using the log-event-settings-modify command. For example, if you
want to turn on TCP events, use the following command:
CLI network-admin@switch > log-event-settings-modify tcp
CLI network-admin@switch > log-event-settings-show
TCP is now turned
on.
switch:
pleiades24
system:
on
port:
on
tcp:
on
stp:
off
igmp:
off
lldp:
off
lacp:
off
vdp:
off
ecp:
off
evb:
off
ptp:
off
openflow: off
storage: on
tacacs:
on
openstack:on
Displaying Log Counters Information
You can display information about the number of events that have occurred on the network by using the
log-system-counters-show command:
CLI network-admin@switch > log-system-counters-show layout vertical
switch:
critical:
error:
warn:
note:
pleiades24
0
0
1061
9
Pluribus Networks Configuration Guide
111
www.pluribusnetworks.com
To reset the log counters, use the log-system-counters-reset command.
Formatting and Filtering of Logging Messages
There are many options for filtering and formatting of log messages returned by these commands. Use the <tab>
completion method and ? to explore them.
The log files are also available using SFTP, switch-ip:/sftp/nvOS/logs and NFS,
/net/switch-name/nvOS/logs if you have enabled the services.
Many systems support a syslog facility for sending or receiving log messages. Pluribus Networks infrastructure can
send messages to syslog servers using either RFC 5424 (Structure) or RFC 3164 (legacy) formats.
Sending Log Messages to Syslog Servers
To configure the switch to send all log messages to a syslog server with an IP address of 172.21.16.144, use the
following command:
CLI network-admin@switch > admin-syslog-create name log-all scope fabric host
172.21.16.144
To display the configuration use the admin-syslog-show command:
CLI network-admin@switch > admin-syslog-show
name
scope
log-all fabric
host
172.21.16.144
port
514
message-format
legacy
To specify sending the syslog messages in structured format, per RFC5424, add the message-format option to the
configuration.
CLI network-admin@switch > admin-syslog-modify name log-all message-format
structured
You can also modify the port that the service listens on to another port. More than one syslog listening service can
be configured and appropriate syslog messages are sent to each one.
By default, all log messages are forwarded to syslog servers. To filter the log messages, use the msg-level option
to specify the severity or other options:
CLI network-admin@switch > admin-syslog-match-add syslog-name log-all name
critical-msgs msg-level critical
You can modify syslog matching using the admin-syslog-match-modify command, or remove matching
criteria using the admin-syslog-match-remove command.
To display the configuration, use the show command:
CLI network-admin@switch > admin-syslog-match-show
syslog-name
log-all
msg-level
critical
name
critical-msgs
The parameters to match include msg-start, msg-end, msg-duration, msg-starting-point,
msg-length, and msg-reverse.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
112
Using Facility Codes with Log Messages
Log messages are labeled with a facility code indicating the area of the software that generated the log message.
nvOSuses the following facility codes by default:
 Log_Daemon
for events and system messages
 Log_AUDIT for audit messages
The following severities are used by default:
 Log_INFO for events and audit messages
 Log_Critical = critical
 Log_ERROR = error
 Log_WARNING = warn
 Log_NOTICE = note
You can override the default values by configuring matches for each syslog configuration which allows nvOSto
translate log messages into fields that the syslog servers understand.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
113
www.pluribusnetworks.com
Configuring SNMP
Simple Network Management Protocol (SNMP) is a widely used protocol for monitoring the health and welfare of
network equipment such as routers, computer equipment and even devices like UPSs. nvOS has implemented SNMP
using Net-SNMP version 5.7.2.
SNMP generally works the same in most implementations and this document does not provide indepth information
about SNMP overall. You can locate many resources on SNMP functionality on the Internet.
SNMP v1, v2, and v3 are now supported in nvOS. The SNMP daemon runs as a service and is launched by using the
following command:
CLI network-admin@switch > admin-service-modify if mgmt snmp
This command launches the daemon, subagents, and opens a port so that remote queries can reach the daemon.
SNMP Communities
Communities are used in SNMPv1 as a method of controlling access to information. You can create a community
using the following command:
CLI network-admin@switch > snmp-community-create community-string name-string
community-type read-only|read-write
To create a SNMP community string named, snmp-group, with read-only privileges, use the following command:
CLI network-admin@switch > snmp-community-create community-string snmp-group
community-type read-only
To modify the SNMP community, snmp-group, to write-only, use the following command:
CLI network-admin@switch > snmp-community-modify community-string snmp-group
community-type write-only
To display information about the SNMP community, snmp-group, use the following command:
CLI network-admin@switch > snmp-community-show community-string snmp-group
switch
-----pleiades24
community-string
---------------snmp-group
community-type
-------------read-only
To delete the SNMP community, snmp-group, use the following command:
CLI network-admin@switch > snmp-community-delete community-string snmp-group
Users and SNMPv3
SNMPv3 creates users as access control mechanisms, and creating users is more complex but also more secure and
more flexible. You can also require that users must authenticate and use encryption. Use the following command to
create a user
CLI network-admin@switch > snmp-user-create user-name name-string
auth-password [auth|no-auth] priv-password [priv|no-priv]
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
114
To create the user, snmp-admin, with authentication, password m0nk3ys, use the following command:
CLI network-admin@switch > snmp-user-create user-name snmp-admin auth-password
auth
auth password: ********
confirm password: ********
To modify the SNMP user and add private with the password, b33hiv3, use the following command:
CLI network-admin@switch > snmp-user-modify user-name snmp-admin auth-password
auth
priv-password priv
auth password: ********
confirm password: ********
priv password: ******
confirm password: ******
To display information about the SNMP user, use the following command:
CLI network-admin@switch > snmp-user-show user-name snmp-user
switch
-------pleiades24
user-name auth priv
--------- ---- ---snmp-user yes yes
To delete the SNMP user, use the snmp-user-delete command.
After you create the user, you must grant permission, using View Access Control Model (VACM) to view SNMP
objects:
CLI network-admin@switch > snmp-vacm-create user-name name-string user-type
[rouser|rwuser] oid-restrict string [auth|no-auth] [priv|no-priv]
The parameter, oid-restrict, is an optional argument that specifies a MIB sub-tree that the view is restricted.
In other words, if you specify an OID, only that OID and the descendants in the tree are visible in this view.
To continue with the previous example, snmp-user is a read-only user restricted only to sysContact OID:
CLI network-admin@switch > snmp-vacm-create user-name snmp-user user-type
rouser oid-restrict sysContact no-auth no-priv
To modify the VACM configuration and change no authentication to authentication, use the following command:
CLI network-admin@switch > snmp-vacm-modify user-name snmp-user user-type
rouser auth
To display information about the VACM configuration, use the snmp-vacm-show command:
switch
user-type user-name oid-restrict view auth priv
-------------- --------- ------------ ---- ---- ---pleiades24 rouser
snmp-user sysContact
no
no
To delete the VACM user from the SNMP configuration, use the snmp-vacm-delete command:
CLI network-admin@switch > snmp-vacm-delete user-name snmp-user
Pluribus Networks Configuration Guide
115
www.pluribusnetworks.com
Supported MIBs
nvOS customized MIBs:
 IfTable
 IfXTable
 EntPhySensorTable
OpenSolaris-supported MIBs:
 SNMPv2
 DISMAN-EVENT — monitors disks, processes and execs
 IF — monitors interfaces
 IP — monitors IP addresses and related information such as ipForwarding, ipForwarding, ipDefaultTTL,
ipInReceives, ipInHdrErrors, ipInAddrErrors, ipForwDatagrams, ipInUnknownProtos, ipInDiscards, ipInDelivers,
ipOutRequests, ipOutDiscards

ipOutNoRoutes

ipReasmTimeout

ipReasmReqds

ipReasmOKs

ipReasmFails

ipFragOKs

ipFragFails

ipFragCreates

ipAddrTable

ipRouteTable

ipNetToMediaTable

ipRoutingDiscards

Last bit mask
 TCP — monitors TCP packet information such as tcpRtoAlgorithm, tcpRtoMin, tcpRtoMax, tcpMaxConn,
tcpActiveOpens, tcpPassiveOpens, tcpAttemptFails, tcpEstabResets, tcpCurrEstab, tcpInSegs, tcpOutSegs,
tcpRetransSegs, tcpConnTable, tcpInErrs, tcpOutRsts,
 UDP — monitors UDP packet information
 HOST-RESOURCES
 NOTIFICATION-LOG
 SNMPv2-SMI
 IF-EXT
 ENTITY-SENSOR
See additional supported MIBs in Table , “”.
Additional commands that support SNMPv1, SNMPv2, and SNMPv3:
— The SNMP engine ID is a unique string of 28 characters that identifies the device
for administrative purposes. This command displays the identification of the local SNMP engine and all remove
engines configured on the switch.
 snmp-engineid-show
— Used to enable notifications about link conditions and common system
errors. This is used with the snmp-monitor commands.
 snmp-trap-enable-modify
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
116
 snmp-trap-enable-show
— Display enabled SNMP traps.
 snmp-trap-sink-create
— Used to specify a SNMPv1 trap receiver.
 snmp-trap-sink-delete
— Remove SNMP sink traps.
 snmp-trap-sink-modify
— Modify SNMP sink traps.
 snmp-trap-sink-show
— Display SNMP sink traps.
 snmp-v3-trap-sink-create
- Used to specify a SNMPv3 trap receiver.
 snmp-v3-trap-sink-delete
— Used to delete a SNMPv3 trap receiver.
 snmp-v3-trap-sink-modify
— Used to modify a SNMPv3 trap receiver.
 snmp-v3-trap-sink-show
— Used to display a SNMPv3 trap receiver.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
117
www.pluribusnetworks.com
Supported MIBs
Table 2: Supported MIBs
MIB
Description
AgentX
This is the MIB module for the SNMP Agent Extensibility Protocol
(AgentX). This MIB module is implemented by the master agent.
Bridge
The Bridge MIB module for managing devices that support IEEE
802.1D.
Disman-Event
The MIB module for defining event triggers and actions for network
management.
Disman-Schedule
This MIB module defines a MIB which provides mechanisms to
schedule SNMP set operations periodically or at specific points in
time.
Disman-Script
This MIB module defines a set of objects that allow you to delegate
management scripts to distributed managers.
Entity
The MIB module for representing multiple logical entities supported
by a single SNMP agent.
Entity-Sensor
This module defines Entity MIB extensions for physical sensors.
Ether-Like
The MIB module that describes generic objects for Ethernet-like
network interfaces.
HCNUM-TC
A MIB module containing textual conventions for high capacity data
types. This module addresses an immediate need for data types not
directly supported in the SMIv2. This short-term solution is meant
to be deprecated as a long-term solution is deployed.
Host-Resources
This MIB is for use in managing host systems. The term `host' is
construed to mean any computer that communicates with other
similar computers attached to the Internet and that is directly used
by one or more human beings. Although this MIB does not
necessarily apply to devices whose primary function is
communications services (e.g., terminal servers, routers, bridges,
monitoring equipment), such relevance is not explicitly precluded.
This MIB instruments attributes common to all Internet hosts
including, for example, both personal computers and systems that
run variants of Unix.
Host-Resources-Types
This MIB module registers type definitions for storage types, device
types, and file system types.
IANA-Address-FamilyNumbers
The MIB module defines the AddressFamilyNumbers textual
convention.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
118
Table 2: Supported MIBs
MIB
Description
IANA-Language
The MIB module registers object identifier values for well-known
programming and scripting languages. Every language registration
MUST describe the format used when transferring scripts written in
this language.
Any additions or changes to the contents of this MIB module
require Designated Expert Review as defined in the Guidelines for
Writing IANA Considerations Section document. The Designated
Expert will be selected by the IESG Area Director of the OPS Area.
Note, this module does not have to register all possible languages
since languages are identified by object identifier values. It is
therefore possible to registered languages in private OID trees. The
references given below are not normative with regard to the
language version. Other references might be better suited to
describe some newer versions of this language. The references are
only provided as `a pointer into the right direction'.
IANA-RTPROTO
This MIB module defines the IANAipRouteProtocol and
IANAipMRouteProtocol textual conventions for use in MIBs which
need to identify unicast or multicast routing mechanisms.
IANAifType
This MIB module defines the IANAifType Textual Convention, and
thus the enumerated values of the ifType object defined in MIB-II's
ifTable.
IF-Inverted-Stack
The MIB module which provides the Inverted Stack Table for
interface sub-layers.
IF
The MIB module to describe generic objects for network interface
sub-layers. This MIB is an updated version of the ifTable for MIB-II,
and incorporates the extensions defined in RFC 1229.
INET-Address
This MIB module defines textual conventions for representing
Internet addresses. An Internet address can be an IPv4 address, an
IPv6 address, or a DNS domain name. This module also defines
textual conventions for Internet port numbers, autonomous system
numbers, and the length of an Internet address prefix.
IP-Forward
The MIB module for the management of CIDR multipath IP Routes.
IP
The MIB module for managing IP and ICMP implementations, but
excluding their management of IP routes.
IPv6-Flow-Label
This MIB module provides commonly used textual conventions for
IPv6 Flow Labels.
IPv6-ICMP
The MIB module for entities implementing the ICMPv6.
IPv6
The MIB module for entities implementing the IPv6 protocol.
IPv6-TC
Imports Integer32 From SNMPv2-SMI
IPv6-TCP
The MIB module for entities implementing TCP over IPv6.
IPv6-UDP
The MIB module for entities implementing UDP over IPv6.
Pluribus Networks Configuration Guide
119
www.pluribusnetworks.com
Table 2: Supported MIBs
MIB
Description
NET-SNMP-AGENT
Defines control and monitoring structures for the Net-SNMP agent.
NET-SNMP-EXAMPLES
Example MIB objects for agent module example implementations
NET-SNMP-EXTEND
Defines a framework for scripted extensions
NET-SNMP
Top-level infrastructure of the Net-SNMP project enterprise MIB
tree
NET-SNMP-PASS
Example MIB objects for "pass" and "pass-persist" extension script
NET-SNMP-TC
Textual conventions and enumerations for the Net-SNMP project
NET-SNMP-VACM
Defines Net-SNMP extensions to the standard VACM view table.
NOTIFICATION-Log
The MIB module for logging SNMP Notifications, that is, Traps and
Informs.
RFC-1215
This module is a empty module. It has been created solely for the
purpose of allowing other modules to correctly import the
TRAP-TYPE clause from RFC-1215 where it should be imported
from. It's a built in type in the UCD-SNMP code, and in fact
RFC-1215 doesn't actually define a mib at all; it only defines macros.
However, importing the TRAP-TYPE is conventionally done from an
import clause pointing to RFC-1215.
RFC-1155-SMI
Exports everything including Lnternet, directory, mgmt,
experimental, private, enterprises, OBJECT-TYPE, ObjectName,
ObjectSyntax, SimpleSyntax, ApplicationSyntax, NetworkAddress,
IpAddress, Counter, Gauge, TimeTicks, Opaque;
RFC-1213
Imports mgmt, NetworkAddress, IpAddress, Counter, Gauge,
TimeTicks
RMON
Imports MODULE-IDENTITY, OBJECT-TYPE, OBJECT-IDENTITY,
NOTIFICATION-TYPE, mib-2, Counter32, Integer32, TimeTicks
FROM SNMPv2-SMI, and TEXTUAL-CONVENTION, DisplayString
FROM SNMPv2-TC, and MODULE-COMPLIANCE, OBJECT-GROUP,
NOTIFICATION-GROUP FROM SNMPv2-CONF
SCTP
The MIB module for managing SCTP implementations.
SMUX
Imports enterprises
FROM RFC1155-SMI
DisplayString
FROM SNMPv2-TC
OBJECT-TYPE
FROM RFC-1212;
SNMP-Community
This MIB module defines objects to help support coexistence
between SNMPv1, SNMPv2c, and SNMPv3.
SNMP-Framework
The SNMP Management Architecture MIB
SNMP-MPD
The MIB for Message Processing and Dispatching
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
120
Table 2: Supported MIBs
MIB
Description
SNMP-Notification
This MIB module defines MIB objects which provide mechanisms to
remotely configure the parameters used by an SNMP entity for the
generation of notifications.
SNMP-Proxy
This MIB module defines MIB objects which provide mechanisms to
remotely configure the parameters used by a proxy forwarding
application.
SNMP-Target
This MIB module defines MIB objects which provide mechanisms to
remotely configure the parameters used by an SNMP entity for the
generation of SNMP messages.
SNMP-User-Based-SM
The management information definitions for the SNMP User-based
Security Model.
SNMP-USM-AES
Definitions of Object Identities needed for the use of AES by SNMP's
User-based Security Model.
SNMP-USM-DH-Objects The management information definitions for providing forward
secrecy for key changes for the usmUserTable, and for providing a
method for 'kickstarting' access to the agent via a Diffie-Helman key
agreement.
SNMP-View-Based-ACM The management information definitions for the View-based Access
Control Model for SNMP.
SNMPv2-Conf
Imports ObjectName, NotificationName, ObjectSyntax from
SNMPv2-SMI
SNMPv2
The MIB module for SNMP entities.
SNMP-SMI
The MIB module that provides the notation for writing SNMP MIBs.
SNMP-TC
Imports TimeTicks from SNMPv2-SMI
SNMP-TM
The MIB module for SNMP transport mappings.
TCP
The MIB module for managing TCP implementations.
Transport-Address
This MIB module provides commonly used transport address
definitions.
Tunnel
The MIB module for management of IP Tunnels, independent of the
specific encapsulation scheme in use.
UCD-Demo
SMIv2 version converted from older MIB definitions.
UCD-DISKIO
This MIB module defines objects for disk IO statistics.
UCD-DLMOD
This file defines the MIB objects for dynamic loadable MIB
modules.
Pluribus Networks Configuration Guide
121
www.pluribusnetworks.com
Table 2: Supported MIBs
MIB
Description
UCD-IPFWACC
This module defines MIB components for reading information from
the accounting rules IP Firewall. This would typically let you read
the rules and the counters. I did not include some flags and fields
that I considered irrelevant for the accounting rules. Resetting the
counters of the rules by SNMP would be simple, but I don't consider
it so useful. I gave no consideration to implementing write access
for allowing modification of the accounting rules.
UCD-SNMP
This file defines the private UCD SNMP MIB extensions.
UDP
The MIB module for managing UDP implementations.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
122
High Availability
Pluribus Networks switches automatically perform functions that ease your administrative burden. In the case of
high availability, switches in a fabric automatically detect other switches in the fabric. If multiple connections exist
between two switches, they automatically create an 801.3ad Link Aggregation Group (LAG) between the two
switches for resiliency and load balancing. Other features require configuration such as connecting one device to
two switches, or if LAGs are desired between Pluribus switches and other manufacturers’ equipment.
Configuring a Cluster
If you have two Pluribus switches, and want them to work together to provide networking services in the event one
of the switches fails, the switches must be members of the same fabric, and you must configure them as a cluster.
Pluribus Networks Configuration Guide
123
www.pluribusnetworks.com
To set up a cluster of two switches, pleiades4 and pleiades6, you must verify that they are members of the existing
fabric:
CLI network-admin@switch > fabric-node-show layout vertical
name:
fab-name:
mgmt-ip:
mgmt-vlan:
fab-tid:
out-port:
version:
state:
name:
fab-name:
mgmt-ip:
mgmt-vlan:
fab-tid:
out-port:
version:
state:
pleiades4
corp-fab
10.9.9.141/16
0
29
0
0.18.2789,pn-nvOS-b144a
online
pleiades6
corp-fab
10.9.9.139/0
0
29
60
0.18.2789,pn-nvOS-b144a
online
To create a cluster configuration, use the following command:
CLI network-admin@switch > cluster-create name cluster1 cluster-node-1
pleiades4 cluster-node-2 pleiades6
To verify the status of the cluster, use the cluster-show command:
CLI network-admin@switch > cluster-show
name
cluster1
state
online
cluster-node-1
pleiades4
cluster-node-2
pleiades6
This feature enhancement allows a cluster member to “repeer”. The repeer-to-cluster-node option is
used when a replacement cluster member joins the fabric after the failure of a node.
CLI (network-admin@switch) > fabric-join repeer-to-cluster-node e68
Joined fabric e68. Restarting nvOS...
Local objects that are present within members of a cluster are replicated across the cluster, so when the “repeer”
occurs all local objects are rebuilt in service of the cluster.
To display information about the cluster, use the cluster-info command:
CLI network-admin@switch > cluster-info format all layout vertical
name:
id:
state:
cluster-node-1:
cluster-node-2:
tid:
ports:
validate:
vlag
a000030:1
online
167772208
167772196
1
26
yes
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
124
If you want to connect the cluster nodes to an uplink switch, you must configure a VLAG between the ports on the
cluster nodes and the uplink switch.
Informational Note: Before you can create a VLAG, you must configure the two switches in a cluster.
For example, if pleiades6 has port 53 connected to the uplink switch and pleiades4 has port 19 connected to the
uplink switch, create a VLAG by executing the vlag-create command on either of the switches:
CLI network-admin@switch > vlag-create name vlag-uplink local-port 53
peer-switch pleiades4 peer-port 19
This example assumes that you’ve entered the command on pleiades6.
To verify the configuration, use the following command:
CLI network-admin@switch > vlag-show
name
vlag-uplink
local-port
53
peer-switch
pleiades4
peer-port
19
status
online
Support for Configuration Changes if a Cluster Node is not Responding
Transactions are allowed to proceed if at least one node in the cluster is reachable. If a cluster node is offline when a
configuration change is requested the transaction proceeds even though one of the cluster members is offline.
Nodes that were ignored for transactions automatically try to recover the transactions. Auto-recovery is enabled by
default but may be disabled. You can also configure the length of time between retry attempts between the nodes.
This feature is enabled by default, but may be disabled.
The following is a sample CLI output with one cluster node offline:
CLI (network-admin@switch1) > vlan-create id 24 scope fabric
Warning: cluster node switch2 not reachable, continuing anyway
The following is a sample of CLI output with both cluster nodes offline:
CLI (network-admin@switch2) > vlan-create id 33 scope fabric
Warning: cluster node switch1 not reachable, continuing anyway
vlan-create: fabric error: switch1 unreachable, both cluster nodes offline
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
125
www.pluribusnetworks.com
Configuring Fabric-based Physical Storage Pools
You can create storage pools on the disks shipped with your switch and create physical storage resources. These
resources can be virtualized and allocated to individual virtual networks. Physical storage consists of hard disk drives
(HDD), solid-state disk drives (SSD), or high-IOPS Fusion-IO Flash-based storage.
Informational Note: Additional storage is not available on the E68 series. For the F64 series, additional
storage is available and must be ordered as an additional component to the switch.
When the switch is booted up, it performs checks for uninitialized storage devices. If found, the devices are
automatically formatted and a storage pool is created on each one.
Informational Note: If you prefer other pool layouts, such as a RAID 1 mirror created from two disks,
then delete the pools on the disks you want to use and add the now-free disks to other pools.
Before you start, display information about the storage set up on the switch:
CLI network-admin@switch > storage-pool-show
switch
------------pleiades01
pleiades01
name
-------datapool
rpool
raid-type
--------no_raid
no_raid
used
----213G
87.5G
avail
----1.58T
21.7G
status
-----ok
ok
state
-----ONLINE
ONLINE
You can also display the physical storage media installed on the switch that is available to create a new storage pool:
CLI network-admin@switch > storage-device-show
switch
------------pleiades01
pleiades01
pleiades01
pleiades01
pleiades01
name
----disk0
disk1
disk4
disk5
disk6
label
---------internal-0
internal-1
back-0
back-1
internal
disk
-----c6t0d0
c6t1d0
c6t4d0
c6t5d0
c1d0p0
type capacity
---- -------disk 112G
disk 112G
disk 932G
disk 932G
flash1.35T
in-use
-----yes
yes
yes
yes
yes
data-set
-------rpool
datapool
datapool
pooldisk1
The column, data-set, refers to the ZFS root pool parameter which identifies the location for storage.
The column, type, identifies the type of storage media as disk or flash.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
126
To create a new physical storage pool, with no RAID protection, using available disk disk3, enter the following
command at the command prompt:
CLI network-admin@switch > storage-pool-create name store-new device1 disk3
raid-type no_raid
storage-pool-show
switch
name
--------------pleiades01
rpool
pleiades01
store-new
raid-type
--------no_raid
no_raid
used
---62.7G
92.5K
avail
----10.2G
457G
By default, the storage-pool-create command creates a disk library and image library within the new
storage pool, and exports the libraries to the network by using NFS sharing. Since disk and image library storage is
limited to storage pools other than rpool, optional disk storage is needed to implement those features.
To verify that the disk library is created, use the following command:
CLI network-admin@switch > disk-library-show storage-pool store-new layout
vertical
switch:
pleiades01
name:
disk-lib-pluribus
storage-pool:
store-new
sharing:
nfs
import-share:
pleiades01:/disk-lib/newpool/import
export-share:
pleiades01:/disk-lib/newpool/export
switch:
pleiades01
name:
disk-lib-pool-disk1
storage-pool:
pool-disk1
sharing:
nfs
import-share:
pleiades01:/disk-lib/pool-disk1/import
export-share:
pleiades01:/disk-lib/pool-disk1/export
To display the ISO image library, use the following command:
CLI network-admin@switch > iso-library-show storage-pool store-new layout
vertical
switch: pleiades01
name: iso-lib-store-new
storage-pool: store-new
sharing: nfs
import-share: pleiades24:/iso-lib/store-new/import
export-share: pleiades24:/iso-lib/store-new/export
dedup: no
To delete the physical storage pool, store-new, use the following command:
CLI network-admin@switch > storage-pool-delete name store-new
To verify that the storage pool is deleted, use the storage-pool-show command:
CLI network-admin@switch > storage-pool-show
switch
pleiades01
name
rpool
raid-type
no-raid
used
62.7G
avail
10.2G
status
ok
state
ONLINE
Pluribus Networks Configuration Guide
127
www.pluribusnetworks.com
To verify that the disk space is now free, use the storage-device-show command:
storage-device-show
switch
name label
--------- ----pleiades01 disk0 internal-0
pleiades01 disk1 internal-1
pleiades01 disk3 back-0
disk
---c6t0d0
c6t1d0
c6t3d0
type
---disk
disk
disk
capacity
-------74.5G
74.5G
466G
in-use
-----yes
yes
no
data-set
-------rpool
rpool
Displaying and Downloading Storage Images
You can use the storage-image commands to view downloaded image files, refresh the list, and download files.
1. Refresh the image list:
CLI network-admin@switch > storage-image-refresh
2. Display the available images:
CLI network-admin@switch > storage-image-show
switch
------------mitch-aquila2
mitch-aquila2
mitch-aquila2
mitch-aquila2
name
--------------------------------CentOS-6.4-x86_64-bin-DVD1.iso.gz
CentOS-6.5-x86_64-bin-DVD1.iso.gz
openstack-centos-neutron.vhd.gz
openstack-centos.vhd.gz
size
----3.94G
4.04G
2.81G
4.31G
status
----------downloaded
downloaded
downloaded
server-only
3. The status, downloaded, means that the images are already downloaded from the server, and the status,
server-only, means that the image is available for downloading.
4. To download the openstack-centos.vhd.gz image, use the following syntax:
CLI network-admin@switch > storage-image-download name openstack-centos.vhd.gz
Periodically run the storage-image-show command to check the status of the download. Once the status
changes to downloaded, you can use the image to create VMs on the switch.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
128
Creating Virtual Storage for a Virtual Network (VNET)
Virtual storage is useful to store virtual machine (VM) images for a elastic compute pool and as a data share for a
virtual network. Elasticity, in this case, means that you can shift and pool resources across your infrastructure
without over provisioning the network. Virtual storage is available to hosts on the VNET through the NFS protocol.
1. Create an IP pool and VNET to host the servers in the elastic compute pool.
CLI network-admin@switch > vnet-create name elas-com-pool scope local
mgr-eth1-vlan 10 vnet-mgr-name ecp1_vmgr mgr-eth0-ip 10.11.37.4
mgr-eth0-netmask 16
Vnet created.
CLI network-admin@switch > ip-pool-create name vpool vnet elas-com-pool
start-ip 192.168.1.1 end-ip 192.168.1.254 netmask 24
2. Create the virtual storage for VMs with the maximum size of 80GB and set the performance optimization to
latency:
CLI network-admin@switch > storage-folder-create elas-com-pool storage-pool
store-new max-space 80g optimization latency sharing nfs
3. Use the storage-folder-show command to display the storage folder configuration:
name
storage-pool vnet max-space backup sharing dedup optimization
ec1_vstor store-new
0:0
80
no
nfs
no
latency
To delete the storage folder, ec1_vstor, use the storage-folder-delete command.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Managing Host Operating Systems
You can set up host operating system ISO images and disk images on your switch. Host OS images are useful to
automatically provision servers assigned to a virtual network in a stateless computing environment, and create local
Netvisor VMs.
With stateless computing, the underlying compute resources, server hardware, are completely transparent to the
OS or applications using it. This allows an OS or application to move from one server to another very easily.
In this example, the VM image is an ISO file named ubuntu-12.10-desktop-i386.iso that you copy and then install on
the switch.
Pluribus Networks Configuration Guide
129
www.pluribusnetworks.com
Using the storage pool, store-new, verify that you have enough disk space and that an ISO library is created:
CLI network-admin@switch > storage-pool-show
switch
pleiades24
name
store-new
raid-type
no_raid
used
92.5K
avail
457G
CLI network-admin@switch > iso-library-show layout vertical
switch:
name:
storage:
sharing:
import-share:
export-share:
dedup:
pleiades24
iso-lib-pool-store-new
store-new
nfs
pleiades24:/iso-lib/pool/store-new/import
pleiades24:/iso-lib/pool/store-new/export
no
1. Copy the VM image to your switch from another computer using the ISO library NFS share that was added when
the storage pool was created. Copying the image depends on your computer’s OS, but on a Mac OS platform, use
the $ showmount -e ip-address using the IP address of your switch and the Terminal application.
$showmount -e 10.10.20.147
Exports list on 10.10.20.147:
/disk-lib/store-new/export
Everyone
/nvOS/log
Everyone
/mnt/vmiso/ubuntu-11.04-amd64
Everyone
/disk-lib/new-store/import
Everyone
/mnt/vmiso/centOS-6.5-x86_64
Everyone
/mnt/vmiso/centOS-6.4-x86_64
Everyone
/nvOS/vlb-web-svr-mgr/kickstarts
Everyone
$cd /net/10.10.20.147/disk-lib/store-new/import
$cp ubuntu-12.10-desktop-i386.iso
2. Add the new VM image to your switch using the iso-library-image-import command:
CLI network-admin@switch > iso-library-image-import iso-library-name
iso-lib-pool-disk1 image-label ubuntu-12 image-file
ubuntu-12.10-desktop-i386.iso
Your VM image is now transferred to the virtual store and available for installation on bare metal or virtualized
servers.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
130
3. To display a list of VM images on your switch, use the following command:
CLI network-admin@switch > iso-library-image-show iso-library-name
iso-lib-disk1
switch
------------pleiades24
pleiades24
pleiades24
pleiades24
pleiades24
pleiades24
pleiades24
pleiades24
iso-library-name
--------------------iso-lib-pool-datapool
iso-lib-pool-datapool
iso-lib-pool-datapool
iso-lib-pool-datapool
pluribus
pluribus
pluribus
pluribus
label
-----------------ubuntu-13.iso
vmware-setup.iso
ubuntu-12.iso
ubuntu-13.1
ubuntu-11.04-amd64
centOS-6.4-x86_64
centOS-6.5-x86_64
Netvisor-b144b-kvm
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Provisioning Bare Metal Servers
A bare metal environment is a computer system or a network in which a virtual machine is installed directly on
hardware rather than within a host operating system (OS). The term, bare metal, refers to the hard disk where a
computer’s OS is typically installed.
Preboot Execution Environment (PXE - pronounced “pixie”) is an industry standard client and server interface that
allows networked computers without an OS to be configured and booted remotely. PXE provides three things:
 DHCP which allows the client to receive an IP address and gain access to the network servers.
 A set of Application Programming Interfaces (API) used by the client’s Basic Input/Output System (BIOS) or a
Network Bootstrap Program (NBP) that automates the booting of the OS.
 A standard method of initializing the PXE code in the PXE ROM chip or boot disk.
How does PXE work? The process consists of the following steps:
1. The client notifies the switch that it uses PXE.
2. Since the switch is configured for PXE, it sends the client a list of boot servers that contain the available OS.
3. The client finds the boot server that it can use and receives the name of the file to download.
4. The client downloads the file and executes it.
Before You Begin
Before you start the PXE process and provisioning a bare metal server, be sure that you have the following
parameters configured:
 The switch is configured as part of a fabric.
 You have at least one VNET configured.
 Create an IP address pool for the DHCP server.
CLI network-admin@switch > ip-pool-create name dhcppool vnet pxevnet network
172.24.100.0 netmask 24
Pluribus Networks Configuration Guide
131
www.pluribusnetworks.com
 The DHCP server provides IP addresses to clients that are PXE booting, and using the parameter pxe-boot
all-hosts allows any host to receive an IP address from the IP address pool.
CLI network-admin@switch > dhcp-create name pxedhcp vnet pxevnet
initial-ip-pool dhcppool pxe-boot all hosts
If you specify the parameter, pxe-boot by-host-mac, only PXE-booting systems with registered MAC
addresses are allowed to PXE boot and get an IP address.
1. Rack your bare metal server hardware and connect it to your switch. If you are not using the option pxe-boot
all-hosts, write down the MAC address of the network adapter.
2. To boot a specific MAC address with hostname r5-d4 using PXE boot, use the following command:
CLI network-admin@switch > dhcp-host-add dhcp-name pxedhcp hostname r5-d4 mac
00:25:90:63:8c:26 pxe-boot
3. Power on the bare metal server.
4. After the server has PXE booted, it obtains an IP address from the DHCP server and downloads pxelinux.0
bootloader code.
5. The PXE Boot Menu is displayed on the bare metal server.
6. Select an installation type from the list to install on the bare metal server and complete the installation.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Customizing PXE Boot Options
To create a custom PXE boot image, copy the desired file to the switch, and be sure that an ISO library is created, and
NFS automounting is configured:
cp CentOS-6.2-x86_64-bin-DVD1.iso
/net/server-ip-address/iso-lib/pool-name/import
The IP address is the IP address of the switch, and the pool-name is the storage pool created in the ISO library. Be
sure to import a CD/DVD image that includes the PXE boot files.
Configure the ISO image as an available image for the switch to use in PXE boot environments using the following
syntax:
CLI network-admin@switch > iso-library-image-import iso-library-name store-new
image-label centOS-6.2-x86_64dvd image-file Centos-6.2-x86_64-bin-DVD1.iso
image-library store-new
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
132
You can use the dhcp-pxe-menu-show to display the default values for the menu:
CLI network-admin@switch > dhcp-pxe-menu-show dhcp-name pxedhcp
dhcp-name: pn-dhcp-dns
name: centOS-6.2-amd64-install
iso-library: pluribus
iso-label: centOS-6.2-x86_64dvd
menu-label: CentOS 6.2 amd64 Install
kernel-iso-path: images/pxeboot/vmlinuz
initrd-iso-path: images/pxeboot/initrd.img
append: initrd=<initrd-path> ks=http://<dhcp-server-ip>:
80/kickstarts/centos.ks ksdevice=eth0 interface=eth0
iso-url: http://::/vmiso/centOS-6.2-x86_64
name
The name of the PXE boot menu item.
iso-label
The name chosen when the ISO image was added.
menu-label
The label for the file as it appears in the PXE boot menu.
kernel-iso-path
The path to the kernel on the ISO image.
initrd-iso-path
The path to initrd on the ISO image
append
Any arguments to pass to the kernel at boot time.
iso-url
The location of the ISO image
The server-ip is the IP address of the switch, and the initrd-path is the path to the copied file on the TFTP
server and is replaced when the PXE menu is generated. You are likely to find any append arguments on the Linux
DVD in the pxelinux.cfg/default file.
Some arguments depend on your switch configuration. The first argument is the DHCP server IP address. The second
argument is the path to the copied initrd file. This file is shared on the TFTP server and is replaced when the PXE
boot menu is generated. Connect using TFTP and download the file to inspect it.
Creating a Custom PXE Boot Menu
You can create your own PXE boot menu based on the details of the ISO image:
CLI network-admin@switch > dhcp-pxe-menu-add dhcp-name pxedhcp name centos-6.5
iso-library iso-lib-pool-disk1 iso-label centOS-5.5-x86_64 kernel-iso-path
/image/pxeboot/vmlinuz initrd-iso-path images/pxeboot/initrd.img append
“initrd=10.10.20.147” menu-label CentOS-6.5
Pluribus Networks Configuration Guide
133
www.pluribusnetworks.com
Use the dhcp-pxe-menu-show command to display the menu:
CLI network-admin@switch > dhcp-pxe-menu-show
name:
centOS-6.5
iso-library:
pluribus
iso-label:
centOS-6.5-x86_64
menu-label:
CentOS 6.5
kernel-iso-path: images/pxeboot/vmlinuz
initrd-iso-path: images/pxeboot/initrd.img
append:
initrd=<initrd-path>
ks=http://<server-ip>:<web-port>/kickstarts/centos-6_5.ks ksdevice=eth0
interface=eth0
iso-url:
http://172.16.23.1/vmiso/centOS-6.5-x86_64
dhcp-interface:
dhcp-name:
vlb-dhcp
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
134
External Disk Drive Installation Guide
For Pluribus Networks hardware models F64 and E28Q, you can install external hard drive disks for additional
storage. You can install either SSD or Fusion I/O disk types.
Be sure to follow all appropriate precautions to prevent Electrostatic Discharge on the new hard drive disk.
Take care when removing the disk from the ESD bag, and installing it in the hard drive carrier.
Locating the Disk Drive Carrier
The disk drive carrier is located on the rear of the F64 and E28Q models.
External Drive
Location
Before adding or removing disks from the switch, power down the switch.
To remove the disk drive from the switch, use the following steps:
1. Locate the small slot in the drive button, and using a small slot screwdriver or a small coin, turn the slot to align
with the Unlock icon.
Button slot
aligned with
Unlock icon.
2. Press the button to release the drive carrier from the drive slot and release the front latch.
3. Use the latch to carefully pull the drive carrier from the slot.
4. Place the external memory drive into the drive carrier.
5. Line up the holes on the memory drive with the holes on the carrier.
Pluribus Networks Configuration Guide
135
www.pluribusnetworks.com
6. Insert the screws on each side and using a Phillips head screwdriver, hand tighten the screws into the disk.
7. Return the carrier to the empty slot on the switch, and push the drive into the slot.
8. Close the latch of the drive carrier and be sure that it clicks into place.
9. With a slot screwdriver or small coin, turn the slot in the round button to a vertical position. This locks the drive
into the switch.
10. Power on the switch and the new disk is initialized during the boot process.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
136
Configuring High Availability for Storage Folders
Informational Note: Before you begin configuring this feature, there are two pre-requisites for it:
• You must create a storage folder using the storage-folder-create command.
• You must have the name of the peer storage pools to add to the configuration.
Storage folders can be replicated between two switches by configuring a vFolder on one switch. This creates a
similar folder on the second switch which is replicated from the active switch to the peer switch at the configured
backup interval.
You can also configure an IP address for the vFolder that allows you to share the folder using NFS or SFTP.
In this example, there are two switches in the fabric, pleiades24 and pleiades25. You configured a storage folder,
iso-images, on Pleiades24. The VLAN 110 has the scope fabric, and has a IP pool of 192.168.11.0/24. To backup the
vFolder every 30 minutes, configure the backup interval to 30 minutes. Pleiades25 has a storage pool, datapool,
configured on it.
1. Create a vFolder on pleaides24 and add pleaides25 as the peer switch:
CLI network-admin@switch > storage-vfolder-create name my-backup folder
iso-files local-switch pleiades24 peer-switch pleiades25 peer-pool datapool
backup-interval 30 ha-ip 192.168.11.17 ha-netmask 24 ha-vlan 110 ha-if data
2. Display the configuration using the storage-vfolder-show command:
CLI network-admin@switch > storage-vfolder-show format all layout vertical
name:
folder:
local-switch:
local_pool:
peer-switch:
peer-pool:
backup-interval:
last-backup:
active-sw:
ha-nic:
ha-ip:
ha-vlan:
ha-vxlan:
ha-if:
failover_controller:
failover_action:
force:
my-backup
iso-files
pleiades24
pool-disk4
pleiades25
datapool
1800
10:23:51
pleiades24
eth2.110
192.168.11.17/24
110
0
mgmt
0
stop-old
false
The show output displays the failover controller as 0, the failover-action as stop-old, and force as
false by default.
Currently, failover to the peer switch does not occur automatically. When you issue the
storage-vfolder-failover command, you failover to the peer switch to become the active switch.
CLI network-admin@switch > storage-vfolder-failover name my-backup active-sw
pleiades25
Pluribus Networks Configuration Guide
137
www.pluribusnetworks.com
When you issue this command, the following actions occur on the local switch:
 The folder, my-backup on the current active switch is deactivated. It is unshared, and unmounted on the local
switch.
 The folder, my-backup, on the peer switch is activated.
 If a HA IP address is configured, it is added to the new primary switch.
 If the local folder is shared over NFS or SFTP, the sharing is activated on the new primary folder.
 The local switch begins replicating the folder, my-backup, onto the peer switch.
Using the Force Option for vFolder Failover
During vFolder failover, if the primary switch is not available, the failover operation fails and returns an error
message. If the force option is specified, the failover operation continues by enabling the folder on the peer switch.
The vFolder on the primary switch is not deactivated.
To use the force option, use the following syntax:
CLI network-admin@switch > storage-vfolder-failover name mybackup active-sw
pleiades25 force
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
138
Configuring a Linux Netvisor KVM
There are three ways to create a Netvisor KVM:
 From a bootable ISO image that runs in memory and is not persistent.
 From a bootable ISO image used to install the Linux distribution onto a disk-image within the switch.
 From an already created disk image imported onto the switch from another switch.
Informational Note: You cannot store disk images and ISO libraries in the root storage pool, rpool.
Storage outside of rpool must be configured using storage-pool commands before you can store images
and ISOs.
1. Your developer virtual machine requires a disk volume to install and store the operating system. Verify that your
switch has sufficient physical storage capacity (GB):
CLI network-admin@switch > storage-pool-show
switch
-----pbg-nvos
pbg-nvos
name
---pool-disk1
rpool
raid-type
--------no_raid
no_raid
used
---422K
21.2G
avail
----5.88G
10G
Using the storage-pool-show command also displays any problems with storage pools, such as failed disks or
degraded RAID states.
Creating a storage pool also creates a disk library. After you create a storage pool, verify that a disk library was
created:
CLI network-admin@switch > disk-library-show layout vertical
switch:
name:
sharing:
import-share:
export-share:
pbg-nvos
disk-lib-pool-disk1
nfs
pbg-nvos:/disk-lib/pool-disk1/import
pbg-nvos:/disk-lib/pool-disk1/export
Look for available ISO images on the switch:
CLI network-admin@switch > iso-library-image-show
switch
-----pbg-nvos
label
----ubuntu-12
library
------iso-lib-pool-disk1
By default, creating a Netvisor KVM occurs on a non-rpool storage pool randomly chosen when you use the
netvisor-kvm-create command. To specify the storage pool for the Netvisor KVM, use the parameter
storage-pool pool-name when creating the Netvisor KVM.
Pluribus Networks Configuration Guide
139
www.pluribusnetworks.com
2. To create a Netvisor KVM from a bootable ISO image for temporary use, you can use the CentOS-6.5 ISO image on
the switch and add 2 GB of memory for it.
CLI network-admin@switch > netvisor-kvm-create name test vnet VNET33 iso-label
centOS-6.5-x86_64 enable storage-pool p1-testpool memory 2g cpus 2 hda-size
10g boot-order hdisk,cdrom hda-lib disk-lib-vnet1 hda-if ide
Netvm created. Please use netvm-interface-add to add interfaces and
netvm-start to boot.
3. Add a network interface to the Netvisor KVM:
CLI network-admin@switch > netvisor-kvm-interface-add netvm-name vm-temp if
mgmt
4. Verify the interface is added:
CLI network-admin@switch > netvisor-kvm-interface-show
netvisor-kvm-name nic
----------------- --vm-temp
ip
---
assignment mac
--------------
vm-temp.eth0::/0 none
vlan vxlan if
---------- --
66:0e:94:11:ae:cc 0
0
mgmt
5. Now, you can start the NetVM, using the netvisor-kvm-start command:
CLI network-admin@switch > netvisor-kvm-start name vm-temp
VM running. From outside switch, connect to vnc port :1.
Ex: vncviewer 172.17.245.201:1
The IP address for the VNC is the same as the IP address of the KVM interface.
6. To display the status of the Netvisor KVM, use the netvisor-kvm-show command:
CLI network-admin@switch > netvisor-kvm-show layout vertical
name:
type:
scope:
vnet:
vnet-service:
gateway:
memory(MB):
cpus:
vm-state:
boot-order:
iso-label:
hda-label:
hdb-label:
hdc-label:
hdd-label:
vnc-port:
vm-temp
netvm
fabric
corp-fabric
dedicated
::
2000
1
running
cdrom,hdisk
centOS-6.5
1
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
140
7. To access the Netvisor KVM virtual console, use a compatible VNC viewer.
vncviewer 172.17.245.201:1
TigerVNC Viewer for X version 1.0.0
...
8. The installation interface for the Ubuntu image is displayed.
Informational Note: The KVM exists until the switch is reset by a reboot or power loss. In this case, you
need to recreate the KVM.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Creating a Disk-based Netvisor KVM
To create a disk-based Netvisor KVM, use the Ubuntu ISO image, 2GB of memory, and create a virtual disk for the
Netvisor KVM. You can use the Netvisor KVM disk library created when you create the Netvisor KVM.
Pluribus Networks Configuration Guide
141
www.pluribusnetworks.com
1. Create the Netvisor KVM and disk library:
CLI network-admin@switch > netvisor-kvm-create name disk-vm vnet corp-fabric
iso-label unbuntu-12 memory 2g hda-size 5g hda-lib disk-lib-pool-disk1
Netvm created. Please use netvm-interface-add to add interfaces, and then
netvm-start to boot
2. Add a network interface to the Netvisor KVM, and then start the Netvisor KVM.
CLI network-admin@switch > netvisor-kvm-interface-add netvm-name disk-vm if
mgmt
CLI network-admin@switch > netvisor-kvm-start name disk-vm
VM running. From outside switch, connect to vnc port :2.
Ex: vncviewer 172.17.245.203:2
3. Display the Netvisor KVM information:
CLI network-admin@switch > netvisor-kvm-show layout vertical
name:
type:
scope:
vnet:
vnet-service:
gateway:
memory:
cpus:
vm-state:
boot-order:
iso-label:
hda-label:
hdb-label:
hdc-label:
hdd-label:
vnc-port:
disk-vm
netvm
fabric
corp-fabric
dedicated
::
2GB
1
running
cdrom,hdisk
ubuntu-12
netvm-disk-vm-hda
2
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
142
Creating a KVM by Importing an ISO Image
To create a NetVM from an imported ISO image, you must copy the image to the disk-library where you install the
NetVM.
1. Copy the ISO image to the disk library:
% cp vm-disk2.img /mnt/tmp/disk-lib/newpool/import
2. Verify that the image is available:
CLI network-admin@switch > disk-library-imports-showname disk-lib-newpool
name
---vm-disk2.img
3. Import the ISO image into the disk library:
CLI network-admin@switch > disk-library-image-import disk-library-name
disk-lib-newpool image-label vm-disk2 image-file vm-disk2.img
4. Create the NetVM that uses the disk image:
CLI network-admin@switch > netvisor-kvm-create name vm-disk2 vnet corp-fabric
hda-lable vm-disk2 memory 2g cpus 2
Netvm created. Please use netvm-insterface-add interfaces, and then
netvm-start to boot.
Adding Virtual Machine (VM) Instances to the Server-Switch
Bhyve images (VMM) provides support for virtual machines but provides better throughput than KVM.
Kernel-based Virtual Machine (KVM) is a Linux kernel virtualization hypervisor that can host different guest
operating systems. VMM is used in a similar manner as KVM, but does not support a graphical user interface (GUI).
Informational Note: nvOS does not have VM-compatible images in the ISO library. You must import
compatible images onto the switch.
You cannot run KVM and VM on the same switch. You must shut down any KVM instances before you can
start VM instances.
To create a VM for CentOS 6.5 with a 20G disk space, and 4G memory on VNET, centos, use the following steps:
Informational Note: VM supports only 1 CPU per virtual machine and does not support a graphical user
interface (GUI).
Pluribus Networks Configuration Guide
143
www.pluribusnetworks.com
1. Create the VMM disk and storage:
CLI network-admin@switch > netvisor-vm-create name centos6.5 vnet centos scope
fabric iso-label centos-6.5-86_64 memory 4g hda-size 20g
boot-at-console-connect true
Netvisor vm created. Please use interface-add to add interfaces and then
start to boot.
2. Add the interface to the VM:
CLI network-admin@switch > netvisor-vm-interface-add name centos6.5 vlan 100 if
mgmt
3. Start the VMM image:
CLI network-admin@switch > netvisor-vm-start name centos6.5
VM running. Use vmm-console to connect to VM
4. Log into the VM:
CLI network-admin@switch > netvisor-vm-console-login
5. Complete the VM configuration using the CLI interface for CentOS 6.5.
To display a list of VMs on the switch, use the following command:
CLI network-admin@switch > netvisor-vm-show format all layout vertical
id:
name:
type:
scope:
vnet:
vnet-service:
state:
location:
storage-pool:
gateway:
template:
memory:
cpus:
vm-state:
iso-label:
hda-label:
vmm-hda-if:
hdb-label:
vmm-hdb-if:
hdc-label:
vmm-hdc-if:
hdd-label:
vmm-hdd-if:
boot-at-console-connect:
delete-hda:
a0000dd:10
centos-6.5
netvmm
fabric
test-b
dedicated
enabled
techpubs-aquila1
rpool
::
no
4G
1
running
centOS-6.5-x86_64
netvisor-vm-centos6.5-hda
ahci-hd
ahci-hd
ahci-hd
ahci-hd
true
false
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
144
To view a list of VMM interfaces, use the netvisor-vm-interface-show command:
CLI network-admin@switch > netvisor-vm-interface-show format all layout
vertical
netvisor-vmm-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
alias-on:
exclusive:
nic-config:
nic-state:
netvisor-vmm-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
alias-on:
exclusive:
nic-config:
nic-state:
netvisor-vmm-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
alias-on:
exclusive:
nic-config:
nic-state:
netvisor-vmm-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
alias-on:
exclusive:
nic-config:
nic-state:
b33h1v3
eth0.106
::/0
none
66:0e:94:dd:69:df
106
0
mgmt
no
enable
down
test-bee
eth1.110
::/0
none
66:0e:94:dd:16:42
110
0
mgmt
no
enable
down
ubuntu-11
eth0.13
::/0
none
66:0e:94:dd:dd:02
13
0
mgmt
no
enable
down
centos65
eth1.101
::/0
none
66:0e:94:dd:1f:78
101
0
mgmt
no
enable
down
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
145
www.pluribusnetworks.com
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
146
Managing Linux VM Images
Linux NetVMs enable you to write software that runs directly on the switch with Linux OS. If the NetVM is
configured on a VNET with the scope fabric, then software that runs on the VMs has access to the complete set of
Pluribus Networks nvOS APIs which provide an open, programmatic interface to the network.
1. To display the list of all VMs on the switch, use the netvisor-kvm-show command.
2. To start the NetVM named vm-disk, use the netvisor-kvm-start command.
3. To modify the NetVM, use the netvisor-kvm-modify command.
CLI network-admin@switch > netvisor-kvm-modify name vm-disk [disable|enable]
memory cpus hda-size hda-lib boot-order iso-label hda-label hdb-label
hdc-label hdd-label
4. To reset a NetVM, use the netvisor-kvm-reset command.
5. To shutdown the NetVM, use the netvisor-kvm-shutdown command.
6. To immediately halt the NetVM, use the netvisor-kvm-kill command.
7. To permanently delete the NetVM, use the netvisor-kvm-delete command.
The disk library images with NetVM content are not automatically deleted when the NetVM is deleted. The images
remain available if you want to reinstall them. To delete the disk library image and free space in the disk library, use
the disk-library-image-remove command.
Changing the State of a NetVM
The command, netvisor-kvm-kill, is similar to pressing the power button for an extended period on the
virtual system with the NetVM. The command, netvisor-kvm-shutdown, sends an ACPL shutdown signal to
the NetVM and may display a dialog box with a message asking if you want to shutdown the NetVM. The command,
netvisor-kvm-reset sends an ACPI reset signal to the NetVM.
Since netvisor-kvm-shutdown and netvisor-kvm-reset send an ACPI signal to the NetVM, the NetVM
is running until the guest OS shuts it down. The command, netvisor-kvm-show may display a status of running
even after a state change command is issued.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
147
www.pluribusnetworks.com
Configuring and Implementing NetZones
Overview
NetZones allow you to execute code within the switches, and allows you to execute x86 Solaris code, either custom
programs or pre-compiled applications. NetVMs allow you to install x86 Linux distributions and execute x86 Linux
code, either custom programs or pre-compiled applications. Software installed in a NetZone or a NetVM can access
the nvOSAPIs which provide an open, programmatic interface to the network.
A NetZone or NetVM can implement one or more standard network interfaces which allows the NetZone or NetVM
to send and receive data on networks. The network interfaces can access the span and data network ports, and
vflow commands can send specific data to the network ports so applications can access the data.
Informational Note: The nvOS APIs are declared in the following C header files:
• /usr/include/nvc_client.h
• /usr/include/nvOS.h
The Java bindings are documented in /usr/java/doc/libnvos/index.html
Only C and Java APIs are supported by nvOS.
Configuring a NetZone
The following tasks assist you with creating an OpenSolaris NetZone.
1. Create a NetZone on the switch using the following command:
CLI network-admin@switch > netvisor-zone-create name netzone-solaris vnet
corp-fabric user admin
netzone admin password:*******
confirm netzone admin password:*******
CLI network-admin@switch > netvisor-zone-show layout vertical
name:
type:
scope:
vnet:
vnet-service:
state:
gateway:
user:
password:
floodlight-enable:
netzone-solaris
netzone
fabric
corp-fabric
dedicated
enabled
::
admin
no
The output specifies the name of the NetZone as netzone-solaris with the scope of fabric. The scope of the NetZone
is the same as the VNET where you created the NetZone. In this case, the default VNET has the scope of fabric and
the NetZone has access to all switches in the fabric.
Informational Note: When you create a Netvisor zone, the zone is created in the rpool storage pool
unless you specify a datapool location to create the zone. Use the storage-pool parameter to
specify a storage pool.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
148
2. To allow traffic to flow through the NetZone, you create an interface and add an IP address:
CLI network-admin@switch > netvisor-zone-interface-add netzone-name
netzone-solaris if data ip 172.17.176.11/16
CLI network-admin@switch > netvisor-zone-interface-show layout vertical
netzone-name:
ip:
assignment:
mac:
vlan:
vxlan:
if:
netzone-solaris
172.17.176.11/16
static
66:0e:94:11:26:5c
0
0
data
The NetZone is assigned the IP address 172.17.176.11 on the switch interface for data. If you want access to the
NetZone through the management ports, then you should create another interface and add the parameter, mgmt,
instead of data.
3. To access the NetZone, use SSH and any terminal application:
% ssh 172.17.176.11 -t admin
Password:********
Last login: Tue Jan 31 22:07:31 2012 from 172.17.176.100
Pluribus Networks, Inc. SunOS 5.11 pn-snv137 January 2012
4. Display the sample code installed in the admin home directory:
-bash-4.0$ ls -lr
.:
total 3
drwxr-xr-x
6 pbg
./samples:
total 12
drwxr-xr-x
drwxr-xr-x
drwxr-xr-x
drwxr-xr-x
...
2pbg
2pbg
2pbg
2pbg
staff
6 May 30 19:03 samples
staff
staff
staff
staff
5
5
5
5
May
May
May
May
30
30
30
30
19:03
19:03
19:03
19:03
Events
Snoop
events
nvsnoop
-bash-4.0$ cd samples/nvsnoop/
-bash-4.0$ ls
Makefile
README
nvsnoop.c
5. gcc and gmake are preinstalled in the developer zone. Use gmake to build the sample code:
-bash-4.0$ gmake
gcc -pthreads -c nvsnoop.c
gcc -pthreads -o nvsnoop nvsnoop.o -lnvOS -lsocket -lnsl
Pluribus Networks Configuration Guide
149
www.pluribusnetworks.com
6. You can now run the nvsnoop sample program. Use the admin password that you configured when you installed
the switch.
-bash-4.0$ nvsnoop --vnet myfabric-global --vlan 5 --user network-admin \
--pass <password>
Displaying captured packets. Press Ctrl-C to stop.
switch: b000038, flow: b000038:25, port: 15, size: 102
src-mac: 02:08:20:23:a4:da, dst-mac: 02:08:20:67:ca:2f, vlan: 5, etype: ip
src-ip: 192.168.3.125, dst-ip: 192.168.3.115, proto: icmp
switch: b000038, flow: b000038:25, port: 54, size: 102
src-mac: 02:08:20:67:ca:2f, dst-mac: 02:08:20:23:a4:da, vlan: 5, etype: ip
src-ip: 192.168.3.115, dst-ip: 192.168.3.125, proto: icmp
To delete the NetZone, use the netzone-delete command.
The NetZone is configured with the created user, in this case, admin, as a sudo-er which means that the user can be
the root and install software packages or configure the NetZone to facilitate the creation of the correct environment
for your application.
If the NetZone is configured as part of the global VNET, you can use privileged nvOS CLI commands and call privilege
nvOS API library routines.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
150
Configuring vRouter Services
 Configuring Prefix Lists for BGP and OSPF
 Configuring Packet Relay for DHCP Servers
 Configuring Hardware Routing for a vRouter
 Configuring BGP on a vRouter
 Configuring Open Shortest Path First (OSPF)
 Configuring Routing Information Protocol (RIP)
 Configuring Static Routes
 Adding IGMP Static Joins to a vRouter
 Configuring Virtual Router Redundancy Protocol
 Configuring Multicast Listener Discovery (MLD)
Overview
Virtual Routers (vRouters) are an important part of fabric functionality. For example, for a VNET to communicate
with other VNETs, or networks external to the fabric, it may need a vRouter that spans the VNET and the external
network. vRouter commands can only be executed at the fabric level by the fabric administrator, so there is no
network disruption by VNET administrators. You cannot use the vRouter commands as a VNET administrator.Routing
protocols essentially work the same way on virtual routers as physical routers. Detailed information about routing
protocols is not covered in this overview.
The vRouter feature supports common routing protocols such as BGP, OSPF, RIP, and static routes.
To create a vRouter on the global VNET, and create a gateway between two networks that connect to the switch
ports, use the following command:
CLI network-admin@switch > vrouter-create name default-gateway vnet
fabricname-global
CLI network-admin@switch > vrouter-interface-add vrouter-name default-gateway
ip 172.16.23.33/24 if data
CLI network-admin@switch > vrouter-interface-add vrouter-name default-gateway
ip 10.9.18.147/16 if data
You just created an interface for the external network (10.9.18.147) and the internal network (172.16.23.33). By
default a static route is created between interfaces added to a vRouter.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
151
www.pluribusnetworks.com
Configuring Prefix Lists for BGP and OSPF
Prefix lists allow you to permit or deny host IP addresses from route distribution in BGP and OSPF configurations. To
configure prefix lists for BGP, this example assumes that you have a vRouter configured for BGP, vrouter-bgp, and
you want to deny the IP address, 172.26.0.0 with the netmask 255.255.0.0, sequence number 5, and minimum
prefix length 17 bits:
CLI network-admin@switch > vrouter-prefix-list-add vrouter-name vrouter-bgp
name deny-bits action deny prefix 172.26.0.0 netmask 255.255.0.0 seq 5
min-prefix-len 17
This prefix list rejects any subnets of 172.26.0.0/16 with prefixes 17 bits or longer. For example, the subnets
172.26.16.9/30 and 172.26.101.0/24 are rejected from route distribution.
The sequence number allows you to insert or remove new lines in a prefix list as well as at the beginning or end. It is
recommended that you increment the sequence numbers by 10 so you can easily add or subtract lists from the
configuration.
Configuring Packet Relay for DHCP Servers
You can configure a vRouter to relay DHCP requests from local clients to a centralized DHCP server. Because the
initial DHCP request arrives from a client that typically does not have an IP address, the client must find the DHCP
server using a Layer 2 broadcast.
The DHCP server must know the subnet and the MAC address of the client before the server can allocate an IP
address to the client. The DHCP server needs the subnet information to ensure that the IP address that the client
receives can work on the client’s subnet. The MAC address is necessary so that the DHCP server can find any
information that is unique to the client.
When you configure the vRouter as a DHCP proxy, the vRouter converts the local broadcast packet from the client to
a unicast packet and forward it to the server.
Because the DHCP client does not have an IP address when it sends the DHCP request packet, the client uses the IP
address, 0.0.0.0, as the source IP address and the general broadcast address 255.255.255.255 for the destination.
The vRouter replaces the source address with the IP address assigned to the interface where the request is received,
and replaces the destination IP address with the address you specify in the vRouter packet-relay command.
To configure packet-relay for a DHCP server with the IP address 172.16.21.34 and vRouter interface eth11.100, use
the following syntax:
CLI network-admin@switch > vrouter-packet-relay add vrouter-name vrouter-dhcp
forward-proto dhcp forward-ip 172.16.21.34 nic eth11.100
Once you’ve added the configuration, you cannot modify it. If you made a mistake or want to add a new
configuration, you must use the vrouter-packet-relay-remove command.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Configuring Hardware Routing for a vRouter
Hardware routing implements the same mechanisms as software routing for the control plane. You create interfaces
on hardware routers and map them to VNICs in the vRouter zone. You can configure up to seven (7) hardware
routers on a platform.
The supported protocols are as follows:
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
152
 OSPF — OSPF does not use a TCP/IP transport protocol such as UDP or TCP, but is encapsulated in the IP
datagram with protocol number 89. OSPF uses multicast addressing for route flooding on a broadcast domain.
For nonbroadcast network, special provisions in the configuration facilitate neighbor discovery. OSPF reserves
the multicast addresses 224.0.0.5/6 for IPv4 or FF02::5/6 for IPv6.
 BGP — BGP uses TCP and port number 179.
 RIP — uses the following parameters:
• RIPv1 — IPv4 uses UDP and port 520, and advertise address - broadcasting
• RIPv2 — IPv4 uses UDP and port 520, and advertise address - 224.0.0.9
• RIPng — IPv6 uses UDP and port 521, and advertise address - FF02::9
 PIM — IPv4 uses protocol 103 with multicast address 224.0.0.13
To create a hardware routing on a vRouter, hwtest, on VNET, fabricname-global, use the following
command:
CLI network-admin@switch > vrouter-create hwtest vnet fabricname-global
router-type hardware
Use the same commands as software routing to add protocols and interfaces.
Configuring Multicast Listener Discovery (MLD)
Multicast Listener Discovery (MLD) is a Layer 3 multicast protocol used between IPv6 hosts and routers similar to
how IGMP is used for IPv4. MLD snooping allows a switch to examine MLD packets and make forwarding decisions
based on their content.
MLD Snooping constrains IPv6 multicast traffic at Layer 2 by configuring Layer 2 LAN ports dynamically to forward
IPv6 multicast traffic only to those ports that want to receive it.
MLD Snooping supports MLDv1 and MLDv2.
MLD snooping constrains IPv6 multicast traffic at Layer 2 by configuring Layer 2 LAN ports dynamically to forward
IPv6 multicast traffic only to those ports that want to receive it.
When MLD Snooping is not configured, the default behavior is to forward all multicast traffic to all the switch ports,
impacting switch performance.
To modify MLD Snooping parameters from local to fabric scope, use the following syntax:
CLI network-admin@switch > mld-snooping-modify scope fabric enable
To display MLD Snooping configurations, use the mld-snooping-show command:
CLI network-admin@switch > mld-snooping-show
switch:
enable:
switch:
enable:
Leaf01
no
Spine01
no
Pluribus Networks Configuration Guide
153
www.pluribusnetworks.com
To display MLD statistics, use the mld-stats-show command:
CLI network-admin@switch > mld-stats-show
switch
------pubdev02
pubdev02
pubdev02
pubdev02
vlan
---1
3
5
7
v1-queries
---------0
0
0
0
v2-queries
---------0
0
0
0
v1-member-reports
----------------0
0
0
0
v2-member-reports
----------------0
0
0
0
done-group
---------0
0
0
0
Enabling MLD Static Groups
You can create MLD static group membership to test multicast forwarding without a receiver host. When you enable
MLD static group membership, data is forwarded to an interface without that interface receiving membership
reports from downstream hosts.
To configure an MLD static group with the multicast IP address of 224.0.1.0, use the following syntax:
CLI network-admin@switch > mld-static-group-create group-ip 224.0.1.0 vlan 5
ports all
The group IP address is a global scope multicast IP address between 224.0.1.0 and 239.255.255.255.
To display the configuration, use the mld-static-group-show command:
CLI network-admin@switch > mld-static-group-show
switch
group-ip vlan ports
-------- --------- ---- ----leaf01
224.0.1.0 5
1-255
Configuring MLD Static Sources
The Multicast Listener Discovery (MLD) Protocol manages the membership of hosts and routers in multicast groups.
IP version 6 (IPv6) multicast routers use MLD to learn, for each of their attached physical networks, which groups
have interested listeners. Each router maintains a list of host multicast addresses that have listeners for each subnet,
as well as a timer for each address. However, the router does not need to know the address of the listeners—just the
address of the hosts. The router provides addresses to the multicast routing protocol it uses; this ensures that
multicast packets are delivered to all subnets where there are interested listeners. In this way, MLD is used as the
transport for the Protocol Independent Multicast (PIM) protocol.
You can add MLD static sources using the following command:
CLI network-admin@switch > mld-static-source-create source-ip 10.9.100.100
group-ip 224.0.1.0 vlan 5 ports all
To display the configuration, use the mld-static-source-show command:
switch
-------pubdev02
pubdev02
pubdev02
pubdev02
pubdev02
pubdev02
pubdev02
...
group-ip
--------224.0.1.0
224.0.1.0
224.0.1.0
224.0.1.0
224.0.1.0
224.0.1.0
224.0.1.0
vlan
---5
5
5
5
5
5
5
source-ip
-----------10.9.100.100
10.9.100.100
10.9.100.100
10.9.100.100
10.9.100.100
10.9.100.100
10.9.100.100
host_ip
------0.0.0.0
0.0.0.0
0.0.0.0
0.0.0.0
0.0.0.0
0.0.0.0
0.0.0.0
ports
----255
254
253
252
251
250
249
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
154
To display switches participating in MLD Snooping, use the mld-switches-show command:
CLI network-admin@switch > mld-switches-show
switch
-------------Leaf01
Leaf01
Leaf01
Leaf01
node-ip
------fe80::
::
::
::
vlan
---1
1
2
3
port
---57
3
3
3
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
155
www.pluribusnetworks.com
Configuring BGP on a vRouter
Border Gateway Protocol (BGP) is a path-vector protocol and is the most commonly used routing protocol on the
Internet. It advertises the paths required to reach a certain destination. BGP is also a Layer 4 protocol that sits on top
of TCP, and is simpler than Open Shortest Path First (OSPF). In Figure 1 Configuring BGP for Two VLANs, you want
network traffic from the source host to reach the destination host. But when different VLANs are configured, the
source host traffic is not aware of the route between the source host and the destination host. However, there is a
VLAN that spans VLAN 33 and VLAN 55. You solve this problem by configuring BGP in the same Autonomous System
(AS) 100 that sends traffic over VLAN 35. This allows the source host to learn the route to the destination host.
Using a loopback address for peering is useful when there are multiple paths between the BGP peers which would
otherwise tear down the BGP session if the physical interface us ed for establishing goes down. It also allows the
vRouters running BGP with multiple links between them to load balance over the available paths.
Figure 1: Configuring BGP for Two VLANs
This example assumes that you have two VLANs, VLAN33 and VLAN55. Also, that you have added ports to the
configuration.
Begin by configuring vRouter1, a software vRouter, on VLAN 33 with the BGP information:
CLI network-admin@switch > vrouter-create name vrouter1 vnet fabricname-global
router-type software bgp-as 100 bgp-redist-connected-metric none
Additional BGP parameters include the following:
 bgp-redist-static-metric — redistribute static BGP route metric number
 bgp-redist-connected-metric — redistribute connected BGP route metric
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
156
 bgp-redist-rip-metric
— redistribute BGP into RIP process metric
 bgp-redist-ospf-metric — redistribute BGP into OSPF process metric
 bgp-cluster-id — the ID assigned to the BGP cluster.
 bgp-max-paths — maximum number of BGP paths
 bgp-ibgp-multipath — allow the BGP vRouter to select multiple paths for load sharing.
 bgp-bestpath-as-path — allow BGP to use the best path for traffic forwarding.
 bgp-dampening|no-bgp-dampening — suppress flapping routes so they are not advertised.
 bgp-graceful-restart|no-bgp-graceful-restart — mechanism for BGP that helps minimize
the negative effects on routing caused by BGP restart.
 bgp-stalepath-time — how long a router waits before deleting stale routes after an end of record (EOR)
message is received from the restarting router.
Add the IP addresses and VLANs:
CLI network-admin@switch > vrouter-interface-add vrouter-name vrouter1 ip
10.16.35.33/24 vlan 35
CLI network-admin@switch > vrouter-interface-add vrouter-name vrouter1 ip
10.16.33.1/24 vlan 33
Add the BGP information:
CLI network-admin@switch > vrouter-bgp-add vrouter-name vrouter1 neighbor
10.16.35.55 remote-as 100
CLI network-admin@switch > vrouter-bgp-add vrouter-name vrouter1 network
10.16.33.0/24
Pluribus Networks Configuration Guide
157
www.pluribusnetworks.com
Display the interface information for vrouter33:
CLI network-admin@switch > vrouter-interface-show format all layout vertical
vrouter-name:
vrouter33
nic:
eth1.33
ip:
10.9.100.100/16
assignment:
static
mac:
66:0e:94:30:c6:92
vlan:
33
vxlan:
0
if:
data
alias-on:
exclusive:
no
nic-config:
enable
nic-state:
up
secondary-macs:
vrouter-name:
vrouter33
nic:
eth2.33
ip:
192.168.42.11/24
assignment:
static
mac:
66:0e:94:30:25:5e
vlan:
33
vxlan:
0
if:
data
alias-on:
exclusive:
no
nic-config:
enable
nic-state:
up
secondary-macs:
If you want to filter IP hosts, you can add prefix lists to the BGP configuration. See Configuring Prefix Lists for BGP
and OSPF.
Then, configure vRouter2 on VNET 55:
CLI network-admin@switch > vrouter-create name vrouter2 vnet fabricname-global
router-type software bgp-as 100 bgp-redist-connected-metric none
Add the IP addresses and VLANs:
CLI network-admin@switch > vrouter-interface-add vrouter-name vrouter2 ip
10.16.35.55/24 vlan 35
CLI network-admin@switch > vrouter-interface-add vrouter-name vrouter2 ip
10.16.55.1/24 vlan 55
Then add the BGP information:
CLI network-admin@switch > vrouter-bgp-add vrouter-name vrouter2 neighbor
10.16.35.33 remote-as 100
CLI network-admin@switch > vrouter-bgp-add vrouter-name vrouter2 network
10.16.55.0/24
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
158
And finally, add the loopback address:
CLI network-admin@switch > vrouter-loopback-interface-add vrouter-name
vrouter1 index 5 ip 1.1.1.1
The index value is a number that uniquely identifies the vRouter in the AS.
Display the vRouter BGP configuration:
CLI network-admin@switch > vrouter-bgp-show format all layout vertical
vrouter-name:
ip:
neighbor:
remote-as:
next-hop-self:
route-reflector-client:
override-capability:
soft-reconfig-inbound:
max-prefix-warn-only:
vrouter-name:
ip:
network:
vrouter-name:
ip:
neighbor:
remote-as:
next-hop-self:
route-reflector-client:
override-capability:
soft-reconfig-inbound:
max-prefix-warn-only:
vrouter-name:
ip:
network:
vrouter33
10.16.35.55
10.16.35.55
100
no
no
no
no
no
vrouter33
10.16.33.0
10.16.33.0/24
vrouter55
10.16.35.33
10.16.35.33
100
no
no
no
no
no
vrouter55
10.16.55.0
10.16.55.0/24
To reset BGP neighbors, use the vrouter-bgp-neighbor-reset command.
Pluribus Networks Configuration Guide
159
www.pluribusnetworks.com
To display BGP neighbors, use the vrouter-bgp-neighbor-show command.
CLI network-admin@switch > vrouter-bgp-neighbor-show
vrouter-name:
neighbor:
ver:
remote-as:
msg_rcvd:
msg_sent:
tblver:
inQ:
outQ:
up/down:
state/pfxrcd:
vrouter-name:
neighbor:
ver:
remote-as:
msg_rcvd:
msg_sent:
tblver:
inQ:
outQ:
up/down:
state/pfxrcd:
vrouter1
10.9.100.201
4
100
11
19
0
0
0
00:54:04
Connect
vrouter2
10.9.100.101
4
100
12
18
0
0
0
00:53:37
Connect
Additional BGP Parameters
There are additional BGP parameters that you can use to optimize your BGP network. Add any of the following
parameters:
— a value for external BGP to accept or attempt BGP connections to external peers, not
directly connected, on the network. This is a value between 1 and 255.
 ebgp-multihop
vrouter — the source IP address of BGP packets sent by the router. This parameter is
required if you want BGP to perform peering over a loopback interface.
 update-source
 prefix-list-in — specify a list of incoming prefixes for route redistribution.
 prefix-list-out — specify a list of outgoing prefixes for route redistribution.
 override-capability — override the result of capability negotiation with the local configuration. This
parameter allows you to ignore a remote peer’s capability value.
 soft-reconfig-inbound — defines the route refresh capability by allowing the local device to reset
inbound routing tables dynamically by exchanging route refresh requests to supporting peers.
 max-prefix — allows you to specify the maximum number of IP prefixes to filter.
 max-prefix-warn — add a parameter to warn when the maximum number of prefixes is reached.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
160
Configuring Open Shortest Path First (OSPF)
Open Shortest Path First (OSPF) is a robust link-state interior gateway protocol (IGP). You can use it when Router
Internet Protocol (RIP) is not enough for your network or when you need fast convergence on the network. It uses
Autonomous Systems (AS) and the concept of Areas which allows further segmentation on the network.
OSPF uses link-state information to make routing decisions, and make route calculations using the shortest path first
(SPF) algorithm. Each vRouter configured for OSPF floods link-state advertisements throughout the AS or area that
contains information about the router’s attached interfaces and routing metrics.
You can add more configuration options, such as hello intervals, for OSPF using the
vrouter-interface-config commands. In addition, you can add stub or not-so-stubby areas to the OSPF
configuration.
You can also manually change the OSPF cost for the configuration. Cost is the metric used by OSPF to judge the
feasibility of a path. If you specify 0 as the cost, the vRouter automatically calculates the cost based on the
bandwidth of the interface.
Informational Note: For switches with ONVL, the only available VNET is a global VNET created when a
fabric is created for the first time. Use tab complete in the CLI to display the VNET and continue the
configuration.
In this example, you configure OSPF for two vRouters with an area of 5. The network has the following configuration:
 VLAN 35 with IP addresses 10.16.35.0/24
 VLAN 45 with IP addresses 10.16.55.0/24
Figure 1: OSPF
1. First, create the vRouter for VNET33, vrouter1.
CLI network-admin@switch > vrouter-create name vrouter1 vnet fabricname-global
Pluribus Networks Configuration Guide
161
www.pluribusnetworks.com
2. Add vRouter interfaces to the vRouter:
CLI network-admin@switch > vrouter-interface-add vrouter-name vrouter1 ip
10.16.35.1 netmask 24 vlan 35 if data nic-enable
CLI network-admin@switch > vrouter-interface-add vrouter-name vrouter1 ip
10.16.55.1 netmask 24 vlan 55 if data nic-enable
3. Add the subnets, 10.16.35.0/24 and 10.16.45.0/24, to VLAN33 with the area 0:
CLI network-admin@switch > vrouter-ospf-add vrouter-name vrouter1 network
10.16.35.0/24 ospf-area 0
4. Add the second IP address with the area 0.
CLI network-admin@switch > vrouter-ospf-add vrouter-name vrouter1 network
10.16.55.0/24 ospf-area 0
5. Add interfaces for OSPF hello intervals of 30 seconds:
CLI network-admin@switch > vrouter-interface-config-add name vrouter1 nic
eth0.35 ospf-hello-interval 30 ospf-cost 0
CLI network-admin@switch > vrouter-interface-config-add name vrouter1 nic
eth0.55 ospf-hello-interval 30 ospf-cost 0
If you specify 0 as the cost value, the vRouter calculates the OSPF cost automatically based on the bandwidth of the
interface.
When you modify the OSPF hello interval, the ospf-dead-interval is automatically reset to 4 times the hello interval.
6. Display the configuration by using the vrouter-ospf-show command:
CLI network-admin@switch > vrouter-ospf-show layout vertical
vrouter-name:
network:
netmask:
ospf-area:
vrouter-name:
network:
netmask:
ospf-area:
stub-area:
stub-type:
ospf-hello-interval:
metric:
vrouter1
10.16.35.0
24
0
vrouter1
10.16.55.0
24
0
11
stub
30
34
The metric value can reflect the cost of routes advertised as OSPF routes. It may also reflect the cost of routes
advertised with other protocols.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
162
Adding Areas and Prefix Lists to OSPF
You can now configure OSPF areas as a stub area, stub-no-summary area, or a not so stubby area (NSSA). Stub areas
see detailed routing information from other areas, but only summary information about networks outside of the AS.
Stub-no-summary areas summarize external routes and routes from other areas. Routers in this type of area only
see routing information local to their area. Not so stubby areas (NSSA) connects to the external network by
introducing a Link State Advertisement (LSA) used within the area to carry external routes originating with boundary
routers connected to this area.
To add a stub area to vRouter, vrouter-ospf, with area 100, use the following command:
CLI network-admin@switch > vrouter-ospf-area-add vrouter-name vrouter-ospf
area 100 stub-type stub
The parameter, stub-type, is a required parameter.
In addition, you can add prefix lists to filter host IP addresses. To add prefix lists to OSPF areas, see Configuring Prefix
Lists for BGP and OSPF.
Pluribus Networks Configuration Guide
163
www.pluribusnetworks.com
Configuring Routing Information Protocol (RIP)
Routing Information Protocol (RIP) is the oldest routing protocol and provides networking information to routers.
Routers need to know what networks are available and how the distance required to reach it.
RIP is a distance vector protocol, and uses hop counts to determine distance and destination. Every 30 seconds, RIP
sends routing information to UDP port 50. If the router is default gateway, it advertises itself by sending 0.0.0.0 with
a metric of 1.
Figure 1:I RIP
1. Create vRouter1 on VNET33:
CLI network-admin@switch > vrouter-create name vrouter1 vnet fabricname-global
You can also specify how RIP routes are distributed using the parameter, rip-redistribute
static|connected|ospf|bgp.
2. Add network 10.16.33.0/24 to vrouter1:
CLI network-admin@switch > vrouter-rip-add vrouter-name vrouter1 network
10.16.33.0/24 metric 2
3. Add network 10.16.35.0/24 to vrouter1:
CLI network-admin@switch > vrouter-rip-add vrouter-name vrouter1 network
10.16.55.0/24 metric 2
4. To view the configuration, use the vrouter-rip-show command. This displays all RIP routes configured using
the vrouter-rip-add command.
To view RIP routes not configured using the vrouter-rip-add command, use the
vrouter-rip-routes-show command.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
164
Configuring Static Routes
vRouters forward packets using either routing information from route tables manually configured or routing
information calculated using dynamic routing algorithms.
Static routes define explicit paths between two vRouters and are not automatically updated. When network changes
occur, you have to reconfigure static routes. However, static routes use less bandwidth than dynamic routes.
Figure 1: Configuring a Static Route
In this example, you configure a static route on vRouter1 for the network, 172.16.10.10/24 with a gateway IP
address, 172.16.20.1:
CLI network-admin@switch > vrouter-static-route-add vrouter-name vrouter1
network 172.16.10.10/24 gateway-ip 172.16.20.1
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
165
www.pluribusnetworks.com
Adding IGMP Static Joins to a vRouter
Internet Group Membership Protocol (IGMP) is used to inform vRouters about multicast groups that hosts want to
join on the network, and vRouters use IGMP to verify that a host is interested in listening to a multicast group.
You can add IGMP static group membership to a vRouter in a VNET. When you enable static group membership, data
is forwarded to an interface without the interface receiving membership reports from downstream hosts. This
allows fast switching for multicast traffic.
You must create IGMP static groups before configuring IGMP static joins. To configure IGMP static groups, use the
following command:
CLI network-admin@switch > igmp-static-group-create group-ip 239.4.9.3 vlan 33
ports 5-7
To configure an IGMP static join for group 239.4.9.3, and source IP address 192.0.2.3, use the following command:
CLI network-admin@switch > vrouter-igmp-static-join-add vrouter-name vrouter1
name igmp-vrouter-group group-ip 239.4.9.3 source-ip 192.0.2.3 interface
vrouter33
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
166
Configuring Virtual Router Redundancy Protocol
Virtual Router Redundancy Protocol (VRRP) is an election protocol that enables virtual routing functions for a master
or standby routing infrastructure for a given IP address. A virtual router is defined by a virtual router identifier (VRID)
and a virtual router IP address (VIP). The scope of the virtual routers is restricted to a single VLAN.
VRRP provides information on the state of a virtual router, not the routes processed and exchanged by the router. It
increases the availability and reliability of routing paths by automatic gateway selections on an IP subnetwork.
VRRP provides rapid transition from master to standby and from standby to master. The master router sends
advertisements every second. If the master VRRP advertisements are not received within a window of time, three
(3) seconds, then the standby virtual router becomes the master virtual router and begins performing routing for
the virtual router. If the master router becomes active again, it can become the master again or allow the standby to
continue as the master router. The role depends on the value assigned to VRRP priority.
Configuring VRRP Priority
The Priority is a value used by the VRRP router for master election. The valid priority range for a virtual router is from
1 to 254. 1 is the lowest priority and 254 is the highest priority. The default value for standby routers is 100. Higher
values indicate higher priority for the virtual router.
Configuring the VRRP ID
The Virtual Router Identifier is a configurable value between 1 and 255. There is no default value.
Example Configuration
In this example, you have the following configurations on two switches (SW1 and SW2) on the network:
 VLAN 100 with IP address range 192.168.11.0/24
 VNET with the name vrrp-router and scope fabric
1. On SW1, configure a vRouter:
CLI network-admin@switch > vrouter-create name vrrp-rtr1 vnet vrrp-router
router-type software enable
VRRP is supported on hardware and software routers, but for this example, software is the router type on both
switches.
Informational Note: You can configure up to seven hardware routers for VRRP, and only one VLAN for
VRRP.
2. Add the first vRouter interface:
CLI network-admin@switch > vrouter-interface-add vrouter-name vrrp-rtr1 ip
192.168.11.3 netmask 24 vlan 100 if data
Pluribus Networks Configuration Guide
167
www.pluribusnetworks.com
3. Use the vrouter-interface-show command to see the name of the interface:
CLI network-admin@switch > vrouter-interface-show format all layout vertical
vrouter-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
alias-on:
exclusive:
nic-config:
nic-state:
vrrp-rtr1
eth0.100
192.168.11.3/24
static
66:0e:94:dd:18:c4
100
0
data
no
enable
up
4. Now create the VRRP interface:
CLI (switch)>vrouter-interface-add vrouter-name vrrp-rtr1 ip 192.168.11.2 netmask 24
vlan 100 if data vrrp-id 10 vrrp-primary eth0.100 vrrp-priority 100
5. Now, create the vRouter and interfaces on SW2:
CLI network-admin@switch > vrouter-create name vrrp-rtr2 vnet vrrp-router
router-type software dedicated-vnet-service
Note that the second vRouter is created as a dedicated VNET service because a VNET supports only one shared
vRouter service.
6. Add the vRouter interface:
CLI network-admin@switch > vrouter-interface-add vrouter-name vrrp-rtr2 ip
192.168.11.4 netmask 24 vlan 100 if data
7. Use the vrouter-interface-show command to see the name of the interface:
CLI network-admin@switch > vrouter-interface-show format all layout vertical
vrouter-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
alias-on:
exclusive:
nic-config:
nic-state:
vrrp-router2
eth2.100
192.168.11.3/24
static
66:0e:94:21:a9:6c
100
0
data
no
enable
up
8. Now create the VRRP interface:
CLI network-admin@switch > vrouter-interface-add vrouter-name vrrp-rtr2 ip
192.168.11.2 netmask 24 vlan 100 if data vrrp-id 10 vrrp-primary eth0.100
vrrp-priority 50
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
168
9. Display the information about the VRRP setup:
CLI network-admin@switch > vrouter-interface-show format all layout vertical
vrouter-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
alias-on:
exclusive:
nic-config:
nic-state:
vrouter-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
alias-on:
exclusive:
nic-config:
nic-state:
vrrp-id:
vrrp-primary:
vrrp-priority:
vrrp-state:
vrouter-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
alias-on:
exclusive:
nic-config:
nic-state:
vrouter-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
alias-on:
exclusive:
nic-config:
nic-state:
vrrp-id:
vrrp-router1
eth0.100
192.168.11.3/24
static
66:0e:94:dd:18:c4
100
0
data
no
enable
up
vrrp-router1
eth1.100
192.168.11.2/24
static
00:00:5e:00:01:0a
100
0
data
no
enable
up
10
eth1.100
100
master
vrrp-router2
eth3.100
192.168.11.4/24
static
66:0e:94:21:54:07
100
0
data
no
enable
up
vrrp-router2
eth3.100
192.168.11.2/24
static
00:00:5e:00:01:0a
100
0
data
no
enable
down
10
Pluribus Networks Configuration Guide
169
www.pluribusnetworks.com
vrrp-primary: eth3.100
vrrp-priority: 50
vrrp-state:
slave
When you intentionally disable the VRRP interface, the slave interface becomes the master interface:
vrouter-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
alias-on:
exclusive:
nic-config:
nic-state:
vrrp-id:
vrrp-primary:
vrrp-priority:
vrrp-state:
vrrp-router2
eth3.100
192.168.11.1/24
static
00:00:5e:00:01:0a
100
0
data
no
enable
up
10
eth3.100
50
master
When you re-enable the VRRP interface, it becomes the master again, and the second interface returns to the slave:
vrouter-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
alias-on:
exclusive:
nic-config:
nic-state:
vrrp-id:
vrrp-primary:
vrrp-priority:
vrrp-state: slave
vrrp-router2
eth3.100
192.168.11.2/24
static
00:00:5e:00:01:0a
100
0
data
no
enable
down
10
eth3.100
50
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
170
Configuring Virtual Load Balancing
Virtual load balancing (vLB) uses virtual servers instead of physical servers to balance traffic across the network.
Each virtual server points to a cluster of services that reside on one or more physical hosts.
VLB uses the following transactions:
1. The client attempts to connect to the service on the load balancer.
2. The load balancer accepts the connection and then decides which host receives the connection. The port and destination IP address are changed to match the service of the selected host.
3. The host accepts the connection and responds to the original source, the client, through the default route which
is the load balancer.
4. The load balancer intercepts the return packet from the host and changes the source IP and port to match the virtual server IP and port, and forwards the packet back to the client.
5. The client receives the return packet and continues the process.
VLB uses four different algorithms to control and distribute traffic as well as load distribution and server selection.
 roundrobin — In a round-robin algorithm, the load balancer assigns requests to a list of servers on a rotating
basis. Once a server is assigned a request, the server moves to the bottom of the list.
 hash-ip — In the source IP hash method, the load balancer selects a server based on the hash value of the
source IP address of the incoming request.
 hash-ip-port — In the source virtual IP, port hash method, the load balancer selects a server based on the
hash value of the source IP address, and the source port of the incoming request.
 hash-ip-vip — In the source IP, VIP hash method, the load balancer selects a server based on the hash value
of the source IP address, and the destination IP address of the incoming requests.
If you already have servers that you want to use for VLB, you can following the instructions below. If you want to
install Ubuntu servers as virtual machines on the switch, see Configuring Virtual Load Balancing with Ubuntu 11.04
Servers and nvOS.
If you are configuring VLB as a dedicated service on a VNET or you have not defined network interfaces for the VNET,
use the vlb-interface-add command to create the vNICs.
CLI network-admin@switch > vlb-create name vlb-vnet1 vnet vnet1
dedicated-vnet-service
You need two interfaces to configure VLB: one for the external address and one for the internal address. To create
the interfaces, use the following commands:
CLI network-admin@switch > vlb-interface-add vlb-name vlb-vnet1 ip
192.168.100.27 netmask 24 assignment none vlan 57 if data
CLI network-admin@switch > vlb-interface-add vlb-name vlb-vnet1 ip 10.10.10.113
netmask 24 assignment none vlan 58 if data
171
Pluribus Networks nvOS Version 2.3.2
Display the configuration information:
CLI network-admin@switch > vlb-interface-show vlb-name vnet1-vlb layout
vertical
vlb-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
vlb-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
vnet1-vlb
vnet1.mgr.eth0
10.10.10.113/24
static
66:0e:94:4b:b8:0c
123
0
data
vnet1-vlb
vnet1.mgr.eth1
192.168.100.27/24
static
66:0e:94:4b:9d:cc
124
0
data
Create a VLB to balance TCP port 80 (HTTP) requests in full NAT mode between the external and internal interfaces.
Full NAT mode sends all traffic to and from the servers and route through the load balancer.
CLI network-admin@switch > vlb-group-add vlb-name vnet1-vlb name vnet1-vlb-http
topology full-nat proto tcp start-port 80 ext-interface vnet.mgr.eth0
int-interface vnet1.mgr.eth1
When you create a vLB group, you can also add the following parameters:
 vip — the destination IP address for incoming requests
 proxy-src-ip — the proxy host source IP address
 proxy-src-netmask — the proxy host source netmask
 start-port — the starting port of the vLB group
 end-port — the ending port of the vLB group
 healthcheck — the name of a healthcheck configuration
CLI network-admin@switch > vlb-group-show layout vertical
vlb-name:
vnet1-vlb
name:
vnet1-vlb-http
topology:
full-nat
proto:
tcp
ext-interface:
vnet1.mgr.eth0
int-interface:
vnet1.mgr.eth1
start-port:
80
end-port:
80
group-enable: group-enable
Pluribus Networks nvOS Version 2.3.2
172
Configure the VLB service to load balance incoming requests on group vnet-vlb-http to a pod of five Web servers:
CLI network-admin@switch > vlb-server-add vlb-name vnet1-vlb ip 192.168.18.3
group vnet1-vlb-http
CLI network-admin@switch > vlb-server-add vlb-name vnet1-vlb ip 192.168.18.4
group vnet1-vlb-http
CLI network-admin@switch > vlb-server-add vlb-name vnet1-vlb ip 192.168.18.5
group vnet1-vlb-http
CLI network-admin@switch > vlb-server-add vlb-name vnet1-vlb ip 192.168.18.6
group vnet1-vlb-http
CLI network-admin@switch > vlb-server-add vlb-name vnet1-vlb ip 192.168.18.7
group vnet1-vlb-http
Display the server information:
CLI network-admin@switch > vlb-server-show
vlb-name
-------vnet1-vlb
vnet1-vlb
vnet1-vlb
vnet1-vlb
vnet1-vlb
group
----vnet1-vlb-http
vnet1-vlb-http
vnet1-vlb-http
vnet1-vlb-http
vnet1-vlb-http
ip
-192.168.18.3
192.168.18.4
192.168.18.5
192.168.18.6
192.168.18.7
server-enable
------------server-enable
server-enable
server-enable
server-enable
server-enable
id
-_vnet1-vlb-http.0
_vnet1-vlb-http.1
_vnet1-vlb-http.2
_vnet1-vlb-http.3
_vnet1-vlb-http.4
CLI network-admin@switch > vlb-show
name
type scope vnet
vnet-service state
gateway
------------- ---- ------ --------- ------------ ------- --------vlb-web
vlb fabric vlb-web
shared
enabled 10.12.1.1
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Monitoring the Health of VLB
You can configure health monitoring for your VLBs so that network traffic can determine if the server is available
before attempting to send connections to it. Basic monitoring is simply pinging the host and determining if the host
is active. Or you can send service pings ranging from simple TCP connections or using scripting interaction.
To create a VLB health monitor for vlb-vnet1 using ping, timeout 10 seconds, attempts 5, and 120 seconds interval
between checks:
CLI network-admin@switch > vlb-health-config-add vlb-name vlb-vnet1 name
vlb-health type ping timeout 10 attempts 5 interval 120
To remove the VLB health configuration, use the vlb-health-config-remove command.
To display the VLB health configuration, use the vlb-health-config-show command.
173
Pluribus Networks nvOS Version 2.3.2
To display the status of the VLB health configuration, use the vlb-health-status-show command:
CLI network-admin@switch > vlb-health-status-show layout vertical
vlb-name:
name:
id:
status:
fail:
last:
next:
rtt:
vlb-vnet1
vlb-health
_vlbgroup
alive
0
13:47:16
13:47:30
1836
Viewing vLB Group Statistics
You can view vLB Group statistics using the vlb-group-stats-show command:
CLI network-admin@switch > CLI vlb-group-stats-show format all layout vertical
switch:
name:
group:
processed-bytes:
processed-pkts:
dropped-bytes:
dropped-pkts:
switch:
name:
group:
processed-bytes:
processed-pkts:
dropped-bytes:
dropped-pkts:
switch:
name:
group:
processed-bytes:
processed-pkts:
dropped-bytes:
dropped-pkts:
pubdev01
vlb-1
vlb-group
0
0
0
0
pubdev03
vlb-1
vlb-group
0
0
0
0
pubdev02
vlb-1
vlb-group
0
0
0
0
Pluribus Networks nvOS Version 2.3.2
174
Configuring Virtual Load Balancing with Ubuntu 11.04 Servers and nvOS
In this example, you configure the following features:
 VNET
 IP Pool
 DHCP Server
 Ubuntu 11.04 Servers (2)
 Apache Services
 VLB
 VLB Health
Configuring the VLB VNET
1. Using the name, vlb-web, scope fabric, and vlans 200, configure the VNET:
CLI network-admin@switch > vnet-create name vlb-web scope fabric vlans 200
2. Create the IP pool, web-ip-pool, with the IP address range of 172.16.23.0, netmask 24:
CLI network-admin@switch > ip-pool-create name web-ip-pool vnet vlb-web
start-ip 172.16.23.0 end-ip 172.16.23.254 netmask 24 vlan 200
175
Pluribus Networks nvOS Version 2.3.2
3. Create the DHCP server, web-dhcp, and add the gateway:
CLI network-admin@switch > dhcp-create name web-dhcp vnet vlb-web
initial-ip-pool web-ip-pool
CLI network-admin@switch > dhcp-pool-modify dhcp-name web-dhcp dhcp-pool-name
web-ip-pool gateway-ip 172.16.23.1
4. Add connectivity to your network. You’ll need this to download Apache2.
Informational Note: This step varies depending on the setup of your corporate network. In this example, the
corporate network is a 10.0.0.0/16 network.
CLI network-admin@switch > vnet-manager-interface-add vnet-manager-name
vlb-web-mgr ip 10.0.0.0 netmask 16 if mgmt vlan 0
CLI network-admin@switch > vnet-manager-modify name vlb-web-mgr gateway
10.0.0.1 enable
5. Create the Ubuntu servers using KVMs on the switch:
Informational Note: There is no requirement that the Ubuntu servers reside on the same switch. For this
purpose, the servers are on the same switch.
CLI network-admin@switch > netvisor-kvm-create name vlb-web-svr1 vnet vlb-web
iso-label ubuntu-11.04-amd64 memory 4g cpus 2 hda-size 20g storage-pool
pool-disk4
Netvisor vm created. Please use interface-add to add interfaces and then
start to boot
CLI network-admin@switch > netvisor-kvm-interface-add netvisor-kvm-name
vlb-web-svr1 if mgmt vlan 0
CLI network-admin@switch > netvisor-kvm-interface-add netvisor-kvm-name
vlb-web-svr1 if data vlan 200
CLI network-admin@switch > netvisor-kvm-start name vlb-web-svr1
VM running. From outside switch, connect to vnc port :2.
Ex: vncviewer 10.9.11.147:2
Pluribus Networks nvOS Version 2.3.2
176
The Ubuntu server installation takes 20-30 minutes. In the meantime, configure the KVM for vlb-web-svr2:
CLI network-admin@switch > netvisor-kvm-create name vlb-web-svr2 vnet vlb-web
iso-label ubuntu-11.04-amd64 memory 4g cpus 2 hda-size 20g storage-pool
pool-disk4
Netvisor vm created. Please use interface-add to add interfaces and then
start to boot
CLI network-admin@switch > netvisor-kvm-interface-add netvisor-kvm-name
vlb-web-svr2 if mgmt vlan 0
CLI network-admin@switch > netvisor-kvm-interface-add netvisor-kvm-name
vlb-web-svr2 if data vlan 200
CLI network-admin@switch > netvisor-kvm-start name vlb-web-svr2
VM running. From outside switch, connect to vnc port :2.
Ex: vncviewer 10.9.11.147:3
The Ubuntu server installation takes 20-30 minutes. In the meantime, configure the KVM for vlb-web-svr3:
CLI network-admin@switch > netvisor-kvm-create name vlb-web-svr3 vnet vlb-web
iso-label ubuntu-11.04-amd64 memory 4g cpus 2 hda-size 20g storage-pool
pool-disk4
Netvisor vm created. Please use interface-add to add interfaces and then start to boot
CLI network-admin@switch > netvisor-kvm-interface-add netvisor-kvm-name
vlb-web-svr3 if mgmt vlan 0
CLI network-admin@switch > netvisor-kvm-interface-add netvisor-kvm-name
vlb-web-svr3 if data vlan 200
CLI network-admin@switch > netvisor-kvm-start name vlb-web-svr2
VM running. From outside switch, connect to vnc port :3.
Ex: vncviewer 10.9.11.147:3
6. Next, you install Apache2 on each Ubuntu server by executing the following commands on each one. Open your
VNC application and connect to an Ubuntu server:
sudo apt-get install apache2
sudo vi/var/www/index.html
7. Create the virtual load balancer:
CLI network-admin@switch > vlb-create name vlb-web vnet vlb-web
shared-vnet-service enable
CLI network-admin@switch > vlb-show
name
type scope vnet
vnet-service state
gateway
----------- ---- ------ ----------- ------------ ------- ------vlb-web
vlb fabric vlb-web
shared
enabled ::
177
Pluribus Networks nvOS Version 2.3.2
8. Create the health check for the VLB service:
CLI network-admin@switch > vlb-health-config-add vlb-name vlb-web switch
pleiades24 name web-http type http timeout 3 attempt 3 interval 11
This configuration means that the health check is performed every 11 seconds, and it verifies the service 3 times and
times out after 3 seconds.
9. Create the virtual load balancing group. Note that the group name must be less than 14 characters:
CLI network-admin@switch > vlb-group-add vlb-name vlb-web name web-svc-grp
proto tcp algorithm roundrobin vip 172.16.23.20 topology full-nat proxy-src-ip
172.16.23.20 proxy-src-netmask 24 start-port 80 healthcheck web-http
group-enable
10. Add the Ubuntu Apache servers to the VLB group:
CLI network-admin@switch > vlb-server-add vlb-name vlb-web ip 172.16.23.3 port
80 group vlb-web-group
CLI network-admin@switch > vlb-server-add vlb-name vlb-web ip 172.16.23.4 port
80 group vlb-web-group
CLI network-admin@switch > vlb-server-add vlb-name vlb-web ip 172.16.23.5 port
80 group vlb-web-group
11. Display the configuration:
CLI network-admin@switch > vlb-show
12. Display the VLB servers:
CLI network-admin@switch > vlb-server-show
vlb-name
group
----------- ----------vlb-web
web-svc-grp
vlb-web
web-svc-grp
vlb-web
web-svc-grp
13. Display the VLB group:
ip
----------172.16.23.2
172.16.23.3
172.16.23.4
port server-enable id
---- ------------- -------------80
server-enable _web-svc-grp.0
80 server-enable _web-svc-grp.1
80 server-enable _web-svc-grp.2
CLI network-admin@switch > vlb-group-show layout vertical
vlb-name:
name:
topology:
proto:
algorithm:
vip:
proxy-src-ip:
start-port:
end-port:
group-enable:
healthcheck:
vlb-web
web-svc-grp
full-nat
tcp
roundrobin
172.16.23.7
172.16.23.7/24
80
80
group-enable
http-service
Pluribus Networks nvOS Version 2.3.2
178
14. Display the VLB health status:
vlb-health-status-show layout vertical
switch:
mitch-aquila2
vlb-name: vlb-web
name:
http-service
id:
_web-svc-grp.0
status:
alive
fail:
0
last:
09:53:01
next:
09:53:17
rtt:
507
switch:
mitch-aquila2
vlb-name: vlb-web
name:
http-service
id:
_web-svc-grp.1
status:
alive
fail:
0
last:
09:53:14
next:
09:53:28
rtt:
572
switch:
mitch-aquila2
vlb-name: vlb-web
name:
http-service
id:
_web-svc-grp.2
status:
alive
fail:
0
last:
09:53:14
next:
09:53:28
rtt:
578
15. Stop the Apache2 service on one of the Ubuntu servers by connecting with VNC and executing the command:
sudo etc/init.d/apache2 stop
179
Pluribus Networks nvOS Version 2.3.2
16. Display the VLB health status again to verify that the server is in a failed state:
CLI network-admin@switch > vlb-health-status-show
CLI (network-admin@mitch-aquila2) > vlb-health-status-show layout vertical
switch:
mitch-aquila2
vlb-name: vlb-web
name:
http-service
id:
_web-svc-grp.0
status:
alive
fail:
0
last:
09:54:42
next:
09:54:57
rtt:
568
switch:
mitch-aquila2
After stopping the Web service on
vlb-name: vlb-web
server 1, the status changes to
name:
http-service
dead.
id:
_web-svc-grp.1
status:
dead
fail:
3
last:
09:54:42
next:
09:54:57
rtt:
565
switch:
mitch-aquila2
vlb-name: vlb-web
name:
http-service
id:
_web-svc-grp.2
status:
alive
fail:
0
last:
09:54:42
next:
09:54:57
rtt:
572
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Adding Virtual Router Redundancy Protocol to VLB Interfaces
You can add VRRP to the VLB configuration so that if one interface becomes unavailable, then the second interface
becomes the virtual router. Add interfaces to the VLB configuration with VRRP parameters. To configure Web server
1 as the master, use the following commands:
Informational Note: You must use the same VRRP ID for both interfaces. Otherwise, the configuration is
invalid. You must also create a VRRP priority with a higher value for the primary interface and a lower
VRRP priority for the secondary interface.
CLI network-admin@switch > vlb-interface-add vlb-name vlb-web if data vlan 200
CLI network-admin@switch > vlb-interface-modify vlb-name vlb-web-svr1 nic
eth1.200 vrrp-id 10 vrrp-primary vlb-web-svr1 vrrp-priority 100
Pluribus Networks nvOS Version 2.3.2
180
To add Web server 2 as the secondary virtual router, use the following command:
CLI network-admin@switch > vlb-interface-add vlb-name vlb-web if data vlan 200
CLI network-admin@switch > vlb-interface-modify vlb-name vlb-web-svr2 nic
eth2.200 vrrp-id 10 vrrp-primary vlb-web-svr1 vrrp-priority 50
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
181
Pluribus Networks nvOS Version 2.3.2
Configuring Roles and Users
Role-Based Access Control (RBAC) is a secure method of restricting access to authorized users. This method enables
the network administrator to add users and assign each user to specific roles. Each role has specific permissions and
allows users to perform various actions based on the scope of their role.
In this context, users are personnel that can log into the switch, and perform certain functions.
A role defines the level of access for a user account. By assigning roles to users, you can allow multiple users to
complete their tasks. RBAC limits risk by ensuring that users do not have access beyond their training or level of
control.
nvOS allows you to create roles and assign them to users. You can create the following types of roles:
 Scope — A role can apply to the scope of local, or fabric.
 Access — You allow read-only access or read-write access.
 Configuration — A role can apply to the running configuration or not.
Once you create a user with a scope of local or fabric, you cannot modify the user scope. If you decide that your user
needs local scope rather than fabric scope, you must delete the user and create a new one.
There are three types of roles configured for user access:
 network-admin — this is a super user role and can perform all functions on the switch.
 read-only-network-admin — this is a read only role and the user can only execute show commands
from the CLI.
 fabric-admin — this role can perform fabric-wide functions only.
Configuring Custom Roles
You can create custom roles in addition to the preconfigured ones in nvOS. When you create a role, you configure
the following parameters:
 name — create a name for the role
 scope — specify fabric or local. Once you’ve configured the role as local or fabric, you can’t modify it. To
change the scope, you must delete the role and create a new one.
 access — specify the type of access for the user. You can specify any of the following types of access:
• read-write — the role can display information and make changes to the configuration. You can modify
this role to read-only if you decide that the role can only use show commands at the CLI.
• running-config — the role has access to the running configuration on the switch.
• no-running-config — the role cannot access the running configuration on the switch.
For example, create the role, local-admin, with scope local, read-write access to the running configuration:
CLI network-admin@switch > role-create name fabric-admin scope local access
read-write running-config
To modify the role parameter, access to read-only, use the following command:
CLI network-admin@switch > user-role-modify name fabric-admin scope fabric
access read-only
When you modify the role, you can also specify to remove the role from users with the delete-from-users
parameter.
Pluribus Networks nvOS Version 2.3.2
182
To delete the role, local-admin, use the user-role-delete command:
CLI network-admin@switch > user-role-delete name fabric-admin
To display the role configuration, use the role-show command.
CLI network-admin@switch > role-show
role-show format all layout vertical
id:
6000021:402
name:
web-svr-admin
scope:
fabric
access:
read-write
running-config:
deny
id:
6000021:404
name:
test-vnet-admin
scope:
fabric
access:
read-write
running-config:
deny
id:
6000021:405
name:
test-admin
scope:
fabric
access:
read-write
running-config:
deny
id:
6000021:406
name:
vlan-test-admin
scope:
fabric
access:
read-write
running-config:
deny
switch:
pleiades24
id:
0:0
name:
network-admin
scope:
local
access:
read-write
running-config:
permit
switch:
pleiades24
id:
0:1
name:
read-only-network-admin
scope:
local
access:
read-only
running-config:
deny
183
Pluribus Networks nvOS Version 2.3.2
This user has
read-write access but
not to the running
configuration.
Creating and Managing Users
You can create users and apply roles to them to manage access to the switch or network. To create a user, jdoe,
scope local, password p1zz@, and initial role, local-admin, use the following syntax:
CLI network-admin@switch > user-create name jdoe scope local password p1zz@
initial-role local-admin
password:
Confirm password:
Informational Note: Once you configure the scope for a user, you cannot modify it. To change the
scope, delete the user, and create a new one with the intended scope.
To modify the initial role from local-admin to network-admin, use the following command:
CLI network-admin@switch > user-modify name jdoe initial-role network-admin
To delete the user, use the user-delete command.
To add roles to a user, jdoe, role name fabric-admin, use the following syntax:
CLI network-admin@switch > user-role-add name jdoe role fabric-admin
You can assign multiple roles to a user. For instance, if jdoe is a fabric-admin, and you also want to assign the role,
local-admin, use the following command:
CLI network-admin@switch > user-role-add user-name jdoe role local-admin
CLI (network-admin@mitch-aquila2) > user-role-show
switch
user-name
role
------------- ----------------- ----------------------network-admin
network-admin
vlb-web-svr-admin vlb-web-svr-admin
jdoe now has two
test-admin
roles assigned. test-admin
test-admin
test-admin-admin
vlan-test-admin
vlan-test-admin
jdoe
network-admin
jdoe
local-admin
ops-test1-admin
fabric-admin
pleiades01
java-api-admin
java-api-admin
To remove a role from the user, jdoe, use the following command:
CLI network-admin@switch > user-role-remove name jdoe role fabric-admin
Pluribus Networks nvOS Version 2.3.2
184
To display user roles, use the user-role-show command.
CLI (network-admin@pleiades24)>
switch
user-name
------------- ----------------network-admin
vlb-web-svr-admin
test-admin
test-admin
vlan-test-admin
laurap
ops-test1-admin
pleiades01
java-api-admin
user-role-show
role
----------------------network-admin
vlb-web-svr-admin
test-admin
test-admin-admin
vlan-test-admin
read-only-network-admin
fabric-admin
java-api-admin
To display information about all users configured in nvOS, use the user-show command:
CLI network-admin@switch > user-show
name
network-admin
ops-mgmt-admin
ext-50-admin
www-51-admin
jdoe
scope
fabric
fabric
fabric
fabric
fabric
uid
39999
40000
40001
40002
40003
The User ID (UID) is assigned by nvOS and is not configurable. You need the UID to configure user passwords for
TACACS+ authentication.
To configure user, jdoe, on a TACACS+ server, use the following command:
CLI network-admin@switch > user-set-password name jdoe scope fabric uid 4003
server aaa-tacacs
See Configuring TACACS+.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
185
Pluribus Networks nvOS Version 2.3.2
Configuring TACACS+
About TACACS+
Terminal Access Controller Access Control System (TACACS+) is an Authentication, Authorization, and Accounting
(AAA) protocol that was introduced in the early 2000s. The main goal of TACACS+ is to provide a centralized
database to use for authentication. It uses a client server approach by which the client queries a server and the
server replies with a pass or fail for authentication. The communication between the client and server uses TCP as
the connection protocol, and requires a secret key.
nvOS can be configured to use external TACACS+ servers for authentication, authorization, and accounting. You can
configure any number of TACACS+ servers, and each server may be configured to handle any combination of
authentication, session authorization, command authorization, session accounting, and command accounting.
It is important to note that the default “network-admin” account is exempt from all TACACS+ integration, as a
fail-safe account for sites without TACACS+ and to allow access to Pluribus Networks facilities if TACACS+ is
unavailable or unreachable.
TACACS+ is configured using the aaa-tacacs-create command, and using options to specify the IP address,
port, password, priority, authentication methods, and accounting options. Once set up, a user can login to the
switch and get CLI access using an account configured on the specified TACACS+ server.
The TACACS+ server determines what role the user has by returning a “role” attribute. The roles include
“networkadmin” for full access and "read-only-network-admin" users who can only run show commandsPAP, CHAP,
and MS-CHAP authentication protocols are supported.
Figure 1 illustrates a simple TACACS+ implementation.
Figure 1: TACACS+ AAA with a nvOS switch
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
186
Configuring TACACS+
Using Figure 1 as an example, you can configure TACACS+ access to the switch with the following command:
CLI network-admin@switch > aaa-tacacs-create name tacacs-server scope fabric
port 34 m0nk3y6 priority 3 authen authen-method ms-chap sess-acct
This command configures basic access from a user on the network to the switch. You can add the following optional
parameters to the configuration:
 Session accounting
 Command accounting
 Session Authorization
 Command Authorization
To add optional parameters or to modify the current configuration, use the aaa-tacacs-modify command.
To display the status of the TACACS server, use the aaa-tacacs-status command.
To delete the configuration, use the aaa-tacacs-delete command.
Pluribus Networks Configuration Guide
187
www.pluribusnetworks.com
Creating and Implementing Access Control Lists (ACLs)
Access Control Lists (ACLs) allow you to configure basic traffic filtering for IP addresses and MAC addresses. The ACL
controls if routed packets are forwarded or blocked on the network. The packet is examined by the switch and then
determines if the packet is forwarded or dropped based on the criteria configured in the ACLs. nvOS supports Layer
2 (MAC) or Layer 3 (IP) ACLs.
ACL criteria can be based on source or destination addresses or the protocol type. nvOS supports UDP, TCP, IGMP,
and IP protocols.
You can use ACLs to restrict contents of routing updates or provide traffic flow control. ACLs can allow one host to
access part of your network and prevent another host from accessing the same area. You can also use ACLs to decide
what types of traffic are forwarded or blocked.
If you need more background on ACLs and using them on your network, refer to the many networking resources
available.
Using a Deny IP ACL to Block Network Traffic
In this example, a network is shown with a Finance server on one part of the network, and an Engineering server on
another part. You want to block the Engineering server from the Finance server in order to protect company
sensitive information. See Configuring an Internal Deny ACL to review the configuration sample.
Figure 1: Network Example - IP ACL for Internal Servers
Or you may discover that an external source is attempting to access your network, and ping your servers for IP
addresses. You can use an ACL to block the specific source using an IP ACL.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
188
Figure 2:IP ACL Blocking External Access
See Configuring an External Deny ACL to review the configuration example.
Using IP ACLs to Allow Network Traffic
In the same manner, you can allow specific traffic to a destination such as the external server in Figure 2 IP ACL
Blocking External Access. To allow HTTP traffic to 209.225.113.24, see Configuring an External Allow IP ACL to review
the configuration example.
Pluribus Networks Configuration Guide
189
www.pluribusnetworks.com
Figure 3:IP ACL Allowing HTTP Traffic
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
190
Using MAC ACLs to Deny Network Traffic
You can create ACLs based on MAC addresses to deny network traffic from a specific source. MAC addresses are
Layer 2 protocols and most often assigned by the hardware manufacturer. Figure 4 MAC ACL Blocking Access shows
an example of a MAC address and Ethernet type that you want to block from the network.
Figure 4: MAC ACL Blocking Access
See Configuring a MAC ACL to Deny Network Traffic to review the example configuration.
Using MAC ACLs to Allow Network Traffic
So now that you’ve blocked the MAC address, let’s reverse the scenario and allow IPv4 network traffic from the MAC
address to the network.
Pluribus Networks Configuration Guide
191
www.pluribusnetworks.com
Figure 5:MAC ACL Allowing Access
See Configuring a MAC ACL to Allow Network Traffic to review the example configuration.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
192
Configuring IP ACLs
From Figure 1 Network Example - IP ACL for Internal Servers, the following information is available:
 Source IP address
 Source netmask
 Destination IP address
 Destination netmask
 Type of protocol to deny - IP
 Ports
 VLAN
Configuring an Internal Deny ACL
Configure the ACL for denying traffic from the Engineering server to the HR server and name the ACL, deny-hr:
CLI network-admin@switch > acl-ip-create name deny-hr action deny scope local
src-ip 192.168.10.2 src-ip-mask 24 dst-ip 192.168.200.3 dst-ip-netmask 24
proto ip src-port 55 dst-port 33 vlan 1505
To review the configuration, use the acl-ip-show command:
CLI network-admin@switch > acl-ip-show name deny-hr layout vertical
name:
deny-ip
id:
b00011:20
action:
deny
proto:
ip
src-ip:
192.168.10.2/24
src-port:
55
dst-ip:
192.168.200.3/24
dst-port:
33
vlan:
1505
scope:
local
port:
0
Now, when you attempt to access the Finance server from the Engineering server, the packets are dropped.
Configuring an External Deny ACL
From Figure 2 IP ACL Blocking External Access, the following information is available:
 IP Address
 Port Number
To configure an ACL to deny traffic from the external server, use the acl-ip-create command to create an ACL
named deny-external:
CLI network-admin@switch > >acl-ip-create name deny-external scope fabric
src-ip 209.255.113.24/28
Pluribus Networks Configuration Guide
193
www.pluribusnetworks.com
To review the configuration, use the acl-ip-show command:
CLI network-admin@switch > acl-ip-show name deny-external layout vertical
name:
id:
action:
proto:
src-ip:
src-port:
dst-ip:
dst-port:
vlan:
scope:
port:
deny-external
b000022:20
deny
tcp
209.225.113.24/28
0
::/0
0
0
fabric
0
Configuring an External Allow IP ACL
To allow HTTP traffic to the external server, 209.225.113.24 with a netmask of 255.255.255.240 and a
scope of fabric, you can create an IP ACL called allow-http using the following syntax:
CLI network-admin@switch > acl-ip-create name allow-http permit scope fabric
src-ip 0.0.0.0. src-mask 255.255.255.255 dst-ip 209.225.113.24 dst-ip-mask
255.255.255.240 protocol tcp dst-port 57
To review the configuration, use the acl-ip-show command:
CLI network-admin@switch > >acl-ip-show name allow-http layout vertical
name:
id:
action:
proto:
src-ip:
src-port:
dst-ip:
dst-port:
vlan:
scope:
port:
allow-http
b000025:20
allow
tcp
0.0.0.0/255.255.255.255
0
209.225.113.24/28
57
0
fabric
0
To delete the ACL configuration, use the acl-ip-delete command.
To modify the ACL configuration, use the acl-ip-modify command.
Configuring a MAC ACL to Deny Network Traffic
To deny IPv4 network traffic from MAC address, 01:80:c2:00:00:0X, for the scope fabric, create the MAC
ACL, deny-MAC, using the following syntax:
CLI network-admin@switch > acl-mac-create name deny-mac action deny src-mac
01:80:c2:00:00:0X ether-type ipv4 scope fabric
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
194
To review the configuration, use the acl-mac-show command:
CLI network-admin@switch > acl-mac-show name deny-mac layout vertical
name:
id:
action:
src-mac:
dst-mac:
dst-mac-mask:
ether-type:
vlan:
scope:
port:
deny-mac
b000015:12
deny
01:80:c2:00:00:0X
00:00:00:00:00:00
aa:aa:aa:aa:aa:aa
ipv4
0
fabric
0
Configuring a MAC ACL to Allow Network Traffic
To allow IPv4 network traffic from MAC address, 01:80:c2:00:00:0X, for the scope fabric, create the MAC
ACL, allow-MAC, using the following syntax:
CLI network-admin@switch > acl-mac-create name allow-mac action permit src-mac
01:80:c2:00:00:0X ether-type ipv4 scope fabric
To review the configuration, use the acl-mac-show command:
CLI network-admin@switch > acl-mac-show name deny-mac layout vertical
name:
id:
action:
src-mac:
dst-mac:
dst-mac-mask:
ether-type:
vlan:
scope:
port:
deny-mac
b000015:12
deny
01:80:c2:00:00:0X
00:00:00:00:00:00
aa:aa:aa:aa:aa:aa
ipv4
0
fabric
0
To delete the ACL configuration, use the acl-mac-delete command.
To modify the ACL configuration, use the acl-mac-modify command.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
195
www.pluribusnetworks.com
Configuring vFlow for Analytics
A vFlow can be used to capture packets for analysis, and you can determine if the vFlow captures packets across the
fabric or on a single switch. Packets are captured by forwarding them from the data plane of the switch to the
control plane.
A vFlow that directs packets to the switch CPU can be configured to save packets to a file by enabling the log-packets
parameter. The file is written using a libcap compatible format so that programs like TCPdump and Wireshark can be
used to read the file. The file is exported to clients using NFS or SFTP.
Packet capture data is available with switch or fabric scope. The pcap files are stored over NFS in the following
locations:
/net/<ServerSw_Name>/nvOS/global/flow/<Flow_Name>/switch/<Switch_Name>/pcap
/net/<ServerSw_Name>/nvOS/vnet/<VNET_Name>/flow/<Flow_Name>/
switch/<Switch_Name>/pcap
/net/<ServerSw_Name>/nvOS/global/flow/<Flow_Name>/fabric/pcap
/net/<ServerSw_Name>/nvOS/vnet/<VNET_Name>/flow/<Flow_Name>/
fabric/pcap
Snooping only works if you use the parameters, copy-to-cpu or to-cpu. The copy-to-cpu parameter
ensures that the data plane forwards the packets and sends a copy to the CPU. Use this parameter if you want traffic
to flow through the switch. The to-cpu parameter doesn’t forward packets and interrupts traffic on the switch. To
snoop all application flow packets of protocol type TCP, enter the following CLI commands at the prompt:
CLI network-admin@switch > vflow-create name snoop_all scope local proto tcp
action copy-to-cpu
Then use the following command to display the output:
CLI network-admin@switch > vflow-snoop
switch: pleiades24, flow: snoop_all, port: 65, size: 66, time:
20:07:15.03867188
smac: 64:0e:94:28:00:fa, dmac: 64:0e:94:2c:00:7a, etype: ip
sip: 192.168.2.51, dip: 192.168.2.31, proto: tcp
sport: 42120, dport: 33399
switch: pleiades24, flow: snoop_all, port: 65, size: 184, time:
20:07:15.03882961
smac: 64:0e:94:28:00:fa, dmac: 64:0e:94:2c:00:7a, etype: ip
sip: 192.168.2.51, dip: 192.168.2.31, proto: tcp
sport: 42120, dport: 33399
switch: pleiades24, flow: snoop_all, port: 43, size: 66, time:
20:07:15.03893740
smac: 64:0e:94:2c:00:7a, dmac: 64:0e:94:28:00:fa, etype: ip
sip: 192.168.2.31, dip: 192.168.2.51, proto: tcp
sport: 33399, dport: 42120
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
196
To restrict the flows captured to TCP port 22, SSH traffic, create the following vFlow:
CLI network-admin@switch > vflow-create name snoop_ssh scope local action
copy-to-cpu src-port 22 proto tcp vflow-add-filter name snoop_ssh
Then use the vflow-snoop command to display the results:
switch: pleiades24, flow: snoop_ssh, port: 41, size: 230, time:
10:56:57.05785917 src-mac: 00:15:17:ea:f8:70, dst-mac:
f4:6d:04:0e:77:60, etype: ip src-ip: 10.9.11.18, dst-ip: 10.9.10.65,
proto: tcp src-port: 22, dst-port: 62356
switch: pleiades24, flow: snoop_ssh, port: 41, size: 118, time:
10:56:57.05922560 src-mac: 00:15:17:ea:f8:70, dst-mac:
f4:6d:04:0e:77:60, etype: ip src-ip: 10.9.11.18, dst-ip: 10.9.10.65,
proto: tcp src-port: 22, dst-port: 62356
The optional parameter vflow-add-filter restricts the output of the vflow-snoop command to the
packets matching the snoop_ssh flow definition.
To capture traffic packets for a flow across the entire fabric, you create a flow with the scope of fabric. To copy the
packets to a pcap file, add the log-packets option:
CLI network-admin@switch > vflow-create name fab_snoop_all scope fabric action
copy-to-cpu port 22 log-packets yes
If you enable log-packets, the separate pcap files for all switches are available on any switch. In addition a
consolidated pcap file is available that aggregates the packets from all switches in the entire fabric.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Analyzing Live Traffic Using Wireshark
Wireshark is a well known network protocol analyzer and one of many applications used for network protocol
analysis. Wireshark can interactively browse packet data from a live network or from a previously save pcap file.
Informational Note:You can download Wireshark from http://www.wireshark.org
To use Wireshark to decode a previously saved packet flow capture file, export the file from the switch and analyze it
with Wireshark.
Informational Note:
The path to a Pluribus Networks switch pcap file has the format:
/net/<ServerSw_Name>/nvOS/global/flow/<Flow_Name>/<Switch_Name>/pcap
Pluribus Networks Configuration Guide
197
www.pluribusnetworks.com
To use Wireshark to interactively analyze packets in real time, you need to capture a packet traffic flow, either on a
specific switch or across the entire fabric using the scope option. Include the log-packets option to send packets to
the associated pcap files, for example
CLI network-admin@switch > vflow-snoop scope fabric src-ip 112.168.3.105 action
copy-to-cpu log-packets
Next, create a fifo on the host running Wireshark.
mkfifo /tmp/pcap
Start Wireshark, and select Options from the Capture menu.
Enter the fifo path that you created in the Interface field: /tmp/pcap
Use tail to copy the pcap file to the FIFO:
tail +0f \
/net/ServerSw_Name/nvOS/global/flow/Flow_Name/switch/Switch_Name/
pcap/tmp/pcap
You need to substitute ServerSw_Name, Flow_Name and Switch_Name to match your environment. Live capture
continues until the packet capture file is rotated. By default, the maximum packet capture file size is 10MB but it is
configurable with the packet-log-max option of the vflow-create and vflow-modify commands.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
198
TIP! The mkfifo command used in this task is a standard feature of UNIX-like operating
systems, including MacOS. For Windows platforms, you may need to install the GNU
CoreUtils package available at http://gnuwin32.sourceforge.net/packages/coreutils.htm.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
199
www.pluribusnetworks.com
Using vFlows to Disable Communication
vFlows can be used to specify communications that are not allowed with a switch or a fabric. Use the following steps
to create a vFlow as a firewall:
1. Define a VLAN and destination IP-based flow and specify that the flow is dropped by the switch, with statistics
monitoring enabled:
CLI network-admin@switch > vflow-create name flow3 scope local vlan 99 dst-ip
172.168.24.1 action drop stats enable
Display the statistics for the new flow above as the traffic is dropped:
CLI network-admin@switch > vflow-stats-show name flow3 show-diff-interval 5
switch
aquila02
switch
aquila02
name
flow3
name
flow3
packets
864
packets
5
bytes
116K
bytes
936K
cpu-packets
0
cpu-packets
0
cpu-bytes
0
cpu-bytes
0
There are many options available for creating vFlows, and vFlows can be used to shape traffic, capture statistics,
capture flow metadata, capture packets, or manage communications. The options include:
 vlan
 vnet
 in-port
 out-port
 ether-type
 src-mac
 src-mac-mask
 dst-mac
 dst-mac-mask
 src-ip
 src-ip-mask
 dst-ip
 dst-ip-mask
 src-port
 dst-port
 dscp
 tos
 proto
 flow-class
 uplink-ports
 bw-min
 bw-max
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
200
 precedence
 action
 action-value
 no-mirror
 mirror
 no-process-mirror
 process-mirror
 no-log-packets
 log-packets
 packet-log-max
 stats
 stats-interval
 duration
 no-transient
 transient
 vxlan
 vxlan-ether-type
 vxlan-proto
Pluribus Networks Configuration Guide
201
www.pluribusnetworks.com
Use Case Scenario
In a real use case, the command connection-show server-ip 10.9.10.117 was used to analyze a
suspicious connections to server 10.9.10.117:
Switch vlan client-ip server-ip service dur(s) latency(us) out-bytes in-by
tes active
------ ---- --------- --------- ------- ------ ----------- --------- ------- -----switch:
switch02
vlan:
1
client-ip:
10.9.9.33
server-ip:
10.9.9.107
service:
http
dur(s):
0
latency(us):
65
out-bytes:
0
in-bytes:
0
active:
yes
switch:
switch02
vlan:
1
client-ip:
10.9.9.33
server-ip:
10.9.9.107
service:
http
dur(s):
210
latency(us):
7
out-bytes:
48804
in-bytes:
6120
active:
yes
switch:
switch02
vlan:
1
client-ip:
10.9.9.33
server-ip:
10.9.9.107
service:
http
dur(s):
328
latency(us):
30
out-bytes:
48720
in-bytes:
612620
active:
yes
Configuring Mirroring for vFlows and Ports
A Pluribus Networks fabric administrator can run services and applications within the switch. Consider the use case
of an application that needs access to data that is flowing through the switch, but does not want to impede that
flow. The port-mirroring feature provides this functionality.
The system predefines a mirror configuration, but does not insert any traffic into that mirror. Use the following steps
to setup mirroring to send from all of the data ports to the span port (port 66). In this version of nvOS, the
port-mirror command is deprecated and replaced with the command mirror-modify to allow support for
vFlow-based and port-based mirroring. The command syntax for mirror-modify is as follows:
CLI network-admin@switch > mirror-modify out-port port-list in-port port-list
[policy port|vflow] mirroring|no-mirroring
CLI network-admin@switch > mirror-show [format fields-to-display]
[parsable-delim character] [sort-asc] [sort-desc] [show dups] [layout
vertical|horizontal] [show-interval seconds-interval]
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
202
View the status of mirroring by entering the following at the CLI command prompt:
CLI network-admin@switch > mirror-show
switch: aquila19
direction: bidirection
out-port:
in-port:
mirroring: disable
The parameter out-port is not configured and mirroring is disabled therefore, no data mirroring can occur.
To modify the mirroring configuration, use the following steps:
1. Use the mirror-modify command to set the output to the span port. However, if there is more than 10Gb of
traffic on ports 1-64, do not execute this command.
CLI network-admin@switch > mirror-modify in-port 1-64 out-put 66 mirroring
mirror-show
switch:
pleiades24
direction:
bidirection
out-put:
66
in-port:
1-64
mirroring:
enable
To disable the configuration, use the following command:
CLI network-admin@switch > mirror-modify no-mirroring
mirror-show
switch: aquila19
direction: bidirection
out-port: 66
in-port: 1-64
mirroring: disable
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
203
www.pluribusnetworks.com
Managing Traffic Classes
nvOS provides a full set of traffic class features, including the ability to view and create traffic classes, as well as
assign traffic classes to flows to manage the quality of service of the flow traffic and shape the traffic passing
through an nvOS fabric.
To display the currently defined traffic classes:
CLI network-admin@switch > vflow-class-show
name
------------meter
guaranteed_bw
lossless
control
scope
-----fabric
fabric
fabric
fabric
type
-----system
system
system
system
priority
-------0
9
10
11
The higher the priority number, the higher the priority of the class. To add a vflow class, use the
vflow-class-create command:
CLI network-admin@switch > vflow-class-create name traffic-1 scope fabric
priority 5
This creates a traffic class with a scope of fabric and medium priority.
To add a traffic class to a vFlow, create a vFlow and assign a traffic class. In this case the flow is for a single IP address:
CLI network-admin@switch > vflow-create name losslessflow scope local src-ip
10.11.1.10 src-ip-mask 255.255.255.255 action none flow-class lossless
CLI network-admin@switch > vflow-show name losslessflow layout vertical
switch: aquila12
name: losslessflow
scope: local
type: vflow
vlan: 0
vnet:
in-port:
out-port:
ether-type: 0
src-ip: 10.11.1.10
dst-ip:::
src-port: 0
dst-port: 0
proto: ip
flow-class: lossless
bw-max: 0
pri: 0
action: none
action-value: 0
transient: no
Traffic from IP address 10.11.1.10 now has a very high priority throughout the switch. For a similar high priority
throughout the fabric use scope fabric rather than scope local.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
204
When a TCP session goes through the NPU, and capacity is exceeded, the return traffic with TCP ACK packets can get
dropped from the session. To avoid this, create a flow that matches the TCP ACK packets and set a higher
precedence for it.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
205
www.pluribusnetworks.com
Using Application Flows and Statistics
Displaying Standard Statistics
You can display standard statistics that consist of flow-based information collected and tracked continuously by the
switch.
To modify statistics logging, use the stats-log-modify command and disable or enable statistical logging as
well as change the interval, in seconds, between statistical events.
To display statistical logging information, use the stats-log-show command:
CLI network-admin@switch > stats-log-show
switch:
enable:
interval:
pleiades24
yes
60
To show connection-level statistics, traffic flows between a pair of hosts for an application service, including current
connections and all connections since the creation of the fabric, enter the following CLI command at the prompt:
CLI network-admin@switch > connection-stats-show
switch:
mac:
vlan:
ip:
port:
iconns:
oconns:
ibytes:
obytes:
total-bytes:
last-seen-ago:
switch:
mac:
vlan:
ip:
port:
iconns:
oconns:
ibytes:
obytes:
total-bytes:
last-seen-ago:
pleiades24
00:e0:81:e4:02:12
200
100.200.1.3
53
80
0
0
0
0
4d19h32m23s
pleiades24
00:12:c0:80:1e:85
200
100.200.1.4
16
0
70684
578M
890M
1.43G
46s
From the information displayed in the output, you can see statistics for each switch, VLANs, client and server IP
addresses, as well as the services on each connection. Latency and other information is also displayed.
The latency(us) column displays the running latency measurement for the TCP connection in microseconds. It
indicates end-to-end latency and includes the protocol stack processing for the connected hosts and all intermediary
network hops.
This is not the same latency measurement experience by a packet transiting the switch port-to-port. The
port-to-port latency is platform-dependent and you should refer to the datasheet for your switch model.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
206
To display specific types of connections, use the additional parameters with the command. For instance to display
active connections,
CLI network-admin@switch > connection-stats-show active
switch
switch12
switch12
switch12
switch12
switch12
. . .
vlan
1
5
1
1
1
vxlan vnet client-ip
server-ip
service active age
0
10.9.10.152 96.17.77.96 http
yes 35m27s
0
10.12.1.47 10.9.10.204 445
yes 7m56s
0
10.9.9.21
23.62.97.88 http
yes 3m41s
0
10.9.9.21
23.60.129.224http
yes 3m44s
0
10.9.10.72 10.9.99.23
http
yes 7s
To display a summary of traffic statistics for each application service, use the service-stats-show command.
CLI network-admin@switch > service-stats-show
switch
pleiades24
pleiades24
pleiades24
pleiades24
pleiades24
pleiades24
service
53495
8084
59475
imap
35356
54341
bytes
584
845M
33.9K
1.83M
106
584
From the information displayed in the output, you can review each switch, service, and the number of bytes used by
each service.
To display storage traffic statistics, use the storage-stats-show command:
CLI network-admin@switch > storage-stats-show
switch
server-ip
port read-bytes write-bytes
----------------- ---------- ----------switch12 10.9.9.9
65
3.63T
302K
switch12 10.9.10.113 nfs 0
0
switch12 10.9.9.33
nfs 284G
6.15K
switch12 10.9.11.18
65
137G
6.02K
switch12 10.9.10.69
nfs 46.0G
402K
. . .
From the information displayed in the output, you can review the storage data for each server, the port, and the
number of read-write bytes.
Pluribus Networks Configuration Guide
207
www.pluribusnetworks.com
To display interface statistics, use the interface-stats-show command:
CLI network-admin@switch > interface-stats-show
switch:
time:
nic:
ibytes:
ipkts:
ierrs:
obytes:
opkts:
oerrs:
switch:
time:
nic:
ibytes:
ipkts:
ierrs:
obytes:
opkts:
oerrs:
switch:
time:
nic:
ibytes:
ipkts:
ierrs:
obytes:
opkts:
oerrs:
switch:
time:
nic:
ibytes:
ipkts:
ierrs:
obytes:
opkts:
oerrs:
pleiades24
09:20:27
data
100M
302K
0
126M
453K
0
pleiades24
09:20:27
span
11.7M
396K
0
0
0
0
pleiades24
09:20:27
ops.mgmt.mgr.eth1
64.2M
774K
0
46.2K
1.10K
0
pleiades24
09:20:27
ext.50.mgr.eth0
2.41M
34.2K
0
679K
11.9K
0
From the information displayed in the output, you can review the inbound and outbound traffic for each NIC on the
switch. You can also check for errors in the inbound and outbound traffic.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Understanding vFlow Statistics
Virtual network-based flows, vflows, display statistics for packet traffic flows on a switch and across the fabric.
vFlows are very powerful and provide many features such as quality of service (QoS), traffic shaping, packet redirect,
drop actions, mirror, and capture.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
208
A vFLow can be configured to store log statistics to a file accessible to clients using NFS and SFTP. If statistics logging
is enabled, nvOS periodically polls the switch for the most recent statistics for each flow and saves the statistics to an
exported file. nvOS also saves individual statistics received from other switches in the fabric and combines the
statistics from all switches to record aggregate statistics for the entire fabric.
The switch consists of two components, the switch and the server. vFlows with operations like drop are executed
within the switch component. Some vFlows operations for QoS take place in the switch component, while others
operate within the coprocessor by directing pertinent traffic to the coprocessor. There, the traffic is managed and
then sent back to the switch component.
Other actions such as copy-to-cpu sends the match traffic to the server component where the traffic is managed
and then forwards packets for delivery. In general, the details are managed by nvOS including fabric scope
commands that cause all switches within a fabric to participate in an operation and then sends the compiled results
to the CLI or to log files.
Before you can access the files, you must enable NFS or SFTP access to the log files by using the
admin-service-modify command.
.
CLI network-admin@switch > vflow-share-show
switch
pleiades24
pleiades24
pleiades24
pleiades24
pleiades24
vnet
fab1-global
ops-mgmt
ext-50
www-51
folsom
enable
no
no
no
no
no
share-path
pleiades24:/nvOS/vnet/fab1-global
pleiades24:/nvOS/vnet/ops-mgmt
pleiades24:/nvOS/vnet/ext-50
pleiades24:/nvOS/vnet/www-51
pleiades24:/nvOS/vnet/folsom
CLI network-admin@switch > vflow-share-modify vnet fab1-global enable
vflow-share-show
switch
vnet
pleiades24 fab1-global
pleiades24 ops-mgmt
pleiades24 ext-50
pleiades24 www-51
pleiades24 folsom
enable
yes
no
no
no
no
share-path
pleiades24:/nvOS/vnet/fab1-global
pleiades24:/nvOS/vnet/ops-mgmt
pleiades24:/nvOS/vnet/ext-50
pleiades24:/nvOS/vnet/www-51
pleiades24:/nvOS/vnet/folsom
You can then access the statistics log files using NFS in the following locations:
For the switch scope, the files are located in
/net/switch-name/nvos/vnet/vnet-name/flow/flow-name/switch/
switch-name/stats
For the fabric scope, the files are located in
/net/switch-name/nvos/vnet/vnet-name/flow/flow-name/fabric/
stats
To create a vFLow for example, Host-Agent-Discover, and measure statistics, enter the following command:
CLI network-admin@switch > vflow-create name Host-Agent-Discover scope local
system
Pluribus Networks Configuration Guide
209
www.pluribusnetworks.com
To view all vFlows currently tracked by the switch or fabric, use the vflow-show command:
CLI network-admin@switch > vflow-show
switch:
name:
scope:
type:
dst-ip:
precedence:
action:
switch:
name:
scope:
type:
in-port:
src-port:
proto:
precedence:
action:
switch:
name:
scope:
type:
dst-ip:
precedence:
action:
switch:
name:
scope:
type:
in-port:
src-port:
proto:
precedence:
action:
pleiades24
Host-Agent-Discover
local
system
224.4.9.6
2
copy-to-cpu
pleiades24
DHCP-client
local
system
1-68
68
udp
2
copy-to-cpu
pleiades24
Host-Agent-Discover
local
system
224.4.9.6
2
copy-to-cpu
pleiades24
DHCP-client
local
system
1-68
68
udp
2
copy-to-cpu
From the information displayed in the output, you can review the switch, the name of the vFlow, scope, type of
vFlow, destination IP address, precedence, and action for the vFlow.
To display statistics for all vFlows, use the vflow-stats-show command:
CLI network-admin@switch > vflow-stats-show
switch
name
--------pleiades24IGMP-Flow
pleiades24 LLDP-Flow
pleiades24 Host-Agent
pleiades24 ECP
packets
------368K
82.9K
17.8K
0
bytes
----23.0M
26.3M
1.11M
0
cpu-packets
----------392K
82.9K
0
0
cpu-bytes
--------23.0M
26.0M
0
0
To monitor statistics of a vFlow and update every 10 seconds, use the following syntax:
CLI network-admin@switch > vflow-stats-show name flow1 show-diff-interval 10
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
210
To log persistent records of flow statistics, use the logging parameter and collect statistics every 10 seconds:
CLI network-admin@switch > vflow-create name monitor-flow scope local
ether-type arp stats log stats-interval 5
You can display the statistics logs for the new flow using the vflow-stats-show command.
Informational Note: Conflicting vFlows
Multiple vFlows can be active at once, but nvOS cannot apply them at the same time. You can
use the precedence parameter is used to set the order of the vFlows. If you set the
precedence to a higher value (0 - 10 with 0 as the lowest precedence), the vFlow has a higher
precedence than those with lower values. If you’re seeing error messages about vFlow conflicts,
try adding a precedence value to new or existing vFlows.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Creating vFlows with the Scope Fabric
To create vFlows across the entire fabric, configure the vFlow with the scope fabric and stats enable option. Using
these parameters enables statistics for the flow on all switches that are members of the fabric and you can display
the statistics for any switch in the fabric.
To create a vFlow for VLAN1 with the scope fabric, use the following syntax:
CLI network-admin@switch > vflow-create name fab_flow1 scope fabric stats
enable vlan 1
To display the statistics for the new vFlow for any switch in the fabric, use the following syntax:
CLI network-admin@switch > switch switch-name vflow-stats-show name fab_flow1
name
packets
---------fab_flow1 51.4K
bytes
----13.8M
cpu-packets
----------50.1K
cpu-bytes
--------13.1M
If you omit the switch name, all vFlow statistics for the fabric are displayed.
switch
-----pleiades1
pleiades2
name
---fab_flow1
fab_flow1
packets
------1.32K
910
bytes
----305K
256K
cpu-packets
----------1.29K
884
cpu-bytes
--------291K
243K
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
211
www.pluribusnetworks.com
Example Use Cases for vFlows
The following examples illustrate how to use vFlows to impact traffic on the switch. You can regulate bandwidth,
create multiple vFlows, or share bandwidth.
Regulating Bandwidth for a VNET
To regulate bandwidth for all hosts in a VNET, create a vFlow and associate it with the appropriate flow class:
1. Create a VNET, bwvnet, using the vnet-create command:
CLI network-admin@switch > vnet-create name bwvnet scope fabric
2. All traffic associated with this VNET has a bandwidth of 5 Gbps. Create a vFlow:
CLI network-admin@switch > vflow-create name bwflow scope fabric vnet bwvnet
flow-class guaranteed-bw bw-min 5g
vflow-create:In order to use bw-min, please use vrg-modify to specify a min
bandwidth for vrg bwvnet-vrg
Creating the vFlow failed because a flow can only use the minimum bandwidth parameter if the associated VRG
(Virtual Resource Group) has minimum bandwidth allocated to it. You need to modify the VRG associated with the
VNET before assigning a minimum bandwidth to the vFlow.
3. Modify the VRG:
CLI network-admin@switch > vrg-modify name bwvnet-vrg data-bw-min 5g
4. Now create the vFlow for regulating bandwidth:
CLI network-admin@switch > vflow-create name bwflow scope fabric vnet bwvnet
flow-class guarantee-bw bw-min 5g
Informational Note: Before you assign minimum bandwidth to a vFlow, the associated VRG must have the
same bandwidth value or higher allocated to it.
You can also regulate bandwidth to a certain speed using vFlows.
5. Modify the VRG associated with the VNET:
CLI network-admin@switch > vrg-modify name bwvnet-vrg data-bw-max 5g
6. And then create the vFlow:
CLI network-admin@switch > vflow-create name bw-reg scope fabric vnet bwvnet
flow-class meter bw-max 5g
This creates a vFlow that allows bandwidth of up to 5 Gbps for all traffic on the VNET, bwvnet.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
212
Suppose you want to offer guaranteed bandwidth on a VNET, and cap the bandwidth to a fixed value. Add another
vFlow to perform this service:
CLI network-admin@switch > vflow-create name gw-bw scope fabtic vnet bwvnet
flow-class guaranteed-bw bw-min 5g bw-max 8g
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Creating Multiple vFlows for the Same VNET
You can create multiple vFlows for the same VNET and add precedence values to the vFlows. The packet is matched
to the vFlow with the highest precedence. For example,
Informational Note: You cannot create a new vFlow if a packet matches an existing flow.
1. Create the first vFlow:
CLI network-admin@switch > vflow-create name client-flow1 scope fabric vnet
bwvnet flow-class meter bw-max 2g
2. Create the second vFlow:
CLI network-admin@switch > vflow-create name client-flow2 scope fabric vnet
bwvnet flow-class meter bw-max 5g src-ip 192.168.20.1
vflow-create: Flow conflicts with Flow client-flow1, ID68: specify fields to
make flows mutually exclusive or change the flow precedence
The error message is generated because the vFlow configurations conflict with each other. To differentiate
between the two flows, assign a different precedence to client-flow2:
CLI network-admin@switch > vflow-create name client-flow2 scope fabric vnet
bwvnet flow-class meter bw-max 5g src-ip 192.168.20.1 precedence 5
Configuring Bandwidth Sharing for a Single VLAN with Different IP Addresses or Subnets
In some instances, you want to allow different subnets to share a guaranteed bandwidth on the same VNET. To do
this, you must create a VRG with the required bandwidth:
CLI network-admin@switch > vrg-create name admin-vrg vlans 100 data-bw-min 1g
data-bw-max 2g scope fabric
Pluribus Networks Configuration Guide
213
www.pluribusnetworks.com
You have now created a VRG with the guaranteed bandwidth of 1 Gbps and limited to a maximum of 2 Gbps. Now,
create a vFLow for each IP address:
CLI network-admin@switch > vflow-create name vfl-1 scope fabric vlan 100 src-ip
1.1.1.1
CLI network-admin@switch > vflow-create name vfl-2 scope fabric vlan 100 src-ip
2.2.2.2
CLI network-admin@switch > vflow-create name vfl-3 scope fabric vlan 100 src-ip
3.3.3.3
CLI network-admin@switch > vflow-create name vfl-4 scope fabric vlan 100 src-ip
4.4.4.4
In this example, the specified IP addresses each have a guaranteed bandwidth between 1 Gbps and 2 Gbps.
If you want to specify a subnet, 100.100.100.0/28, and VLAN 53 with maximum bandwidth of 50 Mbps, use the
following syntax:
CLI network-admin@switch > vrg-create name vrg-custom scope fabric data-bw-min
50M data-bw-max 50M vlan 53
CLI network-admin@switch > vflow-create name vfl-cust scope fabric src-ip
100.100.100.0 src-ip-mask 255.255.255.240 vlan 53
But later on, you found that sixteen IP addresses were not enough and you needed an additional 8 with the subnet,
101.101.101.8/29 that require the same bandwidth as the previous subnet. Use the following syntax:
CLI network-admin@switch > vflow-create name vfl-cust-2 scope fabric src-ip
101.101.101.8 src-ip-mask 255.255.255.248 vlan 53
You now have two vFlows on VLAN 53.
Then, you discover that 50 Mbps is not sufficient to support the network traffic affected by the vFlow, and you want
to upgrade to 80 Mbps:
CLI network-admin@switch > vrg-modify name vrg-custom data-bw-min 80M
data-bw-max 80M
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
214
Configuring VXLANs and Tunnels
 Configuring a VXLAN with nvOS
 Configuration Example
 Creating Tunnels
In today’s virtualized environments, there is increasing demand on MAC address tables of switches that connect to
servers. Instead of learning one MAC address per server link, the switch now has to learn the MAC addresses of
individual VMs, and if the MAC address table overflows, the switch may stop learning new MAC addresses until idle
entries age out.
Virtual Extensible LAN (VXLAN) is essentially a Layer 2 overlay scheme over a Layer 3 network, and each overlay is
called a VXLAN segment. Only VMs within the same VXLAN segment can communicate with each other. Each VXLAN
segment is identified by a 24 bit segment ID called the VXLAN Network Identifier (VNI).
VXLANs increase the scalability of your network up to 16 million logical networks and is used to contain broadcast,
multicast, and unknown unicast traffic.
Because of this encapsulation, VXLAN could also be called a tunneling scheme to overlay Layer 2 networks over top
of Layer 3 networks. However, the tunnel does not terminate on the switch, and the switch sits in the middle of the
tunnel and sees packets as L3 tunneled packets. These packets are then forwarded using L2 or L3 forwarding.
Pluribus Networks supports two scenarios for VXLAN:
1. The tunnel does not terminate on the switch and VTEP is not supported. Though the switch does not participate in
the creation of a tunnel, the following tasks are still performed.
a. Analytics Collection — All TCP control packets are captured as well as ARP packets traversing the tunnel.
These packets are used to build connection statistics and provide visibility as to which VXLAN nodes are on
specific ports.
b. ARP Optimization — An ARP request is captured and if an L2 entry exists in the switch L2 table, a response is
sent back to the sender of the ARP request over the tunnel. Otherwise, the ARP request is re-injected into the
tunnel without any modification to continue crossing the tunnel.
2. The tunnels are terminated at a switch and the switch performs the role of a VTEP. In this scenario, the switch is
responsible for encapsulating packets that arrive from non-VXLAN nodes on a L2 network and transmitting them
over the tunnel. Similarly, the packets arriving through the tunnel are decapsulated and the inner packet is forwarded over the L2 network. The switch also collects statistics and optimizes ARP requests as in the first scenario.
Informational Note: There is a one to one mapping of VXLAN to VLAN. Multicast traffic is not supported.
VXLAN has the scope local on all switches, and must be in the same subnet.
Configuring a VXLAN with nvOS
For the first scenario, no additional configuration is required. The second scenario requires the following steps, in
order:
1. Create a hardware vRouter.
2. Add interfaces to the vRouter, one per tunnel. The tunnel endpoint IP address should be routable.
3. Create one or more tunnels.
4. Create the VXLAN with the VNI, and add the tunnels created in the previous steps.
215
Pluribus Networks nvOS Version 2.3.2
To create a VXLAN, vx-seg1, with the VNID 25, scope fabric, and turn off deep inspection, use the following syntax:
CLI network-admin@switch > vxlan-create name vx-seg1 vnid 25 scope fabric
deep-inspection no
To delete a VXLAN, use the vxlan-delete command.
To display information about VXLANs, use the vxlan-show command.
If you added a port to the VXLAN configuration, use the vxlan-port-remove command.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Configuration Example
The following example assumes that one VTEP is on the generic switch and the other VTEP is on a Pluribus Networks
switch. Also, the nodes are connected on a L3 IP network, and the tunnel is formed between the generic switch and
the Pluribus Networks switch.
The example also includes VLAN 10 and port 47 on Host2 as well as the VNET fab-global.
1. Create the vRouter using the vrouter-create command:
CLI (server-switch)> vrouter-create name vx-vrouter vnet fab-global router-type
hardware
2. Add the vRouter interface:
CLI (server-switch)>vrouter-interface-add vrouter-name vx-vrouter ip 192.168.0.1
netmask 255.255.255.0 vlan 10
3. Create the tunnel:
CLI (server-switch)>tunnel-create name vx-tunnel scope local local-ip 192.168.0.1
remote-ip 192.168.5.1 next-hop 192.168.0.2 next-hop-mac 00:01:02:03:04:05 router-if
vx-router.eth0
4. Create the VXLAN:
CLI (server-switch)>vxlan-create vnid 14593470 scope local name vxlan1 vlan 10
If VLAN 10 does not exist, then the vxlan-create command creates it on the switch, but you may need to add
local ports to the VLAN.
5. Add port 47 to the VXLAN:
CLI (server-switch)>vxlan-port-add vxlan-name vxlan1 ports 47
This associates all packets from port 47 on VLAN 10 with the VXLAN ID, 14593470.
Pluribus Networks nvOS Version 2.3.2
216
6. Add the tunnel to the VXLAN:
CLI (server-switch)>vxlan-tunnel-add vxlan-name vxlan1 tunnel-name vx-tunnel
To display the configuration, use the vxlan-show command.
You cannot configure different VLANs for the tunnel and the local hosts, and you cannot associate different VLANs
on different ports for the same VXLAN.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Creating Tunnels
You can create tunnels to encapsulate protocols on the network. You can create tunnels for IP-in-IP, VXLAN, and
NVGRE network traffic. However, tunnels are supported on the local scope only and do not use any discovery
mechanism.
IP-in-IP protocol encapsulates an IP header with an outer IP header for tunneling. The outer IP header source and
destination identifies the endpoints of a tunnel. The inner IP header source and destination identify the original
sender and recipient of the datagram.
In addition to the IP header and the VXLAN header, the VTEP also inserts a UDP header. During ECMP, the switch
includes this UDP header to perform the hash function. The VTEP calculates the source port by performing the hash
of the inner Ethernet frame's header. The Destination UDP port is the VXLAN port.
The outer IP header contains the Source IP address of the VTEP performing the encapsulation. The destination IP
address is the remote VTEP IP address or the IP Multicast group address.
Network Virtualization using Generic Routing Encapsulation (NVGRE) uses GRE to tunnel Layer 2 packets over Layer
3 networks. NVGRE is similar to VXLAN but it doesn’t rely on IP multicast for address learning.
To create a tunnel for IP-in-IP traffic, local IP address 192.168.100.35, and the router, tunnel-network, use the
following syntax:
CLI network-admin@switch > tunnel-create scope local name ipinip type ip-in-ip
local-ip 192.168.100.35 router-if vrouter-hw-if eth0.0
To remove a tunnel, use the tunnel-delete command.
To modify a tunnel, use the tunnel-modify command.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
217
Pluribus Networks nvOS Version 2.3.2
Edge Virtual Bridging
Understanding Edge Virtual Bridging
Edge Virtual Bridging (EVB) is a software capability on a switch running Pluribus Networks nvOS® that allows
multiple VMs to communicate with each other and with external hosts in the Ethernet network.
Virtual Ethernet Port Aggregator (VEPA) is a software capability on a server that collaborates with an adjacent,
external switch to provide bridging support between multiple VMs and external networks. The VEPA collaborates
with the adjacent switch by forwarding all VM-originated frames to the adjacent switch for frame processing and
frame relay, including hairpin forwarding, and by steering and replicating frames received from the VEPA uplink to
the appropriate destinations.
Why Use VEPA instead of Virtual Ethernet Bridging (VEB)?
Even though VMs are capable of sending packets directly to one another with a technology called Virtual Ethernet
Bridging (VEB), physical switches are used for L2/L3 forwarding because VEB uses server hardware to accomplish the
task. Instead of using VEB, you can install VEPA on a server to offload switching functions to an adjacent physical
switch that offers less expensive L2/L3 forwarding.
Additional advantages of using VEPA include the following:
 VEPA reduces complexity and allows higher performance on the server
 VEPA takes advantage of the physical switch security and tracking features.
 VEPA provides visibility of inter-VM traffic to management tools designed for network switches.
 VEPA reduces the amount of network configuration required by server administrators, and as a consequence,
reduces workload for a network administrator.
How Does EVB Work?
EVB uses two protocols to work: Virtual Station Interface (VSI) Discovery and Configuration Protocol (VDP) and Edge
Control Protocol (ECP), to program policies for each individual virtual switch instance.
EVB maintains the following information for each VSI instance:
 VLAN ID
 VSI type
 VSI type version
 MAC address of the server
VDP is used by the VEPA server to propagate VSI information to the switch. This allows the switch to program
policies on individual VSIs and supports VM migration by implementing logic to pre-associate a VSI with a particular
interface.
ECP is an LLDP (Link Layer Discovery Protocol)-like transport layer that allows multiple upper layer protocols to send
and receive protocol data units (PDUs). ECP improves upon LLDP by implementing sequencing, retransmission and
an ACK mechanism. ECP is implemented in an EVB configuration when you configure LLDP on ports that you have
configured for EVB. In other words, you configure LLDP, not ECP.
You can configure EVB on a switch when that switch is adjacent to a server that includes VEPA technology. In general,
this is how to implement EVB:
 A network administrator creates a set of VSI types. Each VSI type is represented by a VSI type ID and a VSI
version. You can deploy one or several VSI versions at any time.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
218
 The VM administrator configures VSI which is a virtual station interface for a VM represented by a MAC address
and VLAN ID pair. The VM administrator queries available VSI type IDs (VTIDs) and creates a VSI instance
consisting of a VSI Instance ID and the chosen VTID. This instance is known as VTDB and contains a VSI manager
ID, a VSI type ID, a VSI version, and a VSI instance ID.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Configuring Edge Virtual Bridging
Remember, EVB does not convert packets, but it ensures that packets from one VM destined to another VM on the
same server are switched. When the source and destination of a packet are on the same port, EVB delivers the
packet, reflective relay, which otherwise would not happen because standard switching never forwards a packet to
the port from which it received the packet.
Before You Begin
Be sure that you have performed the following:
 Configured packet aggregation on the server connected to the port on the switch used for EVB.
 Configured the EVB port for all VLANs located on the VMs.
1. To enable VDP processing on all ports, enter the following CLI command at the prompt:
CLI network-admin@switch > vdp-modify enable
You can verify if VDP is enabled on a switch by using the vdp-show command.
2. To display the VSI instances and their state, use the vsi-state-show command:
CLI network-admin@switch > vsi-state-show
port mgrid vsiid_format
tate keepalive
49
::
mac
ASSOC
109
49
::
mac
10Gbps
20%
109
vsiid
linkspeed bw_limit traffic_class s
02:08:20:a8:13:67 10Gbps
10%
0
02:08:20:b0:25:39
0
ASSOC
3. To display ECP protocol statistics, use the following command:
CLI network-admin@switch > ecp-port-show
port ipkts opkts timeouts retransmits tx_errors last_rx_seqno last_ack_seqno
49
987
987
27
27
0
481
481
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
219
www.pluribusnetworks.com
Implementing OpenFlow with FloodLight
Floodlight Open Software Defined Network (SDN) Controller is an enterprise-class, Apache-licensed, Java-based
OpenFlow controller. It works with both physical and virtual switches that can interpret the OpenFlow protocol.
Since it is Apache licensed, you can use Floodlight for almost any purpose.
Informational Note: For more information about Floodlight Controller, go to
http://www.floodlight.org.
In this example, you create a NetZone to enable Floodlight, and use the VNET, vnet-engr, with the username
admin-opf, and the IP address 10.13.0.203/24:
CLI network-admin@switch > netzone-create name floodlight1 vnet vnet-engr user
admin-opf
netzone user password: password
confirm netzone user password: password
CLI network-admin@switch > netzone-interface-add netzone-name floodlight1 ip
10.13.0.203 netmask 24
CLI network-admin@switch > netzone-modify name floodlight1 floodlight-enable
By default, Floodlight OpenFlow Controller listens for OpenFlow protocol messages on port 6633 and exposes the
REST API to applications on port 8080.
Now, you can configure the OpenFlow daemon for the VNET, vnet-engr:
CLI network-admin@switch > openflow-connection-add name floodlight1 vlan 10
controller-ip 10.13.0.203 failmode standalone(open) control-port 6633
To begin using the Floodlight OpenFlow Controller within the NetZone, you can SSH to the NetZone using the IP
address that you configured in the previous example.
For additional documentation on using Floodlight, go to
http://docs.projectfloodlight.org/display/floodlightcontroller/Floodlight+Documentation
Pluribus Networks nvOS Version 2.3.2
220
Configuring OpenFlow
 Enabling a Virtual Network for an OpenFlow Controller
 Creating OpenFlow Controllers with Multiple VLANs
 Configuring the OpenFlow Controller
 Configuring Open Virtual Switch (OVS) for OpenFlow
OpenFlow is the first standard communications interface defined between the control and forwarding layers of an
SDN architecture. OpenFlow allows direct access to the forwarding plane and allows you to manipulate the
forwarding plane of network devices such as switches and routers, both physical and virtual. Because current
networking devices lack an open interface, it has led to the characterization of the devices as monolithic, closed, and
mainframe-like. There is no other standard protocol like OpenFlow and an OpenFlow is needed to move network
control out of the networking switches to logically centralized control software.
The OpenFlow protocol is a key enabler for software-defined networks and is currently the only standardized SDN
protocol that allows direct access and manipulation of the forwarding plane on network devices.
For more information about OpenFlow, go to http://www.opennetworking.org.
Enabling a Virtual Network for an OpenFlow Controller
You can enable OpenFlow for a virtual network (VNET) with one or more VLANs and connecting the VLANs to an
OpenFlow controller.
If the VNET assigned to OpenFlow has the scope, local, the switch ports configured for the VNET appears to the
OpenFlow controller as a traditional, standalone OpenFlow switch with those ports.
If the VNET assigned to OpenFlow has the scope, fabric, the OpenFlow controller is presented with the abstraction of
a single logical big switch containing the ports from each switch in the fabric configured for the VNET. The Pluribus
Networks Netvisor (nvOS®) ensures that the state is distributed and rules are programmed into the individual
physical switch tables as necessary to present the abstraction of a single big switch.
Informational Note: The switch supports OpenFlow version 1.0 protocol. For more information about
the OpenFlow 1.0 protocol, go to http://www.opennetworking.org/index.php.
A switch or fabric can virtualize the physical network for one or more OpenFlow networks. Use the following steps to
create a VNET:
1. Create a virtual network and assign it to a VLAN, for example, VLAN10.
CLI network-admin@switch > vnet-create name openflow-1 scope fabric vlans 10
vnet created.
You can apply the standard VNET parameters such as bandwidth guarantee by configuring a virtual resource
group (VRG).
221
Pluribus Networks nvOS Version 2.3.2
2. Create an OpenFlow service for the VNET:
CLI network-admin@switch > openflow-create name openflow-1 vnet openflow-vnet
3. Create an OpenFlow daemon for the VNET, openflow-1 with the IP address of 192.168.1.11 on port 6633. Port
6633 is the well-known port for OpenFlow.
CLI network-admin@switch > openflow-connection-add name openflow-1 vlan 10
controller-ip 192.168.1.11 control-port 6633 failmode standalone(open | secure
(timeout)
The failure mode dictates the policy to follow if OpenFlow controllers configured for the VNET are unresponsive.
In standalone(open) failure mode, the VNET performs as a legacy Layer 2 switch. When connected to
a controller again, the existing flow entries remain. The controller can then delete all flow entries.
In secure(timeout) failure mode, packets and messages sent to the OpenFlow controllers are dropped from
the network. Flows expire according to the configured timeouts.
The default failure mode is standalone(open) mode.
4. Repeat the previous step for each OpenFlow controller on the VNET. For example, you may want to configure a
primary OpenFlow controller and a secondary OpenFlow controller as a backup option.
There may be certain times that you want to reset the connection from the VNET, openflow-1, to the OpenFlow
controller. You can use the openflow-restart command to perform this action.
To remove an OpenFlow controller from a VNET, specify the IP address associated with the OpenFlow controller. For
example,
CLI network-admin@switch > openflow-connection-remove name openflow-1 vlan 10
controller-ip 192.168.1.11
To remove all OpenFlow controllers from the VNET, omit the IP address from the command.
CLI network-admin@switch > openflow-connection-remove name openflow-1 vlan 10
To check the status of an OpenFlow connections, use the openflow-connection-show command.
Pluribus Networks nvOS Version 2.3.2
222
Creating OpenFlow Controllers with Multiple VLANs
If a VNET contains multiple VLANS, then each VLAN is controlled by a separate OpenFlow controller. In this example,
you have VLANs 0, 595, and 222, IP address 10.9.21.72/16, and you are creating a fabric named corp-fabric.
CLI network-admin@switch > fabric-create name corp-fabric
CLI network-admin@switch > vnet-create name vnet-engr scope fabric vlans
595,222
CLI network-admin@switch > vnet-manager-interface-add vnet-manager-name
vnet-engr-mgr ip 10.9.21.72/16 vlan 0 if mgmt
CLI network-admin@switch > vnet-manager-interface-add vnet-manager-name
vnet-engr assignment none vlan 595
CLI network-admin@switch > vnet-manager-interface-add vnet-manager-name
vnet-engr assignment none vlan 222
CLI network-admin@switch > openflow-create name engr-openflow vnet vnet-engr
CLI network-admin@switch > openflow-connection-add name engr-openflow
controller ip 10.9.21.17 failmode secure(timeout) vlan 595
CLI network-admin@switch > openflow-connection-add name engr-openflow
controller ip 10.9.21.17 failmode secure(timeout) vlan 222
CLI network-admin@switch > vlan-port-add vlan-id 595 untagged ports 46,49
CLI network-admin@switch > vlan-port-add vlan-id 222 untagged ports 45,50
After executing these commands on the switch, the fabric is in the following state:
 OpenFlow service, engr-openflow, is created on the VNET, vnet-engr.
 OpenFlow connection, engr-openflow, is added to VLAN 595 and VLAN 222.
 Ports 46 and 49 are added to VLAN 595.
 Ports 45 and 50 are added to VLAN 222.
223
Pluribus Networks nvOS Version 2.3.2
Configuring the OpenFlow Controller
nvOS has a built-in OpenFlow controller, Floodlight, that you can enable and then explore switch information using
the OpenFlow protocol. nvOS provides commands that allows you to send and receive data from the OpenFlow
controller.
For more information about the Floodlight controller, go to http://www.projectfloodlight.org/floodlight/
1. To enable the built-in OpenFlow controller, use the following commands:
CLI network-admin@switch > netvisor-zone-create name floodlight vnet openflow-1
user admin
netzone user password: <password>
confirm netzone user password: <password>
CLI network-admin@switch > netvisor-zone-interface-add netvisor-zone
floodlight ip 192.168.11.13 netmask 24
CLI network-admin@switch > netvisor-zone-modify name floodlight
floodlight-enable
Use an IP address on your network that allows you to access the Floodlight OpenFlow controller.
2. Now add the OpenFlow daemon to the virtual network:
CLI network-admin@switch > openflow-connection-add name floodlight vlan 10
controller-ip 192.168.11.13 failmode standalone(open)control-port 6633
The failure mode dictates the policy that is followed if all OpenFlow controllers configured for the virtual network
are unresponsive.
You can now begin using your built-in Floodlight OpenFlow controller with the Netvisor Zone that you just created.
For documentation on the configuration and management steps for Floodlight, go to
http://www.projectfloodlight.org/documentation/
Configuring Open Virtual Switch (OVS) for OpenFlow
Open Virtual Switch (OVS) is a production quality, multilayer virtual switch licensed under the open source Apache
2.0 license. It is designed to enable massive network automation through programmatic extension, while still
supporting standard management interfaces and protocols, for example, NetFlow, sFlow, IPFIX, RSPAN, CLI, LACP,
and 802.1ag.
After you create OpenFlow version 1.3 on your switch, you can add OVS as your OpenFlow controller by creating a
zone in the same manner as Floodlight.
CLI network-admin@switch > openvswitch-create name openflow13 vnet openflow
dedicated-vnet-service storage-pool diskpool1 gateway 192.168.11.13
db-conn-type default db-ip 192.168.11.15 db-port 6633
And then start the OVS using the openvswitch-start command.
Pluribus Networks nvOS Version 2.3.2
224
About sFlow
Overview
Because businesses rely on network services for mission critical applications, small changes in network usage can
impact network performance and reliability. As a result, these changes can also impact a business’ ability to conduct
key business functions and increase the cost of maintaining network services.
Figure 1: Overview of sFlow
sFlow provides the visibility into network usage and active routes on the network by providing the data required to
effectively control and manage network usage. This ensures that network services provide a competitive edge to the
business.
A few examples of sFlow applications include the following:
Detecting, diagnosing, and fixing network problems
Real-time congestion management
Understanding application mixes such as P2P, Web, DNS
Usage accounting for billing
Audit trail analysis to identify unauthorized network activity and trace sources of Denial of Service (DoS)
attacks
Route profiling and optimizing peers
Trending and capacity planning
sFlow is an open source sampling tool providing constant traffic flow information on all enabled interfaces
simultaneously. sFlow data is sent to a collector that formats the data into charts and graphs while recording and
identifying trends on the network. You can use this information for troubleshooting a network, perform diagnostics,
and analysis of data.
225
Pluribus Networks nvOS Version 2.3.2
The sFlow agent on the switch samples packets from data flows and forwards headers of the sample packet to a
collector at regular intervals. You can specify the number of packets to sample from the total packets which is called
the sample rate. The packets are stored and sent to the collector at an interval that you can configure on the switch.
This is called the polling interval. You can sample different types of packets such as frames sent to the CPU or
interfaces of the switch, routed packets, flooded packets, and multicast packets. However, the following packet types
are not sampled by sFlow:
LACP frames
LLDP frames
STP RPDUs
IGMP packets
Ethernet PAUSE frames
Frames with CRC errors
PIM_HELLO packets
Packets dropped by ACLs
Packets dropped as a result of VLAN violations
Routed packets with IP options or MTU violations
Counter Sampling
For counter sampling, also called polling, the sFlow agent periodically polls the hardware interface statistics
registers, counters, in the switch chip for per port statistics, and stores them in RAM until it is time to send the next
message to the sFlow collector. Overall port statistics such as the number of broadcasts, errors, are collected by the
sFlow agent.
The agent then includes the statistics in the sFlow datagrams sent to the sFlow collector along with the packet
sampling information. From these statistics, the sFlow obtains information about the actual utilization of each port.
For instance, information about broadcast to multicast to unicast rations is captured.
When you configure the agent for counter sampling, it sends an sFlow datagram at intervals of a second, at most.
The datagram contains a snapshot of the counters cached in RAM from the most recent polling of interface
counters.
Packet Sampling
Packet sampling is used to characterize network traffic. If the sFlow agent is configured for packet sampling, the
agent takes copies of random samples of packets forwarded within the switch CPU and sends them to the switch for
processing. The CPU sends a configured portion of the sampled packet, containing a number of protocol headers
and possibly some of the payload data to the sFlow collector. Random sampling prevents the synchronization of
periodic traffic patterns. On the average, 1 in every N packets is captured analyzed. The sampling can apply to
ingress and egress frames independently. The rate that the agent sends datagrams depends on the sampling rate,
the traffic rate, and the configured maximum datagram size. Typically, several samples are included in the datagram.
Agent to Collector Datagrams
After gathering packet and counter samples, each sFlow agent creates a packet of the data and sends it to an sFlow
collector in UCP datagrams. The datagrams contain the IP address of the sFlow collector and the standard UDP
destination port number of 6343. Using a standardized port helps avoid configuration between sFlow agents and
collectors. If the sFlow agent is configured for counter sampling or packet sampling, or both, an sFlow datagram can
contain either interface counters, packet samples, or a mixture of both.
Pluribus Networks nvOS Version 2.3.2
226
The following table provides information about the contents of sFlow datagrams:
Packet Header
Information
Version
The sFlow version used on the network.
IP Address Type
An IPv4 or IPv6 address
Source IP Address
The IP address of the sFlow agent
Sequence Number
The sequence number of the datagram
System Uptime
The length of time that the system is
operational.
Sample Count
The number of samples in the datagram
Ingress Interfaces
The ifindex of the switch port where the
packets entered the agent.
Egress Interfaces
The ifindex of the switch port where the
packets exited the agent.
Sample dataset
sFlow-specific parameters:
•
•
•
•
Sequence Numbers
Sampling Rate
Total Packets available for sampling
Number of sampled packets dropped
because there was no processing resource for
them.
Packet Samples
Packet sample information and may
contain several samples.
Packet data
The sampled data that may include the
packet payload data and the number on
length of protocol headers. This
information depends on the size of the
size, up to 200 bytes.
Counter Sample
Counter statistical information - fitted in
where space permits.
If index
The ifindex of the interface related to the
counters.
Physical Interface Parameters
•
•
•
•
Speed
Duplex mode
Admin status
Operational status of the interface
In Counters
•
•
•
•
•
•
•
ifInOctets
ifInUnicastPkts
ifInMultiPkts
ifInBroadcastPkts
ifInDiscards
ifInErrors
ifInUnknownProbs
227
Pluribus Networks nvOS Version 2.3.2
Packet Header
Information
Out Counters
•
•
•
•
Promiscuous Mode
The private VLAN promiscuous mode of
the interface
Ethernet Statistics
•
•
•
•
•
•
•
•
ifOutOctets
ifOutUcastPkts
ifOutDiscards
ifOutErrors
Alignment Errors
FCS Errors
SQE Errors
Deferred Transmission
Internal MAC errors
Carrier sense errors
Overlength frame errors
Symbol errors
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks nvOS Version 2.3.2
228
Configuring sFlow
From the following network diagram, let’s configure sFlow and sFlow agents.
Figure 1: sFlow Network with IP Addresses
Configuring the sFlow Collector
Before configuring the sFlow agents, you must configure the sFlow collector. The sFlow collector receives sFlow
datagrams from the sFlow agents. In this example, the sFlow collector has an IP address of 10.1.1.243, and a default
port of 6343. The collector name is net-man-all, and the scope is fabric. If the scope is fabric, then additional
switches that join the fabric receive the sFlow collector configuration. If the scope is local, then the sFlow collector is
configured only on one switch.
CLI network-admin@switch > sflow-collector-create collector-ip 10.1.1.243
collector-port 6343 name net-man-all scope fabric
You can add as many collectors as needed for your configuration.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Enabling sFlow on the Network
You must configure and enable sFlow on each switch that you want to use for monitoring network traffic. You can
only configure one sFlow per switch.
Pluribus Networks Configuration Guide
229
www.pluribusnetworks.com
On each switch in the example diagram, use the following command to enable sFlow, net-monitor, on ingress ports
57-59, sample type raw, sample-rate 4096, sample interval 5 seconds, trunc-length 160 bytes, on VLAN 200:
CLI network-admin@switch > sflow-create name net-monitor sample-type raw ports
57-59 sample-rate 4096 trunc-length 160 vlan 200
Adding Additional Ports to sFlow
To add the ports, 61-62, to the sFlow configuration, you must use the following command on each switch:
CLI network-admin@switch > sflow-port-add sflow-name net-monitor switch
10.1.1.23 ports 61-62
In this example, the IP address of the switch is used as the name of the switch.
Removing Ports from the sFlow Configuration
You can remove ports from the sFlow configuration by using the sflow-port-remove command:
CLI network-admin@switch > sflow-port-remove sflow-name net-monitor switch
10.1.1.23 ports 61-62
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks Configuration Guide
www.pluribusnetworks.com
230
Internet Protocol Flow Information Export (IPFIX)
IPFIX (Internet Protocol Flow Information Export) is an IETF protocol created by the need for a common, universal
standard of export for Internet Protocol flow information from routers, probes and other devices that are used by
mediation systems, accounting/billing systems and network management systems to facilitate services such as
measurement, accounting and billing. The IPFIX standard defines how IP flow information is to be formatted and
transferred from an exporter to a collector.
IPFIX Architecture
A Metering Process collects data packets at an Observation Point, optionally filters them and then aggregates
information about these packets. Using the IPFIX protocol, an Exporter then sends this information to a Collector.
Exporters and Collectors are in a many-to-many relationship as one Exporter can send data to many Collectors and
one Collector can receive data from many Exporters.
IPFIX Protocol
IPFIX considers a flow to be any number of packets observed in a specific timeslot and sharing a number of
properties such as same source, same destination, or same protocol. Using IPFIX, devices such as routers can send
information to a central monitoring station about their view of a potentially larger network.
IPFIX is a push protocol, meaning each sender periodically sends IPFIX messages to configured receivers without any
interaction by the receiver.
The actual makeup of data in IPFIX messages is largely up to the sender. IPFIX introduces the makeup of these
messages to the receiver with the help of special Templates. The sender also accepts user-defined data types in the
messages, so the protocol is freely extensible and can adapt to different scenarios.
IPFIX prefers the Stream Control Transmission Protocol (SCTP) as the transport layer protocol, but also allows the use
of the Transmission Control Protocol (TCP) or User Datagram Protocol (UDP). SCTP provides some of the same
service features of both TCP and UDP. SCTP is message-oriented like UDP and ensures reliable, in-sequence
transport of messages with congestion control like TCP. It differs from the two protocols when providing
multi-homing and redundant paths to increase resilience and reliability.
IPFIX Collector
Flow collectors are able to dynamically read the templates exported by flow capable hardware and store the flows
being sent. Most IPFIX collectors provide reporting on the data and some even provide behavior analysis to help
detect network threats.
When flows from the same switches are sent to multiple IPFIX collectors, an IPFIX replicator is deployed. A replicator
speeds the process up exponentially as you can visit a single appliance to forward flows from 800 routers rather than
using telnet to update the configuration on each individual device. An IPFIX replicator is most often found in
environments where the security team wants to send the flows to more than one IPFIX collector for Internet security
reasons.
Analyzing flow data can add another layer of internet security to a company's overall network security solution.
NetFlow and IPFIX threat detection systems compile the flows received and perform network behavior analysis.
During this process, IP addresses within the flows are often compared to a constantly updated host reputation list,
and TCP flags are reviewed in an effort to identify certain types of network scans such as SYN, XMAS, RST/ACK, or
other scans. Flow ratios are also examined, which helps detect DDoS attacks.
On each Pluribus switch, nvOS embeds a real-time non-sampled IPFIX metering process, and each switch can be
configured as an IPFIX exporter. In addition, nvOS supports exporting to multiple collectors.
Pluribus Networks
231
www.pluribusnetworks.com
Bidirectional Flow Support
nvOS supports bidirectional flows for IPFIX in that every flow record contains the attribute of both endpoints. Many
flow analysis tasks benefit from association of the upstream and downstream flows of a bidirectional
communication, for example, separating answered and unanswered TCP requests, calculating round trip times, and
more. Metering processes that are not part of an asymmetric routing infrastructure, especially those deployed at a
single point through which bidirectional traffic flows, are well positioned to observe bidirectional flows (Biflows). In
such topologies, the total resource requirements for Biflow assembly are often lower if the Biflows are assembled at
the measurement interface as opposed to the IPFIX Collector. The IPFIX Protocol requires only information model
extensions to be complete as a solution for exporting Biflow data.
Information Elements
Information in messages of the IPFIX protocol is modeled in terms of Information Elements of the IPFIX information
model.
All Information Elements specified for the IPFIX protocol has the following properties defined:
 name - a unique and meaningful name for the Information Element.
 elementId - A numeric identifier of the Information Element. If this identifier is used without an enterprise
identifier, then it is globally unique, and the list of allowed values is administered by IANA. It is used for compact
identification of an Information Element when encoding Templates in the protocol.
 description - The semantics of this Information Element. It describes how the Information Element is derived
from the Flow or other information available to the observer. Information Elements of dataType string or
octetArray that have length constraints such as fixed length, minimum and/or maximum length, state these
constraints in the descriptions.
 dataType - One of the types listed in DataTypes or registered in the IANA "IPFIX Information Element Data
Types" subregistry. The type space for attributes is constrained to facilitate implementation. The existing type
space encompasses most primitive types used in modern programming languages, as well as some derived
types such as ipv4Address, that are common to this domain.
 status - The status of the specification of this Information Element. Allowed values are current and deprecated.
All newly defined Information Elements are in the current status.
 enterpriseId - You can define Information Elements without registering them with IANA, for example, for
enterprise internal purposes. For such Information Elements, the Information Element identifier is not sufficient
when the Information Element is used outside the enterprise. If specifications of enterprise-specific Information
Elements are made public and if enterprise-specific identifiers are used by the IPFIX protocol outside the
enterprise, then the enterprise-specific identifier is globally unique by combining it with an enterprise identifier.
Valid values for the enterpriseId are defined by IANA as Structure of Management Information (SMI) network
management private enterprise numbers, defined at [IANA-PEN]
Abstract Data Types Supported by IPFIX
Abstract data types unsigned8, unsigned16, unsigned32, unsigned64, signed8, signed16, signed32, and signed64
are integral data types. These data type semantics can be further specified, for example, by totalCounter,
deltaCounter, identifier, or flags.
Abstract Data Type
Description
unsigned8
Represents a non-negative integer value in the range of 0 to 255.
unsigned16
Represents a non-negative integer value in the range of 0 to 65535.
unsigned32
Represents a non-negative integer value in the range of 0 to 4294967295.
Pluribus Networks
www.pluribusnetworks.com
232
Abstract Data Type
Description
unsigned64
Represents a non-negative integer value in the range of 0 to
18446744073709551615.
signed8
Represents an integer value in the range of -128 to 127.
signed16
Represents an integer value in the range of -32768 to 32767.
signed32
Represents an integer value in the range of -2147483648 to 2147483647.
signed64
Represents an integer value in the range of
-9223372036854775808 to 9223372036854775807
float32
Corresponds to an IEEE single-precision 32-bit floating-point type
float64
Corresponds to an IEEE single-precision 64-bit floating-point type
boolean
Represents a binary value. The only allowed values are true and false.
macAddress
Represents a MAC-48 address
octetArray
Represents a finite-length string of octets.
string
Represents a finite-length string of valid characters from the Unicode coded
character set. Unicode incorporates ASCII and the characters of many other
international character sets.
dateTimeSeconds
Represents a time value expressed with second-level precision.
dateTimeMilliseconds
Represents a time value expressed with millisecond-level precision.
dateTimeMicrosecond
Represents a time value expressed with microsecond-level precision
dateTimeNanoseconds
Represents a time value expressed with nanosecond-level precision.
ipv4Address
Represents an IPv4 address.
ipv6Address
Represents an IPv6 address.
basicList
Supports structured data export.
subTemplateList
Supports structured data export.
subTemplateMultiList
supports structured data export.
Pluribus Networks
233
www.pluribusnetworks.com
Data Type Semantics Supported by IPFIX
These semantics apply only to numeric types, as noted in the description of each semantic below.
Abstract Data Type
Description
quantity
A numeric (integral or floating point) value representing a measured value
pertaining to the record. This is distinguished from counters that represent an
ongoing measured value whose "odometer" reading is captured as part of a
given record. This is the default semantic type of all numeric data types.
totalCounter
an integral value reporting the value of a counter. Counters are unsigned and
wrap back to zero after reaching the limit of the type. For example, an
unsigned64 with counter semantics continues to increment until reaching the
value of 2**64 - 1. At this point, the next increment will wrap its value to zero
and continue counting from zero. The semantics of a total counter is similar to
the semantics of counters used in the Simple Network Management Protocol
(SNMP), such as Counter32 . The only difference between total counters and
counters used in SNMP is that the total counters have an initial value of 0. A total
counter counts independently of the export of its value.
deltaCounter
An integral value reporting the value of a counter. Counters are unsigned and
wrap back to zero after reaching the limit of the type. For example, an
unsigned64 with counter semantics continues to increment until reaching the
value of 2**64 - 1. At this point, the next increment wraps its value to zero and
continue counting from zero. The semantics of a delta counter is similar to the
semantics of counters used in SNMP, such as Counter32. The only difference
between delta counters and counters used in SNMP is that the delta counters
have an initial value of 0. A delta counter is reset to 0 each time it is exported
and/or expires without export.
identifier
An integral value that serves as an identifier. Specifically, mathematical
operations on two identifiers (aside from the equality operation) are
meaningless. For example, Autonomous System ID 1 * Autonomous System ID 2
is meaningless. Identifiers MUST be one of the signed or unsigned data types.
flags
An integral value that represents a set of bit fields. Logical operations are
appropriate on such values, but other mathematical operations are not. Flags
MUST always be of an unsigned data type.
Information Elements Supported by nvOS and IPFIX
Data Field
Element Name
ID
Description
proto
4
The value of the protocol number in the IP
unsigned8
packet header.
The protocol number identifies the IP packet
payload type. Protocol numbers are defined in
the IANA Protocol Numbers registry.
protocolIdentifier
Pluribus Networks
www.pluribusnetworks.com
Data Type
Units
Data Type
Semantic
identifier
234
Data Field
Element Name
ID
Description
Data Type
Units
cur-state
6
tcpControlBits
TCP control bits observed for the packets of
unsigned16
this Flow. This information is encoded as a bit
field. For each TCP control bit, there is a bit in
this set. The bit is set to 1 if any observed
packet of this Flow has the corresponding TCP
control bit set to 1. The bit is cleared to 0
otherwise.
flags
src-port
7
sourceTransportPort
The source port identifier in the transport
unsigned16
header. For the transport protocols UDP, TCP,
and SCTP, this is the source port number in the
respective header. This field MAY also be used
for future transport protocols with 16-bit
source port identifiers.
identifier
src-ip
8
sourceIPv4Address
The IPv4 source address in the IP packet
header.
default
src-switch-port
10
ingressInterface
The index of the IP interface where packets of unsigned32
this Flow are received. The value matches the
value of managed object 'ifIndex'. Note that
ifIndex values are not assigned statically to an
interface and that the interfaces may be
renumbered every time the device's
management system is re-initialized,
identifier
dst-port
11
destinationTransportPort
The destination port identifier in the transport unsigned16
header. For the transport protocols UDP, TCP,
and SCTP, this is the destination port number
in the respective header. This field MAY also
be used for future transport protocols
with 16-bit destination port identifiers.
identifier
dst-ip
12
destinationIPv4Address
The IPv4 destination address in the IP packet
header.
default
dst-switch-port
14
egressInterface
The index of the IP interface where packets of unsigned32
this Flow are sent. The value matches the
value of managed object 'ifIndex' .
Note that ifIndex values are not assigned
statically to an interface and that the
interfaces may be renumbered every time the
device's management system is re-initialized.
started-time
21
flowEndSysUpTime
The relative timestamp of the last packet of
unsigned32
this Flow. It indicates the number of
milliseconds since the last (re-)initialization of
the IPFIX Device (sysUpTime). sysUpTime can
be calculated from
systemInitTimeMilliseconds.
milliseconds
ended-time
22
flowStartSysUpTime
The relative timestamp of the first packet of
unsigned32
this Flow. It indicates the number of
milliseconds since the last (re-)initialization of
the IPFIX Device (sysUpTime). sysUpTime can
be calculated from
systemInitTimeMilliseconds.
milliseconds
ipv4Address
ipv4Address
identifier
Pluribus Networks
235
Data Type
Semantic
www.pluribusnetworks.com
Data Field
Element Name
ID
Description
Data Type
Units
Data Type
Semantic
40
exportedOctetTotalCount
The total number of octets the Exporting
unsigned64
Process has sent since the Exporting Process
(re-)initialization to a particular Collecting
Process. The value of this Information Element
is calculated by summing up the IPFIX
Message Header length values of all IPFIX
Messages successfully sent to the Collecting
Process. The reported number excludes octets
in the IPFIX Message that carries the counter
value.
If this Information Element is sent to a
particular Collecting Process, then by default,
it specifies the number of octets sent to the
Collecting Process.
octets
totalCount
er
41
exportedMessageTotalCount
The total number of IPFIX Messages the
unsigned64
Exporting Process has sent since the Exporting
Process (re-)initialization to a particular
Collecting Process. The reported number
excludes the IPFIX Message that carries the
counter value. If this Information Element is
sent to a particular Collecting Process, then by
default, it specifies the number of IPFIX
Messages sent to the Collecting Process.
messages
totalCount
er
42
exportedFlowRecordTotalCount
The total number of Flow Records that the
unsigned64
Exporting Process has sent as Data Records
since the Exporting Process (re-)initialization
to a particular Collecting Process. The
reported number excludes Flow Records in the
IPFIX Message with the counter value.
If this Information Element is sent to a
particular Collecting Process, then by default,
it specifies the number of Flow Records sent
to the process.
flows
totalCount
er
55
postIpClassOfService
The definition of this IE is identical to the
definition of IE 'ipClassOfService', except it
reports a potentially modified value caused by
a middlebox function after the packet passed
the Observation Point.
src-mac
56
sourceMacAddress
The IEEE 802 source MAC address field.
macAddress
default
vlan
58
vlanId
Virtual LAN identifier associated with ingress
interface.
unsigned16
identifier
dst-mac
80
destinationMacAddress
The IEEE 802 source MAC address field.
macAddress
default
dst-port
96
applicationName
Specifies the name of an application.
string
default
Pluribus Networks
www.pluribusnetworks.com
236
Data Field
Element Name
ID
Description
Data Type
Units
cur-state
136
flowEndReason
The reason for Flow termination.The range of unsigned8
values includes the following:
• 0x01: idle timeout —The Flow was
terminated because it was considered to be
idle.
• 0x02: active timeout — The Flow was
terminated for reporting purposes while it
was still active, for example, after the
maximum lifetime of unreported Flows was
reached.
• 0x03: end of Flow detected — The Flow was
terminated because the Metering Process
detected signals indicating the end of the
Flow, for example, the TCP FIN flag.
• 0x04: forced end — The Flow was
terminated because of some external event,
for example, a shutdown of the Metering
Process initiated by a network management
application.
• 0x05: lack of resources — The Flow was
terminated because of lack of resources
available to the Metering Process and/or
the Exporting Process.
identifier
TBD
145
templateId
unsigned16
An identifier of a Template that is locally
unique within a combination of a Transport
session and an Observation Domain.
Template IDs 0-255 are reserved for Template
Sets, and Options Template Sets.
Template IDs of Data Sets are numbered.
Typically, this IE is used for limiting the scope
of other IEs.
Note that after a re-start of the Exporting
Process Template,
identifiers may be re-assigned.
identifier
started-time
150
flowStartSeconds
The absolute timestamp of the first packet of
this Flow.
dateTimeSeco seconds
nds
default
ended-time
151
flowEndSeconds
The absolute timestamp of the last packet of
this Flow.
dateTimeSeco seconds
nds
default
started-time
158
flowStartDeltaMicroseconds
A relative timestamp only valid within the
unsigned32
scope of a single IPFIX Message. It contains
the negative time offset of the first observed
packet of this Flow relative to the export time
specified in the IPFIX Message Header.
microseconds
ended-time
159
flowEndDeltaMicroseconds
A relative timestamp only valid within the
unsigned32
scope of a single IPFIX Message. It contains
the negative time offset of the last observed
packet of this Flow relative to the export time
specified in the IPFIX Message Header.
microseconds
Pluribus Networks
237
www.pluribusnetworks.com
Data Type
Semantic
Data Field
dscp
Element Name
ID
Description
173
flowKeyIndicator
This set of bit fields is used for marking the IEs unsigned64
of a Data Record serving as a Flow Key. Each
bit represents an Information Element in the
Data Record with the n-th bit representing the
n-th Information Element.
A bit set to value 1 indicates that the
corresponding Information Element is a Flow
Key of the reported Flow.
A bit set to value 0 indicates that this is not the
case. If the Data Record contains more than 64
IEs, the corresponding Template SHOULD be
designed such that all Flow Keys are among
the first 64 IEs, because the flowKeyIndicator
only contains 64 bits. If the Data Record
contains less than 64 IEs, then the bits in the
flowKeyIndicator with no corresponding IE
MUST have the value 0.
flags
195
IpDiffServCodePoint
The value of a Differentiated Services Code
Point (DSCP)
encoded in the Differentiated Services field.
The Differentiated Services field spans the
most significant 6 bits of the IPv4 TOS field or
the IPv6 Traffic Class field, respectively.
This IE encodes only the 6 bits of the
Differentiated Services field. Therefore, the
value may range from 0 to 63.
unsigned8
identifier
211
CollectorIPv4Address
An IPv4 address to which the Exporting
Process sends Flow
information.
ipv4Address
default
213
exportinterface
The index of the interface where IPFIX
unsigned32
Messages sent by the Exporting Process to a
Collector leave the IPFIX Device. The value
matches the value of managed object 'ifIndex'.
Note that ifIndex values are not assigned
statically to an interface, the interfaces may be
renumbered every time the device's
management system is re-initialized,
identifier
214
exportProtocolVersion
The protocol version used by the Exporting
Process for sending Flow information. The
protocol version is given by the value of the
Version Number field in the Message Header.
The protocol version is 10 for IPFIX.
A value of 0 indicates that no export protocol
is in use.
identifier
Pluribus Networks
www.pluribusnetworks.com
Data Type
Units
Data Type
Semantic
unsigned8
238
Data Field
Element Name
ID
Description
Data Type
Units
215
exportTransportProtocol
The value of the protocol number used by the unsigned8
Exporting Process for sending Flow
information.
The protocol number identifies the IP packet
payload type.
Protocol numbers are defined in the IANA
Protocol Numbers
registry.
In Internet Protocol version 4 (IPv4), this is
carried in the
Protocol field. In Internet Protocol version 6
(IPv6), this is carried in the Next Header field
in the last extension header of the packet.
identifier
216
collectorTransportPort
The destination port identifier used by the
Exporting process to send Flow information.
For the transport protocols UDP, TCP, and
SCTP, this is the destination port number.
This field MAY also be used for future
transport protocols with 16-bit source port
identifiers.
unsigned16
identifier
217
exporterTransportPort
unsigned16
The source port identifier used bythe
Exporting
Process to send Flow information. For the
transport protocols UDP, TCP, and SCTP, this is
the source port number.
This field MAY also be used for future
transport protocols
with 16-bit source port identifiers. This field
maybe useful for distinguishing multiple
Exporting Processes
that use the same IP address.
identifier
218
tcpSynTotalCount
The total number of packets of a Flow with
TCP "Synchronize sequence numbers" (SYN)
flag set.
unsigned64
packets
totalCount
er
219
tcpFinTotalCount
The total number of packets of a Flow with
unsigned64
TCP "No more data from sender" (FIN) flag set.
packets
totalCount
er
222
tcpAckTotalCount
The total number of packets of a Flow withTCP unsigned64
"Acknowledgment field significant" (ACK) flag
set.
packets
totalCount
er
obytes
231
InitiatorOctets
The total number of Layer 4 payload bytes in a unsigned64
flow from the initiator. The initiator is the
device triggering the session creation, and
remains the same for the life of the session.
octets
deltaCount
er
ibytes
232
responderOctets
The total number of Layer 4 payload bytes in a unsigned64
flow from the responder. The responder is the
device that replies to the initiator, and remains
the same for the life of the session.
octets
deltaCount
er
Pluribus Networks
239
www.pluribusnetworks.com
Data Type
Semantic
Data Field
Element Name
ID
Description
0x01
239
biflowDirection
unsigned8
A description of the direction assignment
method used to assign the Biflow Source and
Destination. This IE MAY be present in a Flow
Data Record, or applied to all flows exported
from an Exporting Process or Observation
Domain using IPFIX Options.
If this IE is not present in a Flow Record or
associated with a Biflow using a scope, it is
assumed that the configuration of the
direction assignment method is done
out-of-band.
Note that when using IPFIX Options to apply
this IE to all flows within an Observation
Domain or from an Exporting Process, the
Option SHOULD be sent reliably. If reliable
transport is not available, for example, using
UDP, this IE SHOULD appear in each Flow
Record. This field may take the following
values:
• 0x00: arbitrary — Direction was assigned
arbitrarily.
• 0x01: initiator — The Biflow Source is the
flow initiator, as determined by the
Metering Process' best effort to detect the
initiator.
• 0X02: reverseInitiator — The Biflow
Destination is the flow initiator, as
determined by the Metering Process' best
effort to This value is provided for the
convenience of Exporting Processes to
revise and initiator estimate without
re-encoding the Biflow Record.
• 0x03: perimeter — The Biflow Source is the
endpoint outside of a defined perimeter.
The perimeter's definition is implicit in the
set of Biflow Source and Biflow Destination
addresses exported in the Biflow Records.
identifier
src-switch-port
252
ingressPhysicalInterface
The index of a networking device's physical
interface, for example, a switch port, where
the flow packets are received.
unsigned32
unsigned32
dst-switch-port
253
egressPhysicalInterface
The index of a networking device's physical
interface for example, a switch port, where
the flow packets are sent.
unsigned32
identifier
ether-type
256
ethernetType
The Ethernet type field of an Ethernet frame
identifying the MAC client protocol carried in
the payload.
unsigned16
identifier
Pluribus Networks
www.pluribusnetworks.com
Data Type
Units
Data Type
Semantic
240
Data Field
Element Name
ID
Description
Data Type
Units
257
postIpPrecedence
The definition of this Information Element is
unsigned8
identical to the definition of IE 'ipPrecedence',
except that it reports a potentially modified
value caused by a middlebox function after
the packet passed the Observation Point.
identifier
258
CollectionTimeMilliseconds
The absolute timestamp at which the data
within the scope containing this IE was
received by a Collecting Process. This IE
SHOULD be bound to the containing IPFIX
Message through IPFIX Options and the
messageScope IE.
default
259
exportSctpStream
The value of the SCTP Stream Identifier used unsigned16
by the Exporting Process for exporting IPFIX
Message data. This is carried in the Stream
Identifier field of the header of the SCTP DATA
chunk containing the IPFIX Message(s).
identifier
260
maxExportSeconds
The absolute Export Time of the latest IPFIX
dateTimeSeco seconds
Message within the scope containing this IE.
nds
The IE SHOULD be bound to the containing
IPFIX Transport Session through IPFIX Options
and the sessionScope IE.
default
261
maxFlowEndSeconds
The latest absolute timestamp of the last
dateTimeSeco seconds
packet within any Flow with the scope
nds
containing this IE, rounded up to the second if
necessary. This IE SHOULD be bound to the
containing IPFIX Transport Session through
IPFIX Options and the sessionScope IE.
default
262
messageMD5Checksum
The MD5 checksum of the IPFIX Message
octetArray
containing this record. The IE SHOULD be
bound to the containing IPFIX Message
through an options record and the
messageScope IE, and SHOULD appear only
once in a given IPFIX Message. To calculate the
value of this IE, first buffer the containing
IPFIXMessage, setting the value of the IE to all
zeroes. Then calculate the MD5 checksum of
the resulting buffer, place the resulting value
in the IE, and export the buffered message.
The IE is intended as a simple checksum only.
Therefore collision resistance and algorithm
agility are not required, and MD5 is an
appropriate message digest.
This Information Element has a fixed length of
16 octets.
default
349
virtualStationUUID
Unique Identifier of a Virtual Station. A Virtual octetArray
Station is an end station instance. It can be a
virtual machine or a physical host.
default
350
virtualStationName
Name of a Virtual Station. A Virtual Station is
an end station instance. It can be a virtual
machine or a physical host.
default
dateTimeSeco milliseconds
nds
string
Pluribus Networks
241
www.pluribusnetworks.com
Data Type
Semantic
Data Field
Element Name
ID
Description
Data Type
Units
Data Type
Semantic
351
layer2Segment
The identifier of a Layer 2 network segment in unsigned64
an overlay network.
The most significant byte identifies the Layer 2
network overlay network encapsulation type:
• 0x00 reserved
• 0x01 VxLAN
• 0x02 NVGRE
The three lowest significant bytes hold the
value of the Layer 2 overlay network segment
identifier.
For example:
• a 24 bit segment ID VXLAN Network
Identifier (VNI)
• a 24 bit Tenant Network Identifier (TNI) for
NVGRE
identifier
368
ingressInterfaceType
The type of interface where packets of this
Flow are received.
The value matches the value of managed
object 'ifType'.
unsigned32
identifier
369
egressInterfaceType
The type of interface where packets of this
Flow are sent.
The value matches the value of managed
object 'ifType'.
unsigned32
identifier
401
transportOctetDeltaCount
The number of octets, excluding IP header(s)
and Layer 4 transport protocol header(s),
observed for this Flow at the Observation
Point since the previous report.
unsigned64
octets
deltaCount
er
Configuring IPFIX
To configure IPFIX from the CLI, you must have a host IP address as the destination for the IPFIX collector. nvOS uses
port 9090 by default, and the default transport protocol type is TCP.
CLI network-admin@switch > ipfix-collector-create name ipfix-host1 port 9090
transport-protocol tcp dscp 3
To enable the IPFIX service, use the command, ipfix-service-modify enable. You can also set the
collection interval using this command. To set the collection interval to one hour, use the following syntax:
CLI network-admin@switch > ipfix-service-modify enable export-interval
0d1h0m0s
Pluribus Networks
www.pluribusnetworks.com
242
Pluribus Networks
243
www.pluribusnetworks.com
About
Pluribus Networks
Pluribus Networks provides data center solutions
that allow your business to run unconstrained.
Our software-defined, open networking, fabricbased solutions transform existing network
infrastructures into flexible and strategic assets
fully aligned with today’s digital business needs.
Our Virtualization-Centric Fabric (VCF™)
architecture provides unprecedented insight,
agility and security to customers seeking to
simplify operations, run more cost effectively
and bring new applications online faster.
Learn more at www.pluribusnetworks.com and
@pluribusnet.
Pluribus Networks, Inc.
2455 Faber Place, Suite 100, Palo Alto, CA 94303
1-855-GET-VNET / +1 650-289-4717
Copyright© 2016 Pluribus Networks, Inc. All rights reserved.
P/N 17-0005 Rev A
January 2016