Download nvOS Introduction 5 Introduction to nvOS Fabric

nvOS Introduction 5
Introduction to nvOS Fabric 13
Adding Switches to the Fabric 13
Displaying Fabric Statistics 17
Displaying Information about Nodes in the Fabric 18
Using the Fabric Transaction Commands 20
Displaying Fabric Statistics 22
Troubleshooting the Fabric 22
Configuring Basic Server-Switch Functionality 25
Using the Serial Console Port for Initial Configuration 25
Changing Other Switch Setup Parameters 27
Creating an Initial Fabric 29
Adding License Keys to nvOS 29
Modifying and Upgrading Software 31
Updating nvOS on the Server-Switch 32
Saving and Restoring Server-Switch Configurations 35
Changing the IP Port for vManage 38
Configuring Virtual Network Interface Cards (vNICs) 39
Displaying Layer 2 Networking Details 41
Rebooting, Powering Off, and Resetting the Server-Switch 43
Installing the nvOS Linux API 44
Configuring Rapid Spanning Tree Protocol (RSTP) 45
Configuring Link Aggregation Control Protocol (LACP) 49
Configuring Trunking for Link Aggregation (LAG) 50
Configuring Layer 2 Multipathing for Virtual Chassis Link Aggregation 51
Configuring Active-Active VLAG 54
Configuring Tagged and Untagged VLANs 59
Displaying VLAN Statistics 61
Implementing Virtual Networks 63
Overview 63
Specifying the Type of VNET Interface 64
Creating a Virtual Network (VNET) 65
Related Tasks 65
Creating a Virtual Network 67
Adding DHCP Service to a VNET 68
Verify Administrator User Creation 69
Configuring Administration Login Using SSH 69
Adding a Default Gateway to the VNET 72
Adding Ports to the VNET 73
Configuring Virtual Resource Groups 77
About Virtual Resource Group (VRG) Bandwidth Enforcement 78
Configuring Network Services - DHCP and DNS 79
Overview of DHCP and DNS 79
Configuring IP Pools 81
Configuring DHCP Services 83
Adding DHCP Interfaces 84
Adding DHCP and DNS Records 84
Removing DHCP and DNS Services 85
Pluribus Networks nvOS Version 2.3.2
i
Configuring DNS Services 87
Adding a DNS Server 87
Overview of NAT and Hardware NAT 89
Hardware NAT 89
NAT and Hardware NAT Use Cases and Scenarios 90
Configuring Network Address Translation Services 93
Configuring Port Forwarding for NAT 94
Configuring Static NAT 94
Configuring Hardware-based Network Address Translation(NAT) 97
nvOS System Logging and SNMP 99
Configuring System Logging 101
Sending Log Messages to Syslog Servers 102
Viewing Log Events 105
Sending Log Messages to Syslog Servers 108
Configuring SNMP 111
SNMP Communities 111
Users and SNMPv3 111
Supported MIBs 115
High Availability 121
Configuring a Cluster 121
Configuring Fabric-based Physical Storage Pools 125
Creating Virtual Storage for a Virtual Network (VNET) 128
Managing Host Operating Systems 128
Provisioning Bare Metal Servers 130
External Disk Drive Installation Guide 135
Configuring High Availability for Storage Folders 137
Configuring a Linux Netvisor KVM 139
Creating a Disk-based Netvisor KVM 141
Creating a KVM by Importing an ISO Image 143
Adding Virtual Machine (VM) Instances to the Server-Switch 143
Managing Linux VM Images 147
Configuring and Implementing NetZones 149
Overview 149
Configuring a NetZone 149
Configuring vRouter Services 153
Overview 153
Configuring Prefix Lists for BGP and OSPF 153
Configuring Packet Relay for DHCP Servers 154
Configuring Hardware Routing for a vRouter 154
Configuring BGP on a vRouter 157
Additional BGP Parameters 161
Configuring Open Shortest Path First (OSPF) 163
Adding Areas and Prefix Lists to OSPF 165
Configuring Routing Information Protocol (RIP) 167
Configuring Static Routes 169
Adding IGMP Static Joins to a vRouter 171
Configuring Virtual Router Redundancy Protocol 173
Configuring Virtual Load Balancing 177
ii
Pluribus Networks nvOS Version 2.3.2
Configuring Virtual Load Balancing with Ubuntu 11.04 Servers and nvOS 181
Adding Virtual Router Redundancy Protocol to VLB Interfaces 186
Configuring Roles and Users 189
Configuring TACACS+ 193
About TACACS+ 193
Configuring TACACS+ 194
Creating and Implementing Access Control Lists (ACLs) 195
Using a Deny IP ACL to Block Network Traffic 195
Using IP ACLs to Allow Network Traffic 196
Using MAC ACLs to Deny Network Traffic 198
Using MAC ACLs to Allow Network Traffic 198
Configuring IP ACLs 201
Configuring an Internal Deny ACL 201
Configuring an External Deny ACL 201
Configuring an External Allow IP ACL 202
Configuring a MAC ACL to Deny Network Traffic 202
Configuring a MAC ACL to Allow Network Traffic 203
Configuring vFlow for Analytics 205
Using vFlows to Disable Communication 209
Configuring Mirroring for vFlows and Ports 211
Managing Traffic Classes 213
Using Application Flows and Statistics 215
Displaying Standard Statistics 215
Understanding vFlow Statistics 217
Example Use Cases for vFlows 221
Configuring VXLANs and Tunnels 225
Creating Tunnels 227
Edge Virtual Bridging 229
Understanding Edge Virtual Bridging 229
Configuring Edge Virtual Bridging 230
Implementing OpenFlow with FloodLight 231
Configuring OpenFlow 233
Enabling a Virtual Network for an OpenFlow Controller 233
Creating OpenFlow Controllers with Multiple VLANs 235
Configuring the OpenFlow Controller 236
Configuring Open Virtual Switch (OVS) for OpenFlow 236
About sFlow 237
Overview 237
Configuring sFlow 241
Configuring the sFlow Collector 241
Enabling sFlow on the Network 241
Adding Additional Ports to sFlow 242
Pluribus Networks nvOS Version 2.3.2
iii
iv
Pluribus Networks nvOS Version 2.3.2
THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE
WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO
BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE
FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.
THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE
INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF
YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR PLURIBUS
NETWORKS REPRESENTATIVE FOR A COPY.
NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE ARE PROVIDED “AS IS”
WITH ALL FAULTS. PLURIBUS NETWORKS DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING WITHOUT
LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR
ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.
IN NO EVENT SHALL PLURIBUS NETWORKS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL
DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA, ARISING OUT OF THE
USE OR INABILITY TO USE THIS MANUAL, EVEN IF PLURIBUS NETWORKS HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
Any Internet Protocol (IP) addresses used in this document are not intended to be actual addresses. Any examples,
command display output, and figures included in the document are shown for illustrative purposes only. Any use of
actual IP addresses in illustrative content is unintentional and coincidental.
© 2016 PLURIBUS NETWORKS, INC. ALL RIGHTS RESERVED.
Pluribus Networks nvOS Version 2.3.2
v
vi
Pluribus Networks nvOS Version 2.3.2
Preface
This preface includes the following sections:
 Audience
 Organization
 Conventions
 Related Documentation
 Obtaining Documentation and Submitting a Service Request
This preface describes the audience, organization, and conventions of this publication, and provides information
about obtaining related documentation.
Audience
This publication is for experienced network administrators responsible for configuring and maintaining Pluribus
Networks switches with some expertise in the following areas:
 Network administration
 Storage administration
 Server administration
 Application delivery administration
 Network security administration
Organization
This publication is organized as follows:
 Layer 2 and Layer 3 Services
 VNETs, Network Services, Cluster (SDF)
 vFlows, OpenStack, OpenFlow, and Netvisor features
Conventions
This document uses the following conventions:
Table 2: CLI Conventions
Convention
Indication
Bold font
Keywords, user interface elements, and user-entered text appear in bold
font.
Italic font
Document titles, new or emphasized terms, and variables that you
supply values are in italic font.
[]
Elements in square brackets are optional.
{x|y|z}
Required elements are grouped in curly braces and are separated by
vertical bars.
[x|y|z]
Optional parameters are grouped in brackets and separated by vertical
bars.
String
A non-quoted set of characters. Do not use quotation marks around the
string or the string includes the quotation marks.
Pluribus Networks nvOS Version 2.3.2
1
Table 2: CLI Conventions
Convention
Indication
courier font
Command Line Interface (CLI) commands and samples appear in
courier font.
<>
Nonprinting characters such as passwords are indicated by angle
brackets.
[]
Default responses to system prompts are in angle brackets.
CLI
network-admin@switch
>Indicates that you enter the following text at the command prompt.
Informational Note:
Indicates information of special interest.
Indicates a situation that could cause equipment failure or loss of data.
TIP!
TIP!Indicates information that can help you solve a problem.
Timesaver:
Indicates information that can help you save time.
Related Documentation
The Pluribus Networks switch nvOS documentation set includes the following publications:
 Pluribus Networks Hardware Installation Guide
 Pluribus Networks vManage® Administrative Guide
2
Pluribus Networks nvOS Version 2.3.2
 Release Notes for Pluribus Networks nvOS Releases
 Pluribus Networks nvOS Configuration Guide
 Pluribus Networks Command Reference
For a complete list of all Pluribus Networks documentation, see the Pluribus Networks support site at
www.plurisbusnetworks.com/support.
Additional documentation describing log messages and MIBs are also available for download at
www.plurisbusnetworks.com/support.
Documentation Feedback
To provide technical feedback on this document, or to report an error or omission, please send your comments to
[email protected]. We appreciate your feedback.
Obtaining Documentation and Submitting a Service Request
For information on obtaining documentation, submitting a service request, and gathering additional information,
please visit www.pluribusnetworks.com/support.
Pluribus Networks nvOS Version 2.3.2
3
4
Pluribus Networks nvOS Version 2.3.2
nvOS Introduction
This chapter provides information for understanding and using the Pluribus Networks nvOS command line interface
(CLI) on a Pluribus Networks switch. Included in this chapter is the following information:
 Before You Start
 Important Terms
 Entering Commands and Getting Help
 Finding Command Options
 Understanding Role-based Access Control
 Specifying IP Address Netmasks
 Specifying Capacity, Throughput, and Scale
 Customizing Show Output Formats
 Using the CLI String Search
 Specifying a Switch or Fabric for Command Scope
Before You Start
Pluribus Netvisor uses the concept of Fabric-Cluster to describe the interconnectivity of devices into a single logical
network. The concept of a virtual network (VNET) describes is a “slice” of resources that apply to a single entity with
assigned resources within the fabric. The VNET contains services and resources that apply only to that VNET.
Typically, VNETs are used to house different tenants within a single large network. For switches with nvOS, the only
available VNET is a global VNET created when a fabric is created for the first time.
Since you are just getting started with your switches and Netvisor, you may decide to configure a single switch first
or you may have purchased a single switch. It’s important to understand that a single switch can consist of a single
fabric and a single VNET, or many switches and VNETs.
Important Terms
The following list of important terms and concepts and their definitions is important for understanding Pluribus
Networks features and determine the best configuration to meet your needs.
Term
Meaning
API
Application Programming Interface to the Pluribus Networks switch.
It has a similar scope as the CLI.
CLI
Command Line Interface to the Pluribus Networks switch. Depending
on the command, it can be executed for an individual switch, a
cluster, or a fabric.
Cluster
A pair of Pluribus Networks switches configured as a high availability
group. You can configure many clusters in the fabric, but a switch can
be a member of only one cluster.
Disk-library
Virtual machine storage within a storage pool.
eth0...ethX
Virtual network interface names associated with virtual services.
Fabric
A set of Pluribus Networks switches configured as a single entity. Any
switch can be a member of only one fabric. Up to 4096 switches can
be configured a single fabric.
Pluribus Networks nvOS Version 2.3.2
5
Term
Meaning
Flow
A communication from one device outside of the fabric to another
device outside of the fabric and traveling through the fabric.
GUI
Graphic User Interface to the Pluribus Networks switch. It has a
similar scope as the CLI.
In-band Management
Address
The IP address of the switch on a production or management
network for administration and inter-switch communication.
ISO- library
ISO (operating system) image storage within a storage pool
LACP
Link Aggregation Control Protocol allows a non-Pluribus Networks
device to have multiple connections to the same switch, for example,
IEEE 802.3ad trunks.
Netvisor Zone, Netvisor
KVM, Netvisor VMM
A virtual machine running within the Pluribus Networks switch. A
NetZone runs natively on the Unix-compatible operating system. A
NetVM allows the use of arbitrary x86 operating systems and
applications.
Server-switch
A Pluribus Networks hardware device with aspects of both a server
and a switch.
Storage-device
Disk or PCI-based storage connected to the switch.
Storage-pool
Storage in a RAID set available for use by storage commands.
Storage-folder
General purpose file sharing system available within a storage pool.
vFlow
A logical, manageable connection within or through the fabric.
VLAG
Virtual Link Aggregation Group is the Pluribus Networks method for
multiple connecting hosts to multiple switches, switches to each
other, and switches to other switches.
VNET
A virtual network configured within the fabric. All traffic within one
VNET segregated from the traffic of all other VNETs. A VNET is an
administrative entity as well, limiting the effects of changes to a
single VNET. Everything in a Pluribus Networks server is associated
with a VNET.
Entering Commands and Getting Help
Commands, options, and arguments are entered at the CLI prompt. A command name must be typed, but included
command-completion and help features contribute to the command entry process.
To display a list of command that you can use within a command mode, enter a question mark (?), or use the tab
key, or type help at the command prompt. You can also display keywords and arguments for each command with
this context-sensitive help feature. You can also use complete commands and display keywords and arguments for
each command using the tab key to assist with context-sensitive command help and completion.
6
Pluribus Networks nvOS Version 2.3.2
Table 3 lists the command that you can enter to get help specific to a command, keyword, or argument.
Table 3: Getting Help
abbreviated- command-entry?
Displays a list of commands that begin with a specific
character string. Do not leave a space between the string
and question mark.
abbreviated- command-entry <tab>
Completes a partial command name.
?
Lists all commands.
command ?
Lists all keywords for the command. Leave a space
between the command and the question mark.
command keyword ?
Lists all arguments for the keyword. Leave a space
between the command and the question mark.
Where a text string is used, such as name-string, the following characters are allowed as part of the text string:
a-z, A-Z, 0-9, _ (underscore), . (period), , (comma), : (colon), and - (dash).
Informational Note: If you enter a command that is invalid, then using the ? and tab key have no effect and
do not return any changes to the CLI.
Informational Note: The CLI has an editing ability similar to UNIX and Linux functionality using emacs keys.
For example, ˄p steps backward through previous commands, ˄n moves to the next command in the history, ˄a
moves to the first character in the command and ˄e moves to the end of the line, ˄u erases the current line, and
˄w erases the previous word.
Informational Note: Also you can use the up and down arrows on your keyboard to retrieve the last
command entered at the CLI.
Finding Command Options
The syntax can consist of optional or required keywords. To display keywords for a command, enter a question mark
(?) at the command prompt or after entering part of a command followed by a space. nvOS® CLI displays a list of
available keywords along with a brief description of the keywords. For example, if you want to see all of the
keywords for the command user, enter user ?.
Table , “Getting Help” displays examples of using the question mark (?) to assist you with entering commands.
Pluribus Networks nvOS Version 2.3.2
7
Table 4: Finding Command Options
CLI network-admin@switch > ?
All commands:
acl-ip-create
acl-ip-delete
...
Displays a list of commands that begin with a specific
character string. Do not leave a space between the string
and question mark.
Switch> user auth
User: <user>
Password: <password>
Completes a partial command name.
?
Lists all commands.
command ?
Lists all keywords for the command. Leave a space
between the command and the question mark.
command option ?
Lists all arguments for the option. Leave a space between
the command and the question mark.
Informational Note: Other useful options, especially for displaying statistics, include sort, interval,
duration, and show diff interval.
Additional Information on the Command Line Interface
For some commands, the parameter delete is used, and in other commands, the parameter remove is used. This
may appear as inconsistent usage, but the explanation is quite simple.
delete is used for top level commands, such as acl-ip-delete, or vlan-delete. The following list is a
sample of top level commands:
 aaa-tacacs-delete
 dhcp-delete
 ip-pool-delete
 nat-delete
remove is used for commands with additional options, such as iso-library-image-remove where the top
level command is iso-library and the additional option image is added to the top level command. The
following list is a sample of top level commands with additional parameters that use remove:
 dhcp-host-remove
 disk-library-image-remove
 dns-interface-remove
 sflow-port-remove
8
Pluribus Networks nvOS Version 2.3.2
The same logic also applies to the usage of create and add. create is used for top level commands and add is
used with top level commands with additional options. For example, sflow-create and sflow-port-add are
two instances where this usage occurs in the CLI.
Alternate Command Format
The CLI has an alternate command format in that the commands start with a verb instead of a noun. This format
omits the hyphen in the command names. For example, connection-stats-show can also be entered as show
connection-stats. The command formats have the same features and can be used interchangeably.
Understanding Role-based Access Control
Pluribus Networks nvOS® supports flexibly defined roles so that data centers can use the same best practices for
managing discrete servers, storage, and networks to operate a Pluribus Networks fabric. You can create user roles
with privileges that reflect user responsibilities in the data center. For example, you can create the following types of
roles:
 Fabric administrator roles with control over all fabric-wide tasks
 Cluster administrator roles with control over all cluster-wide tasks
 Switch-server administrator roles with control over single switch configuration tasks
 Virtual Network (VNET) administrator roles with control over one or multiple VNET configuration tasks
 Virtual network services administrator with control over one or multiple network service(s) configuration tasks.
Specifying IP Address Netmasks
Some commands call for the specification of an IP address netmask. Pluribus Networks nvOS supports both CIDR
and subnet notations.
For example, the range of IP addresses from 192.168.0.0 to 192.168.0.255 can be specified by either entering
192.160.0.0 for the IP address input for a CLI command or either 24 or 255.255.255.0 for the netmask.
Specifying Capacity, Throughput, and Scale
Many commands include input and output of capacity and throughput. Network values are always in bits and
storage values in bytes. Scale factors are allowed on input and displayed in output as well as shown in Table 5, “Scale
Numbers”.
Table 5: Scale Numbers
Scale Indicator
Meaning (Networking)
Meaning (Storage)
K or k
Kilobits
Kilobytes
M or m
Megabits
Megabytes
G or g
Gigabits
Gigabytes
T or t
Terabits
Terabytes
Pluribus Networks nvOS Version 2.3.2
9
Customizing Show Output Formats
The output generated by the show commands can be customized by using the optional arguments described in
Table 6, “Show Output Formats”.
Table 6: Show Output Formats
format
<column_name1>,
<column_name2>,
<column_nameX>
Displays only the columns matching the list of column header names.
NOTE: The list of column names is comma-separated without spaces.
format all
Displays all available column headers. This output is also called verbose
mode.
By default, show commands output a terse set of the most commonly
useful column headers.
parsable-delim <separator> Displays the output of show command by separating columns by the
specified <separator> character(s).
For example, parsable-delim , produces a comma-separated output
(CSV).
NOTE: If the parsable-delim option is specified, the column header
names (titles) are suppressed from the output.
Using the CLI String Search
The pattern in the command output is referred to as a string. The CLI string search feature allows you to search or
filter any show or more command output and allows you to search and filter at --More-- prompts. This feature is
useful when you need to sort through large amounts of output, or if you want to exclude output that you don’t want
to see.
With the search function, you can begin unfiltered output at the first line that contains a specified regular
expression. You can then specify a maximum of one filter per command or start a new search from the --More-prompt.
You can perform three types of filtering:
Use the begin keyword to begin output with the line that contains a specified regular expression.
Use the include keyword to include output lines containing a specified regular expression.
Use the exclude keyword to exclude output lines containing a specified regular expression.
10
Pluribus Networks nvOS Version 2.3.2
You can then search this filtered output at the --More-- prompts.
Most commands optionally preceded by integer argument k. Defaults in brackets. Star
(*) indicates argument becomes the new default.
---------------------------------------------------------------<space>
Display next k lines of text [current screen size]
z
Display next k lines of text [current screen size]
<return.
Display next k lines of text [1]*
d or ctrl-D
Scroll k lines [current scroll size, initally 11]*
q or Q or ,interrupt>
Exit from more
s
Skip forward k lines of text [11]
f
Skip forward k screenfuls of text [1]
b or ctrl-b
Skip backwards k screenfuls of text [1]
‘
Go to place where previous search started
=
Display current line number
/<regular expression>
Search for kth occurrence of regular expression [1]
n
Search for kth occurrence of last r.e. [1]
h
Display this message
ctrl-l
Redraw the screen
:n
Go to kth next file [1]
:p
Go to kth previous file [1]
.
Repeat previous command
For example, to only display output that includes the IP address, 10.9.9, type the following at the --More-prompt:
/10.9.9<return>
Informational Note:
The CLI search function does not allow you to search or filter backward through previous
output.
Specifying a Switch or Fabric for Command Scope
While a switch is the building block of a fabric, the goal of the Pluribus Networks design is that a fabric of switches is
easy to manage as a single switch. Because of this, the CLI can be used to run commands on the local switch, a
cluster of switches, other switches in the fabric, or the entire fabric. You don’t have to log into each switch that you
want to run commands.
By default, commands are run on the switch you’re logged into and for example, the command
port-config-modify port 5 disable disables port 5 on the switch you’re logged into on the network.
To specify a different switch for a single command, use the switch prefix. For example, switch pleiades23
port-config-modify port 28 enable enables port 28 on pleiades23, even if the CLI is connected to a
different switch in the fabric.
To specify a different switch for a series of commands, use the switch prefix with no command. For example, type
switch pleiades24 <return>. The CLI prompt changes to indicate that pleiades24 is the switch you are
executing commands. Additional commands are run on pleiades24 rather than the switch that you’re physically
connected.
For most CLI show commands, the command displays results from all switches in the fabric by default. For example,
when the CLI command port-show is entered on the switch, it shows the ports of all switches in the fabric.
To specify that a CLI show command should apply to a specific switch, use the switch prefix to the CLI command.
For example, for the port-show command to only show the ports of the switch named pleiades24, type the
command switch pleiades24 port-show.
Pluribus Networks nvOS Version 2.3.2
11
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
12
Pluribus Networks nvOS Version 2.3.2
Introduction to nvOS Fabric
 Adding Switches to the Fabric
  Directly Connected Switches in a Fabric
 Fabric Over Management Interface
 Displaying Fabric Statistics
 Displaying Information about Nodes in the Fabric
 Using the Fabric Transaction Commands
 More Information About Undo Commands and Transactions
At Pluribus Networks, a fabric is defined as a distributed architecture based on a collection of compute clustering
techniques to present an open, standard-based Ethernet fabric as one logical switch. Every node shares the same
view of the fabric including MAC and IP addresses, connections, and application flows.
When you add switches to the fabric, all switches are under a single management domain which is highly available
through multiple link aggregation and load balancing between network resources.
The fabric performs a classic database 3-phase commit for configuration changes. All members of the fabric must
accept the configuration changes before the change is made in the fabric. Figure 1 Fabric Architecturedisplays the
fabric architecture of nvOS.
Figure 1: Fabric Architecture
Adding Switches to the Fabric
For this example, the switches are connected as in Figure 2:
Pluribus Networks nvOS Version 2.3.2
13
Figure 2: Directly Connected Switches in a Fabric
When you have more than one switch, you must add it to the fabric to take advantage of the features offered by the
fabric. To add the new switch, use the following command on one of the switches:
CLI network-admin@switch > fabric-join name pn-EBC4 fab1
You can join the fabric using either the fabric name or the switch IP address. If you use the Tab key to display the
available options, all fabrics that the switch is aware of are displayed as options.
If you specify a password for the fabric, you must type it in twice. The password is used to encrypt communication
between the nodes in the fabric. When you join the fabric from a node, you must type in the password to join it.
You can specify a specific VLAN for the fabric when you create a new one, or by default, the fabric uses VLAN1.
However, you cannot change the fabric VLAN without recreating the fabric.
Informational Note:Avoid creating fabrics with the same name.
When the fabric is created, the switch begins sending multicast messages out on Layer 2 looking for other switches.
These messages are not propagated to other networks. This is how Switch B in Figure 2 learns about the fabric.
Once Switch B joins the fabric, the fabric configuration (commands with scope fabric) is downloaded on Switch B and
the switch reboots.
If you want to connect to a switch over Layer 3, you must specify the IP address for the switch in the fabric using the
following command:
CLI network-admin@switch > fabric-join switch-ip 192.168.11.1
Fabric Over Management Interface
You can now configure fabric communication run over either the management interface or the in-band interface.
Because fabric communication over the in-band interface can be disrupted due to STP, ports going up/down, and
other factors, fabric communication over management provides a more consistent configuration.
14
Pluribus Networks nvOS Version 2.3.2
If you create a fabric with the management interface, any nodes joining the fabric inherit this setting. All nodes in a
single fabric all run on the same network type. You cannot run a mixed configuration of management and in-band
interfaces. Fabrics advertised on an incompatible network are not available for when you issue the fabric-join
command. This keeps a switch from joining an incompatible fabric.
If the fabric is configured on the management interface, all fabric-communication is on the management network,
except for the following:
Cluster synchronization-related traffic such as VLAG synchronizations and forwarded STP packets.
Cluster keep-alive packets on the fabric
Fabric keep-alive packets and global-discovery packets because both run on mgmt and in-band interfaces.
Two options, network-type and control-network are added to the command, fabric-create:
CLI network-admin@switch > fabric-create
name name-string
any of the following options:
vlan 0..4095
password
fabric-network in-band|mgmt
control-network in-band|mgmt
delete-conflicts|abort-on-conflict
If not specified, the network defaults to in-band. Note the commands, fabric-join and fabric-unjoin,
remain unchanged.
Specifying the fabric-network parameter sets the data path for fabric administration, which includes
configuration changes and show commands.
Specifying the control-network parameter sets the data path for control plane traffic, which includes status
updates, vLAG syncs, cluster syncs, and other control plane traffic.
Two new states are added to the state field of fabric-node-show:
fabric-node-show ?
[state offline|online|in-band-only-online|mgmt-only-online|
fabric-joined|eula-required|setup-required|fabric-required| fresh-install]
Because there are now two networks for nvOS to monitor for connectivity, online means both management and
in-band are reachable; in-band-only-online means the switch is only reachable through the in-band
network; mgmt-only-online means it is only reachable through the management network; and offline
means the switch is not reachable on either network.
Monitoring and reporting are reported on both the management and in-band network connectivity.
Pluribus Networks nvOS Version 2.3.2
15
Displaying Fabric Information
You can display information about the fabric using the fabric-info command:
CLI network-admin@switch > fabric-info format all layout vertical
name:
switch-ip:
id:
mcast-ip:
tid:
16
pn-EBC4
::
a0000c5:53ab701e
239.4.10.111
327
tid is the fabric
transcation ID
assigned by nvOS
Pluribus Networks nvOS Version 2.3.2
Displaying Fabric Statistics
You can also display statistical information about fabric and node activity.
CLI network-admin@switch > fabric-stats-show format all layout vertical
switch:
id:
servers:
storage:
VM:
vxlan:
tcp-syn:
tcp-est:
tcp-completed:
tcp-bytes:
udp-bytes:
arp:
vlan:
switch:
id:
servers:
storage:
VM:
vxlan:
tcp-syn:
tcp-est:
tcp-completed:
tcp-bytes:
udp-bytes:
arp:
vlan:
switch:
id:
servers:
storage:
VM:
vxlan:
tcp-syn:
tcp-est:
tcp-completed:
tcp-bytes:
udp-bytes:
arp:
vlan:
corp-sw1
0
0
0
0
0
3
1
17
3.56M
0
0
0
corp-Leaf-1
0
0
0
0
0
42.5K
7.20K
1.99M
4.63T
0
0
0
corp-Spine1
0
0
0
0
0
115K
50.2K
106M
222T
0
0
0
Pluribus Networks nvOS Version 2.3.2
17
Displaying Information about Nodes in the Fabric
You can also display information about the nodes in the fabric. It is important to take note of the fab-tid value. If
the fab-tid values do not match for each node, you can use the commands transaction-rollback-to or
transaction-rollforward-toto resynchronize the fabric.
id:
name:
fab-name:
fab-id:
cluster-id:
fab-mcast-ip:
local-mac:
mgmt-nic:
mgmt-ip:
...
in-band-ip:
...
fab-tid:
out-port:
version:
state:
firmware_upgrade:
device_state:
ports:
id:
name:
fab-name:
fab-id:
cluster-id:
fab-mcast-ip:
local-mac:
mgmt-nic:
mgmt-ip:
...
in-band-ip:
...
fab-tid:
out-port:
version:
state:
firmware_upgrade:
device_state:
ports:
id:
name:
fab-name:
fab-id:
cluster-id:
fab-mcast-ip:
local-mac:
mgmt-nic:
mgmt-ip:
18
167772619
Leaf2
fab1
a0001c8:53e2601b
0:0
239.4.10.94
64:0e:94:28:06:f2
192.168.1.14/24
192.168.254.14/24
9
0
2.1.201015836,pn-nvOS-2.0.2-2000212196
online
not-required
ok
72
201326827
Leaf1
fab1
a0001c8:53e2601b
0:0
239.4.10.94
64:0e:94:30:03:97
192.168.1.11/24
192.168.254.11/24
9
129
2.1.201015836,pn-nvOS-2.0.2-2000212196
online
not-required
ok
72
167772618
Spine2
fab1
a0001c8:53e2601b
0:0
239.4.10.94
64:0e:94:28:06:ee
192.168.1.13/24
Pluribus Networks nvOS Version 2.3.2
An example of a fabric that is out of sync for two nodes in the fabric:
CLI network-admin@switch > fabric-node-show format all layout vertical
id:
100663365
name:
CBF-switch
fab-name:
pn-CBF4
fab-id:
a0000c5:53ab701e
cluster-id:
0:0
fab-mcast-ip:
239.4.10.111
local-mac:
64:0e:94:18:01:03
mgmt-nic:
mgmt-ip:
192.168.1.61/24
...
in-band-ip:
192.168.77.61/24
...
fab-tid:
328
out-port:
128
version:
2.1.201005800,pn-nvOS-2.0.2-2000212196
state:
online
firmware_upgrade:
not-required
device_state:
ok
ports:
68
id:
201326771
name:
CBF-Leaf-1
fab-name:
corp-CBF4
fab-id:
a0000c5:53ab701e
cluster-id:
0:0
fab-mcast-ip:
239.4.10.111
local-mac:
64:0e:94:30:02:4d
mgmt-nic:
mgmt-ip:
192.168.1.53/24
...
in-band-ip:
192.168.77.53/24
...
fab-tid:
329
out-port:
128
version:
2.1.201005800,pn-nvOS-2.0.2-2000212196
state:
online
firmware_upgrade:
not-required
device_state:
ok
ports:
72
id:
167772357
name:
CBF-Spine1
fab-name:
pn-CBF4
fab-id:
a0000c5:53ab701e
cluster-id:
0:0
fab-mcast-ip:
239.4.10.111
local-mac:
64:0e:94:28:02:de
mgmt-nic:
mgmt-ip:
192.168.1.51/24
...
in-band-ip:
192.168.77.51/24
f you apply a configuration to the fabric, and a node does not respond to it, you can evict the node from the fabric,
and then troubleshoot the problem. To evict a node, use the following command:
CLI network-admin@switch > fabric-node-evict name pleiades25
Pluribus Networks nvOS Version 2.3.2
19
or
CLI network-admin@switch > fabric-node-evict id b000021:52a1b620
Using the Fabric Transaction Commands
You can roll back the fabric to a specific fabric transaction number. If a failure occurs on the fabric, transactions on
nodes in the fabric can go out of synch. Once transactions are out of synch, no further transactions can be executed
across the scope of local, fabric, or cluster. Unjoining and rejoining the fabric causes the node to lose its
configuration.
As part of a single node transaction recovery, you can roll back the transaction number to a previous one. If multiple
nodes are out of synch, you must recover each node separately.
You can also roll the fabric transaction ID forward on a node if it is out of synch with the rest of the fabric.
In the previous example, the switch, CBF-Switch2, is out of synch with the rest of the fabric. The fabric transaction ID
is 327 and the rest of the nodes have a transaction ID of 328. In this case, you can roll the node, CBF-Switch2,
forward to transaction ID 328. Enter the following command on node CBF-Switch2:
CLI network-admin@switch > transaction-forward-to scope fabric tid 328
This command produces output when an error occurs during the transaction. If there is no output, the transaction is
successful.
To display transaction information for CBF-Switch2,use the transaction-show command:
CLI network-admin@switch > transaction-show format all layout vertical
start-time:
03-19,13:46:42
end-time:
03-19,13:46:43
scope:
fabric
tid:
33
state:
remote-commit
command:
--unrecoverable-- vlan-delete id 22
undo-command: --unrecoverable-- vlan-create id 22 nvid a000030:16 scope
fabric name vlan-22 active yes stats vrg 0:0 ports 1-72,128-129,255
untagged-ports none send-ports 31,41,47-48,51,65-66 active-edge-ports
none ports-specified false flags
---------------------------------------start-time:
09:36:09
end-time:
09:36:09
scope:
fabric
tid:
34
state:
remote-commit
command:
vlan-create id 35 scope fabric stats ports-specified true
The scope parameter indicates which set of transactions to display as each scope has an independent set of
transactions associated with it. The default scope is fabric unless another scope is specified.
You cannot copy and paste commands and undo-commands because they include information that cannot apply to
new commands. These fields are informational-only and allow you to see exactly what happens to the configuration
when you roll forward or roll back the transaction ID.
Once you decide which node you want to modify and the transaction that you want to roll forward or roll back, you
use the transaction-rollforward-to or transaction-rollback-to commands to re-run the
command (roll forward) or undo the command (rollback) on the node. This applies only to the local node.
20
Pluribus Networks nvOS Version 2.3.2
More Information About Undo Commands and Transactions
You may see output similar to this output:
start-time:
end-time:
scope:
tid:
state:
command:
undo-command:
21:54:53
21:54:53
local
3
commit
port-config-modify port 9 enable
port-config-modify port 9 enable
This output is actually correct. The undo information is taken from the current state on the fabric. So if the port is
currently enabled, and you try to enable it again, you see the undo-command in the output, since the previous
state is also enabled. If you actually disable the port first, and then enable it, you see the expected undo information
in the transaction log.
start-time:
10:05:22
end-time:
10:05:22
scope:
local
tid:
20
state:
commit
command:
port-config-modify port 12
undo-command: port-config-modify port 12
---------------------------------------start-time:
10:05:48
end-time:
10:05:48
scope:
local
tid:
21
state:
commit
command:
port-config-modify port 12
undo-command: port-config-modify port 12
disable
enable
enable
disable
So undo is not necessarily the opposite of the current command, but allows you to go back to the state before the
command was issued. This may be the exact same state as before.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks nvOS Version 2.3.2
21
Displaying Fabric Statistics
To display fabric statistics, use the following command:
CLI network-admin@switch > fabric-stats-show
switch:
id:
servers:
storage:
VM:
vxlan:
tcp-syn:
tcp-est:
tcp-completed:
tcp-bytes:
udp-bytes:
arp:
vlan:
switch:
id:
servers:
storage:
VM:
vxlan:
tcp-syn:
tcp-est:
tcp-completed:
tcp-bytes:
udp-bytes:
arp:
vlan:
switch:
id:
servers:
storage:
VM:
vxlan:
tcp-syn:
tcp-est:
tcp-completed:
tcp-bytes:
udp-bytes:
arp:
vlan:
pleiades23
0
0
0
0
0
229K
171
7.19K
3.53G
0
0
0
pleiades24
0
0
0
0
0
85.6K
125
11.6K
3.95G
0
0
0
pleiades25
0
0
0
0
0
179K
20.9K
1.60M
485G
0
0
0
Troubleshooting the Fabric
There may be instances when you need to troubleshoot the fabric. The following is a list of helpful port numbers,
multicast information, and communication on the fabric.
 Internal Keepalive
Multicast IP: 239.4.9.7
UDP Destination Port: 23399
22
Pluribus Networks nvOS Version 2.3.2
This packet is sent from the CPU to the internal port to ensure that the CPU path to the switch is working and
the internal port is up.
 Fabric Keepalive
UDP Destination Port: 23394
Point to point UDP fabric keepalive
If these messages don't get through, the fabric node may go to offline state.
 Global Discovery
Multicast IP: 239.4.9.3
UDP destination port: 23399
Each node periodically multicasts a message about the fabric. This enables fabric-show on L2-connected
nodes to show available packets and also enables fabric-join name name. It also enables you to join a
fabric over Layer 3 connectivity by specify an IP address.
 Proxy commands
TCP Destination Port: 23397 SSL
Used for nvOSd-to-nvOSd commands. Used for internal purposes and also to implement commands executed
on other switches from a local switch.
 Status propagation
TCP Destination Port: 23398 SSL
Port changes and vport changes propagated to other nodes in the fabric.
 TCP API clients
TCP Destination Port: 23396 SSL
C API clients connect to this port. Can be disabled using admin-service-modify if <mgmt/data>
no-net-api command.
 File System replication
TCP Destination Port: 23392 SSL
For ZFS send and ZFS receive messages when replicating file systems across the fabric.
 L2 ARP/DMAC miss/Broadcast encapsulation
UDP Destination Port: 23389
These are VXLAN-encapsulated packets sent from CPU to CPU between two L2 connected switches.
 L3 ARP/DMAC miss/Broadcast encapsulation
UDP Destination Port: 23388
These are VXLAN-encapsulated packets sent from CPU to CPU between two L3 connected switches.
 vPORT status
Multicast IP: 239.4.9.4
UDP Destination Port: 23390
vPort updates from hypervisors or hosts in the fabric.
 vFlow CPU packets
UDP Destination Port: 23398
These packets are sent point-to-point for vflow-snoop of a fabric-scoped vFlow.
Pluribus Networks nvOS Version 2.3.2
23
All of these messages need to be able to get through in order to keep an L2 fabric healthy. The multicast messages
don't propagate through routers so they aren't used for L3 fabrics.
fabric-node-show displays information about nvOS internal data structures for each node in the fabric. If no
keepalive or other messages are received from a fabric node for about 20 seconds, the node is marked as offline.
Anything that prevents keepalive or other kinds of messages from flowing freely between fabric nodes can cause
problems for fabric connectivity.
If the fabric transaction IDs become unsynchronized, use the transaction commands to either roll forward or back
the transaction IDs. See Using the Fabric Transaction Commands.
Configuring Transaction Settings
Transactions are allowed to proceed if at least one node in the cluster is reachable. If a cluster node is offline when a
configuration change is requested the transaction proceeds even though one of the cluster members is offline.
Nodes that were ignored for transactions automatically try to recover the transactions. Auto-recovery is enabled by
default but may be disabled. You can also configure the length of time between retry attempts between the nodes.
This feature is enabled by default, but may be disabled.
The following is a sample CLI output with one cluster node offline:
CLI (network-admin@switch1) > vlan-create id 24 scope fabric
Warning: cluster node switch2 not reachable, continuing anyway
The following is a sample of CLI output with both cluster nodes offline:
CLI (network-admin@switch2) > vlan-create id 33 scope fabric
Warning: cluster node switch1 not reachable, continuing anyway
vlan-create: fabric error: switch1 unreachable, both cluster nodes offline
To configure transaction settings, use the transaction-settings-modify command and configure the
following options:
 allow-offline-cluster-nodes — select this option to allow transactions to proceed on cluster
configurations even if the cluster is offline.
 auto-recover
— select this option to automatically recover missed transactions.
 auto-recover-retry-time — specify the duration of the retry time in days, hours, minutes, or seconds.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
24
Pluribus Networks nvOS Version 2.3.2
Configuring Basic Server-Switch Functionality
 Using the Serial Console Port for Initial Configuration
 Aggregation for Management Network Interface Card (NIC)
 Creating an Initial Fabric
 Changing Other Switch Setup Parameters
 Confirming Connectivity on the Network
 Updating nvOS on the Server-Switch
 Implementing a Fabric Upgrade or a “Rolling” Fabric Upgrade
 Saving and Restoring Server-Switch Configurations
 Copying and Importing Configuration Files
 Configuring Virtual Network Interface Cards (vNICs)
 Displaying Physical Port Details
 Displaying Layer 2 Networking Details
 Rebooting, Powering Off, and Resetting the Server-Switch
 Topic Feedback
Using the Serial Console Port for Initial Configuration
This procedure assumes that you have installed the server-switch in the desired location and it is powered on.
CAUTION! Do not connect any ports to the network until the server-switch is configured. You can accidentally
create loops or cause IP address conflicts on the network.
If you are going to cable host computers to the switch, there is an option to enable or disable host ports by default.
1. Connect the console port on the rear or front (depending on the model) of the server-switch to your laptop or terminal concentrator using a serial cable.
2. From the terminal emulator application on your computer, log into the switch with the username network-admin
and the default password admin.
3. . You can begin initial configuration using the setup questions displayed:
switch console login: network-admin
Password: admin
Last login: Fri Oct 3 12:23:04 on console
Pluribus Command Line Interface v1.2.2
System setup required:
System Name (switch): pleaides01 <return>
network-admin Password: password <return>
Re-enter Password:****** <return>
Enable mgmt link aggregation (no): yes
This might reset SSH connections after the
setup.Are you Sure? (no): yes
LACP mode of the mgmt LAG interface[active|passive|off]
(passive): invalid
Please answer "active", "passive", or "off"
LACP mode of the mgmt LAG interface[active|passive|off]
(passive): active
Pluribus Networks nvOS Version 2.3.2
25
Mgmt IP/Netmask (10.9.19.107
Mgmt IP/Netmask: ip-address/netmask <return>
In-band IP/Netmask: ip-address/netmask
Gateway IP (0.0.0.0): 192.168.100.254 <return> or ip-address
Primary DNS IP (0.0.0.0): 192.168.100.253 <return> or ip-address
Secondary DNS IP (0.0.0.0): 192.168.200.253 <return> or ip-address
Domain name (pluribusnetworks.com): domain-name <return>
Automatically Upload Diagnostics (yes): <return>
Enable host ports by default (yes): no
nvOS system info:
serial number: 1245LC8500018
hostid: a000044
user auth cookie val = 152895552
Switch Setup:
Switch Name:
pleaides01
Switch Mgmt IP:
192.168.100.1/24
Switch In-band IP:
192.168.200.1/24
Switch Gateway:
192.168.100.254
Switch DNS Server:
192.168.100.254
Switch DNS2 Server:
192.168.100.253
Switch Domain Name:
pluribusnetworks.com
Switch NTP Server:
0.us.pool.ntp.org
Switch Timezone:
US/Pacific
Switch Date:
2013-10-03, 13:02:39
Upload Crash Reports:
yes
Fabric required. Please use fabric-create/join/show
Connected to Switch pluribus; nvOS Identifier:0x000044; Ver: 0.19.3398
Aggregation for Management Network Interface Card (NIC)
Out of band management interfaces areaggregated to provide high availability (HA) and failover capabilities in nvOS
in the presence of two management NICs. You can configure nvOS to pool two management NICs into a single logical
management interface to increase bandwidth of the management link and add redundancy to the out of band
connection. By default, management link aggregation is disabled. When you configure link aggregation, a new
interface is created on the platform and a trunk link is also created. Physical management interfaces, MGMT0 and
MGMT1, are added to it. The IPv4 and IPv6 addresses are copied from MGMT0 if configured.
LACP is disabled by default, but can be enabled using the switch-setup-modify mgmt-lacp-mode command. The
default aggregation mode is active-active, and after configuring the link aggregation interface, nvOS waits for a short
interval to ensure that the interface is receiving packets. If no packets are seen on the second physical interface
configuration reverts back to the single management interface, and the appropriate error message is generated.
You are now ready to begin the rest of the configuration on the switch.
Informational Note: In order to use the “phone home” feature, you must open ports 8084 and 8443
on your firewall.
Changing the Default Timezone
The default timezone is US/Pacific Standard Time (PST). To change the timezone, use the
switch-setup-modify command:
CLI network-admin@switch > switch-setup-modify timezone timezone
26
Pluribus Networks nvOS Version 2.3.2
Changing Other Switch Setup Parameters
You can also modify other switch parameters including the following:
 Switch name
 Management IPv4 and IPv6 addresses
 Management IPv4 and IPv6 netmasks
 Management IPv4 and IPv6 address assignments
 In-band IP address
 In-band netmask
 Gateway IPv4 address
 Gateway IPv6 address
 Primary and secondary IPv4 addresses for DNS services
 Domain name
 NTP server
 End User License Agreement (EULA) acceptance and timestamp
 Password
 Date
 Phone home for software updates
 Analytics store (storage type)
 Message of the Day (MOTD)
 Banner
CLI network-admin@switch > switch-setup-modify mgmt-ip6 2001::2/64 gateway-ip
10.10.10.1 gateway-ip6 2001::35 dns-ip 10.10.10.11 dns-secondary-ip 10.10.10.1
domain-name corpinfo.com ntp-server 0.us.pool.ntp.org timezone US/Pacific
<return>
To display the configured settings, use the switch-setup-show command:
CLI network-admin@switch > switch-setup-show
name:
mgmt-ip:
mgmt-ip6:
in-band-ip:
gateway-ip:
gateway-ip6:
dns-ip:
dns-secondary-ip:
domain-name:
ntp-server:
timezone:
date:
phone-home:
analytics-store:
pleiades01
10.10.10.79/16
2001::2/64
192.168.21.1/24
10.10.10.1
2001::35
10.10.9.1
10.10.10.1
corpinfo.com
0.us.pool.ntp.org
US/Pacific
2013-10-31, 16:00:00
yes
optimized
Pluribus Networks nvOS Version 2.3.2
27
The analytics-store parameter refers to the storage location of nvOS analytics. The parameter, optimized,
indicates that a Fusion IO card is installed on the switch. You can now store statistics for connections, hosts, client
servers, and CPU package logs on the Fusion IO card. When you specify optimized, the statistics are stored on the
IO card with the highest amount of free space. If you select default, the statistics are stored on the nvOShard
drive.
Informational Note: Fusion IO cards are only available as an additional upgrade or when you purchase the
F68-F1LT model.
You can also configure a “Message of the Day” for users to see when logging into the switch. You may enter up to
511 characters including spaces. If you use spaces, enclose the MOTD in quotes. The MOTD can be used as a
temporary or short term message to display downtime or other activity. To add the message, “switch down 2-4pm
3/31/15” use the following syntax:
CLI network-admin@switch > switch-setup-modify motd “switch down 2-4pm 3/31/15”
When you log into the switch, the MOTD is displayed after the software version:
admin@pubdev03:~$ cli
Netvisor OS Command Line Interface 2.2
Please enter username and password:
Username (network-admin):
Password:
Connected to Switch pubdev03; nvOS Identifier:0xa0000e3; Ver: 2.2.202036795
pubdev03 down 2-4pm 3/31/15
You can also configure static banners to display switch information such as server identity.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Confirming Connectivity on the Network
After you’ve connected your server-switch, you may want to take the time to ensure that you have connectivity by
pinging an external IP address, and pinging a domain to ensure that you can resolve a domain name.
To ping the external network from the server-switch, use the ping command:
CLI network-admin@switch > ping 98.138.253.109
98.138.253.109 is alive.
To ping a domain, use the ping command again:
CLI network-admin@switch > ping yahoo.com
yahoo.com is alive.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
28
Pluribus Networks nvOS Version 2.3.2
Creating an Initial Fabric
After you complete the initial setup on the switch, you must create a new fabric for the switch or join an existing
fabric. When switches form a fabric, the fabric becomes one logical switch, and shares state information as well as
communicates commands so that any scope of a fabric- command is executed on each switch in the fabric. A
switch must be in a fabric in order to keep track of the fabric state. However, a switch can be a member of fabric, and
consist of a single switch. A switch leaving one fabric and joining another loses the fabric state of the first fabric and
learns the fabric state of the second fabric.
1. To create a new fabric over Layer 2, use the following command:
CLI network-admin@switch > fabric-create name name-string
2. Create a name for the new fabric.
To require a password before joining the fabric, use the password option. Press the return key after typing the
password parameter:
CLI network-admin@switch > fabric-create name name-string <return>
password:*******
Re-enter password:*******
By default, the fabric is created on VLAN1. You can specify a different VLAN, but if you change the VLAN, you must
recreate the fabric.
To create a fabric over Layer 3, use the fabric-join command and the switch IP address. For example,
CLI network-admin@switch > fabric-join switch-ip 192.168.2.2 vlan 20
3. To show fabric details, use the fabric-show command:
CLI network-admin@switch > fabric-show
name
---------------info-dev
ursa-lyon
id
---------------a000030:5537b46c
6000210:566621ee
vlan
---3
0
fabric-network
-------------in-band
mgmt
control-network
--------------in-band
in-band
tid
---365
4928
You can also specify to send network traffic over the fabric network or the control plane network. To specify the
fabric network, use the fabric-network parameter, specify the in-band or management IP address.
Specifying the fabric-network parameter sends traffic over the data path for fabric administration, which includes
configuration changes and show commands.
To specify the control plane network, use the control-network parameter, and specify the in-band or
management IP address.
Using the control-network parameter specifies the data path for control plane traffic, which includes status updates,
vlag syncs, cluster syncs, and similar traffic.
Adding License Keys to nvOS
The license key for nvOS is bound to the serial number of the Pluribus Network switch and ships with the switch.
To install the license key, use the following syntax:
CLI network-admin@switch > software-license-install key license-key
Pluribus Networks nvOS Version 2.3.2
29
The license key has the format of four words separated by commas. For example.
License Key:
rental,deer,sonic,solace
Once the license key is installed, you can display information about the key using the following command:
CLI network-admin@switch > software-license-show format all layout vertical
switch:
Pleaides01
license-id:
F-ASDF-NVOS2.0
description: Freedom F-Line Advanced Software Defined Fabric License for
Netvisor 2.x
key:
rental,deer,sonic,solace
feature:
all
upgrade-from:
To display the status of the server-switch, use the switch-status-show command:
CLI (switch)>switch-status-show
switch
name
value units
state
-------- --------------- ----- --------- ----pluribus Switch Temp
41
degrees-C ok
pluribus CPU1 Temp
57
degrees-C ok
pluribus CPU2 Temp
49
degrees-C ok
pluribus System Temp
46
degrees-C ok
pluribus Peripheral Temp 30
degrees-C ok
pluribus PCH Temp
43
degrees-C ok
pluribus VTT
volts
ok
pluribus CPU1 Vcore
volts
ok
pluribus CPU2 Vcore
volts
ok
pluribus VDIMM AB
volts
ok
pluribus VDIMM CD
volts
ok
pluribus VDIMM EF
volts
ok
pluribus VDIMM GH
volts
ok
pluribus +1.1 V
volts
ok
pluribus +1.5 V
volts
ok
pluribus 3.3V
volts
ok
pluribus +3.3VSB
volts
ok
pluribus 5V
volts
ok
pluribus +5VSB
volts
ok
pluribus 12V
volts
ok
pluribus VBAT
volts
ok
pluribus switch-3.3v
volts
ok
pluribus switch-1.1v
volts
ok
pluribus switch-vcore
volts
ok
pluribus switch-5.0v
volts
ok
pluribus switch-2.5v
volts
ok
pluribus switch-0.95v
volts
ok
pluribus switch-1.8v
volts
ok
pluribus switch-1.2v
volts
ok
pluribus fan-1
3525 rpm
ok
pluribus fan-2
3760 rpm
ok
pluribus fan-3
3525 rpm
ok
pluribus fan-4
3760 rpm
ok
This command displays the physical status of the switch including fan speed, electrical voltage, temperature.
30
Pluribus Networks nvOS Version 2.3.2
To display additional physical information about the switch, use the switch-info-show command:
CLI (switch)>switch-info-show
switch:
model:
chassis-serial:
cpu1-type:
cpu2-type:
system-mem:
switch-device:
switch-version:
polaris-device:
gandalf-version:
fan1-status:
fan2-status:
fan3-status:
fan4-status:
ps1-status:
ps2-status:
pluribus
F64-HWENT
1243PN8500014
Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
64.0G
ok
b2
ok
caff0044
ok
ok
ok
ok
ok
n/a
To display information about a specific switch, specify the name of the switch in the command:
CLI network-admin@switch > switch-info-show name name-string
If you don’t specify the name of the switch, all switches in the fabric are displayed.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Modifying and Upgrading Software
A switch can contact an upgrade server, either directly or through a proxy, to download and upgrade to a newer
version of nvOS. You can modify the upgrade process for the switch and add a proxy host.
Informational Note: This upgrade procedure applies to only one switch. To upgrade switches on the
fabric or to create a “rolling upgrade” on the fabric, see
What are Software Tracks?
Software tracks are a method for Pluribus Networks to manage different software releases available to customers.
The software track, release, is the default standard track, but other tracks, such as Beta, may be available for
download.
CLI network-admin@switch > software-modify phone-home
Pluribus Networks nvOS Version 2.3.2
31
Updating nvOS on the Server-Switch
Pluribus Networks switches can send “phone home” messages to the Pluribus Networks update servers to
determine if a new release of software is available for download.
1. To view the current version of nvOS on the switch, use the following command:
CLI network-admin@switch > software-show
version:
track:
upgrade-status:
version-available:
auto-upgrade:
use-proxy:
2.2.1-202016524
2.2-release
available
2.2.0-202006524 -> 2.2.1-202016554
disable
no
2. If the upgrade status indicates that a newer version of nvOS is available, request an update from the server:
CLI network-admin@switch > software-upgrade
upgrade successful. rebooting...
To check the status while the switch is upgrading, use the software-upgrade-status-show command.
3. To check the status of the switch after upgrading, reconnect to the switch, and enter the following command:
CLI network-admin@switch > software-show
version:
track:
upgrade-status:
auto-upgrade:
use-proxy:
2.2.1-202016554
2.2-release
up-to-date
disable
no
Informational Note: Allow plenty of time for the switch to download and install the new version
of software. Do not interrupt the operation while the upgrade is in progress. When the
upgrade is complete, the switch reboots and loads the latest version of the software.
If you encounter any problems with the new version of the software, a previous version can be
selected as the boot software. See “Topic Feedback” on page 1–33
Informational Note: Upgrading without an Internet connection - If the switch does not have direct
access to the Internet but can use a proxy server, enter the software-modify
use-proxy command to configure the proxy and then check for software upgrade
availability. If there is no access to the Internet from the switch, contact Pluribus Technical
Support for instructions on upgrading a switch offline.
To upgrade the current nvOS to a later release, use the software-upgrade command.
CLI network-admin@switch > software-upgrade package nvos-2.3.1-203018600.tgz
The parameter package allows you to specify the name of the upgrade file.
32
Pluribus Networks nvOS Version 2.3.2
To display information about the software upgrade path, you can use the software-track-show command.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Implementing a Fabric Upgrade or a “Rolling” Fabric Upgrade
You can now implement a fabric-wide upgrade and reboot the switches at the same time or in a sequential order. A
fabric upgrade requires downloading the new nvOS software package to each switch, and rolling upgrade downloads
the software packages from the update server and then copies the software to each switch as the upgrade proceeds.
The upgrade controller is the switch where the fabric-upgrade-start command is issued. All upgrade
commands should be executed from the upgrade controller.
The fabric upgrade feature has two phases:
 Upgrade — start the upgrade which creates and updates nvOS to new boot environments but does not reboot
the fabric.
 Reboot — reboots the entire fabric after all server-switches are upgraded to new boot environments. It is also
possible during this phase to abort the process and discard the new boot environments.
The fabric is locked during the entire process and you cannot change any configurations during the process.
Before You Begin the Fabric Upgrade
Before you begin, you may want to consider the following options for the fabric-upgrade-start command:
 auto-finish — you can specify to automatically reboot the entire fabric after the upgrade is complete.
 rolling — specify if you want to perform a rolling fabric upgrade. A rolling fabric upgrade performs the
upgrade procedure on a switch-by-switch basis and copies the software package from the controller to other
switches in the fabric. If you specify no-rolling, all switches are booted after the upgrade.
 abort-on-failure — specify if you want the upgrade to stop if there is a failure during the process.
 manual-reboot — specify if you want to manually reboot individual switches after the upgrade process. If
you specify no-manual-reboot, all switches reboot automatically after the upgrade is complete.
 prepare — specify if you want to perform setup steps prior to performing the upgrade. This step copies the
offline software package and then extracts and prepares it for the final upgrade process. Once you begin the
prepare process, you cannot add new switches to the fabric.
— specify to reboot switches in parallel if the switches are in a cluster configuration. Or,
you can reboot them one at time using the reboot-single option.
 reboot-parallel
 reboot-group — specify the number of switches to reboot as a group in parallel mode. The default is the
maximum number of switches in the fabric up to 100 switches.
Starting the Fabric Upgrade
1. Download the latest nvOS software from the update server onto a switch in the fabric.
2. Copy the nvOS software package to each switch in the fabric.
3. Select a switch in the fabric to act as the upgrade controller switch, and use the fabric-upgrade-start
command to begin the upgrade.
4. Depending on the options selected, the upgrade completes by reboot the fabric or rebooting all of the switches.
Pluribus Networks nvOS Version 2.3.2
33
Starting the Rolling Fabric Upgrade
If you opted for a rolling fabric upgrade, then the upgrade controller switch begins copying to software packages to
other switches in the fabric. Other than this step, the rolling fabric upgrade has the same behavior as a fabric
upgrade depending on the selected options.
You can check the status of the upgrade using the fabric-upgrade-status-show command:
CLI (network-admin@sw1) > fabric-upgrade-status-show
log
switch
state
----------------------------------------------- -------- -----------------(0:00:36)Upgrading software upgrade framework
sw3
Running
(0:00:08)Computing package update requirements. sw2
Running
(0:00:12)Agent needs restart
sw1* Agent restart wait
The first entry in the log is the duration of the upgrade process. It does not include waiting time. The switch with the
asterisk (*) is the controller server-switch where the fabric-upgrade-start command was issued.
Additional commands for the fabric upgrade feature:
— you can issue this command at any time during the fabric upgrade to reboot
all nodes in the fabric and complete the upgrade. Once the upgrade phase is complete, all server-switches
display the “Upgrade complete” message in the log field. You can then safely reboot the fabric.
 fabric-upgrade-finish
— aborts the software upgrade process. All changes to the server-switches are
cleaned up and the server-switches do not reboot. The configuration lock on the fabric is also released.
 fabric-upgrade-abort
If you issue the fabric-upgrade-abort command during the upgrade process, it may take some time before the
process stops because the upgrade has to reach a logical completion point before the changes are rolled back
on the fabric. This allows the proper cleanup of the changes.
 fabric-upgrade-prepare-cancel — cancels a fabric upgrade that was prepared earlier.
 fabric-upgrade-prepare-resume — resume a fabric upgrade that was prepared earlier.
 fabric-upgrade-prepare-show — displays the status of prepared upgrades on the fabric nodes.
Enabling Administrative Services
There are many features of the Pluribus Networks fabric that require or can be enhanced using remote access. For
example, when packets are written to a log file, you may want to transfer that file from a switch to a different system
for analysis. Also, if you are creating a NetVM environment, an IOS image of the guest OS must be loaded on the
switch.
There are two file transfer methods:
 Secure File Transfer Protocol (SFTP)
 Network File System (NFS)
Both methods must be enabled before you can use them. Because SFTP relies on Secure Shell (SSH), you must
enable SSH before enabling SFTP.
1. To check the status of SFTP, use the following command:
CLI network-admin@switch > admin-service-show
switch
-------
nic
---
ssh
---
nfs
---
web
---
web-port
--------
snmp
----
net-api
-------
icmp
----
pleiades24
mgmt
off
on
off
80
off
off
off
34
Pluribus Networks nvOS Version 2.3.2
2. To enable SSH, use the following command:
CLI network-admin@switch > admin-service-modify nic mgmt ssh
admin-sftp-modify enable
sftp password: <password>
confirm sftp password: <password>
The default SFTP username is sftp and the password can be change using the admin-sftp-modify
command:
CLI network-admin@switch > admin-sftp-modify
sftp password: <password>
confirm sftp password: <password>
CLI network-admin@switch > admin-service-show
switch
-----pleiades24
nic
--mgmt
ssh
--on
nfs
--on
web
--off
web-port
-------80
snmp
---off
net-api
------off
icmp
---off
CLI network-admin@switch > admin-sftp-show
switch:
sftp-user:
enable:
pleiades24
sftp
yes
Use SFTP from a host to the switch, and login with the username sftp and the password that you configured for SFTP.
Then you can download the available files or upload files to the switch.
3. You can check the status of NFS service and enable it using the following command:
CLI network-admin@switch > admin-service-show
switch
nic
ssh
nfs
web
web-port
snmp
net-api
icmp
------
---
---
---
---
--------
----
-------
----
on
off
on
80
off
on
on
pleiades01 mgmt
To enable NFS, use the following command:
CLI network-admin@switch > admin-service-modify nic mgmt nfs
After you enable NFS, the directory /nvOS is mountable using NFS through the management IP addresses for access
to the files in that directory.
Saving and Restoring Server-Switch Configurations
A switch contains local configuration information such as port settings as well as fabric configuration information.
Fabric configurations are stored on every switch in the fabric and does not require that you save and restore before
replacing a switch. When a switch is replaced, removed, or otherwise disrupted, you can save and restore the local
configuration information.
The information that is saved and restored on the local switch includes the following:
 VNETs with VNET manager running on the switch
Pluribus Networks nvOS Version 2.3.2
35
 Port VLAN associations
 Netvisor Zone configuration details, but not any modifications to NetZones such as installed applications
 Netvisor VMM configuration details, but not ISO images or disk images
 Netvisor KVM configuration details, but not ISO images or disk images
 Network services running on the switch
To display a full list of the current configuration details for a switch, use the running-config-show command.
SFTP and NFS can be used to transfer the configuration file, but you must enable the two features before using
them.
Caution! There is a potential for data loss when restoring a configuration. The configuration on the
switch is replaced by the configuration stored in the import file. Although ISO images and disk-library
images are not likely to disappear, you should only perform switch-config-import on a
switch that doesn’t have important data stored on it.
As a precaution, consider using the command switch-config-export to save the data on
the switch that you are importing the configuration file.
Also, copy the ISO images and disk images from the switch using the iso-image-library and
disk-library-image-export commands and copying the files from the switch.
1. To save the switch configuration to a file, use the following command:
CLI network-admin@switch > switch-config-export export-file pleiades24
Exported configuration to /nvOS/export/pleiades24.2013-11-04T22.33.31.tar.gz
2. To display the files available for import and export, use the following command:
CLI network-admin@switch > switch-config-show
switch
pleiades24
export-file
pleiades24.2013-11-04T22.33.31.tar.gz
You can now copy the configuration file to a different host using SFTP or NFS. For example, you can SFTP to the
switch-ip-address, and login using the SFTP password. Then use cd/nvOS/import, and use get to
download the configuration file.
The switch-config-export command is used to export the configuration of the local switch. The file that
is created is a tar file that includes a number of configuration files for the switch. The file is created under
/nvOS/export. This is the command used to export the current configuration on the local switch. vAlso, each
time you reset the switch using the command, switch-config-reset, a backup of the configuration is made
and places a file in the same location.
Once the switch configuration is exported, it becomes available to import on the same switch, by using the
switch-config-copy-to-import command. nvOS copies the configuration tar file from the
/nvOS/export to the /nvOS/import directory. Once in the /nvOS/import directory, it is possible to use
the switch-config-import command to import the switch configuration.
The switch-config-import command is used to import a configuration on the local switch. When using
that command, the intention is to import a switch configuration t previously exported by the same switch.
The switch-config-import command has a few parameters to it. The ignore-system-config and
the apply-system-config parameters are 2 parameters that allow the imported configuration of the switch
to override or not override the currently configured information found under the switch-setup-show
command. When you select the ignore-system-config parameter, the local configuration is saved to an archive. If
you select apply-system-config, the settings in the tar file are applied to the local switch.
36
Pluribus Networks nvOS Version 2.3.2
When you import a configuration using the switch-config-import command, the current configuration on
the switch is overwritten by the imported configuration file.
The skip-fabric-join option imports the fabric configuration from the tar file. However, this information
may be out of date with respect to the fabric if transactions have occurred on the fabric since the file was
exported which causes the imported configuration to be out-of-sync with the current fabric. The alternative is to
specify do-fabric-join, which extracts the fabric name from the tar file, and attempts to join the fabric and
download the current fabric configuration, so that it is in sync with the rest of the fabric. The fabric configuration
in the tar file is ignored, but cluster and local configurations are imported from the tar file.
When a switch that was part of a cluster is replaced, the fabric-join repeer-to-cluster-node
command is used for the new switch to receive all required switch configuration, including the local configuration.
To upload a configuration file to a switch and set the configuration for the switch using the configuration file, you
must transfer the configuration file to the target switch using the following sequence of commands:
sftp sftp@<switch-ip-address>
Connecting to switch-ip-address
Password: <password>
sftp> cd nvOS/import
sftp> put pleiades24.2013-11-04T22.33.31.tar.gz
Informational Note: The configuration file must use the *.tar.gz extension to be recognized by
nvOS.
CAUTION! Loading the configuration file causes nvOS to restart which results in a brief interruption to
switch traffic flow.
Now load the configuration file which replaces the current configuration on the switch with the information in the
file.
CLI network-admin@switch > switch-config-import import-file
pleiades24.2013-11-04T22.33.31.tar.gz
New configuration imported. Restarting nvOS...
Connected to Switch pleiades24; nvOS Identifier:0xb000011; Ver: 0.19.3747
There are many options available that allow you to control how the switch-config-import modifies the
switch, including the following:
 ignore-system-config
- ignore the current system configuration. The settings in the *.tar file are not
applied to the local switch.
— apply the system configuration in the imported file. The settings in the *.tar file
are applied to the local switch. You typically do not want to use this option as it changes the in-band IP address
and other settings.
 apply-system-config
— opt out of joining the fabric. This setting imports the fabric configuration from the
*.tar file, but this information may be out of date with respect to the fabric if additional transactions occur on
the fabric since the file was exported.
 skip-fabric-join
Pluribus Networks nvOS Version 2.3.2
37
— join the current fabric. This setting extracts the fabric name from the *.tar file and
attempts to join the fabric. Then the switch contacts the current fabric to download the configuration so that
the switch is in sync with the rest of the fabric. Cluster and local configurations are imported from the *.tar file.
 do-fabric-join
 no-replace-switch
— do not replace the current switch.
— replace the current switch. This setting is used to replace a faulty switch and after
importing the file, has the same configuration as the replaced switch. This replaces all of the local, cluster, and
fabric configuration by downloading the configurations from peer switches. No configuration is necessary or
advised before running this command. However, you need to run the initial quickstart to obtain an in-band IP
address.
 replace-switch
By default, the initial switch system configuration, management IP addresses and other parameters, are not applied
if there is another switch in the fabric with the same settings. To apply the initial settings, use the
apply-system-config option. Also, by default, the imported configuration attempts to join the same fabric
that the original switch was a member. If that join fails, then the import fails. You can avoid this issue by using the
skip-fabric-join option. Finally, if the original switch is still on the network and you want to copy the
configuration to a new switch, but you want to prevent the new switch from taking ownership of any objects specific
to the original switch, such as VNET services, or VLAN port settings, you must use the no-replace-switch
option.
Copying and Importing Configuration Files
You can create a configuration file to import to another switch by using the
switch-config-copy-to-import command. To create a configuration file with the name config-092613 to
import on another switch, use the following syntax:
CLI network-admin@switch > switch-config-copy-to-import export-file
config-092613
After you create the configuration file, you can export it to /nvOS/export/ directory, and SFTP to it from the
target switch.
To review the available files for import and export, use the following syntax:
CLI network-admin@switch > switch-config-show
switch
pbg-nvos
export-file
config-092613.tar.gz
Depending on the available remote access services, you can now copy the configuration file to a different switch. For
example, you can SFTP to another switch using the IP address of the switch, login as SFTP with the password that
you previously set, cd /nvOS/import and get the configuration file.
To upload the configuration file to the target switch and set the configuration from the configuration file, transfer
the configuration file to the target switch with the IP address, 192.168.3.35.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Changing the IP Port for vManage
vManage is a Web-based service and it listens on an IP port to accept communications. By default, vManage listens
on port 80 on the management IP address that you set during the initial configuration, and can be reached using a
supported Web browser such as Safari, Firefox, or Chrome using the URL http://mgmt-ip. In some cases, you may
want to configure vManage to listen on a different port as in the case of a virtual load balancer sending traffic
arriving on port 80 of the management IP address to other systems. In this case, vManage cannot listen on port 80.
38
Pluribus Networks nvOS Version 2.3.2
Use the admin-service command to change the listening port. Changing the port disrupts any current
connections to vManage.
1. To change the listening port to 8080 for vManage, use the following syntax:
CLI network-admin@switch > admin-service-modify nic mgmt web-port 8080
2. To check the status of admin services, use the following command:
CLI network-admin@switch > admin-service-show
switch
nic
ssh
pleiades24 mgmt on
sftp
on
nfs
on
web
on
web-port
8080
snmp
off
net-api
on
icmp
on
After this change, you use the URL http://mgmt-ip:8080.
Configuring Virtual Network Interface Cards (vNICs)
You can create vNICs on the switch to provide connectivity for some virtual services. You can use the vNICs for data
or management purposes. To create a vNIC with the IP address 172.16.21.33/24 on VLAN 301 for data traffic and the
type of vNIC is e1000, use the following command:
CLI network-admin@switch > switch-vnic-create ip 172.16.21.33/24 assignment
none vlan 301 if data vm-nic-type e1000
To modify the configuration, use the switch-vnic-modify command, and to delete the vNIC, use the
switch-vnic-delete command. To display information about vNICs, use the switch-vnic-show
command.
CLI network-admin@switch > switch-vnic-show layout vertical
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
to_vnic_flow_name:
global.eth0
10.12.111.103/24
static
66:0e:94:21:c8:a2
10
0
data
Informational Note: There are three types of interfaces in nvOS:
• Physical
• vNIC applies to virtual interfaces created for the server-switch.
• VNET interfaces created for virtual services.
Pluribus Networks nvOS Version 2.3.2
39
Displaying System Statistics on a Server-Switch
You display system statistics on a server-switch using the system-stats-show command:
CLI network-admin@switch > system-stats-show layout vertical
switch:
uptime:
used-mem:
used-swap:
swap-scan:
cpu-user:
cpu-sys:
cpu-idle:
pleiades24
1h22m26s
27%
0%
0
0%
1%
98%
The swap-scan output displays the number of scans performed on the swap. A nonzero number indicates that
memory is paged from the physical memory (RAM) to virtual memory (disk or swap). A consistently high value
indicates that all memory, both physical and virtual, is exhausted and the system may stop responding.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
40
Pluribus Networks nvOS Version 2.3.2
Displaying Layer 2 Networking Details
To display fabric-wide Layer 2 (L2) networking table, use the l2-table-show command. This table displays the
MAC addresses associated with IP addresses and the ports that the MAC addresses appeared.
CLI network-admin@switch > l2-table-show
switch:
mac:
ip:
vlan:
vxlan:
state:
create_time:
last-seen:
hit:
migrate:
drops:
switch:
mac:
ip:
vlan:
vxlan:
last_time
hit:
migrate:
drops:
switch:
mac:
ip:
vlan:
vxlan:
last_time
hit:
migrate:
drops:
pleiades24
00:04:f2:41:cb:d4
10.10.11.210
1
0
active
007-10,10:00:13
2013-05-23,12:41:51
1983
0
0
pleiades24
00:25:90:62:12:3a
10.10.10.115
1
0
2013-05-23,12:53:51
89803
2398022
1863639
pleiades24
64:0e:94:28:00:fa
10.13.3.23
1
0
2013-05-23,12:57:53
13989
2
177
Pluribus Networks nvOS Version 2.3.2
41
Using the command options can help quickly determine fabric activity. For example, using the l2-table-show
sort-desc reveals the MAC address that appears most frequently and the associated port:
CLI network-admin@switch > l2-table-show sort-desc hit
switch:
mac:
vlan:
intf:
ports:
state:
create-time:
last-seen:
hit:
switch:
mac:
vlan:
create-time:
last-seen:
hit:
migrate:
drops:
pubdev02
06:a0:00:0e:30:81
38
128
47-48
active
04-13,15:03:08
09:29:57
112
pubdev02
06:a0:00:0e:30:81
22
01-21,11:02:30
01-30,11:22:16
60
1119
74
This information may lead to further investigation of the events by using the connection-stats-show
commands:
CLI network-admin@switch > connection-stats-show ip 10.10.11.3
switch:
mac:
vlan:
ip:
port:
iconns:
oconns:
ibytes:
obytes:
total-bytes:
first-seen:
last-seen:
last-seen-ago:
switch:
mac:
vlan:
ip:
port:
iconns:
oconns:
ibytes:
obytes:
total-bytes:
first-seen:
last-seen:
last-seen-ago:
switch:
mac:
vlan:
42
pleiades24
66:0e:94:21:0e:7b
14
172.16.23.1
65
13
0
132K
375M
375M
06-16,08:15:24
06-16,08:19:11
31d30m19s
pleiades24
66:0e:94:21:f3:34
14
172.16.23.1
65
14
0
132K
375M
375M
06-16,11:54:12
06-16,11:58:25
30d20h51m5s
pleiades24
66:0e:94:21:67:e1
11
Pluribus Networks nvOS Version 2.3.2
ip:
port:
iconns:
oconns:
ibytes:
obytes:
total-bytes:
first-seen:
last-seen:
last-seen-ago:
switch:
mac:
vlan:
ip:
port:
iconns:
oconns:
ibytes:
obytes:
total-bytes:
first-seen:
last-seen:
last-seen-ago:
172.16.23.1
65
57
0
398K
1.10G
1.10G
06-20,15:05:39
07-02,09:44:05
14d23h5m25s
pleiades24
66:0e:94:21:78:2e
14
172.16.23.1
65
69
1
662K
1.83G
1.83G
06-16,14:58:42
06-17,11:12:48
29d21h36m42s
Checking and Fixing Layer 2 Table Issues
You can use the command, l2-check-show, to display any discrepancies with the Layer 2 entries:
CLI network-admin@switch > l2-check-show
pubdev01: Matched: 12
To repair any issues with the Layer 2 table, use the l2-check-fix command:
CLI network-admin@switch > l2-check-fix
OK:12
Rebooting, Powering Off, and Resetting the Server-Switch
There are two recommended ways to reboot a switch:
CLI command switch-reboot
Power button
To reboot the switch using the CLI, use the following command:
CLI network-admin@switch > switch-reboot
Informational Note: The switch-reboot command applies only to the switch where the command is
executed. You cannot reboot a remote switch using this command.
Alternatively, you can use the power button located on the front of the switch to power off.
Pluribus Networks nvOS Version 2.3.2
43
To power off the switch, press and hold the front power button for approximately ten seconds until the power
button light changes from a rapid blink to a slow flashing cycle. The power button light tuns off and now the switch
is powered off.
You can also use the command, switch-poweroff, to turn off a switch.
To complete the process, switch the power toggle on the rear of the switch from 1 to 0. The system is now
completely powered off.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Installing the nvOS Linux API
nvOS is bundled with a Linux API that allows installation of nvOS on any Linux-based server. The API installs libraries
under /lib64, documents under /usr/share/java/doc/libnvOS/index.html, and sample code under
/usr/share/src/nvOS/samples.
Informational Note: You must physically connect the Linux host to the switch.
1. Modify the SFTP permissions on the switch using the admin-sftp-modify enable command.
To install the API on a Linux platform, use the following command:
CLI network-admin@switch > api-install linux-host name linux-host-string user
user-string
To run nvOS on the Linux host, use the following command:
cli --host switch-name ip
44
Pluribus Networks nvOS Version 2.3.2
Configuring Rapid Spanning Tree Protocol (RSTP)
Spanning Tree Protocol (STP) is a standard inter-switch protocol to ensure that an ad hoc network topology is
loop-free at Layer 2, on a per-VLAN basis. If your network connections form loops and STP is disabled, packets
re-circulate between the switches, causing a degradation of network performance. If you are certain that your
network connections are loop-free, you do not need to enable STP.
To build a loop-free topology, switches (“bridges”) have to determine the root bridge and compute the port roles,
root, designated, or blocked. To do this, the bridges use special data frames called Bridge Protocol Data Units
(BPDUs) to exchange information about bridge IDs and root path costs. BPDUs are exchanged regularly, typically at
two second intervals, and enable switches to keep track of network topology changes and to start and stop
forwarding on ports as required. Hosts should not send BPDUs to their switch ports and to avoid malfunctioning or
malicious hosts from doing so, the switch can filter or block BPDUs. If you enable BPDU filtering on a port, BPDUs
received on that port are dropped but other traffic is forwarded as usual. If you enable BPDU blocking on a port,
BPDUs received on that port are dropped and the port is shut down.Pluribus Networks switches support the Per
VLAN Spanning Tree (PVST) variation of STP, and if a PVST BPDU is detected on a port, PVST is used on that port.
Rapid Spanning Tree Protocol is also supported by modifying an STP port and configuring it as an edge port.
Informational Note: RSTP is enabled on the switch by default.
Before you begin, view the status of STP on the switch by using the following command:
CLI network-admin@switch > stp-show
switch:
enable:
bridge-priority:
hello-time:
forwarding-delay:
max-age:
switch:
enable:
bridge-priority:
hello-time:
forwarding-delay:
max-age:
pleiades24
yes
32768
2
15
20
pleiades23
yes
32768
2
15
20
1. To disable STP, use the following command:
CLI network-admin@switch > stp-modify disable
Pluribus Networks nvOS Version 2.3.2
45
2. To display the STP state, use the following command:
CLI network-admin@switch > stp-state-show
switch:
vlan:
name:
bridge-id:
bridge-priority:
root-id:
root-priority:
root-port:
hello-time:
forwarding-delay:
max-age:
disabled:
learning:
forwarding:
discarding:
edge:
designated:
alternate:
backup:
vlag-mirror:
46
techpubs-aquila2
1
stg-default-stg
64:0e:94:18:00:8f
32769
64:0e:94:18:00:8f
32769
128
2
15
20
none
none
65-66,255
128
65-66,255
65-66,255
none
none
none
Pluribus Networks nvOS Version 2.3.2
To display information about STP on ports, use the stp-port-show command:
CLI network-admin@switch > stp-port-show
switch
-------pubdev03
pubdev03
pubdev03
pubdev03
pubdev03
pubdev03
pubdev03
pubdev03
pubdev03
pubdev02
pubdev02
pubdev02
pubdev02
pubdev02
pubdev02
pubdev02
pubdev02
pubdev01
pubdev01
pubdev01
pubdev01
pubdev01
pubdev01
pubdev01
pubdev01
pubdev01
port
---65
66
67
68
69
70
71
72
255
65
66
67
68
69
70
71
72
65
66
67
68
69
70
71
72
255
block
----off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
filter
-----off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
off
guard
----no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
3. To filter BPDUs on port 17, use the following command:
CLI network-admin@switch > stp-port-modify port 17 filter
4. To block BPDUs on port 17 and shut down the port if BPDUs are received on the port, use the following command:
CLI network-admin@switch > stp-port-modify port 17 block
5. To stop blocking BPDUs on port 17, use the following command:
CLI network-admin@switch > stp-port-modify port 17 no-block
6. You can disable STP on a port or a group of ports. If the devices connected to the switch ports are hosts and not
downstream switches, or you know that a loop is not possible, then disable STP and the port is enabled much faster
when the switch restarts.
7. To enable RSTP on port 35, use the following command:
CLI network-admin@switch > stp-port-modify port 35 edge
8. To enable STP, use the following command:
CLI network-admin@switch > stp-modify enable
Pluribus Networks nvOS Version 2.3.2
47
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
48
Pluribus Networks nvOS Version 2.3.2
Configuring Link Aggregation Control Protocol (LACP)
 Configuring Trunking for Link Aggregation (LAG)
 Configuring Layer 2 Multipathing for Virtual Chassis Link Aggregation
 Configuring Active-Active VLAG
 Active-Active VLAG over a Trunk with a Server-Switch and Host
Link Aggregation Control Protocol (LACP) is part of the IEEE specification 802.3ad that allows you to bundle several
physical ports to form a single logical channel. When you change the number of active bundled ports on a port
channel, traffic patterns reflect the rebalanced state of the port channel.
LACP supports the automatic creation of Gigabit Ethernet port trunks by exchanging LACP packets between ports. It
learns the capabilities of port groups and informs the other ports. Once LACP identifies correctly matched Ethernet
links, it facilitates grouping the links into a Gigabit Ethernet port trunks.
LACP packets are exchanged between ports in these modes:
 Active — Places a port into an active negotiating state, and the port initiates negotiations by sending LACP
packets.
 Passive — Places a port into a passive negotiating state where the port responds to LACP packets it receives but
does not initiate LACP negotiation. In this mode, the port channel group attaches the interface to the bundle.
 Off — LACP is not enabled on the switch port or trunk.
Active and passive modes allow LACP to negotiate between ports to determine if they can form a port channel
based on criteria such as port speed and trunking state.
To enable or disable LACP, or change the system priority, use the following command:
CLI network-admin@switch > lacp-modify enable system-priority 35000
The default system priority value is 32768 with a range from 0 to 65535.
LACP system priority can be configured on each switch running LACP. The configuration uses the default value or you
can use another value. LACP uses the system priority with the MAC address to form the system ID and also during
negotiation with other systems.
To create a trunk with LACP, use the following command:
CLI network-admin@switch > trunk-create name trunk23 port 20-36 lacp-mode
active
To modify a trunk with LACP, use the following command:
CLI network-admin@switch > trunk-modify name trunk23 lacp-mode passive
To modify a port configuration and add LACP priority to the port, use the following command:
CLI network-admin@switch > port-config-modify port 33 lacp-priority 34
LACP port priority is configured on each port using LACP. You can use the default value, 32768, or configure a specific
value from 0 to 65535. LACP uses the port priority with the port number to form the port identifier. The port priority
determines which ports should be in standby mode when there is a hardware limitation that prevents all compatible
ports from aggregating.
Pluribus Networks nvOS Version 2.3.2
49
Configuring Trunking for Link Aggregation (LAG)
Informational Note: You must create unique names for each VLAG.
To configure a trunk for aggregating the links connected to ports 1, 2, 3, use the following steps:
1. Create a trunk called trunk-1 on ports 1, 2, 3, enter the following command:
CLI network-admin@switch > trunk-create name trunk-1 port 1,2,3
2. To verify the configuration, use the trunk-show command:
CLI network-admin@switch > trunk-show
name
trunk-1
port
1-3
speed
10g
autoneg
off
jumbo
off
3. Modify the trunk configuration by removing port 2:
CLI network-admin@switch > trunk-modify name trunk-1 port 1,3
4. Verify the updated trunk configuration.
CLI network-admin@switch > trunk-show
name
trunk-1
port
1,3
speed
10g
autoneg
off
jumbo
off
Notice that the ports have changed from 1-3 to 1,3 indicating that port 2 is no longer a member of the trunk
configuration.
5. Delete the trunk configuration from the switch:
CLI network-admin@switch > trunk-delete name trunk-1
Verify that the trunk configuration is removed by using the trunk-show command.
LACP Control Changes
This feature enables ports in a static LACP trunk to operate as individual ports in the absence of proper LACP
negotiation with network peer. Once any port member hears a LACP PDU from the peer, all port members of the
trunk are bundled to operate as a trunk. This feature is useful for servers with multiple network interfaces that
would use PXE boot.
Informational Note: This feature is not supported on virtual link aggregation (vLAG) configurations.
50
Pluribus Networks nvOS Version 2.3.2
With this configuration, nvOS creates the trunk in the switch, but does not add any of the port to the trunk. The
ports continue to operate individually until LACP PDUs are heard on any of the ports that constitute the trunk. Once
LACP PDUs are heard from the peer, then all ports of the trunk cease to operate individually and are added to the
trunk.
If no LACP PDUs are received for the number of seconds configured as the fallback timeout, nvOS LACP checks if
LACP negotiation has expired if LACP negotiation has expired, the ports return to individual mode. If LACP
negotiation has not expired, another fallback timer is scheduled at a value equal to the fallback timeout.
Notes
 LACP fallback timeout is set to 50 seconds and LACP negotiation is set to default 90 seconds.
 After 50 seconds, fallback timer is rescheduled because LACP negotiation has not expired.
 After an additional 40 seconds (90 total) LACP negotiation expires and become inactive. Another 10 seconds
passes (100 seconds total) when the fallback timer expires and the ports fallback to individual.
Configuring Layer 2 Multipathing for Virtual Chassis Link Aggregation
You can aggregate links between two switches by configuring Layer 2 multipathing and virtual chassis Link
Aggregation.
A virtual chassis Link Aggregation Group (VLAG) allows links that are physically connected to two different switches
to appear as a single Ethernet trunk to a third device. The third device can be a server, switch, or any other
networking device. A VLAG can create Layer 2 multipathing which allows you to create redundancy, enabling
multiple parallel paths between nodes.
A VLAG requires that a least one cross connection between the two switches, also called peers, where the VLAG links
terminate. The specific ports that connect the different switches, do not require explicit configuration before
creating a VLAG.
VLAGs can provide the following benefits:
 Allows a single device to use an Ethernet trunk across two access layer (top of rack) switches.
 Eliminates Spanning Tree Protocol (STP) blocked ports.
 Provides a loop-free topology
 Provides fast convergence if a link or device fails.
 Provides link-level resiliency.
 Helps ensure high availability.
Pluribus Networks nvOS Version 2.3.2
51
VLAG Topology Examples
Figure 1:L2 Design - Leaf and Spine with Active-Passive VLAG
Figure 2:L2 Design - Leaf and Spine with Active-Active VLAG
52
Pluribus Networks nvOS Version 2.3.2
Figure 3:L2 Design - Leaf and Third Party Spine without Multichassis LAG or VPC Mode
Figure 4:L2 Design - Leaf and Third Party Spine with Multichassis LAG, vPC and MLAG
To create a VLAG for aggregating links connected to ports 70 on the local switch and the peer called, eng-switch-b,
you must first create a cluster configuration between the two switches. Pluribus Networks switches must be
members of a cluster configuration before you can add VLAGs to them.
Third Party Interoperability with nvOS
Operating System
Host
PN Switch
SmartOS, OpenSolaris,
Illuminos, Oracle Solaris
Create aggr with
lacp-mode passive.
Create lacp-mode active
ad lacp-timeout fast.
Red Hat, Linux
Create bond with
mode 3.
Create lacp-mode off.
Pluribus Networks nvOS Version 2.3.2
53
Operating System
Host
PN Switch
CentOS
Create bond with
mode 4.
Create lacp-mode on.
Configuring Active-Active VLAG
Using the sample topology in Figure 5 Active-Active VLAG over a Trunk with a Server-Switch and Host, use the
following steps to configure Active-Active VLAG:
Informational Note: There must be a physical connection between PN-0 and PN-1 before you can configure VLAG.
Figure 5:Active-Active VLAG over a Trunk with a Server-Switch and Host
Three Pluribus Networks switches in a common fabric with the Spine switch as the RSTP root. It is important to note
that ports 19-22 on PN-0 and PN-1 are ports connected to PN-2 (Spine). Port 26 connects PN-0 to PN-1 for the
cluster configuration required for VLAG.
1. On PN-2, use the following command:
CLI network-admin@switch > stp-modify bridge-priority 4096
2. Create the fabric and add the switches:
54
Pluribus Networks nvOS Version 2.3.2
On PN-2, use the fabric-create command:
CLI network-admin@switch > fabric-create name fab-vlag
On PN-1, join the fabric:
CLI network-admin@switch > fabric-join name fab-vlag
On PN-0, join the fabric:
CLI network-admin@switch > fabric-join name fab-vlag
3. Create VLAN connectivity from the top switch to the bottom:
On PN-2, create the VLAN with scope fabric:
CLI network-admin@switch > vlan-create id 25 scope fabric
On PN-0, add the VLAN and untag the port connected to the host.
CLI network-admin@switch > vlan-port-add vlan-id 25 untagged ports 9
On PN-1, add the VLAN and untag the port connected to the host.
CLI network-admin@switch > vlan-port-add vlan-id 25 untagged ports 9
On PN-0, modify the host STP port to be an edge port.
CLI network-admin@switch > stp-port-modify port 9 edge
On PN-1, modify the host STP port to be an edge port.
CLI network-admin@switch > stp-port-modify port 9 edge
4. Create a cluster configuration between PN-1 and PN-0. This creates the cluster across port 26.
On PN-0, enter the cluster-create command:
CLI network-admin@switch > cluster-create name vlag cluster-node-1 PN-0
cluster-node-2 PN-1
5. You must disable ports between PN-2 and PN-0, and then create a static trunk between them:
On PN-0, modify the ports facing PN-2:
CLI network-admin@switch > port-config-modify port 19,20 disable
Pluribus Networks nvOS Version 2.3.2
55
Then create the trunk on PN-0:
CLI network-admin@switch > trunk-create name pn0-to-pn2 port 19,20 lacp-mode
off
CLI network-admin@switch > trunk-show format all layout vertical
switch:
intf:
name:
port:
speed:
autoneg:
jumbo:
enable:
lacp-mode:
lacp-priority:
lacp-timeout:
reflect:
edge-switch:
pause:
description:
loopback:
mirror-only:
unknown-ucast-level:
unknown-mcast-level:
broadcast-level:
lport:
rswitch-default-vlan:
port-mac-address:
status:
config:
send-port:
PN-0
128
pn0-to-pn2
19-20
10g
off
off
off
off
32768
slow
off
no
no
off
off
100%
100%
100%
0
0
06:60:00:02:10:80
0
From the above output, you can find the name of the trunk configuration, pn0-to-pn2. You need this information
to create the VLAG.
Then, on PN-1, repeat the same commands to create a trunk between PN-1 and PN-2.
6. You must disable ports between PN-2 and PN-1, and then create a static trunk between them:
On PN-1, modify the ports facing PN-2:
56
Pluribus Networks nvOS Version 2.3.2
port-config-modify port 21,22 disable
CLI network-admin@switch > trunk-create name pn1-to-pn2 port 21,22 lacp-mode
off
CLI network-admin@switch > trunk-show format all layout vertical
switch:
intf:
name:
port:
speed:
autoneg:
jumbo:
enable:
lacp-mode:
lacp-priority:
lacp-timeout:
reflect:
edge-switch:
pause:
description:
loopback:
mirror-only:
lport:
rswitch-default-vlan:
port-mac-address:
status:
config:
send-port:
0
PN-0
129
pn1-to-pn2
21-22
10g
off
off
off
off
32768
slow
off
no
no
off
off
0
0
06:60:00:02:10:80
7. Now create the VLAG from the bottom switches going upward and static trunk from the top down. Keep one side
of the VLAG disabled while you configure this step.
On PN-0, use the vlag-create command:
CLI network-admin@switch > vlag-create name to-spine port 128 peer-port 129
peer-switch PN-1 lacp-mode off mode active-active
On PN-2, create a trunk with the name trunk-pn:
CLI network-admin@switch > trunk-create name trunk-pn port 19,20,21,22
lacp-mode off
8. Now, you can enable ports on all switches:
On PN-2, enter the port-config-modify command:
CLI network-admin@switch > port-config-modify port 19,20,21,22 enable
On PN-0, enter the port-config-modify command:
CLI network-admin@switch > port-config-modify port 19,20 enable
On PN-1, enter the port-config-modify command:
CLI network-admin@switch > port-config-modify port 21,22 enable
Pluribus Networks nvOS Version 2.3.2
57
9. Create the server-facing VLAG:
On PN-0, enter the vlag-create command:
CLI network-admin@switch > vlag-create name to-spine port 9 peer-port 9
peer-switch PN-1 lacp-mode active mode active-active
Display the VLAG configuration information:
CLI network-admin@switch > vlag-show format all layout vertical
id:
name:
cluster:
mode:
switch:
port:
peer-switch:
peer-port:
failover-move-L2:
status:
local-state:
lacp-mode:
lacp-timeout:
lacp-key:
lacp-system-id:
a000024:0
to-spine
vlag
active-active
pubdev02
trunk2
pubdev01
129
no
normal
enabled,up
off
slow
26460
110013777969246
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
58
Pluribus Networks nvOS Version 2.3.2
Configuring Tagged and Untagged VLANs
Creating untagged VLANs is useful for connecting the switch to devices that do not support IEEE 802.1Q VLAN tags.
You can configure ports to map untagged packets to a VLAN.
Reserved VLANs and VLAN 0 and 1
The VLAN identifier is a 12-bit field in the header of each packet. Therefore, the maximum number of VLANs you can
define is 4096. Pluribus Networks switches reserve VLANs 0, 1, 4093, 4094, and 4095 for internal use.VLAN 0 is not a
standard VLAN in nvOS. It is used to represent all untagged or non-VLAN traffic. VLAN 1 is the default untagged
traffic VLAN. Untagged traffic can be mapped to any VLAN, but by default, it is mapped to VLAN 1.
It’s important to note that if you create a VLAN with scope fabric and untag all ports, you can cause problems
with the fabric communication.
Informational Note: The untagged VLAN feature is not the same as the default VLAN using the IEEE
802.1Q tag 1.
1. To create a VLAN on the current switch, with the identifier 595, use the following command:
CLI network-admin@switch > vlan-create name VLAN595 id 595 scope local
By default, all ports are trunked on the new VLAN. If you want to specify ports that are trunked, use the optional
parameter, ports, with a comma separated list of ports, or specify a range of ports.
In some cases, you may not want the VLAN created on all ports. You can specify none to apply the VLAN to
internal ports only.
CLI network-admin@switch > vlan-create id 35 scope fabric ports none
CLI network-admin@switch > vlan-show
switch:
id:
nvid:
scope:
name:
active:
stats:
vrg:
ports:
untagged-ports:
active-edge-ports:
switch:
pubdev01
35
a000030:23
fabric
vlan-35
yes
yes
0:0
65-72,255
none
none
pubdev02
To map ports on different switches into the scope fabric VLAN, use the following command:
CLI network-admin@switch > vlan-port-add switch switch-name ports
Pluribus Networks nvOS Version 2.3.2
59
To modify a VLAN name, use the vlan-modify command to modify VLAN 25 name from blue to red:
CLI network-admin@switch > vlan-modify id 25 name blue
To modify the port list, use the vlan-port-add and the vlan-port-remove commands.
2. To display the VLANs configured on the switch, use the vlan-show command.
CLI network-admin@switch > vlan-show format all layout vertical
switch:
id:
nvid:
scope:
name:
active:
stats:
vrg:
ports:
untagged-ports:
active-edge-ports:
active-edge-ports:
switch:
id:
nvid:
scope:
name:
active:
stats:
vrg:
ports:
untagged-ports:
pubdev01
1
a000030:1
local
default-1
yes
yes
0:0
1-72,128,255
1-72,128,255
31,45-46,66,128
65,128-129
pubdev02
1
a000024:1
local
default-1
yes
yes
0:0
1-72,128-129,255
1-72,128-129,255
3. To configure ports 17 and 18 to accept untagged packets and map them to VLAN 595, use the following command:
CLI network-admin@switch > vlan-port-add vlan-id 595 ports 17,18 untagged
60
Pluribus Networks nvOS Version 2.3.2
Displaying VLAN Statistics
You can display network traffic statistics per VLAN using the vlan-stats-show command. This may be useful
when troubleshooting network issues.
CLI network-admin@switch > vlan-stats-show format all layout vertical
switch:
time:
vlan:
ibytes:
ipkts:
idrops-bytes:
idrops-pkts:
obytes:
opkts:
odrops-bytes:
odrops-pkts:
switch:
time:
vlan:
ibytes:
ipkts:
idrops-bytes:
idrops-pkts:
obytes:
opkts:
odrops-bytes:
odrops-pkts:
switch:
time:
vlan:
ibytes:
ipkts:
idrops-bytes:
idrops-pkts:
obytes:
opkts:
odrops-bytes:
odrops-pkts:
pubdev03
10:51:02
1
36.2T
89.0G
119M
313K
0
0
0
0
pubdev03
10:51:02
35
10.8K
154
0
0
0
0
0
0
pubdev02
10:51:02
1
34.9T
84.6G
3.03M
5.69K
0
0
0
0
The output displays the following information:
 switch
 time
 VLAN ID
 incoming and outgoing bytes
 incoming and outgoing packets
 incoming and outgoing dropped bytes
incoming and outgoing dropped packetsTopic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks nvOS Version 2.3.2
61
62
Pluribus Networks nvOS Version 2.3.2
Implementing Virtual Networks
 Overview
 Using VNETs with nvOS
 Creating a Virtual Network
 Adding DHCP Service to a VNET
 Verify Administrator User Creation
 Configuring Administration Login Using SSH
 Adding a Default Gateway to the VNET
 Adding Ports to the VNET
 Configuring Virtual Resource Groups
Overview
A Virtual Network (VNET) is an abstract network resource realized across a fabric of Pluribus Networks switches.
Using VNETs, you can segregate a physical fabric into many logical networks, each with its own resources, network
services, and Quality of Service (QoS) guarantees. A VNET allows you to completely separate all traffic in one VNET
from the traffic of other VNETs.
Figure 1:Using VNETs with nvOS
Pluribus Networks nvOS Version 2.3.2
63
Each VNET has a single point of management. As the fabric administrator, you can create VNETs and assign
ownership of each VNET to individuals with responsibility for managing those resources. You can create separate
usernames and passwords for each VNET manager. Using the separate VNET administration credentials, the VNET
admin can use Secure Shell (SSH) to connect to the VNET manager and access a subset of the nvOS® CLI commands
to manage that VNET. This way, multiple tenants can share a fabric with each managing a VNET with security, traffic,
and resource protection from other VNETs.
VNETs are very flexible and can be used to create complex network architectures. For example, a Pluribus Networks
switch, or a fabric of switches, can be used to create multiple tenant environments in an OpenStack deployment. In
Figure 1 Using VNETs with nvOS, there are three VNETs, each with a management interface and a data interface.
Each VNET is assigned an IP address pool used for DHCP assignment of IP addresses to each node, server, or OS
component.
Underlying each VNET is the VNET manager. Each VNET manager runs in an OpenSolaris zone. When services are
created for a VNET they occupy the same zone on a server-switch. This is called a shared service and it is the default
when creating services. However, each zone can only support a single instance of a service. If a second service
instance is needed for a VNET, then it needs to occupy a separate zone. This is called a dedicated service. In most
cases, you can create services as shared unless you specifically want to create a dedicated service.
When a fabric is created, a VNET is automatically created and named fabric-name-global. This VNET owns all
resources within the fabric, and as new VNETs are created, resources are moved from the default VNET to the new
VNETs. Global services remain in the default VNET unless assigned specifically to a VNET. The software license for IPS
allows only the global VNET, but you can use it to create DHCP servers and other services for the entire switch.
Specifying the Type of VNET Interface
The mgmt, data, and span keywords used in different commands specify the path used to connect to the network
service. For example, to specify an out-of-band connection to a management interface of a VNET, the interface is
specified using the mgmt keyword. If in-band access to that management interface of the VNET is required, then the
data or span keywords are used in the specific command. The keywords, data and span, are essentially
equivalent but apply to two separate paths. To maximize throughput between the server and the switch
components, it is recommended to use both. The data keyword applies to port 65, and the span keyword applies
to port 66.
Each VNET can have one or more isolating zones and network services are applied to each zone. Network services
have their own zone or share the zone with the VNET manager which is the zone that the VNET user logs into to
manage the VNET. In shared zones, the network interfaces are available to all network services in the shared zones,
regardless of the service that created the network interface.
Informational Note: This is an important concept as you can use service commands such as vlb-interface-add to
add an interface or you can use vnet-manager-interface-add to add interfaces to a VNET. If you want the
service to be specific to a VNET as a dedicated service, then add the interfaces using the service-interface-add
commands.
64
Pluribus Networks nvOS Version 2.3.2
Creating a Virtual Network (VNET)
To separate resources, including switch ports, IP addresses, VLANs, and VXLANs, into separate management spaces,
create a VNET and place the resources in the VNET. Then configure a separate VNET admin to manage the network.
Informational Note: You cannot create another VNET inside of a VNET.
There is no performance impact when you send network traffic through a VNET. Packets are switched in the
hardware with full line-rate bandwidth and the same latency even if the packets are on a VNET or not. But, the VNET
allows you to provide different Service Level Agreements (SLAs) to each VNET when there are multiple VNETs on a
physical switch and there is resource contention based on traffic loads.
Related Tasks
 Creating a Virtual Network
 Configuring Virtual Resource Groups
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks nvOS Version 2.3.2
65
66
Pluribus Networks nvOS Version 2.3.2
Creating a Virtual Network
To separate resources, including switch ports, IP addresses, VLANs, and VXLANs, into separate management spaces,
create a VNET and add those resources to the VNET. Then configure a separate administrator for that VNET.
To create a VNET named vnet1 with VLANs, 125 to 130, and a scope of fabric, use the following command:
CLI network-admin@switch > vnet-create name vnet1 scope fabric vlans 123-130
Vnet created.
To confirm that the VLAN is created, use the vnet-show command:
CLI network-admin@switch > vnet-show layout vertical
switch:
name:
scope:
vlans:
managed-ports:
admin:
vnet-mgr-name:
switch:
name:
scope:
vlans:
managed ports:
admin:
vnet-mgr-name:
antares10
vnet1
fabric
125-130
none
vnet1-admin
vnet1-mgr
antares15
vnet2
fabric
131-135
none
vnet2-admin
vnet2-mgr
When you add VLANs to a VNET, you can either assign a range of VLANs, such as 100-199, or a number of VLANs,
such as 5, which then assigns 5 VLANs from nvOS, starting with the lowest number of the available VLANs. You can
see the difference by using the num-vlans parameter to assign VLANs:
CLI network-admin@switch > vnet-create name tester-1 scope fabric num-vlans 3
CLI network-admin@switch > vnet-show name tester-1 layout vertical
switch:
antares10
name:
vnet1
scope:
fabric
vrg:
vnet1-vrg
num-vlans:
3
vlans:
5-7
managed-ports:
none
admin:
vnet1-admin
vnet-mgr-name:
vnet1-mgr
switch:
antares15
name:
vnet2
scope:
fabric
vlans:
123-130
managed ports:
none
admin:
vnet2-admin
vnet-mgr-name:
vnet2-mgr
All switches in the fabric are now in this VNET.
Pluribus Networks nvOS Version 2.3.2
67
Each VNET is associated with a VNET manager (VNM). The default VNM appends the suffix “mgr” to the name
created for the VNET. If you want to create a different name, use the vnet-mgr-option when creating a VNET.
The VNM represents the management interface to the VNET. You can log into the VNM in the same way you can log
into the management plane of the overall logical switch. In multi-tenant environments, access to the VNM is
typically provided to individual VNET administrators such as cloud tenants or application managers. This way the
VNET administrators can manage the configurations and properties of their VNETs. .
Informational Note: Command Execution Time
Some commands may take a few seconds to complete since there are multiple steps in the
commands.
Informational Note: Storage Pool Use
Use the vnet-create command option vnet-mgr-storage-pool to place the VNET
into a storage pool other than the default storage pool.
Adding Untagged VLANs to a VNET
To add untagged VLANs to a VNET, use the vlan-port-add command:
CLI network-admin@switch > vlan-port-add vlan-id 311 ports 15-20 untagged
Adding DHCP Service to a VNET
To add a pool of IP addresses used by a DHCP service, create the IP pool first. For example, you can create the IP
Pool, dhcp-pool, and addresses in the 172.16.23.0/24 network:
CLI network-admin@switch > ip-pool-create name dhcp-pool vnet vnet1 start-ip
172.16.23.0 end-ip 172.16.23.254 netmask 24
Then create the DHCP service:
CLI network-admin@switch > dhcp-create name dhcp-vnet1 vnet vnet1
initial-ip-pool dhcp-pool
The final step is creating the gateway for the DHCP service:
CLI network-admin@switch > dhcp-pool-modify dhcp-name dhcp-vnet1
dhcp-pool-name dhcp-pool gateway-ip 172.16.23.1
Now when you add Virtual Machines (VMs) such as Ubuntu 11.04 or CentOS 6.5, the interfaces receive IP addresses
from the DHCP service assigned to the VNET.
Informational Note: You can only run one instance of a DHCP service per VNET.
68
Pluribus Networks nvOS Version 2.3.2
Verify Administrator User Creation
When a VNET is created, an administrator for that VNET is automatically created in addition to the VNET manager. In
this example, the VNET, vnet1, is created, and the user vnet1-admin is created. The keyword, admin, is
appended to the name of the VNET. This is the default value, so if you want to create an administrator with a
different name, use the vnet-create admin option. vnet1-admin and the superuser, network-admin can
log into the VNET and manage it.
To confirm that the user was created, use the user-show command:
CLI network-admin@switch > user-show
name
vnet1-admin
scope
fabric
uid
20001
Use the user-modify command to change the password for the VNET administrator. The default password is the
same as the account name, vnet1-admin, in this example.
CLI network-admin@switch > user-modify name vnet1-admin
password:********
confirm password:*********
CAUTION!
It is not recommended to change the initial role for a VNET administrator. User roles have different implications and allow
access to the entire switch instead of just the VNET.
Configuring Administration Login Using SSH
In order for the vnet1-admin to login and administer the VNET using SSH, you must add an IP address on either the
switch data port or the mgmt interface. You cannot access the VNET through the management IP address of the
switch. To add the IP address, use the following command:
CLI network-admin@switch > vnet-manager-interface-add vnet-manager-name
vnet1-mgr if data ip 10.100.1.1/24
If you do not specify a VLAN, the interface is added, by default, to the lowest numbered VLAN in the VNET. To verify
that the interface was added, use the vnet-manager-interface-show command:
CLI network-admin@switch > vnet-manager-interface-show vnet-manager-name
vnet1-mgr layout vertical
vnet-manager-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
to_vnic_flow_name:
vnet1-mgr
vnet1.mgr.eth0
10.100.1.1/24
static
66:0e:94:4b:68:96
123
0
data
Pluribus Networks nvOS Version 2.3.2
69
Now you can SSH to the VNET, using the following syntax:
ssh [email protected]
70
Pluribus Networks nvOS Version 2.3.2
Once you log into the VNET, you are placed directly into the CLI for nvOS. The following commands are available to a
VNET administrator:
acl-ip
acl-mac
client-server-stats
connection
connection-latency
connection-stats
dhcp
dhcp-lease
disk-library
dns
fabric
fabric-node
fabric-stats
igmp
igmp-static-group
igmp-static-source
interface-stats
ip-pool
iso-library
l2-history
l2-table
lldp
log-audit
log-event
log-system-counters
log-system
mcast
nat
netvisor-kvm
netvisor-vmm
netvisor-zone
openflow
openstack
openstack-plugin
ping
port-config
port
port-stats
port-vlan
role
running-config-show
sflow
software-license
software
ssh
ssh-known-hosts-delete
storage-folder
storage-pool
stp-port-event
stp-state
tech-support-show
user
vflow
vflow-share
Pluribus Networks nvOS Version 2.3.2
71
vflow-stats
vlan
vlan-stats
vlb
vnet-manager
vnet-service
vnet
vrouter
vrouter-cached-routes
pager
switch
help
quit
exit
Once you are logged into the VNET, you can add VMs or other features to it. For instance, you can install CentOS and
run applications on it or add Ubuntu servers to the VNET.
To remove an interface from the VNET manager, use the vnet-manager-interface-remove command.
Adding a Default Gateway to the VNET
Use the vnet-manager-modify command to add the gateway, 10.100.1.254 to the configuration.
CLI network-admin@switch > vnet-manager-modify name vnet1-mgr gateway
10.100.1.254
To verify the configuration, use the vnet-manager-show command:
CLI network-admin@switch > vnet-manager-show name vnet1-mgr layout vertical
name:
type:
scope:
vnet:
vnet-service:
state:
gateway:
vnet1mgr
vnet-mgr
fabric
vnet1
shared
enabled
10.100.1.254
Modifying and Displaying VNET Manager Services
You can modify the services on the VNET manager using the vnet-manager-service-modify command. If,
for example, you want to disable Web access to the interface, use the following syntax:
CLI network-admin@switch > vnet-manager-services-modify name pn-lab-vnet-mgr
if pn.lab.vnet.mgr.eth0 no-web
72
Pluribus Networks nvOS Version 2.3.2
To display information about the VNET services, use the vnet-services-show command:
CLI (server-switch)>vnet-service-show layout vertical
name:
type:
scope:
vnet:
vnet-service:
state:
gateway:
name:
type:
scope:
vnet:
vnet-service:
state:
gateway:
pn-dhcp-dns
dhcp
fabric
pn-fab-global
shared
enabled
10.9.9.1
lab-dhcp
dhcp
fabric
pn-lab-vnet
shared
enabled
::
To display information about VNET Manager services, use the vnet-manager-service-show command:
CLI network-admin@switch > vnet-manager-service-show layout vertical
vnet-manager-name:
if:
ssh:
web:
web-ssl:
web-ssl-port:
web-port:
icmp:
vnet-manager-name:
if:
ssh:
web:
web-ssl:
web-ssl-port:
web-port:
icmp:
pn-lab-vnet-mgr
pn.lab.vnet.mgr.eth0
on
on
off
443
80
on
pn-lab-vnet-mgr
pn.lab.vnet.mgr.eth1
on
on
off
443
80
on
Adding Ports to the VNET
Ports can be managed by the VNET, but the VNET does not have absolute control over the port. Untagged traffic on
the port can be tagged to a VLAN that is assigned to the VNET. In most cases, it is not necessary to add a port to the
VNET.
Now, add ports, 5-8, 20-30, to the VNET on the local switch and a remote switch.
CLI network-admin@switch > vnet-port-add vnet-name vnet1 ports 5-8,20-30
CLI network-admin@switch > switch antares15 vnet-port-add vnet-name vnet1 ports
20-50
ports added.
Pluribus Networks nvOS Version 2.3.2
73
To verify the ports, use the vnet-show command:
CLI network-admin@switch > vnet-show name vnet1 layout vertical
switch:
name:
scope:
vlans:
managed-ports:
admin:
vnet-mgr-name:
switch:
name:
scope:
vlans:
managed-ports:
admin:
antares15
vnet1
fabric
123-130
5-8,20-30
vnet1-admin
vnet1-mgr
pleiades15
vnet1
fabric
123-130
5-8,20-30
vnet1-admin
Adding a vRouter to the VNET
If you have a VLAN 10 with a subnet 192.168.10.0/24 and a VLAN 12 with a subnet 192.168.12.0/24 on the same
VNET, net-resources, and you want to route traffic between the two VLANs, use the following steps:
1. Create the VNET.
CLI network-admin@switch > create-vnet name net-resources scope local vlans
10,12
2. Create VLAN 10.
CLI network-admin@switch > vlan-create id 10 scope local ports 10
untagged-ports 10
3. Create VLAN 12.
CLI network-admin@switch > vlan-create id 12 scope local ports 12
untagged-ports 12
4. Create the vRouter, subnets.
CLI network-admin@switch > vrouter-create name subnets vnet net-resources
enable
5. Add a vRouter interface for VLAN 10.
CLI network-admin@switch > vrouter-interface-add vrouter-name subnets ip
192.168.10.254 netmask 255.255.255.0 vlan 10
6. Add a vRouter interface for VLAN 12.
CLI network-admin@switch > vrouter-interface-add vrouter-name subnets ip
192.168.12.254 netmask 255.255.255.0 vlan 12
74
Pluribus Networks nvOS Version 2.3.2
To view the configuration, use the vrouter-interface-show command:
CLI network-admin@switch > vrouter-interface-show layout vertical
switch:
vrouter-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
switch:
vrouter-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
pleiades24
subnets
net-resources.mgr.eth1
192.168.10.254/24
static
66:0e:94:24:34:31
10
0
data
pleiades24
subnets
net-resources.mgr.eth2
192.168.12.254/24
static
66:0e:94:24:f8:s9
12
0
data
Informational Note: Network Services Locations and Migration
All network services, such as VNET managers, DHCP servers, and virtual load balancers,
consume disk space, CPU, and memory on one of the switches in a fabric. There may be
instances when you need to move a service, for example, when a disk space shortage occurs,
or you replace a switch. The migrate commands, such as vnet-manager-migrate,
provide the ability to move the service to a different disk pool if you specify the
storage-pool option, or to a different switch within the fabric, if the location option is
specified.
You cannot migrate NetVMs and NetZones. Instead, you export and import them from the
configuration using the commands iso-image-library-export and
disk-library-image-export.
To complete the VNET configuration, you can assign a Virtual Resource Group (VRGs) to the VNET. VRGs allow you
allocate resources to each VNET so that a single VNET does not consume all of the resources on a switch. See
Configuring Virtual Resource Groups.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks nvOS Version 2.3.2
75
76
Pluribus Networks nvOS Version 2.3.2
Configuring Virtual Resource Groups
After creating a VNET, a corresponding Virtual Resource Group (VRG) is created. You can configure VRGs to limit the
resources assigned to a VNET so that a single VNET cannot monopolize all of the resources of the fabric. The VRG can
be modified to limit the specific resources allocated to a VNET.
To create a VRG, use the following command:
CLI network-admin@switch > vrg-create name vnet1-vrg scope fabric num-vlans 8
vlans 123-150
To check the status of a VRG, use the vrg-show command:
CLI network-admin@switch > vrg-show name vnet1-vrg layout vertical
switch:
name:
scope:
num-vlans:
vlans:
ports:
num-flows:
rack-bw-limit (Mbps):
rack-bw(Mbps):
storage-bw(Mbps):
dc-bw(Mbps):
wan-bw(Mbps):
traffic-class:
priority:
restricted resources:
antares15
vnet1-vrg
fabric
8
123-130
0
0
0
0
0
0
0
0
If you want to limit the data bandwidth to 400 Mbps for the VNET, you can modify the VRG:
CLI network-admin@switch > vrg-modify name vnet1-vrg data-bw 400m
CLI network-admin@switch > vrg-show name vnet1-vrg layout vertical
switch:
name:
scope:
num-vlans:
vlans:
ports:
num-flows:
data-bw:
storage-bw:
service-bs:
restricted resources:
antares15
vnet1-vrg
fabric
8
123-130
None
0
400
0
0
data-bw
Pluribus Networks nvOS Version 2.3.2
77
And finally, you want to assign the VRG to a VNET so the resource limitations apply to the VNET:
CLI network-admin@switch > vnet-modify name vnet1 vrg vnet1-vrg
vnet-show name vnet1 format all layout vertical
switch:
antares15
id:
a1634:0
name:
vnet1
scope:
fabric
vrg:
vnet1-vrg
num-vlans:
1
vlans:
150
managed-ports:
admin:
vnet1-admin
vnet-mgr-name:
vnet1-mgr
switch:
antares16
id:
a1635:0
name:
vnet1
scope:
fabric
vrg:
vnet1-vrg
num-vlans:
1
vlans:
150
managed-ports:
admin:
vnet1-admin
vnet-mgr-name:
vnet1-mgr
Timesaver: If the VRG is created before you assign it to a VNET, you can save a step by specifying
the VRG when the VNET is created.
About Virtual Resource Group (VRG) Bandwidth Enforcement
The resources available in a fabric of nvOS devices can be managed by allocating them to Virtual Resource Groups
(VRGs). Each VRG can include an allocation of VLANs and a guarantee of a minimum network bandwidth. VNETs are
then assigned to a VRG. The VNET can also include VLANs as well as other services and resources.
In this implementation, each VRG is assigned a Guaranteed Bandwidth (GBW) parameter specified in Mbps. To
enforce the GBW allocation, all network traffic associated with the VRG is sent to the Networking Processor Unit
(NPU). Flows running on VLANs associated with a VRG is assigned a portion of the GBW assigned to the VRG.
This version has the following limitations:
 Bandwidth guarantees for services and data are supported.
 Storage bandwidth guarantees are not supported.
 Available bandwidth is not enforced per VNET when there are multiple VNETs assigned to the same VRG. Only
VRGs and vFlows are allowed specified guaranteed bandwidth.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
78
Pluribus Networks nvOS Version 2.3.2
Configuring Network Services - DHCP and DNS
 Overview of DHCP and DNS
 Configuring IP Pools
 Configuring DHCP Services
 Adding DHCP Interfaces
 Adding DHCP and DNS Records
 Removing DHCP and DNS Services
 Configuring DNS Services
 Creating a DNS Server
 Configuring Network Address Translation Services
 Configuring Hardware-based Network Address Translation(NAT)
Overview of DHCP and DNS
In general, network services are associated with a VNET. When a fabric is created, a global VNET is also created and
should be used if the network service is available to all Server-Switches and all nodes on the network. Select a
specific VNET if the network service applies to a single VNET, limited to the VNETs resources, and is managed by the
VNET manager. Another option is to decide if the network service is applicable to the same logical zone as the VNET
(shared) or applicable to another separate zone (dedicated). For example, the zone on the VNET may already have a
service running, and another instance of the service is needed to avoid a conflict on the network. In the dedicated
instance, the VNET and the dedicated zone must be configured to see the same network traffic, for example, on the
same VLAN.
This topic describes configuring two virtual services, DNS and DHCP.
Figure 1:VNETs Configured for DHCP and DNS
Pluribus Networks nvOS Version 2.3.2
79
Related Tasks
 Configuring IP Pools
 Configuring DHCP Services
 Adding DHCP and DNS Records
 Removing DHCP and DNS Services
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
80
Pluribus Networks nvOS Version 2.3.2
Configuring IP Pools
IP addresses are resources managed as pools. An IP address pool must be associated with a VNET, because a service
associated with the IP address pool, and the supported service must reside in a VNET. The VNET can be the default
fabric VNET created when the fabric is first created, and if this is the case, the IP address pool or pools are available
fabric-wide and have no resource limitations. If you want to assign restrictions to the IP pool, for example, assign it
to a VLAN or set of VLANs. Create a VNET, and then assign the IP address pool to the VNET.
A private IP address pool consists of private IPv4 addresses, which means that the addresses are not routable on the
Internet. However, you can later create and associate a virtual network address translation (vNAT) service between
the external network IP addresses and internal private IP addresses.
Create an IP address pool with the name dhcp-pool on VNET vnet1 using the IP address pool of 192.168.18.2
through 192.168.18.255 and specifying the optional VLAN group 124.
CLI network-admin@switch > ip-pool-create name dhcp-pool vnet vnet1 start-ip
192.168.18.2 end-ip 192.168.18.255 netmask 24 vlan 124
Pool created successfully.
CLI network-admin@switch > ip-pool-show layout vertical
name:
vnet:
scope:
vlan:
start-ip:
end-ip:
network:
dhcp-pool
vnet1
fabric
124
192.168.18.2
192.168.18.254
192.168.18.0/24
The IP address, 192.168.18.1, is excluded from this configuration because you need to configure it as the gateway IP
address of the DNS and DHCP services.
To modify an IP pool, use the ip-pool-modify command. You cannot modify the assigned VNET. If you decide
that you want to use the IP address pool on another VNET, you must delete the IP pool, and create a new one for the
new VNET.
To delete an IP pool, use the ip-pool-delete command.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks nvOS Version 2.3.2
81
82
Pluribus Networks nvOS Version 2.3.2
Configuring DHCP Services
In this configuration, you use the IP address reserved from the IP address pool to create the DCHP service.
Informational Note:
Once you assign an IP address pool to a DHCP service that allocates dynamic IP addresses, you cannot
assign the same addresses as static IP addresses by other virtual network services.
Before you begin, see Configuring DNS Services to configure the DNS service shared by the DHCP.
1. Use the following command to create the DHCP service for VNET, vnet1. The DHCP server uses the assigned IP
address pool to allocate IP addresses to clients on the VNET.
CLI network-admin@switch > dhcp-create name vnet1-dhcp vnet vnet1
initial-ip-pool dhcp-pool
dhcp-show layout
name:
type:
scope:
vnet:
vnet-service:
state:
pxe-boot:
vertical
vnet1-dhcp
dhcp
fabric
vnet1
shared
enabled
disabled
2. Create the DHCP server for the VNET. Assign the IP pool configured earlier to the DHCP server which is used to distribute IP addresses.
CLI network-admin@switch > dhcp-create name vnet1-dhcp vnet vnet1
initial-ip-pool dhcp-pool
3. To display the configuration, use the dhcp-show command:
CLI network-admin@switch > dhcp-show layout vertical
dhcp-show
name:
type:
scope:
vnet:
vnet-service:
state:
pxe-boot:
vnet1-dhcp
dhcp
fabric
vnet1
shared
enabled
disabled
It is not necessary to add a network interface for the DHCP server since it is sharing the DNS service. In this case, the
vNIC is shared between DHCP and DNS.
Pluribus Networks nvOS Version 2.3.2
83
4. To display the vNIC information, use the dhcp-interface-show command:
CLI network-admin@switch > dhcp-interface-show
dhcp-name
nic
ip
mac
vlan if
--------- --------- -vnet1-dhcp vnet1.mgr.eth0 10.100.1.1/24
66:0e:94:4b:a3:e8 123 mgmt
vnet1-dhcp vnet1.mgr.eth1 192.168.18.1/24 66:0e:94:4b:af:75 124 data
5. Configure the options that the DHCP provides to DHCP clients. You can add the default route using the gateway IP
address, DNS domain name, and the IP address of the DNS server.
CLI network-admin@switch > dhcp-pool-modify dhcp-name vnet-dhcp name dhcp-pool
gateway-ip 192.168.18.1 ddns-domain pluribusnetworks.com dns-ip 192.168.18.1
Adding DHCP Interfaces
You can add DHCP services to an interface on the switch. To add DHCP to interface, dhcp-eng, with the IP address,
172.21.16.25, use the following command:
CLI network-admin@switch > dhcp-interface-create name dhcp-eng ip 172.21.16.25
netmask 32 assignment dhcp vlan 25
To modify the DHCP interface, use the dhcp-interface-modify command.
To remove the interface, use the dhcp-interface-remove command.
To display information about the DHCP interfaces, use the dhcp-interface-show command:
CLI network-admin@switch > dhcp-interface-show layout vertical
dhcp-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
to_vnic_flow_name:
dhcp-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
ext-50-dhcp
ext.50.mgr.eth0
10.111.1.1/24
static
66:0e:94:23:c4:7e
50
0
mgmt
www-51-dhcp
www.51.mgr.eth0
10.222.1.1/24
static
66:0e:94:23:bd:f6
51
0
data
Adding DHCP and DNS Records
The DHCP service adds hostname and IP address records dynamically to the DNS service if the DHCP client specifies
a hostname or if there is a static DHCP record for the client. You can also add hostname and IP address records
manually to the DHCP and DNS services.
84
Pluribus Networks nvOS Version 2.3.2
To manually add a static DHCP record, use the dhcp-host-add command:
CLI network-admin@switch > dhcp-host-add dhcp-name vnet1-dhcp hostname host1
fixed-ip 192.168.18.20 mac 10:0a:dd:ee:ff
When this DHCP client obtains a DHCP lease, the hostname and IP address pair are automatically added to the DNS
service.
To manually add a DNS record, use the dns-record-add command:
CLI network-admin@switch > dns-record-add dns-name vnet1-dns domain
pluribusnetworks.com host host2 ip 192.168.18.1
CLI network-admin@switch > dns-record-show
dns-name
vnet1-dns
vnet1-dns
ip
192.168.18.1
192.168.18.21
host
vnet-dns.pluribusnetworks.com
host2.plurisbusnetworks.com
Removing DHCP and DNS Services
To remove the configured DHCP and DNS services and the IP address pool, use the following commands:
CLI network-admin@switch > dhcp-delete name vnet1-dhcp
Deleted vnet1-dhcp
CLI network-admin@switch > dns-delete name vnet1-dns
Deleted vnet1-dns
CLI network-admin@switch > ip-pool-delete name dhcp-pool
Pool dhcp-pool deleted
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks nvOS Version 2.3.2
85
86
Pluribus Networks nvOS Version 2.3.2
Configuring DNS Services
In this topic, the necessary tasks required to configure DNS as a service to provide name translations for the IP
addresses assigned to the DHCP service.
Adding a DNS Server
Add a DNS server for the fabric-wide VNET, vnet1. The DNS and DHCP services are going to share the service zone
with the VNET manager.
1. To add the DNS server, use the following command:
CLI network-admin@switch > dns-create name vnet1-dns vnet vnet1
shared-vnet-service
2. The DNS service must communicate to hosts on the switch ports, so you must create a virtual NIC (vNIC) and add
an IP address. You have to specify the netmask and VLAN for the vNIC.
CLI network-admin@switch > dns-interface-add dns-name vnet1-dns if data ip
192.168.18.1/24 vlan 24
3. To display the configuration, use the dns-interface-show command:
CLI network-admin@switch > dns-interface-show layout vertical
dns-name:
nic:
ip:
assignment:
mac:
vlan:
if:
dns-name:
nic:
ip:
assignment:
mac:
vlan:
if:
vnet1-dns
vnet1.mgr.eth0
10.100.1.1/24
static
66:0e:94:4b:a3:e8
123
data
vnet1-dns
vnet1.mgr.eth1
192.168.18.1/24
static
66:0e:94:4b:af:75
124
data
This is a shared service, so in addition to the interface you just configured, the interface for the VNET manager is also
present.
Multiple domain names can be associated with an IP address. A reverse lookup is a query of the DNS for a domain
names when the IP address is known. This configuration requires that you define a reverse lookup pool IP addresses.
Pluribus Networks nvOS Version 2.3.2
87
4. Configure the DNS server for the domain and the reverse lookup pool for the DNS.
CLI network-admin@switch > dns-domain-add dns-name vnet1-dns domain
pluribusnetworks.com reverse-lookup-ip-pool dhcp-pool dns-ip 192.168.18.1
dns-domain-show layout vertical
dns-name:
domain:
type:
dns-ip:
reverse-lookup-ip-pool:
reverse-lookup-network:
forwarding:
forwarder:
vnet1-dns
pluribusnetworks.com
master
192.168.18.1
dpool
192.168.10.0/24
none
::
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
88
Pluribus Networks nvOS Version 2.3.2
Overview of NAT and Hardware NAT
 Hardware NAT
 NAT and Hardware NAT Use Cases and Scenarios
  Static Mapping of Individual Private IP Addresses to Public IP Addresses
 Configuring Network Address Translation Services
 Configuring Hardware-based Network Address Translation(NAT)
Network Address Translation (NAT) substitutes the real address in a packet with a mapped address that is routable
on the destination network. NAT uses two steps: 1) translating a real address into mapped address, and 2) reversing
the process for returning traffic.
Just as you can assign DHCP and DNS services to a VNET, you can assign NAT services to a VNET. When you create the
NAT service, you can optionally configure it as a dedicated service, in a separate zone, or shared, in the same logical
zone, on a VNET, and assign a storage pool to it. You can also disable and enable the NAT service on the VNET.
Hardware NAT
Previously, NAT services were available only in ONVL software.Hardware-based NAT has the following functionality:
 HW-NAT only translates traffic that travels between different IP address realms and is configured for HW-NAT.
 The IP addresses inside of an internal domain can be re-used by other internal domains such as a VNET.
 A HW-NAT-enabled router, a vRouter, has an IP address translation table to translate addresses between realms.
 A HW-NAT-enabled router translates IP addresses in packets before forwarding the packets according to the
translation table lookup result.
 Endpoints are unaware of the NAT translation.
 If there is more than one exit point, for example, from internal to external realms, each NAT-enabled router
must have the same IP address translation table.
nvOS supports the following types of hardware-based NAT:
 Static basic NAT (Outbound NAT)
 Static basic NAT with subnet mask
 Dynamic NAT
 NAT-Protocol Translation (PT)
 1K bi-directional NAT sessions or subnets
 Only traditional NAT (outbound NAT) is supported. Two way NAT, bi-directional NAT and Twice NAT are not
supported.
 Applications with IP addresses in the payload, for example FTP, are supported with software NAT.
Pluribus Networks nvOS Version 2.3.2
89
NAT and Hardware NAT Use Cases and Scenarios
Figure 1: Static Mapping of Individual Private IP Addresses to Public IP Addresses
In Figure 1, a simple NAT diagram of mapping two internal IP addresses to a single external IP addresses.
90
Pluribus Networks nvOS Version 2.3.2
Figure 2: Dynamic NAT and NAT-PT
Figure 3: Static NAT
Pluribus Networks nvOS Version 2.3.2
91
Figure 4: NAT with Port Forwarding
Figure 5: NAT with Dynamic Mapping
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
92
Pluribus Networks nvOS Version 2.3.2
Configuring Network Address Translation Services
To create a NAT service, vnet-nat1, on VNET, vnet-customer, as a dedicated service and enable it, use the following
command:
CLI network-admin@switch > nat-create name vnet-nat1 vnet vnet-customer
dedicated-vnet-service enable
Since this is a dedicated service, or if you have not created any network interfaces, then use the
nat-interface-add command to create the vNICs.
CLI network-admin@switch > nat-interface-add vnet-nat1 ip 10.100.1.1/24
assignment none vlan 123 if data
CLI network-admin@switch > nat-interface-add vnet-nat1 ip 192.168.18.1/24
assignment none vlan 124 if data
To modify the configuration, use the nat-interface-modify command. For instance, to change the VLAN
from 124 to 201, use the following syntax:
CLI network-admin@switch > nat-interface-modify vnet-nat1 ip 192.168.18.1/24
vlan 201
To display the configuration, use the nat-interface-show command:
CLI network-admin@switch > nat-interface-show nat-name vnet1-nat layout
vertical
nat-name: vnet1-nat
nic: vnet1.mgr.eth0
ip: 10.100.1.1/24
assignment: static
mac: 66:0e:94:4b:b8:0c
vlan: 123
vxlan: 0
if: data
nat-name: vnet1-nat
nic: vnet1.mgr.eth1
ip: 192.168.18.1/24
assignment: static
mac: 66:0e:94:4b:9d:cc
vlan: 201
vxlan: 0
if: data
To remove the NAT interfaces, use the nat-interface-remove command.
To delete the NAT service, use the nat-delete command. This command removes the entire NAT configuration
including the associated interfaces.
To modify the NAT service, use the nat-modify command.
Pluribus Networks nvOS Version 2.3.2
93
To enable dynamic NAT for internal IP addresses within the VNET, use the nat-map-add command. Traffic from
the interface is sent to the external IP address of the VNET.
CLI network-admin@switch > nat-map-add nat-name vnet1-nat name to-internal
ext-interface vnet1.mgr.eth0 network 192.168.18.2/24
To display the configuration, use the nat-map-show command:
CLI network-admin@switch > nat-map-show
nat-name
-------vnet-1-nat
name
---to-internal
ext-interface
------------vnet1.mgr.eth0
network
------192.168.18.2/24
The hosts on the VNET must have a default router with the internal IP address of the VNET manager. In this example,
the IP address is 192.168.18.1.
To remove the NAT mapping, use the nat-map-remove command.
Configuring Port Forwarding for NAT
Port forwarding or port mapping consists of configuring a gateway to send all packets received on a particular port
to a specific device on the internal network. For example, if the external network requires access a Web server with
port 80 and IP address 192.168.1.2, it is necessary to define a port forwarding rule on the gateway. The rule redirects
all TCP packets received on port 80 to machine 192.168.1.2.
To configure port forwarding from IP address 10.100.1.1:8888 to the internal IP address 192.168.18.4 and port 22,
use the following command:
CLI network-admin@switch > nat-port-forward-add nat-name vnet1-nat name vm1_ssh
ext-port 8888 int-ip 192.168.18.4 int-port 22
The NAT service now forwards from external address 10.100.1.1 port 8888 to the internal address 192.168.18.4 port
22 and permit Secure Shell connections on the well-known SSH port 22.
To remove the NAT port forwarding configuration, use the nat-port-forward-remove command.
To display NAT port forwarding information, use the nat-port-forward-show command.
Configuring Static NAT
Static NAT maps an unregistered IP address to a registered IP address on a one-to-one basis. This is useful when a
device needs to be accessible from outside the network. To configure a one-to-one mapping of the internal address
192.168.18.4 to the external IP address 10.100.1.1, use the following command:
CLI network-admin@switch > nat-static-nat-add nat-name gateway external-ip
10.100.1.1 internal-ip 192.168.18.4
To display the static NAT configuration, use the nat-static-nat-show command.
To remove the static NAT configuration, use the following syntax:
CLI network-admin@switch > nat-static-nat-remove nat-name gateway external-ip
10.100.1.1
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
94
Pluribus Networks nvOS Version 2.3.2
Pluribus Networks nvOS Version 2.3.2
95
96
Pluribus Networks nvOS Version 2.3.2
Configuring Hardware-based Network Address
Translation(NAT)
Before you can add the hardware-based NAT router, you must configure a fabric, VLAN, and vRouter interface. In this
example, we have the following configuration information:
 fabric-name — corp-fabric
 VLANs — VLAN 2 and VLAN 3
 ports — 53 and 55
 IP addresses — 2.2.2.1/24, 20.20.20.1/24, and 20.20.20.2/24
1. Create the fabric:
CLI network-admin@switch > fabric-create name corp-fabric
2. Create the vRouter:
CLI network-admin@switch > vrouter-create name hw-nat vnet global-default
router-type hardware
3. Add the VLANs to the configuration:
CLI network-admin@switch > vlan-create id 2 scope local ports all
untagged-ports 53
CLI network-admin@switch > vlan-create id 3 scope local ports all
untagged-ports 55
4. Add the vRouter interfaces:
CLI network-admin@switch > vrouter-interface-add vrouter-name hw-nat ip
2.2.2.1/24 vlan 2 if data
CLI network-admin@switch > vrouter-interface-add vrouter-name hw-nat ip
20.20.20.1/24 vlan 3 if data
CLI network-admin@switch > vrouter-interface-add vrouter-name hw-nat ip
20.20.20.2/24 alias-on hw.nat.eth1
5. Add the hardware-based NAT configuration:
CLI network-admin@switch > hw-nat-create name nat1 vrouter-name hw-router
Configuring Static NAT
To add a static NAT configuration to the hardware-NAT vRouter, add the following commands, and use the IP address
20.20.20.2 for an additional interface:
CLI network-admin@switch > hw-nat-static-nat-add hw-nat-name nat1 name
static-nat1 internal-ip 2.2.2.10 external-ip 20.20.20.1
CLI network-admin@switch > hw-nat-static-nat-add hw-nat-name nat1 name
static-nat2 internal-ip 2.2.2.20 external-ip 20.20.20.2
Pluribus Networks nvOS Version 2.3.2
97
Configuring NAT with Port Forwarding
To add port forwarding from Host 1 using ports 1122 and 3344 to Host 2, add the following statements to the
configuration:
1. Remove the static NAT configuration from the previous example:
CLI network-admin@switch > hw-nat-static-nat-remove hw-nat-name nat1
CLI network-admin@switch > hw-nat-static-nat-remove hw-nat-name nat2
2. Add the port forwarding configuration:
CLI network-admin@switch > hw-nat-port-forward-add hw-nat-name nat1 name pf1
ext-ip 20.20.20.1 ext-port 80 int-ip 2.2.2.10 int-port 1122
CLI network-admin@switch > hw-nat-port-forward-add hw-nat-name nat1 name pf1
ext-ip 20.20.20.1 ext-port 80 int-ip 2.2.2.10 int-port 3344
Configuring Dynamic Mapping for NAT
To add dynamic mapping for hardware NAT, remove the port forwarding configuration and add the dynamic
mapping statements:
CLI network-admin@switch > hw-nat-port-forward-remove hw-nat-name nat1 name pf1
CLI network-admin@switch > hw-nat-port-forward-remove hw-nat-name nat1 name pf2
CLI network-admin@switch > hw-nat-map-add hw-nat-name nat1 name map1 network
2.2.2.1/24 ext-ip 20.20.20.1
To display the dynamic mapping, use the hw-nat-session-show:
CLI network-admin@switch > hw-nat-session-show
98
Pluribus Networks nvOS Version 2.3.2
nvOS System Logging and SNMP
 Configuring System Logging
 Displaying Log Counters Information
 Sending Log Messages to Syslog Servers
 Sending Log Messages to Syslog Servers
 Viewing Log Events
 Modifying and Displaying Log Event Settings
 Configuring SNMP
 SNMP Communities
 Users and SNMPv3
 Supported MIBs
Overview
nvOS logs all important activities that occur on the switch and fabrics created on them. Logging is enabled by default
and is viewable using the CLI. You can also configure system logging to send syslog-formatted messages to other
servers configured to receive them as part of centralized logging and monitoring.
Figure 1: nvOS Switch with Syslog Server
nvOS Switch
Pluribus Networks nvOS Version 2.3.2
99
There are three types of activities logged by nvOS:
Table 7: Log Events
Type
Description
Event
Records action observed or performed by switches. Each Event type
can be enabled or disabled. Events are collected on a best effort
basis. If events occur too rapidly to be recorded, the event log is
annotated with the number of events lost. The following are
examples of event types:
•
•
•
•
Port state changes
TCP connections
STP port changes
PTP time corrections
Audit
When an administrative change to the configuration is made, an
audit log is recorded. An audit log consists of the command and
parameters along with the success or failure indication. When a
command fails, an error message is also recorded.
System
The system log records error conditions and conditions of interest.
There are four levels in the system log:
•
•
•
•
Perror
critical
error
warn
note
The perror log records messages on standard error output,
describing the last error encountered.
Each log message includes the following information:
 Category - event, audit, or system
 Timestamp within a microsecond
 Process name and process ID of the process producing the message
 Unique message name
 Unique five digit numerical message code
 Message: additional message-specific parameters and explanation
A log message may include optional parameters, including associated VLAN, VXLAN, or switch port.An audit log
message includes additional information:
 User
 Process ID
 Client IP of the remote computer issuing the command
An event log also includes the event type.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
100
Pluribus Networks nvOS Version 2.3.2
Configuring System Logging
To view event logs using the CLI, enter the following command:
CLI network-admin@switch > log-event-show
category
event
event
event
...
time
name
code event-type port message
2013-06-04,13:12:18.304740 port_up 62
port
62
up
2013-06-04,13:12:18.304740 port_up 62
port
50
up
2013-06-04,13:12:18.304740 port_up 62
port
10
up
To view audit log entries, enter the following command:
CLI network-admin@switch > log-audit-show
category time
name
code user
message
audit
2013-06-04,13:12:18.304740 command 1101 network-admin Command
create vnet id=b000011:! name=vnet1 scope=fabric vrg=b000011:0 vlans=100
vnet_mgr_id=b00001
audit
2013-06-04,13:12:18.304740 command 1101 network-admin Command
create vrouter id=b000011:! name=vnet1 scope=fabric vrg=b000011:0
vlans=100 vnet_mgr_id=b00001
To view system log entries, use the following command:
CLI network-admin@switch > log-system-show
time:
name:
level:
time:
name:
level:
time:
name:
level:
2015-09-17, 06:28:09.351514-07:00
11006
warn
2015-09-17, 11:28:09.351514-07:00
11006
warn
2015-09-17, 13:28:09.351514-07:00
11006
warn
Modifying and Displaying Log Event Settings
By default, only system and port events are logged. Other logging is possible, and you can add other events using the
log-event-settings-modify command. You can modify the way nvOS logs events by using the
log-event-settings-modify command to remove or add log events. For instance to remove logging of PTP
events, use the following command:
CLI network-admin@switch > log-event-settings-modify no-ptp
To display log event settings information, use the log-event-settings-show command.
Pluribus Networks nvOS Version 2.3.2
101
Displaying Log Counters Information
You can display information about the number of events that have occurred on the network by using the
log-system-counters-show command:
CLI network-admin@switch > log-system-counters-show layout vertical
switch:
critical:
error:
warn:
note:
pleiades24
0
0
1061
9
To reset the log counters, use the log-system-counters-reset command.
Formatting and Filtering of Logging Messages
There are many options for filtering and formatting of log messages returned by these commands. Use the <tab>
completion method and ? to explore them.
The log files are also available using SFTP, switch-ip:/sftp/nvOS/logs and NFS,
/net/switch-name/nvOS/logs if you have enabled the services.
Many systems support a syslog facility for sending or receiving log messages. Pluribus Networks infrastructure can
send messages to syslog servers using either RFC 5424 (Structure) or RFC 3164 (legacy) formats.
Sending Log Messages to Syslog Servers
To configure the switch to send all log messages to a syslog server with an IP address of 172.16.21.67, use the
following command:
CLI network-admin@switch > admin-syslog-create name log-all scope fabric host
172.16.21.76
To display the configuration use the admin-syslog-show command:
CLI network-admin@switch > admin-syslog-show
name
scope
log-all fabric
host
172.16.21.67
port
514
message-format
legacy
To specify sending the syslog messages in structured format, per RFC5424, add the message-format option to the
configuration.
CLI network-admin@switch > admin-syslog-modify name log-all message-format
structured
You can also modify the port that the service listens on to another port. More than one syslog listening service can
be configured and appropriate syslog messages are sent to each one.
By default, all log messages are forwarded to syslog servers. To filter the log messages, use the msg-level option
to specify the severity or other options:
CLI network-admin@switch > admin-syslog-match-add syslog-name log-all name
critical-msgs msg-level critical
102
Pluribus Networks nvOS Version 2.3.2
You can modify syslog matching using the admin-syslog-match-modify command, or remove matching
criteria using the admin-syslog-match-remove command.
To display the configuration, use the show command:
CLI network-admin@switch > admin-syslog-match-show
syslog-name
log-all
msg-level
critical
name
critical-msgs
Using Facility Codes with Log Messages
Log messages are labeled with a facility code indicating the area of the software that generated the log message.
ONVLuses the following facility codes by default:
 Log_Daemon
for events and system messages
 Log_AUDIT for audit messages
The following severities are used by default:
 Log_INFO for events and audit messages
 Log_Critical = critical
 Log_ERROR = error
 Log_WARNING = warn
 Log_NOTICE = note
You can override the default values by configuring matches for each syslog configuration which allows ONVLto
translate log messages into fields that the syslog servers understand.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks nvOS Version 2.3.2
103
104
Pluribus Networks nvOS Version 2.3.2
Viewing Log Events
For information about specific log events and their meaning, see the Pluribus Networks Log Message Reference
Guide.
A log message consists of common parameters separated by spaces and a colon (:), and optional parameters such as
key and value pairs, another colon, and then the log-specific message.
To view event logs using the CLI, enter the following command:
CLI network-admin@switch > log-event-show
category:
time:
switch:
program:
pid:
name:
code:
event-type:
vnet:
port:
vlan:
message:
category:
time:
switch:
program:
pid:
name:
code:
event-type:
vnet:
port:
vlan:
message:
category:
time:
event
2014-07-17,07:37:17.466173-07:00
pleiades24
nvOSd
6344
mac_ip_changed
11023
port
global-default
65
200
ip address change: mac=50:33:a5:e0:7f:fd ip=172.16.23.7
event
2014-07-17,07:37:50.109133-07:00
pleiades24
nvOSd
6344
mac_ip_changed
11023
port
vlb-web-svr
65
200
ip address change: mac=50:33:a5:e0:7f:fd ip=172.16.23.1
event
2014-07-17,07:42:17.418349-07:00...
Pluribus Networks nvOS Version 2.3.2
105
To view audit log entries, enter the following command:
CLI network-admin@switch > log-audit-show layout vertical
category:
time:
name:
code:
user:
message:
category:
time:
name:
code:
user:
message:
category:
time:
name:
code:
user:
message:
category:
time:
name:
code:
audit
2014-04-01,14:56:40.763626-07:00
user_command
11001
network-admin
Command "vlan-create id 25
audit
2014-04-01,14:56:40.765839-07:00
logout
11100
network-admin
logout
audit
2014-04-01,14:56:40.847912-07:00
login
11099
network-admin
login
audit
2014-04-01,14:56:40.888363-07:00
logout
11100
...
To view system log entries, use the following command:
CLI network-admin@switch > log-system-show
time:
name:
level:
time:
name:
level:
time:
name:
level:
2013-09-17, 06:28:09.351514-07:00
11006
warn
2013-09-17, 11:28:09.351514-07:00
11006
warn
2013-09-17, 13:28:09.351514-07:00
11006
warn
Modifying and Displaying Log Event Settings
By default, only system and port events are logged. Other logging is possible, and you can add other events using the
log-event-settings-modify command. You can modify the way nvOS logs events by using the
log-event-settings-modify command to remove or add log events. For instance to remove logging of PTP
events, use the following command:
CLI network-admin@switch > log-event-settings-modify no-ptp
106
Pluribus Networks nvOS Version 2.3.2
To display log event settings information, use the log-event-settings-show command.
CLI network-admin@switch > log-event-settings-show
switch:
system:
port:
tcp:
stp:
igmp:
lldp:
lacp:
vdp:
ecp:
evb:
ptp:
openflow:
storage:
tacacs:
pleiades24
on
on
off
off
off
off
off
off
off
off
off
off
on
on
You can modify the log event settings using the log-event-settings-modify command. For example, if you
want to turn on TCP events, use the following command:
CLI network-admin@switch > log-event-settings-modify tcp
CLI network-admin@switch > log-event-settings-show
TCP is now turned
on.
switch:
pleiades24
system:
on
port:
on
tcp:
on
stp:
off
igmp:
off
lldp:
off
lacp:
off
vdp:
off
ecp:
off
evb:
off
ptp:
off
openflow: off
storage: on
tacacs:
on
openstack:on
Displaying Log Counters Information
You can display information about the number of events that have occurred on the network by using the
log-system-counters-show command:
CLI network-admin@switch > log-system-counters-show layout vertical
switch:
critical:
error:
warn:
note:
pleiades24
0
0
1061
9
Pluribus Networks nvOS Version 2.3.2
107
To reset the log counters, use the log-system-counters-reset command.
Formatting and Filtering of Logging Messages
There are many options for filtering and formatting of log messages returned by these commands. Use the <tab>
completion method and ? to explore them.
The log files are also available using SFTP, switch-ip:/sftp/nvOS/logs and NFS,
/net/switch-name/nvOS/logs if you have enabled the services.
Many systems support a syslog facility for sending or receiving log messages. Pluribus Networks infrastructure can
send messages to syslog servers using either RFC 5424 (Structure) or RFC 3164 (legacy) formats.
Sending Log Messages to Syslog Servers
To configure the switch to send all log messages to a syslog server with an IP address of 172.21.16.144, use the
following command:
CLI network-admin@switch > admin-syslog-create name log-all scope fabric host
172.21.16.144
To display the configuration use the admin-syslog-show command:
CLI network-admin@switch > admin-syslog-show
name
scope
log-all fabric
host
172.21.16.144
port
514
message-format
legacy
To specify sending the syslog messages in structured format, per RFC5424, add the message-format option to the
configuration.
CLI network-admin@switch > admin-syslog-modify name log-all message-format
structured
You can also modify the port that the service listens on to another port. More than one syslog listening service can
be configured and appropriate syslog messages are sent to each one.
By default, all log messages are forwarded to syslog servers. To filter the log messages, use the msg-level option
to specify the severity or other options:
CLI network-admin@switch > admin-syslog-match-add syslog-name log-all name
critical-msgs msg-level critical
You can modify syslog matching using the admin-syslog-match-modify command, or remove matching
criteria using the admin-syslog-match-remove command.
To display the configuration, use the show command:
CLI network-admin@switch > admin-syslog-match-show
syslog-name
log-all
msg-level
critical
name
critical-msgs
The parameters to match include msg-start, msg-end, msg-duration, msg-starting-point,
msg-length, and msg-reverse.
108
Pluribus Networks nvOS Version 2.3.2
Using Facility Codes with Log Messages
Log messages are labeled with a facility code indicating the area of the software that generated the log message.
ONVLuses the following facility codes by default:
 Log_Daemon
for events and system messages
 Log_AUDIT for audit messages
The following severities are used by default:
 Log_INFO for events and audit messages
 Log_Critical = critical
 Log_ERROR = error
 Log_WARNING = warn
 Log_NOTICE = note
You can override the default values by configuring matches for each syslog configuration which allows ONVLto
translate log messages into fields that the syslog servers understand.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks nvOS Version 2.3.2
109
110
Pluribus Networks nvOS Version 2.3.2
Configuring SNMP
Simple Network Management Protocol (SNMP) is a widely used protocol for monitoring the health and welfare of
network equipment such as routers, computer equipment and even devices like UPSs. ONVL has implemented
SNMP using Net-SNMP version 5.7.2.
SNMP generally works the same in most implementations and this document does not provide indepth information
about SNMP overall. You can locate many resources on SNMP functionality on the Internet.
SNMP v1, v2, and v3 are now supported in nvOS. The SNMP daemon runs as a service and is launched by using the
following command:
CLI network-admin@switch > admin-service-modify if mgmt snmp
This command launches the daemon, subagents, and opens a port so that remote queries can reach the daemon.
SNMP Communities
Communities are used in SNMPv1 as a method of controlling access to information. You can create a community
using the following command:
CLI network-admin@switch > snmp-community-create community-string name-string
community-type read-only|write-only
To create a SNMP community string named, snmp-group, with read-only privileges, use the following command:
CLI network-admin@switch > snmp-community-create community-string snmp-group
community-type read-only
To modify the SNMP community, snmp-group, to write-only, use the following command:
CLI network-admin@switch > snmp-community-modify community-string snmp-group
community-type write-only
To display information about the SNMP community, snmp-group, use the following command:
CLI network-admin@switch > snmp-community-show community-string snmp-group
switch
-----pleiades24
community-string
---------------snmp-group
community-type
-------------read-only
To delete the SNMP community, snmp-group, use the following command:
CLI network-admin@switch > snmp-community-delete community-string snmp-group
Users and SNMPv3
SNMPv3 creates users as access control mechanisms, and creating users is more complex but also more secure and
more flexible. You can also require that users must authenticate and use encryption. Use the following command to
create a user:
CLI network-admin@switch > snmp-user-create user-name name-string
auth-password [auth|no-auth] priv-password [priv|no-priv]
Pluribus Networks nvOS Version 2.3.2
111
To create the user, snmp-admin, with authentication, password m0nk3ys, use the following command:
CLI network-admin@switch > snmp-user-create user-name snmp-admin auth-password
auth
auth password: ********
confirm password: ********
To modify the SNMP user and add private with the password, b33hiv3, use the following command:
CLI network-admin@switch > snmp-user-modify user-name snmp-admin auth-password
auth
priv-password priv
auth password: ********
confirm password: ********
priv password: ******
confirm password: ******
To display information about the SNMP user, use the following command:
CLI network-admin@switch > snmp-user-show user-name snmp-user
switch
-------pleiades24
user-name auth priv
--------- ---- ---snmp-user yes yes
To delete the SNMP user, use the snmp-user-delete command.
After you create the user, you must grant permission, using View Access Control Model (VACM) to view SNMP
objects:
CLI network-admin@switch > snmp-vacm-create user-name name-string user-type
[rouser|rwuser] oid-restrict string [auth|no-auth] [priv|no-priv]
The parameter, oid-restrict, is an optional argument that specifies a MIB sub-tree that the view is restricted.
In other words, if you specify an OID, only that OID and the descendants in the tree are visible in this view.
To continue with the previous example, snmp-user is a read-only user restricted only to sysContact OID:
CLI network-admin@switch > snmp-vacm-create user-name snmp-user user-type
rouser oid-restrict sysContact no-auth no-priv
To modify the VACM configuration and change no authentication to authentication, use the following command:
CLI network-admin@switch > snmp-vacm-modify user-name snmp-user user-type
rouser auth
To display information about the VACM configuration, use the snmp-vacm-show command:
switch
user-type user-name oid-restrict view auth priv
-------------- --------- ------------ ---- ---- ---pleiades24 rouser
snmp-user sysContact
no
no
To delete the VACM user from the SNMP configuration, use the snmp-vacm-delete command:
CLI network-admin@switch > snmp-vacm-delete user-name snmp-user
112
Pluribus Networks nvOS Version 2.3.2
Supported MIBs
nvOS customized MIBs:
 IfTable
 IfXTable
 EntPhySensorTable
OpenSolaris-supported MIBs:
 SNMPv2
 DISMAN-EVENT — monitors disks, processes and execs
 IF — monitors interfaces
 IP — monitors IP addresses and related information such as ipForwarding, ipForwarding, ipDefaultTTL,
ipInReceives, ipInHdrErrors, ipInAddrErrors, ipForwDatagrams, ipInUnknownProtos, ipInDiscards, ipInDelivers,
ipOutRequests, ipOutDiscards

ipOutNoRoutes

ipReasmTimeout

ipReasmReqds

ipReasmOKs

ipReasmFails

ipFragOKs

ipFragFails

ipFragCreates

ipAddrTable

ipRouteTable

ipNetToMediaTable

ipRoutingDiscards

Last bit mask
 TCP — monitors TCP packet information such as tcpRtoAlgorithm, tcpRtoMin, tcpRtoMax, tcpMaxConn,
tcpActiveOpens, tcpPassiveOpens, tcpAttemptFails, tcpEstabResets, tcpCurrEstab, tcpInSegs, tcpOutSegs,
tcpRetransSegs, tcpConnTable, tcpInErrs, tcpOutRsts,
 UDP — monitors UDP packet information
 HOST-RESOURCES
 NOTIFICATION-LOG
 SNMPv2-SMI
 IF-EXT
 ENTITY-SENSOR
See additional supported MIBs in Table , “”.
Additional commands that support SNMPv1, SNMPv2, and SNMPv3:
— The SNMP engine ID is a unique string of 24 characters that identifies the device
for administrative purposes. This command displays the identification of the local SNMP engine and all remove
engines configured on the switch.
 snmp-engineid-show
— Used to enable notifications about link conditions and common system
errors. This is used with the snmp-monitor commands.
 snmp-trap-enable-modify
Pluribus Networks nvOS Version 2.3.2
113
 snmp-trap-enable-show
— Display enabled SNMP traps.
 snmp-trap-sink-create
— Used to specify a SNMPv1 trap receiver.
 snmp-trap-sink-delete
— Remove SNMP sink traps.
 snmp-trap-sink-modify
— Modify SNMP sink traps.
 snmp-trap-sink-show
— Display SNMP sink traps.
 snmp-v3-trap-sink-create
- Used to specify a SNMPv3 trap receiver.
 snmp-v3-trap-sink-delete
— Used to delete a SNMPv3 trap receiver.
 snmp-v3-trap-sink-modify
— Used to modify a SNMPv3 trap receiver.
 snmp-v3-trap-sink-show
— Used to display a SNMPv3 trap receiver.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
114
Pluribus Networks nvOS Version 2.3.2
Supported MIBs
Table 8: Supported MIBs
MIB
Description
AgentX
This is the MIB module for the SNMP Agent Extensibility Protocol
(AgentX). This MIB module is implemented by the master agent.
Bridge
The Bridge MIB module for managing devices that support IEEE
802.1D.
Disman-Event
The MIB module for defining event triggers and actions for network
management.
Disman-Schedule
This MIB module defines a MIB which provides mechanisms to
schedule SNMP set operations periodically or at specific points in
time.
Disman-Script
This MIB module defines a set of objects that allow you to delegate
management scripts to distributed managers.
Entity
The MIB module for representing multiple logical entities supported
by a single SNMP agent.
Entity-Sensor
This module defines Entity MIB extensions for physical sensors.
Ether-Like
The MIB module that describes generic objects for Ethernet-like
network interfaces.
HCNUM-TC
A MIB module containing textual conventions for high capacity data
types. This module addresses an immediate need for data types not
directly supported in the SMIv2. This short-term solution is meant
to be deprecated as a long-term solution is deployed.
Host-Resources
This MIB is for use in managing host systems. The term `host' is
construed to mean any computer that communicates with other
similar computers attached to the Internet and that is directly used
by one or more human beings. Although this MIB does not
necessarily apply to devices whose primary function is
communications services (e.g., terminal servers, routers, bridges,
monitoring equipment), such relevance is not explicitly precluded.
This MIB instruments attributes common to all Internet hosts
including, for example, both personal computers and systems that
run variants of Unix.
Host-Resources-Types
This MIB module registers type definitions for storage types, device
types, and file system types.
IANA-Address-FamilyNumbers
The MIB module defines the AddressFamilyNumbers textual
convention.
Pluribus Networks nvOS Version 2.3.2
115
Table 8: Supported MIBs
116
MIB
Description
IANA-Language
The MIB module registers object identifier values for well-known
programming and scripting languages. Every language registration
MUST describe the format used when transferring scripts written in
this language.
Any additions or changes to the contents of this MIB module
require Designated Expert Review as defined in the Guidelines for
Writing IANA Considerations Section document. The Designated
Expert will be selected by the IESG Area Director of the OPS Area.
Note, this module does not have to register all possible languages
since languages are identified by object identifier values. It is
therefore possible to registered languages in private OID trees. The
references given below are not normative with regard to the
language version. Other references might be better suited to
describe some newer versions of this language. The references are
only provided as `a pointer into the right direction'.
IANA-RTPROTO
This MIB module defines the IANAipRouteProtocol and
IANAipMRouteProtocol textual conventions for use in MIBs which
need to identify unicast or multicast routing mechanisms.
IANAifType
This MIB module defines the IANAifType Textual Convention, and
thus the enumerated values of the ifType object defined in MIB-II's
ifTable.
IF-Inverted-Stack
The MIB module which provides the Inverted Stack Table for
interface sub-layers.
IF
The MIB module to describe generic objects for network interface
sub-layers. This MIB is an updated version of the ifTable for MIB-II,
and incorporates the extensions defined in RFC 1229.
INET-Address
This MIB module defines textual conventions for representing
Internet addresses. An Internet address can be an IPv4 address, an
IPv6 address, or a DNS domain name. This module also defines
textual conventions for Internet port numbers, autonomous system
numbers, and the length of an Internet address prefix.
IP-Forward
The MIB module for the management of CIDR multipath IP Routes.
IP
The MIB module for managing IP and ICMP implementations, but
excluding their management of IP routes.
IPv6-Flow-Label
This MIB module provides commonly used textual conventions for
IPv6 Flow Labels.
IPv6-ICMP
The MIB module for entities implementing the ICMPv6.
IPv6
The MIB module for entities implementing the IPv6 protocol.
IPv6-TC
Imports Integer32 From SNMPv2-SMI
IPv6-TCP
The MIB module for entities implementing TCP over IPv6.
IPv6-UDP
The MIB module for entities implementing UDP over IPv6.
Pluribus Networks nvOS Version 2.3.2
Table 8: Supported MIBs
MIB
Description
NET-SNMP-AGENT
Defines control and monitoring structures for the Net-SNMP agent.
NET-SNMP-EXAMPLES
Example MIB objects for agent module example implementations
NET-SNMP-EXTEND
Defines a framework for scripted extensions
NET-SNMP
Top-level infrastructure of the Net-SNMP project enterprise MIB
tree
NET-SNMP-PASS
Example MIB objects for "pass" and "pass-persist" extension script
NET-SNMP-TC
Textual conventions and enumerations for the Net-SNMP project
NET-SNMP-VACM
Defines Net-SNMP extensions to the standard VACM view table.
NOTIFICATION-Log
The MIB module for logging SNMP Notifications, that is, Traps and
Informs.
RFC-1215
This module is a empty module. It has been created solely for the
purpose of allowing other modules to correctly import the
TRAP-TYPE clause from RFC-1215 where it should be imported
from. It's a built in type in the UCD-SNMP code, and in fact
RFC-1215 doesn't actually define a mib at all; it only defines macros.
However, importing the TRAP-TYPE is conventionally done from an
import clause pointing to RFC-1215.
RFC-1155-SMI
Exports everything including Lnternet, directory, mgmt,
experimental, private, enterprises, OBJECT-TYPE, ObjectName,
ObjectSyntax, SimpleSyntax, ApplicationSyntax, NetworkAddress,
IpAddress, Counter, Gauge, TimeTicks, Opaque;
RFC-1213
Imports mgmt, NetworkAddress, IpAddress, Counter, Gauge,
TimeTicks
RMON
Imports MODULE-IDENTITY, OBJECT-TYPE, OBJECT-IDENTITY,
NOTIFICATION-TYPE, mib-2, Counter32, Integer32, TimeTicks
FROM SNMPv2-SMI, and TEXTUAL-CONVENTION, DisplayString
FROM SNMPv2-TC, and MODULE-COMPLIANCE, OBJECT-GROUP,
NOTIFICATION-GROUP FROM SNMPv2-CONF
SCTP
The MIB module for managing SCTP implementations.
SMUX
Imports enterprises
FROM RFC1155-SMI
DisplayString
FROM SNMPv2-TC
OBJECT-TYPE
FROM RFC-1212;
SNMP-Community
This MIB module defines objects to help support coexistence
between SNMPv1, SNMPv2c, and SNMPv3.
SNMP-Framework
The SNMP Management Architecture MIB
SNMP-MPD
The MIB for Message Processing and Dispatching
Pluribus Networks nvOS Version 2.3.2
117
Table 8: Supported MIBs
MIB
Description
SNMP-Notification
This MIB module defines MIB objects which provide mechanisms to
remotely configure the parameters used by an SNMP entity for the
generation of notifications.
SNMP-Proxy
This MIB module defines MIB objects which provide mechanisms to
remotely configure the parameters used by a proxy forwarding
application.
SNMP-Target
This MIB module defines MIB objects which provide mechanisms to
remotely configure the parameters used by an SNMP entity for the
generation of SNMP messages.
SNMP-User-Based-SM
The management information definitions for the SNMP User-based
Security Model.
SNMP-USM-AES
Definitions of Object Identities needed for the use of AES by SNMP's
User-based Security Model.
SNMP-USM-DH-Objects The management information definitions for providing forward
secrecy for key changes for the usmUserTable, and for providing a
method for 'kickstarting' access to the agent via a Diffie-Helman key
agreement.
SNMP-View-Based-ACM The management information definitions for the View-based Access
Control Model for SNMP.
118
SNMPv2-Conf
Imports ObjectName, NotificationName, ObjectSyntax from
SNMPv2-SMI
SNMPv2
The MIB module for SNMP entities.
SNMP-SMI
The MIB module that provides the notation for writing SNMP MIBs.
SNMP-TC
Imports TimeTicks from SNMPv2-SMI
SNMP-TM
The MIB module for SNMP transport mappings.
TCP
The MIB module for managing TCP implementations.
Transport-Address
This MIB module provides commonly used transport address
definitions.
Tunnel
The MIB module for management of IP Tunnels, independent of the
specific encapsulation scheme in use.
UCD-Demo
SMIv2 version converted from older MIB definitions.
UCD-DISKIO
This MIB module defines objects for disk IO statistics.
UCD-DLMOD
This file defines the MIB objects for dynamic loadable MIB
modules.
Pluribus Networks nvOS Version 2.3.2
Table 8: Supported MIBs
MIB
Description
UCD-IPFWACC
This module defines MIB components for reading information from
the accounting rules IP Firewall. This would typically let you read
the rules and the counters. I did not include some flags and fields
that I considered irrelevant for the accounting rules. Resetting the
counters of the rules by SNMP would be simple, but I don't consider
it so useful. I gave no consideration to implementing write access
for allowing modification of the accounting rules.
UCD-SNMP
This file defines the private UCD SNMP MIB extensions.
UDP
The MIB module for managing UDP implementations.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks nvOS Version 2.3.2
119
120
Pluribus Networks nvOS Version 2.3.2
High Availability
Pluribus Networks switches automatically perform functions that ease your administrative burden. In the case of
high availability, switches in a fabric automatically detect other switches in the fabric. If multiple connections exist
between two switches, they automatically create an 801.3ad Link Aggregation Group (LAG) between the two
switches for resiliency and load balancing. Other features require configuration such as connecting one device to
two switches, or if LAGs are desired between Pluribus switches and other manufacturers’ equipment.
Configuring a Cluster
If you have two Pluribus switches, and want them to work together to provide networking services in the event one
of the switches fails, the switches must be members of the same fabric, and you must configure them as a cluster.
Pluribus Networks nvOS Version 2.3.2
121
To set up a cluster of two switches, pleiades4 and pleiades6, you must verify that they are members of the existing
fabric:
CLI network-admin@switch > fabric-node-show layout vertical
name:
fab-name:
mgmt-ip:
mgmt-vlan:
fab-tid:
out-port:
version:
state:
name:
fab-name:
mgmt-ip:
mgmt-vlan:
fab-tid:
out-port:
version:
state:
pleiades4
corp-fab
10.9.9.141/16
0
29
0
0.18.2789,pn-nvOS-b144a
online
pleiades6
corp-fab
10.9.9.139/0
0
29
60
0.18.2789,pn-nvOS-b144a
online
To create a cluster configuration, use the following command:
CLI network-admin@switch > cluster-create name cluster1 cluster-node-1
pleiades4 cluster-node-2 pleiades6
To verify the status of the cluster, use the cluster-show command:
CLI network-admin@switch > cluster-show
name
cluster1
state
online
cluster-node-1
pleiades4
cluster-node-2
pleiades6
To replace a failed cluster node, use the cluster-repeer command. However, you must evict the failed node
from the fabric, and then run the cluster-repeer command on an active node after replacing the failed node.
To display information about the cluster, use the cluster-info command:
CLI network-admin@switch > cluster-info format all layout vertical
name:
id:
state:
cluster-node-1:
cluster-node-2:
tid:
ports:
validate:
122
vlag
a000030:1
online
167772208
167772196
1
26
yes
Pluribus Networks nvOS Version 2.3.2
If you want to connect the cluster nodes to an uplink switch, you must configure a VLAG between the ports on the
cluster nodes and the uplink switch.
Informational Note: Before you can create a VLAG, you must configure the two switches in a cluster.
For example, if pleiades6 has port 53 connected to the uplink switch and pleiades4 has port 19 connected to the
uplink switch, create a VLAG by executing the vlag-create command on either of the switches:
CLI network-admin@switch > vlag-create name vlag-uplink local-port 53
peer-switch pleiades4 peer-port 19
This example assumes that you’ve entered the command on pleiades6.
To verify the configuration, use the following command:
CLI network-admin@switch > vlag-show
name
vlag-uplink
local-port
53
peer-switch
pleiades4
peer-port
19
status
online
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks nvOS Version 2.3.2
123
124
Pluribus Networks nvOS Version 2.3.2
Configuring Fabric-based Physical Storage Pools
You can create storage pools on the disks shipped with your switch and create physical storage resources. These
resources can be virtualized and allocated to individual virtual networks. Physical storage consists of hard disk drives
(HDD), solid-state disk drives (SSD), or high-IOPS Fusion-IO Flash-based storage.
Informational Note: Additional storage is not available on the E68 series. For the F64 series, additional
storage is available and must be ordered as an additional component to the switch.
When the switch is booted up, it performs checks for uninitialized storage devices. If found, the devices are
automatically formatted and a storage pool is created on each one.
Informational Note: If you prefer other pool layouts, such as a RAID 1 mirror created from two disks,
then delete the pools on the disks you want to use and add the now-free disks to other pools.
Before you start, display information about the storage set up on the switch:
CLI network-admin@switch > storage-pool-show
switch
------------pleiades01
pleiades01
name
-------datapool
rpool
raid-type
--------no_raid
no_raid
used
----213G
87.5G
avail
----1.58T
21.7G
status
-----ok
ok
state
-----ONLINE
ONLINE
You can also display the physical storage media installed on the switch that is available to create a new storage pool:
CLI network-admin@switch > storage-device-show
switch
------------pleiades01
pleiades01
pleiades01
pleiades01
pleiades01
name
----disk0
disk1
disk4
disk5
disk6
label
---------internal-0
internal-1
back-0
back-1
internal
disk
-----c6t0d0
c6t1d0
c6t4d0
c6t5d0
c1d0p0
type capacity
---- -------disk 112G
disk 112G
disk 932G
disk 932G
flash1.35T
in-use
-----yes
yes
yes
yes
yes
data-set
-------rpool
datapool
datapool
pooldisk1
The column, data-set, refers to the ZFS root pool parameter which identifies the location for storage.
The column, type, identifies the type of storage media as disk or flash.
Pluribus Networks nvOS Version 2.3.2
125
To create a new physical storage pool, with no RAID protection, using available disk disk3, enter the following
command at the command prompt:
CLI network-admin@switch > storage-pool-create name store-new device1 disk3
raid-type no_raid
storage-pool-show
switch
name
--------------pleiades01
rpool
pleiades01
store-new
raid-type
--------no_raid
no_raid
used
---62.7G
92.5K
avail
----10.2G
457G
By default, the storage-pool-create command creates a disk library and image library within the new
storage pool, and exports the libraries to the network by using NFS sharing. Since disk and image library storage is
limited to storage pools other than rpool, optional disk storage is needed to implement those features.
To verify that the disk library is created, use the following command:
CLI network-admin@switch > disk-library-show storage-pool store-new layout
vertical
switch:
pleiades01
name:
disk-lib-pluribus
storage-pool:
store-new
sharing:
nfs
import-share:
pleiades01:/disk-lib/newpool/import
export-share:
pleiades01:/disk-lib/newpool/export
switch:
pleiades01
name:
disk-lib-pool-disk1
storage-pool:
pool-disk1
sharing:
nfs
import-share:
pleiades01:/disk-lib/pool-disk1/import
export-share:
pleiades01:/disk-lib/pool-disk1/export
To display the ISO image library, use the following command:
CLI network-admin@switch > iso-library-show storage-pool store-new layout
vertical
switch: pleiades01
name: iso-lib-store-new
storage-pool: store-new
sharing: nfs
import-share: pleiades24:/iso-lib/store-new/import
export-share: pleiades24:/iso-lib/store-new/export
dedup: no
To delete the physical storage pool, store-new, use the following command:
CLI network-admin@switch > storage-pool-delete name store-new
To verify that the storage pool is deleted, use the storage-pool-show command:
CLI network-admin@switch > storage-pool-show
switch
pleiades01
126
name
rpool
raid-type
no-raid
used
62.7G
avail
10.2G
Pluribus Networks nvOS Version 2.3.2
status
ok
state
ONLINE
To verify that the disk space is now free, use the storage-device-show command:
storage-device-show
switch
name label
--------- ----pleiades01 disk0 internal-0
pleiades01 disk1 internal-1
pleiades01 disk3 back-0
disk
---c6t0d0
c6t1d0
c6t3d0
type
---disk
disk
disk
capacity
-------74.5G
74.5G
466G
in-use
-----yes
yes
no
data-set
-------rpool
rpool
Displaying and Downloading Storage Images
You can use the storage-image commands to view downloaded image files, refresh the list, and download files.
1. Refresh the image list:
CLI network-admin@switch > storage-image-refresh
2. Display the available images:
CLI network-admin@switch > storage-image-show
switch
------------mitch-aquila2
mitch-aquila2
mitch-aquila2
mitch-aquila2
name
--------------------------------CentOS-6.4-x86_64-bin-DVD1.iso.gz
CentOS-6.5-x86_64-bin-DVD1.iso.gz
openstack-centos-neutron.vhd.gz
openstack-centos.vhd.gz
size
----3.94G
4.04G
2.81G
4.31G
status
----------downloaded
downloaded
downloaded
server-only
3. The status, downloaded, means that the images are already downloaded from the server, and the status,
server-only, means that the image is available for downloading.
4. To download the openstack-centos.vhd.gz image, use the following syntax:
CLI network-admin@switch > storage-image-download name openstack-centos.vhd.gz
Periodically run the storage-image-show command to check the status of the download. Once the status
changes to downloaded, you can use the image to create VMs on the switch.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks nvOS Version 2.3.2
127
Creating Virtual Storage for a Virtual Network (VNET)
Virtual storage is useful to store virtual machine (VM) images for a elastic compute pool and as a data share for a
virtual network. Elasticity, in this case, means that you can shift and pool resources across your infrastructure
without over provisioning the network. Virtual storage is available to hosts on the VNET through the NFS protocol.
1. Create an IP pool and VNET to host the servers in the elastic compute pool.
CLI network-admin@switch > vnet-create name elas-com-pool scope local
mgr-eth1-vlan 10 vnet-mgr-name ecp1_vmgr mgr-eth0-ip 10.11.37.4
mgr-eth0-netmask 16
Vnet created.
CLI network-admin@switch > ip-pool-create name vpool vnet elas-com-pool
start-ip 192.168.1.1 end-ip 192.168.1.254 netmask 24
2. Create the virtual storage for VMs with the maximum size of 80GB and set the performance optimization to
latency:
CLI network-admin@switch > storage-folder-create elas-com-pool storage-pool
store-new max-space 80g optimization latency sharing nfs
3. Use the storage-folder-show command to display the storage folder configuration:
name
storage-pool vnet max-space backup sharing dedup optimization
ec1_vstor store-new
0:0
80
no
nfs
no
latency
To delete the storage folder, ec1_vstor, use the storage-folder-delete command.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Managing Host Operating Systems
You can set up host operating system ISO images and disk images on your switch. Host OS images are useful to
automatically provision servers assigned to a virtual network in a stateless computing environment, and create local
Netvisor VMs.
With stateless computing, the underlying compute resources, server hardware, are completely transparent to the
OS or applications using it. This allows an OS or application to move from one server to another very easily.
In this example, the VM image is an ISO file named ubuntu-12.10-desktop-i386.iso that you copy and then install on
the switch.
128
Pluribus Networks nvOS Version 2.3.2
Using the storage pool, store-new, verify that you have enough disk space and that an ISO library is created:
CLI network-admin@switch > storage-pool-show
switch
pleiades24
name
store-new
raid-type
no_raid
used
92.5K
avail
457G
CLI network-admin@switch > iso-library-show layout vertical
switch:
name:
storage:
sharing:
import-share:
export-share:
dedup:
pleiades24
iso-lib-pool-store-new
store-new
nfs
pleiades24:/iso-lib/pool/store-new/import
pleiades24:/iso-lib/pool/store-new/export
no
1. Copy the VM image to your switch from another computer using the ISO library NFS share that was added when
the storage pool was created. Copying the image depends on your computer’s OS, but on a Mac OS platform, use
the $ showmount -e ip-address using the IP address of your switch and the Terminal application.
$showmount -e 10.10.20.147
Exports list on 10.10.20.147:
/disk-lib/store-new/export
Everyone
/nvOS/log
Everyone
/mnt/vmiso/ubuntu-11.04-amd64
Everyone
/disk-lib/new-store/import
Everyone
/mnt/vmiso/centOS-6.5-x86_64
Everyone
/mnt/vmiso/centOS-6.4-x86_64
Everyone
/nvOS/vlb-web-svr-mgr/kickstarts
Everyone
$cd /net/10.10.20.147/disk-lib/store-new/import
$cp ubuntu-12.10-desktop-i386.iso
2. Add the new VM image to your switch using the iso-library-image-import command:
CLI network-admin@switch > iso-library-image-import iso-library-name
iso-lib-pool-disk1 image-label ubuntu-12 image-file
ubuntu-12.10-desktop-i386.iso
Your VM image is now transferred to the virtual store and available for installation on bare metal or virtualized
servers.
Pluribus Networks nvOS Version 2.3.2
129
3. To display a list of VM images on your switch, use the following command:
CLI network-admin@switch > iso-library-image-show iso-library-name
iso-lib-disk1
switch
------------pleiades24
pleiades24
pleiades24
pleiades24
pleiades24
pleiades24
pleiades24
pleiades24
iso-library-name
--------------------iso-lib-pool-datapool
iso-lib-pool-datapool
iso-lib-pool-datapool
iso-lib-pool-datapool
pluribus
pluribus
pluribus
pluribus
label
-----------------ubuntu-13.iso
vmware-setup.iso
ubuntu-12.iso
ubuntu-13.1
ubuntu-11.04-amd64
centOS-6.4-x86_64
centOS-6.5-x86_64
Netvisor-b144b-kvm
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Provisioning Bare Metal Servers
A bare metal environment is a computer system or a network in which a virtual machine is installed directly on
hardware rather than within a host operating system (OS). The term, bare metal, refers to the hard disk where a
computer’s OS is typically installed.
Preboot Execution Environment (PXE - pronounced “pixie”) is an industry standard client and server interface that
allows networked computers without an OS to be configured and booted remotely. PXE provides three things:
 DHCP which allows the client to receive an IP address and gain access to the network servers.
 A set of Application Programming Interfaces (API) used by the client’s Basic Input/Output System (BIOS) or a
Network Bootstrap Program (NBP) that automates the booting of the OS.
 A standard method of initializing the PXE code in the PXE ROM chip or boot disk.
How does PXE work? The process consists of the following steps:
1. The client notifies the switch that it uses PXE.
2. Since the switch is configured for PXE, it sends the client a list of boot servers that contain the available OS.
3. The client finds the boot server that it can use and receives the name of the file to download.
4. The client downloads the file and executes it.
Before You Begin
Before you start the PXE process and provisioning a bare metal server, be sure that you have the following
parameters configured:
 The switch is configured as part of a fabric.
 You have at least one VNET configured.
 Create an IP address pool for the DHCP server.
CLI network-admin@switch > ip-pool-create name dhcppool vnet pxevnet network
172.24.100.0 netmask 24
130
Pluribus Networks nvOS Version 2.3.2
 The DHCP server provides IP addresses to clients that are PXE booting, and using the parameter pxe-boot
all-hosts allows any host to receive an IP address from the IP address pool.
CLI network-admin@switch > dhcp-create name pxedhcp vnet pxevnet
initial-ip-pool dhcppool pxe-boot all hosts
If you specify the parameter, pxe-boot by-host-mac, only PXE-booting systems with registered MAC
addresses are allowed to PXE boot and get an IP address.
1. Rack your bare metal server hardware and connect it to your switch. If you are not using the option pxe-boot
all-hosts, write down the MAC address of the network adapter.
2. To boot a specific MAC address with hostname r5-d4 using PXE boot, use the following command:
CLI network-admin@switch > dhcp-host-add dhcp-name pxedhcp hostname r5-d4 mac
00:25:90:63:8c:26 pxe-boot
3. Power on the bare metal server.
4. After the server has PXE booted, it obtains an IP address from the DHCP server and downloads pxelinux.0
bootloader code.
5. The PXE Boot Menu is displayed on the bare metal server.
6. Select an installation type from the list to install on the bare metal server and complete the installation.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Customizing PXE Boot Options
To create a custom PXE boot image, copy the desired file to the switch, and be sure that an ISO library is created, and
NFS automounting is configured:
cp CentOS-6.2-x86_64-bin-DVD1.iso
/net/server-ip-address/iso-lib/pool-name/import
The IP address is the IP address of the switch, and the pool-name is the storage pool created in the ISO library. Be
sure to import a CD/DVD image that includes the PXE boot files.
Configure the ISO image as an available image for the switch to use in PXE boot environments using the following
syntax:
CLI network-admin@switch > iso-library-image-import iso-library-name store-new
image-label centOS-6.2-x86_64dvd image-file Centos-6.2-x86_64-bin-DVD1.iso
image-library store-new
Pluribus Networks nvOS Version 2.3.2
131
You can use the dhcp-pxe-menu-show to display the default values for the menu:
CLI network-admin@switch > dhcp-pxe-menu-show dhcp-name pxedhcp
dhcp-name: pn-dhcp-dns
name: centOS-6.2-amd64-install
iso-library: pluribus
iso-label: centOS-6.2-x86_64dvd
menu-label: CentOS 6.2 amd64 Install
kernel-iso-path: images/pxeboot/vmlinuz
initrd-iso-path: images/pxeboot/initrd.img
append: initrd=<initrd-path> ks=http://<dhcp-server-ip>:
80/kickstarts/centos.ks ksdevice=eth0 interface=eth0
iso-url: http://::/vmiso/centOS-6.2-x86_64
name
The name of the PXE boot menu item.
iso-label
The name chosen when the ISO image was added.
menu-label
The label for the file as it appears in the PXE boot menu.
kernel-iso-path
The path to the kernel on the ISO image.
initrd-iso-path
The path to initrd on the ISO image
append
Any arguments to pass to the kernel at boot time.
iso-url
The location of the ISO image
The server-ip is the IP address of the switch, and the initrd-path is the path to the copied file on the TFTP
server and is replaced when the PXE menu is generated. You are likely to find any append arguments on the Linux
DVD in the pxelinux.cfg/default file.
Some arguments depend on your switch configuration. The first argument is the DHCP server IP address. The second
argument is the path to the copied initrd file. This file is shared on the TFTP server and is replaced when the PXE
boot menu is generated. Connect using TFTP and download the file to inspect it.
Creating a Custom PXE Boot Menu
You can create your own PXE boot menu based on the details of the ISO image:
CLI network-admin@switch > dhcp-pxe-menu-add dhcp-name pxedhcp name centos-6.5
iso-library iso-lib-pool-disk1 iso-label centOS-5.5-x86_64 kernel-iso-path
/image/pxeboot/vmlinuz initrd-iso-path images/pxeboot/initrd.img append
“initrd=10.10.20.147” menu-label CentOS-6.5
132
Pluribus Networks nvOS Version 2.3.2
Use the dhcp-pxe-menu-show command to display the menu:
CLI network-admin@switch > dhcp-pxe-menu-show
name:
centOS-6.5
iso-library:
pluribus
iso-label:
centOS-6.5-x86_64
menu-label:
CentOS 6.5
kernel-iso-path: images/pxeboot/vmlinuz
initrd-iso-path: images/pxeboot/initrd.img
append:
initrd=<initrd-path>
ks=http://<server-ip>:<web-port>/kickstarts/centos-6_5.ks ksdevice=eth0
interface=eth0
iso-url:
http://172.16.23.1/vmiso/centOS-6.5-x86_64
dhcp-interface:
dhcp-name:
vlb-dhcp
Pluribus Networks nvOS Version 2.3.2
133
134
Pluribus Networks nvOS Version 2.3.2
External Disk Drive Installation Guide
For Pluribus Networks hardware models F64 and E28Q, you can install external hard drive disks for additional
storage. You can install either SSD or Fusion I/O disk types.
Be sure to follow all appropriate precautions to prevent Electrostatic Discharge on the new hard drive disk.
Take care when removing the disk from the ESD bag, and installing it in the hard drive carrier.
Locating the Disk Drive Carrier
The disk drive carrier is located on the rear of the F64 and E28Q models.
External Drive
Location
Before adding or removing disks from the switch, power down the switch.
To remove the disk drive from the switch, use the following steps:
1. Locate the small slot in the drive button, and using a small slot screwdriver or a small coin, turn the slot to align
with the Unlock icon.
Button slot
aligned with
Unlock icon.
2. Press the button to release the drive carrier from the drive slot and release the front latch.
3. Use the latch to carefully pull the drive carrier from the slot.
4. Place the external memory drive into the drive carrier.
5. Line up the holes on the memory drive with the holes on the carrier.
Pluribus Networks nvOS Version 2.3.2
135
6. Insert the screws on each side and using a Phillips head screwdriver, hand tighten the screws into the disk.
7. Return the carrier to the empty slot on the switch, and push the drive into the slot.
8. Close the latch of the drive carrier and be sure that it clicks into place.
9. With a slot screwdriver or small coin, turn the slot in the round button to a vertical position. This locks the drive
into the switch.
10. Power on the switch and the new disk is initialized during the boot process.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
136
Pluribus Networks nvOS Version 2.3.2
Configuring High Availability for Storage Folders
Informational Note: Before you begin configuring this feature, there are two pre-requisites for it:
• You must create a storage folder using the storage-folder-create command.
• You must have the name of the peer storage pools to add to the configuration.
Storage folders can be replicated between two switches by configuring a vFolder on one switch. This creates a
similar folder on the second switch which is replicated from the active switch to the peer switch at the configured
backup interval.
You can also configure an IP address for the vFolder that allows you to share the folder using NFS or SFTP.
In this example, there are two switches in the fabric, pleiades24 and pleiades25. You configured a storage folder,
iso-images, on Pleiades24. The VLAN 110 has the scope fabric, and has a IP pool of 192.168.11.0/24. To backup the
vFolder every 30 minutes, configure the backup interval to 30 minutes. Pleiades25 has a storage pool, datapool,
configured on it.
1. Create a vFolder on pleaides24 and add pleaides25 as the peer switch:
CLI network-admin@switch > storage-vfolder-create name my-backup folder
iso-files local-switch pleiades24 peer-switch pleiades25 peer-pool datapool
backup-interval 30 ha-ip 192.168.11.17 ha-netmask 24 ha-vlan 110 ha-if data
2. Display the configuration using the storage-vfolder-show command:
CLI network-admin@switch > storage-vfolder-show format all layout vertical
name:
folder:
local-switch:
local_pool:
peer-switch:
peer-pool:
backup-interval:
last-backup:
active-sw:
ha-nic:
ha-ip:
ha-vlan:
ha-vxlan:
ha-if:
failover_controller:
failover_action:
force:
my-backup
iso-files
pleiades24
pool-disk4
pleiades25
datapool
1800
10:23:51
pleiades24
eth2.110
192.168.11.17/24
110
0
mgmt
0
stop-old
false
The show output displays the failover controller as 0, the failover-action as stop-old, and force as
false by default.
Currently, failover to the peer switch does not occur automatically. When you issue the
storage-vfolder-failover command, you failover to the peer switch to become the active switch.
CLI network-admin@switch > storage-vfolder-failover name my-backup active-sw
pleiades25
Pluribus Networks nvOS Version 2.3.2
137
When you issue this command, the following actions occur on the local switch:
 The folder, my-backup on the current active switch is deactivated. It is unshared, and unmounted on the local
switch.
 The folder, my-backup, on the peer switch is activated.
 If a HA IP address is configured, it is added to the new primary switch.
 If the local folder is shared over NFS or SFTP, the sharing is activated on the new primary folder.
 The local switch begins replicating the folder, my-backup, onto the peer switch.
Using the Force Option for vFolder Failover
During vFolder failover, if the primary switch is not available, the failover operation fails and returns an error
message. If the force option is specified, the failover operation continues by enabling the folder on the peer switch.
The vFolder on the primary switch is not deactivated.
To use the force option, use the following syntax:
CLI network-admin@switch > storage-vfolder-failover name mybackup active-sw
pleiades25 force
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
138
Pluribus Networks nvOS Version 2.3.2
Configuring a Linux Netvisor KVM
There are three ways to create a Netvisor KVM:
 From a bootable ISO image that runs in memory and is not persistent.
 From a bootable ISO image used to install the Linux distribution onto a disk-image within the switch.
 From an already created disk image imported onto the switch from another switch.
Informational Note: You cannot store disk images and ISO libraries in the root storage pool, rpool.
Storage outside of rpool must be configured using storage-pool commands before you can store images
and ISOs.
1. Your developer virtual machine requires a disk volume to install and store the operating system. Verify that your
switch has sufficient physical storage capacity (GB):
CLI network-admin@switch > storage-pool-show
switch
-----pbg-nvos
pbg-nvos
name
---pool-disk1
rpool
raid-type
--------no_raid
no_raid
used
---422K
21.2G
avail
----5.88G
10G
Using the storage-pool-show command also displays any problems with storage pools, such as failed disks or
degraded RAID states.
Creating a storage pool also creates a disk library. After you create a storage pool, verify that a disk library was
created:
CLI network-admin@switch > disk-library-show layout vertical
switch:
name:
sharing:
import-share:
export-share:
pbg-nvos
disk-lib-pool-disk1
nfs
pbg-nvos:/disk-lib/pool-disk1/import
pbg-nvos:/disk-lib/pool-disk1/export
Look for available ISO images on the switch:
CLI network-admin@switch > iso-library-image-show
switch
-----pbg-nvos
label
----ubuntu-12
library
------iso-lib-pool-disk1
By default, creating a Netvisor KVM occurs on a non-rpool storage pool randomly chosen when you use the
netvisor-kvm-create command. To specify the storage pool for the Netvisor KVM, use the parameter
storage-pool pool-name when creating the Netvisor KVM.
Pluribus Networks nvOS Version 2.3.2
139
2. To create a Netvisor KVM from a bootable ISO image for temporary use, you can use the CentOS-6.5 ISO image on
the switch and add 2 GB of memory for it.
CLI network-admin@switch > netvisor-kvm-create name test vnet VNET33 iso-label
centOS-6.5-x86_64 enable storage-pool p1-testpool memory 2g cpus 2 hda-size
10g boot-order hdisk,cdrom hda-lib disk-lib-vnet1 hda-if ide
Netvm created. Please use netvm-interface-add to add interfaces and
netvm-start to boot.
3. Add a network interface to the Netvisor KVM:
CLI network-admin@switch > netvisor-kvm-interface-add netvm-name vm-temp if
mgmt
4. Verify the interface is added:
CLI network-admin@switch > netvisor-kvm-interface-show
netvisor-kvm-name nic
----------------- --vm-temp
ip
---
assignment mac
--------------
vm-temp.eth0::/0 none
66:0e:94:11:ae:cc 0
5. Now, you can start the NetVM, using the netvisor-kvm-start command:
CLI network-admin@switch > netvisor-kvm-start name vm-temp
VM running. From outside switch, connect to vnc port :1.
Ex: vncviewer 172.17.245.201:1
The IP address for the VNC is the same as the IP address of the KVM interface.
6. To display the status of the Netvisor KVM, use the netvisor-kvm-show command:
CLI network-admin@switch > netvisor-kvm-show layout vertical
name:
type:
scope:
vnet:
vnet-service:
gateway:
memory(MB):
cpus:
vm-state:
boot-order:
iso-label:
hda-label:
hdb-label:
hdc-label:
hdd-label:
vnc-port:
140
vlan vxlan if
---------- --
vm-temp
netvm
fabric
corp-fabric
dedicated
::
2000
1
running
cdrom,hdisk
centOS-6.5
1
Pluribus Networks nvOS Version 2.3.2
0
mgmt
7. To access the Netvisor KVM virtual console, use a compatible VNC viewer.
vncviewer 172.17.245.201:1
TigerVNC Viewer for X version 1.0.0
...
8. The installation interface for the Ubuntu image is displayed.
Informational Note: The KVM exists until the switch is reset by a reboot or power loss. In this case, you
need to recreate the KVM.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Creating a Disk-based Netvisor KVM
To create a disk-based Netvisor KVM, use the Ubuntu ISO image, 2GB of memory, and create a virtual disk for the
Netvisor KVM. You can use the Netvisor KVM disk library created when you create the Netvisor KVM.
Pluribus Networks nvOS Version 2.3.2
141
1. Create the Netvisor KVM and disk library:
CLI network-admin@switch > netvisor-kvm-create name disk-vm vnet corp-fabric
iso-label unbuntu-12 memory 2g hda-size 5g hda-lib disk-lib-pool-disk1
Netvm created. Please use netvm-interface-add to add interfaces, and then
netvm-start to boot
2. Add a network interface to the Netvisor KVM, and then start the Netvisor KVM.
CLI network-admin@switch > netvisor-kvm-interface-add netvm-name disk-vm if
mgmt
CLI network-admin@switch > netvisor-kvm-start name disk-vm
VM running. From outside switch, connect to vnc port :2.
Ex: vncviewer 172.17.245.203:2
3. Display the Netvisor KVM information:
CLI network-admin@switch > netvisor-kvm-show layout vertical
name:
type:
scope:
vnet:
vnet-service:
gateway:
memory:
cpus:
vm-state:
boot-order:
iso-label:
hda-label:
hdb-label:
hdc-label:
hdd-label:
vnc-port:
disk-vm
netvm
fabric
corp-fabric
dedicated
::
2GB
1
running
cdrom,hdisk
ubuntu-12
netvm-disk-vm-hda
2
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
142
Pluribus Networks nvOS Version 2.3.2
Creating a KVM by Importing an ISO Image
To create a NetVM from an imported ISO image, you must copy the image to the disk-library where you install the
NetVM.
1. Copy the ISO image to the disk library:
% cp vm-disk2.img /mnt/tmp/disk-lib/newpool/import
2. Verify that the image is available:
CLI network-admin@switch > disk-library-imports-showname disk-lib-newpool
name
---vm-disk2.img
3. Import the ISO image into the disk library:
CLI network-admin@switch > disk-library-image-import disk-library-name
disk-lib-newpool image-label vm-disk2 image-file vm-disk2.img
4. Create the NetVM that uses the disk image:
CLI network-admin@switch > netvisor-kvm-create name vm-disk2 vnet corp-fabric
hda-lable vm-disk2 memory 2g cpus 2
Netvm created. Please use netvm-insterface-add interfaces, and then
netvm-start to boot.
Adding Virtual Machine (VM) Instances to the Server-Switch
Bhyve images (VMM) provides support for virtual machines but provides better throughput than KVM.
Kernel-based Virtual Machine (KVM) is a Linux kernel virtualization hypervisor that can host different guest
operating systems. VMM is used in a similar manner as KVM, but does not support a graphical user interface (GUI).
Informational Note: nvOS does not have VM-compatible images in the ISO library. You must import
compatible images onto the switch.
You cannot run KVM and VM on the same switch. You must shut down any KVM instances before you can
start VM instances.
To create a VM for CentOS 6.5 with a 20G disk space, and 4G memory on VNET, centos, use the following steps:
Informational Note: VM supports only 1 CPU per virtual machine and does not support a graphical user
interface (GUI).
Pluribus Networks nvOS Version 2.3.2
143
1. Create the VMM disk and storage:
CLI network-admin@switch > netvisor-vm-create name centos6.5 vnet centos scope
fabric iso-label centos-6.5-86_64 memory 4g hda-size 20g
boot-at-console-connect true
Netvisor vm created. Please use interface-add to add interfaces and then
start to boot.
2. Add the interface to the VM:
CLI network-admin@switch > netvisor-vm-interface-add name centos6.5 vlan 100 if
mgmt
3. Start the VMM image:
CLI network-admin@switch > netvisor-vm-start name centos6.5
VM running. Use vmm-console to connect to VM
4. Log into the VM:
CLI network-admin@switch > netvisor-vm-console-login
5. Complete the VM configuration using the CLI interface for CentOS 6.5.
To display a list of VMs on the switch, use the following command:
CLI network-admin@switch > netvisor-vm-show format all layout vertical
id:
name:
type:
scope:
vnet:
vnet-service:
state:
location:
storage-pool:
gateway:
template:
memory:
cpus:
vm-state:
iso-label:
hda-label:
vmm-hda-if:
hdb-label:
vmm-hdb-if:
hdc-label:
vmm-hdc-if:
hdd-label:
vmm-hdd-if:
boot-at-console-connect:
delete-hda:
144
a0000dd:10
centos-6.5
netvmm
fabric
test-b
dedicated
enabled
techpubs-aquila1
rpool
::
no
4G
1
running
centOS-6.5-x86_64
netvisor-vm-centos6.5-hda
ahci-hd
ahci-hd
ahci-hd
ahci-hd
true
false
Pluribus Networks nvOS Version 2.3.2
To view a list of VMM interfaces, use the netvisor-vm-interface-show command:
CLI network-admin@switch > netvisor-vm-interface-show format all layout
vertical
netvisor-vmm-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
alias-on:
exclusive:
nic-config:
nic-state:
netvisor-vmm-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
alias-on:
exclusive:
nic-config:
nic-state:
netvisor-vmm-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
alias-on:
exclusive:
nic-config:
nic-state:
netvisor-vmm-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
alias-on:
exclusive:
nic-config:
nic-state:
b33h1v3
eth0.106
::/0
none
66:0e:94:dd:69:df
106
0
mgmt
no
enable
down
test-bee
eth1.110
::/0
none
66:0e:94:dd:16:42
110
0
mgmt
no
enable
down
ubuntu-11
eth0.13
::/0
none
66:0e:94:dd:dd:02
13
0
mgmt
no
enable
down
centos65
eth1.101
::/0
none
66:0e:94:dd:1f:78
101
0
mgmt
no
enable
down
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks nvOS Version 2.3.2
145
146
Pluribus Networks nvOS Version 2.3.2
Managing Linux VM Images
Linux NetVMs enable you to write software that runs directly on the switch with Linux OS. If the NetVM is
configured on a VNET with the scope fabric, then software that runs on the VMs has access to the complete set of
Pluribus Networks nvOS® APIs which provide an open, programmatic interface to the network.
1. To display the list of all VMs on the switch, use the netvisor-kvm-show command.
2. To start the NetVM named vm-disk, use the netvisor-kvm-start command.
3. To modify the NetVM, use the netvisor-kvm-modify command.
CLI network-admin@switch > netvisor-kvm-modify name vm-disk [disable|enable]
memory cpus hda-size hda-lib boot-order iso-label hda-label hdb-label
hdc-label hdd-label
4. To reset a NetVM, use the netvisor-kvm-reset command.
5. To shutdown the NetVM, use the netvisor-kvm-shutdown command.
6. To immediately halt the NetVM, use the netvisor-kvm-kill command.
7. To permanently delete the NetVM, use the netvisor-kvm-delete command.
The disk library images with NetVM content are not automatically deleted when the NetVM is deleted. The images
remain available if you want to reinstall them. To delete the disk library image and free space in the disk library, use
the disk-library-image-remove command.
Changing the State of a NetVM
The command, netvisor-kvm-kill, is similar to pressing the power button for an extended period on the
virtual system with the NetVM. The command, netvisor-kvm-shutdown, sends an ACPL shutdown signal to
the NetVM and may display a dialog box with a message asking if you want to shutdown the NetVM. The command,
netvisor-kvm-reset sends an ACPI reset signal to the NetVM.
Since netvisor-kvm-shutdown and netvisor-kvm-reset send an ACPI signal to the NetVM, the NetVM
is running until the guest OS shuts it down. The command, netvisor-kvm-show may display a status of running
even after a state change command is issued.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks nvOS Version 2.3.2
147
148
Pluribus Networks nvOS Version 2.3.2
Configuring and Implementing NetZones
Overview
NetZones allow you to execute code within the switches, and allows you to execute x86 Solaris code, either custom
programs or pre-compiled applications. NetVMs allow you to install x86 Linux distributions and execute x86 Linux
code, either custom programs or pre-compiled applications. Software installed in a NetZone or a NetVM can access
the nvOS®APIs which provide an open, programmatic interface to the network.
A NetZone or NetVM can implement one or more standard network interfaces which allows the NetZone or NetVM
to send and receive data on networks. The network interfaces can access the span and data network ports, and
vflow commands can send specific data to the network ports so applications can access the data.
Informational Note: The nvOS® APIs are declared in the following C header files:
• /usr/include/nvc_client.h
• /usr/include/nvOS.h
The Java bindings are documented in /usr/java/doc/libnvos/index.html
Only C and Java APIs are supported by nvOS®.
Configuring a NetZone
The following tasks assist you with creating an OpenSolaris NetZone.
1. Create a NetZone on the switch using the following command:
CLI network-admin@switch > netvisor-zone-create name netzone-solaris vnet
corp-fabric user admin
netzone admin password:*******
confirm netzone admin password:*******
CLI network-admin@switch > netvisor-zone-show layout vertical
name:
type:
scope:
vnet:
vnet-service:
state:
gateway:
user:
password:
floodlight-enable:
netzone-solaris
netzone
fabric
corp-fabric
dedicated
enabled
::
admin
no
The output specifies the name of the NetZone as netzone-solaris with the scope of fabric. The scope of the NetZone
is the same as the VNET where you created the NetZone. In this case, the default VNET has the scope of fabric and
the NetZone has access to all switches in the fabric.
Informational Note: When you create a Netvisor zone, the zone is created in the rpool storage pool
unless you specify a datapool location to create the zone. Use the storage-pool parameter to
specify a storage pool.
Pluribus Networks nvOS Version 2.3.2
149
2. To allow traffic to flow through the NetZone, you create an interface and add an IP address:
CLI network-admin@switch > netvisor-zone-interface-add netzone-name
netzone-solaris if data ip 172.17.176.11/16
CLI network-admin@switch > netvisor-zone-interface-show layout vertical
netzone-name:
ip:
assignment:
mac:
vlan:
vxlan:
if:
netzone-solaris
172.17.176.11/16
static
66:0e:94:11:26:5c
0
0
data
The NetZone is assigned the IP address 172.17.176.11 on the switch interface for data. If you want access to the
NetZone through the management ports, then you should create another interface and add the parameter, mgmt,
instead of data.
3. To access the NetZone, use SSH and any terminal application:
% ssh 172.17.176.11 -t admin
Password:********
Last login: Tue Jan 31 22:07:31 2012 from 172.17.176.100
Pluribus Networks, Inc. SunOS 5.11 pn-snv137 January 2012
4. Display the sample code installed in the admin home directory:
-bash-4.0$ ls -lr
.:
total 3
drwxr-xr-x
6 pbg
./samples:
total 12
drwxr-xr-x
drwxr-xr-x
drwxr-xr-x
drwxr-xr-x
...
2pbg
2pbg
2pbg
2pbg
staff
6 May 30 19:03 samples
staff
staff
staff
staff
5
5
5
5
May
May
May
May
30
30
30
30
19:03
19:03
19:03
19:03
Events
Snoop
events
nvsnoop
-bash-4.0$ cd samples/nvsnoop/
-bash-4.0$ ls
Makefile
README
nvsnoop.c
5. gcc and gmake are preinstalled in the developer zone. Use gmake to build the sample code:
-bash-4.0$ gmake
gcc -pthreads -c nvsnoop.c
gcc -pthreads -o nvsnoop nvsnoop.o -lnvOS -lsocket -lnsl
150
Pluribus Networks nvOS Version 2.3.2
6. You can now run the nvsnoop sample program. Use the admin password that you configured when you installed
the switch.
-bash-4.0$ nvsnoop --vnet myfabric-global --vlan 5 --user network-admin \
--pass <password>
Displaying captured packets. Press Ctrl-C to stop.
switch: b000038, flow: b000038:25, port: 15, size: 102
src-mac: 02:08:20:23:a4:da, dst-mac: 02:08:20:67:ca:2f, vlan: 5, etype: ip
src-ip: 192.168.3.125, dst-ip: 192.168.3.115, proto: icmp
switch: b000038, flow: b000038:25, port: 54, size: 102
src-mac: 02:08:20:67:ca:2f, dst-mac: 02:08:20:23:a4:da, vlan: 5, etype: ip
src-ip: 192.168.3.115, dst-ip: 192.168.3.125, proto: icmp
To delete the NetZone, use the netzone-delete command.
The NetZone is configured with the created user, in this case, admin, as a sudo-er which means that the user can be
the root and install software packages or configure the NetZone to facilitate the creation of the correct environment
for your application.
If the NetZone is configured as part of the global VNET, you can use privileged nvOS® CLI commands and call
privilege nvOS® API library routines.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks nvOS Version 2.3.2
151
152
Pluribus Networks nvOS Version 2.3.2
Configuring vRouter Services
 Configuring BGP on a vRouter
 Configuring Open Shortest Path First (OSPF)
 Configuring Routing Information Protocol (RIP)
 Configuring Static Routes
 Adding IGMP Static Joins to a vRouter
 Configuring Virtual Router Redundancy Protocol
Overview
Virtual Routers (vRouters) are an important part of fabric functionality. For example, for a VNET to communicate
with other VNETs, or networks external to the fabric, it may need a vRouter that spans the VNET and the external
network. vRouter commands can only be executed at the fabric level by the fabric administrator, so there is no
network disruption by VNET administrators. You cannot use the vRouter commands as a VNET administrator.Routing
protocols essentially work the same way on virtual routers as physical routers. Detailed information about routing
protocols is not covered in this overview.
The vRouter feature supports common routing protocols such as BGP, OSPF, RIP, and static routes.
To create a vRouter on the global VNET, and create a gateway between two networks that connect to the switch
ports, use the following command:
CLI network-admin@switch > vrouter-create name default-gateway vnet
fabricname-global
CLI network-admin@switch > vrouter-interface-add vrouter-name default-gateway
ip 172.16.23.33/24 if data
CLI network-admin@switch > vrouter-interface-add vrouter-name default-gateway
ip 10.9.18.147/16 if data
You just created an interface for the external network (10.9.18.147) and the internal network (172.16.23.33). By
default a static route is created between interfaces added to a vRouter.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Configuring Prefix Lists for BGP and OSPF
Prefix lists allow you to permit or deny host IP addresses from route distribution in BGP and OSPF configurations. To
configure prefix lists for BGP, this example assumes that you have a vRouter configured for BGP, vrouter-bgp, and
you want to deny the IP address, 172.26.0.0 with the netmask 255.255.0.0, sequence number 5, and minimum
prefix length 17 bits:
CLI network-admin@switch > vrouter-prefix-list-add vrouter-name vrouter-bgp
name deny-bits action deny prefix 172.26.0.0 netmask 255.255.0.0 seq 5
min-prefix-len 17
This prefix list rejects any subnets of 172.26.0.0/16 with prefixes 17 bits or longer. For example, the subnets
172.26.16.9/30 and 172.26.101.0/24 are rejected from route distribution.
Pluribus Networks nvOS Version 2.3.2
153
The sequence number allows you to insert or remove new lines in a prefix list as well as at the beginning or end. It is
recommended that you increment the sequence numbers by 10 so you can easily add or subtract lists from the
configuration.
Configuring Packet Relay for DHCP Servers
You can configure a vRouter to relay DHCP requests from local clients to a centralized DHCP server. Because the
initial DHCP request arrives from a client that typically does not have an IP address, the client must find the DHCP
server using a Layer 2 broadcast.
The DHCP server must know the subnet and the MAC address of the client before the server can allocate an IP
address to the client. The DHCP server needs the subnet information to ensure that the IP address that the client
receives can work on the client’s subnet. The MAC address is necessary so that the DHCP server can find any
information that is unique to the client.
When you configure the vRouter as a DHCP proxy, the vRouter converts the local broadcast packet from the client to
a unicast packet and forward it to the server.
Because the DHCP client does not have an IP address when it sends the DHCP request packet, the client uses the IP
address, 0.0.0.0, as the source IP address and the general broadcast address 255.255.255.255 for the destination.
The vRouter replaces the source address with the IP address assigned to the interface where the request is received,
and replaces the destination IP address with the address you specify in the vRouter packet-relay command.
To configure packet-relay for a DHCP server with the IP address 172.16.21.34 and vRouter interface eth11.100, use
the following syntax:
CLI network-admin@switch > vrouter-packet-relay add vrouter-name vrouter-dhcp
forward-proto dhcp forward-ip 172.16.21.34 nic eth11.100
Once you’ve added the configuration, you cannot modify it. If you made a mistake or want to add a new
configuration, you must use the vrouter-packet-relay-remove command.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Configuring Hardware Routing for a vRouter
Hardware routing implements the same mechanisms as software routing for the control plane. You create interfaces
on hardware routers and map them to VNICs in the vRouter zone. You can configure up to seven (7) hardware
routers on a platform.
The supported protocols are as follows:
 OSPF — OSPF does not use a TCP/IP transport protocol such as UDP or TCP, but is encapsulated in the IP
datagram with protocol number 89. OSPF uses multicast addressing for route flooding on a broadcast domain.
For nonbroadcast network, special provisions in the configuration facilitate neighbor discovery. OSPF reserves
the multicast addresses 224.0.0.5/6 for IPv4 or FF02::5/6 for IPv6.
 BGP — BGP uses TCP and port number 179.
 RIP — uses the following parameters:
• RIPv1 — IPv4 uses UDP and port 520, and advertise address - broadcasting
• RIPv2 — IPv4 uses UDP and port 520, and advertise address - 224.0.0.9
• RIPng — IPv6 uses UDP and port 521, and advertise address - FF02::9
 PIM — IPv4 uses protocol 103 with multicast address 224.0.0.13
154
Pluribus Networks nvOS Version 2.3.2
To create a hardware routing on a vRouter, hwtest, on VNET, fabricname-global, use the following
command:
CLI network-admin@switch > vrouter-create hwtest vnet fabricname-global
router-type hardware
Use the same commands as software routing to add protocols and interfaces.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks nvOS Version 2.3.2
155
156
Pluribus Networks nvOS Version 2.3.2
Configuring BGP on a vRouter
Border Gateway Protocol (BGP) is a path-vector protocol and is the most commonly used routing protocol on the
Internet. It advertises the paths required to reach a certain destination. BGP is also a Layer 4 protocol that sits on top
of TCP, and is simpler than Open Shortest Path First (OSPF). In Figure 1 Configuring BGP for Two VLANs, you want
network traffic from the source host to reach the destination host. But when different VLANs are configured, the
source host traffic is not aware of the route between the source host and the destination host. However, there is a
VLAN that spans VLAN 33 and VLAN 55. You solve this problem by configuring BGP in the same Autonomous System
(AS) 100 that sends traffic over VLAN 35. This allows the source host to learn the route to the destination host.
Using a loopback address for peering is useful when there are multiple paths between the BGP peers which would
otherwise tear down the BGP session if the physical interface us ed for establishing goes down. It also allows the
vRouters running BGP with multiple links between them to load balance over the available paths.
Figure 1: Configuring BGP for Two VLANs
This example assumes that you have two VLANs, VLAN33 and VLAN55. Also, that you have added ports to the
configuration.
Begin by configuring vRouter1, a software vRouter, on VLAN 33 with the BGP information:
CLI network-admin@switch > vrouter-create name vrouter1 vnet fabricname-global
router-type software bgp-as 100 bgp-redist-connected-metric none
Additional BGP parameters include the following:
 bgp-redist-static-metric — redistribute static BGP route metric number
 bgp-redist-connected-metric — redistribute connected BGP route metric
Pluribus Networks nvOS Version 2.3.2
157
 bgp-redist-rip-metric
— redistribute BGP into RIP process metric
 bgp-redist-ospf-metric — redistribute BGP into OSPF process metric
 bgp-cluster-id — the ID assigned to the BGP cluster.
 bgp-max-paths — maximum number of BGP paths
 bgp-ibgp-multipath — allow the BGP vRouter to select multiple paths for load sharing.
 bgp-bestpath-as-path — allow BGP to use the best path for traffic forwarding.
 bgp-dampening|no-bgp-dampening — suppress flapping routes so they are not advertised.
 bgp-graceful-restart|no-bgp-graceful-restart — mechanism for BGP that helps minimize
the negative effects on routing caused by BGP restart.
 bgp-stalepath-time — how long a router waits before deleting stale routes after an end of record (EOR)
message is received from the restarting router.
Add the IP addresses and VLANs:
CLI network-admin@switch > vrouter-interface-add vrouter-name vrouter1 ip
10.16.35.33/24 vlan 35
CLI network-admin@switch > vrouter-interface-add vrouter-name vrouter1 ip
10.16.33.1/24 vlan 33
Add the BGP information:
CLI network-admin@switch > vrouter-bgp-add vrouter-name vrouter1 neighbor
10.16.35.55 remote-as 100
CLI network-admin@switch > vrouter-bgp-add vrouter-name vrouter1 network
10.16.33.0/24
158
Pluribus Networks nvOS Version 2.3.2
Display the interface information for vrouter33:
CLI network-admin@switch > vrouter-interface-show format all layout vertical
vrouter-name:
vrouter33
nic:
eth1.33
ip:
10.9.100.100/16
assignment:
static
mac:
66:0e:94:30:c6:92
vlan:
33
vxlan:
0
if:
data
alias-on:
exclusive:
no
nic-config:
enable
nic-state:
up
secondary-macs:
vrouter-name:
vrouter33
nic:
eth2.33
ip:
192.168.42.11/24
assignment:
static
mac:
66:0e:94:30:25:5e
vlan:
33
vxlan:
0
if:
data
alias-on:
exclusive:
no
nic-config:
enable
nic-state:
up
secondary-macs:
If you want to filter IP hosts, you can add prefix lists to the BGP configuration. See Configuring Prefix Lists for BGP
and OSPF.
Then, configure vRouter2 on VNET 55:
CLI network-admin@switch > vrouter-create name vrouter2 vnet fabricname-global
router-type software bgp-as 100 bgp-redist-connected-metric none
Add the IP addresses and VLANs:
CLI network-admin@switch > vrouter-interface-add vrouter-name vrouter2 ip
10.16.35.55/24 vlan 35
CLI network-admin@switch > vrouter-interface-add vrouter-name vrouter2 ip
10.16.55.1/24 vlan 55
Then add the BGP information:
CLI network-admin@switch > vrouter-bgp-add vrouter-name vrouter2 neighbor
10.16.35.33 remote-as 100
CLI network-admin@switch > vrouter-bgp-add vrouter-name vrouter2 network
10.16.55.0/24
Pluribus Networks nvOS Version 2.3.2
159
And finally, add the loopback address:
CLI network-admin@switch > vrouter-loopback-interface-add vrouter-name
vrouter1 index 5 ip 1.1.1.1
The index value is a number that uniquely identifies the vRouter in the AS.
Display the vRouter BGP configuration:
CLI network-admin@switch > vrouter-bgp-show format all layout vertical
vrouter-name:
ip:
neighbor:
remote-as:
next-hop-self:
route-reflector-client:
override-capability:
soft-reconfig-inbound:
max-prefix-warn-only:
vrouter-name:
ip:
network:
vrouter-name:
ip:
neighbor:
remote-as:
next-hop-self:
route-reflector-client:
override-capability:
soft-reconfig-inbound:
max-prefix-warn-only:
vrouter-name:
ip:
network:
vrouter33
10.16.35.55
10.16.35.55
100
no
no
no
no
no
vrouter33
10.16.33.0
10.16.33.0/24
vrouter55
10.16.35.33
10.16.35.33
100
no
no
no
no
no
vrouter55
10.16.55.0
10.16.55.0/24
To reset BGP neighbors, use the vrouter-bgp-neighbor-reset command.
160
Pluribus Networks nvOS Version 2.3.2
To display BGP neighbors, use the vrouter-bgp-neighbor-show command.
CLI network-admin@switch > vrouter-bgp-neighbor-show
vrouter-name:
neighbor:
ver:
remote-as:
msg_rcvd:
msg_sent:
tblver:
inQ:
outQ:
up/down:
state/pfxrcd:
vrouter-name:
neighbor:
ver:
remote-as:
msg_rcvd:
msg_sent:
tblver:
inQ:
outQ:
up/down:
state/pfxrcd:
vrouter1
10.9.100.201
4
100
11
19
0
0
0
00:54:04
Connect
vrouter2
10.9.100.101
4
100
12
18
0
0
0
00:53:37
Connect
Additional BGP Parameters
There are additional BGP parameters that you can use to optimize your BGP network. Add any of the following
parameters:
— a value for external BGP to accept or attempt BGP connections to external peers, not
directly connected, on the network. This is a value between 1 and 255.
 ebgp-multihop
vrouter — the source IP address of BGP packets sent by the router. This parameter is
required if you want BGP to perform peering over a loopback interface.
 update-source
 prefix-list-in — specify a list of incoming prefixes for route redistribution.
 prefix-list-out — specify a list of outgoing prefixes for route redistribution.
 override-capability — override the result of capability negotiation with the local configuration. This
parameter allows you to ignore a remote peer’s capability value.
 soft-reconfig-inbound — defines the route refresh capability by allowing the local device to reset
inbound routing tables dynamically by exchanging route refresh requests to supporting peers.
 max-prefix — allows you to specify the maximum number of IP prefixes to filter.
 max-prefix-warn — add a parameter to warn when the maximum number of prefixes is reached.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks nvOS Version 2.3.2
161
162
Pluribus Networks nvOS Version 2.3.2
Configuring Open Shortest Path First (OSPF)
Open Shortest Path First (OSPF) is a robust link-state interior gateway protocol (IGP). You can use it when Router
Internet Protocol (RIP) is not enough for your network or when you need fast convergence on the network. It uses
Autonomous Systems (AS) and the concept of Areas which allows further segmentation on the network.
OSPF uses link-state information to make routing decisions, and make route calculations using the shortest path first
(SPF) algorithm. Each vRouter configured for OSPF floods link-state advertisements throughout the AS or area that
contains information about the router’s attached interfaces and routing metrics.
You can add more configuration options, such as hello intervals, for OSPF using the
vrouter-interface-config commands. In addition, you can add stub or not-so-stubby areas to the OSPF
configuration.
You can also manually change the OSPF cost for the configuration. Cost is the metric used by OSPF to judge the
feasibility of a path. If you specify 0 as the cost, the vRouter automatically calculates the cost based on the
bandwidth of the interface.
Informational Note: For switches with ONVL, the only available VNET is a global VNET created when a
fabric is created for the first time. Use tab complete in the CLI to display the VNET and continue the
configuration.
In this example, you configure OSPF for two vRouters with an area of 5. The network has the following configuration:
 VLAN 35 with IP addresses 10.16.35.0/24
 VLAN 45 with IP addresses 10.16.55.0/24
Figure 1: OSPF
1. First, create the vRouter for VNET33, vrouter1.
CLI network-admin@switch > vrouter-create name vrouter1 vnet fabricname-global
Pluribus Networks nvOS Version 2.3.2
163
2. Add vRouter interfaces to the vRouter:
CLI network-admin@switch > vrouter-interface-add vrouter-name vrouter1 ip
10.16.35.1 netmask 24 vlan 35 if data nic-enable
CLI network-admin@switch > vrouter-interface-add vrouter-name vrouter1 ip
10.16.55.1 netmask 24 vlan 55 if data nic-enable
3. Add the subnets, 10.16.35.0/24 and 10.16.45.0/24, to VLAN33 with the area 0:
CLI network-admin@switch > vrouter-ospf-add vrouter-name vrouter1 network
10.16.35.0/24 ospf-area 0
4. Add the second IP address with the area 0.
CLI network-admin@switch > vrouter-ospf-add vrouter-name vrouter1 network
10.16.55.0/24 ospf-area 0
5. Add interfaces for OSPF hello intervals of 30 seconds:
CLI network-admin@switch > vrouter-interface-config-add name vrouter1 nic
eth0.35 ospf-hello-interval 30 ospf-cost 0
CLI network-admin@switch > vrouter-interface-config-add name vrouter1 nic
eth0.55 ospf-hello-interval 30 ospf-cost 0
If you specify 0 as the cost value, the vRouter calculates the OSPF cost automatically based on the bandwidth of the
interface.
When you modify the OSPF hello interval, the ospf-dead-interval is automatically reset to 4 times the hello interval.
6. Display the configuration by using the vrouter-ospf-show command:
CLI network-admin@switch > vrouter-ospf-show layout vertical
vrouter-name:
network:
netmask:
ospf-area:
vrouter-name:
network:
netmask:
ospf-area:
stub-area:
stub-type:
ospf-hello-interval:
metric:
vrouter1
10.16.35.0
24
0
vrouter1
10.16.55.0
24
0
11
stub
30
34
The metric value can reflect the cost of routes advertised as OSPF routes. It may also reflect the cost of routes
advertised with other protocols.
164
Pluribus Networks nvOS Version 2.3.2
Adding Areas and Prefix Lists to OSPF
You can now configure OSPF areas as a stub area, stub-no-summary area, or a not so stubby area (NSSA). Stub areas
see detailed routing information from other areas, but only summary information about networks outside of the AS.
Stub-no-summary areas summarize external routes and routes from other areas. Routers in this type of area only
see routing information local to their area. Not so stubby areas (NSSA) connects to the external network by
introducing a Link State Advertisement (LSA) used within the area to carry external routes originating with boundary
routers connected to this area.
To add a stub area to vRouter, vrouter-ospf, with area 100, use the following command:
CLI network-admin@switch > vrouter-ospf-area-add vrouter-name vrouter-ospf
area 100 stub-type stub
The parameter, stub-type, is a required parameter.
In addition, you can add prefix lists to filter host IP addresses. To add prefix lists to OSPF areas, see Configuring Prefix
Lists for BGP and OSPF.
Pluribus Networks nvOS Version 2.3.2
165
166
Pluribus Networks nvOS Version 2.3.2
Configuring Routing Information Protocol (RIP)
Routing Information Protocol (RIP) is the oldest routing protocol and provides networking information to routers.
Routers need to know what networks are available and how the distance required to reach it.
RIP is a distance vector protocol, and uses hop counts to determine distance and destination. Every 30 seconds, RIP
sends routing information to UDP port 50. If the router is default gateway, it advertises itself by sending 0.0.0.0 with
a metric of 1.
Figure 1:I RIP
1. Create vRouter1 on VNET33:
CLI network-admin@switch > vrouter-create name vrouter1 vnet fabricname-global
You can also specify how RIP routes are distributed using the parameter, rip-redistribute
static|connected|ospf|bgp.
2. Add network 10.16.33.0/24 to vrouter1:
CLI network-admin@switch > vrouter-rip-add vrouter-name vrouter1 network
10.16.33.0/24 metric 2
3. Add network 10.16.35.0/24 to vrouter1:
CLI network-admin@switch > vrouter-rip-add vrouter-name vrouter1 network
10.16.55.0/24 metric 2
4. To view the configuration, use the vrouter-rip-show command. This displays all RIP routes configured using
the vrouter-rip-add command.
To view RIP routes not configured using the vrouter-rip-add command, use the
vrouter-rip-routes-show command.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks nvOS Version 2.3.2
167
168
Pluribus Networks nvOS Version 2.3.2
Configuring Static Routes
vRouters forward packets using either routing information from route tables manually configured or routing
information calculated using dynamic routing algorithms.
Static routes define explicit paths between two vRouters and are not automatically updated. When network changes
occur, you have to reconfigure static routes. However, static routes use less bandwidth than dynamic routes.
Figure 1: Configuring a Static Route
In this example, you configure a static route on vRouter1 for the network, 172.16.10.10/24 with a gateway IP
address, 172.16.20.1:
CLI network-admin@switch > vrouter-static-route-add vrouter-name vrouter1
network 172.16.10.10/24 gateway-ip 172.16.20.1
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks nvOS Version 2.3.2
169
170
Pluribus Networks nvOS Version 2.3.2
Adding IGMP Static Joins to a vRouter
Internet Group Membership Protocol (IGMP) is used to inform vRouters about multicast groups that hosts want to
join on the network, and vRouters use IGMP to verify that a host is interested in listening to a multicast group.
You can add IGMP static group membership to a vRouter in a VNET. When you enable static group membership, data
is forwarded to an interface without the interface receiving membership reports from downstream hosts. This
allows fast switching for multicast traffic.
You must create IGMP static groups before configuring IGMP static joins. To configure IGMP static groups, use the
following command:
CLI network-admin@switch > igmp-static-group-create group-ip 239.4.9.3 vlan 33
ports 5-7
To configure an IGMP static join for group 239.4.9.3, and source IP address 192.0.2.3, use the following command:
CLI network-admin@switch > vrouter-igmp-static-join-add vrouter-name vrouter1
name igmp-vrouter-group group-ip 239.4.9.3 source-ip 192.0.2.3 interface
vrouter33
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks nvOS Version 2.3.2
171
172
Pluribus Networks nvOS Version 2.3.2
Configuring Virtual Router Redundancy Protocol
Virtual Router Redundancy Protocol (VRRP) is an election protocol that enables virtual routing functions for a master
or standby routing infrastructure for a given IP address. A virtual router is defined by a virtual router identifier (VRID)
and a virtual router IP address (VIP). The scope of the virtual routers is restricted to a single VLAN.
VRRP provides information on the state of a virtual router, not the routes processed and exchanged by the router. It
increases the availability and reliability of routing paths by automatic gateway selections on an IP subnetwork.
VRRP provides rapid transition from master to standby and from standby to master. The master router sends
advertisements every second. If the master VRRP advertisements are not received within a window of time, three
(3) seconds, then the standby virtual router becomes the master virtual router and begins performing routing for
the virtual router. If the master router becomes active again, it can become the master again or allow the standby to
continue as the master router. The role depends on the value assigned to VRRP priority.
Configuring VRRP Priority
The Priority is a value used by the VRRP router for master election. The valid priority range for a virtual router is from
1 to 254. 1 is the lowest priority and 254 is the highest priority. The default value for standby routers is 100. Higher
values indicate higher priority for the virtual router.
Configuring the VRRP ID
The Virtual Router Identifier is a configurable value between 1 and 255. There is no default value.
Example Configuration
In this example, you have the following configurations on two switches (SW1 and SW2) on the network:
 VLAN 100 with IP address range 192.168.11.0/24
 VNET with the name vrrp-router and scope fabric
1. On SW1, configure a vRouter:
CLI network-admin@switch > vrouter-create name vrrp-rtr1 vnet vrrp-router
router-type software enable
VRRP is supported on hardware and software routers, but for this example, software is the router type on both
switches.
Informational Note: You can configure up to seven hardware routers for VRRP, and only one VLAN for
VRRP.
2. Add the first vRouter interface:
CLI network-admin@switch > vrouter-interface-add vrouter-name vrrp-rtr1 ip
192.168.11.3 netmask 24 vlan 100 if data
Pluribus Networks nvOS Version 2.3.2
173
3. Use the vrouter-interface-show command to see the name of the interface:
CLI network-admin@switch > vrouter-interface-show format all layout vertical
vrouter-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
alias-on:
exclusive:
nic-config:
nic-state:
vrrp-rtr1
eth0.100
192.168.11.3/24
static
66:0e:94:dd:18:c4
100
0
data
no
enable
up
4. Now create the VRRP interface:
CLI (switch)>vrouter-interface-add vrouter-name vrrp-rtr1 ip 192.168.11.2 netmask 24
vlan 100 if data vrrp-id 10 vrrp-primary eth0.100 vrrp-priority 100
5. Now, create the vRouter and interfaces on SW2:
CLI network-admin@switch > vrouter-create name vrrp-rtr2 vnet vrrp-router
router-type software dedicated-vnet-service
Note that the second vRouter is created as a dedicated VNET service because a VNET supports only one shared
vRouter service.
6. Add the vRouter interface:
CLI network-admin@switch > vrouter-interface-add vrouter-name vrrp-rtr2 ip
192.168.11.4 netmask 24 vlan 100 if data
7. Use the vrouter-interface-show command to see the name of the interface:
CLI network-admin@switch > vrouter-interface-show format all layout vertical
vrouter-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
alias-on:
exclusive:
nic-config:
nic-state:
vrrp-router2
eth2.100
192.168.11.3/24
static
66:0e:94:21:a9:6c
100
0
data
no
enable
up
8. Now create the VRRP interface:
CLI network-admin@switch > vrouter-interface-add vrouter-name vrrp-rtr2 ip
192.168.11.2 netmask 24 vlan 100 if data vrrp-id 10 vrrp-primary eth0.100
vrrp-priority 50
174
Pluribus Networks nvOS Version 2.3.2
9. Display the information about the VRRP setup:
CLI network-admin@switch > vrouter-interface-show format all layout vertical
vrouter-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
alias-on:
exclusive:
nic-config:
nic-state:
vrouter-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
alias-on:
exclusive:
nic-config:
nic-state:
vrrp-id:
vrrp-primary:
vrrp-priority:
vrrp-state:
vrouter-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
alias-on:
exclusive:
nic-config:
nic-state:
vrouter-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
alias-on:
exclusive:
nic-config:
nic-state:
vrrp-id:
vrrp-router1
eth0.100
192.168.11.3/24
static
66:0e:94:dd:18:c4
100
0
data
no
enable
up
vrrp-router1
eth1.100
192.168.11.2/24
static
00:00:5e:00:01:0a
100
0
data
no
enable
up
10
eth1.100
100
master
vrrp-router2
eth3.100
192.168.11.4/24
static
66:0e:94:21:54:07
100
0
data
no
enable
up
vrrp-router2
eth3.100
192.168.11.2/24
static
00:00:5e:00:01:0a
100
0
data
no
enable
down
10
Pluribus Networks nvOS Version 2.3.2
175
vrrp-primary: eth3.100
vrrp-priority: 50
vrrp-state:
slave
When you intentionally disable the VRRP interface, the slave interface becomes the master interface:
vrouter-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
alias-on:
exclusive:
nic-config:
nic-state:
vrrp-id:
vrrp-primary:
vrrp-priority:
vrrp-state:
vrrp-router2
eth3.100
192.168.11.1/24
static
00:00:5e:00:01:0a
100
0
data
no
enable
up
10
eth3.100
50
master
When you re-enable the VRRP interface, it becomes the master again, and the second interface returns to the slave:
vrouter-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
alias-on:
exclusive:
nic-config:
nic-state:
vrrp-id:
vrrp-primary:
vrrp-priority:
vrrp-state: slave
vrrp-router2
eth3.100
192.168.11.2/24
static
00:00:5e:00:01:0a
100
0
data
no
enable
down
10
eth3.100
50
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
176
Pluribus Networks nvOS Version 2.3.2
Configuring Virtual Load Balancing
Virtual load balancing (vLB) uses virtual servers instead of physical servers to balance traffic across the network.
Each virtual server points to a cluster of services that reside on one or more physical hosts.
VLB uses the following transactions:
1. The client attempts to connect to the service on the load balancer.
2. The load balancer accepts the connection and then decides which host receives the connection. The port and destination IP address are changed to match the service of the selected host.
3. The host accepts the connection and responds to the original source, the client, through the default route which
is the load balancer.
4. The load balancer intercepts the return packet from the host and changes the source IP and port to match the virtual server IP and port, and forwards the packet back to the client.
5. The client receives the return packet and continues the process.
VLB uses four different algorithms to control and distribute traffic as well as load distribution and server selection.
 roundrobin — In a round-robin algorithm, the load balancer assigns requests to a list of servers on a rotating
basis. Once a server is assigned a request, the server moves to the bottom of the list.
 hash-ip — In the source IP hash method, the load balancer selects a server based on the hash value of the
source IP address of the incoming request.
 hash-ip-port — In the source virtual IP, port hash method, the load balancer selects a server based on the
hash value of the source IP address, and the source port of the incoming request.
 hash-ip-vip — In the source IP, VIP hash method, the load balancer selects a server based on the hash value
of the source IP address, and the destination IP address of the incoming requests.
If you already have servers that you want to use for VLB, you can following the instructions below. If you want to
install Ubuntu servers as virtual machines on the switch, see Configuring Virtual Load Balancing with Ubuntu 11.04
Servers and nvOS.
If you are configuring VLB as a dedicated service on a VNET or you have not defined network interfaces for the VNET,
use the vlb-interface-add command to create the vNICs.
CLI network-admin@switch > vlb-create name vlb-vnet1 vnet vnet1
dedicated-vnet-service
You need two interfaces to configure VLB: one for the external address and one for the internal address. To create
the interfaces, use the following commands:
CLI network-admin@switch > vlb-interface-add vlb-name vlb-vnet1 ip
192.168.100.27 netmask 24 assignment none vlan 57 if data
CLI network-admin@switch > vlb-interface-add vlb-name vlb-vnet1 ip 10.10.10.113
netmask 24 assignment none vlan 58 if data
Pluribus Networks nvOS Version 2.3.2
177
Display the configuration information:
CLI network-admin@switch > vlb-interface-show vlb-name vnet1-vlb layout
vertical
vlb-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
vlb-name:
nic:
ip:
assignment:
mac:
vlan:
vxlan:
if:
vnet1-vlb
vnet1.mgr.eth0
10.10.10.113/24
static
66:0e:94:4b:b8:0c
123
0
data
vnet1-vlb
vnet1.mgr.eth1
192.168.100.27/24
static
66:0e:94:4b:9d:cc
124
0
data
Create a VLB to balance TCP port 80 (HTTP) requests in full NAT mode between the external and internal interfaces.
Full NAT mode sends all traffic to and from the servers and route through the load balancer.
CLI network-admin@switch > vlb-group-add vlb-name vnet1-vlb name vnet1-vlb-http
topology full-nat proto tcp start-port 80 ext-interface vnet.mgr.eth0
int-interface vnet1.mgr.eth1
When you create a vLB group, you can also add the following parameters:
 vip — the destination IP address for incoming requests
 proxy-src-ip — the proxy host source IP address
 proxy-src-netmask — the proxy host source netmask
 start-port — the starting port of the vLB group
 end-port — the ending port of the vLB group
 healthcheck — the name of a healthcheck configuration
CLI network-admin@switch > vlb-group-show layout vertical
vlb-name:
vnet1-vlb
name:
vnet1-vlb-http
topology:
full-nat
proto:
tcp
ext-interface:
vnet1.mgr.eth0
int-interface:
vnet1.mgr.eth1
start-port:
80
end-port:
80
group-enable: group-enable
178
Pluribus Networks nvOS Version 2.3.2
Configure the VLB service to load balance incoming requests on group vnet-vlb-http to a pod of five Web servers:
CLI network-admin@switch > vlb-server-add vlb-name vnet1-vlb ip 192.168.18.3
group vnet1-vlb-http
CLI network-admin@switch > vlb-server-add vlb-name vnet1-vlb ip 192.168.18.4
group vnet1-vlb-http
CLI network-admin@switch > vlb-server-add vlb-name vnet1-vlb ip 192.168.18.5
group vnet1-vlb-http
CLI network-admin@switch > vlb-server-add vlb-name vnet1-vlb ip 192.168.18.6
group vnet1-vlb-http
CLI network-admin@switch > vlb-server-add vlb-name vnet1-vlb ip 192.168.18.7
group vnet1-vlb-http
Display the server information:
CLI network-admin@switch > vlb-server-show
vlb-name
-------vnet1-vlb
vnet1-vlb
vnet1-vlb
vnet1-vlb
vnet1-vlb
group
----vnet1-vlb-http
vnet1-vlb-http
vnet1-vlb-http
vnet1-vlb-http
vnet1-vlb-http
ip
-192.168.18.3
192.168.18.4
192.168.18.5
192.168.18.6
192.168.18.7
server-enable
------------server-enable
server-enable
server-enable
server-enable
server-enable
id
-_vnet1-vlb-http.0
_vnet1-vlb-http.1
_vnet1-vlb-http.2
_vnet1-vlb-http.3
_vnet1-vlb-http.4
CLI network-admin@switch > vlb-show
name
type scope vnet
vnet-service state
gateway
------------- ---- ------ --------- ------------ ------- --------vlb-web
vlb fabric vlb-web
shared
enabled 10.12.1.1
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Monitoring the Health of VLB
You can configure health monitoring for your VLBs so that network traffic can determine if the server is available
before attempting to send connections to it. Basic monitoring is simply pinging the host and determining if the host
is active. Or you can send service pings ranging from simple TCP connections or using scripting interaction.
To create a VLB health monitor for vlb-vnet1 using ping, timeout 10 seconds, attempts 5, and 120 seconds interval
between checks:
CLI network-admin@switch > vlb-health-config-add vlb-name vlb-vnet1 name
vlb-health type ping timeout 10 attempts 5 interval 120
To remove the VLB health configuration, use the vlb-health-config-remove command.
To display the VLB health configuration, use the vlb-health-config-show command.
Pluribus Networks nvOS Version 2.3.2
179
To display the status of the VLB health configuration, use the vlb-health-status-show command:
CLI network-admin@switch > vlb-health-status-show layout vertical
vlb-name:
name:
id:
status:
fail:
last:
next:
rtt:
vlb-vnet1
vlb-health
_vlbgroup
alive
0
13:47:16
13:47:30
1836
Viewing vLB Group Statistics
You can view vLB Group statistics using the vlb-group-stats-show command:
CLI network-admin@switch > CLI vlb-group-stats-show format all layout vertical
switch:
name:
group:
processed-bytes:
processed-pkts:
dropped-bytes:
dropped-pkts:
switch:
name:
group:
processed-bytes:
processed-pkts:
dropped-bytes:
dropped-pkts:
switch:
name:
group:
processed-bytes:
processed-pkts:
dropped-bytes:
dropped-pkts:
180
pubdev01
vlb-1
vlb-group
0
0
0
0
pubdev03
vlb-1
vlb-group
0
0
0
0
pubdev02
vlb-1
vlb-group
0
0
0
0
Pluribus Networks nvOS Version 2.3.2
Configuring Virtual Load Balancing with Ubuntu 11.04 Servers and nvOS
In this example, you configure the following features:
 VNET
 IP Pool
 DHCP Server
 Ubuntu 11.04 Servers (2)
 Apache Services
 VLB
 VLB Health
Configuring the VLB VNET
1. Using the name, vlb-web, scope fabric, and vlans 200, configure the VNET:
CLI network-admin@switch > vnet-create name vlb-web scope fabric vlans 200
2. Create the IP pool, web-ip-pool, with the IP address range of 172.16.23.0, netmask 24:
CLI network-admin@switch > ip-pool-create name web-ip-pool vnet vlb-web
start-ip 172.16.23.0 end-ip 172.16.23.254 netmask 24 vlan 200
Pluribus Networks nvOS Version 2.3.2
181
3. Create the DHCP server, web-dhcp, and add the gateway:
CLI network-admin@switch > dhcp-create name web-dhcp vnet vlb-web
initial-ip-pool web-ip-pool
CLI network-admin@switch > dhcp-pool-modify dhcp-name web-dhcp dhcp-pool-name
web-ip-pool gateway-ip 172.16.23.1
4. Add connectivity to your network. You’ll need this to download Apache2.
Informational Note: This step varies depending on the setup of your corporate network. In this example, the
corporate network is a 10.0.0.0/16 network.
CLI network-admin@switch > vnet-manager-interface-add vnet-manager-name
vlb-web-mgr ip 10.0.0.0 netmask 16 if mgmt vlan 0
CLI network-admin@switch > vnet-manager-modify name vlb-web-mgr gateway
10.0.0.1 enable
5. Create the Ubuntu servers using KVMs on the switch:
Informational Note: There is no requirement that the Ubuntu servers reside on the same switch. For this
purpose, the servers are on the same switch.
CLI network-admin@switch > netvisor-kvm-create name vlb-web-svr1 vnet vlb-web
iso-label ubuntu-11.04-amd64 memory 4g cpus 2 hda-size 20g storage-pool
pool-disk4
Netvisor vm created. Please use interface-add to add interfaces and then
start to boot
CLI network-admin@switch > netvisor-kvm-interface-add netvisor-kvm-name
vlb-web-svr1 if mgmt vlan 0
CLI network-admin@switch > netvisor-kvm-interface-add netvisor-kvm-name
vlb-web-svr1 if data vlan 200
CLI network-admin@switch > netvisor-kvm-start name vlb-web-svr1
VM running. From outside switch, connect to vnc port :2.
Ex: vncviewer 10.9.11.147:2
182
Pluribus Networks nvOS Version 2.3.2
The Ubuntu server installation takes 20-30 minutes. In the meantime, configure the KVM for vlb-web-svr2:
CLI network-admin@switch > netvisor-kvm-create name vlb-web-svr2 vnet vlb-web
iso-label ubuntu-11.04-amd64 memory 4g cpus 2 hda-size 20g storage-pool
pool-disk4
Netvisor vm created. Please use interface-add to add interfaces and then
start to boot
CLI network-admin@switch > netvisor-kvm-interface-add netvisor-kvm-name
vlb-web-svr2 if mgmt vlan 0
CLI network-admin@switch > netvisor-kvm-interface-add netvisor-kvm-name
vlb-web-svr2 if data vlan 200
CLI network-admin@switch > netvisor-kvm-start name vlb-web-svr2
VM running. From outside switch, connect to vnc port :2.
Ex: vncviewer 10.9.11.147:3
The Ubuntu server installation takes 20-30 minutes. In the meantime, configure the KVM for vlb-web-svr3:
CLI network-admin@switch > netvisor-kvm-create name vlb-web-svr3 vnet vlb-web
iso-label ubuntu-11.04-amd64 memory 4g cpus 2 hda-size 20g storage-pool
pool-disk4
Netvisor vm created. Please use interface-add to add interfaces and then start to boot
CLI network-admin@switch > netvisor-kvm-interface-add netvisor-kvm-name
vlb-web-svr3 if mgmt vlan 0
CLI network-admin@switch > netvisor-kvm-interface-add netvisor-kvm-name
vlb-web-svr3 if data vlan 200
CLI network-admin@switch > netvisor-kvm-start name vlb-web-svr2
VM running. From outside switch, connect to vnc port :3.
Ex: vncviewer 10.9.11.147:3
6. Next, you install Apache2 on each Ubuntu server by executing the following commands on each one. Open your
VNC application and connect to an Ubuntu server:
sudo apt-get install apache2
sudo vi/var/www/index.html
7. Create the virtual load balancer:
CLI network-admin@switch > vlb-create name vlb-web vnet vlb-web
shared-vnet-service enable
CLI network-admin@switch > vlb-show
name
type scope vnet
vnet-service state
gateway
----------- ---- ------ ----------- ------------ ------- ------vlb-web
vlb fabric vlb-web
shared
enabled ::
Pluribus Networks nvOS Version 2.3.2
183
8. Create the health check for the VLB service:
CLI network-admin@switch > vlb-health-config-add vlb-name vlb-web switch
pleiades24 name web-http type http timeout 3 attempt 3 interval 11
This configuration means that the health check is performed every 11 seconds, and it verifies the service 3 times and
times out after 3 seconds.
9. Create the virtual load balancing group. Note that the group name must be less than 14 characters:
CLI network-admin@switch > vlb-group-add vlb-name vlb-web name web-svc-grp
proto tcp algorithm roundrobin vip 172.16.23.20 topology full-nat proxy-src-ip
172.16.23.20 proxy-src-netmask 24 start-port 80 healthcheck web-http
group-enable
10. Add the Ubuntu Apache servers to the VLB group:
CLI network-admin@switch > vlb-server-add vlb-name vlb-web ip 172.16.23.3 port
80 group vlb-web-group
CLI network-admin@switch > vlb-server-add vlb-name vlb-web ip 172.16.23.4 port
80 group vlb-web-group
CLI network-admin@switch > vlb-server-add vlb-name vlb-web ip 172.16.23.5 port
80 group vlb-web-group
11. Display the configuration:
CLI network-admin@switch > vlb-show
12. Display the VLB servers:
CLI network-admin@switch > vlb-server-show
vlb-name
group
----------- ----------vlb-web
web-svc-grp
vlb-web
web-svc-grp
vlb-web
web-svc-grp
13. Display the VLB group:
ip
----------172.16.23.2
172.16.23.3
172.16.23.4
port server-enable id
---- ------------- -------------80
server-enable _web-svc-grp.0
80 server-enable _web-svc-grp.1
80 server-enable _web-svc-grp.2
CLI network-admin@switch > vlb-group-show layout vertical
vlb-name:
name:
topology:
proto:
algorithm:
vip:
proxy-src-ip:
start-port:
end-port:
group-enable:
healthcheck:
184
vlb-web
web-svc-grp
full-nat
tcp
roundrobin
172.16.23.7
172.16.23.7/24
80
80
group-enable
http-service
Pluribus Networks nvOS Version 2.3.2
14. Display the VLB health status:
vlb-health-status-show layout vertical
switch:
mitch-aquila2
vlb-name: vlb-web
name:
http-service
id:
_web-svc-grp.0
status:
alive
fail:
0
last:
09:53:01
next:
09:53:17
rtt:
507
switch:
mitch-aquila2
vlb-name: vlb-web
name:
http-service
id:
_web-svc-grp.1
status:
alive
fail:
0
last:
09:53:14
next:
09:53:28
rtt:
572
switch:
mitch-aquila2
vlb-name: vlb-web
name:
http-service
id:
_web-svc-grp.2
status:
alive
fail:
0
last:
09:53:14
next:
09:53:28
rtt:
578
15. Stop the Apache2 service on one of the Ubuntu servers by connecting with VNC and executing the command:
sudo etc/init.d/apache2 stop
Pluribus Networks nvOS Version 2.3.2
185
16. Display the VLB health status again to verify that the server is in a failed state:
CLI network-admin@switch > vlb-health-status-show
CLI (network-admin@mitch-aquila2) > vlb-health-status-show layout vertical
switch:
mitch-aquila2
vlb-name: vlb-web
name:
http-service
id:
_web-svc-grp.0
status:
alive
fail:
0
last:
09:54:42
next:
09:54:57
rtt:
568
switch:
mitch-aquila2
After stopping the Web service on
vlb-name: vlb-web
server 1, the status changes to
name:
http-service
dead.
id:
_web-svc-grp.1
status:
dead
fail:
3
last:
09:54:42
next:
09:54:57
rtt:
565
switch:
mitch-aquila2
vlb-name: vlb-web
name:
http-service
id:
_web-svc-grp.2
status:
alive
fail:
0
last:
09:54:42
next:
09:54:57
rtt:
572
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Adding Virtual Router Redundancy Protocol to VLB Interfaces
You can add VRRP to the VLB configuration so that if one interface becomes unavailable, then the second interface
becomes the virtual router. Add interfaces to the VLB configuration with VRRP parameters. To configure Web server
1 as the master, use the following commands:
Informational Note: You must use the same VRRP ID for both interfaces. Otherwise, the configuration is
invalid. You must also create a VRRP priority with a higher value for the primary interface and a lower
VRRP priority for the secondary interface.
CLI network-admin@switch > vlb-interface-add vlb-name vlb-web if data vlan 200
CLI network-admin@switch > vlb-interface-modify vlb-name vlb-web-svr1 nic
eth1.200 vrrp-id 10 vrrp-primary vlb-web-svr1 vrrp-priority 100
186
Pluribus Networks nvOS Version 2.3.2
To add Web server 2 as the secondary virtual router, use the following command:
CLI network-admin@switch > vlb-interface-add vlb-name vlb-web if data vlan 200
CLI network-admin@switch > vlb-interface-modify vlb-name vlb-web-svr2 nic
eth2.200 vrrp-id 10 vrrp-primary vlb-web-svr1 vrrp-priority 50
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks nvOS Version 2.3.2
187
188
Pluribus Networks nvOS Version 2.3.2
Configuring Roles and Users
Role-Based Access Control (RBAC) is a secure method of restricting access to authorized users. This method enables
the network administrator to add users and assign each user to specific roles. Each role has specific permissions and
allows users to perform various actions based on the scope of their role.
In this context, users are personnel that can log into the switch, and perform certain functions.
A role defines the level of access for a user account. By assigning roles to users, you can allow multiple users to
complete their tasks. RBAC limits risk by ensuring that users do not have access beyond their training or level of
control.
nvOS allows you to create roles and assign them to users. You can create the following types of roles:
 Scope — A role can apply to the scope of local, or fabric.
 Access — You allow read-only access or read-write access.
 Configuration — A role can apply to the running configuration or not.
Once you create a user with a scope of local or fabric, you cannot modify the user scope. If you decide that your user
needs local scope rather than fabric scope, you must delete the user and create a new one.
There are three types of roles configured for user access:
 network-admin — this is a super user role and can perform all functions on the switch.
 read-only-network-admin — this is a read only role and the user can only execute show commands
from the CLI.
 fabric-admin — this role can perform fabric-wide functions only.
Configuring Custom Roles
You can create custom roles in addition to the preconfigured ones in nvOS. When you create a role, you configure
the following parameters:
 name — create a name for the role
 scope — specify fabric or local. Once you’ve configured the role as local or fabric, you can’t modify it. To
change the scope, you must delete the role and create a new one.
 access — specify the type of access for the user. You can specify any of the following types of access:
• read-write — the role can display information and make changes to the configuration. You can modify
this role to read-only if you decide that the role can only use show commands at the CLI.
• running-config — the role has access to the running configuration on the switch.
• no-running-config — the role cannot access the running configuration on the switch.
For example, create the role, local-admin, with scope local, read-write access to the running configuration:
CLI network-admin@switch > role-create name fabric-admin scope local access
read-write running-config
To modify the role parameter, access to read-only, use the following command:
CLI network-admin@switch > user-role-modify name fabric-admin scope fabric
access read-only
When you modify the role, you can also specify to remove the role from users with the delete-from-users
parameter.
Pluribus Networks nvOS Version 2.3.2
189
To delete the role, local-admin, use the user-role-delete command:
CLI network-admin@switch > user-role-delete name fabric-admin
To display the role configuration, use the role-show command.
CLI network-admin@switch > role-show
role-show format all layout vertical
id:
6000021:402
name:
web-svr-admin
scope:
fabric
access:
read-write
running-config:
deny
id:
6000021:404
name:
test-vnet-admin
scope:
fabric
access:
read-write
running-config:
deny
id:
6000021:405
name:
test-admin
scope:
fabric
access:
read-write
running-config:
deny
id:
6000021:406
name:
vlan-test-admin
scope:
fabric
access:
read-write
running-config:
deny
switch:
pleiades24
id:
0:0
name:
network-admin
scope:
local
access:
read-write
running-config:
permit
switch:
pleiades24
id:
0:1
name:
read-only-network-admin
scope:
local
access:
read-only
running-config:
deny
190
Pluribus Networks nvOS Version 2.3.2
This user has
read-write access but
not to the running
configuration.
Creating and Managing Users
You can create users and apply roles to them to manage access to the switch or network. To create a user, jdoe,
scope local, password p1zz@, and initial role, local-admin, use the following syntax:
CLI network-admin@switch > user-create name jdoe scope local password p1zz@
initial-role local-admin
password:
Confirm password:
Informational Note: Once you configure the scope for a user, you cannot modify it. To change the
scope, delete the user, and create a new one with the intended scope.
To modify the initial role from local-admin to network-admin, use the following command:
CLI network-admin@switch > user-modify name jdoe initial-role network-admin
To delete the user, use the user-delete command.
To add roles to a user, jdoe, role name fabric-admin, use the following syntax:
CLI network-admin@switch > user-role-add name jdoe role fabric-admin
You can assign multiple roles to a user. For instance, if jdoe is a fabric-admin, and you also want to assign the role,
local-admin, use the following command:
CLI network-admin@switch > user-role-add user-name jdoe role local-admin
CLI (network-admin@mitch-aquila2) > user-role-show
switch
user-name
role
------------- ----------------- ----------------------network-admin
network-admin
vlb-web-svr-admin vlb-web-svr-admin
jdoe now has two
test-admin
roles assigned. test-admin
test-admin
test-admin-admin
vlan-test-admin
vlan-test-admin
jdoe
network-admin
jdoe
local-admin
ops-test1-admin
fabric-admin
pleiades01
java-api-admin
java-api-admin
To remove a role from the user, jdoe, use the following command:
CLI network-admin@switch > user-role-remove name jdoe role fabric-admin
Pluribus Networks nvOS Version 2.3.2
191
To display user roles, use the user-role-show command.
CLI (network-admin@pleiades24)>
switch
user-name
------------- ----------------network-admin
vlb-web-svr-admin
test-admin
test-admin
vlan-test-admin
laurap
ops-test1-admin
pleiades01
java-api-admin
user-role-show
role
----------------------network-admin
vlb-web-svr-admin
test-admin
test-admin-admin
vlan-test-admin
read-only-network-admin
fabric-admin
java-api-admin
To display information about all users configured in nvOS, use the user-show command:
CLI network-admin@switch > user-show
name
network-admin
ops-mgmt-admin
ext-50-admin
www-51-admin
jdoe
scope
fabric
fabric
fabric
fabric
fabric
uid
39999
40000
40001
40002
40003
The User ID (UID) is assigned by nvOS and is not configurable. You need the UID to configure user passwords for
TACACS+ authentication.
To configure user, jdoe, on a TACACS+ server, use the following command:
CLI network-admin@switch > user-set-password name jdoe scope fabric uid 4003
server aaa-tacacs
See Configuring TACACS+.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
192
Pluribus Networks nvOS Version 2.3.2
Configuring TACACS+
About TACACS+
Terminal Access Controller Access Control System (TACACS+) is an Authentication, Authorization, and Accounting
(AAA) protocol that was introduced in the early 2000s. The main goal of TACACS+ is to provide a centralized
database to use for authentication. It uses a client server approach by which the client queries a server and the
server replies with a pass or fail for authentication. The communication between the client and server uses TCP as
the connection protocol, and requires a secret key.
nvOS can be configured to use external TACACS+ servers for authentication, authorization, and accounting. You can
configure any number of TACACS+ servers, and each server may be configured to handle any combination of
authentication, session authorization, command authorization, session accounting, and command accounting.
It is important to note that the default “network-admin” account is exempt from all TACACS+ integration, as a
fail-safe account for sites without TACACS+ and to allow access to Pluribus Networks facilities if TACACS+ is
unavailable or unreachable.
TACACS+ is configured using the aaa-tacacs-create command, and using options to specify the IP address,
port, password, priority, authentication methods, and accounting options. Once set up, a user can login to the
switch and get CLI access using an account configured on the specified TACACS+ server.
The TACACS+ server determines what role the user has by returning a “role” attribute. The roles include
“networkadmin” for full access and "read-only-network-admin" users who can only run show commandsPAP, CHAP,
and MS-CHAP authentication protocols are supported.
Figure 1 illustrates a simple TACACS+ implementation.
Figure 1: TACACS+ AAA with a nvOS switch
Pluribus Networks nvOS Version 2.3.2
193
Configuring TACACS+
Using Figure 1 as an example, you can configure TACACS+ access to the switch with the following command:
CLI network-admin@switch > aaa-tacacs-create name tacacs-server scope fabric
port 34 m0nk3y6 priority 3 authen authen-method ms-chap sess-acct
This command configures basic access from a user on the network to the switch. You can add the following optional
parameters to the configuration:
 Session accounting
 Command accounting
 Session Authorization
 Command Authorization
To add optional parameters or to modify the current configuration, use the aaa-tacacs-modify command.
To display the status of the TACACS server, use the aaa-tacacs-status command.
To delete the configuration, use the aaa-tacacs-delete command.
194
Pluribus Networks nvOS Version 2.3.2
Creating and Implementing Access Control Lists (ACLs)
Access Control Lists (ACLs) allow you to configure basic traffic filtering for IP addresses and MAC addresses. The ACL
controls if routed packets are forwarded or blocked on the network. The packet is examined by the switch and then
determines if the packet is forwarded or dropped based on the criteria configured in the ACLs. ONVL supports Layer
2 (MAC) or Layer 3 (IP) ACLs.
ACL criteria can be based on source or destination addresses or the protocol type. nvOS supports UDP, TCP, IGMP,
and IP protocols.
You can use ACLs to restrict contents of routing updates or provide traffic flow control. ACLs can allow one host to
access part of your network and prevent another host from accessing the same area. You can also use ACLs to decide
what types of traffic are forwarded or blocked.
If you need more background on ACLs and using them on your network, refer to the many networking resources
available.
Using a Deny IP ACL to Block Network Traffic
In this example, a network is shown with a Finance server on one part of the network, and an Engineering server on
another part. You want to block the Engineering server from the Finance server in order to protect company
sensitive information. See Configuring an Internal Deny ACL to review the configuration sample.
Figure 1: Network Example - IP ACL for Internal Servers
Or you may discover that an external source is attempting to access your network, and ping your servers for IP
addresses. You can use an ACL to block the specific source using an IP ACL.
Pluribus Networks nvOS Version 2.3.2
195
Figure 2:IP ACL Blocking External Access
See Configuring an External Deny ACL to review the configuration example.
Using IP ACLs to Allow Network Traffic
In the same manner, you can allow specific traffic to a destination such as the external server in Figure 2 IP ACL
Blocking External Access. To allow HTTP traffic to 209.225.113.24, see Configuring an External Allow IP ACL to review
the configuration example.
196
Pluribus Networks nvOS Version 2.3.2
Figure 3:IP ACL Allowing HTTP Traffic
Pluribus Networks nvOS Version 2.3.2
197
Using MAC ACLs to Deny Network Traffic
You can create ACLs based on MAC addresses to deny network traffic from a specific source. MAC addresses are
Layer 2 protocols and most often assigned by the hardware manufacturer. Figure 4 MAC ACL Blocking Access shows
an example of a MAC address and Ethernet type that you want to block from the network.
Figure 4: MAC ACL Blocking Access
See Configuring a MAC ACL to Deny Network Traffic to review the example configuration.
Using MAC ACLs to Allow Network Traffic
So now that you’ve blocked the MAC address, let’s reverse the scenario and allow IPv4 network traffic from the MAC
address to the network.
198
Pluribus Networks nvOS Version 2.3.2
Figure 5:MAC ACL Allowing Access
See Configuring a MAC ACL to Allow Network Traffic to review the example configuration.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks nvOS Version 2.3.2
199
200
Pluribus Networks nvOS Version 2.3.2
Configuring IP ACLs
From Figure 1 Network Example - IP ACL for Internal Servers, the following information is available:
 Source IP address
 Source netmask
 Destination IP address
 Destination netmask
 Type of protocol to deny - IP
 Ports
 VLAN
Configuring an Internal Deny ACL
Configure the ACL for denying traffic from the Engineering server to the HR server and name the ACL,
deny-hr:
CLI network-admin@switch > acl-ip-create name deny-hr action deny scope
local src-ip 192.168.10.2 src-ip-mask 24 dst-ip 192.168.200.3
dst-ip-netmask 24 proto ip src-port 55 dst-port 33 vlan 1505
To review the configuration, use the acl-ip-show command:
CLI network-admin@switch > acl-ip-show name deny-hr layout vertical
name:
deny-ip
id:
b00011:20
action:
deny
proto:
ip
src-ip:
192.168.10.2/24
src-port:
55
dst-ip:
192.168.200.3/24
dst-port:
33
vlan:
1505
scope:
local
port:
0
Now, when you attempt to access the Finance server from the Engineering server, the packets are
dropped.
Configuring an External Deny ACL
From Figure 2 IP ACL Blocking External Access, the following information is available:
 IP Address
 Port Number
To configure an ACL to deny traffic from the external server, use the acl-ip-create command to
create an ACL named deny-external:
CLI network-admin@switch > >acl-ip-create name deny-external scope
fabric src-ip 209.255.113.24/28
Pluribus Networks ONVL Version 2.3
201
To review the configuration, use the acl-ip-show command:
CLI network-admin@switch > acl-ip-show name deny-external layout
vertical
name:
id:
action:
proto:
src-ip:
src-port:
dst-ip:
dst-port:
vlan:
scope:
port:
deny-external
b000022:20
deny
tcp
209.225.113.24/28
0
::/0
0
0
fabric
0
Configuring an External Allow IP ACL
To allow HTTP traffic to the external server, 209.225.113.24 with a netmask of 255.255.255.240
and a scope of fabric, you can create an IP ACL called allow-http using the following syntax:
CLI network-admin@switch > acl-ip-create name allow-http permit scope
fabric src-ip 0.0.0.0. src-mask 255.255.255.255 dst-ip 209.225.113.24
dst-ip-mask 255.255.255.240 protocol tcp dst-port 57
To review the configuration, use the acl-ip-show command:
CLI network-admin@switch > >acl-ip-show name allow-http layout vertical
name:
id:
action:
proto:
src-ip:
src-port:
dst-ip:
dst-port:
vlan:
scope:
port:
allow-http
b000025:20
allow
tcp
0.0.0.0/255.255.255.255
0
209.225.113.24/28
57
0
fabric
0
To delete the ACL configuration, use the acl-ip-delete command.
To modify the ACL configuration, use the acl-ip-modify command.
Configuring a MAC ACL to Deny Network Traffic
To deny IPv4 network traffic from MAC address, 01:80:c2:00:00:0X, for the scope fabric, create
the MAC ACL, deny-MAC, using the following syntax:
CLI network-admin@switch > acl-mac-create name deny-mac action deny
src-mac 01:80:c2:00:00:0X ether-type ipv4 scope fabric
Pluribus Networks ONVL Version 2.3
202
To review the configuration, use the acl-mac-show command:
CLI network-admin@switch > acl-mac-show name deny-mac layout vertical
name:
id:
action:
src-mac:
dst-mac:
dst-mac-mask:
ether-type:
vlan:
scope:
port:
deny-mac
b000015:12
deny
01:80:c2:00:00:0X
00:00:00:00:00:00
aa:aa:aa:aa:aa:aa
ipv4
0
fabric
0
Configuring a MAC ACL to Allow Network Traffic
To allow IPv4 network traffic from MAC address, 01:80:c2:00:00:0X, for the scope fabric,
create the MAC ACL, allow-MAC, using the following syntax:
CLI network-admin@switch > acl-mac-create name allow-mac action permit
src-mac 01:80:c2:00:00:0X ether-type ipv4 scope fabric
To review the configuration, use the acl-mac-show command:
CLI network-admin@switch > acl-mac-show name deny-mac layout vertical
name:
id:
action:
src-mac:
dst-mac:
dst-mac-mask:
ether-type:
vlan:
scope:
port:
deny-mac
b000015:12
deny
01:80:c2:00:00:0X
00:00:00:00:00:00
aa:aa:aa:aa:aa:aa
ipv4
0
fabric
0
To delete the ACL configuration, use the acl-mac-delete command.
To modify the ACL configuration, use the acl-mac-modify command.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks ONVL Version 2.3
203
Pluribus Networks ONVL Version 2.3
204
Configuring vFlow for Analytics
A vFlow can be used to capture packets for analysis, and you can determine if the vFlow captures packets across the
fabric or on a single switch. Packets are captured by forwarding them from the data plane of the switch to the
control plane.
A vFlow that directs packets to the switch CPU can be configured to save packets to a file by enabling the log-packets
parameter. The file is written using a libcap compatible format so that programs like TCPdump and Wireshark can be
used to read the file. The file is exported to clients using NFS or SFTP.
Packet capture data is available with switch or fabric scope. The pcap files are stored over NFS in the following
locations:
/net/<ServerSw_Name>/nvOS/global/flow/<Flow_Name>/switch/<Switch_Name>/pcap
/net/<ServerSw_Name>/nvOS/vnet/<VNET_Name>/flow/<Flow_Name>/
switch/<Switch_Name>/pcap
/net/<ServerSw_Name>/nvOS/global/flow/<Flow_Name>/fabric/pcap
/net/<ServerSw_Name>/nvOS/vnet/<VNET_Name>/flow/<Flow_Name>/
fabric/pcap
Snooping only works if you use the parameters, copy-to-cpu or to-cpu. The copy-to-cpu parameter
ensures that the data plane forwards the packets and sends a copy to the CPU. Use this parameter if you want traffic
to flow through the switch. The to-cpu parameter doesn’t forward packets and interrupts traffic on the switch. To
snoop all application flow packets of protocol type TCP, enter the following CLI commands at the prompt:
CLI network-admin@switch > vflow-create name snoop_all scope local proto tcp
action copy-to-cpu
Then use the following command to display the output:
CLI network-admin@switch > vflow-snoop
switch: pleiades24, flow: snoop_all, port: 65, size: 66, time:
20:07:15.03867188
smac: 64:0e:94:28:00:fa, dmac: 64:0e:94:2c:00:7a, etype: ip
sip: 192.168.2.51, dip: 192.168.2.31, proto: tcp
sport: 42120, dport: 33399
switch: pleiades24, flow: snoop_all, port: 65, size: 184, time:
20:07:15.03882961
smac: 64:0e:94:28:00:fa, dmac: 64:0e:94:2c:00:7a, etype: ip
sip: 192.168.2.51, dip: 192.168.2.31, proto: tcp
sport: 42120, dport: 33399
switch: pleiades24, flow: snoop_all, port: 43, size: 66, time:
20:07:15.03893740
smac: 64:0e:94:2c:00:7a, dmac: 64:0e:94:28:00:fa, etype: ip
sip: 192.168.2.31, dip: 192.168.2.51, proto: tcp
sport: 33399, dport: 42120
Pluribus Networks nvOS Version 2.3.2
205
To restrict the flows captured to TCP port 22, SSH traffic, create the following vFlow:
CLI network-admin@switch > vflow-create name snoop_ssh scope local action
copy-to-cpu src-port 22 proto tcp vflow-add-filter name snoop_ssh
Then use the vflow-snoop command to display the results:
switch: pleiades24, flow: snoop_ssh, port: 41, size: 230, time:
10:56:57.05785917 src-mac: 00:15:17:ea:f8:70, dst-mac:
f4:6d:04:0e:77:60, etype: ip src-ip: 10.9.11.18, dst-ip: 10.9.10.65,
proto: tcp src-port: 22, dst-port: 62356
switch: pleiades24, flow: snoop_ssh, port: 41, size: 118, time:
10:56:57.05922560 src-mac: 00:15:17:ea:f8:70, dst-mac:
f4:6d:04:0e:77:60, etype: ip src-ip: 10.9.11.18, dst-ip: 10.9.10.65,
proto: tcp src-port: 22, dst-port: 62356
The optional parameter vflow-add-filter restricts the output of the vflow-snoop command to the
packets matching the snoop_ssh flow definition.
To capture traffic packets for a flow across the entire fabric, you create a flow with the scope of fabric. To copy the
packets to a pcap file, add the log-packets option:
CLI network-admin@switch > vflow-create name fab_snoop_all scope fabric action
copy-to-cpu port 22 log-packets yes
If you enable log-packets, the separate pcap files for all switches are available on any switch. In addition a
consolidated pcap file is available that aggregates the packets from all switches in the entire fabric.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Analyzing Live Traffic Using Wireshark
Wireshark is a well known network protocol analyzer and one of many applications used for network protocol
analysis. Wireshark can interactively browse packet data from a live network or from a previously save pcap file.
Informational Note:You can download Wireshark from http://www.wireshark.org
To use Wireshark to decode a previously saved packet flow capture file, export the file from the switch and analyze it
with Wireshark.
Informational Note:
The path to a Pluribus Networks switch pcap file has the format:
/net/<ServerSw_Name>/nvOS/global/flow/<Flow_Name>/<Switch_Name>/pcap
206
Pluribus Networks nvOS Version 2.3.2
To use Wireshark to interactively analyze packets in real time, you need to capture a packet traffic flow, either on a
specific switch or across the entire fabric using the scope option. Include the log-packets option to send packets to
the associated pcap files, for example
CLI network-admin@switch > vflow-snoop scope fabric src-ip 112.168.3.105 action
copy-to-cpu log-packets
Next, create a fifo on the host running Wireshark.
mkfifo /tmp/pcap
Start Wireshark, and select Options from the Capture menu.
Enter the fifo path that you created in the Interface field: /tmp/pcap
Use tail to copy the pcap file to the FIFO:
tail +0f \
/net/ServerSw_Name/nvOS/global/flow/Flow_Name/switch/Switch_Name/
pcap/tmp/pcap
You need to substitute ServerSw_Name, Flow_Name and Switch_Name to match your environment. Live capture
continues until the packet capture file is rotated. By default, the maximum packet capture file size is 10MB but it is
configurable with the packet-log-max option of the vflow-create and vflow-modify commands.
Pluribus Networks nvOS Version 2.3.2
207
TIP! The mkfifo command used in this task is a standard feature of UNIX-like operating
systems, including MacOS. For Windows platforms, you may need to install the GNU
CoreUtils package available at http://gnuwin32.sourceforge.net/packages/coreutils.htm.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
208
Pluribus Networks nvOS Version 2.3.2
Using vFlows to Disable Communication
vFlows can be used to specify communications that are not allowed with a switch or a fabric. Use the following steps
to create a vFlow as a firewall:
1. Define a VLAN and destination IP-based flow and specify that the flow is dropped by the switch, with statistics
monitoring enabled:
CLI network-admin@switch > vflow-create name flow3 scope local vlan 99 dst-ip
172.168.24.1 action drop stats enable
Display the statistics for the new flow above as the traffic is dropped:
CLI network-admin@switch > vflow-stats-show name flow3 show-diff-interval 5
switch
aquila02
switch
aquila02
name
flow3
name
flow3
packets
864
packets
5
bytes
116K
bytes
936K
cpu-packets
0
cpu-packets
0
cpu-bytes
0
cpu-bytes
0
There are many options available for creating vFlows, and vFlows can be used to shape traffic, capture statistics,
capture flow metadata, capture packets, or manage communications. The options include:
 vlan
 vnet
 in-port
 out-port
 ether-type
 src-mac
 src-mac-mask
 dst-mac
 dst-mac-mask
 src-ip
 src-ip-mask
 dst-ip
 dst-ip-mask
 src-port
 dst-port
 dscp
 tos
 proto
 flow-class
 uplink-ports
 bw-min
 bw-max
Pluribus Networks nvOS Version 2.3.2
209
 precedence
 action
 action-value
 no-mirror
 mirror
 no-process-mirror
 process-mirror
 no-log-packets
 log-packets
 packet-log-max
 stats
 stats-interval
 duration
 no-transient
 transient
 vxlan
 vxlan-ether-type
 vxlan-proto
210
Pluribus Networks nvOS Version 2.3.2
Use Case Scenario
In a real use case, the command connection-show server-ip 10.9.10.117 was used to analyze a
suspicious connections to server 10.9.10.117:
Switch vlan client-ip server-ip service dur(s) latency(us) out-bytes in-by
tes active
------ ---- --------- --------- ------- ------ ----------- --------- ------- -----switch:
switch02
vlan:
1
client-ip:
10.9.9.33
server-ip:
10.9.9.107
service:
http
dur(s):
0
latency(us):
65
out-bytes:
0
in-bytes:
0
active:
yes
switch:
switch02
vlan:
1
client-ip:
10.9.9.33
server-ip:
10.9.9.107
service:
http
dur(s):
210
latency(us):
7
out-bytes:
48804
in-bytes:
6120
active:
yes
switch:
switch02
vlan:
1
client-ip:
10.9.9.33
server-ip:
10.9.9.107
service:
http
dur(s):
328
latency(us):
30
out-bytes:
48720
in-bytes:
612620
active:
yes
Configuring Mirroring for vFlows and Ports
A Pluribus Networks fabric administrator can run services and applications within the switch. Consider the use case
of an application that needs access to data that is flowing through the switch, but does not want to impede that
flow. The port-mirroring feature provides this functionality.
The system predefines a mirror configuration, but does not insert any traffic into that mirror. Use the following steps
to setup mirroring to send from all of the data ports to the span port (port 66). In this version of nvOS, the
port-mirror command is deprecated and replaced with the command mirror-modify to allow support for
vFlow-based and port-based mirroring. The command syntax for mirror-modify is as follows:
CLI network-admin@switch > mirror-modify out-port port-list in-port port-list
[policy port|vflow] mirroring|no-mirroring
CLI network-admin@switch > mirror-show [format fields-to-display]
[parsable-delim character] [sort-asc] [sort-desc] [show dups] [layout
vertical|horizontal] [show-interval seconds-interval]
Pluribus Networks nvOS Version 2.3.2
211
View the status of mirroring by entering the following at the CLI command prompt:
CLI network-admin@switch > mirror-show
switch: aquila19
direction: bidirection
out-port:
in-port:
mirroring: disable
The parameter out-port is not configured and mirroring is disabled therefore, no data mirroring can occur.
To modify the mirroring configuration, use the following steps:
1. Use the mirror-modify command to set the output to the span port. However, if there is more than 10Gb of
traffic on ports 1-64, do not execute this command.
CLI network-admin@switch > mirror-modify in-port 1-64 out-put 66 mirroring
mirror-show
switch:
pleiades24
direction:
bidirection
out-put:
66
in-port:
1-64
mirroring:
enable
To disable the configuration, use the following command:
CLI network-admin@switch > mirror-modify no-mirroring
mirror-show
switch: aquila19
direction: bidirection
out-port: 66
in-port: 1-64
mirroring: disable
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
212
Pluribus Networks nvOS Version 2.3.2
Managing Traffic Classes
nvOS provides a full set of traffic class features, including the ability to view and create traffic classes, as well as
assign traffic classes to flows to manage the quality of service of the flow traffic and shape the traffic passing
through an nvOS fabric.
To display the currently defined traffic classes:
CLI network-admin@switch > vflow-class-show
name
------------meter
guaranteed_bw
lossless
control
scope
-----fabric
fabric
fabric
fabric
type
-----system
system
system
system
priority
-------0
9
10
11
The higher the priority number, the higher the priority of the class. To add a vflow class, use the
vflow-class-create command:
CLI network-admin@switch > vflow-class-create name traffic-1 scope fabric
priority 5
This creates a traffic class with a scope of fabric and medium priority.
To add a traffic class to a vFlow, create a vFlow and assign a traffic class. In this case the flow is for a single IP address:
CLI network-admin@switch > vflow-create name losslessflow scope local src-ip
10.11.1.10 src-ip-mask 255.255.255.255 action none flow-class lossless
CLI network-admin@switch > vflow-show name losslessflow layout vertical
switch: aquila12
name: losslessflow
scope: local
type: vflow
vlan: 0
vnet:
in-port:
out-port:
ether-type: 0
src-ip: 10.11.1.10
dst-ip:::
src-port: 0
dst-port: 0
proto: ip
flow-class: lossless
bw-max: 0
pri: 0
action: none
action-value: 0
transient: no
Traffic from IP address 10.11.1.10 now has a very high priority throughout the switch. For a similar high priority
throughout the fabric use scope fabric rather than scope local.
Pluribus Networks nvOS Version 2.3.2
213
When a TCP session goes through the NPU, and capacity is exceeded, the return traffic with TCP ACK packets can get
dropped from the session. To avoid this, create a flow that matches the TCP ACK packets and set a higher
precedence for it.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
214
Pluribus Networks nvOS Version 2.3.2
Using Application Flows and Statistics
Displaying Standard Statistics
You can display standard statistics that consist of flow-based information collected and tracked continuously by the
switch.
To modify statistics logging, use the stats-log-modify command and disable or enable statistical logging as
well as change the interval, in seconds, between statistical events.
To display statistical logging information, use the stats-log-show command:
CLI network-admin@switch > stats-log-show
switch:
enable:
interval:
pleiades24
yes
60
To show connection-level statistics, traffic flows between a pair of hosts for an application service, including current
connections and all connections since the creation of the fabric, enter the following CLI command at the prompt:
CLI network-admin@switch > connection-stats-show
switch:
mac:
vlan:
ip:
port:
iconns:
oconns:
ibytes:
obytes:
total-bytes:
last-seen-ago:
switch:
mac:
vlan:
ip:
port:
iconns:
oconns:
ibytes:
obytes:
total-bytes:
last-seen-ago:
pleiades24
00:e0:81:e4:02:12
200
100.200.1.3
53
80
0
0
0
0
4d19h32m23s
pleiades24
00:12:c0:80:1e:85
200
100.200.1.4
16
0
70684
578M
890M
1.43G
46s
From the information displayed in the output, you can see statistics for each switch, VLANs, client and server IP
addresses, as well as the services on each connection. Latency and other information is also displayed.
The latency(us) column displays the running latency measurement for the TCP connection in microseconds. It
indicates end-to-end latency and includes the protocol stack processing for the connected hosts and all intermediary
network hops.
This is not the same latency measurement experience by a packet transiting the switch port-to-port. The
port-to-port latency is platform-dependent and you should refer to the datasheet for your switch model.
Pluribus Networks nvOS Version 2.3.2
215
To display specific types of connections, use the additional parameters with the command. For instance to display
active connections,
CLI network-admin@switch > connection-stats-show active
switch
switch12
switch12
switch12
switch12
switch12
. . .
vlan
1
5
1
1
1
vxlan vnet client-ip
server-ip
service active age
0
10.9.10.152 96.17.77.96 http
yes 35m27s
0
10.12.1.47 10.9.10.204 445
yes 7m56s
0
10.9.9.21
23.62.97.88 http
yes 3m41s
0
10.9.9.21
23.60.129.224http
yes 3m44s
0
10.9.10.72 10.9.99.23
http
yes 7s
To display a summary of traffic statistics for each application service, use the service-stats-show command.
CLI network-admin@switch > service-stats-show
switch
pleiades24
pleiades24
pleiades24
pleiades24
pleiades24
pleiades24
service
53495
8084
59475
imap
35356
54341
bytes
584
845M
33.9K
1.83M
106
584
From the information displayed in the output, you can review each switch, service, and the number of bytes used by
each service.
To display storage traffic statistics, use the storage-stats-show command:
CLI network-admin@switch > storage-stats-show
switch
server-ip
port read-bytes write-bytes
----------------- ---------- ----------switch12 10.9.9.9
65
3.63T
302K
switch12 10.9.10.113 nfs 0
0
switch12 10.9.9.33
nfs 284G
6.15K
switch12 10.9.11.18
65
137G
6.02K
switch12 10.9.10.69
nfs 46.0G
402K
. . .
From the information displayed in the output, you can review the storage data for each server, the port, and the
number of read-write bytes.
216
Pluribus Networks nvOS Version 2.3.2
To display interface statistics, use the interface-stats-show command:
CLI network-admin@switch > interface-stats-show
switch:
time:
nic:
ibytes:
ipkts:
ierrs:
obytes:
opkts:
oerrs:
switch:
time:
nic:
ibytes:
ipkts:
ierrs:
obytes:
opkts:
oerrs:
switch:
time:
nic:
ibytes:
ipkts:
ierrs:
obytes:
opkts:
oerrs:
switch:
time:
nic:
ibytes:
ipkts:
ierrs:
obytes:
opkts:
oerrs:
pleiades24
09:20:27
data
100M
302K
0
126M
453K
0
pleiades24
09:20:27
span
11.7M
396K
0
0
0
0
pleiades24
09:20:27
ops.mgmt.mgr.eth1
64.2M
774K
0
46.2K
1.10K
0
pleiades24
09:20:27
ext.50.mgr.eth0
2.41M
34.2K
0
679K
11.9K
0
From the information displayed in the output, you can review the inbound and outbound traffic for each NIC on the
switch. You can also check for errors in the inbound and outbound traffic.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Understanding vFlow Statistics
Virtual network-based flows, vflows, display statistics for packet traffic flows on a switch and across the fabric.
vFlows are very powerful and provide many features such as quality of service (QoS), traffic shaping, packet redirect,
drop actions, mirror, and capture.
Pluribus Networks nvOS Version 2.3.2
217
A vFLow can be configured to store log statistics to a file accessible to clients using NFS and SFTP. If statistics logging
is enabled, ONVL periodically polls the switch for the most recent statistics for each flow and saves the statistics to
an exported file. ONVL also saves individual statistics received from other switches in the fabric and combines the
statistics from all switches to record aggregate statistics for the entire fabric.
The switch consists of two components, the switch and the server. vFlows with operations like drop are executed
within the switch component. Some vFlows operations for QoS take place in the switch component, while others
operate within the coprocessor by directing pertinent traffic to the coprocessor. There, the traffic is managed and
then sent back to the switch component.
Other actions such as copy-to-cpu sends the match traffic to the server component where the traffic is managed
and then forwards packets for delivery. In general, the details are managed by nvOS including fabric scope
commands that cause all switches within a fabric to participate in an operation and then sends the compiled results
to the CLI or to log files.
Before you can access the files, you must enable NFS or SFTP access to the log files by using the
admin-service-modify command.
.
CLI network-admin@switch > vflow-share-show
switch
pleiades24
pleiades24
pleiades24
pleiades24
pleiades24
vnet
fab1-global
ops-mgmt
ext-50
www-51
folsom
enable
no
no
no
no
no
share-path
pleiades24:/nvOS/vnet/fab1-global
pleiades24:/nvOS/vnet/ops-mgmt
pleiades24:/nvOS/vnet/ext-50
pleiades24:/nvOS/vnet/www-51
pleiades24:/nvOS/vnet/folsom
CLI network-admin@switch > vflow-share-modify vnet fab1-global enable
vflow-share-show
switch
vnet
pleiades24 fab1-global
pleiades24 ops-mgmt
pleiades24 ext-50
pleiades24 www-51
pleiades24 folsom
enable
yes
no
no
no
no
share-path
pleiades24:/nvOS/vnet/fab1-global
pleiades24:/nvOS/vnet/ops-mgmt
pleiades24:/nvOS/vnet/ext-50
pleiades24:/nvOS/vnet/www-51
pleiades24:/nvOS/vnet/folsom
You can then access the statistics log files using NFS in the following locations:
For the switch scope, the files are located in
/net/switch-name/nvos/vnet/vnet-name/flow/flow-name/switch/
switch-name/stats
For the fabric scope, the files are located in
/net/switch-name/nvos/vnet/vnet-name/flow/flow-name/fabric/
stats
To create a vFLow for example, Host-Agent-Discover, and measure statistics, enter the following command:
CLI network-admin@switch > vflow-create name Host-Agent-Discover scope local
system
218
Pluribus Networks nvOS Version 2.3.2
To view all vFlows currently tracked by the switch or fabric, use the vflow-show command:
CLI network-admin@switch > vflow-show
switch:
name:
scope:
type:
dst-ip:
precedence:
action:
switch:
name:
scope:
type:
in-port:
src-port:
proto:
precedence:
action:
switch:
name:
scope:
type:
dst-ip:
precedence:
action:
switch:
name:
scope:
type:
in-port:
src-port:
proto:
precedence:
action:
pleiades24
Host-Agent-Discover
local
system
224.4.9.6
2
copy-to-cpu
pleiades24
DHCP-client
local
system
1-68
68
udp
2
copy-to-cpu
pleiades24
Host-Agent-Discover
local
system
224.4.9.6
2
copy-to-cpu
pleiades24
DHCP-client
local
system
1-68
68
udp
2
copy-to-cpu
From the information displayed in the output, you can review the switch, the name of the vFlow, scope, type of
vFlow, destination IP address, precedence, and action for the vFlow.
To display statistics for all vFlows, use the vflow-stats-show command:
CLI network-admin@switch > vflow-stats-show
switch
name
--------pleiades24IGMP-Flow
pleiades24 LLDP-Flow
pleiades24 Host-Agent
pleiades24 ECP
packets
------368K
82.9K
17.8K
0
bytes
----23.0M
26.3M
1.11M
0
cpu-packets
----------392K
82.9K
0
0
cpu-bytes
--------23.0M
26.0M
0
0
To monitor statistics of a vFlow and update every 10 seconds, use the following syntax:
CLI network-admin@switch > vflow-stats-show name flow1 show-diff-interval 10
Pluribus Networks nvOS Version 2.3.2
219
To log persistent records of flow statistics, use the logging parameter and collect statistics every 10 seconds:
CLI network-admin@switch > vflow-create name monitor-flow scope local
ether-type arp stats log stats-interval 5
You can display the statistics logs for the new flow using the vflow-stats-show command.
Informational Note: Conflicting vFlows
Multiple vFlows can be active at once, but nvOS cannot apply them at the same time. You can
use the precedence parameter is used to set the order of the vFlows. If you set the
precedence to a higher value (0 - 10 with 0 as the lowest precedence), the vFlow has a higher
precedence than those with lower values. If you’re seeing error messages about vFlow conflicts,
try adding a precedence value to new or existing vFlows.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Creating vFlows with the Scope Fabric
To create vFlows across the entire fabric, configure the vFlow with the scope fabric and stats enable option. Using
these parameters enables statistics for the flow on all switches that are members of the fabric and you can display
the statistics for any switch in the fabric.
To create a vFlow for VLAN1 with the scope fabric, use the following syntax:
CLI network-admin@switch > vflow-create name fab_flow1 scope fabric stats
enable vlan 1
To display the statistics for the new vFlow for any switch in the fabric, use the following syntax:
CLI network-admin@switch > switch switch-name vflow-stats-show name fab_flow1
name
packets
---------fab_flow1 51.4K
bytes
----13.8M
cpu-packets
----------50.1K
cpu-bytes
--------13.1M
If you omit the switch name, all vFlow statistics for the fabric are displayed.
switch
-----pleiades1
pleiades2
name
---fab_flow1
fab_flow1
packets
------1.32K
910
bytes
----305K
256K
cpu-packets
----------1.29K
884
cpu-bytes
--------291K
243K
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
220
Pluribus Networks nvOS Version 2.3.2
Example Use Cases for vFlows
The following examples illustrate how to use vFlows to impact traffic on the switch. You can regulate bandwidth,
create multiple vFlows, or share bandwidth.
Regulating Bandwidth for a VNET
To regulate bandwidth for all hosts in a VNET, create a vFlow and associate it with the appropriate flow class:
1. Create a VNET, bwvnet, using the vnet-create command:
CLI network-admin@switch > vnet-create name bwvnet scope fabric
2. All traffic associated with this VNET has a bandwidth of 5 Gbps. Create a vFlow:
CLI network-admin@switch > vflow-create name bwflow scope fabric vnet bwvnet
flow-class guaranteed-bw bw-min 5g
vflow-create:In order to use bw-min, please use vrg-modify to specify a min
bandwidth for vrg bwvnet-vrg
Creating the vFlow failed because a flow can only use the minimum bandwidth parameter if the associated VRG
(Virtual Resource Group) has minimum bandwidth allocated to it. You need to modify the VRG associated with the
VNET before assigning a minimum bandwidth to the vFlow.
3. Modify the VRG:
CLI network-admin@switch > vrg-modify name bwvnet-vrg data-bw-min 5g
4. Now create the vFlow for regulating bandwidth:
CLI network-admin@switch > vflow-create name bwflow scope fabric vnet bwvnet
flow-class guarantee-bw bw-min 5g
Informational Note: Before you assign minimum bandwidth to a vFlow, the associated VRG must have the
same bandwidth value or higher allocated to it.
You can also regulate bandwidth to a certain speed using vFlows.
5. Modify the VRG associated with the VNET:
CLI network-admin@switch > vrg-modify name bwvnet-vrg data-bw-max 5g
6. And then create the vFlow:
CLI network-admin@switch > vflow-create name bw-reg scope fabric vnet bwvnet
flow-class meter bw-max 5g
This creates a vFlow that allows bandwidth of up to 5 Gbps for all traffic on the VNET, bwvnet.
Pluribus Networks nvOS Version 2.3.2
221
Suppose you want to offer guaranteed bandwidth on a VNET, and cap the bandwidth to a fixed value. Add another
vFlow to perform this service:
CLI network-admin@switch > vflow-create name gw-bw scope fabtic vnet bwvnet
flow-class guaranteed-bw bw-min 5g bw-max 8g
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Creating Multiple vFlows for the Same VNET
You can create multiple vFlows for the same VNET and add precedence values to the vFlows. The packet is matched
to the vFlow with the highest precedence. For example,
Informational Note: You cannot create a new vFlow if a packet matches an existing flow.
1. Create the first vFlow:
CLI network-admin@switch > vflow-create name client-flow1 scope fabric vnet
bwvnet flow-class meter bw-max 2g
2. Create the second vFlow:
CLI network-admin@switch > vflow-create name client-flow2 scope fabric vnet
bwvnet flow-class meter bw-max 5g src-ip 192.168.20.1
vflow-create: Flow conflicts with Flow client-flow1, ID68: specify fields to
make flows mutually exclusive or change the flow precedence
The error message is generated because the vFlow configurations conflict with each other. To differentiate
between the two flows, assign a different precedence to client-flow2:
CLI network-admin@switch > vflow-create name client-flow2 scope fabric vnet
bwvnet flow-class meter bw-max 5g src-ip 192.168.20.1 precedence 5
Configuring Bandwidth Sharing for a Single VLAN with Different IP Addresses or Subnets
In some instances, you want to allow different subnets to share a guaranteed bandwidth on the same VNET. To do
this, you must create a VRG with the required bandwidth:
CLI network-admin@switch > vrg-create name admin-vrg vlans 100 data-bw-min 1g
data-bw-max 2g scope fabric
222
Pluribus Networks nvOS Version 2.3.2
You have now created a VRG with the guaranteed bandwidth of 1 Gbps and limited to a maximum of 2 Gbps. Now,
create a vFLow for each IP address:
CLI network-admin@switch > vflow-create name vfl-1 scope fabric vlan 100 src-ip
1.1.1.1
CLI network-admin@switch > vflow-create name vfl-2 scope fabric vlan 100 src-ip
2.2.2.2
CLI network-admin@switch > vflow-create name vfl-3 scope fabric vlan 100 src-ip
3.3.3.3
CLI network-admin@switch > vflow-create name vfl-4 scope fabric vlan 100 src-ip
4.4.4.4
In this example, the specified IP addresses each have a guaranteed bandwidth between 1 Gbps and 2 Gbps.
If you want to specify a subnet, 100.100.100.0/28, and VLAN 53 with maximum bandwidth of 50 Mbps, use the
following syntax:
CLI network-admin@switch > vrg-create name vrg-custom scope fabric data-bw-min
50M data-bw-max 50M vlan 53
CLI network-admin@switch > vflow-create name vfl-cust scope fabric src-ip
100.100.100.0 src-ip-mask 255.255.255.240 vlan 53
But later on, you found that sixteen IP addresses were not enough and you needed an additional 8 with the subnet,
101.101.101.8/29 that require the same bandwidth as the previous subnet. Use the following syntax:
CLI network-admin@switch > vflow-create name vfl-cust-2 scope fabric src-ip
101.101.101.8 src-ip-mask 255.255.255.248 vlan 53
You now have two vFlows on VLAN 53.
Then, you discover that 50 Mbps is not sufficient to support the network traffic affected by the vFlow, and you want
to upgrade to 80 Mbps:
CLI network-admin@switch > vrg-modify name vrg-custom data-bw-min 80M
data-bw-max 80M
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks nvOS Version 2.3.2
223
224
Pluribus Networks nvOS Version 2.3.2
Configuring VXLANs and Tunnels
 Configuring a VXLAN with nvOS
 Configuration Example
 Creating Tunnels
In today’s virtualized environments, there is increasing demand on MAC address tables of switches that connect to
servers. Instead of learning one MAC address per server link, the switch now has to learn the MAC addresses of
individual VMs, and if the MAC address table overflows, the switch may stop learning new MAC addresses until idle
entries age out.
Virtual Extensible LAN (VXLAN) is essentially a Layer 2 overlay scheme over a Layer 3 network, and each overlay is
called a VXLAN segment. Only VMs within the same VXLAN segment can communicate with each other. Each VXLAN
segment is identified by a 24 bit segment ID called the VXLAN Network Identifier (VNI).
VXLANs increase the scalability of your network up to 16 million logical networks and is used to contain broadcast,
multicast, and unknown unicast traffic.
Because of this encapsulation, VXLAN could also be called a tunneling scheme to overlay Layer 2 networks over top
of Layer 3 networks. However, the tunnel does not terminate on the switch, and the switch sits in the middle of the
tunnel and sees packets as L3 tunneled packets. These packets are then forwarded using L2 or L3 forwarding.
Pluribus Networks supports two scenarios for VXLAN:
1. The tunnel does not terminate on the switch and VTEP is not supported. Though the switch does not participate in
the creation of a tunnel, the following tasks are still performed.
a. Analytics Collection — All TCP control packets are captured as well as ARP packets traversing the tunnel.
These packets are used to build connection statistics and provide visibility as to which VXLAN nodes are on
specific ports.
b. ARP Optimization — An ARP request is captured and if an L2 entry exists in the switch L2 table, a response is
sent back to the sender of the ARP request over the tunnel. Otherwise, the ARP request is re-injected into the
tunnel without any modification to continue crossing the tunnel.
2. The tunnels are terminated at a switch and the switch performs the role of a VTEP. In this scenario, the switch is
responsible for encapsulating packets that arrive from non-VXLAN nodes on a L2 network and transmitting them
over the tunnel. Similarly, the packets arriving through the tunnel are decapsulated and the inner packet is forwarded over the L2 network. The switch also collects statistics and optimizes ARP requests as in the first scenario.
Informational Note: There is a one to one mapping of VXLAN to VLAN. Multicast traffic is not supported.
VXLAN has the scope local on all switches, and must be in the same subnet.
Configuring a VXLAN with nvOS
For the first scenario, no additional configuration is required. The second scenario requires the following steps, in
order:
1. Create a hardware vRouter.
2. Add interfaces to the vRouter, one per tunnel. The tunnel endpoint IP address should be routable.
3. Create one or more tunnels.
4. Create the VXLAN with the VNI, and add the tunnels created in the previous steps.
Pluribus Networks nvOS Version 2.3.2
225
To create a VXLAN, vx-seg1, with the VNID 25, scope fabric, and turn off deep inspection, use the following syntax:
CLI network-admin@switch > vxlan-create name vx-seg1 vnid 25 scope fabric
deep-inspection no
To delete a VXLAN, use the vxlan-delete command.
To display information about VXLANs, use the vxlan-show command.
If you added a port to the VXLAN configuration, use the vxlan-port-remove command.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Configuration Example
The following example assumes that one VTEP is on the generic switch and the other VTEP is on a Pluribus Networks
switch. Also, the nodes are connected on a L3 IP network, and the tunnel is formed between the generic switch and
the Pluribus Networks switch.
The example also includes VLAN 10 and port 47 on Host2 as well as the VNET fab-global.
1. Create the vRouter using the vrouter-create command:
CLI (server-switch)> vrouter-create name vx-vrouter vnet fab-global router-type
hardware
2. Add the vRouter interface:
CLI (server-switch)>vrouter-interface-add vrouter-name vx-vrouter ip 192.168.0.1
netmask 255.255.255.0 vlan 10
3. Create the tunnel:
CLI (server-switch)>tunnel-create name vx-tunnel scope local local-ip 192.168.0.1
remote-ip 192.168.5.1 next-hop 192.168.0.2 next-hop-mac 00:01:02:03:04:05 router-if
vx-router.eth0
4. Create the VXLAN:
CLI (server-switch)>vxlan-create vnid 14593470 scope local name vxlan1 vlan 10
If VLAN 10 does not exist, then the vxlan-create command creates it on the switch, but you may need to add
local ports to the VLAN.
5. Add port 47 to the VXLAN:
CLI (server-switch)>vxlan-port-add vxlan-name vxlan1 ports 47
This associates all packets from port 47 on VLAN 10 with the VXLAN ID, 14593470.
226
Pluribus Networks nvOS Version 2.3.2
6. Add the tunnel to the VXLAN:
CLI (server-switch)>vxlan-tunnel-add vxlan-name vxlan1 tunnel-name vx-tunnel
To display the configuration, use the vxlan-show command.
You cannot configure different VLANs for the tunnel and the local hosts, and you cannot associate different VLANs
on different ports for the same VXLAN.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Creating Tunnels
You can create tunnels to encapsulate protocols on the network. You can create tunnels for IP-in-IP, VXLAN, and
NVGRE network traffic. However, tunnels are supported on the local scope only and do not use any discovery
mechanism.
IP-in-IP protocol encapsulates an IP header with an outer IP header for tunneling. The outer IP header source and
destination identifies the endpoints of a tunnel. The inner IP header source and destination identify the original
sender and recipient of the datagram.
In addition to the IP header and the VXLAN header, the VTEP also inserts a UDP header. During ECMP, the switch
includes this UDP header to perform the hash function. The VTEP calculates the source port by performing the hash
of the inner Ethernet frame's header. The Destination UDP port is the VXLAN port.
The outer IP header contains the Source IP address of the VTEP performing the encapsulation. The destination IP
address is the remote VTEP IP address or the IP Multicast group address.
Network Virtualization using Generic Routing Encapsulation (NVGRE) uses GRE to tunnel Layer 2 packets over Layer
3 networks. NVGRE is similar to VXLAN but it doesn’t rely on IP multicast for address learning.
To create a tunnel for IP-in-IP traffic, local IP address 192.168.100.35, and the router, tunnel-network, use the
following syntax:
CLI network-admin@switch > tunnel-create scope local name ipinip type ip-in-ip
local-ip 192.168.100.35 router-if vrouter-hw-if eth0.0
To remove a tunnel, use the tunnel-delete command.
To modify a tunnel, use the tunnel-modify command.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Pluribus Networks nvOS Version 2.3.2
227
228
Pluribus Networks nvOS Version 2.3.2
Edge Virtual Bridging
Understanding Edge Virtual Bridging
Edge Virtual Bridging (EVB) is a software capability on a switch running Pluribus Networks nvOS® that allows
multiple VMs to communicate with each other and with external hosts in the Ethernet network.
Virtual Ethernet Port Aggregator (VEPA) is a software capability on a server that collaborates with an adjacent,
external switch to provide bridging support between multiple VMs and external networks. The VEPA collaborates
with the adjacent switch by forwarding all VM-originated frames to the adjacent switch for frame processing and
frame relay, including hairpin forwarding, and by steering and replicating frames received from the VEPA uplink to
the appropriate destinations.
Why Use VEPA instead of Virtual Ethernet Bridging (VEB)?
Even though VMs are capable of sending packets directly to one another with a technology called Virtual Ethernet
Bridging (VEB), physical switches are used for L2/L3 forwarding because VEB uses server hardware to accomplish the
task. Instead of using VEB, you can install VEPA on a server to offload switching functions to an adjacent physical
switch that offers less expensive L2/L3 forwarding.
Additional advantages of using VEPA include the following:
 VEPA reduces complexity and allows higher performance on the server
 VEPA takes advantage of the physical switch security and tracking features.
 VEPA provides visibility of inter-VM traffic to management tools designed for network switches.
 VEPA reduces the amount of network configuration required by server administrators, and as a consequence,
reduces workload for a network administrator.
How Does EVB Work?
EVB uses two protocols to work: Virtual Station Interface (VSI) Discovery and Configuration Protocol (VDP) and Edge
Control Protocol (ECP), to program policies for each individual virtual switch instance.
EVB maintains the following information for each VSI instance:
 VLAN ID
 VSI type
 VSI type version
 MAC address of the server
VDP is used by the VEPA server to propagate VSI information to the switch. This allows the switch to program
policies on individual VSIs and supports VM migration by implementing logic to pre-associate a VSI with a particular
interface.
ECP is an LLDP (Link Layer Discovery Protocol)-like transport layer that allows multiple upper layer protocols to send
and receive protocol data units (PDUs). ECP improves upon LLDP by implementing sequencing, retransmission and
an ACK mechanism. ECP is implemented in an EVB configuration when you configure LLDP on ports that you have
configured for EVB. In other words, you configure LLDP, not ECP.
You can configure EVB on a switch when that switch is adjacent to a server that includes VEPA technology. In general,
this is how to implement EVB:
 A network administrator creates a set of VSI types. Each VSI type is represented by a VSI type ID and a VSI
version. You can deploy one or several VSI versions at any time.
Pluribus Networks nvOS Version 2.3.2
229
 The VM administrator configures VSI which is a virtual station interface for a VM represented by a MAC address
and VLAN ID pair. The VM administrator queries available VSI type IDs (VTIDs) and creates a VSI instance
consisting of a VSI Instance ID and the chosen VTID. This instance is known as VTDB and contains a VSI manager
ID, a VSI type ID, a VSI version, and a VSI instance ID.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Configuring Edge Virtual Bridging
Remember, EVB does not convert packets, but it ensures that packets from one VM destined to another VM on the
same server are switched. When the source and destination of a packet are on the same port, EVB delivers the
packet, reflective relay, which otherwise would not happen because standard switching never forwards a packet to
the port from which it received the packet.
Before You Begin
Be sure that you have performed the following:
 Configured packet aggregation on the server connected to the port on the switch used for EVB.
 Configured the EVB port for all VLANs located on the VMs.
1. To enable VDP processing on all ports, enter the following CLI command at the prompt:
CLI network-admin@switch > vdp-modify enable
You can verify if VDP is enabled on a switch by using the vdp-show command.
2. To display the VSI instances and their state, use the vsi-state-show command:
CLI network-admin@switch > vsi-state-show
port mgrid vsiid_format
tate keepalive
49
::
mac
ASSOC
109
49
::
mac
10Gbps
20%
109
vsiid
linkspeed bw_limit traffic_class s
02:08:20:a8:13:67 10Gbps
10%
0
02:08:20:b0:25:39
0
ASSOC
3. To display ECP protocol statistics, use the following command:
CLI network-admin@switch > ecp-port-show
port ipkts opkts timeouts retransmits tx_errors last_rx_seqno last_ack_seqno
49
987
987
27
27
0
481
481
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
230
Pluribus Networks nvOS Version 2.3.2
Implementing OpenFlow with FloodLight
Floodlight Open Software Defined Network (SDN) Controller is an enterprise-class, Apache-licensed, Java-based
OpenFlow controller. It works with both physical and virtual switches that can interpret the OpenFlow protocol.
Since it is Apache licensed, you can use Floodlight for almost any purpose.
Informational Note: For more information about Floodlight Controller, go to
http://www.floodlight.org.
In this example, you create a NetZone to enable Floodlight, and use the VNET, vnet-engr, with the username
admin-opf, and the IP address 10.13.0.203/24:
CLI network-admin@switch > netzone-create name floodlight1 vnet vnet-engr user
admin-opf
netzone user password: password
confirm netzone user password: password
CLI network-admin@switch > netzone-interface-add netzone-name floodlight1 ip
10.13.0.203 netmask 24
CLI network-admin@switch > netzone-modify name floodlight1 floodlight-enable
By default, Floodlight OpenFlow Controller listens for OpenFlow protocol messages on port 6633 and exposes the
REST API to applications on port 8080.
Now, you can configure the OpenFlow daemon for the VNET, vnet-engr:
CLI network-admin@switch > openflow-connection-add name floodlight1 vlan 10
controller-ip 10.13.0.203 failmode standalone(open) control-port 6633
To begin using the Floodlight OpenFlow Controller within the NetZone, you can SSH to the NetZone using the IP
address that you configured in the previous example.
For additional documentation on using Floodlight, go to
http://docs.projectfloodlight.org/display/floodlightcontroller/Floodlight+Documentation
Pluribus Networks nvOS Version 2.3.2
231
232
Pluribus Networks nvOS Version 2.3.2
Configuring OpenFlow
 Enabling a Virtual Network for an OpenFlow Controller
 Creating OpenFlow Controllers with Multiple VLANs
 Configuring the OpenFlow Controller
 Configuring Open Virtual Switch (OVS) for OpenFlow
OpenFlow is the first standard communications interface defined between the control and forwarding layers of an
SDN architecture. OpenFlow allows direct access to the forwarding plane and allows you to manipulate the
forwarding plane of network devices such as switches and routers, both physical and virtual. Because current
networking devices lack an open interface, it has led to the characterization of the devices as monolithic, closed, and
mainframe-like. There is no other standard protocol like OpenFlow and an OpenFlow is needed to move network
control out of the networking switches to logically centralized control software.
The OpenFlow protocol is a key enabler for software-defined networks and is currently the only standardized SDN
protocol that allows direct access and manipulation of the forwarding plane on network devices.
For more information about OpenFlow, go to http://www.opennetworking.org.
Enabling a Virtual Network for an OpenFlow Controller
You can enable OpenFlow for a virtual network (VNET) with one or more VLANs and connecting the VLANs to an
OpenFlow controller.
If the VNET assigned to OpenFlow has the scope, local, the switch ports configured for the VNET appears to the
OpenFlow controller as a traditional, standalone OpenFlow switch with those ports.
If the VNET assigned to OpenFlow has the scope, fabric, the OpenFlow controller is presented with the abstraction of
a single logical big switch containing the ports from each switch in the fabric configured for the VNET. The Pluribus
Networks Netvisor (nvOS®) ensures that the state is distributed and rules are programmed into the individual
physical switch tables as necessary to present the abstraction of a single big switch.
Informational Note: The switch supports OpenFlow version 1.0 protocol. For more information about
the OpenFlow 1.0 protocol, go to http://www.opennetworking.org/index.php.
A switch or fabric can virtualize the physical network for one or more OpenFlow networks. Use the following steps to
create a VNET:
1. Create a virtual network and assign it to a VLAN, for example, VLAN10.
CLI network-admin@switch > vnet-create name openflow-1 scope fabric vlans 10
vnet created.
You can apply the standard VNET parameters such as bandwidth guarantee by configuring a virtual resource
group (VRG).
Pluribus Networks nvOS Version 2.3.2
233
2. Create an OpenFlow service for the VNET:
CLI network-admin@switch > openflow-create name openflow-1 vnet openflow-vnet
3. Create an OpenFlow daemon for the VNET, openflow-1 with the IP address of 192.168.1.11 on port 6633. Port
6633 is the well-known port for OpenFlow.
CLI network-admin@switch > openflow-connection-add name openflow-1 vlan 10
controller-ip 192.168.1.11 control-port 6633 failmode standalone(open | secure
(timeout)
The failure mode dictates the policy to follow if OpenFlow controllers configured for the VNET are unresponsive.
In standalone(open) failure mode, the VNET performs as a legacy Layer 2 switch. When connected to
a controller again, the existing flow entries remain. The controller can then delete all flow entries.
In secure(timeout) failure mode, packets and messages sent to the OpenFlow controllers are dropped from
the network. Flows expire according to the configured timeouts.
The default failure mode is standalone(open) mode.
4. Repeat the previous step for each OpenFlow controller on the VNET. For example, you may want to configure a
primary OpenFlow controller and a secondary OpenFlow controller as a backup option.
There may be certain times that you want to reset the connection from the VNET, openflow-1, to the OpenFlow
controller. You can use the openflow-restart command to perform this action.
To remove an OpenFlow controller from a VNET, specify the IP address associated with the OpenFlow controller. For
example,
CLI network-admin@switch > openflow-connection-remove name openflow-1 vlan 10
controller-ip 192.168.1.11
To remove all OpenFlow controllers from the VNET, omit the IP address from the command.
CLI network-admin@switch > openflow-connection-remove name openflow-1 vlan 10
To check the status of an OpenFlow connections, use the openflow-connection-show command.
234
Pluribus Networks nvOS Version 2.3.2
Creating OpenFlow Controllers with Multiple VLANs
If a VNET contains multiple VLANS, then each VLAN is controlled by a separate OpenFlow controller. In this example,
you have VLANs 0, 595, and 222, IP address 10.9.21.72/16, and you are creating a fabric named corp-fabric.
CLI network-admin@switch > fabric-create name corp-fabric
CLI network-admin@switch > vnet-create name vnet-engr scope fabric vlans
595,222
CLI network-admin@switch > vnet-manager-interface-add vnet-manager-name
vnet-engr-mgr ip 10.9.21.72/16 vlan 0 if mgmt
CLI network-admin@switch > vnet-manager-interface-add vnet-manager-name
vnet-engr assignment none vlan 595
CLI network-admin@switch > vnet-manager-interface-add vnet-manager-name
vnet-engr assignment none vlan 222
CLI network-admin@switch > openflow-create name engr-openflow vnet vnet-engr
CLI network-admin@switch > openflow-connection-add name engr-openflow
controller ip 10.9.21.17 failmode secure(timeout) vlan 595
CLI network-admin@switch > openflow-connection-add name engr-openflow
controller ip 10.9.21.17 failmode secure(timeout) vlan 222
CLI network-admin@switch > vlan-port-add vlan-id 595 untagged ports 46,49
CLI network-admin@switch > vlan-port-add vlan-id 222 untagged ports 45,50
After executing these commands on the switch, the fabric is in the following state:
 OpenFlow service, engr-openflow, is created on the VNET, vnet-engr.
 OpenFlow connection, engr-openflow, is added to VLAN 595 and VLAN 222.
 Ports 46 and 49 are added to VLAN 595.
 Ports 45 and 50 are added to VLAN 222.
Pluribus Networks nvOS Version 2.3.2
235
Configuring the OpenFlow Controller
nvOS has a built-in OpenFlow controller, Floodlight, that you can enable and then explore switch information using
the OpenFlow protocol. nvOS provides commands that allows you to send and receive data from the OpenFlow
controller.
For more information about the Floodlight controller, go to http://www.projectfloodlight.org/floodlight/
1. To enable the built-in OpenFlow controller, use the following commands:
CLI network-admin@switch > netvisor-zone-create name floodlight vnet openflow-1
user admin
netzone user password: <password>
confirm netzone user password: <password>
CLI network-admin@switch > netvisor-zone-interface-add netvisor-zone
floodlight ip 192.168.11.13 netmask 24
CLI network-admin@switch > netvisor-zone-modify name floodlight
floodlight-enable
Use an IP address on your network that allows you to access the Floodlight OpenFlow controller.
2. Now add the OpenFlow daemon to the virtual network:
CLI network-admin@switch > openflow-connection-add name floodlight vlan 10
controller-ip 192.168.11.13 failmode standalone(open)control-port 6633
The failure mode dictates the policy that is followed if all OpenFlow controllers configured for the virtual network
are unresponsive.
You can now begin using your built-in Floodlight OpenFlow controller with the Netvisor Zone that you just created.
For documentation on the configuration and management steps for Floodlight, go to
http://www.projectfloodlight.org/documentation/
Configuring Open Virtual Switch (OVS) for OpenFlow
Open Virtual Switch (OVS) is a production quality, multilayer virtual switch licensed under the open source Apache
2.0 license. It is designed to enable massive network automation through programmatic extension, while still
supporting standard management interfaces and protocols, for example, NetFlow, sFlow, IPFIX, RSPAN, CLI, LACP,
and 802.1ag.
After you create OpenFlow version 1.3 on your switch, you can add OVS as your OpenFlow controller by creating a
zone in the same manner as Floodlight.
CLI network-admin@switch > openvswitch-create name openflow13 vnet openflow
dedicated-vnet-service storage-pool diskpool1 gateway 192.168.11.13
db-conn-type default db-ip 192.168.11.15 db-port 6633
And then start the OVS using the openvswitch-start command.
236
Pluribus Networks nvOS Version 2.3.2
About sFlow
Overview
Because businesses rely on network services for mission critical applications, small changes in network usage can
impact network performance and reliability. As a result, these changes can also impact a business’ ability to conduct
key business functions and increase the cost of maintaining network services.
Figure 1: Overview of sFlow
sFlow provides the visibility into network usage and active routes on the network by providing the data required to
effectively control and manage network usage. This ensures that network services provide a competitive edge to the
business.
A few examples of sFlow applications include the following:
Detecting, diagnosing, and fixing network problems
Real-time congestion management
Understanding application mixes such as P2P, Web, DNS
Usage accounting for billing
Audit trail analysis to identify unauthorized network activity and trace sources of Denial of Service (DoS)
attacks
Route profiling and optimizing peers
Trending and capacity planning
sFlow is an open source sampling tool providing constant traffic flow information on all enabled interfaces
simultaneously. sFlow data is sent to a collector that formats the data into charts and graphs while recording and
identifying trends on the network. You can use this information for troubleshooting a network, perform diagnostics,
and analysis of data.
Pluribus Networks nvOS Version 2.3.2
237
The sFlow agent on the switch samples packets from data flows and forwards headers of the sample packet to a
collector at regular intervals. You can specify the number of packets to sample from the total packets which is called
the sample rate. The packets are stored and sent to the collector at an interval that you can configure on the switch.
This is called the polling interval. You can sample different types of packets such as frames sent to the CPU or
interfaces of the switch, routed packets, flooded packets, and multicast packets. However, the following packet types
are not sampled by sFlow:
LACP frames
LLDP frames
STP RPDUs
IGMP packets
Ethernet PAUSE frames
Frames with CRC errors
PIM_HELLO packets
Packets dropped by ACLs
Packets dropped as a result of VLAN violations
Routed packets with IP options or MTU violations
Counter Sampling
For counter sampling, also called polling, the sFlow agent periodically polls the hardware interface statistics
registers, counters, in the switch chip for per port statistics, and stores them in RAM until it is time to send the next
message to the sFlow collector. Overall port statistics such as the number of broadcasts, errors, are collected by the
sFlow agent.
The agent then includes the statistics in the sFlow datagrams sent to the sFlow collector along with the packet
sampling information. From these statistics, the sFlow obtains information about the actual utilization of each port.
For instance, information about broadcast to multicast to unicast rations is captured.
When you configure the agent for counter sampling, it sends an sFlow datagram at intervals of a second, at most.
The datagram contains a snapshot of the counters cached in RAM from the most recent polling of interface
counters.
Packet Sampling
Packet sampling is used to characterize network traffic. If the sFlow agent is configured for packet sampling, the
agent takes copies of random samples of packets forwarded within the switch CPU and sends them to the switch for
processing. The CPU sends a configured portion of the sampled packet, containing a number of protocol headers
and possibly some of the payload data to the sFlow collector. Random sampling prevents the synchronization of
periodic traffic patterns. On the average, 1 in every N packets is captured analyzed. The sampling can apply to
ingress and egress frames independently. The rate that the agent sends datagrams depends on the sampling rate,
the traffic rate, and the configured maximum datagram size. Typically, several samples are included in the datagram.
Agent to Collector Datagrams
After gathering packet and counter samples, each sFlow agent creates a packet of the data and sends it to an sFlow
collector in UCP datagrams. The datagrams contain the IP address of the sFlow collector and the standard UDP
destination port number of 6343. Using a standardized port helps avoid configuration between sFlow agents and
collectors. If the sFlow agent is configured for counter sampling or packet sampling, or both, an sFlow datagram can
contain either interface counters, packet samples, or a mixture of both.
238
Pluribus Networks nvOS Version 2.3.2
The following table provides information about the contents of sFlow datagrams:
Packet Header
Information
Version
The sFlow version used on the network.
IP Address Type
An IPv4 or IPv6 address
Source IP Address
The IP address of the sFlow agent
Sequence Number
The sequence number of the datagram
System Uptime
The length of time that the system is
operational.
Sample Count
The number of samples in the datagram
Ingress Interfaces
The ifindex of the switch port where the
packets entered the agent.
Egress Interfaces
The ifindex of the switch port where the
packets exited the agent.
Sample dataset
sFlow-specific parameters:
•
•
•
•
Sequence Numbers
Sampling Rate
Total Packets available for sampling
Number of sampled packets dropped
because there was no processing resource for
them.
Packet Samples
Packet sample information and may
contain several samples.
Packet data
The sampled data that may include the
packet payload data and the number on
length of protocol headers. This
information depends on the size of the
size, up to 200 bytes.
Counter Sample
Counter statistical information - fitted in
where space permits.
If index
The ifindex of the interface related to the
counters.
Physical Interface Parameters
•
•
•
•
Speed
Duplex mode
Admin status
Operational status of the interface
In Counters
•
•
•
•
•
•
•
ifInOctets
ifInUnicastPkts
ifInMultiPkts
ifInBroadcastPkts
ifInDiscards
ifInErrors
ifInUnknownProbs
Pluribus Networks nvOS Version 2.3.2
239
Packet Header
Information
Out Counters
•
•
•
•
Promiscuous Mode
The private VLAN promiscuous mode of
the interface
Ethernet Statistics
•
•
•
•
•
•
•
•
ifOutOctets
ifOutUcastPkts
ifOutDiscards
ifOutErrors
Alignment Errors
FCS Errors
SQE Errors
Deferred Transmission
Internal MAC errors
Carrier sense errors
Overlength frame errors
Symbol errors
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
240
Pluribus Networks nvOS Version 2.3.2
Configuring sFlow
From the following network diagram, let’s configure sFlow and sFlow agents.
Figure 1: sFlow Network with IP Addresses
Configuring the sFlow Collector
Before configuring the sFlow agents, you must configure the sFlow collector. The sFlow collector receives sFlow
datagrams from the sFlow agents. In this example, the sFlow collector has an IP address of 10.1.1.243, and a default
port of 6343. The collector name is net-man-all, and the scope is fabric. If the scope is fabric, then additional
switches that join the fabric receive the sFlow collector configuration. If the scope is local, then the sFlow collector is
configured only on one switch.
CLI network-admin@switch > sflow-collector-create collector-ip 10.1.1.243
collector-port 6343 name net-man-all scope fabric
You can add as many collectors as needed for your configuration.
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
Enabling sFlow on the Network
You must configure and enable sFlow on each switch that you want to use for monitoring network traffic. You can
only configure one sFlow per switch.
Pluribus Networks nvOS Version 2.3.2
241
On each switch in the example diagram, use the following command to enable sFlow, net-monitor, on ingress ports
57-59, sample type raw, sample-rate 4096, sample interval 5 seconds, trunc-length 160 bytes, on VLAN 200:
CLI network-admin@switch > sflow-create name net-monitor sample-type raw ports
57-59 sample-rate 4096 trunc-length 160 vlan 200
Adding Additional Ports to sFlow
To add the ports, 61-62, to the sFlow configuration, you must use the following command on each switch:
CLI network-admin@switch > sflow-port-add sflow-name net-monitor switch
10.1.1.23 ports 61-62
In this example, the IP address of the switch is used as the name of the switch.
Removing Ports from the sFlow Configuration
You can remove ports from the sFlow configuration by using the sflow-port-remove command:
CLI network-admin@switch > sflow-port-remove sflow-name net-monitor switch
10.1.1.23 ports 61-62
Topic Feedback
Was this topic useful to you? Please provide feedback to improve the content.
242
Pluribus Networks nvOS Version 2.3.2