Download OTV Technical Deep Dive

OTV
Technology Introduction
Natale Ruello
Technical Marketing Engineer – Nexus 7000
Addressing Business Goals with LAN
Extensions
Business Goals
99.999% Global
Availability
Service Velocity and
On-demand Capacity
LAN Extensions
Enable Distributed Clusters to improve
Application Availability without
compromising Network Resiliency
Attributes
Availability
Unleash Compute Virtualization beyond
a single physical data center for fast service and
capacity additions
Adaptability
Maximize Asset
Utilization
Supports migration of workloads and
consolidation of servers across locations
Streamline Operations
& Reduce OPEX
Enables improved change management
methods across multiple physical locations
Non-disruptive model for minimal
operational overhead
BRKRST-2033
to avoid power/cooling hot spots or compute/network idleness
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Public
Cost
Optimization
3
Enabling IT Solutions with LAN extensions
Vmotion
Data Center
Cluster
Data Center
A
B
MSCS
Cluster
LAN Extension Solaris Sun Cluster Enterprise
RAC (Real Appl.Cluster)
HACMP
Legato Automated Availability Mgr
Metro Cluster
Metrocluster
BACnet (building automation/control)
BRKRST-2033
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Public
4
Overlay Transport Virtualization (OTV)
O
T
V
Overlay - A solution that is independent of the
infrastructure technology and services, flexible
over various inter-connect facilities
Transport - Transporting services for layer 2 and
layer 3 Ethernet and IP traffic
Virtualization - Provides virtual connections,
connections that are in turn virtualized and
partitioned into VPNs, VRFs, VLANs
OTV LAN Extensions
BRKRST-2033
OTV delivers a virtual L2 transport
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Public
5
Challenges with LAN Extensions
Real Problems Solved by OTV
 Extensions over any transport (IP, MPLS)
Fault
Domain
North
Data
Center
Fault
Domain
 Failure Boundary Preservation
 Site independence / Isolation
 Optimal BW utilization (no head-end
replication)
 Resiliency/multi-homing
LAN Extension
 Built-in end-to-end Loop Prevention
 Multi-site connectivity (inter and intra DC)
 Scalability
 VLANs, Sites, MACs
Only 5 CLI
commands
 ARP, Broadcasts/Floods
 Operations Simplicity
Fault
Domain
 Topology Flexibility
BRKRST-2033
© 2009 Cisco Systems, Inc. All rights reserved.
Fault
Domain
South
Data
Center
Cisco Public
6
Traditional Layer 2 VPNs
EoMPLS
Dark Fiber
VPLS
Presentation_ID
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
7
Flooding Behavior
 Traditional Layer 2 VPN technologies rely on flooding to propagate MAC
reachability.
 The flooding behavior causes failures to propagate to every site in the L2-VPN.
x2
Site A
Site C
MAC 1
MAC 1
propagation
Site B
 A solution that provides layer 2 connectivity, yet restricts the reach of the flood
domain is necessary in order to contain failures and preserve the resiliency.
Presentation_ID
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
8
Pseudo-wires Maintenance
 Before any learning can happen a full mesh of pseudo-wires/tunnels must be in place.
 For N sites, there will be N*(N-1)/2 pseudo-wires. Complex to add/remove sites.
 Head-end replication for multicast and broadcast. Sub-optimal BW utilization.
 A simple overlay protocol with built-in functionality and point-to-cloud provisioning is key
to reducing the cost of providing this connectivity
Presentation_ID
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
9
Multi-Homing
 Require additional protocols to support Multi-homing.
 STP is often extended across the sites of the Layer 2 VPN. Very difficult to
manage as the number of sites grows.
 Malfunctions on one site will likely impact all sites on the VPN.
Active
Active
L2 VPN
L2 Site
L2 Site
 A solution that natively provides automatic detection of multi-homing without
the need of extending the STP domains is key.
Presentation_ID
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
10
What can be improved
 Data Plane Learning  Control Plane Learning
Moving to a Control Plane protocol that proactively advertises MAC
addresses and their reachability instead of the current flooding
mechanism.
 Pseudo-wires and Tunnels  Dynamic Encapsulation
No static tunnel or pseudo-wire configuration required.
Optimal replication of traffic done closer to the destination, which
translates into much more efficient bandwidth utilization in the core.
 Multi-Homing  Native Built-in Multi-homing
Ideally a multi-homed solution should allow load balancing of flows
within a single VLAN across the active devices in the same site,
while preserving the independence of the sites.
STP confined within the site (each site with its own STP Root bridge)
Presentation_ID
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
11
Overlay Transport Virtualization
Technology Pillars
OTV is a “MAC in IP” technique for
supporting Layer 2 VPNs
OVER ANY TRANSPORT.
Dynamic Encapsulation
Protocol Learning
No Pseudo-Wire State
Maintenance
Built-in Loop Prevention
Optimal Multicast
Replication
Preserve Failure
Boundary
Multi-point Connectivity
Seamless Site
Addition/Removal
Point-to-Cloud Model
Automated Multi-homing
BRKRST-2033
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Public
12
OTV at a Glance
 Ethernet traffic between sites is encapsulated in IP: “MAC in IP”
 Dynamic encapsulation based on MAC routing table
 No Pseudo-Wire or Tunnel state maintained
IP packet
Ethernet Frame
Encap
MAC
IF
MAC1
Eth1
MAC2
IP B
MAC3
IP B
Decap
OTV
OTV
IP B
IP A
West
Site
BRKRST-2033
Ethernet Frame
Ethernet Frame
© 2009 Cisco Systems, Inc. All rights reserved.
Communication between
MAC1 (West) and MAC2 (East)
Cisco Public
MAC
IF
MAC1
IP A
MAC2
Eth 1
MAC3
Eth 2
East
Site
13
OTV Data Plane: Unicast
MAC Table contains
MAC addresses reachable through
IP addresses
MAC TABLE
OTV Inter-Site Traffic
1
Layer 2
Lookup
MAC TABLE
VLAN
MAC
IF
100
MAC 1
Eth 2
100
MAC 2
Eth 1
100
MAC 3
IP B
100
MAC 4
IP B
VLAN
MAC
100
MAC 1
IP A
5
100
MAC 2
IP A
Layer 2
Lookup
100
MAC 3
Eth 3
100
MAC 4
Eth 4
OTV
OTV
MAC 2
Eth 2
MAC 1  MAC 3
External
IP B
External
IP A
Eth 1
MAC 1  MAC 3
L2
3
Encap
MAC 1
Eth 4
Eth 3
Core
2
MAC 4
MAC 1MAC
 MAC
1  3MACIP3 A  IP B
IP A  IP B
L3
IF
6
L3 L2
MAC 1  MAC 3
4
Decap
MAC 3
East
West
No Pseudo-Wire state is maintained.
The encapsulation is done based on a Layer 2 destination lookup.
The encapsulation is done in hardware by the Forwarding Engine.
BRKRST-2033
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Public
14
Building the MAC tables
The OTV Control Plane
 The OTV control plane proactively advertises MAC reachability (controlplane learning).
 The MAC addresses are advertised in the background once OTV has
been configured.
 No protocol specific configuration is required.
MAC Addresses
Reachability
Core
IP A
IP B
East
West
IP C
South
BRKRST-2033
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Public
15
OTV Control Plane
MAC address advertisements – Multicast Core
 Every time an Edge Device learns a new MAC address, the OTV
control plane will advertise it together with its associated VLAN IDs
and IP next hop.
 The IP next hops are the addresses of the Edge Devices through
which these MACs are reachable in the core.
 A single update reaches all neighbors.
OTV update is replicated
by the core
4
VLAN
1
3 New MACs are
learned on VLAN 100
Vlan 100
MAC A
Vlan 100
MAC B
Vlan 100
MAC C
3
Core
2
MAC
IF
100
MAC A
IP A
100
MAC B
IP A
100
MAC C
IP A
East
IP A
VLAN
West
3
MAC
IF
100
MAC A
IP A
100
MAC B
IP A
100
MAC C
IP A
4
South-East
BRKRST-2033
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Public
16
Multicast Groups in the Core
Summary of the Multicast groups used by OTV
 OTV will leverage the multicast capabilities of the core.
 This is the summary of the Multicast groups used by OTV:
 An ASM/Bidir group to exchange MAC reachability.
 An SSM group range for the multicast data generated by the site.
Presentation_ID
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
17
What if core multicast is not an option?
OTV in Unicast Mode – The Adjacency Server Mode
 The use of multicast in the core provides significant benefits:
Reduce the amount of hellos and updates OTV must issue
Streamline neighbor discovery, site adds and removes
Optimize the handling of broadcast and multicast data traffic
 However multicast may not always be available
 The OTV Adjacency Server Mode of operation provides a
unicast based solution.
Presentation_ID
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
18
Adjacency Server
Core with no support for multicast: Adjacency Server
 Despite the naming this is NOT a physical server. It is just a mode of
operation of the Edge Devices.
 An OTV node which sends a multicast packet on a non-multicast
capable network will “unicast replicate (head-end)” the packet.
 One of the OTV Edge Devices will be configured as an Adjacency
Server and it will be responsible for communicating the IP addresses
where the other Edge Devices can be reached.
 The group of IP addresses is called overlay Adjacency List (oAL)
 Two configuration steps:
1. Configure an OTV Edge Device to be the Adjacency Server
2. Configure the other Edge Devices to point to the Adjacency Server to
retrieve each other IP address
Presentation_ID
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
19
Adjacency Server
 At first, the Adjacency Server knows about no other OTV Edge Devices because
their oAL is empty.
 Once other OTV Edge Devices start sending to the Adjacency Servers their site-id
and IP address, the Adjacency Server will build up its oAL.
 The contents of the oAL is advertised and sent unicast to each member of the oAL.
Now the Edge Devices can communicate with each other.
Site 2
Site 3
IP C
IP B
oAL
Site 1, IP
Site 2, IP
Site 3, IP
Site 4, IP
Site 5, IP
Site 1
A
B
C
D
E
Core
IP A
Adjacency
Server
Presentation_ID
© 2009 Cisco Systems, Inc. All rights reserved.
IP E
IP D
SiteCisco4 Confidential
Site 5
20
STP BPDU Handling
 When STP is configured at a site, an Edge Device will send and
receive BPDUs on the internal interfaces.
 An OTV Edge Device will not originate or forward BPDUs on the
overlay network.
 An OTV Edge Device can become (but it is not required to) a root of
one or more spanning trees within the site.
 An OTV Edge Device will take the typical action when receiving
Topology Change Notification (TCNs) messages.
The BPDUs
stop here
OTV
Core
Presentation_ID
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
21
Unknown Unicast Packet Handling
 Flooding of unknown unicast over the overlay is not required and is
therefore suppressed.
 Any unknown unicasts that reach the OTV edge device will not be
forwarded onto the overlay.
 The assumption here is that the end-points connected to the network
are not silent or uni-directional.
 MAC addresses for uni-directional host are learnt and advertised by
snooping the host’s ARP reply
No MAC 3 in the
MAC Table
MAC TABLE
OTV
VLAN
MAC
IF
100
MAC 1
Eth1
100
MAC 2
IP B
Core
MAC 1  MAC 3
Presentation_ID
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
22
Controlling ARP traffic
Proxy ARP
 OTV Edge Devices can proxy ARP replies on behalf of remote hosts
 ARP traffic spanning multiple sites can thus be significantly reduced
 An ARP cache is maintained by every OTV edge device
 The ARP cache is populated by snooping ARP replies
 Initial ARP requests are broadcasted to all sites
 Subsequent ARP requests are suppressed and answered locally
 The ARP cache could also be populated at MAC learning time, this would
allow the suppression of all ARP related broadcast
2
4
Subsequent
ARP requests
(IP A)
5
Proxy ARP
reply (IP A)
OTV
OTV
1
First
ARP
request
(IP A)
BRKRST-2033
ARP
reply
Core
3
ARP Cache
MAC 1
IP 1
MAC 2
IP 2
© 2009 Cisco Systems, Inc. All rights reserved.
Snoop &
cache
ARP
reply
AED
One time traffic
Cisco Public
23
OTV solves Layer 2 fault propagation
Summary
 STP Isolation: BPDUs are not forwarded over the
overlay
 Unknown unicasts are not flooded across sites
Selective flooding is optional
 Cross site ARP traffic is reduced with Proxy ARP
 Broadcast can be controlled based on a white list as
well as a rate limiting profile
BRKRST-2033
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Public
24
Multi-Homing: Loop Condition Handling
 OTV includes the logic necessary to avoid the creation of loops in
multi-homed site scenarios.
 Each site will have its own STP domain, which is separate and
independent from the STP domains in other sites, even though all
sites will be part of common Layer 2 domain.
STP
domain 1
No STP
OTV
OTV
STP
domain 2
OTV
Core
OTV
Presentation_ID
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
25
Authoritative Edge Device
 OTV provides loop-free multi-homing by electing a designated forwarding
device per site for each VLAN.
 The designated forwarder is referred to as the Authoritative Edge
Device (AED).
 The Edge Devices at the site peer with each other on the internal
interfaces to elect the AED
 The AED is the only edge device that will forward multicast and broadcast
traffic between a site and the overlay.
OTV
OTV
OTV
Core
OTV
AED
AED
Presentation_ID
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
26
Multi-Homing: AED & Broadcast
 A broadcast packet gets to all the Edge Devices within a site.
 The AED for the VLAN is the only Edge Device that forwards broadcast
packets on the overlay network.
 All the Edge Devices at a remote site will receive the broadcast packet, but
only the AED at the remote site will forward the packet into the site.
 Once sent into the site, the packet gets to all switches on the site specific
Spanning Tree.
OTV
Broadcast
stops here
OTV
Broadcast
stops here
OTV
Core
OTV
Bcast
pkt
AED
AED
Presentation_ID
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
27
Multi-Homing
AED & Unicast Forwarding
 One AED is elected for each VLAN on each site
 Different AEDs can be elected for each VLAN to balance traffic load
 Only the AED forwards unicast traffic to and from the overlay
 Only the AED advertises MAC addresses for any given site/VLAN
 Unicast routes will point to the AED on the corresponding remote
site/VLAN
MAC TABLE
VLAN
MAC
100
MAC 1
200
MAC 2
IF
OTV
IP A
OTV
IP B
AED
AED
IP A
OTV
Core
OTV
IP B
AED
AED
BRKRST-2033
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Public
28
Configuration
OTV CLI configuration
Connects to the core. Used to join the Overlay network.
Its IP address is used as source IP for the OTV encap
ASM/Bidir group in the core used for the
OTV Control Plane.
interface Overlay0
SSM group range used to carry the site’s
mcast traffic data.
description otv-demo
otv join-interface Ethernet1/1
otv control-group 239.1.1.1
otv data-group 232.192.1.2/32
otv extend-vlan 100-150
Site VLANs being extended by OTV
otv site-vlan 100
VLAN used within the Site for communication
between the site’s Edge Devices
BRKRST-2033
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Public
29
Configuration
OTV CLI configuration with adjacency server
Connect to the core. Used to join the core mcast groups.
Their IP addresses are used as source IP for the OTV encap
Configures this Edge device as an
Adjacency Server
interface Overlay0
description otv-demo
Use a remote Edge Device as the Adjacency
Server (mutually exclusive with the previous
line)
otv join-interface Ethernet1/1
otv adjacency-server
or otv use-adjacency-server 10.10.10.10
otv extend-vlan 100-150
otv site-vlan 100
Site VLANs being extended by OTV
VLAN used within the Site for communication
between the site’s Edge Devices
BRKRST-2033
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Public
30
Nexus 7000 Rollout plan
 EFT
Target start date: Mid January, 2010
 FCS
Q1CY2010
Presentation_ID
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
31