Download Cisco IP SLA`s

CISCO IOS IP SERVICE LEVEL
AGREEMENTS: TECHNICAL
OVERVIEW
TOM ZINGALE
INTERNET TECHNOLOGIES DIVISION
SEPTEMBER 2004
Cisco IOS IP SLA,
Technical, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Internal Use Only
1
Cisco IOS IP Service Level Agreement:
A New Direction
• Cisco solution that assures IP service levels, proactively verifies
network operation, and accurately measures network performance
Comprehensive hardware support
Committed Cisco partner support
Cisco IOS Software, the world’s leading network infrastructure software
Enterprise and Small Medium Business
Understand Network
Performance &
Ease Deployment
Access
Service Providers
Verify Service Levels
Verify Outsourced SLAs
Enterprise
Premise Edge
Enterprise Backbone
Measure and provide
SLAs
Service Provider
Aggregation Edge
Service Provider Core
Cisco IOS Software
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
2
The Need for IP-Based Service Levels
PROBLEM
40% of companies delay launching new
applications due to network performance
concerns2
RESULT
Reduced business
productivity
59% of companies simply add bandwidth to ensure
application efficiency2
Increased network costs
55% of companies only identify some of their
network traffic2
Reduced understanding of
network behavior
Cost of application downtime and degradation is
$13,000 per minute for an ERP application3
Lowered network
performance can be costly
2003 Infonetics Research Study “Cost of Enterprise Downtime”
www.infonetics.com/services/green.shtml?2004/service.provider.and.user.plans.shtml
2 2003 Network World Application Performance Market Study
www.nwfusion.com
3 Forrester Research
Cisco IOSwww.forrester.com
IP SLA, and
1
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
3
Cisco IOS IP SLA Benefits
OPTIMIZED APPLICATIONS
& SERVICES
• Performance visibility
• Prove service levels
• Enhance Customer satisfaction
• Enhance acceptance of businesscritical services
REDUCED TOTAL COST OF
OWNERSHIP AND OpEx
• Reduce deployment time
• Lower mean time to restore and
downtime
• Proactive identification of issues
enforces higher reliability
Continuous
Predictable Reliable
Measurements and Metrics
Automated Intelligence
Proactive
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
4
Cisco IOS IP SLAs Life Cycle
Baseline network
performance
Verify network readiness
for new services with
Cisco IOS IP SLA
capabilities.
2
1
Quantify results
• Reduce deployment time
• Prove service and
application differentiation
• Verify service levels
• Reduce network down
time
• Manage demand for the
network
Cisco IOS IP SLA, and
NetFlow, 9/04
Understand network
performance baseline
Confidence to deploy
new IP services
and applications
© 2004 Cisco Systems, Inc. All rights reserved.
Assure
application
and service
deployment
3
4
Cisco Confidential
Fine tune and
optimize
Ongoing
measurements to
understand behavior
with proactive
notification
5
Example: Multi-Protocol Measurement
and Management with Cisco IOS IP SLAs
Applications
Network
Performance
Monitoring
Availability
VoIP
Monitoring
Service Level
Agreement
(SLA)
Monitoring
Network
Assessment
Multiprotocol
Label
Switching
(MPLS)
Monitoring
Trouble
Shooting
Measurement Metrics
Packet
Loss
Latency
Network
Jitter
Dist. of
Stats
Connectivity
Protocols
Jitter
FTP
DNS
DHCP
DLSW
ICMP
UDP
TCP
HTTP
LDP
H.323
SIP
RTP
Radius
Video
IP Server
Defined Packet Size, Spacing
COS and Protocol
IP Server
Cisco IOS
Software
Source
IP SLA
MIB Data
Cisco IOS
Software
Active Generated Traffic to
measure the network
Destination
Cisco IOS
Software
IP SLA
IP SLA
Responder
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
6
Comprehensive Hardware Support
Core
Enterprise & Aggregation/Edge
Cisco IOS Software Release 12.2S
Cisco 7200
Series
Cisco Catalyst
Cisco 10000 6500; Cisco
Series
7600 Series
Cisco 7300
Series
Cisco 12000
Series
Access
Cisco IOS Software Releases 12.3T and 12.4
Cisco 800 Cisco 1700
1800
Series
Series
Cisco IOS IP SLA, and
NetFlow, 9/04
Cisco 2600
2800
Series
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco 7200 &
Cisco 3700 7300 Series
3800Series
Cisco Confidential
Cisco 2900,
3550, &
3750 Series
7
SLA Verification and Management
• Access router may be managed or unmanaged
• Data typically provided by the service provider for the customer
includes availability, QoS, and Jitter SLAs
• Service Provider needs visibility in the Customer Edge, in order to
commit to SLAs
• Enterprise will verify SP SLAs by using access router edge to edge
measurements
Enterprise may provide restricted Simple Network Management Protocol
(SNMP) (RTT, Latency, QoS) visibility into Access router for Service
Provider
Service Provider with restricted access can report SLA as a service back
to the enterprise
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
8
Network Monitoring
• Cisco IOS IP SLA answers the following question:
What is the jitter, latency, or packet loss between any two points
in the network?
• IP Services can be simulated by specifying various packet
sizes, ports, class of service, packet spacing, and
measurement frequencies
• Uni-directional and highly accurate measurements
• Measurements per class of service to validate service
differentiation for data, voice, and video
• Cisco IOS IP SLA will identify an edge to edge network
performance baseline and allow the user to understand trends
and anomalies from the baseline
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
9
IP Network Readiness
• Network assessment tool built into Cisco IOS
Software
• Simulate IP Services and verify how well they will
work in the network
• How well is QoS working in the network predeployment
• Post deployment continued verification of network
performance per IP service
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
10
Availability Monitoring
• Cisco IOS IP SLA uses proactive monitoring for periodic,
reliable, and continuous availability measurements
• Connectivity measurements from Cisco router to router or
Cisco router to server
• Threshold notifications when end point is not available
What is the availability of a Network File System (NFS) server
used to store business critical data from a remote site ?
Cisco IOS IP SLA UDP active measurement to specific server
ports is used to test remote site to server connectivity
If server is unavailable, then traps can notify the network
management system
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
11
Troubleshooting with Cisco IOS IP SLA
• Proactive notification of problems and issues
based on threshold alerts
• Testing edge to edge consistently and reliability will
save time in finding and pin pointing network
performance problem areas
• Secondary activation of path operation (ie: path
jitter) or activation of operations at a higher
frequency to isolate and verify problem areas in the
network
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
12
Cisco IOS IP SLA
Source and Responder
• Source Router
Cisco IOS Software router that sends data from operation
Cisco IOS Software may or may not be the target
Some operations require the target to run the IP SLA responder
Stores results in MIB
• Responder
Responds to IP SLA packets at destination
User defined UDP/TCP ports
IP SLA Control Protocol
MD 5 Authentication
Accurate measurements
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
13
Responder
Source Router
Target Router
Responder
T2
T1
T3
T4
D = T3 - T2
The Responder takes 2 Timestamps (T2 & T3)
• Responder factors out destination processing time making results
highly accurate
• Responder allows for one-way measurements for latency, jitter, packet
loss, and MOS
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
14
Cisco IOS IP SLAs Uses and Metrics
*DATA
TRAFFIC
*VoIP
REQUIREMENT
• Minimize
Delay, Packet
Loss
• Verify Quality
of Service
(QoS)
IP SLA MEASURMENT
•
•
•
•
Jitter
Packet loss
Latency
per QoS
• Minimize
Delay, Packet
Loss, Jitter
•
•
•
•
*SERVICE LEVEL
AGREEMENT
• Measure Delay,
Packet Loss,
Jitter
• One-way
Jitter
Packet loss
Latency
MOS Voice
Quality Score
• Jitter
• Packet loss
• Latency
• One-way
• Enhanced
accuracy
• NTP
*AVAILABILITY
Connectivity
testing
• Connectivity
tests to IP
devices
**STREAMING
VIDEO
• Minimize
Delay, Packet
Loss
• Jitter
• Packet loss
• Latency
* Currently available
**Limited availability in 9/04; complete in CY’05
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
15
Cisco IOS IP SLA Reaction Conditions
• Reaction Trigger to Events
Can send SNMP traps for certain “triggering” events:
Connection Loss and Timeout
Round Trip Time Threshold
Average Jitter Threshold
Unidirectional packet loss, latency, jitter, MOS Scores
Trigger
• Immediate
• Consecutive
• X of Y times
• Average Exceeded
Can trigger another IP SLA operation for further analysis
Threshold
Violation
Alert
No Alert
Threshold
Violation
Alert
100 ms
50 ms
Threshold
violation
Time
Cisco IOS IP SLA, and
NetFlow, 9/04
Technical,
9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Resolution
Cisco
Cisco
Internal
Confidential
Use Only
16
Availability
12.0(5)T
12.0(8)S
12.1E
12.1(1)T
12.2
12.2(2)T
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
UDP Jitter One Way
Latency
X
FTP Get
X
Feature/Release
11.2
12.0(3)T
ICMP Echo
X
X
X
X
X
X
X
X
ICMP Echo Path
SSCP(SNA)
UDP Echo
TCP Connect
UDP Jitter
HTTP
DNS
DHCP
DLSw+
SNMP Support
MPLS/VPN Aware
Frame-Relay (CLI)
ICMP Path Jitter
APM
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
12.2(11)T
(Infra2)
12.2(14)S
12.2(25)S
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Cisco Confidential
X
X
X
X
X
X
X
X
17
Cisco IOS IP SLA Partners
Cisco Network Management Solution
Cisco IP Solution Center
MPLS VPN and SLA Monitoring
Internetworking Performance Monitor
Enterprise performance measurements
THIRD PARTY PRODUCTS
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
18
Cisco IOS IP SLA Performance with
Infrastructure 2: CPU Load by Hardware
•
Jitter probe
•
Versus Release
12.3(3)
Operations/
Second
Operations/
Minute
Cisco 2600
Series
4
8
12
16
240
480
720
960
14
20
29
35
20
24
28
32
36
40
44
1200
1440
1680
1920
2160
2400
2640
41
48
56
63
67
48
52
56
60
2880
3120
3360
3600
•
Cisco
2620XM
Series
7
8
12
15
2,000 active
probes
Cisco 3640
Series
Cisco 3725
Router
6
9
13
17
2
3
2
3
Cisco
7200VXR
NPE225
4
3
3
3
19
24
27
28
31
34
38
22
25
28
31
35
38
43
2
3
3
2
2
3
4
3
3
3
4
3
7
8
42
46
48
52
47
49
43
58
5
5
6
6
8
10
11
11
*Jitter operations are activated sequentially with this testing. Each operation sends 10 packets, 64 bytes each with 20ms spacing
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
19
Cisco IOS IP SLA Performance
Infrastructure 2: CPU Load by Hardware
•
Jitter probe
•
Release 12.3(4)T6
•
2,000 active probes
IP Plus/Firewall/3DES
Operations per
second
4
8
12
16
20
24
28
32
36
40
44
48
52
56
60
Cisco IOS IP SLA, and
NetFlow, 9/04
Operations per
minute
240
480
720
960
1200
1440
1680
1920
2160
2400
2640
2880
3120
3360
3600
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco 831
Router
7
13
23
29
33
35
41
47
52
57
62
66
72
76
81
Cisco 837
Router
10
16
23
30
34
36
41
46
50
56
62
65
68
71
75
Cisco Confidential
Cisco 1751
Router
3
8
10
17
22
27
29
32
35
39
43
48
53
59
62
20
Cisco IOS IP SLA VoIP Measurements
Q1CY’05
Data Center
Gatekeeper
Call Manager
Cluster
Registration
Delay
Discovery
Delay
Headquarters
H323 or SIP
Post Dial Delay
Seattle
LA
San Jose
Sales Office Sales Office Sales Office
Cisco IOS IP SLA, and
NetFlow, 9/04
Responder
© 2004 Cisco Systems, Inc. All rights reserved.
New York
Cleveland
Detroit
Cisco Confidential
Boston
Sales Office Sales Office
21
Digital Signal Processor Based IP SLA
Measurements (Q3CY’05)
• VoIP Active (test call) measurements using Real-time Transport Protocol
(RTP) streams
• Voice quality scores and voice metrics from the Digital Signal Processor
(DSP)
Call Control
VoIP Metrics
RTP
IP SLA
DSP
IP Server
Responder
RTP IP
SLA
Cisco IOS IP SLA, and
NetFlow, 9/04
Cisco IOS IP SLA RTP Operation Data
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
22
New IOS IP SLA
CLI
• The new IOS IP SLA CLI releases Q1CY05 in 12.3(RLS6)T
• Phase 1 changes include new syntax for commands and new
show commands
New show commands: “show ip sla statistics” and “ show ip sla
statistics details”
Older show commands will be deprecated over time and replaced
with the new show commands
The RTR keyword was changed to IP SLA Monitor in CLI
The new syntax is used in the presentation. The old syntax
before 12.3(pi6)T is shown in the Appendix
OLD CLI
Router (config)#rtr 1
Router (config-rtr)#type echo protocol
ipIcmpEcho 1.1.1.1
Router (config)#rtr schedule 1 start-time
now
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
New CLI
Router (config)#ip sla monitor 1
Router (config-sla-monitor)#icmp-echo
1.1.1.1
Router (config)#ip sla monitor schedule
1 start-time now
Cisco Confidential
23
New Cisco IOS IP SLA Show Commands
Q1CY’05
• Jitter operation “show ip sla monitor statistics (details)”
Router#sh ip sla monitor statistics 15
Round trip time (RTT) Index 15
Latest RTT: 1 ms
Latest operation start time: *05:43:28.720 UTC Fri May 28 2004
Latest operation return code: OK RTT Values
Number Of RTT: 10
RTT Min/Avg/Max: 1/1/1 ms
Latency one-way time milliseconds
Number of one-way Samples: 0
Source to Destination one way Latency Min/Avg/Max: 0/0/0 ms
Desination to source one way Latency Min/Avg/Max: 0/0/0 ms
Jitter time milliseconds
Number of Jitter Samples: 9
Source to Destination Jitter Min/Avg/Max: 20/20/23 ms
Destination to Source Jitter Min/Avg/Max: 0/0/0 ms
Packet Loss Values
Loss Source to Destination: 0
Loss Destination to Source: 0
Out Of Sequence: 0
Tail Drop: 0 Packet Late Arrival: 0
Number of successes: 1
Number of failures: 0
Operation time to live: 3567 sec
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
24
New Cisco IOS IP SLA Show Commands
Q1CY’05
• Jitter operation “show ip sla monitor statistics details”
Round trip time (RTT) Index 2004
Latest RTT: 1 ms
Latest operation start time: *08:41:09.937 PST Wed Oct 6 2004
Latest operation return code: OK
Over thresholds occurred: FALSE
RTT Values
Number Of RTT: 10
RTT Min/Avg/Max: 1/1/1 ms
Latency one-way time:
Number of Latency one-way Samples: 0
Source to Destination Latency one way Min/Avg/Max: 0/0/0 ms
Destination to Source Latency one way Min/Avg/Max: 0/0/0 ms
Source to Destination Latency one way Sum/Sum2: 0/0
Destination to Source Latency one way Sum/Sum2: 0/0
Jitter time:
Number of Jitter Samples: 9
Source to Destination Jitter Min/Avg/Max: 0/0/0 ms
Destination to Source Jitter Min/Avg/Max: 0/0/0 ms
Source to destination positive jitter Min/Avg/Max: 0/0/0 ms
Source to destination positive jitter Number/Sum/Sum2: 0/0/0
Source to destination negative jitter Min/Avg/Max: 0/0/0 ms
Source to destination negative jitter Number/Sum/Sum2: 0/0/0
Destination to Source positive jitter Min/Avg/Max: 0/0/0 ms
Destination to Source positive jitter Number/Sum/Sum2: 0/0/0
Destination to Source negative jitter Min/Avg/Max: 0/0/0 ms
Destination to Source negative jitter Number/Sum/Sum2: 0/0/0
Interarrival jitterout: 0
Interarrival jitterin: 0
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
25
Cisco IOS IP SLA Multiple Operations
Scheduling (Release 12.3(8)T)
• Schedule multiple operations in one command
• Scalable and sequential activation of IP SLA operations
If the frequency is not specified, the default frequency will be the same as
that of the schedule period)
Reduced load on the network
Consistent monitoring coverage
Router (config)#ip sla monitor 1
Router (config-sla-monitor)#type echo protocol ipIcmpEcho
1.1.1.1
Router (config)# ip sla monitor 2
Router (config-sla-monitor)#type echo protocol ipIcmpEcho
2.2.2.2
Router (config)# ip sla monitor 3
Router (config-sla-monitor)#type echo protocol ipIcmpEcho
3.3.3.3
Router (config)# ip sla monitor group schedule 1 1-3 sch 20
start now
Router #show ip sla monitor group schedule
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
26
Cisco IOS IP SLA Random Scheduler
Enhancement
• Release 12.4(Rls1)T will introduce the following
functionality:
Randomness for group scheduler during schedule period
Randomness for the frequency of the operations, which are
started by random group scheduler
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
27
Cisco IOS IP SLA Accuracy Feature
• High performance and high accuracy
measurements
• Precision to .1 ms from current 1ms
• Improve Cisco IOS IP SLA accuracy under
forwarding load and for dedicated routers
• Release 12.3(RLS6)T will introduce this
functionality in Q1CY’05
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
28
Cisco IOS IP Service Level Agreement
Roadmap
Feature
Release
Target Date
Release 12.3T Features
MOS and ICPIF Scores
12.3(4)T
November 2003
One way latency, jitter, packet loss and MOS Traps
12.3(7)T
March 2003
Multi-Operation Scheduler – Ease of scheduling
12.3(8)T
June 2003
Post Dial and Gatekeeper Delays with SIP and H323
12.3(pi-6)T
Q1CY’05
High accuracy enhancement
12.3(pi-6)T
Q1CY’05
Ease of use CLI
12.3(pi-6)T
Q1CY’05
Release 12.4T Features
Ease of use CLI Phase 2
12.4(pi-1)T
Q2CY’05
Random scheduler for operations
12.4(pi-1)T
Q2CY’05
Voice gateway integration VoIP measurement using DSP
12.4(pi-2)T
Q3CY’05
Ease of use CLI Phase 3
12.4(pi-2)T
Q3CY’05
Video operation
12.4(pi-2)T
Q3CY’05
Radius response operation
12.4(pi-2)T
Q3CY’05
Release 12.2S Features
IP SLA: Auto MPLS VPN Monitoring
12.2(Rls6)S
Q1CY’05
IP SLA: Auto MPLS VPN Monitoring with ECMP
12.2(Rls7)S
Q3CY’05
IP SLA: Auto MPLS Monitoring with VCCV
12.2(Rls8)S
Radar
IP SLA: Auto MPLS Monitoring with BFD
12.2(Rls8)S
Radar
Radar
IP SLA Multicast
Radar
Radar
Auto IP SLA Monitoring
Radar
Radar
IP SLA with DMVPN
Radar
Radar
ICMP Jitter
Radar
Radar
IP SLA High Availability
Radar
Radar
Cisco IOS IP SLA, and
NetFlow, 9/04
Embedded
2004 Cisco Systems,
All rights reserved.
Event ©Manager
(EEM)Inc.Detector
Radar
Cisco Confidential
Radar
29
NetFlow
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
30
Flow Is Defined By Seven Unique Keys
• Source IP address
• Destination IP address
Traffic
Enable NetFlow
New
SNMP MIB
Interface
• Source port
• Destination port
• Layer 3 protocol type
• TOS byte (DSCP)
• Input logical interface
(ifIndex)
NetFlow
Export
Packets
Traditional Export &
Collector
SNMP Poller
GUI
Cisco
Cisco
IOS
IOS
IPNetFlow
SLA, and
NetFlow,
Overview,
9/04
2/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
31
NetFlow Cache Example
1. Create and update flows in NetFlow cache
Srclf
SrclPadd
Dstlf
DstlPadd
Protocol
TOS
Flgs
Pkts
Src
Port
Src
Msk
Src
AS
Dst
Port
Dst
Msk
Dst
AS
NextHop
Bytes/
Pkt
Active
Idle
Fa1/0
173.100.21.2
Fa0/0
10.0.227.12
11
80
10
11000
00A2
/24
5
00A2
/24
15
10.0.23.2
1528
1745
4
Fa1/0
173.100.3.2
Fa0/0
10.0.227.12
6
40
0
2491
15
/26
196
15
/24
15
10.0.23.2
740
41.5
1
Fa1/0
173.100.20.2
Fa0/0
10.0.227.12
11
80
10
10000
00A1
/24
180
00A1
/24
15
10.0.23.2
1428
1145.5
3
Fa1/0
173.100.6.2
Fa0/0
10.0.227.12
6
40
0
2210
19
/30
180
19
/24
15
10.0.23.2
1040
24.5
14
•
•
•
•
2. Expiration
Inactive timer expired (15 sec is default)
Active timer expired (30 min (1800 sec) is default)
NetFlow cache is full (oldest flows are expired)
RST or FIN TCP Flag
Srclf
SrclPadd
Dstlf
DstlPadd
Protocol
TOS
Flgs
Pkts
Src
Port
Src
Msk
Src
AS
Dst
Port
Dst
Msk
Dst
AS
NextHop
Bytes/
Pkt
Active
Idle
Fa1/0
173.100.21.2
Fa0/0
10.0.227.12
11
80
10
11000
00A2
/24
5
00A2
/24
15
10.0.23.2
1528
1800
4
3. Aggregation
e.g. Protocol-Port Aggregation
Scheme Becomes
4. Export version
5. Transport protocol
Cisco IOS IP SLA, and
NetFlow, 9/04
Export
Packet
© 2004 Cisco Systems, Inc. All rights reserved.
Heade
r
Non-Aggregated Flows—Export Version 5 or 9
Payload
(Flows)
Protocol
Pkts
SrcPort
DstPort
Bytes/Pkt
11
11000
00A2
00A2
1528
Aggregated Flows—Export Version 8 or 9
Cisco Confidential
32
Principle Netflow Benefits
Service Provider
Enterprise
• Traffic Engineering
• Internet access
monitoring (protocol
distribution, where traffic
is going/coming)
• Accounting and billing
• User Monitoring
• Security Monitoring
• Application Monitoring
• Peering arrangements
• Network Planning
• Charge Back billing for
departments
• Security Monitoring
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
33
Tracking Users
Who are the top users?
How long are the users on the network?
What Internet sites do they use?
Where do the users go on the network?
What percentage of traffic do they use?
What applications do they use?
What are the user usage patterns?
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
34
NetFlow for Security:
Flow Information Helps Mitigate Attacks
• Identify the attack
Count the Flows
Inactive flows signal a worm attack
• Classify the attack
Small size flows to same destination
What is being attacked and origination of attack
• Key Partners: Arbor Networks, Protego, NetQos, Adlex
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
35
Capacity Planning
• Capacity planning is the process of determining the network
resources required to prevent a performance or availability
impact on business-critical applications
• Key areas to monitor
Application usage
Identify which applications consume bandwidth
Who are the top ten nodes that consume bandwidth
• Output data circuit forecasts
• Current network utilization and capacity being used
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
36
Billing
• IP Accounting and Billing
• Usage-based billing considerations
Time of day
Within or outside of the network
Application
Distance-based
Quality of Service (QoS) / Class of Service (CoS)
Bandwidth usage
Transit or peer
Data transferred
Traffic class
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
37
How Cisco IT uses NetFlow
• Characterize IP traffic and account for how and where it flows
Total Avoidance of SQL Slammer Worm
Transition from Managed DSL service to Internet VPN
Detection of Unauthorized WAN Traffic
Reduction in Peak WAN Traffic
Validation of QoS Parameters and BW allocation
Analysis of VPN Traffic and Tele-Commuter Behavior
Calculating Total Cost of Ownership for Applications
Use of NetFlow
NMS and Usage
Security Monitoring
Network traffic analysis by application with BGP.
Anomaly detection Arbor Networks
WAN Aggregation and Edge
Network traffic analysis by application, for capacity
planning using NetQOS
Core routers and Nat Gateway
Collection of historical data, useful for forensics and
diagnostics with Flow Tools
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
38
Comprehensive Hardware Support
Enterprise & Aggregation/Edge
Core
Cisco IOS Software Release 12.2S
Cisco 7200
Series
Cisco 7300
Series
Cisco Catalyst
Cisco 10000 6500; Cisco
Series ASIC
7600 Series
Cisco 4500
Series ASIC
Release 12.0S
Cisco 12000
Series
ASIC
Access
Cisco IOS Software Releases 12.3T & 12.4
Cisco 800 Cisco 1700
Series
Series
Cisco IOS IP SLA, and
NetFlow, 9/04
Cisco 2600
Series
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco 3700
Series
Cisco 7200/
7300 Series
Cisco Confidential
39
NetFlow Versions
NetFlow
Version
1
5
7
8
9
Comments
Original
Standard and most common
Specific to Cisco Catalyst 6500 and 7600
Series Switches
Similar to Version 5, but does not include AS,
interface, TCP Flag & TOS information
Choice of eleven aggregation schemes
Reduces resource usage
Flexible, extensible file export format to
enable easier support of additional fields &
technologies; coming out now MPLS,
Multicast, & BGP Next Hop
Cisco Catalyst 6500 Series Router will support
versions 5 & 8 in Cisco IOS Software Release 12.1(13)E
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
40
Version 5 - Flow Export Format
Usage
• Packet Count
• Byte Count
Source IPIPAddress
•• Source
Address
Destination IPIP
Address
•• Destination
Address
From/To
Time
of Day
• Start sysUpTime
• End sysUpTime
• Source TCP/UDP Port
• Destination TCP/UDP Port
Application
Port
Utilization
• Input ifIndex
• Output ifIndex
QoS
• Type of Service
• TCP Flags
• Protocol
• Next Hop Address
• Source AS Number
• Dest. AS Number
• Source Prefix Mask
• Dest. Prefix Mask
Routing
and
Peering
Version 5 used extensively today
Flow information
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
41
Why a New Version 9?
• Fixed export formats are not flexible and adaptable
• With each new version Cisco creates new export
fields
• Partners need to re-engineer for each new version
Solution: Build a flexible and extensible
export format called version 9!
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
42
NetFlow v9 Export Packet
To support technologies such as
MPLS or Multicast, this export format can
be leveraged to easily insert new fields
Template FlowSet
(version,
# packets,
sequence #,
Source ID)
Template
Record
Template ID #1
Template
Record
Template ID #2
(specific Field
types and
lengths)
(specific Field
types and
lengths)
Flows from
Interface A
Flows from
Interface B
Data FlowSet
Data FlowSet
FlowSet ID #1
FlowSet ID #2
Data Record
(Field values)
Option
Template
FlowSet
Data Record
Data Record
Template ID
(Field values)
(Field
values)
(specific
Field types
and lengths)
Option Data
FlowSet
FlowSet ID
Option
Data
Record
Option
Data
Record
(Field
values)
(Field
values)
• Matching ID numbers are the way to associate template to the Data Records
• The Header follows the same format as prior NetFlow versions so Collectors will be
backward compatible
• Each data record represents one flow
• If exported flows have the same fields, then they can be contained in the same
Template Record (ie: unicast traffic) can be combined with multicast records
• If exported flows have different fields, then they cannot be contained in the same
Template Record (ie: BGP next-hop cannot be combined with MPLS Aware NetFlow
Cisco IOSrecords)
IP SLA, and
Cisco Confidential
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
43
NetFlow v9 and IETF
• Internet Protocol Flow Information eXport (IPFIX) is an
IETF Working Group
www.ietf.org/html.charters/ipfix-charter.html
• Netflow version 9 is the basis for the standard in the
IETF
• Standards Track NetFlow version 9
http://www.ietf.org/internet-drafts/draft-ietf-ipfix-protocol-05.txt
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
44
IETF: Packet SAMPling WG (PSAMP)
• PSAMP web site for the charter, email archive,
drafts, etc. psamp.ccrle.nec.de/
• Agreed to use IPFIX for export protocol if suitable
for PSAMP
To be improved: the variable length data type
• Note: NetFlow is already using some sampling
mechanisms
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
45
NetFlow Partners
Traffic Analysis
Flow-Tools
Denial of Service
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Billing
Cisco Confidential
46
Cisco Catalyst 6500 Series Switch and
Cisco 7600 Series Router
• Hybrid: Cisco Catalyst OS on PFC/supervisor and Cisco IOS
software on MSFC
• Native Cisco IOS Software: PFC/supervisor and the MSFC
both run a single bundled Cisco IOS software image
• Export is centrally via the supervisor and MSFC, each linecard
has its own hardware NetFlow cache and forwarding table, i.e.
distributed platform
Hybrid
Native 12.1E
Native 12.2SX
MSFCx
v5
v5
v5, v8*
Sup1a
V7, v8
v7
N/A
Sup2
V7, v8
v5, v7
v5, v7, v8
v5, v7, v8
v5, v7
v5, v7, v8
Sup720
*No NetFlow Support on MSFC with Sup1a
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
47
Cisco Catalyst 6500 and Cisco 7600 Series
Versions and Features
• Cisco IOS Software Release 12.1(13)E1
PFC2 Source/destination interface information (Hybrid 6.3(6))
PFC2 Source/destination AS information
PFC2 Support for V5 NetFlow data export (Hybrid 7.5(1))
IP Next hop
Sampled NetFlow is available on PFC in Cisco IOS
• Cisco IOS Software Release 12.2(14)SX
Version 8 in native mode
• PFC3b (Sup720) cards
ToS byte
• Hybrid Catalyst OS 7.2(1)
L2 switched traffic (vlan x to vlan y) support (doesn’t require MSFC)
• Hybrid Catalyst OS 7.3(1)
Destination and source IfIndex enabled by default
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
48
Cisco Catalyst 4000 Supervisor IV
NetFlow Services Card
NetFlow Service Card Features
• NetFlow Statistics Collection
and Data Export (NDE)
•VLAN Statistics Collection
•CLI support for NetFlow & VLAN Stats
•SNMP support for VLAN Stats
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
• Requirements:
• Supervisor IV or V
• IOS 12.1(13)EW
• NetFlow Versions 1 & 5, 8
w IOS 12.1.19 EW
Cisco Confidential
49
NetFlow Features supported with Version 9
• Multicast NetFlow
Availability: Major Release 12.3(1) and 12.2(18)S
Ingress Accounting of replicated multicast packets
Egress Per user accounting of multicast packets
• MPLS Aware NetFlow
Availability: Release 12.0(26)S
Label and prefix export information
• BGP Next Hop
Availability: Releases 12.0(26)S, 12.2(18)S, and 12.3
Edge to Edge Traffic Matrix
BGP traffic destination information
• NetFlow for IPv6
Availability: Release 12.3(7)T
Export IPv6 source and destination information
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
50
NetFlow Product Update
• Sampled NetFlow
Availability: Releases 12.0(26)S, 12.3(2)T, and 12.2(18)S
Random Sampling of packets per flow with reduce CPU
• NetFlow MIB
Availability: Releases 12.3(7)T and 12.2(25)S
Top N Talker in MIB
NetFlow configuration using MIB
• Input Flow Filters
Availability: Release 12.3(7)T, 12.2(25)S
QOS MQC based Filtering entering NetFlow
• Egress NetFlow
Availability: Release 12.3(11)T, 12.2(Rls6)S-Q1CY05
Accounting for Egress IP Flows
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
51
Random Sampled NetFlow
• Capacity planning may not need every packet per Flow
• Sampling on high speed interfaces will reduce CPU consumption
• Random (select packet to export per statistical principles)
Cisco IOS Software Releases 12.0(26)S, 12.2S(18), and 12.3(1)T
Cisco 800, 1700, 1800, 2600, 2800,3600, 3700, 3800 7200, and 7500
Series Routers
Random sampling Cisco 12000 Series 12.0(28)S
Cisco 12000 Series deterministic sampling today
Cisco Catalyst 6500 Series Random and Time based sampling
12.1(13)E
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
52
NetFlow MIB
• Currently available in Releases 12.3(7)T
• NetFlow information available using SNMP and without NetFlow export
• Administration of Netflow using the MIB interface
• NetFlow MIB cannot be used to retrieve all Flow information but is very useful
for security monitoring and locations where export is not possible
Example objects available:
Packet size distribution
Number of Bytes exported per second
Number of flowsNetFlow MIB with Export of Top N talkers
• Top N Talkers
Top N Flows based on various NetFlow field values ( AS Number, destination,
ports…)
MIB and CLI support
12.2(25)S and 12.3(11)T
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
53
Import Flow Mask Filters
• Prevent flows from entering NetFlow cache by using Flow Filter
• Increase scalability and decrease CPU usage
• Filters are based on QOS MQC CLI class maps
• User can use ACL to match flows from certain port or source
• Define Traffic Class (match ACL) and Flow Sampling per Match
12.0(27)S, 12.3(4)T, 12.2S(25)
Traffic Filter
High Importance
Sample 1:1 from Server B
Packets
Traffic Filter
Low Importance
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Sample 1:100 from Subnet A
Cisco Confidential
54
Egress NetFlow Accounting
Netflow
Egress and Ingress
PE
PE
IP
Servers
IP
IP or MPLS
Netflow
Ingress
Netflow
Egress
12.3(7)T, 12.2(25)S
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
55
Flexible NetFlow and Flexible Accounting
• Flexible NetFlow and Flexible Accounting will replace
most static accounting technologies available today
Flexible NetFlow user defined Flow keys and export
fields within NetFlow
Flexible Accounting user defined permanent flow
with periodic export and account for defined flows
over time
The data can be polled thru a MIB
Flow Groups user defined buckets for specific flow
fields values
Example show me packets and bytes from 1.1.1.1 to 2.2.2.2
on port 21
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
56
SCTP Reliable Transport
• Flows may be sent in Reliable or unreliable or partial mode
• SCTP connection to collector and multiple streams per connection
• Supported with Version 9. Templates may be sent reliably
• Congestion Awareness, retransmission and queuing
Send Queue
Releases 12.4(2nd)T, 12.2S(Rls7)
Data for Export in SCTP Stream
Cisco IOS IP SLA, and
NetFlow, 9/04
Congestion packets marked
unreliable
potentially dropped
© 2004 Cisco Systems, Inc. All rights reserved.
Collector
Cisco Confidential
57
NetFlow Security Enhancement
Releases 12.4(1st)T Q2CY05
• New show commands to understand and parse NetFlow data
For Example, show flows on port X to destination Y
show ip flow top <N> <aggregate-field> <sort-criteria> <matchcriteria>
show ip flow top 10 destination-address packets interface ser0
port-range 100 to 135
• New Flow export fields including Source Mac, TTL, Packet
length, ICMP type, and more
• Also will be available in 12.2(rls7)S
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
58
Upcoming New Features:
NetFlow Product Update
• NetFlow Security Enhancements (Q2CY2005)
New exports and show commands for security monitoring
• Flexible NetFlow and Accounting (Q3CY2005)
Allow user defined flow keys and aggregation with v.9
• Reliable and Congestion Aware Export (Q2CY2005)
SCTP protocol NetFlow export
• NBAR and NetFlow Integration (Radar)
Application flow information export
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
59
NetFlow Roadmap
Enhancing Cisco
technologies’ with
Flow Accounting
Scalability &
Flexibility
Nov
2003
Dec
2003
Jan
2004
Feb
2004
Mar
2004
Apr
2004
May
2004
Jun
2004
Optimizing data
for Flow
processing
Jul
2004
Aug
2004
Sep
2004
Oct
2004
• Input Filter
• NetFlow MIB
& Top Talker
Targeting
12.3(11)T
• NetFlow IPv6
Cisco IOS IP SLA, and
NetFlow, 9/04
Dec
2004
Jan
2005
Feb
2005
Mar
2005
• Egress
NetFlow
Targeting
12.3(2)T
• Input
Filter
Nov
2004
Targeting
12.2(Rls6)S
12.0(27)S
12.3(Rls2)T
Standardization
• Egress
NetFlow
• Flexible Flow
Definition
Reliable
Export
Targeting
12.2(25)S
• NetFlow MIB &
Top Talker
Targeting
12.4(Rls1)T
• Input Filter
Security
Exports
© 2004 Cisco Systems, Inc. All rights reserved.
Targeting
12.2(Rls7)S
Cisco Confidential
Security
Exports
MIB Phase 2
60
Cisco IOS IP SLA, and
NetFlow, 9/04
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
61