Download 31004011 VidTran10 TIA-921

Telecommunications
Industry
Association
Telecommunications
Industry
Association
By Teleconference
April 26, 2010
TR-30.3/10-04-011
TR41.N.n-YY-MM-XXX
Document Cover Sheet
Project Number
PN-3-0062-RV2
Document Title
31004011 VidTran10 TIA-921 - G.1050 Presentation
Source
PacketStorm
Contact
Name: Jack Douglass
Complete Address:
Distribution
TR-30.3
Intended Purpose
of Document
(Select one)
X
Phone:
Cell: 949-436-0663
Home/Office: 949-480-1220
Fax: 949-480-1220
Email: [email protected]
For Incorporation Into TIA Publication
For Information
Other (describe) - Liaison
The document to which this cover statement is attached is submitted to a Formulating Group or
sub-element thereof of the Telecommunications Industry Association (TIA) in accordance with the
provisions of Sections 6.4.1–6.4.6 inclusive of the TIA Engineering Manual dated October 2009, all
of which provisions are hereby incorporated by reference.
Abstract
Video Services Forum (VSF) VidTrans10 presentation.
Standards Based Network Model for Evaluating
Multimedia Transmission Performance Over IP
Jack Douglass
VP of Marketing and Business Development
Chairman of TIA TR30.3 Committee
Agenda
• Standards Based IP Network Model
– ITU-T G.1050 / ANSI TIA-921
• G.1050-2007 / TIA-921A
– Top Down Approach
• G.1050–201X / TIA-921B (proposed revision)
– Bottom-Up Approach
• Example Results
• Applications for G.1050/TIA-921 IP Network Model
Standards Based IP Network Model
ITU-T G.1050 / ANSI TIA-921
• Committees
– TIA TR30.3 Committee
– ITU-T SG12 Question 14
• Range of test scenarios
– Video, VoIP and Data
– “Real-World” Network Characteristics
• Service Level Agreements (SLAs) based
impairments (ITU-T Y.1541)
– Well-Managed
– Partially-Managed
– Unmanaged conditions
Top Down Approach
G.1050-2007 / TIA-921A (Current Model)
• Surveyed many networks to evaluate the jitter and loss
characteristics (Bursty, Not Random)
• Level of the impairment characteristics were adjusted to match the
service levels in Y.1541
• Created impairment combinations based on Impairment Severity
Levels and LAN/Access Rates
• Multiple Two State Time Series Modified Gilbert-Elliott / Markov
Models were used to emulate loss and delay characteristics
– Emulated bursty packet loss and jitter that are similar to the surveyed
network characteristics
– Time based model with pre-calculated delay and loss
Network Impairments
• Network Architecture
• Types of Access Links
• QoS controlled Edge
Routing
• MTU Size
• Packet Loss (Frame
Loss)
• Out of order packets
• One Way Delay
(Latency)
• Variable Delays (Jitter)
• Occupancy (Background
Traffic, Congestion, Network
Load)
• Route Flapping
• Network faults
• Link Failures
Network Topologies
ITU-T G.1050-2007 / TIA-921A
DUT
A
LAN A
LAN Rates
----------------Occupancy level
Packet loss
Local
Access
A
Core IP
Network
Route flapping
Local
Access
B
LAN B
Access Rates
Link Failures
Access Rates
LAN Rates
----------------
One-way delay
----------------
-----------------
Occupancy level
Jitter
Occupancy level
Occupancy level
Qos Edge Router
Packet loss
Qos Edge Router
Packet loss
Packet Reorder
LAN-to-LAN IP Network Model
DUT
A
Core IP
Network
Route flapping
Local
Access
B
LAN B
Link Failures
One-way delay
Access Rates
LAN Rates
Jitter
----------------
-----------------
Packet loss
Occupancy level
Occupancy level
Packet Reorder
Qos Edge Router
Packet loss
Core-to-LAN
DUT
B
DUT
B
Test Profiles based on ITU-T Y1541
ITU-T G.1050-2007 / TIA-921A
Different test profiles for different Service Level Agreements (SLAs)
Impairment Type
Units
Profile A
Well-Managed
Range (min to max)
Profile B
Partially-Managed
Range (min to max)
Profile C
Unmanaged
Range (min to max)
One Way Latency
ms
20 to 100 (regional)
90 to 300 (intercontinental)
20 to 100 (regional)
90 to 400 (intercontinental)
20 to 500
Jitter (peak to peak)
ms
0 to 50
0 to 150
0 to 500
Sequential Packet Loss
ms
Random loss only (except
when link failure occurs)
40 to 200
40 to 10,000
Rate of Sequential Loss
sec-1
Random loss only (except
when link failure occurs)
< 10-3
< 10-1
Random Packet Loss
%
0 to 0.05
0 to 2
0 to 20
Reordered Packets
%
0 to 0.001
0 to 0.01
0 to 0.1
8 Impairment Severity Combinations
ITU-T G.1050-2007 and TIA-921A
Severity=>
Impairment
A
B
C
D
E
F
G
H*
Units
Profie A LOO
%
50
30
15
5
0
0
0
0
Profile B LOO
%
5
25
30
25
10
5
0
0
Profile C LOO
%
5
5
10
15
20
25
15
5
Source Location (A) Parameters
LAN A Occupancy
%
1
2
3
5
8
12
16
20
Access A Occupancy
%
0
1
2
4
8
15
30
50
512
512
1508
1508
1508
1508
1508
1508
MTU A
bytes
Core Network Impairments
Route flap interval
seconds
0
3600
1800
900
480
240
120
60
Route flap delay
ms
0
2
4
8
16
32
64
128
Delay
ms
4
8
16
32
64
128
256
512
Jitter (peak to peak)
ms
5
10
24
40
70
100
150
500
Link fail interval
seconds
0
3600
1800
900
480
240
120
60
Link fail duration
ms
0
64
128
256
400
800
1600
3000
Packet loss
%
0
0.01
0.02
0.04
0.1
0.2
0.5
1
Reordered packets
%
0
0.00025
0.0005
0.001
0.005
0.01
0.05
0.1
Destination Location (B) Parameters
Access B Occupancy
MTU B
LAN B Occupancy
%
bytes
%
0
1
2
4
8
15
30
50
512
512
1508
1508
1508
1508
1508
1508
1
2
3
5
8
12
16
20
Impairment Severity Combinations
LAN-Access Rate Combinations
ITU-T G.1050-2007 and TIA-921A
15 Core-to-LAN (Excluding IPTV≤ 3 Mbit/s) Rate Combinations
120 Impairment Combinations (15 rates * 8 Severities)
168 LAN-to-LAN Scenario Rate Combinations
1344 Impairment Combinations (168 rates * 8 Severities)
6 IPTV Core-to-LAN (> 3 Mbit/s) Rate Combinations
48 Impairment Combinations (6 rates * 8 Severities)
Testing with Fixed Levels of IP Impairments
Fixed
Packet
Jitter
Fixed
Jitter &
Packet
Loss
• Real IP Network Impairments Vary Over Time significantly over
time and are bursty in nature
• IP Impairment network emulators exist today that provide fixed
or semi-fixed levels of IP impairments.
Real World Impairment Characteristics
Good
Bad
Route Flap
Link Failure
Severe
1344 General Model
120 Core to LAN ≤ 3 Mbit/s
48 Core to LAN > 3 Mbit/s
IP Impairments IPTV Testing
G.1050-2007/TIA-921A
1XA
1XB
1XC
1XD
1XE
1XF
1XG
1XH
48 Core to LAN > 3 Mbit/s
Impairment Combinations
Reasons Revising G.1050-2007 / TIA-921A
• Increased Realism
• Keep in step with evolving IP network
• Reduced number of test cases
–
–
–
•
•
•
•
Small set of Standard Long Duration Test Cases
Focused Test Cases for transient impairments such as link failures and route flaps
Short burst tests (15 seconds) for voice/video subjective testing and fax
True bi-directional model
Ability to test with mixed traffic
Statistically repeatable tests
Ability for users to customize test cases and use their own .pcap
files to model interferers.
• Methods for verifying that test equipment implementations match
the model
Bottom-Up Approach
G.1050-201X / TIA-921B (Revised Model)
•
Model interfering traffic
–
–
–
•
Model the Mechanisms that cause Jitter, Delay and Loss
–
•
Capture different types of real network traffic
Create Packet Delay Variation (PDV) Histograms from captured traffic
Make interferers scalable
Packet loss and delay are caused by interferers and self-interferers
Model of each Network Element (Core, Access, LAN)
–
–
Capture characteristics of each Network Elements
Capture characteristics of Access and LAN technologies
•
–
–
•
•
Bandwidth Throttling, Smoothing and Shaping Effects of network elements
Bi-directional characteristics
Model Quality of Service (QoS)
Adjust and scale the amounts and types of interfering traffic
–
–
•
DSL, Cable, Fiber Optic, WiMAX, LTE; LAN – Wi-Fi, Ethernet, HPNA
Match Surveyed Network Characteristics
Match Service Levels in Y.1541
A similar model is being considered in ITU-T SG15 Question 13 – G.8261
Test Profiles based on ITU-T Y1541
ITU-T G.1050-201X / TIA-921B (Revised Model)
Different test profiles for different Service Level Agreements (SLAs)
Impairment Type
Units
Profile A
Well-Managed
Range (min to max)
Profile B
Partially-Managed
Range (min to max)
Profile C
Unmanaged
Range (min to max)
One Way Latency
ms
20 to 100 (regional)
90 to 300 (intercontinental)
20 to 100 (regional)
90 to 400 (intercontinental)
20 to 500
Jitter (peak to peak)
ms
0 to 50
0 to 150
0 to 500
Sequential Packet Loss
ms
Random loss only (except
when link failure occurs)
40 to 200
40 to 10,000
Rate of Sequential Loss
sec-1
Random loss only (except
when link failure occurs)
< 10-3
< 10-1
Random Packet Loss
%
0 to 0.05
0 to 2
0 to 20
Reordered Packets
%
0 to 0.001
0 to 0.01
0 to 0.1
Impairment Combination Table
Network Element
QoS
Residential Impairment Combinations in the direction from the core
Well Managed
Partially Managed
Impairment
Sev1
Sev2
Sev3
Sev4
Sev5
Sev6
IPTV
VoIP (primary)
Data
Test stream
2
1
3
Y
2
1
3
Y
2
1
3
Y
2
1
3
?
2
2
3
?
Sev7
2
3
3
N
Unmanaged
Sev8
Sev9
3
3
3
N
3
3
3
N
3
3
3
N
LAN - wired
Effective Rate (Mbit/s)
Peer-to-peer
POP3
HTTP
Web video
VoIP (secondary)
FoIP (secondary)
Gaming
SAP
100
0
0
0
0
0
0
0
0
100
0
0
1
0
1
0
0
0
100
1
1
1
0
0
1
0
0
100
Effective Rate (Mbit/s)
Peer-to-peer
POP3
HTTP
Web video
VoIP (secondary)
FoIP (secondary)
Gaming
SAP
20
0
1
1
0
0
0
0
0
20
0
1
1
0
0
0
1
0
20
0
1
2
1
1
0
1
1
4
100
100
100
100
LAN - wireless
Firewall
Buffer size
Rate (Mbit/s)
Proc delay (µs)
64 KB
Buffer size
Proc delay
VoIP (primary)
FoIP (primary)
IPTV service
64 KB
64 KB
100
100
64 KB
10
100
100
"Modem"
Access (pick one technology)
DSL
Access Rate (Mbit/s)
Cable
Access Rate (Mbit/s)
ONT
Access Rate (Mbit/s)
64 KB
64 KB
0
0
0
33
100
100
22
100
100
14
50
100
7
100
100
100
100
Network Topologies
ITU-T G.1050-201X / TIA-921B (Revised Model)
LAN-to-LAN IP Network Model
Core-to-LAN
LAN and Access Block Diagram
ITU-T G.1050-201X / TIA-921B (Revised Model)
Edge Router
DSLAM /
CMTS /
OLT
DSL Modem /
Cable Modem /
ONT
Firewall
LAN
Wired
Wireless
Residential Gateway
Network Interferers
Test Streams
Network Interferers
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
TCP
Peer-to-Peer
POP3 email protocol
HTTP web browsing
Web Video (You Tube, Hulu)
VoIP
FoIP
Gaming
IPTV
Web Video
VoIP / FoIP
DATA
TCP
Peer-to-Peer
POP3 email protocol
HTTP web browsing
Web Video (You Tube, Hulu)
VoIP
FoIP
Gaming
Capturing “Real-World” Characteristics
• Capture Packet Delay Variation (PDV) characteristics
–
–
Access and LAN Technologies
Different types of Interferers (network traffic ) using .pcap (from Wireshark)
• Result is composite Packet Delay Variation (PDV) for combined
Access/LAN and Interferers
Basic Model Element
ITU-T G.1050-201X / TIA-921B
Disturbance
load
Disturbance
generator
load
Disturbance
generator
load
generator
Input
packets
Store/
Fwd
Link
Latency
+
Output
packets
Simulated Packet
Queue with Loss
21
Network Model Structure (Simplified)
Stream
Under
Test
DSLAM/
CMTS
OLT
Edge
Router
DSL Modem/
Cable Modem/
ONT
20Mb/s
or
30Mb/s
1G
1G
Firewall/
Router
LAN
100M
100M
STB
1ms
100ns
IPTV
64kB
buffer
64kB
buffer
0ns
64kB
buffer
64kB
buffer
Stream
Under
Test
HTTP
HTTP
POP3
Interferers
From PCAP
Composite Interference
HDTV
HTTP Interference Characteristics
Delay (milliseconds)
Packet Delay Variation (PDV) Histogram
Delay vs. Time
IPTV Interference (VBR) Characteristics
Delay (milliseconds)
Packet Delay Variation (PDV) Histogram
Delay vs. Time
POP3 Interference Characteristics
Delay (milliseconds)
Packet Delay Variation (PDV) Histogram
Delay vs. Time
Example #1
Edge Router
DSLAM /
CMTS /
OLT
DSL Modem /
Cable Modem /
ONT
Firewall
LAN
Wired
Wireless
Residential Gateway
Access
Network Interferers
PC#1, HTTP web browsing – QoS 2
PC#2, POP3 email protocol – QoS 2
STB#2, IPTV – SD (2 Mb/s) – QoS 1
STB#3 , IPTV – SD (2 Mb/s) – QoS 1
DSL
20Mb/s
Test Streams
STB#1 IPTV – SD 2Mb/s – QoS 1
Network Interferers
PC#1, HTTP web browsing – QoS 2
PC#2, POP3 email protocol – QoS 2
STB#2, IPTV – SD (2 Mb/s) – QoS 1
STB#3 , IPTV – SD (2 Mb/s) – QoS 1
Example #1
Network Interferers
PC#1, HTTP web browsing – QoS 2
Test Streams
STB#1, IPTV – SD 2Mb/s – QoS 1
PC#2, POP3 email protocol – QoS 2
Access
DSL
20Mb/s
STB#2, IPTV – SD (2 Mb/s) – QoS 1
STB#3 , IPTV – SD (2 Mb/s) – QoS 1
Example #1
Test Stream
Packet Delay Variation (PDV) Histogram
Test Streams
STB#1, IPTV – SD 2Mb/s – QoS 1
Access
DSL
20Mb/s
Delay vs. Time
Example #2
Edge Router
DSLAM /
CMTS /
OLT
DSL Modem /
Cable Modem /
ONT
Firewall
LAN
Wired
Wireless
Residential Gateway
Access
Network Interferers
PC#1, HTTP web browsing – QoS 2
PC#2, POP3 email protocol – QoS 2
STB#2, IPTV – SD (2 Mb/s) – QoS 1
STB#3 , IPTV – SD (2 Mb/s) – QoS 1
DSL
20Mb/s
Test Streams
STB#1, IPTV – HD 8Mb/s – QoS 1
Network Interferers
PC#1, HTTP web browsing – QoS 2
PC#2, POP3 email protocol – QoS 2
STB#2, IPTV – SD (2 Mb/s) – QoS 1
STB#3 , IPTV – SD (2 Mb/s) – QoS 1
Example #2
Network Interferers
PC#1, HTTP web browsing – QoS 2
Test Streams
STB#1, IPTV – HD 8Mb/s – QoS 1
PC#2, POP3 email protocol – QoS 2
Access
DSL
20Mb/s
STB#2, IPTV – SD (2 Mb/s) – QoS 1
STB#3 , IPTV – SD (2 Mb/s) – QoS 1
Example #2
Test Stream
Packet Delay Variation (PDV) Histogram
Test Streams
STB#1, IPTV – HD 8Mb/s – QoS 1
Access
DSL
20Mb/s
Delay vs. Time
Example #3
Edge Router
DSLAM /
CMTS /
OLT
DSL Modem /
Cable Modem /
ONT
Firewall
LAN
Wired
Wireless
Residential Gateway
Access
Network Interferers
PC#1, HTTP web browsing – QoS 2
PC#2, POP3 email protocol – QoS 2
STB#2, IPTV – SD (2 Mb/s) – QoS 1
STB#3 , IPTV – SD (2 Mb/s) – QoS 1
DSL
30Mb/s
Test Streams
STB#1, IPTV – HD 8Mb/s – QoS 1
Network Interferers
PC#1, HTTP web browsing – QoS 2
PC#2, POP3 email protocol – QoS 2
STB#2, IPTV – SD (2 Mb/s) – QoS 1
STB#3 , IPTV – SD (2 Mb/s) – QoS 1
Example #3
Network Interferers
Test Streams
PC#1, HTTP web browsing – QoS 2
PC#2, POP3 email protocol – QoS 2
STB#1, IPTV – HD 8Mb/s – QoS 1
Access
DSL
30Mb/s
STB#2, IPTV – SD (2 Mb/s) – QoS 1
STB#3 , IPTV – SD (2 Mb/s) – QoS 1
Example #3
Test Stream
Packet Delay Variation (PDV) Histogram
Test Streams
STB#1, IPTV – HD 8Mb/s – QoS 1
Access
DSL
30Mb/s
Delay vs. Time
Applications
G.1050/TIA-921– Compare the Encoders and STBs
A
Imp Gen
A
B
B
C
C
Video
Encoders
STBs
G.1050-2007 / TIA-921A Impairments
o Compatibility of Encoder and Decoder
o Effectiveness Decoders to conceal network errors
o Stability of STB under network conditions
o Regression Testing
o Standard, Definitive, Repeatable set of tests
representing Real-World IP Network Conditions
Applications
G.1050/TIA-921– Emulate Field Problems, Error Correcting, Server Loading
Imp Gen
RUDP
A
Server
D
Server
G.1050-2007 / TIA-921A Impairments
o Emulate field problems
o Emulate effectiveness of error correcting scheme
o Test Loading of A&D Server in presence of Network Errors
o Test ability of system to meet required specifications
• Standard, Definitive, Repeatable set of tests representing Service Levels
Applications
G.1050/TIA-921 – Effectiveness of FEC
Imp Gen
Video
Server
FEC
Equipment
FEC
Equipment
G.1050-2007 / TIA-921A Impairments
o Emulate effectiveness of Forward Error Correcting (FEC) Techniques and Equipment
o Test ability of FEC to meet required specifications
• Standard, Definitive, Repeatable set of tests representing Real-World IP Network
Conditions
Applications
G.1050/TIA-921 – General Network Emulation
DUT
A
Imp Gen
Test Any Type of Network Equipment under
Real-World Network Characteristics
DUT
DUT
B
B
TR30.3 and ITU-T SG12
Welcome Your Input to Make
G.1050 More Accurate