Download Basics of Voice and Video over IP performance

Troubleshooting Voice and
Video Performance over IP
using Measurement Tools…
Prasad Calyam and Paul Schopis,
ITEC-Ohio, OARnet and Ohio Supercomputer Center,
Tools Tutorial, Joint Techs, Salt Lake City, Utah, February 2005
Topics of Discussion
Basics of Voice and Video over IP
performance measurement
Measurement Tools we have experimented
with…
“H.323 Beacon” Tool we have developed!
How is this tool different?
Tool Features
Two Case-studies
An Useful H.323 Beacon Tool Supplement!!!
Ethereal RTP Analysis Module
Conclusion
Basics of Voice and Video over IP
performance measurement
Voice and Video Traffic are inherently different from
Data Traffic
They use Signaling protocols such as H.323, SIP, …
Media (Voice and Video payload) is delivered over IP using
RTP packets irrespective of which codec or end-point
technology (PC-based Vs Appliance-based)
General ICMP and UDP based tools fail to totally
capture the performance bottlenecks faced by actual
voice and video traffic at the end-host and in the
network
i.e.- ping, traceroute, Iperf, pathrate, … don’t suffice!
We need to measure end-user experience of Voice and
Video over IP applications also!
Measuring End-user experience…
Two approaches to evaluating end-user
experience of audiovisual quality
Subjective Measurements
Involve human participants to rate audiovisual
quality Can you hear me now?
Mean Opinion Score (MOS) Ranking technique
(ITU-T P.800) Not just “Good”!
Objective Measurements
Automated techniques to rate audiovisual quality
“E-Model” [ITU-T G.107]
Perceptual Evaluation of Speech Quality (PESQ)
[ITU-T P.862]
Mean Opinion Score (MOS) Concept
What are the end-host and network problems?
Scenario I: A Researcher and an Industry
professional want to Videoconference
GigaPOP
OC2
OC192
Internet2 Abilene Network
Case1:Researcher is unable to make a
call!
GigaPOP
OC2
OC192
Internet2 Abilene Network
There was a mis-configured firewall
blocking necessary ports…
GigaPOP
OC2
OC192
Internet2 Abilene Network
Case2: Industry professional is unable
to make a call!
GigaPOP
OC2
OC192
Internet2 Abilene Network
His LAN’s Internet connectivity was
non-functional at that time…
GigaPOP
OC2
OC192
Internet2 Abilene Network
Case3: They connected, but of them
experienced bad audio & video!
GigaPOP
OC2
OC192
Internet2 Abilene Network
There was congestion at one of the
intermediate routers along the path…
GigaPOP
OC2
OC192
Internet2 Abilene Network
There was congestion at one of the
intermediate routers along the path…
GigaPOP
OC2
OC192
Internet2 Abilene Network
There was congestion at one of the
intermediate routers along the path…
GigaPOP
OC2
OC192
Internet2 Abilene Network
The performance problem can be
anywhere in the E2E Path!!!
GigaPOP
OC2
OC192
Internet2 Abilene Network
What are the other common end-to-end
performance problems?
Common problems involving endpoint devices
Failure of audio and video hardware, out dated or buggy
end-point application software
Faulty connections and configurations of audio and video
interfaces
Lack of lip-synchronization
Mis-configured jitter buffer sizes
Non inter-operable end-point application software
Lack of forward error correction and echo cancellation
mechanisms in end-point clients
Lack of end-user training
Common End-to-End performance
problems in VVoIP Systems (2)
Common problems involving network devices
Insufficient network capacity for handling multiple high data
rate videoconferencing calls
Excessive delay, loss, jitter, out of order packets and reordered packets in the network
Duplex mis-match problems
Traffic congestion at peak-usage periods of the network
Mis-configured priorities for real-time audio and video traffic
streams in the network
Asymmetric routing with excessive delays on one path
Lack of network engineering resource personnel
Common End-to-End performance
problems in VVoIP Systems (3)
Common problems involving application service
devices
Misconfigured firewalls that block required ports
Non-H.323 friendly NATs
Misconfigured devices
MCUs
Gatekeeper
Gateway devices
Outdated or buggy application-service device application
software
Measurement Tools we have
experimented with…
MCU-specific Performance Monitors
Per-call monitoring of many network and end-point parameters
(delay, jitter, loss, reordering)
Telchemy VQMon
Based on E-Model Standard; Integrated into many vendor VoIP
troubleshooting products (e.g., Brix Networks, Agilent, etc…)
PESQ
Based on PESQ Standard; Used in many products (e.g., Opticom,
Acterna, etc…)
Hammer Call Analyzer
Packet Analyzer (Ethereal backend) with many features such as
call-flow display, jitter buffer analysis, wave file analysis, etc…
Nettach
Developed originally by PictureTel; Similar to H.323 Beacon where
you emulate Voice and Video traffic and measure performance
Many other good tools… (AppareNet, ViDeNet Scout (based on
NetIQ Chariot), etc.)
MCU-specific Performance Monitors
Increased Video
Sync Loss!
Telchemy VQMon
PESQ
SAMPLE RESULTS:
REFERENCE
DEGRADED
original.wav degraded.wav
PESQMOS
SQValue=2.930
PESQMOS
2.930
Hammer Call Analyzer
Taken from- http://www.empirix.com
Nettach
H.323 Beacon Overview
An application-specific measurement tool
To monitor and qualify the performance of
H.323 Videoconferencing sessions at the host
and in the network (end-to-end)
Useful to an end-user/conference
operator/network engineer
Uses OpenH323 and J323Engine
libraries
Comparison with other existing tools…
Addresses problems due to H.323 protocolspecific idiosyncrasies
Can be generalized to RTP packets performance
over the network
Many in-built tools that generate various kinds of
measurement data for pre/during/post
Videoconference troubleshooting!
An “easy to install and use” tool that is open
source (FREE!)
Commercial tools are VERY expensive!
NetIQ Chariot, Spirent Smartbits, Telchemy VQMon,…
Initial call setup failures and haphazard
disconnections…
Test Status
In-Session, Normal Close, Exception
Close
Exception Close Alarms
“Possible Firewall/NAT presence obstruction”
“No Internet connectivity”
“Incompatible codec being used”
“Insufficient bandwidth”
“Remote H.323 Beacon Server not online”
Initial call setup failures and haphazard
disconnections…
Network Health Status…
Delay, Jitter and Loss data
Real-time, offline raw data and test
session summary
Network Health Plots…
Watermarks for “Good”, “Acceptable” and “Poor” grade
of quality as experienced by end-user
Delay: (0-150)ms, (150-300)ms, > 300ms
Jitter: (0-20)ms, (20-50)ms, > 50ms
Loss: (0-0.5)%, (0.5-1.5)%, >1.5%
Levels obtained from our PAM 2004 paper
Poor
Acceptable
Good
Audio and Video Quality Assessments
Audio and video loopback feature
E-Model-based objective MOS ranking
Slider-based subjective MOS ranking
Customization of tests…
Test results data folder, TCP/UDP/RTP port
settings, H.225 and H.245 parameters,
preferred codec, watermarks for delay, jitter,
loss, …
Use-case I
Problem report
Effect of a mis-configured
firewall on a video stream
Intermittent frame
freezing
Lot of pixilation
No significant audio
problems
Sudden
disconnections
Use-case I (2)
MCU Software
~15 fps for 384Kbps call
and ~0.6% packet loss
~30 fps for 128Kbps call
and ~0% packet loss
H.323 Beacon
Sluggish call-setup
Delayed packet-events
Initial jitter variations in
poor range
Jitter variations indicated by H.323
Beacon client-to-server test
Increasing the rate-limit and firewall reconfiguration solved the problem!
Use-case II
Problem report
Poor audio and video reception at the site of user
Problem only in the streams from Campus A to Campus B
MCU Software
Packet loss (~1%)
Packet re-ordering
H.323 Beacon
Jitter values consistently in poor range
Traceroute and Reverse-Traceroute showed an asymmetric
route
E-Model results
Campus A to Campus B: 3.78
Campus B to Campus A: 4.37
Use-case II (2)
Fig. 1 Time-plot of Equipment Impairment Factor
Fig. 3 Time-plot of Loss
Fig. 2 Time-plot of Delay
Fig. 4 Time-plot of Mean Opinion Score Ranking
Problem was solved by contacting the neighboring
ISP who was suspected to be experiencing problems!
How do you setup the H.323 Beacon in a network?
Scenario I: H.323 Beacon used in a
Network Measurement Infrastructure…
3 Com
CISCO SYSTEMS
GigaPOP
OC2
OC192
3Com
Core Router
CISCO SYSTEMS
Switch
NMS
CDMA Device
Internet2 Abilene Network
Scenario II: In a Multi-point setting…
Scenario II: In a Multi-point setting…
Scenario II: In a Multi-point setting…
Ethereal RTP Analysis Module
An useful H.323 Beacon Tool supplement!!!
Adds advantages of passive measurement to the
H.323 Beacon
In general, to any RTP-based application!
Generates inter-packet jitter, packet-size
distribution, packet loss and re-ordering
information from input traffic traces
We have written simple parsing scripts that
can process data in a format that facilitates
analysis using Minitab or Excel
Use the OPENXTRA version of Ethereal!
http://resource.intel.com/telecom/support/appnotes/9008/9008an.pdf
Ethereal RTP Analysis Module (2)
Steps for analyzing the Traces
Load the packet trace into Ethereal
Trace will contain both forward and reverse direction streams
(Check “Source” and “Destination” IP addresses)
Decode streams as RTP (default is UDP)
This will mark all related packets as belonging to a specific
audio and video codec streams
Analyze individual audio or video streams
Import various information fields as .csv file (“Save as CSV”
option)
Also has wave file generation relating to an audio stream
(“Save Payload” option)
Works only for G.711 Codec streams!
Good for PESQ where you want to compare original and
degraded wave files to obtain Objective MOS information
Ethereal RTP Analysis Module (3)
General UDP Stream
decoded as an H.263
payload stream
Ethereal RTP Analysis Module (4)
Audio Stream
Video Stream
Ethereal RTP Analysis Module (5)
Loss
Re-ordering; 45413 45415 45414
(Observe Sequence #s; could also be 2
consecutive packet losses)
Pink-marked packets relate to either lost or re-ordered packets!
Ethereal RTP Analysis Module (6)
An Imported CSV File!
Ethereal RTP Analysis Module (6)
Create interesting visualizations to understand various RTP packet
characteristics; can do the same for both Voice and Video packets!!!
Questions?
H.323 Beacon Reference:
http://www.itecohio.org/beacon