Download PerceivedQoS - Department of Telematics

1
User-perceived QoS in
networked multimedia services
Harald Øverby, PhD
28/10-2008
2
Topics
• Quality of Service (QoS)
– User-preceived QoS
• Networked multimedia services
– Voice-over-IP (VoIP) (1 hour)
• Shared packet redundancy in VoIP
– Networked online games (1 hour)
• Real Time Strategy (RTS) game Warcraft III
How does QoS influence user experience in
networked multimedia services?
3
Resources
• Y-W Leung, “Shared packet loss recovery for internet
telephony”, IEEE communications letters vol. 9, no. 1, january
2005
• P. O. Osland et al., “Perceived VoIP quality under varying traffic
conditions”, Proceedings of the Nordic Teletraffic Seminar
(NTS), 2004
• N. Sheldon et. al., ”The effect of latency on user performance in
Warcraft III”, Proceedings of the 2nd workshop on Network and
system support for games
• Wikipedia: VoIP: http://en.wikipedia.org/wiki/Voice_over_IP
4
What is Voice over IP (VoIP)
• Voice over IP, IP Telephony,
Internet Telephony, …
– Packet switched telephony
service
– Independent packets routed
over a packet network
• Alternative to circuit switched
telephony service
– End to end circuits established
for each call
– ”Stupid” network vs Intelligent
network
5
VoIP functionality
• Voice is encoded in packets, and transmitted when necessary
• Ability to transmit more than one telephone call down the same
broadband-connected telephone line.
• Many VoIP packages include PSTN features that most telcos
normally charge extra for, or may be unavailable from your local
telco, such as 3-way calling, call forwarding, automatic redial,
and caller ID.
• VoIP can be secured with existing off-the-shelf protocols such
as Secure Real-time Transport Protocol.
• VoIP is location independent.
• VoIP phones can integrate with other services available over the
Internet.
6
QoS in VoIP
• QoS definition (one of many…)
– Degree of compliance of a service to the agreement that exists
between the user and the provider of this service.
• QoS (in broad sense) encompasses
– Performance
– Dependability
– Security
• In this talk we focus on Performance aspects of
VoIP
7
Performance aspects of VoIP
• Performance is characterized by a set of paramters,
which can be measured quantitatively.
• Important performance related QoS parameters in
VoIP includes
–
–
–
–
Available bandwidth
Packet loss
End-to-end delay
Delay jitter
8
User-perceived QoS
• How the user perceives the service quality
• Measured using e.g. questionaires
• E.g.: Will the user see difference between 0 end-toend delay and 0.1 ms end-to-end delay?
• Measured using Mean Opinion score (MOS)
– 5: Max, 1: Minimum
• Influenced by QoS parameters
9
1. Available bandwidth
• Measured in bps
• Depends on the codec
– How the voice signals are decoded
– Trade-off between complexity, bandwidth savings and other
performance related QoS parameters
• Bandwidth requirement dependent on the codec
– H.323 and SIP are popular protocols
• Overhead may constitute a significant part of total
bandwidth consumption
– E.g. data=5.6 kbps, overhead=12.4 kbps, total=18 kbps
10
2. Packet loss
• Measured in ratio of lost packets to all packets sent
• Caused by
– Bit errors
– Queue overflow in routers
– End-to-end delay for packet too high
• Acceptable levels of packet loss ratio is ~2 % for
VoIP
• Dependent on the codec
11
3. End-to-end delay
• Average delay between a user say a word until the received
hears the same word
• The end-to-end delay is comprised of:
–
–
–
–
Local computer delay
Compression delay
Packetizing delay
Transmission delay
• Propagation delay
• Queueing delay
– …
• End-to-end delay below 150 ms is acceptable
12
4. Delay jitter
• Variations in the end-to-end delay
– Difference in the end-to-end delay between independent packets
• Caused by
– Variations in the queueing delay
– Packetizing delay
– Computer processing delay
• Delay jitter below 50 ms is acceptable for VoIP
13
Shared packet redundancy in
VoIP
• Novel mechanism to reduce the packet loss ratio
(PLR)
• Adding redundancy packets to a set of data packets
• Requires additional bandwidth
• Influences the end-to-end delay and delay jitter?
• Complexity issues?
14
Architecture
• Each gateway serves N users
• Packets arrive to the gateway in slots
– n packets in a certain time slot
• The gateway has B channels in each time-slot
• Packets may be lost in the Internet
15
Architecture
slot5
User1
slot4
Data1_5
N=3
Data1_5
n=2
slot2
Data1_3
User2
User3
slot3
n=0
slot1
Data1_1
Data1_3
Data1_2
Data1_3
Data1_2
n=3
n=2
GateWay
B=2
n=1
16
Shared packet redundancy
1. A set of n data packets, add r redundancy packets
2. Transmit all n+r packets
3. If received at least n packets, reconstruction of lost data
packets is possible.
• Example:
1. From 3 data packets we create and add 2 redundancy packets
2. All 5 packets are transmitted
3. If we receive at least 3 packets (among 5 packets),
reconstruction is possible -> no data packets are lost
17
Example
•
2 data packets, 1 redundancy packet
–
–
•
•
•
•
Redundancy packet generated using bitwise XOR in
data packets
Red=Data1Data2
If Data1 is lost, it can be reconstructed as
Data1=Data2Red
If Data2 is lost, it can be reconstructed as
Data2=Data1Red
If two or more packets are lost,
reconstruction is not possible
With more redundancy packets, reed
solomon codes must be used
Data1
Data2
Red
1
1
0
1
0
1
0
1
1
0
0
0
18
Shared packet redundancy in
VoIP
Router
data1
data2
data3
red1
red2
Sent
Received
Reconstruction
data1
data1
data2
data2
data3
data3
red1
red1
red2
red2
data3
19
Parameters
•
•
•
•
•
N: number of customers served
n: outgoing voice streams
B: available capacity (slots)
e: Packet loss ratio
: probability that a session is active
– The probability that a user has a packet to send
• A: activity probability (probability of n active voice
streams)
20
Erasure coding
Packet set
N=4

n=3 with
prob. An
• n=B: No modifications
• n>B: Drop packets
• n<B: Add B-n redundancy
packets
B=5
21
Erasure conding in VoIP
• Always transmit B packets in a packet set
– Case 1: n data packets and B-n redundancy packets
• Packets are lost if less than n packets are received
• Else, lost data packets may be reconstructed
– Case 2: n=B data packets and 0 redundancy packets
• Lost packets cannot be reconstructed
• Packet sets are independent
22
Analysis
23
Analysis: Case 1: n<B
• n voice (data) packets are transmitted
• B-n redundancy packets are added to the data
packet set.
• A packet is lost when
– It is dropped in the network AND
– Less than n of the remaining B-1 packets are dropped
24
Analysis: Case 2: n≥B
• No redundancy packets are added
• Packets are lost due to
– Overflow in the server
– In the network with probability e
• Example
– B=80, n=100, e=0.1
– P(E|An)=(100-80)/100+80*0.1/100=0.28
– Of 100 packets: 20 are lost at server, and 8 in the network
25
Analysis
• Overall loss formula
• Summing over all n
26
Results
B=100
=0.5
27
Qualitative evaluation of the
shared packet redundancy
scheme for VoIP
Parameter
w/o shared packet
redundancy
w/ packet
redundancy
Available
bandwidth
Moderate
Increased
Packet loss
Moderate
Reduction
End-to-end delay
Moderate
Increased
Delay jitter
Moderate
Moderate
Complexity
Moderate
Increased
28
Topics
• Quality of Service (QoS)
– User-preceived QoS
• Networked multimedia services
– Voice-over-IP (VoIP) (1 hour)
• Shared packet redundancy in VoIP
– Networked online games (1 hour)
• Real Time Strategy (RTS) game Warcraft III
How does QoS influence user experience in
networked multimedia services?
29
Motivation
• Computer/video games have become a multibillion industry and is a driving force for
technology development: Hardware,
Software, Networks, etc.
• Games are played by people of all ages female and male.
• Games are used in serious applications
areas: Education, Work training, Physical
exercise, Social training, Brain training, etc.
30
What make games intriguing?
• Characteristics of enjoyable activities (playing games) can be
divided into three categories:
– Challenge:
• Mastering a challenge can give a self-esteem boost.
• Ideally the game should adjust to players abilities.
– Fantasy:
• Fantasies and abstractions enhance and make the experience more interesting.
– Curiosity:
• Curiosity is the motivation to learn and investigate.
• Desire to bring completeness, consistency to their knowledge
31
Game genres
•
•
•
•
•
Action games - shooting games, platform games
Adventure games - logic puzzles in a virtual world
Fighting games - games involving combat
Puzzle games - e.g. Tetris
Role playing game - players assume the role of
imaginative creature or person
• Simulations - the player is put in control of
simplified process. E.g., flight simulators,
SimCity, driving simulation etc.
• Sport games - games involving sports
• Strategy games - typically commanding
armies and warfare planning.
32
Online Gaming in General
• Most new-released games for PCs and game consoles (PS3,
Xbox 360 and Wii) provide online game functionality.
– For some games, online-features are main parts of the game (Battlefield)
– For other games online-features are add-on functionality (Burnout)
– The number of player supported varies from game to game (genre)
• For consoles, Xbox Live and Playstation Network (PSN) are
driving forces for online game functionality/support.
• Xbox Live and PSN have initiated series of simpler online
games XBOX Live Arcade, and PSN games (often retro games).
• XNA from Microsoft enables independent developers to create
own online games distributed on Xbox Live.
33
Main Challenges for Online
Gaming in General
• Latency (main challenge!)
– Ensure smooth gameplay and that other players move realistically
(not in steps, but in one motion).
– Ensure that variation in latency does not affect players
performance/score in the game.
• Bandwidth
– Ensure that variation in bandwidth does not affect players
performance/score in the game.
• Heterogeneity:
– Support various types of hardware (PC, consoles) and services
(Xbox Live, Playstation Network, etc)
34
Mobile Online Gaming
• Mobile online game functionality is becoming more and more
common on Playstation Portable and Nintendo DS (add-on to
original game): E.g., Killzone: Liberation, Test Drive Unlimited
• Few online games for mobile phones:
– Online highscore lists
– Turn-based and slow-paced games
– MMOG on mobile phones in Japan: Samurai Romanesque
• Mobile Online Gaming has high potential as there are plenty of
users and the users are “always” online.
35
MMOG/MMO
• Massively multiplayer online game (MMOG/MMO):
Computer game which is capable of supporting
hundreds or thousands of players simultaneously.
• Games played over the Internet
• Feature at least one persistent world.
• Enable players to cooperate and compete on a grand
scale.
• Most MMOGs require a monthly subscription fee.
• Huge success last decade
36
Types of MMOGs (Massive
Multiplayer Online Games)
• MMORPG (Role-playing Game): Guild Wars, FF XI, WoW,
EverQuest.
• FPS (First-Person Shooter): Counterstrike, Call of Duty.
• RTS (Real-Time Strategy): Shattered Galaxy, Boundless
Planet, Dreamlords, Ballerium.
• MMODG (Dance/Rhythm Games): DANCE! Online, O2 Jam,
Audition Online.
• MMOR (Racing): Test Drive Unlimited, RaceLands, KartRider.
• MMOSG (Social/Strategy/Sports Game): Second Life, Home,
Empire.
• MMO Real-words simulations: flight-simulations, traffic
control, military, etc.
37
Main Challenges for Massive
Multiplayer Online Games
• Scalability/bandwidth to servers! (Main challenge)
– Allow several players to play simultaneously in the same game world.
– Divide the world into zones served by separate servers.
– Run same game worlds/zones in parallel.
• Consistency
– Ensure that changes to the game world is consistently distributed among all
players.
• Latency
– Ensure that the players have a responsive gameplay and that game objects
move around smoothly.
• Bandwidth to clients
– Used to be a challenge when players used modem. With broadband
connections this is no longer a big challenge.
38
The Effects of Latency
on User Performance in
Warcraft III
Nathan Sheldon, Eric Gerard, Seth Borg, Mark Claypool,
Emmanuel Agu
Computer Science Department
Worcester Polytechnic Institute
Worcester, MA, USA
http://www.cs.wpi.edu/~claypool/papers/war3/
39
Network Games and Latency
• Latency degrades performance of interactive
applications
– Web-browsing – seconds
– Audioconference – 100’s of milliseconds
– First Person Shooters (FPS) – 100’s of milliseconds
• Real-Time Strategy (RTS)?
• Knowing effects of latency useful for
– Building better network games
– Building better networks to support games (QoS)
 Effects of Latency on Warcraft III (RTS)
40
Warcraft III Overview
RTS User Interaction
Components:
• Exploration
• Building
• Combat
41
Exploration Map
Performance?
• Time
(to reach end)
42
Building Map
Performance?
• Time
(to build technology tree)
43
Combat Map
Performance?
• Games Won
• Unit Scores
44
Controlling Latency
• Warcraft III uses client-server
– Set computer B as server (also a client)
– Set computer C or D as client
• NIST Net on computer A
– Induce latency [0 ms to 3500 ms]
45
Building and Latency
Build Time vs. Latency
9:36
Linear (Time To Build)
R2 = 0.0516
9:21
Build Time (m:s)
Time To Build
9:07
8:52
8:38
8:24
8:09
7:55
7:40
0
500
1000
1500
2000
Latency (ms)
2500
3000
3500
4000
46
Exploration and Latency
Time To Complete
Linear (Time To Complete)
Explore Time vs Latency
Explore Time (m:s)
5:31
R2 = 0.6334
5:16
5:02
4:48
4:33
4:19
4:04
3:50
0
200
400
600
Latency (ms)
800
1000
1200
47
Combat and Latency (1)
Unit Score Difference
Unit Score Difference vs. Latency
3000
R2 = 0.0138
2000
1000
0
-1000
-2000
-3000
0
200
400
600
800
1000
Latency (ms)
1200
1400
1600
48
Combat and Latency (2)
49
Bandwidth LAN vs. Internet
3.8 Kbps
4.0 Kbps
6.8 Kbps
50
User-Level Analysis
• 0-500 ms latency, users could easily adjust
• 800+ ms, game appeared erratic
– Degradation in gaming experience
• 500-800 ms degradation depended upon
– User
• More skilled were more sensitive
– Strategy
• Micro managers were more sensitive
• Combat managers were more sensitive
51
Summary of the Effect of
Latency on RTS Games
• Typical Internet latencies do not significantly affect user
performance in Warcraft III
– Some effect on exploration
– No statistical effect on building or combat
• RTS game play emphasizes “strategy” (which takes 10s of
seconds or minutes), not “real-time”
• RTS games less sensitive to latency than are FPS
– RTS in QoS class similar to that of Web browsing
• At the network level:
– Small packets with low bandwidth
– Command aggregation at higher latencies