Download Services in CINEMA - Columbia University

Reliable, Scalable and Interoperable
Internet Telephony
PhD thesis presentation
by Kundan Singh
Advisor: Henning Schulzrinne
Computer Science Department, Columbia University, New York
June 21, 2006
My research background/timeline
1997
1999
2000
2001
2002
2003
2004
2005
2
2006
Outline of the presentation
Introduction



Server redundancy



Load sharing and failover in SIP telephony
Comparison of thread models for SIP server
Peer-to-peer (P2P)



SIP servers using external P2P network
Additionally, P2P maintenance using SIP
Enterprise IP telephony





What is the problem? Why important?
My contributions
Multi-platform collaboration using SIP
Scalable centralized conferencing architecture
Interworking between SIP/SDP and H.323
Conclusion
36 slides
3
Telephone reliability & scalability
(PSTN: Public Switched Telephone Network)
database (SCP)
for freephone,
calling card, …
signaling network
(SS7)
local telephone switch
(class 5 switch)
signaling
router
10,000
customers
(STP)
20,000 calls/hour
regional telephone switch
(class 4 switch)
100,000 customers
150,000 calls/hour
“bearer” network
database (SCP)
10 million customers
2 million lookups/hour
signaling router (STP)
1 million customers
1.5 million calls/hour
telephone switch
(SSP)
Switches strive for 99.999% availability.
Lucent’s 5E-XC supports 4 million BHCA.
4
Internet telephony
(SIP: Session Initiation Protocol)
[email protected]
yahoo.com
example.com
INVITE
REGISTER
INVITE
129.1.2.3
[email protected]
192.1.2.4
DB
DNS
Can SIP server provide carrier
grade reliability and scalability
using commodity hardware
5
What are the problems?

Can SIP server provide carrier-grade reliability
and scalability using commodity hardware?


How can we build a server-less selforganizing peer-to-peer VoIP network?


What affects the server performance?
Can this be done in standards compliant way?
Can communication be extended to multiplatform collaboration using existing
protocols?


How well multi-party conferencing scales?
How to interoperate between SIP and H.323?
6
My contributions

Server redundancy




Peer-to-peer (P2P)



Implemented failover using database replication
Two-stage architecture for SIP load sharing
Comparison of thread models for SIP server
SIP servers using external P2P network
Additionally, P2P maintenance using SIP
Enterprise IP telephony



Multi-platform collaboration using SIP
Scalable centralized conferencing architecture
Interworking between SIP/SDP and H.323
New architecture, New algorithm or approach, Implementation, Evaluation
7
Outline of the presentation
Introduction



Server redundancy



Load sharing and failover in SIP telephony
Comparison of thread models for SIP server
Peer-to-peer (P2P)



SIP servers using external P2P network
Additionally, P2P maintenance using SIP
Enterprise IP telephony





What is the problem? Why important?
My contributions
Multi-platform collaboration using SIP
Scalable centralized conferencing architecture
Interworking between SIP/SDP and H.323
Conclusion
8
Server redundancy
The problem: failure or overload
INVITE
REGISTER
9
High availability
Master/
slave
Implementation and analysis

Using MySQL replication

TR
Call setup latency
DNS TTL, time-to-repair,
retry timeout

P2
P1
System reliability
individual component
reliability

Caller
D2
D1
DNS
Implementation
Slave/
master
User unavailability
Most registers are refreshes
Retry timeout, replication
interval, register refresh
interval
P1
Tc
P2
D1
D2
Callee
Td
A
Tc
Tr
TR
A
Tc
10
Scalability
Load sharing: redundant proxies and databases
P1

REGISTER

D1


D2
P3
Write to D1 & D2
INVITE

P2
INVITE
REGISTER
Read from D1 or D2
Database write/
synchronization
traffic becomes
bottleneck
11
Scalability
Load sharing: divide the user space
P1
a-h

D1

P2
i-q
D2
Proxy and database
on the same host
Stateless proxy can
become overloaded

Use many
P3
r-z
D3
12
Scalability
Comparison of the two designs
P1
P1
a-h
D1
D1
P2
P2
i-q
D2
P3
P3
D2
r-z
Low Scalability
High Reliability
D2
High Scalability
Low Reliability
13
Scalability (and reliability)
Two stage architecture
I stage
II stage
a*@example.com
a1
s1
Master
a2
a.example.com
_sip._udp
SRV 0 0 a1.example.com
SRV 1 0 a2.example.com
Slave
sip:[email protected]
s2
sip:[email protected]
b*@example.com
s3
ex
example.com
_sip._udp
SRV 0 40 s1.example.com
SRV 0 40 s2.example.com
SRV 0 20 s3.example.com
SRV 1 0 ex.backup.com
b1
Master
b2
Slave
b.example.com
_sip._udp
SRV 0 0 b1.example.com
SRV 1 0 b2.example.com
Capacity = f(#stateless, #groups)
14
Load Sharing
Performance result: calls/second

calls/sec
2800
3000
2500
2100
1800
2000
1500
1000
900

Using 3+3 servers gives
carrier-grade
performance (10 million
busy hour call attempts)
Registration test:
supports 10 million
subscribers
1050
500
0
On commodity hardware: 3 GHz, Pentium 4, 1 GB memory
Test with UDP, stateless, no DNS and no mempool
15
Server performance
What happens inside a server? What thread/event models possible?
Web server
recv
accept
SIP server
stateful
Response
recvfrom
parse
Request
File read
send
Match
transaction
Modify
response
Stateless proxy
Found
Match
transaction
Update DB
REGISTER
other
Redirect/reject
Lookup DB
Stateless proxy
1.
2.
3.
4.
(Blocking) I/O
Critical section (lock)
Critical section (r/w lock)
Pure event-based (one thread)
Thread-per-msg or transaction
Pool-thread per msg (sipd)
Two stage thread pool
sendto
Build
response
Proxy
Modify
Request
DNS
16
Server performance
Results of my measurement


Event-based performs 30% better than
existing thread-pool architecture on
single-CPU
Two stage thread-pool architecture
gives better performance on multi-CPU


60% more on 4xPentium
Both Pentium and Sparc took 2 MHz of
CPU cycles per call/s on single-CPU
17
Problem with servers
C
C
S
C

Server-based

C

C

P
P
P
P
P

Cost: maintenance, configuration
Central points of failures, catastrophic
failures
Controlled infrastructure (e.g., DNS)
Peer-to-peer



Robust: no central dependency
Self organizing, no configuration
Inherent scalability
18
We built: P2P-SIP

Unlike proprietary Skype architecture


Robust and efficient lookup using DHT
Interoperability


Hybrid architecture



Lookup in SIP+P2P
Inter-domain P2P-SIP
Unlike file-sharing applications



DHT algorithm uses SIP communication
Data storage, caching, delay, reliability
Data model and service model
Disadvantages

Lookup delay and security
19
How to combine SIP + P2P?

SIP-using-P2P


Replace SIP location
service by a P2P
protocol
P2P-over-SIP
Additionally,
implement P2P using
SIP messaging

SIP-using-P2P
P2P SIP proxies
P2P-over-SIP
Maintenance
P2P
P2P
SIP
Lookup
P2P
SIP
SIP
P2P network
FIND
INSERT
INVITE sip:[email protected]
INVITE alice
P2P-SIP
overlay
REGISTER
Alice
128.59.19.194
Alice
128.59.19.194
20
SIP-using-P2P
Logical Operations

Contact management


Key storage



put (subscribee id, signed encrypted subscriber id)
Composition needs service model
Offline message


User certificates and private configurations
Presence


put (user id, signed contact)
put (recipient, signed encrypted message)
NAT and firewall traversal

STUN and TURN server discovery needs service model
XML-based data format
21
SIP-using-P2P
Implementation in SIPc with the help of Xiaotao Wu

OpenDHT


Identity
protection



Using Data
model
Certificate-based
SIP id == email
P2P for
Calls, IM,
presence, offline
message and
name lookup
22
P2P-over-SIP
Architecture and implementation




DHT (Chord) algorithm using SIP messages
with query and update semantics of
REGISTER
Has SIP registrar, proxy, user agent
Other: discovery, NAT traversal, failover
Adaptor: allows existing SIP devices to
become P2P
23
P2P-over-SIP
Analysis: scalability

Computed message load as function of


Refresh rate (keep-alive, finger table, user
registration), call arrival rate, churn (join, leave,
failure), scale (number of peer nodes and users)
Number of nodes = f(individual node
capacity)


Measured performance: 800 register/s.
Assuming a conservative 10 reqister/s capacity,
and aggressive refresh and call rate of 1/min, it
gives more than 16 million peers (super nodes) in
the network.
24
P2P-over-SIP
Analysis: availability and call setup latency

To increase user availability:




Fast failure detection: increase keep-alive rate
Reduce unavailability: frequent registration refresh
Replicate: user and node registrations
Call setup latency:

Same as DHT lookup latency: O(log(N))


Calls to known locations (“buddies”) is direct
Chord: 10000 nodes => 6 hops


At most a few seconds
User availability and retransmission timers
25
SIP-using-P2P vs P2P-over-SIP




Not SIP-specific, hence
no implementation
overhead for non-VoIP
but P2P applications
Low transport and
transaction overhead
No P2P security burden
on SIP
No dependency on
single DHT
implementation


Reuse SIP naming,
routing, security,
NAT/firewall traversal
Easily reuse existing SIP
components without
change



voicemail, conference
Single DHT
implementation
Readily supports service
model
26
Server-based vs peer-to-peer
Reliability,
failover latency
DNS-based. Depends on client
retry timeout, DB replication
latency, registration refresh
interval
DHT self organization and
periodic registration refresh.
Depends on client timeout,
registration refresh interval.
Scalability,
number of users
Depends on number of servers
in the two stages.
Depends on refresh rate,
join/leave rate, uptime
Call setup
latency
One or two steps.
O(log(N)) steps.
Security
TLS, digest authentication,
S/MIME
Additionally needs a reputation
system, working around spy nodes
Maintenance,
configuration
Administrator: DNS, database,
middle-box
Automatic: one time bootstrap
node addresses
PSTN
interoperability
Gateways, TRIP, ENUM
Interact with server-based
infrastructure or co-locate peer node
with the gateway
27
Outline of the presentation
Introduction



Server redundancy



Load sharing and failover in SIP telephony
Comparison of thread models for SIP server
Peer-to-peer (P2P)



SIP servers using external P2P network
Additionally, P2P maintenance using SIP
Enterprise IP telephony





What is the problem? Why important?
My contributions
Multi-platform collaboration using SIP
Scalable centralized conferencing architecture
Interworking between SIP/SDP and H.323
Conclusion
28
Internet telephony infrastructure
CINEMA: Columbia InterNet Extensible Multimedia Architecture
Telephone
switch
Local/long distance
1-212-5551212
CINEMA servers
rtspd: media server
sipconf:
RTSP
Conference server
PSTN
Quicktime
RTSP clients
Department
PBX
Internal
Telephone
Extn: 7040
713x
sipum:
Unified
messaging
sipd:
Proxy, redirect,
Registrar server
SIP/PSTN Gateway
SQL
database
cgi
vxml
Web
server
Web based
configuration
SIP
VXML
Built many components in a
complete system for enterprise
IP telephony and multimedia
collaboration
siph323:
SIP-H.323
translator
My work
H.323
NetMeeting
29
My other work

Communication to collaboration





Centralized conferencing



Comprehensive, multi-platform collaboration using SIP
Unified messaging: The gaps among different media (audio, video,
text), devices (PC, phone) and means of communications (Email,
SIP, IM) disappear for messaging
Novel SIP/RTSP based voicemail and answering machine
SIP interface to VoiceXML browser
Audio mixing, video forwarding, IM, shared web browsing, screen
sharing, web-based configuration and control, floor control
Performance evaluation; cascaded server architecture
SIP-H.323 translation
30
Conference server
Performance evaluation of audio mixer

On commodity PC



About 480 participants in a single
conference with one active speaker
About 80 four-party conferences, with one
speaker each
Both Pentium and Sparc took 6 MHz per
participant
31
Conference server
Cascaded architecture




SIP REFER message is
used to create
cascading

N.(N-1) participants
Higher delay





N2/4 participants
Lower delay
I measured the CPU usage for two
cascaded servers: supports
about 1000 participants
32
Outline of the presentation
Introduction



Server redundancy



Load sharing and failover in SIP telephony
Comparison of thread models for SIP server
Peer-to-peer (P2P)



SIP servers using external P2P network
Additionally, P2P maintenance using SIP
Enterprise IP telephony





What is the problem? Why important?
My contributions
Multi-platform collaboration using SIP
Scalable centralized conferencing architecture
Interworking between SIP/SDP and H.323
Conclusion
33
Revisiting the problems

Can
and


What affects the server performance?
How can we build a server-less selfDeveloped P2P-SIP architecture:
organizing peer-to-peer
VoIP network?
SIP-using-P2P
and P2P-over-SIP


Developed a two stage scalable and
SIP server provide
carrier-grade
reliability
reliable
SIP server architecture:
linear scaling. Usehardware?
event-based.
scalability using commodity
Can this be done in standards compliant way?
Can communication be extended to multiplatform collaboration using existing
Multi-platform collaboration using
protocols?


existing protocols and tools,
How well multi-party conferencing
scales?
unified messaging,
centralized
conferencing
(cascaded),
SIPHow to interoperate between
SIP
and H.323?
H.323 interworking.
34
Conclusions

Impact:








Commercialized by SIPquest (now FirstHand) and sold to many customers.
CINEMA was deployed in our department for a brief period of time.
Used in various other projects at IRT: NG911, firewall controller, presence
scalability, TCP/TLS measurements,…
P2P-SIP is a “hot” topic in industry and IETF now – client desktop,
hardware phone as well as server vendors are pursuing this.
SIP-H.323 requirements eventually became an RFC
Plan to open source SIPc for large scale deployment experience of P2P-SIP
Started working on a P2P-based self organizing servers for 3GPP at Bell
Labs
“So what” (Implications):


Replacing PSTN – better features, quality and performance at lower cost
and maintenance; zero cost VoIP using P2P-SIP
Distributed, multi-provider, component architecture for telephony and
collaboration
35
My publications
Conference, workshop, technical report, magazine/journal
1.
2.
K. Singh and H. Schulzrinne, “Using an external DHT as a SIP location service", Columbia University Technical Report CUCS-007-06, NY, Feb’06.
K. Singh and H. Schulzrinne, “Peer-to-peer Internet telephony using SIP", NOSSDAV, Skamania, Washington, Jun 2005..
K. Singh and H. Schulzrinne, "Peer-to-peer Internet Telephony using SIP", New York Metro Area Networking Workshop, CUNY, NY, Sep 2004.
K. Singh and H. Schulzrinne, "Peer-to-peer Internet Telephony using SIP", Columbia University Technical Report CUCS-044-04, NY, Oct 2004.
3.
K. Singh and H. Schulzrinne, “Failover, load sharing and server architecture in SIP telephony”, Elsevier Computer Communication Journal. To
appear. Aug 2006.
“K. Singh and H. Schulzrinne, “Failover and load sharing in SIP telephony", SPECTS (Symposium on performance evaluation of computer and
telecommunication systems). Philadelphia, PA, Jul 2005.
K. Singh and H. Schulzrinne, "Failover and Load Sharing in SIP Telephony", Columbia University Technical Report CUCS-011-04, NY, May 2004.
4.
5.
H. Schulzrinne, K. Singh and X. Wu, "Programmable Conference Server", Columbia University Technical Report CUCS-040-04, NY, Oct 2004.
K. Singh, Xiaotao Wu, J. Lennox and H. Schulzrinne, "Comprehensive Multi-platform Collaboration", MMCN 2004 - SPIE Conference on
Multimedia Computing and Networking, Santa Clara, CA, Jan 2004.
K. Singh, Xiaotao Wu, J. Lennox and H. Schulzrinne, "Comprehensive Multi-platform Collaboration", Columbia University Technical Report CUCS-027-03,
NY, Nov 2003.
6.
M. Buddhikot, A. Hari, K. Singh and S. Miller, "MobileNAT: A new Technique for Mobility across Heterogeneous Address Spaces", ACM
MONET journal, March 2005.
M. Buddhikot, A. Hari, K. Singh and S. Miller, "MobileNAT: A new Technique for Mobility across Heterogeneous Address Spaces", WMASH 2003 - ACM
International Workshop on Wireless Mobile Applications and Services on WLAN Hotspots, San Diego, CA, Sep 2003.
7.
K. Singh, A. Nambi and H. Schulzrinne, "Integrating VoiceXML with SIP services", ICC 2003 - Global Services and Infrastructure for Next
Generation Networks, Anchorage, Alaska, May 2003.
K. Singh, A. Nambi and H. Schulzrinne, "Integrating VoiceXML with SIP services", Second New York Metro Area Networking Workshop, Columbia
University, NY, Sep 2002.
8.
K. Singh, W. Jiang, J. Lennox, S. Narayanan and H. Schulzrinne, "CINEMA: Columbia InterNet Extensible Multimedia Architecture", Columbia
University Technical Report CUCS-011-02, NY, May 2002.
W. Jiang, J. Lennox, H. Schulzrinne and K. Singh, "Towards Junking the PBX: Deploying IP Telephony", NOSSDAV 2001.
W. Jiang, J. Lennox, S. Narayanan, H. Schulzrinne, K. Singh and X. Wu, "Integrating Internet Telephony Services", IEEE Internet Computing (magazine),
May/June 2002 (Vol. 6, No. 3).
9. K. Singh, Gautam Nair and H. Schulzrinne, "Centralized Conferencing using SIP", 2nd IP-Telephony Workshop (IPTel'2001), April 2001.
10. K. Singh and H. Schulzrinne, "Unified Messaging using SIP and RTSP", IP Telecom Services Workshop 2000, Atlanta, Georgia, U.S.A, Sept 2000.
K. Singh and H. Schulzrinne, "Unified Messaging using SIP and RTSP", Columbia University Technical Report CUCS-020-00, NY, Oct 2000.
11. K. Singh, H.Schulzrinne, "Interworking Between SIP/SDP and H.323", 1st IP-Telephony Workshop (IPTel'2000), April 2000.
K. Singh and H. Schulzrinne, "Interworking Between SIP/SDP and H.323", Columbia University Technical Report CUCS-015-00, NY, May 2000.
36