Download 3rd Edition: Chapter 2

Chapter 2
Application Layer
A note on the use of these ppt slides:
We’re making these slides freely available to all (faculty, students, readers).
They’re in PowerPoint form so you can add, modify, and delete slides
(including this one) and slide content to suit your needs. They obviously
represent a lot of work on our part. In return for use, we only ask the
following:
 If you use these slides (e.g., in a class) in substantially unaltered form,
that you mention their source (after all, we’d like people to use our book!)
 If you post any slides in substantially unaltered form on a www site, that
you note that they are adapted from (or perhaps identical to) our slides, and
note our copyright of this material.
Computer Networking:
A Top Down Approach
Featuring the Internet,
3rd edition.
Jim Kurose, Keith Ross
Addison-Wesley, July
2004.
Thanks and enjoy! JFK/KWR
All material copyright 1996-2005
J.F Kurose and K.W. Ross, All Rights Reserved
2: Application Layer
1
Chapter 2: Application layer
 2.1 Principles of
network applications
 2.2 Web and HTTP
 2.3 FTP
 2.4 Electronic Mail

SMTP, POP3, IMAP
 2.5 DNS
 2.6 P2P file sharing
 2.7 Socket programming
with TCP
 2.8 Socket programming
with UDP
 2.9 Building a Web
server
2: Application Layer
2
Chapter 2: Application Layer
Our goals:
 conceptual,
implementation
aspects of network
application protocols
 transport-layer
service models
 client-server
paradigm

peer-to-peer
paradigm
 learn about protocols
by examining popular
application-level
protocols




HTTP
FTP
SMTP / POP3 / IMAP
DNS
 programming network
applications

socket API
2: Application Layer
3
Some network apps
 E-mail
 Internet telephone
 Web
 Real-time video
 Instant messaging
 Remote login
 P2P file sharing
conference
 Massive parallel
computing
 Multi-user network
games
 Streaming stored
video clips
2: Application Layer
4
Creating a network app
Write programs that



run on different end
systems and
communicate over a
network.
e.g., Web: Web server
software communicates
with browser software
little software written for
devices in network core


network core devices do
not run user application
code
application on end systems
allows for rapid app
development, propagation
application
transport
network
data link
physical
application
transport
network
data link
physical
application
transport
network
data link
physical
2: Application Layer
5
Chapter 2: Application layer
 2.1 Principles of
network applications
 2.2 Web and HTTP
 2.3 FTP
 2.4 Electronic Mail

SMTP, POP3, IMAP
 2.5 DNS
 2.6 P2P file sharing
 2.7 Socket programming
with TCP
 2.8 Socket programming
with UDP
 2.9 Building a Web
server
2: Application Layer
6
Application layer functions
 Applications
 Implement desired functionality within application
protocols when no underlying network service provides
support
• Mail, Web, News, P2P, etc….

Other functions
•
•
•
•
•
•
•
•
Security (S/MIME, PGP, S-HTTP)
Delivery semantics (multicast overlays, anycast)
Reliable data transfer (reliable multicast, reliable UDP)
Quality of service (QoS overlays, scheduling)
Congestion control (Non-TCP applications)
Flow control (Non-TCP applications)
Naming (DNS, URLs)
Routing (overlays)
 Functionality that is common rolled into libraries
and “middleware”
2: Application Layer
7
Application architectures
 Client-server
 Peer-to-peer (P2P)
 Hybrid of client-server and P2P
2: Application Layer
8
Client-server architecture
server:



always-on host
permanent IP address
server farms for scaling
clients:




communicate with
server
may be intermittently
connected
may have dynamic IP
addresses
do not communicate
directly with each other
2: Application Layer
9
Pure P2P architecture
 every participant is both a
client and a server to other
clients


arbitrary end systems
directly communicate
Resource costs shared
 peers are intermittently
connected, change IP
addresses
 example: Gnutella
Highly scalable
But difficult to manage
2: Application Layer
10
Hybrid of client-server and P2P
Napster
File transfer P2P
 File search centralized:

• Peers register content at central server
• Peers query same central server to locate content
Instant messaging
Chatting between two users is P2P
 Presence detection/location centralized:

• User registers its IP address with central server
when it comes online
• User contacts central server to find IP addresses of
buddies
2: Application Layer
11
Processes communicating
Process: program running
within a host.
 within same host, two
processes communicate
using inter-process
communication (defined
by OS).
 processes in different
hosts communicate by
exchanging messages
Client process: process
that initiates
communication
Server process: process
that waits to be
contacted
 Note: applications with
P2P architectures have
client processes &
server processes
2: Application Layer
12
Sockets
 process sends/receives
messages to/from its
socket
 socket analogous to door


sending process shoves
message out door
sending process relies on
transport infrastructure
on other side of door which
brings message to socket
at receiving process
host or
server
host or
server
process
controlled by
app developer
process
socket
socket
TCP with
buffers,
variables
Internet
TCP with
buffers,
variables
controlled
by OS
 API: (1) choice of transport protocol; (2) ability to fix
a few parameters (lots more on this later)
2: Application Layer
13
Addressing processes
 For a process to
receive messages, it
must have an identifier
 A host has a unique32bit IP address
 Q: does the IP address
of the host on which
the process runs
suffice for identifying
the process?
 Answer: No, many
processes can be
running on same host
 Identifier includes
both the IP address
and port numbers
associated with the
process on the host.
 Example port numbers:


HTTP server: 80
Mail server: 25
 More on this later
2: Application Layer
14
App-layer protocol defines
 Types of messages
exchanged, e.g., request
& response messages
 Syntax of message
types: what fields in
messages & how fields
are delineated
 Semantics of the fields,
i.e., meaning of
information in fields
 Rules for when and how
processes send &
respond to messages
Public-domain protocols:
 defined in RFCs
 allows for
interoperability
 e.g., HTTP, SMTP
Proprietary protocols:
 e.g., KaZaA
2: Application Layer
15
What transport service does an app need?
Data loss
 some apps (e.g., audio) can
tolerate some loss
 other apps (e.g., file
transfer, telnet) require
100% reliable data
transfer
Timing
 some apps (e.g.,
Internet telephony,
interactive games)
require low delay to be
“effective”
Bandwidth
 some apps (e.g.,
multimedia) require
minimum amount of
bandwidth to be
“effective”
 other apps (“elastic
apps”) make use of
whatever bandwidth
they get
2: Application Layer
16
Transport service requirements of common apps
Data loss
Bandwidth
Time Sensitive
file transfer
e-mail
Web documents
real-time audio/video
no loss
no loss
no loss
loss-tolerant
no
no
no
yes, 100’s msec
stored audio/video
interactive games
instant messaging
loss-tolerant
loss-tolerant
no loss
elastic
elastic
elastic
audio: 5kbps-1Mbps
video:10kbps-5Mbps
same as above
few kbps up
elastic
Application
yes, few secs
yes, 100’s msec
yes and no
2: Application Layer
17
Internet transport protocols services
TCP service:
 connection-oriented: setup




required between client and
server processes
reliable transport between
sending and receiving process
flow control: sender won’t
overwhelm receiver
congestion control: throttle
sender when network
overloaded
does not provide: timing,
minimum bandwidth
guarantees
UDP service:
 unreliable data transfer
between sending and
receiving process
 does not provide:
connection setup,
reliability, flow control,
congestion control, timing,
or bandwidth guarantee
Q: why bother? Why is
there a UDP?
2: Application Layer
18
Internet apps: application, transport protocols
Application
e-mail
remote terminal access
Web
file transfer
streaming multimedia
Internet telephony
Application
layer protocol
Underlying
transport protocol
SMTP [RFC 2821]
Telnet [RFC 854]
HTTP [RFC 2616]
FTP [RFC 959]
proprietary
(e.g. RealNetworks)
proprietary
(e.g., Vonage,Dialpad)
TCP
TCP
TCP
TCP
TCP or UDP
typically UDP
2: Application Layer
19
Chapter 2: Application layer
 2.1 Principles of
network applications
 2.2 Web and HTTP
 2.3 FTP
 2.4 Electronic Mail

SMTP, POP3, IMAP
 2.5 DNS
 2.6 P2P file sharing
 2.7 Socket programming
with TCP
 2.8 Socket programming
with UDP
 2.9 Building a Web
server
2: Application Layer
20
Programming application
protocols
Goal: learn how to build client/server application that
communicate using sockets without understanding underlying
functions of TCP/IP
 Many possible programming interfaces
• Socket APIs (most common)
– BSD C Socket API (most common)
– Java socket API
– Python socket API
• Other APIs
– Client-side Java URLconnections
– Server-side Java servlets
– Python urllib
– Python HTTPServer
– RPC, CORBA, Java RMI (not covered)
2: Application Layer
21
Socket programming
Socket API
 introduced in BSD4.1 UNIX,
1981
 explicitly created, used,
released by apps
 client/server paradigm
 two types of transport
service via socket API:
 unreliable datagram
 reliable, byte streamoriented
socket
a host-local,
application-created,
OS-controlled interface
(a “door”) into which
application process can
both send and
receive messages to/from
another application
process
2: Application Layer
22
Socket-programming using TCP
Socket: a door between application process and endend-transport protocol (UCP or TCP)
TCP service: reliable transfer of bytes from one
process to another
controlled by
application
developer
controlled by
operating
system
process
process
socket
TCP with
buffers,
variables
host or
server
internet
socket
TCP with
buffers,
variables
controlled by
application
developer
controlled by
operating
system
host or
server
2: Application Layer
23
Socket programming with TCP
Client must contact server
 server process must first
be running
 server must have created
socket (door) that
welcomes client’s contact
Client contacts server by:
 creating client-local TCP
socket
 specifying IP address, port
number of server process
 When client creates
socket: client TCP
establishes connection to
server TCP
 When contacted by client,
server TCP creates new
socket for server process to
communicate with client
 allows server to talk with
multiple clients
 source port numbers
used to distinguish
clients (more in Chap 3)
application viewpoint
TCP provides reliable, in-order
transfer of bytes (“pipe”)
between client and server
2: Application Layer
24
Sockets in action
*,SIP:80
*,SIP:80
CIP:1099,SIP:80
CIP:1100,SIP:80
CIP:1099,SIP:80
*,SIP:80
CIP:1099,SIP:80
CIP:1099,SIP:80
CIP:1100,SIP:80
CIP:1099,SIP:80
CIP:1100,SIP:80
2: Application Layer
25
BSD sockets in C/Unix
 Socket API (socket.h)
• socket(): create unnamed socket (data structure)
– UDP (SOCK_DGRAM), TCP (SOCK_STREAM)
– IP (SOCK_RAW)
• bind(): name socket (bind local address to socket)
• listen(): enable socket to accept connections
• accept(): get connect() request from listen queue, allocate
file descriptor for new socket
• connect(): initiate connection on a socket (TCP handshake)
• send(), sendto(), sendmsg(), writev(), write(): send data
• recv(), recvfrom(), recvmsg(), readv(), read(): receive data
• setsockopt(), getsockopt(): set socket options (such as
buffer sizes, flag fields)
• close(), shutdown(): teardown connection
2: Application Layer
26
BSD sockets in action
UDP example
client
server
socket()
socket()
bind()
bind()
TCP example
client
server
socket()
socket()
bind()
connect()
listen()
sendto()
accept()
recvfrom()
write()
read()
sendto()
write()
recvfrom()
read()
2: Application Layer
27
BSD example
 http://www.thefengs.com/wuchang/work/cou
rses/cs594/socket_example
 TCP socket code

Client
• tcpcli.c

Server
• tcpserv.c
 UDP socket code
 Client
• udpcli.c

Server
• udpserv.c
2: Application Layer
28
Java network programming
Java network applications
java.net package
System-dependent
implementations
2: Application Layer
29
Java installation on linuxlab
 J2SE

javac
• java compiler

java
• java interpreter
• http://www.ibiblio.org/javafaq/javatut
orial.html
2: Application Layer
30
java.net classes
 Low-level networking classes (Sockets and
Packets)
 High-level URL networking classes
 http://java.sun.com/j2se/1.5.0/docs/api/
 java.lang.Object






java.net.Socket
java.net.ServerSocket
java.net.DatagramSocket
java.net.DatagramPacket
java.net.URL
java.net.URLConnection
• java.net.HttpURLConnection


java.net.URLencoder
java.net.InetAddress
2: Application Layer
31
java.net.Socket
 Constructors
 Socket(InetAddress, int)
 Socket(String, int)
 Socket(InetAddress, int, InetAddress, int)
 Some methods
 getInputStream()
 getOutputStream
 getInetAddress()
 getPort()
 getLocalAddress()
 getLocalPort()
 get/set individual socket options
2: Application Layer
32
java.net.ServerSocket
 Constructors
ServerSocket(int)
 ServerSocket(int, int) // backlog specified
 ServerSocket(int, int, InetAddress) // local
address and backlog specified

 Some methods
 accept()
 getInetAddress()
 getLocalPort()
2: Application Layer
33
Stream jargon for Java
network programming
 A stream is a sequence of
characters that flow into
or out of a process.
 An input stream is
attached to some input
source for the process,
e.g., keyboard or socket.
 An output stream is
attached to an output
source, e.g., monitor or
socket.
2: Application Layer
34
Socket programming with TCP
Client
Process
process
input
stream
output
stream
inFromServer
1) client reads line from
standard input (inFromUser
stream) , sends to server via
socket (outToServer
stream)
2) server reads line from socket
3) server converts line to
uppercase, sends back to
client
4) client reads, prints modified
line from socket
(inFromServer stream)
outToServer
Example client-server app:
monitor
inFromUser
keyboard
input
stream
client
TCP
clientSocket
socket
to network
TCP
socket
from network
2: Application Layer
35
Client/server socket interaction: TCP
Server (running on hostid)
Client
create socket,
port=x, for
incoming request:
welcomeSocket =
ServerSocket()
TCP
wait for incoming
connection request connection
connectionSocket =
welcomeSocket.accept()
read request from
connectionSocket
write reply to
connectionSocket
close
connectionSocket
setup
create socket,
connect to hostid, port=x
clientSocket =
Socket()
send request using
clientSocket
read reply from
clientSocket
close
clientSocket
2: Application Layer
36
Example: Java client (TCP)
import java.io.*;
import java.net.*;
class TCPClient {
public static void main(String argv[]) throws Exception
{
String sentence;
String modifiedSentence;
Create
input stream
Create
client socket,
connect to server
Create
output stream
attached to socket
BufferedReader inFromUser =
new BufferedReader(new InputStreamReader(System.in));
Socket clientSocket = new Socket("hostname", 6789);
DataOutputStream outToServer =
new DataOutputStream(clientSocket.getOutputStream());
2: Application Layer
37
Example: Java client (TCP), cont.
Create
input stream
attached to socket
BufferedReader inFromServer =
new BufferedReader(new
InputStreamReader(clientSocket.getInputStream()));
sentence = inFromUser.readLine();
Send line
to server
outToServer.writeBytes(sentence + '\n');
Read line
from server
modifiedSentence = inFromServer.readLine();
System.out.println("FROM SERVER: " + modifiedSentence);
clientSocket.close();
}
}
2: Application Layer
38
Example: Java server (TCP)
import java.io.*;
import java.net.*;
class TCPServer {
Create
welcoming socket
at port 6789
Wait, on welcoming
socket for contact
by client
Create input
stream, attached
to socket
public static void main(String argv[]) throws Exception
{
String clientSentence;
String capitalizedSentence;
ServerSocket welcomeSocket = new ServerSocket(6789);
while(true) {
Socket connectionSocket = welcomeSocket.accept();
BufferedReader inFromClient =
new BufferedReader(new
InputStreamReader(connectionSocket.getInputStream()));
2: Application Layer
39
Example: Java server (TCP), cont
Create output
stream, attached
to socket
DataOutputStream outToClient =
new DataOutputStream(connectionSocket.getOutputStream());
Read in line
from socket
clientSentence = inFromClient.readLine();
capitalizedSentence = clientSentence.toUpperCase() + '\n';
Write out line
to socket
outToClient.writeBytes(capitalizedSentence);
}
}
}
End of while loop,
loop back and wait for
another client connection
2: Application Layer
40
Chapter 2: Application layer
 2.1 Principles of
network applications
 2.2 Web and HTTP
 2.3 FTP
 2.4 Electronic Mail

SMTP, POP3, IMAP
 2.5 DNS
 2.6 P2P file sharing
 2.7 Socket programming
with TCP
 2.8 Socket programming
with UDP
 2.9 Building a Web
server
2: Application Layer
41
Socket programming with UDP
UDP: no “connection” between
client and server
 no handshaking
 sender explicitly attaches
IP address and port of
destination to each packet
 server must extract IP
address, port of sender
from received packet
application viewpoint
UDP provides unreliable transfer
of groups of bytes (“datagrams”)
between client and server
UDP: transmitted data may be
received out of order, or
lost
2: Application Layer
42
java.net.DatagramSocket
 Constructors
 DatagramSocket()
 DatagramSocket(int) // bind to specific port
 DatagramSocket(int, InetAddress) // specify
local address
 Some methods
 getLocalAddress()
 getLocalPort()
 receive(DatagramPacket)
 send(DatagramPacket)
 get/set individual socket options
2: Application Layer
43
java.net.DatagramPacket
 Constructors
 DatagramPacket(byte[], int) // receiving packets
 DatagramPacket(byte[], int, InetAddress, int) // sending
packets
 Some methods
 getAddress() // remote address
 getPort() // remote port
 getLength() // get packet length
 getData() // return data received or to be sent
 setAddress(InetAddress) // set remote address
 setData(byte[]) // set packet data
 setLength(int) // set packet length
 setPort(int) // set remote port
2: Application Layer
44
Client/server socket interaction: UDP
Server (running on hostid)
create socket,
port=x, for
incoming request:
serverSocket =
DatagramSocket()
read request from
serverSocket
write reply to
serverSocket
specifying client
host address,
port number
Client
create socket,
clientSocket =
DatagramSocket()
Create, address (hostid, port=x,
send datagram request
using clientSocket
read reply from
clientSocket
close
clientSocket
2: Application Layer
45
Example: Java client (UDP)
input
stream
Client
process
monitor
inFromUser
keyboard
Process
Input: receives
packet (recall
thatTCP received
“byte stream”)
UDP
packet
receivePacket
packet (recall
that TCP sent
“byte stream”)
sendPacket
Output: sends
client
UDP
clientSocket
socket
to network
UDP
packet
UDP
socket
from network
2: Application Layer
46
Example: Java client (UDP)
import java.io.*;
import java.net.*;
Create
input stream
Create
client socket
Translate
hostname to IP
address using DNS
class UDPClient {
public static void main(String args[]) throws Exception
{
BufferedReader inFromUser =
new BufferedReader(new InputStreamReader(System.in));
DatagramSocket clientSocket = new DatagramSocket();
InetAddress IPAddress = InetAddress.getByName("hostname");
byte[] sendData = new byte[1024];
byte[] receiveData = new byte[1024];
String sentence = inFromUser.readLine();
sendData = sentence.getBytes();
2: Application Layer
47
Example: Java client (UDP), cont.
Create datagram
with data-to-send,
length, IP addr, port
DatagramPacket sendPacket =
new DatagramPacket(sendData, sendData.length, IPAddress, 9876);
Send datagram
to server
clientSocket.send(sendPacket);
Read datagram
from server
clientSocket.receive(receivePacket);
DatagramPacket receivePacket =
new DatagramPacket(receiveData, receiveData.length);
String modifiedSentence =
new String(receivePacket.getData());
System.out.println("FROM SERVER:" + modifiedSentence);
clientSocket.close();
}
}
2: Application Layer
48
Example: Java server (UDP)
import java.io.*;
import java.net.*;
Create
datagram socket
at port 9876
class UDPServer {
public static void main(String args[]) throws Exception
{
DatagramSocket serverSocket = new DatagramSocket(9876);
byte[] receiveData = new byte[1024];
byte[] sendData = new byte[1024];
while(true)
{
Create space for
received datagram
Receive
datagram
DatagramPacket receivePacket =
new DatagramPacket(receiveData, receiveData.length);
serverSocket.receive(receivePacket);
2: Application Layer
49
Example: Java server (UDP), cont
String sentence = new String(receivePacket.getData());
Get IP addr
port #, of
sender
InetAddress IPAddress = receivePacket.getAddress();
int port = receivePacket.getPort();
String capitalizedSentence = sentence.toUpperCase();
sendData = capitalizedSentence.getBytes();
Create datagram
to send to client
DatagramPacket sendPacket =
new DatagramPacket(sendData, sendData.length, IPAddress,
port);
Write out
datagram
to socket
serverSocket.send(sendPacket);
}
}
}
End of while loop,
loop back and wait for
another datagram
2: Application Layer
50
High-level Java networking
classes
 Socket/Packet
Low level building blocks
 Must implement all application-level logic

 Many protocols based on URLs and/or
tunneled in HTTP
Program at a higher-level to hide underlying
protocol details
 Do not re-implement HTTP, URL parsing, MIME
handling for each application

2: Application Layer
51
High-level client-side Java
networking classes
 java.net.URL

Represent a URL object
 java.net.URLConnection
 Represent a connection to a URL which can be
read and written from
 java.net.HttpURLConnection
 Subclass of URLConnection for http:// URLs
 Example
2: Application Layer
52
Java high-level client-side
networking classes example
 http://www.thefengs.com/wuchang/work/c
ourses/cs594/java_example
import java.net.*;
import java.io.*;
public class Test {
public static void main(String argv[]) {
try {
URL u=new URL(http://www.google.com/);
URLConnection uc=u.openConnection();
Object o = (Object) uc.getContent();
} catch (Exception e) {
}
}
2: Application Layer
53
High-level server-side Java
networking classes
 Servlets
 Dynamically generate content
 Implement common protocol header logic
• Example http servlets
– Cookies
– Content-type
– Content-length
 Servlet classes
 javax.servlet.Servlet javax.servlet.HttpServlet
• init()
• service()
• destroy()


javax.servlet.ServletRequest
javax.servlet.ServletResponse
javax.servlet.HttpServletRequest
javax.servlet.HttpServletResponse
2: Application Layer
54
Python network programming
Python network applications
Python network packages
(socket, URLlib, HTTPServer)
System-dependent
implementations
2: Application Layer
55
Python network programming
 Python
Scripting language
 No compilation required
 Language reference: http://www.python.org
 Provides APIs similar to Java

• socket
– Low-level socket interface
• urllib
– HTTP client
• SimpleHTTPServer
– HTTP server
2: Application Layer
56
Python sockets
 Similar to C and Java

Client
import socket
host = “localhost”
port = 7
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host,port))
s.send(“some data to echo”)
print s.recv(20)
s.close
2: Application Layer
57
Python sockets
 Similar to C and Java

Server
import socket
host = “”
port = 7
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((host,port))
s.listen(1)
while (1):
conn, addr = s.accept()
data = conn.recv(20)
conn.send(data)
conn.close()
2: Application Layer
58
Python urllib/urllib2
 Client-side HTTP code similar to Java’s
java.net.URLConnection

Hides socket creation, HTTP request
formatting, HTTP response parsing
Python urllib
Python urllib2
url = sys.argv[1]
sock = urllib.urlopen(url)
htmlSource = sock.read()
sock.close()
print htmlSource
url = sys.argv[1]
txdata = None
txheaders = {
‘Accept-Language’: ‘en-us’
}
req = urllib2.Request(url, txdata, txheaders)
u = urlib2.urlopen(req)
headers = u.info()
2: Application Layer
59
print u.read()
Python BasicHTTPServer
 Server-side HTTP processing libraries
javax.servlet.HTTPServlet
Hides socket creation, HTTP request parsing,
HTTP response formatting
 SocketServer.TCPServer.BaseHTTPServer

2: Application Layer
60
Python network programming
URLs
 http://docs.python.org/lib/internet.html
 http://docs.python.org/lib/module-socket.html
 http://docs.python.org/lib/module-urllib.html
 http://docs.python.org/lib/module-urllib2.html
 http://docs.python.org/lib/module-
SimpleHTTPServer.html
 http://docs.python.org/lib/moduleBaseHTTPServer.html
 http://www.w3journal.com/6/s3.vanrossum.html
2: Application Layer
61
Chapter 2: Application layer
 2.1 Principles of
network applications


app architectures
app requirements
 2.2 Web and HTTP
 2.4 Electronic Mail
 SMTP, POP3, IMAP
 2.5 DNS
 2.6 P2P file sharing
 2.7 Socket programming
with TCP
 2.8 Socket programming
with UDP
 2.9 Building a Web
server
2: Application Layer
62
Web and HTTP
First some jargon
 Web page consists of objects
 Object can be HTML file, JPEG image, Java
applet, audio file,…
 Each object is addressable by a URL
 Web page consists of base HTML-file which
includes several referenced objects
 Example URL:
www.someschool.edu/someDept/pic.gif
host name
path name
2: Application Layer
63
HTTP overview
 HTTP: hypertext transfer
protocol
 Web’s application layer protocol
 client/server model


client: browser that requests,
receives, “displays” Web
objects
server: Web server sends
objects in response to requests
PC running
Explorer
Server
running
Apache Web
server
 HTTP 1.0: RFC 1945

http://www.rfc-editor.org/rfc/rfc1945.txt
 HTTP 1.1: RFC 2068

http://www.rfc-editor.org/rfc/rfc2068.txt
 HTTP state management
(cookies): RFC 2109

Mac running
Navigator
http://www.rfc-editor.org/rfc/rfc2109.txt
2: Application Layer
64
HTTP overview (continued)
Uses TCP:
 client initiates bi-directional
TCP connection (via socket)
to server, port 80
 server accepts TCP
connection from client
 HTTP messages (applicationlayer protocol messages)
exchanged between browser
(HTTP client) and Web
server (HTTP server)

Messages encoded in text
 TCP connection closed
HTTP is “stateless”
 server maintains no
information about
past client requests
aside
Protocols that maintain
“state” are complex!
 past history (state) must
be maintained
 if server/client crashes,
their views of “state” may
be inconsistent, must be
reconciled
2: Application Layer
65
Non-persistent HTTP
connections
 HTTP/0.9 & HTTP/1.0
 One request/response per connection
 simple to implement
 server parses request, responds, and closes connection
 Disadvantages
 Short transfers hard on TCP
• Each connection stuck in slow start
• Loss recovery poor when windows small

Connection setup latency (three-way handshake)
• Several extra round trips added to transfer

Server and network overhead
• Connection handling, extra packets
• TIME_WAIT
– Much larger than # active connections
2: Application Layer
66
Nonpersistent HTTP
(contains text,
Suppose user enters URL
references to 10
www.someSchool.edu/someDepartment/home.index
jpeg images)
1a. HTTP client initiates TCP
connection to HTTP server
(process) at
www.someSchool.edu on port 80
2. HTTP client sends HTTP
request message (containing
URL) into TCP connection
socket. Message indicates
that client wants object
someDepartment/home.index
1b. HTTP server at host
www.someSchool.edu waiting
for TCP connection at port 80.
“accepts” connection, notifying
client
3. HTTP server receives request
message, forms response
message containing requested
object, and sends message
into its socket
time
2: Application Layer
67
Nonpersistent HTTP (cont.)
4. HTTP server closes TCP
5. HTTP client receives response
connection.
message containing html file,
displays html. Parsing html
file, finds 10 referenced jpeg
objects
time 6. Steps 1-5 repeated for each
of 10 jpeg objects
2: Application Layer
68
Response time modeling
Definition of RTT: time to send
a small packet to travel from
client to server and back.
Response time:
 one RTT to initiate TCP
connection
 one RTT for HTTP request
and first few bytes of HTTP
response to return
 file transmission time
total = 2RTT+transmit time
initiate TCP
connection
RTT
request
file
time to
transmit
file
RTT
file
received
time
time
2: Application Layer
69
Single non-persistent example
Client
SYN
0 RTT
Client opens TCP connection
1 RTT
Client sends HTTP request for HTML
Server
SYN
DAT
ACK
2 RTT
ACK
DAT
FIN
Server reads from disk
ACK
Client parses HTML
Client opens TCP connection
FIN
ACK
3 RTT
Client sends HTTP request for image
SYN
SYN
ACK
DAT
Server reads from disk
ACK
4 RTT
Image begins to arrive
DAT
2: Application Layer
70
Parallel non-persistent
connections (Netscape)
 Improve non-persistent latency by using multiple
concurrent connections


Different parts of Web page arrive independently on
separate connections (object demux via connections)
Can grab more of the network bandwidth than other
users
 Doesn’t necessarily improve response time
 TCP loss recovery ends up being timeout dominated
because windows are small
2: Application Layer
71
Persistent HTTP connections
 Default in HTTP/1.1
 Several requests/responses on one TCP connection

On same TCP connection: server, parses request, responds,
parses new request,..
 Benefits greatest for small objects

Up to 2x improvement in response time (avoid handshake)
 Server resource utilization reduced due to fewer connection
establishments and fewer active connections
 TCP behavior improved


Longer connections help adaptation to available bandwidth
Larger congestion window improves loss recovery
 HTTP/1.1 vs. HTTP/1.0 example

Multiple requests to www.cs.pdx.edu
 Problem: serial delivery of objects (head-of-line object
blocking)
2: Application Layer
72
Persistent HTTP connection
example
Server
Client
0 RTT
Client sends HTTP request for HTML
DAT
ACK
Server reads from disk
DAT
1 RTT
ACK
Client parses HTML
Client sends HTTP request for image
DAT
ACK
Server reads from disk
DAT
2 RTT
Image begins to arrive
2: Application Layer
73
Persistent HTTP connections
 Pipelining requests
 Examples
• Getall – request HTML document and all embeds
– Requires server to parse HTML files
– Embeds returned serially
– Doesn’t consider client cached documents
• Getlist – request a set of documents
– Implemented as a simple set of GETs

Problems with pipelined serialized requests
• Stall in one object prevents delivery of others
• Most useful information in first few bytes (layout info)
– Multiple connections allow incremental rendering of images
• Need application-level demux to emulate multiple
connections
2: Application Layer
74
HTTP-NG?
 HTTP-NG, HTTP/2.0, HTTP range requests
Support pipelining, but avoid HOL blocking via
application-layer demux of objects
 Demux within application stream

• Data “labeled” with object it is for

Application specific solution to transport
protocol problems!
 Better approach?
 SCTP: Stream Control Transmission Protocol
• Support above across all applications
2: Application Layer
75
Persistent HTTP
Nonpersistent HTTP issues:
 requires 2 RTTs per object
 OS overhead to allocate
resources for each TCP
connection
 browsers often open parallel
TCP connections to fetch
referenced objects
 Solves demux issue on
multiple objects
Persistent HTTP
 server leaves connection
open after sending response
 subsequent HTTP messages
between same client/server
sent over open connection
Persistent without pipelining:
 client issues new request
only when previous
response has been received
 one RTT for each
referenced object
Persistent with pipelining:
 default in HTTP/1.1
 client sends requests as
soon as it encounters a
referenced object
 as little as one RTT for all
the referenced objects
 Objects still returned one
at a time (HOL blocking
versus parallel nonpersistent HTTP)
2: Application Layer
76
HTTP request message
 two types of HTTP messages: request, response
 HTTP request message:
 ASCII (human-readable format)
request line
(GET, POST,
HEAD commands)
GET /somedir/page.html HTTP/1.1
Host: www.someschool.edu
User-agent: Mozilla/4.0
header Connection: close
lines Accept-language:fr
Carriage return,
line feed
indicates end
of message
(extra carriage return, line feed)
2: Application Layer
77
HTTP request message: general format
2: Application Layer
78
HTTP request line (methods)
HTTP/1.0
 GET

Return object specified
by URI
 POST
 Send data to server
(forms)
 HEAD
 asks server to leave
requested object out of
response
 Return headers only of
GET response
HTTP/1.1
 GET, POST, HEAD
 PUT

uploads file in entity
body to path specified
in URL field
 DELETE
 deletes file specified in
the URL field
 OPTIONS, TRACE,
CONNECT
2: Application Layer
79
HTTP request line (cont.)
 URI

Object to retrieve
• E.g. http://www.cs.pdx.edu/index.html with a proxy
• E.g. /index.html if no proxy
 HTTP version
 Version being used
 HTTP 1.1
• Host: header required
• Connection: header supported
2: Application Layer
80
Common HTTP request headers
 Accept
 Acceptable document types, encodings, languages, character
sets
 If-Modified-Since
 For use with caching
 Referer
 URL which caused this page to be requested
 User-Agent
 Host
 For multiple web sites hosted on same server
 Connection
 Keep connection alive for subsequent request or close
connection
2: Application Layer
81
Other HTTP request headers
 Authorization
 Authentication info for HTTP authentication
 From
 User email (when privacy is disabled)
Rest of HTTP request
 Blank-line
 Separate request headers from POST information
 End of request
 Body
 If POST, send POST information
2: Application Layer
82
Handling user input (forms)
GET method:
 Input is uploaded in
URL field of request
line
POST method:
 Input is uploaded to
server in entity
body
GET search?name=george&animal=monkey HTTP/1.1
Host: www.somesite.com
POST search HTTP/1.1
Host: www.somesite.com
Content-type: application/x-www-form-urlencoded
name=george&animal=monkey
2: Application Layer
83
HTTP response message
status line
(protocol
status code
status phrase)
header
lines
data, e.g.,
requested
HTML file
HTTP/1.1 200 OK
Connection close
Date: Thu, 06 Aug 1998 12:00:15 GMT
Server: Apache/1.3.0 (Unix)
Last-Modified: Mon, 22 Jun 1998 …...
Content-Length: 6821
Content-Type: text/html
data data data data data ...
2: Application Layer
84
HTTP response message: general
format
2: Application Layer
85
HTTP response format
 Status-line
HTTP version
 3 digit response code

•
•
•
•
•

1XX – informational
2XX – success
3XX – redirection
4XX – client error
5XX – server error
Reason phrase
2: Application Layer
86
HTTP response status codes
A few sample codes:
200 OK

request succeeded, requested object later in this message
301 Moved Permanently

requested object moved, new location specified later in
this message (Location:)
400 Bad Request

request message not understood by server
404 Not Found

requested document not found on this server
505 HTTP Version Not Supported
2: Application Layer
87
Common HTTP response
headers
 Server
 server software
 Content-Encoding
 x-gzip
 Content-Length
 Content-Type
 Expires
 Last-Modified
 ETag
2: Application Layer
88
Other HTTP response headers
 Location
 redirection
 WWW-Authenticate
 request for authentication
 Allow
 list of methods supported (GET, HEAD, etc)
Rest of HTTP response
 Blank-line
 Separate headers from data
 Body
 Data being returned to client
2: Application Layer
89
HTTP headers by function
 Authentication
 Client
• Authorization, ProxyAuthorization

Server
 Caching
 General
• Cache-control, Pragma

• If-Modified-Since, IfUnmodified-Since, IfMatch
• WWW-authenticate,
Proxy-Authenticate
 User, server tracking
 Client
• Cookie, Referer, From,
User-agent

Client

Server
• Last-Modified,
Expires, ETag, Age
Server
• Set-cookie, Server
2: Application Layer
90
Trying out HTTP (client side) for yourself
1. Telnet to your favorite Web server:
telnet cis.poly.edu 80
Opens TCP connection to port 80
(default HTTP server port) at cis.poly.edu.
Anything typed in sent
to port 80 at cis.poly.edu
2. Type in a GET HTTP request:
GET /~ross/ HTTP/1.1
Host: cis.poly.edu
By typing this in (hit carriage
return twice), you send
this minimal (but complete)
GET request to HTTP server
3. Look at response message sent by HTTP server!
2: Application Layer
91
Let’s look at HTTP in action
 telnet example
 Ethereal example
 Other examples
 http://www.thefengs.com/courses/cs594/http.txt
 http://www.thefengs.com/courses/cs594/http_po
st.txt
2: Application Layer
92
User-server state: cookies
Many major Web sites
use cookies
Four components:
1) cookie header line of
HTTP response message
Set-cookie:
2) cookie header line in
HTTP request message
Cookie:
3) cookie file kept on
user’s host, managed by
user’s browser
4) back-end database at
Web site
Example:



Susan access Internet
always from same PC
She visits a specific ecommerce site for first
time
When initial HTTP
requests arrives at site,
site creates a unique ID
and creates an entry in
backend database for
ID
2: Application Layer
93
Cookies: keeping “state” (cont.)
client
Cookie file
server
usual http request msg
usual http response +
ebay: 8734
Cookie file
amazon: 1678
ebay: 8734
Set-cookie: 1678
usual http request msg
cookie: 1678
usual http response msg
one week later:
Cookie file
amazon: 1678
ebay: 8734
usual http request msg
cookie: 1678
usual http response msg
server
creates ID
1678 for user
cookiespecific
action
cookiespectific
action
2: Application Layer
94
Cookies (continued)
What cookies can bring:
 authorization
 shopping carts
 Site preferences
 recommendations
 user session state
(Web e-mail)
aside
Cookies and privacy:
 cookies permit sites to
learn a lot about you
 you may supply name
and e-mail to sites
 search engines use
redirection & cookies
to learn yet more
 advertising companies
obtain info across
sites
2: Application Layer
95
Web caches (proxy server)
Goal: satisfy client request without involving origin server
(i.e. do not send content that has not changed)
Why Web caching?
Info on web caching
 Reduce response time for
 http://www.ircache.net/
client request.
 http://www.squid.org
 Reduce traffic on an
 ICP
institution’s access link.
 http://www.rfc Reduce load on servers.
editor.org/rfc/rfc2186.txt
 Enables “poor” content
 http://www.rfcproviders to effectively
editor.org/rfc/rfc2187.txt
deliver content (but so does
P2P file sharing)
2: Application Layer
96
More about Web caching
 Browser sends all HTTP
requests to cache


object in cache: cache
returns object
else cache requests object
from origin server, then
returns object to client
 Done directly at client

origin
server
client
Proxy
server
Via browser web cache
 Along path from client to
origin server



Via proxy web cache
Proxy acts as both client
and server
Typically cache is installed
by ISP (university,
company, residential ISP)
client
origin
server
2: Application Layer
97
Caching example
Assumptions
 average object size = 100,000
bits
 avg. request rate from
institution’s browsers to origin
servers = 15/sec
 delay from institutional router
to any origin server and back
to router = 2 sec
Consequences
origin
servers
public
Internet
1.5 Mbps
access link
institutional
network
10 Mbps LAN
 utilization on LAN = 15%
 utilization on access link = 100%
 total delay
= Internet delay +
access delay + LAN delay
= 2 sec + minutes + milliseconds
institutional
cache
2: Application Layer
98
Caching example (cont)
Possible solution
 increase bandwidth of access
link to, say, 10 Mbps
Consequences
origin
servers
public
Internet
 utilization on LAN = 15%
 utilization on access link = 15%
= Internet delay +
access delay + LAN delay
= 2 sec + msecs + msecs
 often a costly upgrade
10 Mbps
access link
 Total delay
institutional
network
10 Mbps LAN
institutional
cache
2: Application Layer
99
Caching example (cont)
origin
servers
Install cache
 suppose hit rate is .4
Consequence
public
Internet
 40% requests will be
satisfied almost immediately
 60% requests satisfied by
origin server
 utilization of access link
reduced to 60%, resulting in
negligible delays (say 10
msec)
 total avg delay = Internet
delay + access delay + LAN
delay = .6*(2.01) secs +
.4*milliseconds < 1.4 secs
1.5 Mbps
access link
institutional
network
10 Mbps LAN
institutional
cache
2: Application Layer
100
Conditional GET
 Goal: don’t send object if
cache has up-to-date cached
version
 cache: specify date of
cached copy in HTTP request
If-modified-since:
<date>
 server: response contains no
object if cached copy is upto-date:
HTTP/1.0 304 Not
Modified
server
cache
HTTP request msg
If-modified-since:
<date>
HTTP response
object
not
modified
HTTP/1.0
304 Not Modified
HTTP request msg
If-modified-since:
<date>
HTTP response
object
modified
HTTP/1.0 200 OK
<data>
2: Application Layer
101
HTTP caching
 Additional caching methods

ETag and If-Match
• HTTP 1.1 has file signature as well
 When/how often should the original be
checked for changes?
Check every time?
 Check each session? Day? Etc?
 Use Expires header

• If no Expires, often use Last-Modified as estimate
2: Application Layer
102
Example Cache Check Request
GET / HTTP/1.1
Accept: */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
If-Modified-Since: Mon, 29 Jan 2001 17:54:18 GMT
If-None-Match: "7a11f-10ed-3a75ae4a"
User-Agent: Mozilla/4.0 (compatible; MSIE 5.5;
Windows NT 5.0)
Host: www.cs.pdx.edu
Connection: Keep-Alive
2: Application Layer
103
Example Cache Check Response
HTTP/1.1 304 Not Modified
Date: Tue, 27 Mar 2001 03:50:51 GMT
Server: Apache/1.3.14 (Unix) (RedHat/Linux) mod_ssl/2.7.1 OpenSSL/0.9.5a
DAV/1.0.2 PHP/4.0.1pl2 mod_perl/1.24
Connection: Keep-Alive
Keep-Alive: timeout=15, max=100
ETag: "7a11f-10ed-3a75ae4a"
2: Application Layer
104
Content distribution networks (CDNs)
origin server
in North America
• “Active caching” (Akamai)
• Push content close to client a
priori
• Versus pulling content
through cache
CDN distribution node
• CDN company puts content
servers throughout Internet
– in lower-tier ISPs, close to
users
• CDN replicates customers’
content in CDN servers.
• When provider updates
content, CDN updates
servers
CDN server
in S. America CDN server
in Europe
CDN server
in Asia
2: Application Layer
105
CDN example
HTTP request for
www.foo.com/sports/sports.html
1
2
3
Origin server
DNS query for www.cdn.com
CDNs authoritative
DNS server
HTTP request for
www.cdn.com/www.foo.com/sports/ruth.gif
Nearby
CDN server
 origin server



www.foo.com
distributes HTML
Replaces:
http://www.foo.com/sports.ruth.gif
with
http://www.cdn.com/www.foo.com/
sports/ruth.gif
 CDN (cdn.com)


distributes gif files to
content servers
uses its authoritative
DNS server to route
redirect requests
2: Application Layer
106
More about CDNs
routing requests
not just Web pages
• CDN creates a “map”,
• streaming stored
indicating distances
audio/video
from leaf ISPs and
• streaming realCDN nodes
time audio/video
• when query arrives at
authoritative DNS
server:
– server determines ISP
from which query
originates
– uses “map” to
determine best CDN
server
– CDN nodes create
application-layer
multicast overlay
network
2: Application Layer
107
Chapter 2: Application layer
 2.1 Principles of
network applications
 2.2 Web and HTTP
 2.3 FTP
 2.4 Electronic Mail

SMTP, POP3, IMAP
 2.5 DNS
 2.6 P2P file sharing
 2.7 Socket programming
with TCP
 2.8 Socket programming
with UDP
 2.9 Building a Web
server
2: Application Layer
108
FTP: the file transfer protocol
user
at host
FTP
FTP
user
client
interface
file transfer
local file
system
FTP
server
remote file
system
 transfer file to/from remote host
 client/server model
client: side that initiates transfer (either to/from
remote)
 server: remote host
 ftp: RFC 959
 ftp server: port 21

2: Application Layer
109
FTP: separate control, data
connections
 FTP client contacts FTP
server at port 21 using TCP


Control connection for
exchanging commands
Out-of-band signaling
between end-points
 Server maintains “state”
 Authorization status
 Current directory
 Contrast to HTTP
TCP control connection
port 21
FTP
client
TCP data connection
port 20
FTP
server
 For file transfer


Request via control connection
New TCP connection for data
xfer is negotiated and created
• Can be server or client initiated
• Includes IP address and port
information sent in application
layer message
– Why?
– Protocol allows client to
initiate transfer between 2 ftp
servers
• Data connection closed after
transfer
2: Application Layer
110
FTP commands, responses
Sample commands:
Sample return codes
 sent as ASCII text over
 status code and phrase (as
control channel
 USER username
 PASS password
 LIST return list of file in


current directory
 RETR filename retrieves

 STOR filename stores

(gets) file
(puts) file onto remote
host
in HTTP)
331 Username OK,
password required
125 data connection
already open;
transfer starting
425 Can’t open data
connection
452 Error writing
file
2: Application Layer
111
ftp and NAT (network address
translation)
 NAT

Dynamically assign source address to a pool of
available addresses
• “Statistically multiplex” address usage due to IP
address shortage
• Replaces source IP address of packets
• Each machine gets unique, external IP address out of
pool
• What to do if the “pool” runs out?
– NAT with port translation
2: Application Layer
112
ftp and NAT (network address
translation)
 NAPT (NAT with port translation)

Use the source port field (of TCP or UDP) along
with pool of IP addresses
• Example: single, globally routable external IP address
Packet #1
SrcIP=192.168.0.1
SrcPort=1312
DstIP=131.252.220.66
DstPort=21
Packet #2
SrcIP=192.168.0.2
SrcPort=1312
DstIP=131.252.220.66
DstPort=21
192.168.0.1
192.168.0.2
NAPT translator
ExternalIP=129.95.50.3
2: Application Layer
113
ftp and NAT (network address
translation)
 NAPT (NAT with port translation)

Use the source port field (of TCP or UDP) along
with pool of IP addresses
• Example: single, globally routable external IP address
Packet #1
SrcIP=192.168.0.1
SrcPort=1312
DstIP=131.252.220.66
DstPort=21
Packet #2
SrcIP=192.168.0.2
SrcPort=1312
DstIP=131.252.220.66
DstPort=21
Packet #1 after NAPT
SrcIP=129.95.50.3
SrcPort=2000
DstIP=131.252.220.66
DstPort=21
Packet #2 after NAPT
SrcIP=129.95.50.3
SrcPort=2001
DstIP=131.252.220.66
DstPort=21
192.168.0.1
192.168.0.2
NAPT translator
ExternalIP=129.95.50.3
2: Application Layer
114
ftp and NAT (network address
translation)
 NAPT (NAT with port translation)

Use the source port field (of TCP or UDP) along
with pool of IP addresses
• Example: single, globally routable external IP address
Reply #1
SrcIP=131.252.220.66
SrcPort=21
DstIP=129.95.50.3
DstPort=2000
Reply #2
SrcIP=131.252.220.66
SrcPort=21
DstIP=129.95.50.3
DstPort=2001
192.168.0.1
192.168.0.2
NAPT translator
ExternalIP=129.95.50.3
2: Application Layer
115
ftp and NAT (network address
translation)
 NAPT (NAT with port translation)

Use the source port field (of TCP or UDP) along
with pool of IP addresses
• Example: single, globally routable external IP address
Reply #1 after NAPT
SrcIP=131.252.220.66
SrcPort=21
DstIP=192.168.0.1
DstPort=1312
Reply #2 after NAPT
SrcIP=131.252.220.66
SrcPort=21
DstIP=192.168.0.2
DstPort=1312
Reply #1
SrcIP=131.252.220.66
SrcPort=21
DstIP=129.95.50.3
DstPort=2000
Reply #2
SrcIP=131.252.220.66
SrcPort=21
DstIP=129.95.50.3
DstPort=2001
192.168.0.1
192.168.0.2
NAPT translator
ExternalIP=129.95.50.3
2: Application Layer
116
ftp, NAT and PORT command
 Normal FTP mode
 Server has port 20, 21 reserved
 Client initiates control connection to port 21 on server
 Client allocates port X for data connection
 Client passes the data connection port (X) and its IP
address in a PORT command to server
 Server parses PORT command and initiates connection
from its own port 20 to the client on port X
 What if client is behind a NAT device?
• NAT must capture outgoing connections destined for port
21 and look to translate X within payload
– What if NAT doesn’t parse PORT command correctly?
– What if ftp server is running on a different port than 21?

http://www.practicallynetworked.com/support/linksys_ft
p_port.htm
2: Application Layer
117
ftp, NAT, and PORT command
 Passive (PASV) mode
 Client initiates control connection by connecting to port
21 on server
 Client enables “Passive” mode
 Server responds with PORT command giving client the IP
address and port to use for subsequent data connection
(usually port 20, but can be bypassed)
 Client initiates data connection by connecting to
specified port on server
 Most web browsers do PASV-mode ftp
 What if server is behind a NAT device?
• See client issues

What if both client and server are behind NAT devices?
• Problem
• Similar to P2P xfers
2: Application Layer
118
Chapter 2: Application layer
 2.1 Principles of
network applications
 2.2 Web and HTTP
 2.3 FTP
 2.4 Electronic Mail

SMTP, POP3, IMAP
 2.5 DNS
 2.6 P2P file sharing
 2.7 Socket programming
with TCP
 2.8 Socket programming
with UDP
 2.9 Building a Web
server
2: Application Layer
119
Electronic Mail
outgoing
message queue
user mailbox
user
agent
Three major components:
 user agents
 mail servers
mail
server
SMTP
 simple mail transfer
protocol: SMTP
User Agent
 a.k.a. “mail reader”
 composing, editing, reading
mail messages
 e.g., Eudora, Outlook, elm,
Netscape Messenger
 outgoing, incoming messages
stored on server
SMTP
mail
server
user
agent
SMTP
user
agent
mail
server
user
agent
user
agent
user
agent
2: Application Layer
120
Electronic Mail: mail servers
Mail Servers
 mailbox contains incoming
messages for user
 message queue of outgoing (to
be sent) mail messages
 SMTP protocol between mail
servers to send email messages
 Mail servers are both
clients and servers
• client: sending mail server
• server: receiving mail
server
• Peer-to-peer?
 SMTP protocol also between
user agents and mail servers to
send email messages
user
agent
mail
server
SMTP
SMTP
mail
server
user
agent
SMTP
user
agent
mail
server
user
agent
user
agent
user
agent
2: Application Layer
121
Electronic Mail: SMTP [RFC 821]
 uses TCP to reliably transfer email message from client
to server, port 25
 direct transfer

User agent to sending server
• If one runs SMTP locally (postfix, sendmail) or uses SMTP
built-in to client software

Sending server to receiving server
 three phases of transfer
handshaking (greeting)
 transfer of messages
 closure
 command/response interaction
 commands: ASCII text
 response: status code and phrase

 messages must be in 7-bit ASCII
2: Application Layer
122
Scenario: Alice sends message to Bob
1) Alice uses UA to compose
message and “to”
[email protected]
2) Alice’s UA sends message
to her mail server; message
placed in message queue
3) Client side of SMTP opens
TCP connection with Bob’s
mail server
1
user
agent
2
mail
server
3
4) SMTP client sends Alice’s
message over the TCP
connection
5) Bob’s mail server places the
message in Bob’s mailbox
6) Bob invokes his user agent
to read message
mail
server
4
5
6
user
agent
2: Application Layer
123
Sample SMTP interaction
S:
C:
S:
C:
S:
C:
S:
C:
S:
C:
C:
C:
S:
C:
S:
220 hamburger.edu
HELO crepes.fr
250 Hello crepes.fr, pleased to meet you
MAIL FROM: <[email protected]>
250 [email protected]... Sender ok
RCPT TO: <[email protected]>
250 [email protected] ... Recipient ok
DATA
354 Enter mail, end with "." on a line by itself
Do you like ketchup?
How about pickles?
.
250 Message accepted for delivery
QUIT
221 hamburger.edu closing connection
2: Application Layer
124
Try SMTP interaction for yourself:
 telnet servername 25
 see 220 reply from server
 enter HELO, MAIL FROM, RCPT TO, DATA, QUIT
commands
above lets you send email without using email client
(reader)
2: Application Layer
125
SMTP: final words
 SMTP uses persistent
connections
 SMTP requires message
(header & body) to be in 7bit ASCII
 SMTP server uses
CRLF.CRLF to determine
end of message
Comparison with HTTP:
 HTTP: pull
 SMTP: push
 both have ASCII
command/response
interaction, status codes
 HTTP: each object
encapsulated in its own
response msg
 SMTP: multiple objects
sent in multipart msg
2: Application Layer
126
Mail message format
SMTP: protocol for
exchanging email msgs
RFC 822: standard for text
message format:
 header lines, e.g.,
To:
 From:
 Subject:
different from SMTP
commands!

header
blank
line
body
 body

the “message”, ASCII
characters only
2: Application Layer
127
Message format: multimedia extensions
 MIME: multimedia mail extension, RFC 2045, 2056
 additional lines in msg header declare MIME content
type
MIME version
method used
to encode data
multimedia data
type, subtype,
parameter declaration
encoded data
From: [email protected]
To: [email protected]
Subject: Picture of yummy crepe.
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Type: image/jpeg
base64 encoded data .....
.........................
......base64 encoded data
2: Application Layer
128
MIME types
Text
Content-Type: type/subtype
• example subtypes: plain,
html
Image
• example subtypes: jpeg,
gif
Audio
• exampe subtypes: basic
(8-bit mu-law encoded),
32kadpcm (32 kbps
coding)
Video
• example subtypes:
mpeg, quicktime
Application
• other data that must be
processed by reader
before “viewable”
• example subtypes:
msword, octetstream
2: Application Layer
129
Multipart Types
From: [email protected]
To: [email protected]
Subject: Picture of yummy crepe.
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary=98766789
--98766789
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain
Dear Bob,
Please find a picture of a crepe.
--98766789
Content-Transfer-Encoding: base64
Content-Type: image/jpeg
base64 encoded data .....
.........................
......base64 encoded data
--98766789--
2: Application Layer
130
Mail access protocols
user
agent
SMTP
SMTP
sender’s mail
server
access
protocol
user
agent
receiver’s mail
server
 SMTP: delivery/storage to receiver’s server
 Mail access protocol: retrieval from server
 Direct (telnet or ssh followed by “mail”)



POP: Post Office Protocol [RFC 1939]
• authorization (agent <-->server) and download
IMAP: Internet Mail Access Protocol [RFC 1730]
• more features (more complex)
• manipulation of stored msgs on server
HTTP: Hotmail , Yahoo! Mail, Horde/IMP, etc.
2: Application Layer
131
POP3 protocol
authorization phase
 client commands:
user: declare username
 pass: password
 server responses
 +OK
 -ERR

transaction phase, client:
 list: list message numbers
 retr: retrieve message by
number
 dele: delete
 quit
S:
C:
S:
C:
S:
+OK POP3 server ready
user bob
+OK
pass hungry
+OK user successfully logged
C:
S:
S:
S:
C:
S:
S:
C:
C:
S:
S:
C:
C:
S:
list
1 498
2 912
.
retr 1
<message 1 contents>
.
dele 1
retr 2
<message 1 contents>
.
dele 2
quit
+OK POP3 server signing off
2: Application Layer
on
132
POP3 (more) and IMAP
More about POP3
 Previous example uses
“download and delete”
mode.
 Bob cannot re-read email if he changes
client
 “Download-and-keep”:
copies of messages on
different clients
 POP3 is stateless
across sessions
IMAP
 Keep all messages in
one place: the server
 Allows user to
organize messages in
folders
 IMAP keeps user state
across sessions:

names of folders and
mappings between
message IDs and folder
name
2: Application Layer
133
Chapter 2: Application layer
 2.1 Principles of
network applications
 2.2 Web and HTTP
 2.3 FTP
 2.4 Electronic Mail

SMTP, POP3, IMAP
 2.5 DNS
 2.6 P2P file sharing
 2.7 Socket programming
with TCP
 2.8 Socket programming
with UDP
 2.9 Building a Web
server
2: Application Layer
134
Domain Name System (DNS)
 Internet hosts, routers like to use fixed-
length addresses (numbers)

IP address (32 bit) - used for addressing
datagrams
 Humans like to use variable-length names
 www.cs.pdx.edu
 keywords
 DNS, keywords, naming protocols
 Map between IP addresses and names
2: Application Layer
135
Original Name to Address
Mapping
 Flat namespace
/etc/hosts.txt
 SRI kept main copy
 Downloaded regularly

 Problems
 Count of hosts was increasing
• From machine per domain to machine per user
• Many more downloads of hosts.txt
• Many more updates of hosts.txt
2: Application Layer
136
DNS: Domain Name System
(1984)
 application-layer protocol used by hosts
and name servers
communicate to resolve names (address/name
translation)
 core Internet function, implemented as
application-layer protocol

• complexity at network’s “edge”
• Compare to phone network
– Naming (none supported)
– Addressing (complex mechanism within network)
2: Application Layer
137
DNS: Domain Name System (1984)
 distributed database implemented in hierarchy of
many name servers

Goals
•
•
•
•
Scalability
Decentralized maintenance
Robustness, fault-tolerance
Global scope
– Names mean the same thing everywhere
• Don’t need
– Atomicity
– Strong consistency

Why not centralize DNS?
• Not scalable, hard to maintain, single point of failure


http://www.rfc-editor.org/rfc/rfc1034.txt
http://www.rfc-editor.org/rfc/rfc1035.txt
2: Application Layer
138
Other DNS issues
 Host aliasing

Canonical and alias names
 Mail server aliasing
 MX record
 Load distribution

Replicated Web servers: set of IP addresses
for one canonical name
2: Application Layer
139
DNS records
DNS: distributed db storing resource records (RR)
RR format: (name,
 Type=A
 name is hostname
 value is IP address
 Type=NS
 name is domain (e.g.
foo.com)
 value is hostname of
authoritative name
server for this domain
value, type, ttl)
 Type=CNAME
 name is alias name for some
“canonical” (the real) name
www.ibm.com is really
servereast.backup2.ibm.com

value is canonical name
 Type=MX
 value is name of mailserver
associated with name
2: Application Layer
140
DNS MX record type
 Initially only “A” records
 MX records created for mail
Point to mail exchanger for a name
 E.g. mail.acm.org is MX for acm.org

 Addition of MX record type proved to be a
challenge
How to get mail programs to lookup MX record
for mail delivery rather than A record?
 Needed critical mass of such mailers

2: Application Layer
141
DNS hierarchical canonical name
space
 www.cs.pdx.edu
root
org
gwu
edu
net
com
pdx
ucb
cs
uk
bu
ca
mit
ece
www
2: Application Layer
142
Maps closely to hierarchy
Root DNS Servers
com DNS servers
org DNS servers
yahoo.com
amazon.com
DNS servers DNS servers
pbs.org
DNS servers
edu DNS servers
poly.edu
umass.edu
DNS serversDNS servers
 Administrative hierarchy


Organized into regions known as “zones” with “.” as separator
zone = contiguous section of name space
 Zones created by convincing owner node to delegate subzone


umass.edu zone delegates cs.umass.edu to a different set of
authoritative name servers
Each zone contains multiple redundant servers (fault tolerance)
• Primary (master) name server updated manually
• Secondary (redundant) servers updated by zone transfer of name
space
2: Application Layer 143
DNS: Root name servers
 contacted by local name server that can not resolve name
 root name servers



contacts authoritative name server or intermediate name server if
name mapping not known
gets mapping and returns it to local name server
13 root name servers worldwide for fault-tolerance
• All that fit in a 512 octet SOA record
a Verisign, Dulles, VA
c Cogent, Herndon, VA (also Los Angeles)
d U Maryland College Park, MD
k RIPE London (also Amsterdam,
g US DoD Vienna, VA
Frankfurt)
i Autonomica, Stockholm (plus 3
h ARL Aberdeen, MD
j Verisign, ( 11 locations)
other locations)
m WIDE Tokyo
e NASA Mt View, CA
f Internet Software C. Palo Alto,
CA (and 17 other locations)
b USC-ISI Marina del Rey, CA
l ICANN Los Angeles, CA
2: Application Layer
144
TLD Servers
 Top-level domain (TLD) servers: responsible
for com, org, net, edu, etc, and all top-level
country domains uk, fr, ca, jp.
Network Solutions maintains servers for com TLD
 Educause for edu TLD

2: Application Layer
145
Authoritative Servers
 Provides authoritative hostname to IP mappings
 Typically, one per organization
 Hand mappings out for organization’s servers (Web & mail).
 Store parts of the database
 Each name assigned to authoritative server
 Server responds to all queries for name it is the authority
 Can be maintained by organization or service provider
 Root name servers are authorities for TLD names
 Root servers deliver authority for “.edu” TLD
 TLD name servers are authorities for their domains
 “.edu” TLD server delivers authority for “pdx.edu” name
 Each local DNS server has pointers to root servers
 Hard-coded into name server distributions
2: Application Layer
146
Local Name Server
 Does not strictly belong to hierarchy
 Each ISP (residential ISP, company, university)
has one.

Also called “default name server”
 When a host makes a DNS query, query is sent to
its local DNS server



Acts as a proxy, forwards query into hierarchy.
Typically answer queries about local zone directly
Do a lookup of distant host names for local hosts
• Configured with well-known root servers
• Currently {a-m}.root-servers.net
2: Application Layer
147
Client resolver
 Common code on client to query DNS
hierarchy
Typically a library that applications can link to
obtain gethostbyname() function
 Resolver configuration /etc/nsswitch.conf
 Local name servers hand-configured (e.g.
/etc/resolv.conf) or automatically configured
(DHCP)

• Can specify a file /etc/hosts
• Can specify a name server by its IP address (i.e.
129.95.50.2)

Host queries local name server for unknown
names
2: Application Layer
148
Inserting records into DNS
 Example: just created startup “Network Utopia”
 Register name networkuptopia.com at a registrar
(e.g., Network Solutions)


Need to provide registrar with names and IP addresses of
your authoritative name server (primary and secondary)
Registrar inserts two RRs into the com TLD server:
(networkutopia.com, dns1.networkutopia.com, NS)
(dns1.networkutopia.com, 212.212.212.1, A)
 Put in authoritative server
 Type A record for www.networkuptopia.com
 Type MX record for networkutopia.com
 How do people get the IP address of your Web site?
2: Application Layer
149
Query types
recursive query:
 puts burden of name resolution on contacted name server
 Only returns answer or “not found”
 Heavy load?


Root servers currently disable recursive querying through them
RFC 2010
iterated query:
 contacted server replies with name of server to contact
 “I don’t know this name, but ask this server”
 Client iteratively queries
2: Application Layer
150
Recursive query example
root DNS server
2
6
TLD DNS
server
local DNS server
1
wants IP address for
gaia.cs.umass.edu
3
7
dns.poly.edu
 Host at cis.poly.edu
5
4
8
requesting host
authoritative DNS server
dns.cs.umass.edu
cis.poly.edu
gaia.cs.umass.edu
2: Application Layer
151
Iterated query example
root DNS server
iterated
query
2
3
TLD DNS server
 Host at cis.poly.edu
wants IP address for
gaia.cs.umass.edu
4
5
local DNS server
dns.poly.edu
1
8
requesting host
7
6
authoritative DNS server
dns.cs.umass.edu
cis.poly.edu
gaia.cs.umass.edu
2: Application Layer
152
Typical Resolution
 Client does recursive request to local name server
 Local name server does iterative requests to find
name
 Local name server has knowledge of root servers
 Steps for resolving www.pdx.edu






Application calls gethostbyname()
Resolver contacts local name server (S1)
S1 queries root server (S2) for (www.pdx.edu)
S2 returns NS record for pdx.edu (S3)
S1 queries S3 for www.pdx.edu
S3 returns A record for www.pdx.edu
 Can return multiple addresses -> what does this
mean?
2: Application Layer
153
DNS: caching and updating records
 DNS responsescached throughout hierarchy

Quick response for repeated translations
• Other queries may reuse some parts of lookup
– NS records for domains reused often (xxx.yahoo.com)
• Entries timeout after some time (soft state)
– TTL field controlled by authority
– Affects DNS-based load balancing

TLD servers often cached in local name servers
• Thus root name servers not often visited

Negative responses also cached
• Don’t repeat past mistakes (misspellings)
 update/notify mechanisms under design by IETF

RFC 2136

http://www.ietf.org/html.charters/dnsind-charter.html
2: Application Layer
154
DNS Lookup Caching Example
www.cs.pdx.edu?
Client
Local
DNS server
root & edu
DNS server
pdx.edu
DNS server
cs.pdx.edu
DNS
server
2: Application Layer
155
Subsequent Lookup Example
cs.pdx.edu entry cached
ftp.cs.pdx.edu
Client
Local
DNS server
root & edu
DNS server
pdx.edu
DNS server
cs.pdx.edu
DNS
server
2: Application Layer
156
Iterated queries and caching
 Recall
Recursive query to local DNS server
 Iterative query from local DNS server on

 Why not do iterative queries from host?
 Win2k client
• Does iterative queries from host

Caching implications?
2: Application Layer
157
DNS dig example
 Real example using dig and named

Shows iterative queries, recursive queries, and caching
 dig
cat
dig
dig
dig
dig
/etc/resolv.conf
+norecurse www.thefengs.com.
@192.41.162.30 +norecurse www.thefengs.com.
@216.21.226.71 +norecurse www.thefengs.com.
+norecurse www.thefengs.com.
dig +recurse www.thefengs.com.
dig +norecurse www.thefengs.com.
#
#
#
#
#
#
#
#
local DNS server
do an iterative query to local DNS server
do an iterative query to IP addr of L TLD
do an iterative query to IP addr of NS at register.com
do an iterative query again to local DNS server
NOTHING was cached at local DNS server!
now do a recursive query through local DNS server
now you get a cached result
dig +norecurse www.jjkkllmmnnoopp.com.
dig +recurse www.jjkkllmmnnoopp.com.
dig +norecurse www.jjkkllmmnnoopp.com.
#
#
#
#
Negative results also cached
returns pointer to root name servers
returns status: NXDOMAIN
returns status: NXDOMAIN
2: Application Layer
158
DNS protocol, messages
DNS protocol : query and reply messages, both with
same message format
msg header
 identification: 16 bit #
for query, reply to query
uses same #
 flags:
 query or reply
 recursion desired
 recursion available
 reply is authoritative
2: Application Layer
159
DNS protocol, messages
Name, type fields
for a query
RRs in response
to query
records for
authoritative servers
additional “helpful”
info that may be used
2: Application Layer
160
DNS issues
 Poor static configuration (root server list)
 Lack of exponential backoff
 No centralized caching per site
 Each machine runs one caching local server
 UDP used for queries
 Need reliability -> Why not TCP?
 Vulnerability of 13 TLD servers
 Jon Postel and his mobility “experiment”
 Attacks on TLD have occurred
2: Application Layer
161
Chapter 2: Application layer
 2.1 Principles of
network applications


app architectures
app requirements
 2.2 Web and HTTP
 2.4 Electronic Mail
 SMTP, POP3, IMAP
 2.5 DNS
 2.6 P2P file sharing
 2.7 Socket programming
with TCP
 2.8 Socket programming
with UDP
 2.9 Building a Web
server
2: Application Layer
162
Building a simple Web server
 handles one HTTP




request
accepts the request
parses header
obtains requested file
from server’s file
system
creates HTTP response
message:

 after creating server,
you can request file
using a browser (e.g.,
IE explorer)
 see text for details
header lines + file
 sends response to client
2: Application Layer
163
Chapter 2: Summary
Our study of network apps now complete!
 Application architectures
 client-server
 P2P
 hybrid
 application service
requirements:

 specific protocols:
 HTTP
 FTP
 SMTP, POP, IMAP
 DNS
 socket programming
reliability, bandwidth,
delay
 Internet transport
service model


connection-oriented,
reliable: TCP
unreliable, datagrams: UDP
2: Application Layer
164
Chapter 2: Summary
Most importantly: learned about protocols
 typical request/reply
message exchange:


client requests info or
service
server responds with
data, status code
 message formats:
 headers: fields giving
info about data
 data: info being
communicated
 control vs. data msgs
in-band, out-of-band
centralized vs. decentralized
stateless vs. stateful
reliable vs. unreliable msg
transfer
“complexity at network
edge”





2: Application Layer
165
Extra slides
2: Application Layer
166
HTTP Authentication
server
client
Authentication goal: control
access to server documents
usual http request msg
 stateless: client must present
401: authorization req.
authorization in each request
WWW authenticate:
 authorization: typically name,
password
usual http request msg
 authorization: header
+Authorization:cred
line in request
 if no authorization
usual http response msg
presented, server refuses
access, sends
WWW authenticate:
usual http request msg
+ Authorization:cred
header line in response
time
 http://www.sandbox.com/clip
usual http response msg
board/pub-doc/home.jsp
Browser caches name & password so
2: Application Layer 167
that user does not have to repeatedly enter it.
HTTP Authentication example
 http://www.thefengs.com/wuchang/work/c
ourses/cs594/http_ba.txt
2: Application Layer
168
Chapter 2: Application layer
 2.1 Principles of
network applications


app architectures
app requirements
 2.2 Web and HTTP
 2.4 Electronic Mail
 SMTP, POP3, IMAP
 2.5 DNS
 2.6 P2P file sharing
 2.7 Socket programming
with TCP
 2.8 Socket programming
with UDP
 2.9 Building a Web
server
2: Application Layer
169
P2P file sharing
Example
 Alice runs P2P client
application on her
notebook computer
 Intermittently
connects to Internet;
gets new IP address
for each connection
 Asks for “Hey Jude”
 Application displays
other peers that have
copy of Hey Jude.
 Alice chooses one of
the peers, Bob.
 File is copied from
Bob’s PC to Alice’s
notebook: HTTP
 While Alice downloads,
other users uploading
from Alice.
 Alice’s peer is both a
Web client and a
transient Web server.
All peers are servers =
highly scalable!
2: Application Layer
170
P2P: centralized directory
original “Napster” design
1) when peer connects, it
informs central server:


Bob
centralized
directory server
1
peers
IP address
content
2) Alice queries for “Hey
Jude”
3) Alice requests file from
Bob
1
3
1
2
1
Alice
2: Application Layer
171
P2P: problems with centralized directory
 Single point of failure
 Performance
bottleneck
 Copyright
infringement
file transfer is
decentralized, but
locating content is
highly centralized
2: Application Layer
172
P2P: decentralized directory
 Gnutella
 fully distributed
• no central server
– Location service distributed over peers
– More difficult to shut down
• query flooding to search

Issues
• “Bootstrap” node needed
• Excessive query traffic
• Limited scoping
– Query radius may not have content
• Maintenance overhead

public domain protocol
• many Gnutella clients implementing protocol
2: Application Layer
173
Gnutella: protocol
 Query message
sent over existing TCP
connections
 peers forward
Query message
 QueryHit
sent over
reverse
Query
path
File transfer:
HTTP
Query
QueryHit
QueryHit
Scalability:
limited scope
flooding
2: Application Layer
174
Gnutella: Peer joining
Joining peer X must find some other peer in
Gnutella network: use list of candidate peers
2. X sequentially attempts to make TCP with peers
on list until connection setup with Y
3. X sends Ping message to Y; Y forwards Ping
message.
4. All peers receiving Ping message respond with
Pong message
5. X receives many Pong messages. It can then
setup additional TCP connections
Peer leaving: see homework problem!
1.
2: Application Layer
175
Exploiting heterogeneity: KaZaA
 Each peer is either a group
leader or assigned to a group
leader.


TCP connection between
peer and its group leader.
TCP connections between
some pairs of group leaders.
 Group leader tracks the
content in all its children.

Forwards query to other
group leaders if content not
present
 Overlay network

Edges are TCP connections
not links
ordinary peer
group-leader peer
neighoring relationships
in overlay network
2: Application Layer
176
KaZaA: Querying
 Each file has a hash and a descriptor
 Client sends keyword query to its group
leader
 Group leader responds with matches:

For each match: metadata, hash, IP address
 If group leader forwards query to other
group leaders, they respond with matches
 Client then selects files for downloading

HTTP requests using hash as identifier sent to
peers holding desired file
2: Application Layer
177
KaZaA tricks
 Limitations on simultaneous uploads
 Request queuing
 Incentive priorities
 Parallel downloading
For more info:
 J. Liang, R. Kumar, K. Ross, “Understanding KaZaA,”
(available via cis.poly.edu/~ross)
2: Application Layer
178
P2P: BitTorrent
 Previous systems
 Sources/servers
• Have entire copy of content
• Keep content available for others to download

Clients
• Connect to sources/servers with full copy

Problems
• Must force clients to become sources/servers for P2P
application to work
• Can’t redistribute data blocks of a file until it is received in
its entirety (large files problematic)
• No control of content (must rely on naming and popularity to
infer integrity)
– “URL-based P2P?”
2: Application Layer
179
Slides courtesy of Karthik Tamilman
Philosophy
 Author: Bram Cohen
 Based on Tit-for-tat
 Incentive - Uploading while downloading
 Get preference in downloading if you supply good upload
 Pieces of files
 Components
 Ordinary web server to serve up metainfo file (.torrent)
 Client web browser
 BitTorrent tracker (location of which is specified as a URL
in .torrent)
 Original downloader (the first “seed”)
 Client downloader
2: Application Layer
180
Overall Architecture
Tracker
Web Server
Origin
Downloader
[Seed]
C
A
Peer
Peer
[Leech]
B
Downloader
Peer
“US”
[Leech]
[Seed]
2: Application Layer
181
Overall Architecture
Tracker
Web Server
Origin
Downloader
[Seed]
C
A
Peer
Peer
B
[Seed]
[Leech]
Downloader
Peer
“US”
[Leech]
2: Application Layer
182
Overall Architecture
Tracker
Web Server
Origin
Downloader
[Seed]
C
A
Peer
Peer
B
[Seed]
[Leech]
Downloader
Peer
“US”
[Leech]
2: Application Layer
183
Overall Architecture
Tracker
Web Server
Origin
Downloader
[Seed]
C
A
Peer
Peer
[Leech]
B
Downloader
Peer
“US”
[Leech]
[Seed]
2: Application Layer
184
Overall Architecture
Tracker
Web Server
Origin
Downloader
[Seed]
C
A
Peer
Peer
B
[Seed]
[Leech]
Downloader
Peer
“US”
[Leech]
2: Application Layer
185
Overall Architecture
Tracker
Web Server
Origin
Downloader
[Seed]
C
A
Peer
Peer
B
[Seed]
[Leech]
Downloader
Peer
“US”
[Leech]
2: Application Layer
186
Overall Architecture
Tracker
Web Server
Origin
Downloader
[Seed]
C
A
Peer
Peer
[Leech]
B
Downloader
Peer
“US”
[Leech]
[Seed]
2: Application Layer
187
Messages
 Peer – Peer messages

TCP Sockets
 Peer – Tracker messages

HTTP Request/Response
 B-encoding
 http://bitconjurer.org/BitTorrent/protocol
.html
2: Application Layer
188
.torrent
 URL of the tracker
 Dictionary keys for B-encoding scheme
 Pieces <hash1,hash2,….hashn>
 Piece length
 Name
 Length
 Files
Path
 length

2: Application Layer
189
Tracker
 Peer cache

IP, port, peer id
 State information
Completed
 Downloading

 Returns random list
2: Application Layer
190
2: Application Layer
191
Peer Operation
 Choking algorithm

Choke/Unchoke
• Avoid large numbers of TCP connections
• Optimistic unchoke for a peer, rotates every 30 sec.
• Preferred peers do not get choked

Snubbing behavior
• Prevented by Anti-snubbing.
 Upload to interested peers who are not
choking.
2: Application Layer
193
Peer Operation
 Verify on receiving complete piece
 Endgame Behavior
Cancel
 To fix the “waiting” on the last slow peer

2: Application Layer
194
Experimental client
2: Application Layer
195
2: Application Layer
196
Strengths
 Better bandwidth utilization

Never before speeds.
• Up to 7 MB/s from the Internet.
 Limit free riding – tit-for-tat
 Limit leech attack – coupling upload &
download
 Ability to resume a download
2: Application Layer
197
Drawbacks
 Small files – latency, overhead
 Random list of peers - naive
 Scalability
 Millions of peers – Tracker behavior (uses
1/1000 of bandwidth)
 Single point of failure
 Robustness
 System progress dependent on altruistic nature
of seeds (and peers)
 Malicious attacks and leeches.
2: Application Layer
198
Interesting links
 Official site:
http://bitconjurer.org/BitTorrent
 BitTorrent FAQ: http://btfaq.com
 Torrent sites
http://f.scarywater.net
 http://www.suprnova.org
 http://tvtorrents.com
Remember
 leave your download windows open
 Big brother is watching!

2: Application Layer
199