Download Networking Part I - DePaul University

Networking Part I
May 20, 2015
May 20, 2015
JDP Networking Part I
1 of 67
Administrivia: Introductions
Dennis Mumaugh
Undergraduate: BSEE - University of California, Berkeley
MS Computer Science - University of Maryland
Ph.D. Studies - University of Maryland
Teaching at DePaul since September 2000
Work
Senior Engineer - National Security Agency
ARPANet Pioneer, Unix™ Technology Transfer
Member of the Technical Staff - Bell Labs/Lucent Technologies
Unix Development - Current Engineering
IS&R Systems - Knowledge Based Systems
Software Tools and OO Technology
Interests
Operating Systems and System Programming
Software Productivity, Compilers and Software Metrics
Software Engineering
May 20, 2015
JDP Networking Part I
2 of 67
Administrivia: contact details
• Contact Information:
 Email: [email protected]
 Phone: 630-983-1221 (10:00 am - 11:00 pm) except just
before classes
 Office: CDM 432
 Office Hours: Monday, 4:00-5:30
May 20, 2015
JDP Networking Part I
3 of 67
Administrivia: reading materials
• Course home page: http://condor.depaul.edu/dmumaugh/JDP
contains reading assignments, lectures, homework, pointers to API
documentation, more reading material, sample source code
• Textbooks
 Core Java, Volume I – Fundamentals, Eighth Edition, Cay S.
Horstmann and Gary Cornell, ISBN: 978-0132354769
• Other books you may want to read
 Core Java, Volume II – Advanced Features, Eighth Edition, Cay S.
Horstmann and Gary Cornell, ISBN:978-0132354790
 Java Network Programming, Harold, Elliotte Rusty, O'Reilly, 3rd
edition, 2004, ISBN 0-596-00217-3
 Fundamental Networking in Java, Pitt, Esmond, Springer, 2006,
ISBN 1-84628-030-3
May 20, 2015
JDP Networking Part I
4 of 67
Thought for the Day
(In 1970) The original Internet (called the ARPANet)
was intended to be used to provide distributed and
collaborative systems. It was actually used primarily
for sending email.
May 20, 2015
JDP Networking Part I
5 of 67
This Lecture
• Topic: Network programming
• Reading:
 Harold: Chapters 2-3, 9 and 10; pp. 525-542
 Pitt: Chapters 1-3, 13
 Core Java, Vol. 2: pp. 1-64, 169-217
 Core Java, Vol. 1: pp. 516-539
 Sun's [really Oracle] API javadoc documentation on the
Socket and ServerSocket classes.
 Technology Briefs (see class page for URL)
 TCP/IP
 HTTP
 Article on “Reading Data from the Internet”
May 20, 2015
JDP Networking Part I
6 of 67
Java Networking
May 20, 2015
JDP Networking Part I
7 of 67
Introduction to Networking
• What is a network? Why do we care?
 Need to understand what we are working with.
• Terms
 LAN
 WAN
 Internet
May 20, 2015
JDP Networking Part I
8 of 67
Networks
May 20, 2015
JDP Networking Part I
9 of 67
Networks
May 20, 2015
JDP Networking Part I
10 of 67
Introduction to Networking
•
•
•
•
What is a hub?
A switch?
A router?
A WAP?
May 20, 2015
JDP Networking Part I
11 of 67
Networks
Router
Firewall
Router
Firewall
May 20, 2015
JDP Networking Part I
12 of 67
The Internet and network programming
What is the Internet?
 Hosts
 Interfaces
 Routers
 LANS
 Gateways
 Protocols
 SOFTWARE
May 20, 2015
JDP Networking Part I
13 of 67
Network Programming
May 20, 2015
JDP Networking Part I
14 of 67
The Notion of an Internet Protocol
•
•
•
How is it possible to send bits across incompatible LANs
and WANs?
Solution: protocol software running on each host and router
smooths out the differences between the different
networks.
Implements an internet protocol (i.e., set of rules) that
governs how hosts and routers should cooperate when
they transfer data from network to network.
• TCP/IP is the protocol for the global IP Internet.
May 20, 2015
JDP Networking Part I
15 of 67
What Does an Internet Protocol Do?
1.
Provides a naming scheme
 An internet protocol defines a uniform format for host
addresses.
 Each host (and router) is assigned at least one of these
internet addresses that uniquely identifies it.
2. Provides a delivery mechanism
 An internet protocol defines a standard transfer unit
(packet)
 Packet consists of header and payload
 Header: contains info such as packet size, source
and destination addresses.
 Payload: contains data bits sent from source host.
May 20, 2015
JDP Networking Part I
16 of 67
Global IP Internet
• Most famous example of an internet.
• Based on the TCP/IP protocol family
 Level 3 - IP (Internet protocol) :
 Provides basic naming scheme and unreliable
delivery capability of packets (datagrams) from hostto-host.
 Level 4 - UDP (User Datagram Protocol)
 Uses IP to provide unreliable datagram delivery from
process-to-process.
 Level 4 - TCP (Transmission Control Protocol)
 Uses IP to provide reliable byte streams from processto-process over connections.
• Accessed via a mix of Java file I/O and functions from the
sockets interface.
May 20, 2015
JDP Networking Part I
17 of 67
Transferring Data Over an internet
(1)
Host A
Host B
client
server
data
internet packet
(2)
data
(3)
data
(8)
data
protocol
software
protocol
(7)
software
data
PH
LAN1
adapter
LAN2
(6)
adapter
data
PH FH2
PH
PH FH1
Router
LAN1 frame
LAN1
adapter
LAN1
(4)
data
LAN2
adapter
LAN2 frame
LAN2
PH FH1
data PH FH2 (5)
protocol
software
May 20, 2015
JDP Networking Part I
18 of 67
Introduction to Network Programming
•
•
•
•
•
What is IP?
What is a firewall?
A proxy?
Network Address Translation?
What is an RFC?
 Request For Comment
May 20, 2015
JDP Networking Part I
19 of 67
Introduction to Network Programming
• What is a lossy network?
• What is an unordered network?
TCP versus UDP
• What is TCP? What does it provide?
 TCP (Transmission Control Protocol) guarantees that all segments
will arrive at the destination and in the right order. (It makes no
guarantees about how long it will take.)
• What is UDP? Why use it?
 UDP (User Datagram Protocol) makes no such guarantees. So
UDP is mostly like IP, but with the extension that it gets data from
source process to destination process. UDP is used where
performance is needed and occasional loss of data is not critical.
E.g. audio and video streaming, short protocols such as DNS,
name services, etc.

May 20, 2015
UPD is a faster protocol. It has less overhead. But the trade off is
loss and unordered.
JDP Networking Part I
20 of 67
Introduction to Network Programming
Questions to answer:
• What is the client-server model?
• What is the difference between the client and the server?
• What are example client/server pairs?
• Are there other models?
May 20, 2015
JDP Networking Part I
21 of 67
Client-Server Model
• Every network application is based on the client-server
model:
 A server process and one or more client processes
 Server manages some resource.
 Server provides service by manipulating resource for clients.
1. Client sends request
Client
process
4. Client
handles
response
Server
process
3. Server sends response
Resource
2. Server
handles
request
Note: clients and servers are processes running on hosts
(can be the same or different hosts).
May 20, 2015
JDP Networking Part I
22 of 67
Clients
• Examples of client programs
 Web browsers, ftp, telnet, ssh
• How does a client find the server?
 The IP address in the server socket address identifies
the host (more precisely, an adapter on the host)
 The (well-known) port in the server socket address
identifies the service, and thus implicitly identifies the
server process that performs that service.
 Examples of well known ports
 Port 7: Echo server
 Port 23: Telnet server
 Port 25: Mail server
 Port 80: Web server
May 20, 2015
JDP Networking Part I
23 of 67
Servers
• Servers are long-running processes (daemons).
 Created at boot-time (typically) by the init process
(process 1)
 Run continuously until the machine is turned off.
• Each server waits for requests to arrive on a well-known
port associated with a particular service.
 Port 7: echo server
 Port 23: telnet server
 Port 25: mail server
 Port 80: HTTP server
• A machine that runs a server process is also often referred
to as a “server.”
May 20, 2015
JDP Networking Part I
24 of 67
Server Examples
•
Web server (port 80)
 Resource: files/compute cycles (CGI programs)
 Service: retrieves files and runs CGI programs on behalf of the client
•
FTP server (20, 21)
 Resource: files
 Service: stores and retrieve files
•
Telnet server (23)
 Resource: terminal
 Service: proxies a terminal on the server machine
•
Mail server (25)
 Resource: email “spool” file
 Service: stores mail messages in spool file
May 20, 2015
JDP Networking Part I
See /etc/services for
a comprehensive list of
the services available on
a Linux machine.
25 of 67
A Programmer’s View of the Internet
1.
Hosts are mapped to a set of 32-bit IP addresses.
 128.2.203.179
2. The set of IP addresses is mapped to a set of identifiers called Internet
domain names. [A host name].
 128.2.203.179 is mapped to www.cs.cmu.edu
 How do hostnames get matched to IP addresses?
 What is /etc/hosts?
 What is a DNS?
3. A process on one Internet host can communicate with a process on
another Internet host over a connection.
 What is special about addresses 127.0.0.*?
 Each host has a locally defined domain name localhost which
always maps to the loopback address 127.0.0.1
 192.168.*.*? (See notes page for answer).
May 20, 2015
JDP Networking Part I
26 of 67
Internet Connections
•
•
•
•
Clients and servers communicate by sending streams of bytes over
connections:
 Point-to-point, full-duplex (2-way communication), and reliable.
A socket is an endpoint of a connection
 Socket address is an IPaddress:port pair
A port is a 16-bit integer that identifies a process:
 Ephemeral port: Assigned automatically on client when client
makes a connection request
 Well-known port: Associated with some service provided by a
server (e.g., port 80 is associated with Web servers)
A connection is uniquely identified by the socket addresses of its
endpoints (socket pair)
 (cliaddr:cliport, servaddr:servport)
May 20, 2015
JDP Networking Part I
27 of 67
Using Ports to Identify Services
Server host 128.2.194.242
Client host
Service request for
128.2.194.242:80
(i.e., the Web server)
Client
Web server
(port 80)
Kernel
Echo server
(port 7)
Client
Service request for
128.2.194.242:7
(i.e., the echo server)
Web server
(port 80)
Kernel
Echo server
(port 7)
May 20, 2015
JDP Networking Part I
28 of 67
Introduction to Network Programming
• Are there other models?
 Peer-to-peer [Napster]
 Three tiered systems
 Service Oriented Architecture
 CORBA
 Distributed Services
May 20, 2015
JDP Networking Part I
29 of 67
Domain Naming System (DNS)
• The Internet maintains a mapping between IP addresses and
domain names in a huge worldwide distributed database called
DNS.
unnamed root
mil
mit
cs
edu
cmu
gov
berkeley
ece
com
First-level domain names
amazon
www
Second-level domain names
Third-level domain names
208.216.181.15
May 20, 2015
JDP Networking Part I
30 of 67
Querying DNS from the Command Line
•
Domain Information Groper (dig) provides a scriptable
command line interface to DNS.
linux> dig +short kittyhawk.cmcl.cs.cmu.edu
128.2.194.242
linux> dig +short -x 128.2.194.242
KITTYHAWK.CMCL.CS.CMU.EDU.
linux> dig +short aol.com
205.188.145.215
205.188.160.121
64.12.149.24
64.12.187.25
linux> dig +short -x 64.12.187.25
aol-v5.websys.aol.com.
May 20, 2015
JDP Networking Part I
31 of 67
The Eight Fallacies of Distributed Computing
Essentially everyone, when they first build a distributed application, makes
the following eight assumptions. All prove to be false in the long run
and all cause big trouble and painful learning experiences.
1. The network is reliable
2. Latency is zero
3. Bandwidth is infinite
4. The network is secure
5. Topology doesn't change
6. There is one administrator
7. Transport cost is zero
8. The network is homogeneous
May 20, 2015
JDP Networking Part I
32 of 67
Network Programming
May 20, 2015
JDP Networking Part I
33 of 67
Introduction to Network Programming
Protocols
• What is FTP, Telnet, and SMTP?
• What is NNTP?
• What is HTTP?
• Why is Telnet useful (other than for command line
interfaces)?
Markup Languages
• What is SGML?
• HTML?
• XML?
May 20, 2015
JDP Networking Part I
34 of 67
Topics
• Sockets interface
• Writing clients and servers
 How to use the sockets interface to establish Internet
connections between clients and servers
 How to copy data from one host to another over an
Internet connection.
May 20, 2015
JDP Networking Part I
35 of 67
Sockets Interface
Created in the early 80’s as part of the original Berkeley
distribution of Unix that contained an early version of the
Internet protocols.
• Provides a user-level interface to the network.
• Underlying basis for all Internet applications.
• Based on client/server programming model.
The key to network programming is understanding the
answers to the following questions:
• A socket?
• What is a port?
 How do we know who to talk to?
 What is a well-known port?
May 20, 2015
JDP Networking Part I
36 of 67
Sockets
• Sockets as an abstraction provide a conduit through which a
process can send data out onto a network to another
process. (Both processes could be on the same machine.)
• Sockets can be used with both the TCP and the UDP
transport layer protocols.
• Remember that TCP and UDP sockets need IP addresses
and port numbers.
• Conceptually this is all that is needed to specify a socket,
although the details vary somewhat depending on the
programming language and environment used.
May 20, 2015
JDP Networking Part I
37 of 67
Sockets
• What is a socket?
 To the kernel, a socket is an endpoint of communication.
 To an application, a socket is a file descriptor that lets the
application read/write from/to the network.
 Remember: All I/O devices, including networks, are
generally modeled as files.
• Clients and servers communicate with each other by
reading from, and writing to, socket descriptors.
• The main distinction between regular file I/O and socket I/O
is how the application “opens” the socket descriptors.
May 20, 2015
JDP Networking Part I
38 of 67
The Java Socket Class
Sockets can
1. Connect to a remote machine
2. Send data
3. Receive data
4. Close a connection
5. Bind to a port
6. Listen for incoming connection
7. Accept connections from remote machines on a bound
port
May 20, 2015
JDP Networking Part I
39 of 67
The Java Socket Class
The Socket class supports the
1. Connect to a remote machine [socket = new
Socket(…)]
2. Send data [socket.write()]
3. Receive data [socket.read()]
4. Close a connection [socket.close()]
What is the kind of data that can be transmitted?
1. What is a byte stream?
Normally a socket is encapsulated in a InputStream class or
a Reader class. [More next lecture].
May 20, 2015
JDP Networking Part I
40 of 67
The Java ServerSocket Class
The ServerSocket class additionally supports the
5. Bind to a port [server_socket.bind()]
6. Listen for incoming connection
[server_socket.listen()]
7. Accept connections from remote machines on a bound
port [server_socket.accept()]
May 20, 2015
JDP Networking Part I
41 of 67
Network Programming
• General theory
 Create and/or open a socket
 Convert a socket to a standard Java I/O class
 Input stream
 Output stream
 Use standard Java I/O for all operations
• Works for "normal" TCP connections
May 20, 2015
JDP Networking Part I
42 of 67
Basics
•
•
•
•
•
Create a new socket with the Socket() constructor
 Provide a host name and port.
Socket attempts to contact remote host
After connection exchange data
 Connection is full-duplex
After interactions are done close the connection
Normally a socket is encapsulated in an InputStream
class and possibly a Reader class, and an
OutputStream class and possibly a Writer class. [More
next lecture].
May 20, 2015
JDP Networking Part I
43 of 67
Addresses
• Connect to a remote machine
• Must have fully qualified domain name
 Can have short form
• How are names known
 Local files may have name: /etc/host
 System may have an NIS database (Solaris)
 Otherwise you must a priori know the name
• Must have a port number
 How are port numbers found?
May 20, 2015
JDP Networking Part I
44 of 67
Well Known Ports
# Network services, Internet style
# WELL KNOWN PORT NUMBERS
daytime 13/tcp
daytime 13/udp
qotd
17/tcp quote
#Quote of the Day
qotd
17/udp quote
#Quote of the Day
ftp
21/tcp
#File Transfer [Control]
ssh
22/tcp
#Secure Shell Login
telnet 23/tcp
smtp
25/tcp mail
#Simple Mail Transfer
http
80/tcp
#World Wide Web HTTP
nntp
119/tcp usenet #Network News Transfer Protocol
https
443/tcp
#Secure World Wide Web HTTP
syslog 514/udp
May 20, 2015
JDP Networking Part I
45 of 67
Telnet
Using telnet to experiment
• The telnet program can be used to debug a server
• Examples:
 telnet condor.depaul.edu 23
 telnet condor.depaul.edu 80
• You can try out being a client process of many protocols
that use TCP by using telnet.
telnet condor.depaul.edu 80
• But what does a protocol say about the interchange
between client and server?
• See next slide for an example of a transaction.
May 20, 2015
JDP Networking Part I
46 of 67
Using Telnet to Simulate a HTTP Session
$ telnet condor.depaul.edu 80
GET /~dmumaugh/index.html HTTP/1.1
Host: condor.depaul.edu
<blank line>
HTTP/1.1 200 OK
Date: Wed, 02 Apr 2003 20:35:34 GMT
Server: Apache/2.0.39 (Unix) PHP/4.2.1
Last-Modified: Wed, 02 Apr 2003 05:06:49 GMT
ETag: "29261-103d-e21d0c40"
Accept-Ranges: bytes
Content-Length: 4157
Content-Type: text/html; charset=ISO-8859-1
<blank line>
Blah…Blah…Blah…
May 20, 2015
JDP Networking Part I
47 of 67
Examples
May 20, 2015
JDP Networking Part I
48 of 67
Network I/O
• Remember: once a socket is open we have two major
operations:
 read()
 write()
• We may also convert a socket into a byte stream using
 getInputStream()
 getOutputStream()
• We can then “wrap” a byte stream into Reader and Writer
classes and use more powerful methods:
 readLine() – for input streams
 println() – for output streams
• We will discuss more on network I/O next time
May 20, 2015
JDP Networking Part I
49 of 67
Network I/O
Socket s = new Socket( host, port);
• Consider the method:
String BufferedReader.readLine ()
• And:
BufferedReader reader = new BufferedReader
(new InputStreamReader
(s.getInputStream(), "UTF-8"));
String request = reader.readLine ();
• Consider the method:
PrintWriter.println (String msg)
• And:
PrintWriter writer = new PrintWriter
(new OutputStreamWriter
(s.getOutputStream(), "UTF-8"));
writer.println ("GET /index.html HTTP/1.1");
writer.flush ();
May 20, 2015
JDP Networking Part I
50 of 67
Examples
• Examples
 Port Scanner on arbitrary host
 Time of Day Client
 Second Time of Day Client
 Port Scanner on current host
 Reverse Client
 Reverse Server
 Reverse Server Multi-threaded
 Serialize Client
 Serialize Server
May 20, 2015
JDP Networking Part I
51 of 67
Example: Port Scanner
Examine code for the simple port scanner
 http://condor.depaul.edu/dmumaugh/JDP/examples/lect0
1/PortProbe.java
• This will scan the ports on the specified host to see what are
available.
Note this is considered to be an unfriendly act by most
network administrators and can have unpleasant
consequences.
May 20, 2015
JDP Networking Part I
52 of 67
Example: Connecting to a Time Server
What is a time server?
• Let’s look at how to talk with one
 http://condor.depaul.edu/dmumaugh/JDP/examples/lect0
1/DayTime.java
• Another version (slightly different code)
 http://condor.depaul.edu/dmumaugh/JDP/examples/lect0
1/DaytimeClient.java
May 20, 2015
JDP Networking Part I
53 of 67
Servers Class
ServerSocket class
• Listens on a designated port for a connection
• Accepts a connection
 Uses getInputStream() and getOutputStream()
• Communicates with client
• Closes connection
May 20, 2015
JDP Networking Part I
54 of 67
Servers Class
Once a connection is made client and server must
communicate
• Protocol
• Traditional protocol by the server is
 Message number or type
 Message text
 Additional data
May 20, 2015
JDP Networking Part I
55 of 67
Example: A Port Scanner
Check to see what ports on the local host are in use
• Connects as a server locally
http://condor.depaul.edu/dmumaugh/JDP/examples/lect01/L
ocalPortScanner.java
May 20, 2015
JDP Networking Part I
56 of 67
Palindrome Detector
• Reverse Client
 http://condor.depaul.edu/dmumaugh/JDP/examples/lect0
1/ReverseClient.java
• Reverse Server
 http://condor.depaul.edu/dmumaugh/JDP/examples/lect0
1/ReverseServer.java
• Reverse Server Multi-threaded
 http://condor.depaul.edu/dmumaugh/JDP/examples/lect0
1/ReverseServerMT.java
May 20, 2015
JDP Networking Part I
57 of 67
Serialization
May 20, 2015
JDP Networking Part I
58 of 67
Object Serialization
To serialize an object:
OutputStream out = ...;
MyObject anObject = ...;
ObjectOutputStream oOut = new
ObjectOutputStream (out);
oOut.writeObject (anObject);
To unserialize an object:
InputStream in = ...;
ObjectInputStream oIn = new
ObjectInputStream(in);
MyObject anObject =
(MyObject)(oIn.readObject());
May 20, 2015
JDP Networking Part I
59 of 67
Object Serialization
Object output streams provide:
writeObject (Object);
writeInt (int);
writeBoolean (boolean);
...
Object input streams provide:
Object readObject ();
int readInt ();
boolean readBoolean ();
...
By some `magical process', data is sent down the line.
May 20, 2015
JDP Networking Part I
60 of 67
Object Serialization
• Not all objects can be serialized. Why not?
• Objects which can be serialized are called serializable.
• Serializable objects are instances of classes:
class MyClass implements Serializable
{ ... }
• Note that the Serializable interface is empty! This is a really
bad hack!
• Internally the Serializable interface provides version control.
• Many container classes are already marked as Serializable.
See note.
May 20, 2015
JDP Networking Part I
61 of 67
Object Serialization
What happens if we write:
class Unserializable {
stuff you can't send across the network
}
class OhDear implements Serializable {
Unserializable stupid = new Unserializable();
}
then try:
Serializable oh = new OhDear ();
oo.writeObject (oh);
Not one of the best bits of Java!
May 20, 2015
JDP Networking Part I
62 of 67
Object Serialization
Consider
class Example implements Serializable {
String sendMe = "hello";
transient String dontSendMe = "world";
}
At the sender:
Example test = new Example ();
test.sendMe = "fred";
test.dontSendMe = "wilma";
oOut.writeObject (test);
What gets printed by the receiver:
Example copy = (Example)(oIn.readObject ());
System.out.println (copy.sendMe);
System.out.println (copy.dontSendMe);
May 20, 2015
JDP Networking Part I
63 of 67
Object Serialization
What are transient fields good for?
• Suppress special information
 Passwords
• Hold process dependent information
 Idents (PID)
 File names
May 20, 2015
JDP Networking Part I
64 of 67
Object Serialization
Under the hood, the objects are converted into a stream of bytes.
For most classes, this is easy:
• To serialize ints, booleans, etc. just send the binary data (doubles are a
bit trickier but not much).
• To serialize an object, send the name of the class, then recursively
serialize all the fields.
• Other metadata is written as well: Inheritance tree, version ID, number of
fields, description of all fields, etc.; enough to recreate the object exactly.
For example, what is sent when:
class Foo { String a = "AString"; int b = 42; }
class Bar { Foo c = new Foo (); }
oOut.writeObject (new Foo ());
oOut.writeObject (new Bar ());
May 20, 2015
JDP Networking Part I
65 of 67
Serialize Example
• Serialize Server
 http://condor.depaul.edu/dmumaugh/JDP/examples/lect0
1/SerialS.java
• Serialize Client
 http://condor.depaul.edu/dmumaugh/JDP/examples/lect0
1/SerialC.java
May 20, 2015
JDP Networking Part I
66 of 67
Next Time
Topic: I/O streams, Applets, More Networking: UDP and Multicast
Reading:
 Core Java, Vol. 2: pp. 1-64, 169-217
 Sun(Oracles)'s API Javadoc
 Java Streams Basics
May 20, 2015
JDP Networking Part I
67 of 67