Download network programming - Mathematical, Statistical, and Scientific

Network Programming
1
Computer Networks
client/server architecture, protocols, and sockets
2
Network Programming
a simple client/server interaction
the module socket in Python
3
Monte Carlo for π
a pleasingly parallel computation
4
A Simple Multithreaded Server
combining sockets and threads
the SocketServer module
MCS 507 Lecture 28
Mathematical, Statistical and Scientific Software
Jan Verschelde, 31 October 2012
Scientific Software (MCS 507)
network programming
31 October 2012
1 / 40
Network Programming
1
Computer Networks
client/server architecture, protocols, and sockets
2
Network Programming
a simple client/server interaction
the module socket in Python
3
Monte Carlo for π
a pleasingly parallel computation
4
A Simple Multithreaded Server
combining sockets and threads
the SocketServer module
Scientific Software (MCS 507)
network programming
31 October 2012
2 / 40
client/server architecture
Client/server architecture defines the communication between two
computers: one is the server and the other acts as the client.
A client places a request or order to a server.
The server processes the request.
The client does not need to know how the server processes
the requests.
We distinguish between software and hardware
client/server architectures:
web and database servers offer software services;
file and print servers offer hardware services.
Scientific Software (MCS 507)
network programming
31 October 2012
3 / 40
network protocols
HTTP = Hypertext Transfer Protocol for web communications.
Network protocols are rules for network communication.
We consider two types of protocols:
TCP Transmission Control Protocol
First a message is sent that data is coming.
Only after the receiver acknowledges this message will
the sender send the data.
All successful transmissions are confirmed, and
retransmissions are acknowledged.
UDP User Datagram Protocol
Unlike TCP, no connection is established prior to sending
data. The sender just carries on after sending a message.
TCP is connection oriented, UDP is connectionless.
TCP is more reliable whereas UDP is more streamlined.
Scientific Software (MCS 507)
network programming
31 October 2012
4 / 40
sockets
Sockets are objects programs use to connect.
Sockets were introduced in 1981 in BSD Unix.
Originally used for communication between processes,
i.e.: between two programs on same computer.
Sockets support communication across platforms, independent of the
operating system.
In addition to the IP address of the computer, both server and client
must use the same port. A port is a 16-bit integer, some are reserved
for particular protocols.
Any port between 1,024 and 65,535 is free.
Sockets support both TCP and UDP.
Scientific Software (MCS 507)
network programming
31 October 2012
5 / 40
methods on socket objects
Most commonly used methods:
method
accept()
bind()
close()
connect(a)
listen(c)
recv(b)
send(d)
Scientific Software (MCS 507)
description
accepts connection and returns
new socket for passing data
binds a socket to an address
closes the socket
connects to address a
listen for connections
receives data in buffer of size b
sends data in d
network programming
31 October 2012
6 / 40
Network Programming
1
Computer Networks
client/server architecture, protocols, and sockets
2
Network Programming
a simple client/server interaction
the module socket in Python
3
Monte Carlo for π
a pleasingly parallel computation
4
A Simple Multithreaded Server
combining sockets and threads
the SocketServer module
Scientific Software (MCS 507)
network programming
31 October 2012
7 / 40
launching a server
Running tcp_server.py at the command prompt $:
In another terminal window we launch tcp_client.py:
Scientific Software (MCS 507)
network programming
31 October 2012
8 / 40
making connections
As soon as the client is launched,
we see in the terminal window for the server:
and then we type in the window of the client:
Scientific Software (MCS 507)
network programming
31 October 2012
9 / 40
passing a message
In the client terminal window we typed:
Then the server displays:
Scientific Software (MCS 507)
network programming
31 October 2012
10 / 40
Network Programming
1
Computer Networks
client/server architecture, protocols, and sockets
2
Network Programming
a simple client/server interaction
the module socket in Python
3
Monte Carlo for π
a pleasingly parallel computation
4
A Simple Multithreaded Server
combining sockets and threads
the SocketServer module
Scientific Software (MCS 507)
network programming
31 October 2012
11 / 40
sockets in Python
from socket import *
The socket module exports the method socket()
which returns an object representing a socket.
The first argument of socket is the Address Family (AF):
AF_UNIX : for UNIX sockets;
AF_INET : most commonly used for internet
AF_INET supports both TCP and UDP, given respectively by
SOCK_STREAM and SOCK_DGRAM as second argument of socket().
For example:
sock = socket(AF_INET, SOCK_STREAM)
Scientific Software (MCS 507)
network programming
31 October 2012
12 / 40
defining connections
After a socket is created, both server and client define
server_address = (hostname, number)
To bind the socket to the address, the server does
sock.bind(server_address)
and the client contacts the server then via
sock.connect(server_address)
Scientific Software (MCS 507)
network programming
31 October 2012
13 / 40
taking requests
With listen() the server indicates how many incoming connections
will be accepted:
sock.listen(2) # accept at most 2 connections
The server takes requests via the accept() method:
client, client_address = sock.accept()
The accept() method returns
1
a socket client for receiving data;
2
the address of the client.
Scientific Software (MCS 507)
network programming
31 October 2012
14 / 40
sending and receiving data
The client sends data with send()
sock.send(data)
The server receives data applying recv()
data = client.recv(buffer)
to the socket client obtained with accept().
When all is over, both client and server do
sock.close()
Scientific Software (MCS 507)
network programming
31 October 2012
15 / 40
the script tcp_server.py
from socket import *
hostname = ’’
# blank so any address can be used
number = 41267 # number for the port
buffer = 80
# size of the buffer
server_address = (hostname, number)
server = socket(AF_INET, SOCK_STREAM)
server.bind(server_address)
server.listen(1)
print ’server waits for connection’
client, client_address = server.accept()
print ’server accepted connection request from ’,\
client_address
print ’server waits for data’
data = client.recv(buffer)
print ’server received ’, data
server.close()
Scientific Software (MCS 507)
network programming
31 October 2012
16 / 40
the script tcp_client.py
from socket import *
hostname = ’localhost’
number = 41267
buffer = 80
# on same host
# same port number
# size of the buffer
server_address = (hostname, number)
client = socket(AF_INET, SOCK_STREAM)
client.connect(server_address)
print ’client is connected’
data = raw_input(’Give message : ’)
client.send(data)
client.close()
Scientific Software (MCS 507)
network programming
31 October 2012
17 / 40
Network Programming
1
Computer Networks
client/server architecture, protocols, and sockets
2
Network Programming
a simple client/server interaction
the module socket in Python
3
Monte Carlo for π
a pleasingly parallel computation
4
A Simple Multithreaded Server
combining sockets and threads
the SocketServer module
Scientific Software (MCS 507)
network programming
31 October 2012
18 / 40
estimating π
6
'$
The area of the unit disk is π:
-
&%
Generate random uniformly distributed points with coordinates
(x, y) ∈ [−1, +1] × [−1, +1].
We count a success when x 2 + y 2 ≤ 1.
1
generate n points P in [0, 1] × [0, 1]
2
m := #{ (x, y) ∈ P : x 2 + y 2 ≤ 1 }
3
the estimate is then 4 × m/n
→ dual core processor: use two clients.
pleasingly parallel: optimal speedup (twice as fast)
Scientific Software (MCS 507)
network programming
31 October 2012
19 / 40
running server and 2 clients
$ python mc4pi2.py
server waits for connections...
server waits for results...
approximation for pi = 3.141487
$ python mc4pi_client.py
client is connected
client received 1
client computes 0.785253200000
$ python mc4pi_client.py
client is connected
client received 2
client computes 0.785490300000
Scientific Software (MCS 507)
network programming
31 October 2012
20 / 40
one server distributes seeds
...
server.listen(2)
print ’server waits for connections...’
first, first_address = server.accept()
second, second_address = server.accept()
first.send(’1’); second.send(’2’)
print ’server waits for results...’
r1 = first.recv(buffer)
r2 = second.recv(buffer)
result = 2*(float(r1)+float(r2))
print ’approximation for pi =’, result
server.close()
Scientific Software (MCS 507)
network programming
31 October 2012
21 / 40
code for the client
import random
from socket import *
hostname = ’localhost’ # on same host
number = 11267
# same port number
buffer = 80
# size of the buffer
server_address = (hostname, number)
client = socket(AF_INET, SOCK_STREAM)
client.connect(server_address)
print ’client is connected’
s = client.recv(buffer)
print ’client received %s’ % s
random.seed(int(s))
Scientific Software (MCS 507)
network programming
31 October 2012
22 / 40
code for the client continued
N = 10**7
k = 0
for i in range(0,N):
x = random.uniform(0,1)
y = random.uniform(0,1)
if x**2 + y**2 <= 1: k = k + 1
R = float(k)/N
print ’client computes %.12f’ % R
client.send(str(R))
client.close()
Scientific Software (MCS 507)
network programming
31 October 2012
23 / 40
Network Programming
1
Computer Networks
client/server architecture, protocols, and sockets
2
Network Programming
a simple client/server interaction
the module socket in Python
3
Monte Carlo for π
a pleasingly parallel computation
4
A Simple Multithreaded Server
combining sockets and threads
the SocketServer module
Scientific Software (MCS 507)
network programming
31 October 2012
24 / 40
using sockets and threads
The main program will
1
define address, port numbers, server socket,
2
launch the handler threads of the server.
Typically we will have as many handler threads as the number of
connections the server listens to.
Inheriting from the class Thread,
1
the constructor of the handler will store a reference to the socket
server as data attribute,
2
the run contains the code to accept connections
and handle requests.
Scientific Software (MCS 507)
network programming
31 October 2012
25 / 40
a simple server
$ python mtserver.py
give #clients : 2
server is ready for 2 clients
server starts 2 threads
0 accepted request from (’127.0.0.1’, 49152)
0 waits for data
0 received A
1 accepted request from (’127.0.0.1’, 49153)
1 waits for data
1 received B
$
The clients can connect at any time,
one does not have the wait for the other to connect.
Scientific Software (MCS 507)
network programming
31 October 2012
26 / 40
code for the client
from socket import *
hostname = ’localhost’
number = 11267
buffer = 80
# on same host
# same port number
# size of the buffer
server_address = (hostname, number)
client = socket(AF_INET, SOCK_STREAM)
client.connect(server_address)
print ’client is connected’
data = raw_input(’Give message : ’)
client.send(data)
client.close()
Scientific Software (MCS 507)
network programming
31 October 2012
27 / 40
main() in mtserver.py
def main():
"""
Prompts for number of connections,
starts the server and handler threads.
"""
n = input(’give #clients : ’)
b = 80; server = connect(n,b)
print ’server is ready for %d clients’ % n
T = []
for i in range(0,n):
T.append(Handler(str(i),server,b))
print ’server starts %d threads’ % n
for t in T: t.start()
server.close()
Scientific Software (MCS 507)
network programming
31 October 2012
28 / 40
the connect function
def connect(n,b):
"""
Connects a server to listen to n clients,
buffer size is b. Returns the socket.
"""
hostname = ’’
# to use any address
number = 11267 # number for the port
buffer = b
# size of the buffer
server_address = (hostname, number)
server = socket(AF_INET, SOCK_STREAM)
server.bind(server_address)
server.listen(n)
return server
Scientific Software (MCS 507)
network programming
31 October 2012
29 / 40
the handler class
from socket import *
from threading import *
class Handler(Thread):
"""
Defines handler threads.
"""
def __init__(self,n,s,b):
"""
Name of handler is n, server
socket is s, buffer size is b.
"""
Thread.__init__(self,name=n)
self.sv = s
self.bf = b
Scientific Software (MCS 507)
network programming
31 October 2012
30 / 40
defining the handling
def run(self):
"""
Handler accepts connection,
prints message received from client.
"""
n = self.getName()
server = self.sv
buffer = self.bf
client, client_address = server.accept()
print n + ’ accepted request from ’,\
client_address
print n + ’ waits for data’
data = client.recv(buffer)
print n + ’ received ’, data
Scientific Software (MCS 507)
network programming
31 October 2012
31 / 40
Network Programming
1
Computer Networks
client/server architecture, protocols, and sockets
2
Network Programming
a simple client/server interaction
the module socket in Python
3
Monte Carlo for π
a pleasingly parallel computation
4
A Simple Multithreaded Server
combining sockets and threads
the SocketServer module
Scientific Software (MCS 507)
network programming
31 October 2012
32 / 40
the SocketServer module
With the SocketServer module we do not need to import the
socket module for the server script.
Follow these steps:
1
from SocketServer import StreamRequestHandler,
TCPServer
2
Inheriting from StreamRequestHandler
define a request handler class. Override handle().
→ handle() processes incoming requests
3
Instantiate TCPServer with (address,port)
and an instance of the request handler class.
→ this returns a server object
4
Apply the method handle_request() or serve_forever()
to the server object.
Scientific Software (MCS 507)
network programming
31 October 2012
33 / 40
a server to tell the time
In the window running the server:
$ python clockserver.py
server is listening to 12091
press ctrl c to stop server
connected at (’127.0.0.1’, 49154)
read "What is the time?
" from client
writing "Thu Nov 10 21:55:30 2011" to client
In the window running the client:
$ python clockclient.py
client is connected
Thu Nov 10 21:55:30 2011
Scientific Software (MCS 507)
network programming
31 October 2012
34 / 40
code for the client
from socket import *
hostname = ’localhost’
number = 12091
buffer = 25
# on same host
# same port number
# size of the buffer
server_address = (hostname, number)
client = socket(AF_INET, SOCK_STREAM)
client.connect(server_address)
print ’client is connected’
data = ’What is the time?’
client.send(data + (buffer-len(data))*’ ’)
data = client.recv(buffer)
print data
client.close()
Scientific Software (MCS 507)
network programming
31 October 2012
35 / 40
code for the server
from SocketServer import StreamRequestHandler
from SocketServer import TCPServer
from time import ctime
port = 12091
class ServerClock(StreamRequestHandler):
"""
The server tells the clients the time.
"""
def handle(self):
"""
Handler sends time to client.
"""
...
ss = TCPServer((’’,port),ServerClock)
print ’server is listening to’, port
ss.handle_request()
Scientific Software (MCS 507)
network programming
31 October 2012
36 / 40
code for the handler
def handle(self):
"""
Handler sends time to client.
"""
print "connected at", self.client_address
data = self.rfile.read(25)
print ’read \"’ + data + ’\" from client’
now = ctime()
print ’writing \"’ + now + ’\" to client’
self.wfile.write(now)
Scientific Software (MCS 507)
network programming
31 October 2012
37 / 40
about rfile and wfile
attributes in the class StreamRequestHandler
rfile contains input stream to read data from client
example: data = self.rfile.read(25)
client must send exactly 25 characters!
wfile contains output stream to write data to client
example: self.wfile.write(data)
all data are strings of characters!
Scientific Software (MCS 507)
network programming
31 October 2012
38 / 40
using serve_forever()
With serve_forever(), we can
1
serve indefinitely many requests,
2
simultaneously from multiple clients.
ss = TCPServer((’’,port),ServerClock)
print ’server is listening to’, port
try:
print ’press ctrl c to stop server’
ss.serve_forever()
except KeyboardInterrupt:
print ’ ctrl c pressed, closing server’
ss.socket.close()
Scientific Software (MCS 507)
network programming
31 October 2012
39 / 40
Summary + Exercises
For more: Foundations of Python Network Programming: The
comprehensive guide to building network applications with Python by
Brandon Rhodes and John Goerzen, 2nd Edition, Springer, 2010.
1
Make a server to compute determinants. A client first sends the
dimension n and then the server reads n2 floats as the n rows of
an n-by-n matrix A. Before closing down, the server sends det(A)
to the client.
2
Make a multithreaded server to evaluate Python expressions.
Clients send strings which can be evaluated by Python’s eval().
The server sends the result of the evaluation to the client.
Homework due Friday 16 November at 10AM:
exercises 1 and 3 of lecture 21; exercise 1 of lecture 22;
exercise 2 of lecture 23; exercises 2 and 4 of lecture 24;
exercises 1 and 2 of lecture 25; exercises 1 and 3 of lecture 26.
Scientific Software (MCS 507)
network programming
31 October 2012
40 / 40