Download Introduction to Programming - Rensselaer Polytechnic Institute

Distributed Computing with
Python
Carlos Varela
Rensselaer Polytechnic Institute
C. Varela, RPI
1
Distributed Computing with
Python
• Using standard TCP/IP (or UDP) sockets
• Using high-level libraries for remote method invocation
– XML-RPC
– Pyro (PYthon Remote Objects)
C. Varela, RPI
2
Networking with Python Sockets
•
•
•
•
•
Resembles C socket API
Can use UDP (datagrams) or TCP (streamed connections)
Does not require “non-standard” libraries
Low-level, yet relatively easy to use and portable
Enables more efficient communication and is
programming language independent.
• New features require protocol extensions
C. Varela, RPI
3
Sockets: An Echo Server
from socket import *
def main():
s = socket(AF_INET, SOCK_STREAM)
s.bind((‘’, 50007)) # port number
s.listen(1)
conn, (remotehost, remoteport) = s.accept()
print 'connected by', remotehost, remoteport
while 1:
data = conn.recv(1024) # buffer size
if not data: break
conn.send(data)
conn.close()
main()
C. Varela, RPI
4
Sockets: An Echo Client
from socket import *
def main():
s = socket(AF_INET, SOCK_STREAM)
s.connect((‘localhost’, 50007)) # port number
s.send(‘Hello World’)
data = s.recv(1024) # buffer size
s.close()
print data
main()
C. Varela, RPI
5
Remote Procedure Calls with
Python XML-RPC library
• XML-RPC is programming-language/O.S. independent
• Uses XML as message format for parameters and return
values, e.g.:
<params>
<param>
<value><string>Hello World</string></value>
</param>
</params>
• Uses HTTP as the transport protocol.
• Enabling technology for building “web services”.
• Example requires downloading xmlrpcserver.py library:
http://svn.python.org/view/python/trunk/Demo/xmlrpc/
C. Varela, RPI
6
XML-RPC: An Echo Server
import SocketServer; import xmlrpcserver; import xmlrpclib
class EchoReqHandler(xmlrpcserver.RequestHandler):
def call(self, method, params):
print "Dispatching: ", method, params
try:
server_method = getattr(self, method)
except:
raise AttributeError, ”No method: %s" % method
return server_method(params)
def echo(self, value):
return xmlrpclib.dumps(value)
server = SocketServer.TCPServer(('', 8000), EchoReqHandler)
7
server.serve_forever() C. Varela, RPI
XML-RPC: An Echo Client
import xmlrpclib
echosvr = xmlrpclib.Server("http://localhost:8000")
print echosvr.echo("Hello World")
print echosvr.echo(10)
C. Varela, RPI
8
Distributed Computing with
Python Remote Objects (PYRO)
• Resembles Java RMI: Server Daemon, Client Proxy
• To find remote objects:
– By name, e.g., “PYRONAME://echo”, requires name server
– By location, e.g., “PYROLOC://localhost:7766/echo”
– By URI, e.g.,
“PYRO://127.0.0.1:7766/7f0000010512127841288b1892e90a71”
• Method arguments, return values, and exceptions
marshalled (“pickled”) over the network
• Example code requires downloading and installing Pyro at:
http://pyro.sourceforge.net/
C. Varela, RPI
9
Pyro: An Echo Server
import Pyro.core
class Echo(Pyro.core.ObjBase):
def __init__(self):
Pyro.core.ObjBase.__init__(self)
def echo(self, string):
return string;
Pyro.core.initServer()
daemon=Pyro.core.Daemon()
uri=daemon.connect(Echo(),"echo-object")
print "The object's uri is:",uri
daemon.requestLoop()
C. Varela, RPI
10
Pyro: An Echo Client
import Pyro.core
echoProxy = Pyro.core.getProxyForURI(
"PYROLOC://localhost:7766/echo-object")
print echoProxy.echo("Hello World")
C. Varela, RPI
11
Python Remote Objects (PYRO)
• Python programming language-dependent
• Event server for publish-subscribe communication
• Mobile code, but BEWARE: No security
guarantees!
C. Varela, RPI
12
Exercises
1. Compare the round trip time of the Echo application using:
•
•
•
TCP/IP
XML-RPC
Pyro
2. Use the XML-RPC client in Python to connect to an
existing web service (e.g., Google)
•
http://www.xmlrpc.com/googleGateway
3. Modify the echo example in Pyro to use the name service.
What are alternative ways to get to the name server?
4. Does Pyro support distributed garbage collection?
C. Varela, RPI
13