Download ppt - MPJ Express

mpjdev, the low-level MPJ
device
Aamir Shafi
[email protected]
Distributed Systems Group
12th March, 2004
Trinity: Neo... No one has ever done anything like this.
Neo: That's why it's going to work.
Sequence

Introduction



History of MPJ
Overview of java.nio package
‘mpjdev’ implementation



Introduction
Packing/Unpacking at the device level
Communication Protocols




Eager Send
Rendezvous Protocol
Quick Start Guide to mpjdev
What’s next
Introduction

Parallel Programming



Message Passing



Shared Memory
Message Passing
MPI
PVM
Has Java got what it takes to be a
parallel programming language ?
Message Passing in Java (MPJ)


Specification Document
mpiJava 1.2.5

Uses native MPI
Pure java MPI-like libraries

mpiJava 1.2.5





JMPI






Drawback, based on RMI
http://euler.ecs.umass.edu/jmpi/
MPJava


Uses native MPI
Best candidate to be developed to MPJ
http://www.hpjava.org/mpiJava.html
DSG’s project MPJ (http://dsg.port.ac.uk/projects/mpj/) is tightly
coupled to this effort
Based on java.nio
Results encouraging
No source-code
DOGMA – Dead
MPP –


Basic library
http://www.mi.uib.no/~bjornoh/mtj/mpp/
Oracle: You're cuter than I thought. I can see why she likes you.
Sequence

Introduction



History of MPJ
Overview of java.nio package
‘mpjdev’ implementation



Introduction
Packing/Unpacking at the device level
Communication Protocols




Eager Send
Rendezvous Protocol
Quick Start Guide to mpjdev
What’s next
New I/O overview

Buffer classes,



Scalable Applications




Conversion among basic data types
Direct and Indirect Buffer
No more “one thread per client”
One thread, many clients
Multiplex Synchronous I/O using the selectors
SocketChannel – New Abstraction

Pipe – One way communication
Buffer classes in java.nio



‘T’Buffer classes, ‘T’ being all the basic
data types
Any basic data type can be copied onto
the ByteBuffer, (the basic primitive)
Buffer can be,


Direct, allocateDirect(int)
Indirect, allocate(int)
a. ByteBuffer buffer = ByteBuffer.allocate(8);
b. buffer.putInt(4);
c. buffer.flip();
a. ByteBuffer buffer = ByteBuffer.allocate(8);
b. ByteBuffer buffer = ByteBuffer.allocateDirect(8);
Selector

Selector provide, connecting, accepting,
reading & writing facilities to the
SocketChannel


OP_WRITE, OP_READ, OP_ACCEPT,
OP_CONNECT
Depends on native OS facilities

Selection performs differently on Windows
and Linux
Sequence of events for
selectors
Taming the NIO circus

Taming the NIO circus thread


OutOfMemory Exception


http://forum.java.sun.com/thread.jsp?thread=433702&forum=
4&message=2136979
Selectors taking cent percent CPU



http://forum.java.sun.com/thread.jsp?forum=4&thread=45933
8&start=0&range=15&hilite=false&q=
http://forum.java.sun.com/thread.jsp?forum=11&thread=4949
67
http://forum.java.sun.com/thread.jsp?forum=4&thread=49419
4
J2SE 1.5.0 beta solves many problems
Neo: What are you trying to tell me? That I can dodge bullets?
Morpheus: No, Neo. I'm trying to tell you that when you're ready, you
won't have to.
Sequence

Introduction



History of MPJ
Overview of java.nio package
‘mpjdev’ implementation



Introduction
Packing/Unpacking at the device level
Communication Protocols




Eager Send
Rendezvous Protocol
Quick Start Guide to mpjdev
What’s next
mpjdev – Introduction

What is device ?



Sockets
Similar to ADI in MPICH
Meant for library developers (MPJ), not
application programmers

jGMA can use mpjdev
mpjdev – Introduction

Single JVM Implementation


Native implementation


Uses native MPI
LAPI Implementation



Processes are threads in the single JVM
Eager-send
Rendezous
Buffer packing/unpacking are taken from
these implementations
mpjev functions

Comm methods



Void send(Buffer, int dest, int tag)
Req isend(Buffer, int dest, int tag)
Req irecv(Buffer, int src, int tag)




For src, ANY_SRC
For tag, ANY_TAG
Status recv(Buffer, int dest, int tag)
Req methods


Status Wait()
Status Wait(Req[])
‘mpjdev’ – Connectivity
Communication Protocols – Eager
Send
Sender
Huge Memory at
Receiver
Receiver
1
Sel
2
3
Sel
4
5
6
Rendezvous Protocol
1
Sender
Sender 1:
Sender posts send(),
Req stored in Queue
Sender 2:
Sender Queue
1
Sender sends the
control message asking
if matching recv is
posted ?
Sender 3:
Sender receives
response confirming
there is a matching recv
Sender 4:
Sel
Sender sends the actual
data
Receiver 1:
Checks if step 2 has
already posted recv, if
yes, initiates step 3,
else posts a req in Que
1
Receiver
Receiver 2:
Checks if step 1 has
posted a recv request, if
yes, initiates step 3,
else posts a request.
Recv Queue
1
NOTE :- STEP 1 & 2
needs synchronization
Receiver 3:
Writes back to sender
that recv’r is ready to
receive
Receiver 4:
Receives the data
Sel
Packing/Unpacking of Buffers







write(t[] source, int offset,int length)
gather(t[] source, int numEls, int offs, int []
indexes)
strGather(t[] source, int srcOff, int rank, int
exts, int strs, int [] indexes)
read(t[] dest, int dstOff, int numEls)
scatter(t[] dest, int numEls, int offs, int []
indexes)
strScatter(t[] dest, int dstOff, int rank, int
exts, int strs, int [] indexes)
Java objects, same concepts
Message Format

Primary header (Eight bytes),



First four bytes –nothing
Last four byte --size of message
Primary PayLoad,

Each primitive data type is written as a section,

SECTION_HEADER (8 bytes)



SECTION_DATA




First Four bytes –nothing
Last Four bytes – Size of Java Object
Secondary PayLoad


The actual data itself
Should be multiple of 8
Secondary header (Eight Bytes)


First byte – Type of Message, int, float etc
Last four bytes – Number of Elements in this section
Actual java Object in bytes
Can Java object be written as a section in Primary Payload ?
Message Format, Single Section
int intArray[] = new int[2];
int[0] = 1; int[1] = 2;
WriteBuffer wBuffer = new WriteBuffer(24);
wBuffer.write(intArray, 0,2); wBuffer.pack();
8 bytes
XXXX
Size (24)
PRIMARY_HEADER
8 bytes
8 bytes
int T X X X 2(NoEls) int[0]
SECTION_HEADER
int[1]
SECTION_HEADER
Message Format, Multiple Section
int intArray[] = new int[2]; long longArray = new long[2];
int[0] = 1; int[1] = 2; longArray[0] = 1L; longArray[1] = 2L;
WriteBuffer wBuffer = new WriteBuffer(48);
wBuffer.write(intArray, 0,2); wBuffer.write(longArray, 0,2);
wBuffer.pack();
8 bytes
8 bytes
8 bytes
X X X X Size (24) int X X X 2(NoEls) int[0] int[1]
PRIMARY_HEADER SECTION_HEADER SECTION_DATA
8 bytes
LXXX
2
S_HEADER
16 bytes
long[0] long[1]
S_DATA
Control Messages


Used in Rendezvous Protocol
Format





Int
Int
Int
Int
–
–
–
–
Rank of Destination
Rank of Source
Message Length
Message Tag
Should be as small as possible
Morpheus: There is a difference between knowing the path and walking the
path.
Sequence

Introduction



History of MPJ
Overview of java.nio package
‘mpjdev’ implementation



Introduction
Packing/Unpacking at the device level
Communication Protocols




Eager Send
Rendezvous Protocol
Quick Start Guide to mpjdev
What’s next
Directory Structure
Compiling the source files

Dependencies,


J2SE1.5.0-beta
Apache Ant (1.6.1)

If you want to compile from the source,
otherwise


mpjdev.jar is lying in the mpjdev/lib folder
You need to add “mpjdev.jar” in your
CLASSPATH
Running Examples - Config

Configuration



mpjdev/conf/mpjdev.conf
Total Number of Processes
For each Process


FULLY_QUALIFIED_NAME@PORT@RANK
Sorry – IP address won’t work at present

I never had to to write an IP 
Running examples



# cd mpjdev/test
# javac -classpath ../lib/mpjdev.jar:.
BufferTest3.java
# java -classpath ../lib/mpjdev.jar:.
BufferTest3 0


(O is the Rank)
Tracking problems



java –version, is it 1.5. ?
Are you using IP ?
Email me
Writing your own programs

As an example,


Two processes
Initialize the device


Get your ID,


CommImpl.send(Buffer, int dest, int tag)
Other process reads data & unpacks


CommImpl.id()
One process packs & sends data,


CommImpl.init()
CommImpl.recv(Buffer, int src, int tag)
Finalize,

CommImpl.finish();
Neo: Why do my eyes hurt?
Morpheus: You've never used them before.
Sequence

Introduction



History of MPJ
Overview of java.nio package
‘mpjdev’ implementation



Introduction
Packing/Unpacking at the device level
Communication Protocols




Eager Send
Rendezvous Protocol
Quick Start Guide to mpjdev
What’s next
A Few experiments


No tests, yet 
Bottlenecks,



Would like to see how many processes, can
selector handle ?


Transfer of huge messages seems inefficient
Queues, java.util.Vector is not good enough
Potentially thousands
Need bench-marks, keeping in view java
issues and bottlenecks, and test them

Write our own
Future

Badly need a runtime


MPJ,



Shell scripts starting jobs using ‘ssh’ or ‘rsh’ ?
Group communications
I thought, I would have completed the
whole MPJ, by now
Looking for my PhD project, in the
same domain
Morpheus: Unfortunately, no one can be told what the Matrix is. You have to
see it for yourself.
Suggestions
?