Download Session 4 - Personal Web Pages

Session 4: Overview of other assignments
9:15 pm - 9:40 pm
Workshop 20: Teaching a Hands-on Undergraduate
Grid Computing Course
SIGCSE 2010 - The 41st ACM Technical Symposium on
Computer Science Education
Friday March 12, 2010
Dr. Barry Wilkinson
Dr. Clayton Ferner
University of North Carolina
Charlotte
University of North Carolina
Wilmington
© 2010 B. Wilkinson/Clayton Ferner SIGCSE 2010 Workshop 20 session4.ppt Modification date: Feb 27, 2010
1
Session 4: Overview of other assignments
• Condor scheduler assignment
• Assignment installing GT4 core, creating,
deploying, and testing a GT4 Web service
• GridSphere portlet design assignment
2
Condor Scheduler Assignment
Tasks:
•
•
•
•
•
•
•
Issue various Condor commands
Submit jobs to Condor
Manage jobs in job queue
C and Java programs
Vanilla and Java universes
Job and resource ClassAds for matching jobs to
resources
Using DAGMAN to create workflows
Assignment done on severs coit-grid03.uncc.edu and
coit-grid05.uncc.edu
3
Systems used for assignment
coit-grid01.uncc.edu – coit-grid05.uncc.edu
External interfaces
Users
(:8080)
User access
Dual Xeon
processors
Portal
coitgrid04
Used in Condor
assignment as
written
User’s home directory
(/nfs-home) shared
across system
coitgrid03
coitgrid02
CA
coit-grid01
switch
coitgrid05
Four quad-core Xeon
processors, 64GB memory,
1.2 TB RAID disk
4
Check Condor pool status: condor_status.
Name
OpSys
Arch
State
Activity LoadAv Mem
slot1@coit-grid03.
slot2@coit-grid03.
slot3@coit-grid03.
slot4@coit-grid03.
slot10@coit-grid05
slot11@coit-grid05
slot12@coit-grid05
slot13@coit-grid05
slot14@coit-grid05
slot15@coit-grid05
slot16@coit-grid05
slot1@coit-grid05.
slot2@coit-grid05.
slot3@coit-grid05.
slot4@coit-grid05.
slot5@coit-grid05.
slot6@coit-grid05.
slot7@coit-grid05.
slot8@coit-grid05.
slot9@coit-grid05.
LINUX
LINUX
LINUX
LINUX
LINUX
LINUX
LINUX
LINUX
LINUX
LINUX
LINUX
LINUX
LINUX
LINUX
LINUX
LINUX
LINUX
LINUX
LINUX
LINUX
X86_64
X86_64
X86_64
X86_64
X86_64
X86_64
X86_64
X86_64
X86_64
X86_64
X86_64
X86_64
X86_64
X86_64
X86_64
X86_64
X86_64
X86_64
X86_64
X86_64
Unclaimed
Unclaimed
Unclaimed
Unclaimed
Unclaimed
Unclaimed
Unclaimed
Unclaimed
Unclaimed
Unclaimed
Unclaimed
Unclaimed
Unclaimed
Unclaimed
Unclaimed
Unclaimed
Unclaimed
Unclaimed
Unclaimed
Unclaimed
Idle
Idle
Idle
Idle
Idle
Idle
Idle
Idle
Idle
Idle
Idle
Idle
Idle
Idle
Idle
Idle
Idle
Idle
Idle
Idle
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
ActvtyTime
250 0+00:25:04
250 30+10:53:00
250 5+16:29:17
250 30+12:32:14
4028 7+22:59:18
4028 7+22:59:19
4028 7+22:59:20
4028 7+22:59:21
4028 7+22:59:22
4028 7+22:59:23
4028 7+22:59:16
4028 0+03:10:04
4028 4+23:12:43
4028 7+22:59:19
4028 7+22:59:20
4028 7+22:59:21
4028 7+22:59:22
4028 7+22:59:23
4028 7+22:59:16
4028 7+22:59:17
Total Owner Claimed Unclaimed Matched Preempting Backfill
X86_64/LINUX
20
0
0
20
0
0
0
Total
20
0
0
20
0
0
0
5
Submitting a job
Create a job description file
Submit job with
condor_submit hostname_test1
Output:
Contents of hostname_test1:
# comment
Universe = vanilla
Executable = /bin/hostname
Output = hostname.out
Error = hostname.error
Log = hostname.log
Should_transfer_files = YES
When_to_transfer_output
= ON_EXIT
Queue
condor_submit condor_test1
Submitting job(s).
Logging submit event(s).
1 job(s) submitted to cluster 73.
6
Monitoring
Can query status of Condor queue with:
condor_q
Get output of form:
Queue
-- Submitter: coit-grid02.uncc.edu : <152.15.98.25:32821> :
ID
OWNER
SUBMITTED
RUN_TIME ST PRI SIZE CMD
662.0
abw
5/23 17:36
0+00:00:00 I
0
9.8
uptime
16 jobs; 1 idle, 0 running, 15 held
Status: H (hold), R (running), I (idle, waiting for machine), C
(Completed), U (unexpanded, never being run) or X (removed).
7
Other tasks

Managing job



Set up email notifications
Different Condor universes

Write and submit C program to compute pi by Monte Carlo
method

Rewrite in Java and submit to Java universe
ClassAd Job/resource matching

Examine resource ClassAds of computers to find out Java
version, memory and performance

Add job ClassAd to job description to make job run only on
computer with specific version number, memory and
performance
8
Condor’s ClassAd Matchmaking
Mechanism
Include in submit
description file
Held in assessible file
on system
9
Fig 3.14
Condor’s workflow scheduler
DAGMAN (Directed Acyclic Graph Manager )
Example
# diamond.dag
Job A a.sub
Job B b.sub
Job C c.sub
Job D d.sub
Parent A Child B C
Parent B C Child D
Job A
Job B
Job C
Job D
Start with:
condor_submit_dag diamond.dag
10
Some assignment tasks with
DAGMAN

Write program to compute π -- specifying different
random sequences for each of three instances and
then construct a DAG as shown below
Compute π
Compute
average
Job A
Job B
Job C
Job D
11
Assignment -- Installing GT4 core, creating,
deploying, and testing a GT4 Web service
This assignment is based upon a
GT4 service described in the book
Globus Toolkit 4 Programming Java
Services by Borja Sotomayor and
Lisa Childers, 2006.
All the code comes from this book.
The book is based upon the on-line
tutorial available at
http://gdp.globus.org/gt4-tutorial/
12
Assignment Goals
• Show how stateful WSRF Web services can be
created and deployed in Globus 4.0.
• Assignment is to be done on one’s own
computer (or a lab computer).
• Requires one to install Globus 4.0 core and
associated software – all available for download.
13
Testing installation
Start Container
Command:
globus-start-container -nosec
-nosec specifies “no security”, which simplifies
assignment (no need for user certificates)
Will display list of deployed services.
14
Standard list of deployed
services in core
7-1.15
Fig 7.7
Purpose of Service
To store an integer called value, acted upon by three
methods to:

Add a to value

Subtract a from value

Get value
public interface Math {
public void add(int a);
public void subtract(int a);
public int getValueRP();
}
Methods given. Further methods will be implemented.
Service is stateful (value retained between accesses).
16
Resource Properties
Resource
Resource properties
Math Web Service
Client
“value”
(integer)
“last operation
performed”
(string)
Code provided actually two
resource properties
17
Steps in Assignment

Preliminary set-up - Install GT4 and associated software

Download provided files:

WSDL service interface file

Math web service code (in Java)

WSDD Deployment Descriptor file

Client code (Java) to exercise service

Build Math service

Deploy Math service into GT 4 container

Write and compile the client

Start container and execute client.

Add functionality to service
18
GT4 container
After Service Deployed
Mathservice
7-1.19
GridSphere portlet design
assignment



To learn how to deploy portlets within the
Gridsphere Grid portal but on your own
computer.
Fully understand the underlying software
structure of JSR 168 grid portals.
Design portlets as front-ends to Grid services
and applications.
20
Portlets
21
Fig 8.14
JSR 168 portlet
22
Fig 8.15
Step 1:
Download and Install
Tomcat
http://localhost:8080/
Step 2:
Download and Install
Gridsphere
23
http://localhost:8080/gridsphere/
Creating your own portlet
Goal
Create and deploy a portlet that will accept one number
and say whether it is odd or even (“oddeven” portlet)
The code for this portlet is given.
You simply have to deploy it.
Later you will to ceate your own portlets.
24
Final portlet displayed in GridSphere
11a.25
Developing your own portlet
Example
Make a portlet that
allows a user
creates a portlet
that can add,
subtract, multiply
and divide two
numbers
26
Portlet acting as a front-end to a
Web service
27
More Information
on Gridsphere
See the links on
the Gridsphere
portal page
11.28