Download Installing and Building GTLAB

Installing and Building
GTLAB
Marlon Pierce
Indiana University
GTLAB and OGCE
 OGCE contains multiple sub-projects






Portlet-based Grid portal (with GridSphere and Tomcat).
Workflow suite (services and add-ins to the portal)
Information Web services
Gadget container
Cyberaide JavaScript libraries
GTLAB
 These are packaged with Maven and include everything you need
except Java and (for some services) MySQL.
 We try to make things installable with minimal fuss.




Consistent directory structures across projects
Edit one config file (pom.xml)
Run one command (mvn clean install)
You may need to futz a little with MySQL
Overview and Examples
User’s Browser
Portlet, Gadget
Containers
iGoogle, Facebook
GTLAB Gadgets
RPS, RDS, GPIR
Services
TeraGrid Services
(GRAM, GridFTP, INCA)
Google GData Services,
Twitter Feeds, etc
GTLAB’s Relationship to Other
Components
Various GTLAB
applications deployed
as portlets:
Remote directory
browsing, proxy
management, and
LoadLeveler queues.
GTLAB Applications as Google Gadgets:
MOAB dashboard, remote directory
browser, and proxy management.
MyProxy Example
<h:form id="form">
<h:outputText value="Username:"/>
<h:inputText value="#{resource.username}" required="true"/>
<h:outputText value="Password: "/>
<h:inputSecret value="#{resource.password}" required="true"/>
<h:outputText value="MyProxy Server: "/>
<h:inputText value="#{resource.myproxyHostname}" required="true"/>
<o:submit id="submit" value="Submit" binding="#{builder.body}"
action="#{builder.junkAction}" >
<o:multitask id="multi" persistent="true" taskname="#{resource.taskname}">
<o:myproxy id="mypr" hostname="#{resource.myproxyHostname}"
lifetime="2" password="#{resource.password}" port="7512"
username="#{resource.username}"/>
</o:multitask>
</o:submit>
</h:form>
Executing Multiple Task
<o:submit id="submit" value="Submit" binding="#{builder.body}"
action="#{builder.junkAction}" >
<o:multitask id="multi" persistent="true"
taskname="#{resource.taskname}">
<o:jobsubmit id="js" myfaces="true"
arguments="#{resource.arguments}"
executable="#{resource.executable}"
hostname="#{resource.hostname}"
provider="#{resource.provider}"
stdout="#{resource.stdout}"/>
<o:filetransfer id="ft" myfaces="true"
from="#{resource.from}" to="#{resource.to}" />
<o:dependency id="dep" task="ft" dependsOn="js"/>
</o:multitask>
</o:submit>
Resource Prediction Service
Client with GTLAB
<o:submit id="submit" value="Submit"
binding="#{builder.body}"
action="#{builder.junkAction}">
<o:multitask id="multi"
persistent="true"
taskname="#{resources.taskname”>
<o:rpsOptimal
resource="task1-resources"
id="rpsoptimal-exec"
myfaces="true"/>
</o:multitask>
</o:submit>
Example with Dependency
<o:submit id="submit” …>
<o:multitask id="multi” …>
<o:rpsAddModel resource="task1-resources"
id="rpsAddModel-exec"
myfaces="true"/>
<o:rpsManager resource="task1-resources"
id="rpsManager-exec"
myfaces="true"/>
<o:dependency id="deps-rpsstuff”
task="rpsManager-exec"
dependsOn="rpsAddModel-exec"/>
</o:multitask>
</o:submit>
More Detail: Available
Tags and Features
GTLAB Features
 Extends Java Server Faces.
o Tag components wrap major COG Abstraction Layer features
o And Web Service clients, GridShib, etc.
o Supports tag inter-dependencies.
 Allows you to do standalone development.
 Use JSF portlet bridge to convert into portlets
 No new coding, just add/modify XML config files and jars.
 Process is automated
 Or forget about portlets
o Convert into Google Gadgets
o Develop as a Facebook application
o Etc.
Getting GTLAB
 See http://www.collab-ogce.org/ogce/index.php/GTLAB
 You can use your favorite SVN client to check out.
 svn co https://ogce.svn.sourceforge.net/svnroot/ogce/GTLAB
(latest)
 svn co
https://ogce.svn.sourceforge.net/svnroot/ogce/tags/GTLABtg09 (tagged)
 “Latest” will give you easy access to any updates
 “svn update”
 Best option if you want to actively develop and get fixes right
away.
No SVN? Get the TAR
 SourceForge’s SVN/CVS
viewer now provides a
“Download GNU Tar”
option.
 http://ogce.svn.sourc
eforge.net/viewvc/ogc
e/GTLAB/ for latest.
 http://ogce.svn.sourc
eforge.net/viewvc/ogc
e/tags/GTLAB-tg09/
for tag
Levels of Abstraction in GTLAB code
You can use prepackaged gadgets/portlets.
You can develop new applications using
existing tag libraries.
You can make new tags for your services
You can hack or steal the code
 Note JSF is compatible with JSP, so you can mix and
match.
 And you can embed JavaScript
 YUI, Scriptaculous user interface goodies.
 Google and Facebook APIs.
Tag
MyProxy
JobSubmit
FileOperation
FileTransfer
XSubmissionOutput
ResourceDiscovery
ResourcePrediction
MyCommunityProxy
Community Log
Description
Gets a proxy credential (COG)
Runs remote commands (COG)
Create, delete, list remote files and
dirs (COG)
GridFTP downloads, 3rd party
transfers (COG)
Format outputs of job and file
operations
Clients for the RDS
Clients for the RPS
Get and decorate TG Community
credentials with Grid Shib
Log use of TG Community credentials
GTLAB Example
Description
MyProxyExample
Get a credential and store in OGCE
global proxy store.
JobSubmitExamples
Various examples showing how to
submit a grid job and get output.
Dashboard Examples
Runs showq on Big Red, Cobalt, and
Mercury and formats output.
DirectoryBrowser
Displays and downloads files on a
remote host via GridFTP
MultipleTaskExample
Shows how to couple file transfers,
operations, and
These will work as gadgets, portlets,
or standalone applications
Gadgets and Portlets
GTLAB gadgets
can also run as
portlets with no
code changes by
using the JSF
portlet bridge.
iGoogle
Client
Aggregator
Tomcat Web Server
+
Portlet Container
Tomcat
Web
Server
Compile and deploy into server
GTLAB Gadgets
Code
Base
Decorate with bridge
and container jars, xml
config files.
GTLAB Portlets
Code
Base
Making GTLAB Google Gadgets
 A Gadget is a standalone application that is integrated
into your personal iGoogle display.
 They can run on your Web server.
 You can use SSL/HTTPS
 Google Gadgets have two versions
 http: these use Google’s JavaScript APIs
 url: these wrap external applications with Iframes.
 We currently support “url” gadgets.
Example Gadget Config File
<?xml version="1.0" encoding="UTF-8" ?>
<Module>
<ModulePrefs title="MyProxy Gadget Example"
scrolling="true"
height="500"/>
<Content type="url"
href="http://…/:8080/GTLAB/examples/MyProxyExample.jsf">
</Content>
</Module>
Save this as MyProxy.xml and place
in a Web accessible place.
Gadget containers
aggregate content
from multiple
providers. Content
is aggregated on the
client by the user.
Nearly any web
application can be a
simple gadget (as
Iframes)
Other Gadgets
Providers
Tomcat + GTLAB
Gadgets
Other Gadgets
Providers
RSS Feed, Cloud, etc
Services
Grid and Web
Services
(TeraGrid, OSG, etc)
Social Network
Services (Orkut,
LinkedIn,etc)
Click “Add stuff” and
then “Add feed or
gadget”. Type the
URL of your gadget.
GTLAB Applications as Google Gadgets:
MOAB dashboard, remote directory
browser, and proxy management.
Common science
gateway
architecture.
Aggregation is in
the portlet
container.
Users have limited
selections of
components.
HTML/HTTP
Tomcat
+
Portlets and
Container
SOAP/HTTP
Grid and Web
Services
(TeraGrid, OSG, etc)
Grid and Web
Services
(TeraGrid, OSG, etc)
Grid and Web
Services
(TeraGrid, OSG, etc)
Making GTLAB Gadgets into Portlets
 Do all development in jsf_standalone directory
 This is where your template tags will be placed
 When you are ready to convert:
 Download and build the OGCE portal




cd GTLAB/transition
mvn clean process-resources
cd GTLAB/portlets
mvn clean install
 You will need to make small edits to portlet.xml,
group.xml, and layout.xml in
GTLAB/portlet/src/main/webapp/WEB-INF/
 That’s it.
Various GTLAB
applications deployed
as portlets:
Remote directory
browsing, proxy
management, and
LoadLeveler queues.
Getting Started with
GTLAB
GTLAB Directory Structure
Directory
Description
jsf_standalone
Gadget and stand-alone code and web pages
portal_deploy
Deployed applications go here. Includes
Tomcat.
templateTag
Template directory for making new tags
global-config
Global jars and config files for Tomcat
certificates
TeraGrid Certificates. Other Certificates go
here.
transition
Directory for converting standalone applications
into portlets (currently obsolete)
portlet
Portlet versions of the GTLAB standalone
applications
Build GTLAB
 Unpack or checkout code
 Cd GTLAB
 All commands are
executed from here.
 Edit properties at the
top of pom.xml.
 Change IP
 Change project.home if
you unpack someplace
besides $HOME.
 Run “mvn clean install”
<properties>
<portal.server.ip>
156.56.104.143
</portal.server.ip>
<host.base.url>
http://${portal.server.ip}:8080/
</host.base.url>
<project.home>
${env.HOME}/GTLAB
</project.home>
<tomcat.version>
apache-tomcat-5.5.27
</tomcat.version>
<catalina.home>
${project.home}/portal_deploy/${tomcat.version}/
</catalina.home>
<dot.globus.home>
${env.HOME}/.globus/
</dot.globus.home>
</properties>
Run Examples
 From GTLAB, start tomcat with ./startup.sh.
 From GTLAB, stop Tomcat with ./shutdown.sh
 Point browser to http://localhost:8080/GTLAB
 Start with MyProxy Example
Next Steps
 Play with examples.
 These are really bare bones. Make something
interesting.
 Make a Google gadget.
 Mix and match tags in a pipeline to make a new
application.
 Use the dependency tag.
 Note you can mix and match JSF and JSP if you are
not familiar with JSF.
 Try making a new tag.
 Explained next.
Making New Tags and
Examples
Making a New JSF Page from Tags
 I recommend starting from the examples.
 jsf_standalone/src/main/webapp/examples
 “Build” the examples with
 mvn –o clean install –f jsf_standalone/pom.xml
 The “-o” option is used to build offline. Will also avoid
unnecessary Maven repository updates.
 The “-f” specifies only build this specific module.
 I recommend not futzing with the deployed versions under
portal_deploy.
 A computer is a state machine. State must be reproducible.
Making a New Tag
 Run the following command from GTLAB:
 mvn clean process-resources -Dtag.name=test Dprojectname=Test -f templateTag/pom.xml
 Add -Ddest.dir=/tmp for a dry run.
 Replace “test” with the name of your tag.
 Replace “Test” with the name for your Bean.
 This will make 4 files
 TestBean.java, TestTag.java, TestBeanFactory.java,
UITest.java
 Edits also 3 config files
 gtlab-factory.xml, managed-beans.xml, components.xml
 This will compile but to implement something useful, you will
need to edit the highlighted files.
Implementing a Tag
 The place to start is TestBean.java (or whatever
you used for –Dprojectname=…).
 This includes several inherited methods that can be
implemented.
 Most important is submit(). Use the try/catch
block. This is where the action is.
 If you want to hook tags into chains, implement
getOutput() and setInput().
 Also take a look at the other beans.
Suggested Tags: A Wish List
 What can you do in your bean? Anything server-side Java
can do.
 Some suggestions:
 Implement a tag client to a remote Web service. Amazon has
some interesting ones….
 Implement an RSS/Atom feed client to Twitter, your blog,
Facebook, etc.
 Combine the feeds as a mash-up.
 Connect to a database with JDBC.
 Implement a JMS publisher or subscriber.
 Use Google Java APIs to interact with Blogger, Calendar, and
YouTube.
 Try interacting with Facebook.
INCA Tags: An Extended
Example
Example: Make an INCA Dashboard
 INCA is used as the testing framework for the TeraGrid.
 You can get the latest INCA test results from the URL
 http://inca.teragrid.org/inca/XML/kit-statusv1/portal_summary
 See Sangmi’s notes at
http://sangpall.blogspot.com/2009/03/being-nice-clientof-inca-service.html
 You’ll get back some fine looking XML.
 Let’s make a little dashboard out of this.
First, Create a New, Empty Tag
mvn clean process-resources \
-Dtag.name=incastatus \
-Dprojectname=IncaStatus \
-f templateTag/pom.xml
Second, Implement the Bean
 Main thing is to implement IncaStatusBean’s
submit() method.
 Code is hard to put in a PPT slide
 See
http://ogce.svn.sourceforge.net/viewvc/og
ce/GTLAB/jsf_standalone/src/main/java/o
gce/gsf/gridbeans/IncaStatusBean.java
Third, Implement a JSF Page
 Actual tag is shown below.
 Full example is
http://ogce.svn.sourceforge.net/viewvc/ogce/GTL
AB/jsf_standalone/src/main/webapp/examples/In
caStatusExample.jsp?view=log
 It includes some boilerplate (can be automated).
<o:incastatus id=”mytest"
resource="task1-resources”
myfaces="true"/>
Here’s the resulting page. You can make into a portlet in a
couple of steps. You can also tie to other tags in a
pipeline.
Next Steps: Make a Pipeline
 Implement the
setInput() and
getOutput() methods.
 Pipeline it with the
Resource Description
and Resource Prediction
Service tags.
 Integrate QBETS tags
 Currently unwritten
 Integrate with Job
Submission
Find Running Hosts
(INCA or RDS)
Find Best Host for
YourJob (RPS)
Find Host with
Shortest Wait Time
(QBETS)
Submit Job
More Stuff about GTLAB
 Supports linear tag dependencies.
 Removed dependence on COG graph processing (still use the
COG)
 We can support graphs of any tag you care to write.
 Only support pipelines currently, not full DAGs
 Redesigned to simplify writing new tags.
 Extensive use of factories.
 You no longer have to change core code to add a tag
 Templates for 4 Java files and 3 XML config files now can be
done in one (mvn) step
 Typically you will only need to edit one Java file.