Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Building Web Portals as Science Gateways Marlon Pierce [email protected] with slides from Dennis Gannon, Ahmet Sayar, Galip Aydin Indiana University Outline of Talk • Overview of computing portals as gateways to cyberinfrastructure. – With several example projects • Architectures and standards of computing portals. – Review of portal architectures – Review of JSR168 • Building science portals. – With a motivating example • New technologies and techniques – JSF – AJAX Computing for the Greater Good • Recent devastating natural disasters have demonstrated the need for large scale, distributed scientific computing. – Katrina, Rita, … – Christmas 2004 tsunamis – Earthquakes in Pakistan and future major earthquakes in California, Japan, … • 1906 was 99 years ago… • Research in large scale distributed computing can make a difference in people’s lives. • But many challenges lie ahead. Cyberinfrastructure (CI) Overview • Cyberinfrasture combines – Grid computing: • Globus toolkit, Storage Resource Broker, Condor, UK OMII, OGSADAI, etc. • Provides the software for tying together distributed computing resources – High speed networking – Hardware: supercomputers, clusters, workstations – Applications: modeling, simulation, data analysis codes in many fields. – Data: raw, cooked, shared, derived, cataloged, etc. – Information/Internet Technology: web services, portals, databases, XML, semantic web/grid, audio/video collaboration, shared Web spaces, etc. – People • Cyberinfrastructure is supposed to enable remote collaborations and distributed teams. • This is the reality of a lot of science (and science funding) Scientific Challenges Genetics and Disease Susceptibility Science Communities and Outreach • The current and future generations of scientific • Communities problems are: • CERN’s Large Hadron Collider – Data Oriented (CI challenge) experiments • • • Increasingly stream based. • Physicists working in HEP and Often need petabyte archives similarly data intensive scientific But linking data to applications disciplines across groups is a general • National collaborators and those problem. across the digital divide in disadvantaged countries – In need of on-demand computing resources (CI • Scope challenge) • Interoperation between LHC Data Grid Hierarchy and ETF – Conducted by • Create and Deploydistributed Scientific geographically Dataof andspecialists Services Grid Portals teams • Bringdon’t the Power to bear • Who wantoftoETF become on LHCin Physics Analysis: Help experts grid computing. discover the Higgs Boson! Phenotype 1 Phenotype 2 Phenotype 3 Phenotype 4 Ethnicity Environment Age Gender • Partners Identify Genes • Caltech Pharmacokinetic s • UniversityMetabolism ofFlorida Endocrine Biomarker Physiology Proteome • Open Science Grid and Grid3 Signatures Transcriptome Immune Morphometrics • Fermilab • DOE PPDG Predictive Disease Susceptibility Terry Magnuson, UNC •Source: CERN Storms Forming • NSF GriPhyn and iVDGL Forecast Model Streaming• EU LCG and EGEE Data Mining Observations • Brazil (UERJ,…) On-Demand • Pakistan (NUST, …) Storm predictions • Korea (KAIST,…) LHC Data Distribution Model Science Web Portals The goals of a Science Web Portal are – To give a community of researchers easy access to • • • • Computational applications and tools Primary data, shared data, derived data Computational resources All are needed to solve today’s scientific and engineering problems. – To do this in a discipline specific language that is common to the target community. • Science Portals are “gateways” to underlying Grid tools – Grids provide access to computing resources – Gateways manage the user’s interaction with the grid. What is a Science Portal? • A standards-based web portal – Provides each user with • a secure way to login and authenticate identity • A view of a private workspace for accessing and managing data and tools • Tutorials and community news – Announcements of new services • Collaboration tools – Chat, calendaring, group messaging • Core Science Functions … Portal Science Capabilities • Data Access is the most important – Allow the user community access to important shared data resources • Visualize it, publish it, download it, curate it. – Data Discovery • Searchable metadata directories. • Web access to important tools – Web-form interfaces to allow users to run important community codes – Webstart access to common java-based tools • Workflow tools – Allow users to combine community codes into workflows managed by the portal back-end resources. Let’s look at a few real examples (about a dozen … many more exist!) ServoGrid Portal NEESGrid Realtime access to earthquake Shake table experiments at remote sites. Geological Information Grid Portal Mesoscale Meteorology NSF LEAD project - making the tools that are needed to make accurate predictions of tornados and hurricanes. - Data exploration and Grid workflow Renci Bio Portal Providing access to biotechnology tools running on a back-end Grid. - leverage state-wide investment in bioinformatics - undergraduate & graduate education, faculty research - another portal soon: national evolutionary synthesis center SCOOP ADCIRC Interface Set Run Dates Current ADCIRC grid Decomposition • SURA SCOOP – • Southeastern Coastal Ocean Observing Program Advanced Circulation Model (ADCIRC) – Finite Element Hydraulic Model for Coastal Areas, Inlets and Rivers • • • Rick Luettich (UNC), Joannes Westerink (Notre Dame) Randall Kolar (Oklahoma), Clint Dawson (Texas), Corps of Engineers Collaborators: – – Leuttich/Blanton (UNC Marine Sciences) Ramakrishnan/Lander (RENCI) Katrina Storm Surge Prediction (Leuttich/Blanton) X-Ray Crystallography Flood Modeling • University of Texas •TACC •Center for Research in Water Resources • ORNL • Purdue Large-scale flooding along Brays Bayou in central Houston triggered by heavy rainfall during Tropical Storm Allison (June 9, 2001) caused more than $2 billion of damage. Gordon Wells, UT; David Maidment, UT; Budhu Bhaduri, ORNL, Gilbert Rochon, Purdue Teragrid Gateway Portals • TeraGrid – The largest “public” supercomputing grid in the US. • 50 TFlops – Massive Data storage capability • Over 1 Petabyte on-line storage • TeraGrid Gateways – Access points for communities. – Two types: • A gateway between 2 grids – Open Science Grid gate way is one example. – Science Community Portal is the other type • Includes LEAD portal, Nanohub, Renci Bioportal, NVO • An additional set of new science portals just being developed. The Portal as a Service Framework • Many important science portals are designed to access and orchestrate standalone web services and databases • Others are service frameworks that sit on top of an existing Grid, such as TeraGrid. • In both cases: – The architecture is similar – Common portal services are often shared • A Common architecture is emerging The Architecture of Gateway Services The Users Desktop. Grid Portal Server Gateway Services Proxy Certificate Server / vault Application Workflow Application Deployment Application Events Resource Broker App. Resource catalogs User Metadata Catalog Replica Mgmt Core Grid Services Security Services Information Services Self Management Resource Management OGSA-like Layer Physical Resource Layer Execution Management Data Services Next Steps • Many components of this architecture are now shared between different portal groups. • We now look at some of the technology behind these portals. A Portal Architecture Review A Famous Web Portal A Famous Portal, After Login What to Notice After logging in, my colors, layouts, and content all changed. • I get my stock list, my Bloomington weather, my news stories, etc. • I got rid of “Garfield” As we will see later, each of these content fragments (encircled) is managed by a thing called a portlet… • … I don’t guarantee that this is true for Yahoo’s web site but it is true for a large class of enterprise Java portal systems. Portlets are the key to portal software reuse. Let 10,000 Flowers Bloom Many portal projects have been launched since late ’90s. • First Generation: HotPage from SDSC, NCSA efforts, DOD, DOE Portals, NASA IPG • 2002 Special Issue of Concurrency and Computation: Practice and Experience. The field continues to be active • Global Grid Forum 14 Science Gateway workshop in June 2005. • About 15 gateways will be described in upcoming issue of Concurrency. • GCE2005 workshop at Supercomputing 05. How do we share and reuse all of this work? Three-Tiered Architecture Portal User Interface Grid and Web Protocols JDBC, Local, or Remote Connection Portal Client Stub Database Service Database Portal Client Stub Grid Resource Broker Service HPC or Compute Cluster Portal Client Stub Information and Data Services Grid Information Services, SRB Three-tiered architecture is accepted standard for accessing Grid and other services JSR 168 Overview Java Portlet Standard What Is a Portlet? A portlet is a piece of Java code that manages the content of one section of a web portal’s HTML. It can do anything else that a Java web application can do. • You can connect a portlet to a database, invoke a web service, download an RSS feed, etc. It lives in a portlet container, which creates, manages, and destroys all the portlets of the portal. Portlet containers are part of portals. • Portals must do other things like manage login, users, groups, layouts, etc. JSR 168 standardizes two main things: • How the portlet container manages portlet lifecycles • How the portlets are programmed. What is JSR 168? From the portlet development point of view, it is really very simple: • You write a java class that extends GenericPortlet. • You override/implement several methods inherited from GenericPortlet. • You use some supporting classes/interfaces Many are analogous to their servlet equivalents Some (portletsession) actually seem to be trivial wrappers around servlet equivalents in Pluto. A complete example is available from • http://www.servogrid.org/slide/iSERVO/NMI/OG CETutorialMaterial Some Open Source JSR 168 Containers GridSphere • http://www.gridsphere.org uPortal • http://www.uportal.org LifeRay • http://sourceforge.net/projects/lportal eXo platform • http://www.exoplatform.com StringBeans • http://www.nabh.com/projects/sbportal Jetspeed2 • http://portals.apache.org/jetspeed-2/ Some GenericPortlet.java Methods Method Description Init Called when the portlet is created. Override if you need to set initial params. doView Controls what happens immediately before the portlet is displayed in view mode. Normally you override this. doHelp, doEdit Other portlet display modes processAction Place for handling any <form> actions before turning over to the display mode method (like doView). You should override this for web forms. Some Supporting Classes/Interfaces Class Description PortletContext Similar to servlet context; get context info and the RequestDispatcher from here. PortletSession Stores attribute information for a single portlet application across multiple requests. RenderRequest, RenderResponse The request and response objects available to the doView() method. Similar to the normal servlet request ActionRequest,Actio nResponse The request and response objects available to the processAction() method. Similar to the servlet request and response objects. PortletURL Use this to create URLs that reference the portal. PortletRequestDispa tcher Use this to include/forward to a JSP or servlet in the same portlet app. WindowState See if you are in minimized, maximized, normal state. The Big Picture As a portlet developer, the previous set of classes are all you normally touch. The portlet container is responsible for running your portlets. • Init, invoke methods, destroy. Portlets have a very limited way of interacting with the container. • The API is basically one-way. In Action: Get started. public class JunkPortlet extends GenericPortlet { public void init(){ //Do any initialization. } //Rest of the methods on following slides go here. … } Override doView() protected void doView( RenderRequest req, RenderResponse res) throws PortletException, IOException { //Include the desired JSP or HTML page. //We could also use out to write directly to the response. WindowState state=req.getWindowState(); if(!state.equals(WindowState.MINIMIZED)) { res.setContentType("text/html"); PortletRequestDispatcher rd= getPortletContext().getRequestDispatcher(“MyJSP.jsp ”); rd.include(req,res); } } The JSP Page <portlet:defineObjects/> <% PortletSession portletSession=renderRequest.getPortletSession(); portletSession.setAttribute("localattname","localattval"); PortletURL url=renderResponse.createActionURL(); String theActionString=url.toString(); %> HTML Content is here. A form is below. <form method=post action="<%=theActionString%>"> <input type=…> </form> Some Notes • Include the <%portlet:definedObjects/%> tag, which will instantiate renderRequest, renderResponse, and portletConfig objects. – You can then just use them, as with request, response, and other JSP implicit objects. • The renderRequest gives you access to the PortletSession, if you want to store session variables. – One of the trouble points. • The renderResponse gives you access to the PortletURL object. • Use the PortletURL to generate a URL for the <form action> – So that it points to portlet container and gets handled by the processAction() method, rather than going of into space. – Handle href URLs similarly. – This is one of the sticking points. Lastly, Override processAction() • When you invoke the form on the previous JSP, the portlet container will pass the action handling to the processAction method. • The ActionRequest can be used to get any of the <input> parameters in a way similar to the usual HttpServletRequest. • When the processAction method returns, the container then invokes the appropriate do method (usually doView). • If you need to pass <form> parameters on to doView, add them to the ActionResponse. – This will give them to the RenderRequest. – The example shows how to add ALL parameters. public void processAction (ActionRequest request, ActionResponse actionResponse) throws PortletException, java.io.IOException { //Process request parameters … //Add other params // to the renderRequest actionResponse.setRenderPar ameters(request.getParameter Map()); } A Comment on Portlet Coding JSR 168 seems to make some important and dubious assumptions • Developers will gladly ignore other development methodologies/frameworks like Velocity, Struts, and Java Server Faces. • Developers instead want to write a GenericPortlet extension for every single portlet they develop. And write really complicated processAction() and doView() methods. • Developers will like the specific JSR 168 portlet-style Model-View-Controller that it forces on them. Fortunately, these other development environments can be mapped to portlet actions. • In the OGCE project, we have developed support for Velocity portlets. • We are transitioning to Java Server Faces A Grid Portlet Scenario Developing a Simple Grid Portlet Application A Quantum Chemistry Code Submission Form You have been asked to develop a submission form for the Democritos group’s Quantum Espresso (QE) package. These forms should help users set up and run QE applications on the TeraGrid and other Grid installations. • Mix of GT 2, GT 4, Condor, etc., for submission You are told to initially support the Plane Wave Self Consistent Field (PWSCF) code. Other QE applications may follow. • These may be coupled with PWSCF into simple workflows, but this is a later problem. Your Deployment Architecture Your portal server runs at FSU. It acts as a common Gateway to different grid toolkit installations and resources GT 4 @TG TeraGrid LSF GT 2 @UMN MSI PBS Portal Server @FSU Some Issues You decide the JSR 168 style portlets are the way to go… • But of course the PWSCF portlet doesn’t exist yet. You will need to also support other Quantum Espresso codes. • Would like to reuse as much code as possible. • But your PWSCF portlet isn’t reusable at that level. You also would like to simplify saving user input data and session archiving. PWSCF Web Forms for Submission Your Choices JSR 168 will allow you to share your portlet code with other collaborators. The Java COG Kit will help hide the differences between Grid toolkits for common tasks. Vanilla JSF will help simplify your portlet development in several ways. • JSF decouples your backing code from the Servlet API You can write your backing code as pure Java Beans/Plain Old Java Objects. You don’t have to adopt, maintain HTTP parameter name conventions. • Your form input backing beans can be serialized/deserialized with Castor. Coupling JSF and COG will allow you to compose your Grid actions using simple JSF taglibs. • You can reuse your Grid taglibs in other Quantum Espresso portlets. • You can compose composite actions The Java CoG Kit Gregor von Laszewski Argonne National Laboratory University of Chicago [email protected] http://www.cogkit.org (as interpreted by MEP) CoG Kits CoG Kits make Grid programming simple and new technologies are easy to integrate We focus on a CoG Kit for Java • Python also available (K. Jackson, LBNL) • Availability: Java CoG Kit since 1997 The CoG provides two important things • A higher level client programming environment than stubs. • A shield against different versions of the Globus toolkit Same high level API works with GT 2.4, GT 3.0.2, GT 3.2.0, GT 3.2.1, GT 4.0.0 CoG Abstraction Layers Nano materials BioDisaster Informatics Management Applications Portals Development Support CoGGridfaces GridfacesLayer Layer CoG CoG CoG GridIDE GridIDE CoGData Dataand andTask TaskManagement ManagementLayer Layer CoG CoGAbstraction AbstractionLayer Layer CoG CoG CoG GT2 CoG CoG GT3 OGSI classic CoG CoG GT4 WS-RF CoG CoG CoG CoG Condor Unicore CoG CoG SSH CoG CoG Others Avaki SETI Task Handler The class diagram is the same for all grid tasks (running jobs, modifying files, moving data). Task Task Specification Service Security Context Classes also abstract toolkit provider differences. You set these as parameters: GT2, GT4, etc. Service Contact Setting Up Task and Specification Task task=new TaskImpl(“mytask”, Task.JOB_SUBMISSION); task.setProvider(“GT2”); JobSpecification spec= new JobSpecificationImpl(); spec.setExecutable(“rm”); spec.setBatchJob(true); spec.setArguments(“-r”); … task.setSpecification(spec); Setting Up the Service and Security Context Service service=new ServiceImpl(Service.JOB_SUBMISSION); service.setProvider(“GT2”); SecurityContext securityContext= CoreFactory.newSecurityContext(“GT2”); //Use cred object from ProxyManager securityContext.setCredentials(cred); service.setSecurityContext( (SecurityContext)securityContext); Set Up Service Contact and Finish ServiceContact serviceContact= new ServiceContact(“myhost.myorg.org”); service.setServiceContact(serviceContact); task.setService( Service.JOB_SUBMISSION_SERVICE, service); TaskHandler handler=new GenericTaskHandler(); handler.submit(task); Coupling CoG Tasks The COG abstractions also simplify creating coupled tasks. Tasks can be assembled into task graphs with dependencies. • “Do Task B after successful Task A” Graphs can be nested. Java COG Summary The Java COG 4 interfaces provide high level abstractions for building Grid clients. • Abstract out differences between Grid toolkits. • Provide task abstractions that form the basis for constructing DAG-style, file-based workflow. The COG can be used to build a wide range of clients • Desktops, grid shells, and of course portals. • Portlets are a well known way to build reusable portal components. Building Grid Portlets with Java Server Faces Limitations of Portlets Portlets provide a way to bundle and share a complete application. • RSS portlet, GridFTP portlet, SRB portlet, etc. • Portlets combine the user interface view and action code. But in science gateway development, we often need finer grained components. • “When user clicks button, upload file, launch code, and move data someplace when done.” • Combines “GridFTP” and “Job Submit” portlets… • Or maybe OGSA-DAI or SRB or …. We need a way for the view and action code to be developed from reusable parts. JSF and Science Gateways JSF enables you to back your science application input form portlets with Java Beans. • These are independent of the servlet container, so are easy to test and to reuse in other applications. But also, Java Beans can be easily serialized with XML. • Castor, XML Beans • Marshal and un-marshal user input for persistent storage in XML storage services OGSA-DAI, GPIR, WS-Context • Potentially, can develop backing code as XML Schema and generate the code. JSF for Grid Enabled HTML Widgets Natural program: develop Java Bean wrappers around Java COG kit, OGSA-DAI client API, SRB Jargon, etc. • Allows simple integration with JSF. Some issues exist • JSF only manages individual bean instances. • But grid portlets will need to manage an unknown number of bean instances. You may launch and monitor many different jobs. • We need a way of scripting composite actions created out of multiple reusable actions. COG Bean Wrappers Recall the COG structure • Executable tasks abstract basic grid actions and hide toolkit version differences. • These tasks can be collected into filebased DAG workflows. First problem is simple: wrap tasks as beans to make them available to JSF. • GenericGridBean defines the interface Second problem is managing multiple individual tasks. Managing Multiple Grid Tasks We must create and manage multiple beans for each task. • That is, I submit the job four times in one session. • Similarly, we can create multiple task graph clones. We do this by cloning and storing each bean. Beans have listeners and maintain state. • Unsubmitted, submitted, active, suspended, resumed are “live” Stored in live repository • Failed, canceled, completed, unknown are “dead” Stored in archive (WS-Context or other) Alternative approach: use one bean that is a bean factory for GenericGridTask beans. Corresponding JSF snippets <o:taskGraph id="myGraph" method="#{taskgraph.test}" > <o:task id="task1" method="task.create" type="FileTransfer" /> <o:task id="task2" method="task.create" type="JobSubmit" /> <o:task id="task3" method="task.create" type="FileTransfer" /> <o:taskAdd name="task1" method="taskgraph.add" /> <o:taskAdd name="task2" depends="task1“ method="taskgraph.add" /> <o:taskAdd name="task3" depends="task2" method="taskgraph.add" /> </o:taskGraph> <h:panelGrid columns="2"> <h:commandButton id="submit" value="Submit" action="#{taskgraph.submitAction}"/> <h:commandButton value="Clear" type="Reset"/> </h:panelGrid> <h:panelGrid columns="3" > <h:outputText value="Hostname (*) "/> <h:inputText value="#{task.hostname}"/> </h:panelGrid> <h:panelGrid columns="3" > <h:outputText value="Provider (*) "/> <h:inputText value="#{task.provider}"/> </h:panelGrid> Task Monitoring with JSF Data Model Corresponding Java class. public class Job { private String jobId; private String status; private String submitDate; private String finishDate; } <h:dataTable value="#{jobData.jobs}" var="job"> <h:column> <f:facet name="header"> <h:outputText style="font-weight: bold" value="Job ID" /> </f:facet> <h:outputText value="#{job.jobId}"/> </h:column> <h:column> <f:facet name="header"> <h:outputText style="font-weight: bold" value="Submit Date" /> </f:facet> <h:outputText value="#{job.submitDate}"/> </h:column> <h:column> <f:facet name="header"> <h:outputText style="font-weight: bold" value="Finish Date" /> </f:facet> <h:outputText value="#{job.finishDate}"/> </h:column> <h:column> <f:facet name="header"> <h:outputText style="font-weight: bold" value="Status" /> </f:facet> <h:outputText value="#{job.status}"/> </h:column> </h:dataTable> Using AJAX Galip Aydin, Ahmet Sayar, and Marlon Pierce Community Grids Lab Indiana University What Is AJAX? Asynchronous JavaScript and XML is a combination of standard Web technologies • JavaScript, CSS, DOM, XML When done properly, it provides apparently seamless interactivity in browser clients. • No browser reloads • Much smoother than standard request-wait-response for browser forms. Several Google tools are a very well known examples. • Google maps, Gmail, etc. The real key is the standardization of the XMLHttpRequest Object • Originally developed for Internet Explorer • Now supported by all major browsers. Seminal article from Adaptive Path • http://www.adaptivepath.com/publications/essays/archives/ 000385.php AJAX Architecture Taken from http://www.adaptivepath.com/publications/essays/archives/000385.php The Main Idea: Asynchronous JavaScript Calls to Server AJAX’s key concept is the use of XMLHttpRequest to buffer requests and responses within the browser. • Use XMLHttpRequest to make a call to the server and get back the response without displaying it. • The response is stored locally as either plain text (plain or HTML), or XML. • JavaScript + DOM can be used to walk the HTML or XML tree to handle most user interactions. The XMLHttpRequest object in turn can issue requests and process responses independently of the user’s interaction. • Go and fetch additional maps, for example. Asynchronous Invocations How it Works Simple Example: GIS and Google Maps The Web Feature Service is a standard geographical information system service. • It stores geographic features and metadata used to construct maps. • Features are represented with the Geographic Markup Language • For example, the location and properties of earthquake faults and seismic events. The Feature Service is typically constructed around traditional request-response. We would like to combine this with Google Maps to make more interactive user interfaces. So let’s get started. Integrating Google Maps and GIS Services Simple Architecture Client MAP interface Supported Feature Data by the WFS Google Map Server OGC WFS Feature Data Server Layer specific parameters for filtering Making the request Creating an XMLHttpRequest Object • For any browser, except IE var requester = new XMLHttpRequest(); • In IE var requester = new ActiveXObject("Microsoft.XMLHTTP"); Transporting Data using an XMLHttpRequest Object • To retrieve data from the server we use two methods: open() to initialize the connection, send() to activate the connection and make the request. requester.open("GET", "getFaultNames.jsp?State=CA"); requester.send(null); This request is for all California fault information in the Feature Service. • It will be encoded in GML • It may take a bit of time to download…. Checking the Connection Status To find out if the data retrieval is done we check the status of the readyState variable. Object’s status may be any of the following: 0 1 2 3 4 – – – – – Uninitialised Loading Loaded Interactive Completed • requester.onreadystatechange monitors the readyState variables status. if (requester.readyState == 4){ if (requester.status == 200){ success(); } else{ failure(); } } Parsing the Data After a successful request XMLHttpRequest object may hold data in one of the two properties: responseXML or responseText. responseXML stores a DOM-structured XML data. <Fault> <Name>San Andreas</Name> </Fault> We use JavaScript XML parsing methods such as getElementsByTagName(), childNodes[], parentNode… var faultNameNode = requester.responseXML.getElementsByTagName(“Name")[0]; var faultName = faultNameNode.childNodes[0].nodeValue; We can then use Google Map JavaScript functions to create the browser display. responseText stores the data as one complete string in case the content type of the data supplied by the server was text/plain or text/html. Ajax Calls From JSP Pages In the 1st JSP page function checkForMessage() { var url = "relay.jsp"; initRequest(url); req.onreadystatechange = processReqChange; req.open("GET", url, true); req.send(null); } The request is forwarded to relay.jsp In the 2nd JSP page (relay.jsp) <% Bean.getNames(response); %> The response object will contain the XML result object. Integrating Web Feature Service Archives and Google Maps Google maps can be integrated with Web Feature Service Archives to browse earthquake fault records. Faults are typically stored by segment number, so map interfaces are convenient for both verifying continuity and setting up input files for computing problems. Other Useful Examples AJAX is also useful for simulating server data pushing. • Browser interface is updated periodically from locally stored data. • XMLHttpRequest periodically requests updates asyncrhonously. Replace annoying browser reload polling and fragile, non-standard server push. The browser is always available for user interactions while the XMLHttpRequest object is being updated. Examples: Chat applications and GPS streams Real Time GPS and Google Maps Subscribe to live GPS station. Position data from SOPAC is combined with Google map clients. Select and zoom to GPS station location, click icons for more information. Conclusions Computing portals scientific gateways. • Gateways include both user interface components and supporting services. • Manage user interactions with Grid infrastructure. We reviewed standards for building portals out of reusable components. • JSR 168 portlet standard Several new techniques can be used to build portlets. • Java Server Faces: reusable components for portlets. • AJAX: better interactivity for portlets More Information [email protected] Open Grid Computing Environments • http://www.collab-ogce.org Geographical Information System services • http://www.crisisgrid.org QuakeSim • http://quakesim.jpl.nasa.gov Integrating JSF and COG 4 Mehmet Nacar and Marlon Pierce JSF for Grid Enabled HTML Widgets • Natural program: develop Java Bean wrappers around Java COG kit, OGSA-DAI client API, SRB Jargon, etc. – Allows simple integration with JSF. • Some issues exist – JSF only manages individual bean instances. – But grid portlets will need to manage an unknown number of bean instances. • You may launch and monitor many different jobs. – We need a way of scripting composite actions created out of multiple reusable actions. COG Bean Wrappers • Recall the COG structure – Executable tasks abstract basic grid actions and hide toolkit version differences. – These tasks can be collected into file-based DAG workflows. • First problem is simple: wrap tasks as beans to make them available to JSF. – GenericGridBean defines the interface • Second problem is managing multiple individual tasks. JSF Task Management Class Structure «interface» GenericGridBean TaskListenerBean 1 TaskGraphListenerBean 1 TaskBeanManager TaskGraphBeanManager * *1 1* TaskBean TaskGraphBean 1 * * Managing Multiple Grid Tasks • We must create and manage multiple beans for each task. – That is, I submit the job four times in one session. – Similarly, we can create multiple task graph clones. • We do this by cloning and storing each bean. • Beans have listeners and maintain state. – Unsubmitted, submitted, active, suspended, resumed are “live” • Stored in live repository – Failed, canceled, completed, unknown are “dead” • Stored in archive (WS-Context or other) • Alternative approach: use one bean that is a bean factory for GenericGridTask beans. Managing Multiple Tasks HashMap JSF Form Task Manager JSF retrieve Task Bean 1 Task Bean 2 Task Bean register JSF Task Bean 3 Managed Beans Creating Task Graphs • COG Task Graphs correspond to composite JSF Web Form actions. – Do X, Y, and then Z when user clicks the button. • Each of these actions may be reused, but the entire action is new. • We must do two things – Wrap the COG TaskGraphHandler with a bean and bean manager. – Provide tag bindings for defining the custom actions. Constructing Task Graphs JSF Request Form HashMap TaskGraph Manager JSF retrieve TaskGraph 1 register TaskGraph 2 TaskGraphBean JSF Monitoring Form TaskGraph 3 JobSubmit Bean FileTransfer Bean FileOperation Bean Managed Beans Expressing Task Graphs with Tags <%@taglib uri="http://java.sun.com/jsf/core" prefix="f"%> <%@taglib uri="http://java.sun.com/jsf/html" prefix="h"%> <%@taglib uri="http://www.ogce.org/gsf/task" prefix="o"%> ……… ……… <o:taskGraph id="myGraph" method="#{taskgraph.test}" > <o:task id="task1" method="task.create" type="FileTransfer" /> <o:task id="task2" method="task.create" type="JobSubmit" /> <o:task id="task3" method="task.create" type="FileTransfer" /> <o:taskAdd name="task1" method="taskgraph.add" /> <o:taskAdd name="task2" depends="task1" method="taskgraph.add" /> <o:taskAdd name="task3" depends="task2" method="taskgraph.add" /> </o:taskGraph> Corresponding JSF snippets <o:taskGraph id="myGraph" method="#{taskgraph.test}" > <o:task id="task1" method="task.create" type="FileTransfer" /> <o:task id="task2" method="task.create" type="JobSubmit" /> <o:task id="task3" method="task.create" type="FileTransfer" /> <o:taskAdd name="task1" method="taskgraph.add" /> <o:taskAdd name="task2" depends="task1“ method="taskgraph.add" /> <o:taskAdd name="task3" depends="task2" method="taskgraph.add" /> </o:taskGraph> <h:panelGrid columns="2"> <h:commandButton id="submit" value="Submit" action="#{taskgraph.submitAction}"/> <h:commandButton value="Clear" type="Reset"/> </h:panelGrid> <h:panelGrid columns="3" > <h:outputText value="Hostname (*) "/> <h:inputText value="#{task.hostname}"/> </h:panelGrid> <h:panelGrid columns="3" > <h:outputText value="Provider (*) "/> <h:inputText value="#{task.provider}"/> </h:panelGrid> Task Monitoring with JSF Data Model Corresponding Java class. public class Job { private String jobId; private String status; private String submitDate; private String finishDate; } <h:dataTable value="#{jobData.jobs}" var="job"> <h:column> <f:facet name="header"> <h:outputText style="font-weight: bold" value="Job ID" /> </f:facet> <h:outputText value="#{job.jobId}"/> </h:column> <h:column> <f:facet name="header"> <h:outputText style="font-weight: bold" value="Submit Date" /> </f:facet> <h:outputText value="#{job.submitDate}"/> </h:column> <h:column> <f:facet name="header"> <h:outputText style="font-weight: bold" value="Finish Date" /> </f:facet> <h:outputText value="#{job.finishDate}"/> </h:column> <h:column> <f:facet name="header"> <h:outputText style="font-weight: bold" value="Status" /> </f:facet> <h:outputText value="#{job.status}"/> </h:column> </h:dataTable> Extended Slides The Java CoG Kit Gregor von Laszewski Argonne National Laboratory University of Chicago [email protected] http://www.cogkit.org (as interpreted by MEP) CoG Kits • CoG Kits make Grid programming simple and new technologies are easy to integrate • We focus on a CoG Kit for Java – Python also available (K. Jackson, LBNL) – Availability: Java CoG Kit since 1997 • The CoG provides two important things – A higher level client programming environment than stubs. – A shield against different versions of the Globus toolkit • Same high level API works with GT 2.4, GT 3.0.2, GT 3.2.0, GT 3.2.1, GT 4.0.0 CoG Abstraction Layers Nano materials BioDisaster Informatics Management Applications Portals Development Support CoGGridfaces GridfacesLayer Layer CoG CoG CoG GridIDE GridIDE CoGData Dataand andTask TaskManagement ManagementLayer Layer CoG CoGAbstraction AbstractionLayer Layer CoG CoG CoG GT2 CoG CoG GT3 OGSI classic CoG CoG GT4 WS-RF CoG CoG CoG CoG Condor Unicore CoG CoG SSH CoG CoG Others Avaki SETI Task Handler The class diagram is the same for all grid tasks (running jobs, modifying files, moving data). Task Task Specification Service Security Context Classes also abstract toolkit provider differences. You set these as parameters: GT2, GT4, etc. Service Contact Java COG Summary • The Java COG 4 interfaces provide high level abstractions for building Grid clients. – Abstract out differences between Grid toolkits. – Provide task abstractions that form the basis for constructing DAG-style, file-based workflow. • The COG can be used to build a wide range of clients – Desktops, grid shells, and of course portals. – Portlets are a well known way to build reusable portal components. Building Grid Portlets with Java Server Faces Limitations of Portlets • Portlets provide a way to bundle and share a complete application. – RSS portlet, GridFTP portlet, SRB portlet, etc. – Portlets combine the user interface view and action code. • But in science gateway development, we often need finer grained components. – “When user clicks button, upload file, launch code, and move data someplace when done.” – Combines “GridFTP” and “Job Submit” portlets… – Or maybe OGSA-DAI or SRB or …. • We need a way for the view and action code must be developed from reusable parts. PWSCF Web Forms for Submission Java Server Faces Overview • JSF can solve the reusable portlet widget problem. – JSF can also work in “standalone” mode outside of portlets. – Potentially independent of Web applications. • XUL and Swing widget bindings • We will first examine JSF generally • Conclude with integrating JSF and COG Advantages of JSF • JSF hides communication details that connect HTML forms with backing code. – You don’t have to worry about servlet specific request, response, and session objects. – You don’t have to maintain fragile naming conventions for <input> tags. • Developers only need to develop JavaBeans and tag libraries. – Beans are independent of Web applications. – Can be easily written and tested outside of servlet containers. – Compatible popular “Inversion of Control” based systems like JSF and Spring JSF and Science Gateways • JSF enables you back your science application input form portlets with Java Beans. – Again, these are independent of the servlet container, so are easy to test and to reuse in other applications. • But also, Java Beans can be easily serialized with XML. – Castor, XML Beans – Marshal and un-marshal user input for persistent storage in XML storage services • OGSA-DAI, GPIR, WS-Context – Potentially, can develop backing code as XML Schema and generate the code. A JSF Example <HTML> <HEAD> <title>Hello</title> </HEAD> <%@ taglib uri="http://java.sun.com/jsf/html" prefix="h" %> <%@ taglib uri="http://java.sun.com/jsf/core" prefix="f" %> <body bgcolor="white"> <f:view> <h:form id="entryForm" > <h2> Enter some text in the form below</h2> <h:inputText id="userSt" value="#{multiEventBean.userString}"/> <h:commandButton id="submit" action="success" value="Submit" > <f:actionListener type="multiEventTest.Listener1"/> <f:actionListener type="multiEventTest.Listener2"/> </h:commandButton> </h:form> </f:view> </body> </HTML> The JSF Page • Note everything with <f:> or <h:> namespace prefix is a JSF tag. – Usually, <h:> tags mimic HTML widgets. – <f:> is for non-rendered stuff. – Everything else is good old HTML. • There are three different Java classes here. – They are all in the package multiEventTest – multiEventBean.java is just a bean with typical get/set methods. – Listener1.java and Listener2.java implement the javax.faces.event.ActionListener interface. • All 3 classes are called when you click the command button. • Also, take a look at <inputText>. This is roughly equivalent to <input name=“” value=“”>. – But no name needed. JSF handles parameter names for you and connects them to the beans you specify. – This greatly simplifies writing generic actions. A Simple Example: HtmlDataGrid <%@ taglib uri="http://java.sun.com/jsf/html" prefix="h" %> <%@ taglib uri="http://java.sun.com/jsf/core" prefix="f" %> <f:view> <h2> This shows an example of how to use HtmlDataTable to display some results. </h2> <h:dataTable value="#{ValueBean.itemList}" var="values" border="1"> <h:column> <f:facet name="header"> <h:outputText value="Column 1"/> </f:facet> <h:outputText value="#{values.value1}"/> </h:column> <h:column> <f:facet name="header"> <h:outputText value="Column 2"/> </f:facet> <h:outputText value="#{values.value2}"/> <h:outputText value="#{values.value2}"/> </h:column> </h:dataTable> </f:view> It Looks Like This Some Explanation • The <h:dataTable> binds to a particular data set with value=“ValueBean.itemList”. – ValueBean.itemList must be java.util.List, java.sql.ResultSet or similar. – The values of these lists may be beans also. • The var=“” just defines a useful internal value. – Here, each entry in the list is set equal to “values”. – In the example, the items in the list happen to be simple beans with member variables “value1” and “value2” along with appropriate getter/setters. • When you load the page, it just iterates through the entries in ValueBean.itemList and creates the table, as you instructed. • Note again there are no loops in the actual page. Also, you don’t know anything about the data you are getting. JSF Magic: Data Models • The M in MVC is the Data Model – “Abstract” representation of a data structure • That is, not tied to the display of the data (“view”) • JSF DataModel extension classes include – – – – – Arrays: wrap arrays of Java objects Lists: wraps a java.util.List of Java objects Results: for JSTL’s Result object, which itself wraps SQL ResultSets. ResultSets: also for JDBC, wraps the ResultsSet object directly. Scalar: Wraps an individual Java object. • Typically, these should follow Bean patterns for naming member data and their associated getter/setters. • You can write your own specialized Data Models. – XMLDataModel, for wrapping an XML data bean, comes to mind. – RSSDataModel is another that I found. • These are associated with UIData classes in the JSF page for display. – HtmlDataTable is an example. JSF Form Validators • The user interface stuff (<f:> and <h:>) has lots of built-in validators – Verify that input entries are integers, doubles, etc. – Verify that input entries fall within the correct range (1 to 10 in the guessNumber example). – Verify the string has the right length. • You can extend to write your own specialized validators