Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Forschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft The European CrossGrid Project Marcel Kunze Abteilung Grid-Computing und e-Science Forschungszentrum Karlsruhe GmbH On behalf of the CrossGrid Collaboration Special thanks to Marian Bubak and Jesus Marco www.eu-crossgrid.org 5th EU DataGrid Conference, Budapest, September 2002 Outline The CrossGrid Project CrossGrid Testbed Aplications Status Deliverables and Plans 5th EU DataGrid Conference, Budapest, September 2002 EU Funded Grid Project Space (Kyriakos Baxevanidis) - Links with European National efforts - Links with US projects (GriPhyN, PPDG, iVDGL,…) Applications EGSO CROSSGRID GRIA GRIP Middleware & Tools EUROGRID DAMIEN DATAGRID GRIDLAB DATATAG Underlying Infrastructures 5 EU /DataGrid Conference, Budapest, September 2002 Industry business th Science CrossGrid Collaboration 21 institutes 11 countries Ireland: Poland: TCD Dublin Netherlands: UvA Amsterdam Spain: Portugal: LIP Lisbon CSIC Santander Valencia & RedIris UAB Barcelona USC Santiago & CESGA Germany: FZK Karlsruhe TUM Munich USTU Stuttgart Austria: U.Linz Italy: DATAMAT Cyfronet & INP Cracow PSNC Poznan ICM & IPJ Warsaw Slovakia: II SAS Bratislava Greece: Algosystems Demo Athens AuTh Thessaloniki 5th EU DataGrid Conference, Budapest, September 2002 Cyprus: UCY Nikosia Main Objectives EU Vth Framework Programme (IST) Project, started March 2002 21 partners from 11 countries New category of Grid enabled applications Computing and data intensive Distributed Interactive, near real time response (a person in a loop) Layered New programming tools Grid more user friendly, secure and efficient Interoperability with other Grids Implementation of standards 5th EU DataGrid Conference, Budapest, September 2002 Collaboration with other Grid Projects Exchange of Information Software components Partners DATAGRID DATATAG GRIDLAB EUROGRID and GRIP GRIDSTART Participation in GGF 5th EU DataGrid Conference, Budapest, September 2002 Workpackages WP1 – CrossGrid Application Development Weather forecast and air pollution modeling Tools for parallel programming and debugging on the Grid (MPI) Portals and roaming access Resource management Monitoring Optimisation of data access WP4 - International Testbed Organisation Interactive distributed data analysis in HEP WP3 – New Grid Services and Tools Flooding crisis support WP2 - Grid Application Programming Environments Biomedical simulation and visualization Integration team, certification authority, support and test procedures Based on EDG 1.2 (see talk of Marcus Hardt in WP6 parallel session) WP5 - Project Management Coordination, architecture, dissemination 5th EU DataGrid Conference, Budapest, September 2002 Biomedical Application CT / MRI scan Segmentation Medical DB LB flow simulation Medical DB 10 simulations/day 60 GB/simulation > 20 MB/s 5th EU DataGrid Conference, Budapest, September 2002 Visualization VE WD PC PDA HDB Interaction VR-Interaction 5th EU DataGrid Conference, Budapest, September 2002 Cascade of Flood Simulations Data sources Meteorological simulations Users Hydrological simulations Hydraulic simulations Output visualization 5th EU DataGrid Conference, Budapest, September 2002 Example of the Flood Simulation - Flow and Water Depth 5th EU DataGrid Conference, Budapest, September 2002 Distributed Data Analysis in HEP Objectives Distributed data access Distributed data mining techniques with neural networks Issues Typical interactive requests will run on o(TB) distributed data Transfer/replication times for the whole data about one hour Data transfers once and in advance of the interactive session Allocation, installation and set-up of corresponding database servers before the interactive session Integration of user-friendly interactive access (based on PROOF) 5th EU DataGrid Conference, Budapest, September 2002 Parallel ROOT Facility: PROOF Selection Parameters TagD B CPU Procedure PROOF RD B Proc.C Proc.C Proc.C Proc.C Proc.C http://root.cern.ch DB 1 DB 2 DB 3 DB 4 DB 5 DB 6 Local CPU Remote CPU CPU CPU CPU 5th EU DataGrid Conference, Budapest, September 2002 Weather Forecast and Air Pollution Modeling Distributed/parallel codes on the Grid Coupled Ocean/Atmosphere Mesoscale Prediction System STEM-II Air Pollution Code Integration of distributed databases Data mining applied to downscaling weather forecast 5th EU DataGrid Conference, Budapest, September 2002 Key Features of CrossGrid Applications Data To be selected on demand Processing Data sources and data bases geographically distributed Large processing capacity required; both HPC & HTC Interactive Presentation Complex data requires versatile 3D visualisation Support for interaction and feedback to other components 5th EU DataGrid Conference, Budapest, September 2002 Overview of the CrossGrid Architecture Applications Supporting Tools 1.1 BioMed 2.2 MPI Verification 1.2 Flooding 2.3 Metrics and Benchmarks Applications Development Support App. Spec Services Generic Services Fabric MPICH-G 1.1 User Interaction Services 3.2 Scheduling Agents DataGrid Job Submission Service Resource Manager (CE) CPU 1.3 Interactive Distributed Data Access 1.3 Data Mining on Grid (NN) 2.4 Performance Analysis 3.1 Portal & Migrating Desktop 1.4 Meteo Pollution 1.1, 1.2 HLA and others 1.1 Grid Visualisation Kernel 3.4 Optimization of Grid Data Access GRAM Resource Manager (SE) Secondary Storage GridFTP 3.3 Grid Monitoring GIS / MDS Resource Manager 3.4 Optimization of Local Data Access 3.1 Roaming Access GSI Resource Manager Instruments ( Satelites, Radars) Tertiary Storage 5th EU DataGrid Conference, Budapest, September 2002 Globus-IO DataGrid Replica Manager Globus Replica Manager Replica Catalog Replica Catalog Status after M6++ Software Requirements Specifications together with use cases CrossGrid Architecture defined Detailed Design documents for tools and the new Grid services (OO approach, UML) Analysis of security issues and the first proposal of solutions Detailed description of the test and integration procedures Testbed first experience Sites: LIP, FZK, CSIC+USC, PSNC, AuTH+Demo Basic: EDG release 1.2 Applications: EDG HEP simulations (Atlas,CMS) first distributed prototypes using MPI: NN distributed training Evolutionary Algorithms 5th EU DataGrid Conference, Budapest, September 2002 CrossGrid Testbed Map Géant TCD Dublin USC Santiago PSNC Poznan UvA Amsterdam ICM & IPJ Warsaw FZK Karlsruhe CYFRONET Cracow CSIC-UC IFCA Santander II SAS Bratislava LIP Lisbon CSIC RedIris Madrid Auth Thessaloniki UAB Barcelona CSIC IFIC Valencia DEMO Athens 5th EU DataGrid Conference, Budapest, September 2002 UCY Nikosia Sample Testbed Sites Valencia GoG farm, Santander (GridWall), FZK 5th EU DataGrid Conference, Budapest, September 2002 Plans for the Future Participation in production testbed with DataGrid Common DEMO at IST 2002, Copenhagen, November 4th-6th Collaboration with DataGrid in specific points (e.g. user support and helpdesk software) Conference together with RI Forum and the “Across Grids” All sites will be ready to join by end of September Santiago de Compostella, Spain, February 9th-14th,2003 With Proceedings (reviewed papers) CrossGrid workshop, Linz (w/ EuroPVM/MPI 2002), September 28th-29th 5th EU DataGrid Conference, Budapest, September 2002 Linz CrossGrid Workshop Sep.28th-29th Evaluate the current status of all tasks Contact partners who are developing software we are going to use: discuss interfaces and functionality Understand what we may expect as first prototypes Coordinate the operation of testbeds Agree about common rules for software development: Standard Operational Procedures (SOP) document written Start to organize the first CrossGrid EU review Meet with EU DataGrid representatives Discuss the technology for the future (OGSA) Details at http://www.gup.uni-linz.ac.at/crossgrid/workshop/ 5th EU DataGrid Conference, Budapest, September 2002 CrossGrid Schedule and Deliverables M6 (now) WP5 CrossGrid website WP4 First testbed prototype release M11 (Feb. 2003) WP4 internal status report M10 (Jan. 2003) WP4 are working on launching the first testbed. M9 (Dec.2002) WPs 1-3 start to develop prototypes of the applications, tool environment, and the new grid services WP5 Project leaflet/brochure WP5 Dissemination and exploitation report CrossGrid conference in Santiago di Compostella, February 2002 M12 (March 2003) WP1 First software release WP2 First prototypes WP3 First prototypes WP5 Report on requirements on integration and interoperability with DataGrid WP5 Yearly report 5th EU DataGrid Conference, Budapest, September 2002 1980s: Internet 1990s: Web 2000s: Grid Where do we need to get to ? A Grid infrastructure which hides the complexities from the users (“Invisible Computing”) A powerful and flexible network infrastructure Where do we need to invest ? Applications to support an “e-society” (“Cyber-Infrastructure”) Applications targeted at realistic problems in “e-science” Prototypes of Grid infrastructures Maintain and improve the GEANT network Expression of Interest for EU FP6 program: Grid-enabled Applications “Enabling Grids and e-Science in Europe (EGEE)” 5th EU DataGrid Conference, Budapest, September 2002 Prototype Grid Infrastructures Gèant: World Class Networking