Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Grid Computing Overview Thanks to Mark Ellisman Data Acquisition Advanced Visualization Imaging Instruments Analysis Computational Resources Large-Scale Databases Coordinate Computing Resources, People, Instruments in Dynamic Geographically-Distributed Multi-Institutional Environment Treat Computing Resources like Commodities Compute cycles, data storage, instruments Human communication environments No Central Control; No Trust University at Buffalo The State University of New York Center for Computational Research CCR Factors Enabling the Grid Internet is Infrastructure Increased network bandwidth and advanced services Advances in Storage Capacity Terabyte costs less than $5,000 Internet-Aware Instruments Increased Availability of Compute Resources Clusters, supercomputers, storage, visualization devices Advances in Application Concepts Computational science: simulation and modeling Collaborative environments large and varied teams Grids Today Moving towards production; Focus on middleware University at Buffalo The State University of New York Center for Computational Research CCR Computational Grids & Electric Power Grids Similarities/Goals of CG and EPG Ubiquitous Consumer is comfortable with lack of knowledge of details Differences Between CG and EPG Wider spectrum of performance & services Access governed by more complicated issues Security Performance Socio-political factors University at Buffalo The State University of New York Center for Computational Research CCR Growth of Data and Load Courtesy of vs. Moore’s Law Rick Stevens Metabolic Pathways Pharmacogenomics Human Genome Combinatorial Chemistry Computational Load ESTs Genome Data Moore’s Law 1990 2000 University at Buffalo The State University of New York 2010 Center for Computational Research CCR A Short History of the Grid Grand Challenge Problems (1980s) NSF and DOE initiatives “Science is a team sport” Initiate multi-resource projects involving computation, instruments, visualization, data Evolution of Related Communities Parallel computation Address resource limitations Networking Gigabit testbed program CASA Gigabit Testbed (1990s) Investigate potential testbed network architectures Explore usefulness for end-users University at Buffalo The State University of New York Center for Computational Research CCR The Globus Project (Ian Foster and Carl Kesselman) Globus model focuses The Grid as a Layered Set of Services on providing key Applications Grid services Resource access and management Grid FTP Information Service Security services Authentication Authorization Policy Delegation Network reservation, monitoring, control High-level Services and Tools GlobusView DUROC MPI MPI-IO CC++ Testbed Status Nimrod/G globusrun Core Services Nexus Metacomputing Directory Service Gloperf Condor LSF Heartbeat Monitor Local Services MPI Easy GRAM Globus Security Interface NQE University at Buffalo The State University of New York AIX GASS TCP UDP Irix Solaris Center for Computational Research CCR NSF Extensible TeraGrid Facility Caltech: Data collection analysis 0.4 TF IA-64 IA32 Datawulf 80 TB Storage Sun IA64 ANL: Visualization LEGEND Cluster Visualization Cluster Storage Server Shared Memory IA32 IA64 IA32 Disk Storage Backplane Router 1.25 TF IA-64 96 Viz nodes 20 TB Storage IA32 Extensible Backplane Network LA Hub 30 Gb/s Chicago Hub 40 Gb/s 30 Gb/s 30 Gb/s 4 TF IA-64 DB2, Oracle Servers 500 TB Disk Storage 6 PB Tape Storage 1.1 TF Power4 IA64 10 TF IA-64 128 large memory nodes 230 TB Disk Storage GPFS and data mining Sun 30 Gb/s 30 Gb/s Figure courtesy of Rob Pennington, NCSA EV7 6 TF EV68 71 TB Storage 0.3 TF EV7 shared-memory 150 TB Storage Server IA64 EV68 Pwr4 SDSC: Data Intensive NCSA: Compute Intensive University at Buffalo The State University of New York Sun PSC: Compute Intensive Center for Computational Research CCR Critical Resources: WNY Computational & Data Grids Computational & Data Resources (CCR) 10TF Computing & 78TB Storage Instruments (HWI, RPCI) Microarray; Diffractometer; NMR High-Throughput Crystallization Laboratory Data Generation (HWI) 7TB per year Databases (UB-N, UB-S, BGH, CoE) SnB; Multiple Sclerosis; Protein/Genomic University at Buffalo The State University of New York Center for Computational Research CCR Network Connections 1000 Mbps 100 Mbps FDDI 100 Mbps 1.54 Mbps (T1) - RPCI OC-3 - I1 Medical/Dental 44.7 Mbps (T3) - BCOEB 155 Mbps (OC-3) I2 1.54 Mbps (T1) - HWI NYSERNet NYSERNet 350 Main St 350 Main St BCOEB Abilene 622 Mbps (OC-12) University at Buffalo The State University of New York Commercial Center for Computational Research CCR Network Connections (New) 1000 Mbps 100 Mbps 100 Mbps FDDI Medical/Dental RIA RIA UB controlled UB controlled meet-me loc. meet-me loc. OC-3 - I1 1000 Mbps 1000 Mbps BCOEB 1000 Mbps Abilene University at Buffalo The State University of New York NYSERNet NYSERNet 350 Main St 350 Main St 622 Mbps (OC-12) Commercial Center for Computational Research CCR Advanced CCR Data Center (ACDC) Computational Grid Overview Nash: Compute Cluster 75 Dual Processor 1 GHz Pentium III RedHat Linux 7.3 1.8 TB Scratch Space Joplin: Compute Cluster 300 Dual Processor 2.4 GHz Intel Xeon RedHat Linux 7.3 38.7 TB Scratch Space Mama: Compute Cluster 9 Dual Processor 1 GHz Pentium III RedHat Linux 7.3 315 GB Scratch Space ACDC: Grid Portal 4 Processor Dell 6650 1.6 GHz Intel Xeon RedHat Linux 9.0 66 GB Scratch Space Young: Compute Cluster 16 Dual Sun Blades 47 Sun Ultra5 Solaris 8 770 GB Scratch Space SGI Origin 3800 64 - 400 MHz IP35 IRIX 6.5.14m 360 GB Scratch Space Fogerty: Condor Flock Master 1 Dual Processor 250 MHz IP30 IRIX 6.5 Expanding RedHat, IRIX, Solaris, WINNT, etc CCR T1 Connection Computer Science & Engineering 25 Single Processor Sun Ultra5s Crosby: Compute Cluster 19 IRIX, RedHat, & WINNT Processors School of Dental Medicine Hauptman-Woodward Institute 9 Single Processor Dell P4 Desktops 13 Various SGI IRIX Processors Note: Network connections are 100 Mbps unless otherwise noted. University at Buffalo The State University of New York Center for Computational Research CCR ACDC Data Grid Overview 182 GB Storage Joplin: Compute Cluster 300 Dual Processor 2.4 GHz Intel Xeon RedHat Linux 7.3 38.7 TB Scratch Space Nash: Compute Cluster 75 Dual Processor 1 GHz Pentium III RedHat Linux 7.3 1.8 TB Scratch Space 100 GB Storage 70 GB Storage Mama: Compute Cluster 9 Dual Processor 1 GHz Pentium III RedHat Linux 7.3 315 GB Scratch Space ACDC: Grid Portal 56 GB Storage Young: Compute Cluster 4 Processor Dell 6650 1.6 GHz Intel Xeon RedHat Linux 9.0 66 GB Scratch Space 16 Dual Sun Blades 47 Sun Ultra5 Solaris 8 770 GB Scratch Space CSE Multi-Store 2 TB 100 GB Storage Crosby: Compute Cluster 136 GB Storage SGI Origin 3800 64 - 400 MHz IP35 IRIX 6.5.14m 360 GB Scratch Space Network Attached Storage 480 GB Storage Area Network 75 TB Note: Network connections are 100 Mbps unless otherwise noted. University at Buffalo The State University of New York Center for Computational Research CCR WNY Grid Highlights Heterogeneous Computational & Data Grid Currently in Beta with Shake-and-Bake WNY Release in March Bottom-Up General Purpose Implemenation Ease-of-Use User Tools Administrative Tools Back-End Intelligence Backfill Operations Prediction and Analysis of Resources to Run Jobs (Compute Nodes + Requisite Data) University at Buffalo The State University of New York Center for Computational Research CCR Advanced CCR Data Center (ACDC) Computational Grid Overview Nash: Compute Cluster 75 Dual Processor 1 GHz Pentium III RedHat Linux 7.3 1.8 TB Scratch Space Joplin: Compute Cluster 300 Dual Processor 2.4 GHz Intel Xeon RedHat Linux 7.3 38.7 TB Scratch Space Mama: Compute Cluster 9 Dual Processor 1 GHz Pentium III RedHat Linux 7.3 315 GB Scratch Space ACDC: Grid Portal 4 Processor Dell 6650 1.6 GHz Intel Xeon RedHat Linux 9.0 66 GB Scratch Space Young: Compute Cluster 16 Dual Sun Blades 47 Sun Ultra5 Solaris 8 770 GB Scratch Space SGI Origin 3800 64 - 400 MHz IP35 IRIX 6.5.14m 360 GB Scratch Space Fogerty: Condor Flock Master 1 Dual Processor 250 MHz IP30 IRIX 6.5 Expanding RedHat, IRIX, Solaris, WINNT, etc CCR T1 Connection Computer Science & Engineering 25 Single Processor Sun Ultra5s Crosby: Compute Cluster 19 IRIX, RedHat, & WINNT Processors School of Dental Medicine Hauptman-Woodward Institute 9 Single Processor Dell P4 Desktops 13 Various SGI IRIX Processors Note: Network connections are 100 Mbps unless otherwise noted. University at Buffalo The State University of New York Center for Computational Research CCR Data Grid Motivation & Goal Motivation: Large data collections are emerging as important community resources. Data Grids inherently complements Computational Grids, which manipulate data. A data grid denotes a large network of distributed storage resources such as archival systems, caches, and databases, which are linked logically to create a sense of global persistence. Goal: To design and implement transparent management of data distributed across heterogeneous resources, such that the data is accessible via a uniform web interface. University at Buffalo The State University of New York Center for Computational Research CCR Data Grid Summary 544 GB Storage Located on 6 heterogeneous ACDC-Grid resources 480 GB Storage Located on 1 dual processor Dell PowerVault server 75,000 GB Storage (10/03) Served by 4 – 16 processor HP GS1280 servers 2,000 GB Storage Network Attached Storage 480 GB 56 GB Storage 70 GB Storage 100 GB Storage 100 GB Storage CSE Multi-Store 2 TB 136 GB Storage 182 GB Storage Served by Sun Ultra-60 servers 78,024 GB Total Data Grid Storage available and accessible from the ACDC-Grid Portal University at Buffalo The State University of New York Storage Area Network 75 TB Center for Computational Research CCR Grid-Based SnB Objectives Install Grid-Enabled Version of SnB Job Submission and Monitoring over Internet SnB Output Stored in Database SnB Output Mined through Internet-Based Integrated Querying Tool Serve as Template for Chem-Grid & Bio-Grid Experience with Globus and Related Tools University at Buffalo The State University of New York Center for Computational Research CCR Grid Enabled SnB Problem Statement Use all available resources in the ACDC-Grid for determining a single molecular structure. Grid Enabling Criteria All heterogeneous resources in the ACDC-Grid are capable of executing the SnB application. All job results obtained from the ACDC-Grid resources are stored in a corresponding molecular structure database. There are three modes of operation: Continue submitting SnB application jobs until the grid-enabled SnB application determines a solution has been found, or “X” number of trials have been evaluated, or indefinitely (grid job owner determines when a solution has been found). University at Buffalo The State University of New York Center for Computational Research CCR Grid Services and Applications Applications Shake-and-Bake ACDC-Grid Computational Resources High-level Services and Tools Globus Toolkit MPI Apache MPI-IO C, C++, Fortran, PHP Oracle MySQL NWS globusrun Core Services Metacomputing Directory Service Condor LSF Globus Security Interface Local Services Stork MPI PBS Maui Scheduler TCP UDP GRAM GASS RedHat Linux WINNT Irix ACDC-Grid Data Resources Solaris Adapted from Ian Foster and Carl Kesselman University at Buffalo The State University of New York Center for Computational Research CCR Notes Apache – web portal server PHP - used by apache server for dynamic web portal pages MDS – traditional to use MDS with LDAP but we use MDS with MYSql grid portal database to keep information of available resources (we poll every 15 mins) GRAM – Globus Resource Allocation Manager – API for requesting comptuational jobs GASS – Global Access to Secondary Storage – API for accessing files stored on various platforms Stork – Condor module for transporting job files within a flock CCR University at Buffalo The State University of New York Center for Computational Research Grid Enabled SnB Required Layered Grid Services Grid-enabled Application Layer Shake – and – Bake application Apache web server MySQL database High-level Service Layer Globus, NWS, PHP, Fortran, and C Core Service Layer Metacomputing Directory Service, Globus Security Interface, GRAM, GASS Local Service Layer Condor, MPI, PBS, Maui, WINNT, IRIX, Solaris, RedHat Linux University at Buffalo The State University of New York Center for Computational Research CCR Required Grid Services Application Layer Shake-and-Bake Apache web server MySQL database High-level Services Grid Implementation as a Layered Set of Services Applications High-level Services and Tools GlobusView DUROC MPI MPI-IO CC++ Testbed Status Nimrod/G globusrun Globus, PHP, Fortran, C Core Services Metacomputing Directory Service, Globus Security Interface, GRAM, GASS Core Services Nexus Metacomputing Directory Service Gloperf Local Services Condor, MPI, PBS, Maui, WINNT, IRIX, Solaris, RedHat Linux Condor LSF University at Buffalo The State University of New York GRAM Heartbeat Monitor Local Services MPI Easy Globus Security Interface NQE AIX GASS TCP UDP Irix Solaris Center for Computational Research CCR Grid Enabled SnB Execution User defines Grid-enabled SnB job using Grid Portal or SnB supplies location of data files from Data Grid supplies SnB mode of operation Grid Portal assembles required SnB data and supporting files, execution scripts, database tables. determines available ACDC-Grid resources. ACDC-Grid job management includes: automatic determination of appropriate execution times, number of trials, and number/location of processors, logging/status of concurrently executing resource jobs, & automatic incorporation of SnB trial results into the molecular structure database. University at Buffalo The State University of New York Center for Computational Research CCR ACDC-Grid Portal University at Buffalo The State University of New York Center for Computational Research CCR ACDC-Grid Portal Login Grid Portal login screen University at Buffalo The State University of New York Center for Computational Research CCR Data Grid Capabilities Browser view of “mlgreen” user files stored in the Data Grid University at Buffalo The State University of New York Center for Computational Research CCR Data Grid Capabilities Browser view of “miller” group files published by user “rappleye” University at Buffalo The State University of New York Center for Computational Research CCR Data Grid Capabilities Browser view of “public” user files published by user “miller” University at Buffalo The State University of New York Center for Computational Research CCR Data Grid Capabilities University at Buffalo The State University of New York Center for Computational Research CCR Data Grid Capabilities University at Buffalo The State University of New York Center for Computational Research CCR Grid Portal Job Status Grid-enabled jobs can be monitored using the Grid Portal web interface dynamically. Charts are based on: total CPU hours, or total jobs, or total runtime. Usage data for: running jobs, or queued jobs. Individual or all resources. Grouped by: group, or user, or queue. University at Buffalo The State University of New York Center for Computational Research CCR Grid Portal Job Status University at Buffalo The State University of New York Center for Computational Research CCR ACDC-Grid Portal Condor Flock CondorView integrated into ACDC-Grid Portal University at Buffalo The State University of New York Center for Computational Research CCR ACDC-Grid Portal User Management Administrator based University at Buffalo The State University of New York user based Center for Computational Research CCR ACDC-Grid Portal Resource Management Administrator grants a user access to ACDC-Grid resources, software, and web pages. University at Buffalo The State University of New York Center for Computational Research CCR ACDC-Grid Administration University at Buffalo The State University of New York Center for Computational Research CCR ACDC-Grid Administration University at Buffalo The State University of New York Center for Computational Research CCR Grid Enabled Data Mining Problem Statement Use all available resources in the ACDC-Grid for executing a data mining genetic algorithm optimization of SnB parameters for molecular structures having the same space group. Grid Enabling Criteria All heterogeneous resources in the ACDC-Grid are capable of executing the SnB application. All job results obtained from the ACDC-Grid resources are stored in a corresponding molecular structure databases. University at Buffalo The State University of New York Center for Computational Research CCR Grid Enabled Data Mining There are two modes of operation and two sets of stopping criteria: Data mining jobs can be submitted in a dedicated mode (time critical), where jobs are queued on ACDC-Grid resources, or in a back fill mode (non-time critical), where jobs are submitted to ACDC-Grid resource that have unused cycles available. There are two sets of stopping criteria: Continue submitting SnB data mining application jobs until the grid-enabled SnB application determines optimal parameters have been found, or indefinitely (grid job owner determines when optimal parameters have been found). University at Buffalo The State University of New York Center for Computational Research CCR Grid Enabled Data Mining ACDC-Grid Data Grid Data Mining Criteria ACDC-Grid Computational Resources Grid Portal Workflow Job Manager University at Buffalo The State University of New York Molecular Structure Database Center for Computational Research CCR SnB Molecular Structure Database Molecular Structure Database University at Buffalo The State University of New York Center for Computational Research CCR Grid Enabled Data Mining Execution Scenario User defines a Grid-enabled data mining SnB job using the Grid Portal web interface supplying: designate which molecular structures parameter sets to optimize, data file metadata, and Grid-enabled SnB mode of operation dedicated or back fill mode, and Grid-enabled SnB stopping criteria. The Grid Portal assembles the required SnB application data and supporting files, execution scripts, database tables, and submits jobs for parameter optimization based on the current database statistics. ACDC-Grid job management includes: automatic determination of appropriate execution times, number of trials, and number of processors for each available resource, logging and status of all concurrently executing resource jobs, automatic incorporation of SnB trial results into the molecular structure database, and post processing of updated database for subsequent job submissions. University at Buffalo The State University of New York Center for Computational Research CCR ACDC Data Grid Database Schema ACDC-Grid Data Grid University at Buffalo The State University of New York Center for Computational Research CCR Grid Portal Job Status ACDC-Grid Computational Resources University at Buffalo The State University of New York Center for Computational Research CCR Data Grid Overview Enable the transparent migration of data between various resources while preserving uniform access for the user. Maintain metadata information about each file and its location in a global database table. Currently using MySQL tables. Periodically migrate files between machines for more optimal usage of resources. University at Buffalo The State University of New York Center for Computational Research CCR Data Grid Functionality Implement basic file management functions accessible via a platform-independent web interface. Features include: User-friendly menus/ interface. File Upload/ Download to and from the Data Grid Portal. Simple web-based file editor. Efficient search utility. Logical display of files for a given user in three divisions (user/ group/ public). Hierarchical vs. List-based 3 divisions: (user/ group/ public) Sorting capability based on file metadata, i.e. filename, size, modification time, etc. University at Buffalo The State University of New York Center for Computational Research CCR Data Grid Functionality Support multiple access to files in the data grid. Implement basic Locking and Synchronization primitives for version control. Integrate security into the data grid. Implement basic authentication and authorization of users. Decide and enforce policies for data access and publishing. University at Buffalo The State University of New York Center for Computational Research CCR Data Grid File Migration Migration Algorithm File migration depends upon a number of factors: User access time Network capacity at time of migration User profile User disk quotas on various resources University at Buffalo The State University of New York Center for Computational Research CCR Data Grid File Migration We need to mine log files in order to determine How much data to migrate in one migration cycle? What is an appropriate migration cycle length? What is a user’s access pattern for files? What is the overall access pattern for particular files? University at Buffalo The State University of New York Center for Computational Research CCR Data Grid File Aging Global File Aging vs. Local File Aging User aging attribute Indicative of a user’s access across their own files. Attribute of a user’s profile. During migration time, this attribute will determine which user’s files should be migrated off of the grid portal onto a remote resource. Function of (file age, global file aging, resource usage) University at Buffalo The State University of New York Center for Computational Research CCR Data Grid File Aging File aging attribute Indicative of overall access to/migration activity of a particular file. Attribute in file_management table. Scale: 0 to 1 probability of whether or not to migrate file. File_aging_local_param initialized to 1. During migration time after a user has been chosen, this attribute will help determine which files of the user to migrate. i.e. Migrate a maximum of the top 5% of user’s files in any one cycle. University at Buffalo The State University of New York Center for Computational Research CCR Data Grid File Aging For a given user, the average of the file_aging_local_param attributes of all files should be close to 1. Operating tolerance before action is taken is within the range of 0.9 – 1.1. In this way, the user file_aging_global_param can be a function of this average. If the average file_aging_local_param attribute > 1.1, then files of the user are being held to long before being migrated. The file_aging_global_param value should be decreased. If the average file_aging_local_param attribute < 0.9, then files of the user are being accessed at a higher frequency than the file_aging_global_param value. The file_aging_global_param value should be increased. University at Buffalo The State University of New York Center for Computational Research CCR Data Grid Resource Info University at Buffalo The State University of New York Center for Computational Research CCR Data Grid Resource Info Both platforms have reduced bandwidth available for additional transfers University at Buffalo The State University of New York Center for Computational Research CCR Date Grid File Management Table University at Buffalo The State University of New York Center for Computational Research CCR Data Grid File Age File age, access time, and resource id denote: the amount of time since a file was accessed, when the file was accessed, and where the file currently resides respectively. University at Buffalo The State University of New York Center for Computational Research CCR Data Grid Summary Network Attached Storage 480 GB 56 GB Storage 70 GB Storage 100 GB Storage 100 GB Storage CSE Multi-store 2 TB 136 GB Storage 182 GB Storage Storage Area Network 75 TB The Data Grid algorithms are continually evolving to minimize network traffic and maximize disk space utilization on a per user basis by data mining user usage and disk space requirements. University at Buffalo The State University of New York Center for Computational Research CCR ACDC-Grid Development/Maintenance Development Requirements 7 – Person months for Grid Services Coordinator Minimum Maintenance Requirements Including Grid and Database conceptual design and implementation 5 – Person months for Grid Services Programmer Web portal programming 5 – Person months for System Administrator Globus, NWS, MDS, etc. installations 3 – Person months for Database Administrator Grid Portal Database implementation University at Buffalo The State University of New York 1 – Grid Services Coordinator 100% level of effort 1 – Grid Services Programmer 100% level of effort 1 – System Administrator 50% level of effort 1 – Database Administrator 10% level of effort Center for Computational Research CCR Future ACDC Applications Princeton Ocean Model (POM) Genetic Algorithms for Earthquake Structural Design Bioinformatics Computational Chemistry (Q-Chem) Environmental Engineering Applications University at Buffalo The State University of New York Center for Computational Research CCR