Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Introduction to Globus PHENIX experiment uses Grid to transfer 270 TB of data to Japan During the polarized proton-proton run that ended in June at the Relativistic Heavy Ion Collider (RHIC) at Brookhaven, Grid tools were used by the PHENIX experiment to send recently acquired data to a regional computing centre for the experiment in Japan. Brookhaven National Laboratory, on Long Island, New York, is home to the RHIC/ATLAS Computing Facility (RCF/ACF), which is the main computing centre for experiments at RHIC and a Tier-1 computing centre for ATLAS. The PHENIX regional computing centre in Japan (CCJ) is at the RIKEN research centre on its Wako campus close to Tokyo. Daily rates of data transferred from the PHENIX experiment to the CCJ computing centre in Japan (blue), and the integrated data volume (red). Overall, 270 TB of data were transferred. Introduction The Globus Project provides middleware services for grid environments. There are four main components of Globus: The Grid Security Infrastructure (GSI) provides Authentication and authorization services using public key certificates and Kerberos authentication. The Globus Resource Management architecture provides: a language for specifying application requirements mechanisms for immediate and advance reservations of one or more computational components. interfaces for submitting jobs to remote machines. Introduction (cont.) The Globus Information Management architecture provides: a distributed scheme for publishing and retrieving information about resources in the wide area environment. A distributed collection of information servers is accessed by higher-level services that perform resource discovery, configuration and scheduling. The Globus Data Management architecture provides two fundamental components: A universal data transfer protocol for grid computing environments called GridFTP Replica Management infrastructure for managing multiple copies of shared data sets. The Globus Approach The Globus toolkit provides a range of basic Grid services These services are simple and orthogonal Security, information, fault detection, communication, resource management, ... Can be used independently, mix and match Programming model independent For each there are well-defined APIs Standards are used extensively E.g., LDAP, GSS-API, X.509, ... Grid Security Infrastructure Single-sign on, run anywhere [if authorized] Standards based (GSS, SSL, X.509) GSS-API Interface Identity/credential mapping at each resource Limited delegation of rights Integrated into wide variety of tools Globus Resource Management Secure shell, FTP, Storage Resource Broker Grid Security Infrastructure (cont.) Based on public key technology Standard X.509 certificate, same as certificates used for the Web Each user has: a Grid user id (called a Subject Name) /C=US/O=Globus/O=University of Southern California/OU=Information Sciences Institute/CN=Ann Chervenak a private key (like a password) a certificate signed by a Certificate Authority (CA) A “gridmap” file at each site specifies grid-id to local-id mapping Authentication Model Authentication is done on a “user” basis No communication of plaintext passwords Most sites will use conventional account mechanisms Single authentication step allows access to all grid resources You must have an account on a resource to use that resource Sites may use “generic” Grid accounts Not common, but Globus can deal with it Certificate Based Authentication User has a certificate, signed by a trusted “certificate authority” (CA) User’s private key is used to encode a challenge string Public key is used to decode the challenge Certificate contains user’s name and public key Globus project operates a CA If you can decode it, you know the user Treat your private key carefully!! Private key is stored in encrypted form User Proxies Minimize exposure of user’s private key A temporary credential for use by our computations We call this a user proxy certificate Allows process to act on behalf of user User-signed user proxy certificate stored in local file Proxy’s private key is not encrypted Rely on file system security, proxy certificate file must be readable only by the owner Delegation Remote creation of a user proxy Allows remote process to act on behalf of the user Avoids sending passwords or private keys across the network User Assignment of credentials to “user proxies” User Proxy Globus Credential GRAM Process GSI Process Kerberos Single sign-on via “grid-id” Site 2 Site 1 Ticket CREDENTIAL Authenticated interprocess communication Mutual user-resource authentication Process GRAM Process GSI GSSAPI: multiple Public Key low-level mechanisms Certificate Process Process Mapping to local ids Resource Management Globus Resource Allocation Manager (GRAM) Globus Arch. for Reservation and Allocation Uniform interface to resource management Co-allocation of compute resources Immediate and advance reservation of network and computers in prototype form Fault detection service Network measurement tools Code management and distribution infrastructure Resource Management Resource Specification Language (RSL) is used to communicate requirements The Globus Resource Allocation Manager (GRAM) API allows programs to be started on remote resources, despite local heterogeneity A layered architecture allows applicationspecific resource brokers and co-allocators to be defined in terms of GRAM services Resource Management Architecture RSL specialization Broker RSL Queries & Info Application Ground RSL Information Service Co-allocator Simple ground RSL Local resource managers GRAM GRAM GRAM LSF EASY-LL NQE Resource Specification Language Common notation for exchange of information between components RSL provides two types of information: Resource requirements: Machine type, number of nodes, memory, etc. Job configuration: Directory, executable, args, environment API provided for manipulating RSL RSL Syntax Elementary form: parenthesis clauses Operators Supported: <, <=, =, >=, > , != Some supported attributes: (attribute op value [ value … ] ) executable, arguments, environment, stdin, stdout, stderr, resourceManagerContact, resourceManagerName Unknown attributes are passed through May be handled by subsequent tools Constraints: “&” For example: & (count>=5) (count<=10) (max_time=240) (memory>=64) (executable=myprog) “Create 5-10 instances of myprog, each on a machine with at least 64 MB memory that is available to me for 4 hours” Multirequest: “+” A multirequest allows us to specify multiple resource needs, for example + (& (count=5)(memory>=64) (executable=p1)) (&(network=atm) (executable=p2)) Execute 5 instances of p1 on a machine with at least 64M of memory Execute p2 on a machine with an ATM connection Multirequests are central to co-allocation Co-allocation Simultaneous allocation of a resource set Handled via optimistic co-allocation based on free nodes or queue prediction In the future, advance reservations will also be supported A Co-allocation Multirequest +( & (resourceManagerContact= “flash.isi.edu:754:/C=US/…/CN=flash.isi.edu-fork”) (count=1) (label="subjob A") Different resource (executable= my_app1) managers ) Different ( & (resourceManagerContact= counts “sp139.sdsc.edu:8711:/C=US/…/CN=sp097.sdsc.edu-lsf") (count=2) (label="subjob B") Different executables (executable=my_app2) ) Job Submission Interfaces Globus Toolkit includes several command line programs for job submission globus-job-run: Interactive jobs globus-job-submit: Batch/offline jobs globusrun: Flexible scripting infrastructure Others are building better interfaces General purpose Condor-G, PBS, GRD, Hotpage, etc Application specific ECCE’, Cactus, Web portals Grid Information Services Publish and retrieve information about system elements Used for discovery, configuration, scheduling Distributed collection of information servers and index nodes LDAP V3 as wire protocol and API Examples of Useful Information Characteristics of a compute resource Characteristics of a network IP address, software available, system administrator, networks connected to, OS version, load Bandwidth and latency, protocols, logical topology Characteristics of the Globus infrastructure Hosts, resource managers Grid Information Service Provide access to static and dynamic information regarding system components A basis for configuration and adaptation in heterogeneous, dynamic environments Requirements and characteristics Uniform, flexible access to information Scalable, efficient access to dynamic data Access to multiple information sources Decentralized maintenance The Globus Toolkit Metacomputing Directory Service Store information in a distributed directory Directory can be updated by Directory stored in collection of LDAP servers Information providers and tools Applications (i.e., users) Backend tools which generate info on demand Information dynamically available to Tools Applications Directory Service Functions White Pages Yellow Pages Look up the IP number, amount of memory, etc., associated with a particular machine Find all the computers of a particular class or with a particular property Temporary inconsistencies are often considered okay In a distributed system, you often do not know the state of a resource until you actually use it Information is often used as “hints” Information itself can contain ttl, etc. MDS Approach Application Based on LDAP Globus specific schema Lightweight Directory Access Protocol v3 (LDAPv3) Standard data model Standard query protocol LDAP API GRIS … GIIS Host-centric representation Globus specific tools Middleware GRIS, GIIS Data discovery, publication,… SNMP … NWS NIS LDAP Grid Resource Information Service Server which runs on each resource Given the resource DNS name, you can find the GRIS server (well known port = 2135) Provides resource specific information Much of this information may be dynamic “White pages” lookup of resource information Load, process information, storage information, etc. GRIS gathers this information on demand Ex: How much memory does machine have? “Yellow pages” lookup of resource options Ex: Which queues on machine allow large jobs? Grid Index Information Service GIIS describes a class of servers Gathers information from multiple GRIS servers Each GIIS is optimized for particular queries Ex1: Which Alliance machines are >16 process SGIs? Ex2: Which Alliance storage servers have >100Mbps bandwidth to host X? Akin to web search engines Organization GIIS The Globus Toolkit ships with one GIIS Caches GRIS info with long update frequency Useful for queries across an organization that rely on relatively static information Referral Service Links together multiple GRIS and/or GIIS servers into a single LDAP namespace Referral servers contain no actual content Data Grid Services Access to remote data Transport services Uniform access to diverse, remote storage management systems Cache management Standards based (GSI, FTP protocol) Client API, Extensible server, support for third party transfer Replica Management Data Intensive Issues Include … High-speed, reliable access to remote data Automated discovery of “best” copy of data Manage replication to improve performance Co-schedule compute, storage, network Enforce access control on data The Globus Data Grid Two major components: 1. Data Transport and Access Common protocol Secure, efficient, flexible, extensible data movement Family of tools supporting this protocol 2. Replica Management Architecture Simple scheme for managing: multiple copies of files collections of files Motivation for a Common Data Access Protocol Existing distributed data storage systems DPSS, HPSS: focus on high-performance access, utilize parallel data transfer, striping DFS: focus on high-volume usage, dataset replication, local caching SRB: connects heterogeneous data collections, uniform client interface, metadata queries Problems Incompatible protocols Each require custom client Partitions available data sets and storage devices Each protocol has subset of desired functionality A Common, Secure, Efficient Data Access Protocol Common, extensible transfer protocol Decouple low-level data transfer mechanisms from the storage service Advantages: New, specialized storage systems are automatically compatible with existing systems Existing systems have richer data transfer functionality Interface to many storage systems HPSS, DPSS, file systems Plan for SRB integration Replica Management Maintain a mapping between logical names for files and collections and one or more physical locations Important for many applications Example: CERN HLT data Multiple petabytes of data per year Copy of everything at CERN (Tier 0) Subsets at national centers (Tier 1) Smaller regional centers (Tier 2) Individual researchers will have copies Globus Approach to Replica Management Identify replica cataloging and reliable replication as two fundamental services Layer on other Grid services: GSI, transport, information service Use LDAP as catalog format and protocol, for consistency Use as a building block for other tools Advantage These services can be used in a wide variety of situations Replica Manager Components Replica catalog definition Low-level replica catalog API LDAP object classes for representing logical-tophysical mappings in an LDAP catalog globus_replica_catalog library Manipulates replica catalog: add, delete, etc. High-level reliable replication API globus_replica_manager library Combines calls to file transfer operations and calls to low-level API functions: create, destroy, etc. Replica Catalog Structure: A Climate Modeling Example Replica Catalog Logical Collection Logical Collection C02 measurements 1998 C02 measurements 1999 Filename: Jan 1998 Filename: Feb 1998 … Location Location jupiter.isi.edu sprite.llnl.gov Filename: Mar 1998 Filename: Jun 1998 Filename: Oct 1998 Protocol: gsiftp UrlConstructor: gsiftp://jupiter.isi.edu/ nfs/v6/climate Filename: Jan 1998 … Filename: Dec 1998 Protocol: ftp UrlConstructor: ftp://sprite.llnl.gov/ pub/pcmdi Logical File Parent Logical File Logical File Jan 1998 Feb 1998 Size: 1468762 Replica Catalog Services as Building Blocks: Examples Combine with information service to build replica selection services E.g. “find best replica” using performance info from NWS and MDS Use of LDAP as common protocol for info and replica services makes this easier Combine with application managers to build data distribution services E.g., build new replicas in response to frequent accesses Relationship to Metadata Catalogs Metadata services describe data contents Must support a variety of metadata catalogs Have defined a simple set of object classes MCAT being one important example Others include LDAP catalogs, HDF Community metadata catalogs Agree on set of attributes Produce names needed by replica catalog: Logical collection name Logical file name A Model Architecture for Data Grids Metadata Catalog Attribute Specification Logical Collection and Logical File Name Application Selected Replica gsiftp commands Replica Catalog Multiple Locations Replica Selection Tape Library Replica Location 1 Disk Cache Replica Location 2 MDS Performance Information and Predictions Disk Cache Disk Array NWS Replica Location 3 Fault Detection: Globus Heartbeat Monitor Detect and report “failure” of component of a computation Limited by ability to distinguish between network partition and system failure Optionally used within Globus Toolkit to monitor status of system processes Can also be used to construct special fault monitors for applications Example: Netsolve Fault Detection (cont.) Goal: make low-level operations reliable No libraries for checkpoint and restart Can’t “checkpoint” a socket Only application knows how to checkpoint and restart Likewise, storage system must do logging Heartbeat Monitor Application Level Fault Handler ! Application Data Collector HBM Data Collector System Monitoring Tools Process and Host Heartbeat Process and Host Heartbeat Host 1 Process Status Inquiry HBM Monitor Register/ Unregister Host 2 Process Status Inquiry Monitored Process Monitored Process Monitored Process HBM Monitor Register/ Unregister Monitored Process Monitored Process Monitored Process