Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
F2F JRA4 Meeting CERN 16 June 2004 www.eu-egee.org Network Requirements Javier Orellana EGEE is a project funded by the European Union under contract IST-2003-508833 Outline • Components and Requirements • NA4 Requirements • Other Requirements: HEP Biomedical Generic • Requirements from mw Clients <F2F JRA4 Meeting, CERN> <16 June 2004> - 2 Components and Requirements Consumer (User Application) Requirements End User Requirements Operations Grid Middleware Middleware Requirements Computer Element NE-A NE-B Network Element Storage Element NE-X Network (GEANT+NRENs) <F2F JRA4 Meeting, CERN> <16 June 2004> - 3 NA4 : User Interface Requirements • R1.2 User Interface configuration. The user interface should be easy to configure. A pointer to an information service server should be sufficient for the user interface to get all the configuration information it requires. A default information service is expected. • R1.3 Application Programming Interface. The application programming interface to middleware services should be available in C++ and Java. The installation should be lightweight. It should cover all services accessible from the user interface host and the worker node: jobs submission, data manager, information system, etc. If needed, the network related information has to be included in the user interface <F2F JRA4 Meeting, CERN> <16 June 2004> - 4 NA4: Network Requirements • R6.1 Communications encryption It should be possible to encrypt data prior to communication between sites to prevent third party listening. • R6.2 Outbound connectivity Applications will need to access services external to the grid middleware. It should be possible to establish a communication between a grid node and non-grid hosts on any port. AWG UC#externalConnection • R6.3 Guaranteed bandwidth Some applications (e.g. interactive) have a need for a guaranteed network bandwidth. <F2F JRA4 Meeting, CERN> <16 June 2004> - 5 NA4 : Operation Requirements • R8.2 User control It should be possible to grant access to an authorized user and to revoke a user. HEPCAL UC#gridauth HEPCAL UC#gridrevoc • R8.6 VO management (creation, modification and deletion) VO management should allow the creation of user groups within a VO. Each user group should have the possibility to choose whether or not to share resources with the other VO groups. • ie, NREN allowing just 25% of bandwidth to Grid traffic. <F2F JRA4 Meeting, CERN> <16 June 2004> - 6 …Is that all ? • Maybe not. More requirements can be needed What about Network Performance Monitoring? (Nothing is included in the “Information Service” requirements) Are end users and/or operations interested in diagnostic tool? No mention about possible requirements from the mw clients to the network element Guaranteed network bandwidth?, maybe also other network resources like L2 paths or Premium IP. Is that important for the end users? Still key to talk to end users and experts !!!! <F2F JRA4 Meeting, CERN> <16 June 2004> - 7 HEP Case * • Context : Massive number of datasets globally distributed ie. BaBar currently ~1PetaByte Hundreds of Analysis Sites Need for managing global resources optimally The system needs to be managed and monitored globally end to end • Requirements: 100 GBytes extracted from 100 TeraByte data in less than 10 minutes => ~1Gbps bandwidth End to end Quality of Service In 3 years, extract 100TBytes from 100 PBytes => ~100Gbps bandwidth • Lamda flows between end points • Capacity Planning • Key to have Network Performance Monitoring to optimize network usage * Source: High-Performance Network Planning Workshop, US DoE <F2F JRA4 Meeting, CERN> <16 June 2004> - 8 Biomedical Case * • Outpacing number of researchers Highly effective AA solutions Policy-based control and sharing of Grid resources (including network resources) Clear need for VO, resource allocation policies and accounting mechanisms (including network resources) Heavy use of distributed large-scale relational database => end-to-end solutions for database integration and federation • Requirements: Databases ~TeraBytes and soon PetaBytes => ~1Gbps and ~100Gbps Accurate Database replicas with high integrity => highly reliable and accurate transport mechanism, enhancing current ones and QoS mechanisms Multicasting * Source: High-Performance Network Planning Workshop, US DoE <F2F JRA4 Meeting, CERN> <16 June 2004> - 9 In general, other Requirements * • Authenticated data streams for easier site access through firewalls • Robust access to large data files => multiple network paths with • • • • • • backups and redundancy Adequate bandwidth and bounded latency for analysis and visualization of remote datasets (Quality of Service guarantees) Other applications require ~500Mbps sustained bandwidth Co-scheduling of computer and network resources End-to-end Quality of Service Management Network caching (rapid access to intermediate locations) Monitoring and problems diagnostic * Source: High-Performance Network Planning Workshop, US DoE <F2F JRA4 Meeting, CERN> <16 June 2004> - 10 From mw Clients perspective * • Data Replication Manager: Data replicated, picket up from source S and delivered to D by time T Data replicated with deadline, need to be delivered before time Tf Data replicated from S to destinations D > 1, multicast? • Workload Manager: Workload Manager needs to ensure that remote data is accessible for the time window CPUs have been allocated In case remote data is unknown, then at least WM needs to guarantee “on average” access to the dataset • Remote Archive Shadowing or Mirroring (keep remote shadow copies of the same dataset) Keep datasets synchronised that change slowly Need for changing all at once • High Performance Computing Virtual Backplane: Two end point linked with a maximum latency boundary * Paul Mealor, Peter Kunszt and Peter Clarke comments about middleware clients <F2F JRA4 Meeting, CERN> <16 June 2004> - 11 Authorisation Auditing Grid Access Service Grid Middleware Grid Monitoring Accounting Data Management Site Gatekeeper Workload Management Monitoring Authentication Network Performance Interfaces with Middleware Architecture Bandwidth Allocation and Reservation <F2F JRA4 Meeting, CERN> <16 June 2004> - 12 Questions? <F2F JRA4 Meeting, CERN> <16 June 2004> - 13