Download The Worldwide LHC computing GRID

GRID and ROC-LA
Javier Magnin
Brazilian Center for Research in Physics
&
ROC-LA
Workshop in Physics and Technology
at CERN - Bogota, Oct. 20-22, 2010
Outline
• The concept of GRID
• Different GRID “flavors”
• The Worldwide LHC Computing GRID: WLCG
• The ROC-LA
• How to become a site member of ROC-LA
• The typical structure of a site under ROC-LA
• ROC-LA outreach
• Conclusions
The concept of GRID
Web is a service for sharing information over
the Internet, Grid is a service for sharing
computer power and data storage capacity over
the Internet.
The concept of GRID
Web is a service for sharing information over
the Internet, Grid is a service for sharing
computer power and data storage capacity over
the Internet.
The five big ideas
•
•
•
•
•
Resource Sharing
Secure Access
Resource Use
The Death of Distances
Open Standards
The concept of GRID
Cluster
2
Cluster
3
Cluster
1
Isolated resources
Cluster
5
Cluster
4
The concept of GRID
Cluster
2
• Resource sharing  direct access to
Cluster
Cluster
remote computers, software and data 3
1
• Death of distances  high speed
connections between computers
• Open standards  applications made
to run on one resource will run on all
other
Cluster
5
Cluster
4
The concept of GRID
Cluster
2
Security:
Digital certificates
to identify clusters
in the GRID
Cluster
3
Cluster
1
Cluster
5
Cluster
4
The concept of GRID
VO 2
Cluster
VO 1
2VO 4
VO 2
Cluster
VO34
VO 1
Cluster
VO 31
• Secure access  Access policy, authentication
and authorization
• Resource use  you should be able to calculate
the optimal allocation of resources
VO 4
Cluster
VO 25
VO 1
Cluster
VO 43
The concept of GRID
VO 2
Cluster
VO
1
2VO 4
VO 1
Cluster
VO 31
VO 4
Cluster
VO 25
VO (Virtual Organization):
Is a group of people sharing
common interests.
• Individuals
are
VO 2
Cluster
identified by
VO34digital
certificates
• Individuals pertaining
to a given VO have
granted access to
resources supporting the
given VO
VO 1
Cluster
VO 43
The concept of GRID
VO 2
Cluster
VO 1
2VO 4
VO 1
Cluster
VO 31
Monitoring services:
• ROC/NGI´s in the
European GRID
VO 4
Cluster
• Something else
in the
VO 25
American GRID
VO 2
Cluster
VO34
VO 1
Cluster
VO 43
ROC
/NGI
The concept of GRID
COD
The concept of GRID
In summary, the GRID is:
• Hardware  a large amount of computers across
the world
• Network  to interconnect the hardware
• Middleware  the software which brings
together all the hardware across the internet to
setup the GRID infrastructure
• A set of usage rules  Digital Certificates, VO’s,
monitoring structures and goals, etc.
which sound like a “Virtual Super Computer”
Different GRID “flavors”
There exists several GRID initiatives around the world
• Europe
• EGI
• Nordugrid
• WLCG
• etc.
• National projects
• Open Science Grid (OSG) - USA
• TeraGrid - USA
• CNGrid - China
• Garuda - India
• National Grid Service – United Kingdom
• INFN Grid – Italy
• etc
Different GRID “flavors”
They mostly differ in the middleware
• gLite
• Unicore
• ARC
• dCache.org
• Globus Toolkit
• etc.
Different GRID “flavors”
They mostly differ in the middleware
• gLite
• Unicore
• ARC
• dCache.org
• Globus Toolkit
• etc.
EMI (European
Middleware Initiative):
• Common middleware
• Three years project
• Started in May 2010
• 26 partners across
Europe
Different GRID “flavors”
They mostly differ in the middleware
• gLite
• Unicore
• ARC
• dCache.org
• Globus Toolkit
• etc.
Conversations with OSG
EMI (European
Middleware Initiative):
• Common middleware
• Three years project
• Started in May 2010
• 26 partners across
Europe
Different GRID “flavors”
EMI
• Objectives: consolidate a middleware distribution
simplifying services and components, evolve functionalities
following the requirements of the community
• Improve usability
• Improve security
• Standardization
• interoperability
• service integration
• Integration with new technologies
• messaging - for monitoring, accounting, service
management, etc.
• virtualization – usage of virtual machines
The Worldwide LHC computing GRID: WLCG
WLCG is a collaboration linking grid infrastructure and
computer centers worldwide
• More than 130 computing centers in 34 countries
• 210190 CPU cores (Oct. 2010) available to process,
and analyze data produced at LHC
• Large storage capacity (some Pb)
• Equally available to all partners, regardless of
their physical location
• Individual institutions contribute to WLGC with
hardware and human resources
The Worldwide LHC computing GRID: WLCG
WLCG consists of three mains layers or Tiers, made up
of computer centers:
• Tier-0: One site (the CERN computing center). All
data from LHC passes through this central hub.
Contributes with less than 20% of the total
computing power
• Tier-1: 11 large computer centers with large
storage capacity and round-the-clock support. Used
for processing of raw data, data analysis and data
storage.
• Tier-2: about 160 smaller computing centers with
enough computing power and storage for data
analysis and MC generation
The Worldwide LHC computing GRID: WLCG
WLCG consists of three mains layers or Tiers, made up
of computer centers:
The Worldwide LHC computing GRID: WLCG
The ROC-LA
Created as a joint effort by CBPF, ICN-UNAM and
UNIANDES
• It was a response to the
• end of the ROC-CERN as a catch-all ROC
• absence of a ROC infrastructure in LA
• ROC-LA operations started in Sept. 30, 2009
• CBPF, ICN-UNAM and UNIANDES were under
ROC-CERN until Sept. 30, 2009
• ROC-LA technical team trained at CERN during 6
months in 2009 – (financial support by WLCG and
host institutions)
• Fully founded by the host institutions
The ROC-LA
Structure of ROC-LA
• Management Board:
• Dr. Carlos Avila (UNIANDES)
• Dr. Javier Magnin (CBPF)
• Dr Lukas Nelen (ICN-UNAM)
• Technical Board:
• Eng. Luciano Diaz (ICN-UNAM)
• Eng. Andrés Olguin (UNIANDES)
• Eng. Renato Santana (CBPF)
• Technical support team provided by the host
Institutions
The ROC-LA
Objectives of ROC-LA
• To provide a GRID infrastructure for LA (HEP
and non-HEP sites)
• To give support to the GRID infrastructure by
certifying, testing and monitoring sites in Latin
America
Services provided by ROC-LA
• Information  through the central GOCDB, siteBDII, APEL
• Support  GGUS tickets
• Monitoring  Nagios, SAM-tests, GSTAT
The ROC-LA
Responsibilities of ROC-LA
• Follow up and dispatch tickets addressed to ROC-LA
• Review (on a daily basis) the backlog of tickets and
take appropriated actions
• Review (on a daily basis) alarms and take
appropriated actions
• Certify new sites and follow up new sites in
certification process
• Provide a support mailing list
The ROC-LA
CBPF
312 cores
UNIANDES
216 cores
SAMPA
120 cores
ICN-UNAM
58 cores
UTFSM
44 cores
The ROC-LA
CBPF
312 cores
ROC-LA in numbers:
UNIANDES
•5
sites in production (WLCG)
216 cores
• 750 cores
• 1 site in final tests (LNCCCOMCIDIS)
ICN-UNAM
• 1 site in58certification
process
cores
SAMPA
(UFRJ-NACAD)
120 cores
• SupportUTFSM
to the VO’s: LHCb,
44 cores
CMS, ALICE,
ATLAS, biomed,
fussion, Auger, oper.vo.eu-eela.eu,
prod.vo.eu-eela.eu
How to become a site member of ROC-LA
First
•Send e-mail to [email protected] asking to
become a site member describing your hardware and
network infrastructure and human resources
• Describe also your needs in terms of VO’s your site
will support
• Sign up the Letter of Agreement
then
• The ROC-LA technical board will start the process
of certification and setup of your site
The typical structure of a site under ROC-LA
It depends on your needs/resources but at least you
will have
• A Computing Element (CE)  this is the portal of
your site
• A Storage Element (SE)  for data storage –
depending on your needs (VO depending)
• A number of Working Nodes  for processing –
typically a few (as much as you can!) servers with
multi-core processors
• RAM memory  for the WN’s – depending on the
data you will have to process (VO depending)
• A System Manager (of course!)
• Internet connection (of course!)
ROC-LA outreach
Home page:
http://www.roc-la.org
I ROC-LA Workshop
CERN, Oct. 6 to 8, 2010
http://indico.cern.ch/conferenceDisplay.py?ovw=True&confId=108833
ROC-LA outreach
Home page:
http://www.roc-la.org
I ROC-LA Workshop
CERN, Oct. 6 to 8, 2010
Annual workshop open
to all ROC-LA users
http://indico.cern.ch/conferenceDisplay.py?ovw=True&confId=108833
Conclusions
• GRID in LA shows a slow but consistent growth
• GRID activities in LA are mainly driven by the HEP
community (which is the largest user in the region)
• ROC-LA commitment is to provide a sustainable
infrastructure for the GRID community/users in LA
• monitoring services
• technical support
• certification of sites/users
• The ROC-LA born to provide support to the HEP
community, but is open to the non-HEP users too
• ROC-LA is financially supported by its hosts
institution