Download Authorisation Policy Towards a European Policy for

Authorisation Policy
Towards a European Policy for Resource Sharing
CONTOURS OF A TRANSPARANT
GRID ACCESS POLICY
Dr. Patrick Aerts
Director of the Netherlands
National Computing Facilities Foundation
(NCF)
1
eIRG meeting, Dublin, Ireland
April 15 2004
Overview
●
●
●
●
●
●
●
2
The goals
Grid concepts for Europe
The terms, what is involved
Examples, the scope of the problem
Some models presently in place
Complications
Further issues
eIRG meeting, Dublin, Ireland
April 15 2004
The Goals
●
●
●
Access to all resources for scientific computing in Europe
using the grid
A “fair share” for all users
Authentication by National Certification Authorities (CA)
using European formats
●
Authorisation: required, but not not too often
●
Accounting, using European formats
3
eIRG meeting, Dublin, Ireland
April 15 2004
The European grid concept
What are we heading for?
●
Concept 1: a grid of grids
●
Grids get formed by and from communities with a certain
common goal
●
Within these grids things are rather easy:
●
Trust, resource sharing, etc.
From these grids a larger (European) grid may arise
Concept 2: one large grid enabled bunch of resources
●
Owners allow their system(s) to be grid enabled and grid
aware
●
VO’s select their choice from available systems
●
VO’s seek funding for their project
●
●
4
eIRG meeting, Dublin, Ireland
April 15 2004
What is involved in Authorisation and
Accounting (1)
●
Authorisation:
● Who is allowed to access a facility
● Who provides the financial means (allocation)
• Allocating refers to the mechanism that
determines one’s rights to access an entity
●
5
Accounting: refers to the system that keeps track
of the resource units used by a user and the way
the associated cost are billed or properly placed
at the responsible authority (possibly the user).
eIRG meeting, Dublin, Ireland
April 15 2004
What is involved in Authorisation and
Accounting (2)
• Authorisation determines who has rights for
access,
• Allocation determines to what extent.
– Allocation mechanisms may be very different for the
entities within a grid and between grids.
• An authorised person/organisation may have its
own funds too
• Whose responsibility is the reliability
(trustworthyness) of users: at the authentication
level or at the authorisation level?
6
eIRG meeting, Dublin, Ireland
April 15 2004
How it works in The Netherlands
a Use Case (1)
●
●
●
●
●
Scientific projects are submitted to the National
Science Foundation (NWO)
A selection panel awards the project on scientific
merits, after peer review
NCF/NWO awards the necessary computing
resources for these projects, but also for other
qualified projects (also after peer review)
The national Computer Center, like SARA, then
creates an account and installs a budget
SARA bills NCF at the end of each month for the
resources provided in this way
Reality is not much more complicated
7
eIRG meeting, Dublin, Ireland
April 15 2004
But also:from biodiversity: bird migration case(2)
●
●
●
●
●
Subgroup in the biology faculty of the Amsterdam
University
University groups may request resources from NCF
without going through the NWO selection panel
In a simulation the migration of one bird is simulated
Ideally suited for a CPU cluster if one wants to
simulate a flock of birds over a longer time
A VO=bird migration is created and the faculty
members request a certificate from the Dutch CA
8
eIRG meeting, Dublin, Ireland
April 15 2004
Bird
migration
9
eIRG meeting, Dublin, Ireland
April 15 2004
How it (possibly) works in Germany
a Use Case
●
●
●
●
●
Scientific projects are submitted to the Fraunhofer
Gesellschaft
A selection panel awards the project on scientific merits
The Fraunhofer Gesellschaft makes computer resources
available through one of its computer centers like
Karlsruhe FZK
FZK then creates an account and a budget
and bills Fraunhofer at the end of the year for the services
provided
I assume this is how it works in Germany,
reality may be more complicated
But that is not relevant for this argument
10
eIRG meeting, Dublin, Ireland
April 15 2004
A Real Example
from astrophysics: colliding black holes
●
●
●
●
11
For this sort of calculations one needs a
supercomputer
EU Supercomputer project: DEISA
Let us assume that supercomputers are also
accessible through a grid infrastructure
A VO=black holes is created and the
participating scientists all request a certificate
from the German CA
eIRG meeting, Dublin, Ireland
April 15 2004
Colliding
black
holes
12
eIRG meeting, Dublin, Ireland
April 15 2004
Exchange of resources
●
●
●
●
●
13
Assume a bird migration calculation is submitted to
the grid (EGEE) and is send to a cluster of cpu’s at the
Karlsruhe computer center
Assume a colliding black hole simulation is submitted
to the grid (DEISA) and is send to the supercomputer
at SARA in Amsterdam
The control of where a job is executed on the grid
depends on the available resources at any time
For this to work SARA and FZK have to accept jobs
from the bird migration and black holes VO
What is the policy for resource providers in Europe to
accept/not accept VO’s?
eIRG meeting, Dublin, Ireland
April 15 2004
One would hope that ..
●
●
●
●
●
14
The scientists don’t have to worry where their job
migrates to
The scientists don’t have to worry that they can use
resources where their job runs best
The resource providers get the money that their
services cost
A European policy can be defined such that services
can be provided across national borders without
cash flow
In order to fulfill this hope, these issues have to be
subjects of the next chapters of the eIRG
eIRG meeting, Dublin, Ireland
April 15 2004
International Scientific Collaborations
●
15
The case is much simpler in High Energy Physics:
●
The Atlas collaborators have already requested
resources from their national funding agencies
●
The Atlas collaborators are organised in one and the
same Atlas Virtual Organisation VO
●
Budgets exist for this VO on all major sites with
computer resources in Europe
●
The fair sharing of those resources is done at the
collaboration level in a Memorandum of
Understanding with each of the collaborating
institutions
●
The collaborating institutions go through the normal
procedure for resource assignment at a national level
eIRG meeting, Dublin, Ireland
April 15 2004
Smaller National Scientific Projects
●
●
●
●
16
Bird migration simulation was a Dutch initiative
from a small university group
The same in Germany for the colliding black
holes study
Yet resources will be used more efficiently if the
computing would not respect national borders
To achieve this an authorisation policy has to be
put in place and nationally created VO’s must
be recognised Europe-wide, in some way...
eIRG meeting, Dublin, Ireland
April 15 2004
Delegation of Rights
A Push Model
●
In both cases the Authorisation involves
some form of cascading of rights:
●
●
Implemented in DataGrid (EDG) in a push
model
●
●
17
From NCF to SARA to VO to users
GridMapFiles at each site where these rights
per user and VO are described
Push model preferred if AuthZ is needed
globally and instantly (networking)
eIRG meeting, Dublin, Ireland
April 15 2004
Delegation of Rights
A Pull Model
●
●
●
18
It could be implemented the other way
● User to SARA to NCF to Project Description
Depending on the problem this is a better or
worse solution
Shibboleth uses a Pull Model for accessing web
resources
eIRG meeting, Dublin, Ireland
April 15 2004
Delegation of Rights
an Agent Model
●
●
●
●
●
19
Virtual Organisations VO’s are used to describe
large scientific organisations
Not all members have the same rights
Authorisation can be further cascaded
Developed in Virtual Organisation Management
Service (VOMS) in DataGrid and DataTag
Tested now in LHC Grid project LCG
eIRG meeting, Dublin, Ireland
April 15 2004
AuthZ Models
3
1
2
Push
AuthZ
Service
AuthZ
Service
1
Pull
Resource
Resource
1
4
Agent
AuthZ
Service
2
Resource
20
eIRG meeting, Dublin, Ireland
April 15 2004
2
3
3
Acceptable Use Policies
●
●
Use policies are defined at many levels:
institutional, national, scientific collaboration,
etc.
National legislation may also impose use
policies (security, privacy, etc)
●
Often different for different countries
●
Often different for different resources
●
These things seem solvable relatively easy
21
eIRG meeting, Dublin, Ireland
April 15 2004
Complications:
●
●
22
As long as the resources involved are rather homogeneous
and rather simple (like midsize clusters) things are easy
Once relatively expensive or specialised equipment gets
involved things get complicated:
●
One has to make a case for renewal and re-investments
●
Such cases involve accountability, show cases, success
stories
●
Regional/National pride may be involved, etc.
●
This is usually a co-responsibility of the authorisation
bodies
●
So, one does not hand over control over the special
systems in a grid for others to decide on its usage
eIRG meeting, Dublin, Ireland
April 15 2004
Complications (2)
●
●
●
●
23
The European grid is best build from the ansatz that there will
be many different ad hoc build grids.
In practice these grids are to a large extend coinciding with
the VO’s from other concepts.
The convergence from this situation to a situation where all
relevant systems are grid aware and grid enabled to allow
these different grids to glue together has to be guided by the
eIRG.
This means doing things the hard way. But it will keep Europe
ahead of developments elsewhere (Teragrid, US), because
one of the grid added values has to be sharing diversity
rather than sharing homogenity.
eIRG meeting, Dublin, Ireland
April 15 2004
Further complications
●
●
●
●
24
If users or VO’s were only to pay in real money:
Wouldn’t that be nice and easy.
But more often no real money is involved in allocation:
●
Either one gets resource units, implicitly meant to be
spend on a limited number of dedicated systems, or
●
If real money is involved, budgets may cover only a
systems running cost, not the integral cost (including reinvestments)
●
And even then the money is supposed to be spend on a
predetermined (number of) systems
In fact there is no (open) market, but a large number of
closed circuits
eIRG meeting, Dublin, Ireland
April 15 2004
Success stories
●
GEANT
●
Common basis for all AUP*s defined
●
●
●
Big user community: all NRENs in Europe
DataGrid
●
New AUP defined
Small user community: relatively easy!
●
*AUP= Acceptable Use Policiy
●
25
(however: see lecture d. Van dromme)
eIRG meeting, Dublin, Ireland
April 15 2004
Preferred Solution
●
●
●
●
●
26
A schema which encompasses all national AUPs without
making them all the same
A schema which separates the “common” basis from
differences and accounts for those
A schema by which AUPs apply for all resources: cpu’s,
storage, networking, etc.
eIRG should stimulate this development
For the time being: why not have authorisation bodies put
a percentage of the systems they govern into a basket for
European grid-related usage ( the 5% of Mary Spada,
Argonne/SDSC)
eIRG meeting, Dublin, Ireland
April 15 2004
Virtual Organisations
a possible model
●
●
●
●
●
27
In each EU country VOs can easily (through a web form)
be created for scientific projects
When computing resources are assigned to the project the
VO is validated
A validated VO is uploaded with the grid middleware to all
sites but is by default “unsupported”
Each site will “support” all VO’s from countries with which
there is an agreed policy for resource sharing (preferably
all EU countries)
Scheduling priorities among VO’s is still a local or national
policy
eIRG meeting, Dublin, Ireland
April 15 2004
Accounting
●
●
●
●
28
Not all services cost the same:
●
Supercomputers vs. clusters
●
What costs archiving or databases
●
Other non-computer networked facilities
Each resource provider may have an internationally
standardised and man+machine readable SLA per system
Accounting done per user, billing per VO (or user or AutZ
body) by resource provider
Less a problem for larger international scientific
collaborations
eIRG meeting, Dublin, Ireland
April 15 2004
Dutch Presidency
●
Policy for easy creation of VO’s
●
Policy for VO support by resource providers
●
Model for AuthZ
●
●
●
●
Common for CPU, storage and network resources
●
Support for accounting schemes
●
Respecting anonymity
Proposals for the %-basket
Possibly linking to the money follows man (M/F) principle of
European research councils
Common Acceptable Use Policy
29
eIRG meeting, Dublin, Ireland
April 15 2004