Download SC2000 Tutorial

Spitfire
A Relational DB Service for the Grid
Peter Z. Kunszt
European DataGrid Data Management
CERN Database Group
GGF Toronto 19.02.02
Motivation
• Small and large Grid applications
• working together to provide access to and management of
massive amounts of data
• Examples
• replica metadata catalog, service registry, application
metadata, logging and performance monitors
• Use
• easy to use, interoperable and high-performing database
technology
• Otherwise
• continue to use many proprietary approaches towards
metadata storage and retrieval
GGF Toronto 19.02.02
Baseline
• Aim: Unified Grid enabled front end to relational
databases.
• Convenient, scalable and efficient storage, retrieval
and query of data held in any type of local or remote
RDBMS.
• Core SQL functionality is insert, delete, update and
query.
• Implement as Grid Service
GGF Toronto 19.02.02
WP2: What did we sign up for?
Metadata Management
• 'Simple' Grid Persistency
– Grid Metadata
– Application Metadata
• Metadata Replication and Consistency
• Publish information on the metadata service
GGF Toronto 19.02.02
Local Application
Local Database
Local Computing
Grid
Grid Application Layer
Job Management
Data Management
Metadata
Management
Object to File
Mapper
Collective Services
Information &
Monitoring
Replica Manager
Grid Scheduler
Replica Catalog Interface
Replica Optimization
Underlying Grid Services
Spitfire
MetaData
Service
Computing
Element
Services
Storage
Element
Services
Replica
Catalog
Authorisation,
Authentication
and Accounting
Service
Index
Grid
Fabric
Fabric services
Resource
Management
Configuration
Management
Monitoring
and
Fault Tolerance
GGF Toronto 19.02.02
Node
Installation &
Management
Fabric Storage
Management
Spitfire Architecture
•
•
Web/Grid Services Paradigm
– SOAP interfaces
– JDBC interface to RDBMS
But not a 'real' distributed DBMS
– no distributed locking &
transactions (yet?)
– lazy consistency model
SOAP
•
Atomic RDBMS is always
consistent
No local replication of data
Role-based authorization
•
•
Global Spitfire
Layer
Connecting Layer
OracleLayer
OracleLayer
PGLayer
MyLayer
Oracle
DB2
PostGres
MySQL
GGF Toronto 19.02.02
Local Spitfire
Layer
The Local Layer
Required from the local Site:
• Any JDBC-enabled RDBMS backend. (PostGreSQL,
Oracle, ..)
• Any Servlet Container. (Tomcat, Oracle Application Server,
WebSphere…)
• SSL
Provides:
• SOAP & WSDL interface
• Role-based Authorization
• Simple Persistency
• Expiration based on a timestamp
GGF Toronto 19.02.02
The Global Layer
These are all only 'on the table' ; not worked out in detail, see
also Database BoF
• Distributed Querying
– Needs additional information on the given data, like
definition of common schemata and indices.
• Caching/Replication mechanisms
• Consistency
• Expiration
• Cleanup
• Transactions
GGF Toronto 19.02.02
Local Layer Architecture
Client API
SSL Auth
Security Servlet
Translator Servlet
JDBC
RDBMS
GGF Toronto 19.02.02
Expiration Servlet
Security Mechanism
HTTP + SSL
Request + client certificate
Servlet Container
SSLServletSocketFactory
Trusted CAs
Is certificate signed
by a trusted CA?
TrustManager
Revoked Certs
repository
RDBMS
Has certificate
been revoked?
No
Security Servlet
Authorization Module
Does user specify role?
Connection
Pool
No
Find default
Yes
Role repository
Role ok?
Translator Servlet
Role
Connection
mappings
Map role to connection id
GGF Toronto 19.02.02
Request and connection ID
Local Layer Functionality
• DB Administration
– Create Database
– Delete Database
– Create Table
– Drop Table
• Role Administration
– Create Role
– Delete Role
– Update Role
• DB Information
– Quotas
– Memory, Disk space
– User Info
– Schemata
• Core Functionality
– Insert
– Update
– Delete
– Select
• Timestamps
– Set Table Timestamp
– Set Row Timestamp
• Connections
– Open Dedicated
Connection
– Close Connection
GGF Toronto 19.02.02