Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Entity–attribute–value model wikipedia , lookup
Open Database Connectivity wikipedia , lookup
Extensible Storage Engine wikipedia , lookup
Microsoft Jet Database Engine wikipedia , lookup
Concurrency control wikipedia , lookup
Functional Database Model wikipedia , lookup
Relational model wikipedia , lookup
Clusterpoint wikipedia , lookup
LCG POOL, Distributed Database Deployment and Oracle Services@CERN Dirk Düllmann, CERN HEPiX Fall‘04, BNL Outline: • POOL Persistency Framework and its use in LHC Data Challenges • LCG 3D Project scope and first ideas for a LCG Database service • CERN Databases Services for Physics plans LCG POOL, 3D and Oracle@CERN POOL Objectives • To allow the multi-PB of experiment data and associated meta data to be stored in a distributed and Grid enabled fashion – various types of data of different volume and access pattern • event data, physics and detector simulation, detector data and bookkeeping data • Hybrid technology approach, combining – C++ object streaming technology (Root I/O), for the bulk data – transactionally safe Relational Database (RDBMS) services, (MySQL, Oracle) for catalogs, collections and other meta data • In particular, POOL provides – Persistency for C++ transient objects – Transparent navigation from one object across file and technology boundaries – Integrated with a external File Catalog to keep track of the file physical location, allowing files to be moved or replicated LCG POOL, 3D and Oracle@CERN D.Duellmann, CERN 2 POOL Storage Hierarchy • A simple and generic model is exposed to the application • A application may access databases (eg ROOT files) from a set of catalogs • Each database/file has containers of one specific technology (eg ROOT trees) • Smart Pointers are used – to transparently load objects into a client side cache – define object associations across file or technology boundaries LCG POOL, 3D and Oracle@CERN POOL Context FileCatalog Database Container Object D.Duellmann, CERN 3 Mapping to Technologies • Identify commonalties and differences between technologies Generic ROOT(Tree) ROOT(Dir) MySQL Oracle Database File File Database User Space Container Tree/Branch Directory Table Table Object ID Event Number Unique key Prim.Key Prim.Key • Model adapts to (almost) any storage technology with direct access – Record identifier needs to be known before flushing to disk • Use of RDBMS rather conventional – No special object support in SQL required LCG POOL, 3D and Oracle@CERN D.Duellmann, CERN 4 POOL Component Breakdown • POOL is (mainly) a client-side package – Coupling to standard file, database and grid services – No specialized POOL servers! • Storage Manager – Streams transient C++ objects to/from disk – Resolves a logical object reference to a physical object – I/O via ROOT (rfio/dcache) or Database(Oracle/MySQL/SQLite) • File Catalog – Maintains consistent lists of accessible files together with their unique identifiers (FileID), – Used to resolves the logical file reference (from a POOL Object ID) to a physical file • POOL API Storage Service FileCatalog Collections ROOT I/O Storage Svc XML Catalog Explicit Collection Relational Storage Svc MySQL Catalog Implicit Collection Grid Replica Catalog Collections – Defines (large) containers for objects (eg event collections) stored via POOL LCG POOL, 3D and Oracle@CERN D.Duellmann, CERN 5 POOL Grid Connected File Catalog Replica Location Replica Manager Service RootI/O Meta Data LCG POOL LCG POOL, 3D and Oracle@CERN Grid Resources Experiment Framework User Application Collections Grid Dataset Registry Meta Data Catalog Grid Middleware D.Duellmann, CERN 6 POOL Standalone Experiment Framework User Application Collections MySQL or RootIO Collections File Catalog XML or MySQL Catalogs RootI/O Meta Data LCG POOL LCG POOL, 3D and Oracle@CERN MySQL or SQLite Disconnected Laptop D.Duellmann, CERN 7 Why a Relational Abstraction Layer (RAL)? • Goal: Vendor independence for the relational components of POOL, ConditionsDB and user code – Continuation of the component architecture as defined in the LCG Blueprint – File catalog, collections and object storage run against all available RDBMS plug-ins • To reduced code maintenance effort – All RDBMS client components can use all supported back-ends – Bug fixes can be applied once centrally • To minimise risk of vendor binding – Allows to add new RDBMS flavours later or use them in parallel and are picked up by all RDBMS clients – RDBMS market is still in flux.. • To address the problem of distributing data in RDBMS of different flavours – Common mapping of application code to tables simplifies distribution of RDBMS data in a generic application independent way LCG POOL, 3D and Oracle@CERN D.Duellmann, CERN 8 Software Interfaces and Plugins Experiment framework FileCatalog Collection StorageSvc RelationalStorageSvc Relational Catalog Relational Collection ObjectRelationalAccess RelationalAccess MySQL Oracle Seal reflection SQLite uses implements Implementation Abstract interface Technology dependent plugin LCG POOL, 3D and Oracle@CERN D.Duellmann, CERN 9 POOL in 2004 Data Challenges • Experience with POOL framework gained in Data Challenges is positive! – No major POOL-related problems – Close collaboration between POOL developers and experiments invaluable! EDG-RLS as POOL back-end catalog • Deployment based on Oracle services provided by CERN Database group – Stable service throughout the 2004 Data Challenges! – Input concerning performance and required functionality for future Grid File Catalogs • Successful integration and use in LHC Data Challenges! • Data volume stored in POOL: ~400TB! – Similar to that stored in / migrated from Objectivity/DB! LCG POOL, 3D and Oracle@CERN D.Duellmann, CERN 10 Why a LCG Database Deployment Project? • LCG today provides an infrastructure for distributed access to file based data and file replication • Physics applications (and grid services) require a similar services for data stored in relational databases – Several applications and services already use RDBMS – Several sites have already experience in providing RDBMS services • Goals for common project (LCG 3D) – – – – increase the availability and scalability of LCG and experiment components allow applications to access data in a consistent, location independent way allow to connect existing db services via data replication mechanisms simplify a shared deployment and administration of this infrastructure during 24 x 7 operation • Need to bring service providers (site technology experts) closer to database users/developers to define a LCG database service – Time frame: First deployment in 2005 data challenges (autumn ‘05) LCG POOL, 3D and Oracle@CERN D.Duellmann, CERN 11 Project Non-Goals Store all database data • Experiments are free to deploy databases and distribute data under their responsibility Setup a single monolithic distributed database system • Given constraints like WAN connections one can not assume that a single synchronously updated database work and provide sufficient availability. Setup a single vendor system • Technology independence and a multi-vendor implementation will be required to minimize the long term risks and to adapt to the different requirements/constraints on different tiers. Impose a CERN centric infrastructure to participating sites • CERN is one equal partner of other LCG sites on each tier Decide on an architecture, implementation, new services, policies • Produce a technical proposal for all of those to LCG PEB/GDB LCG POOL, 3D and Oracle@CERN D.Duellmann, CERN 12 Database Services at LCG Sites Today • In contact with database teams at – Tier1 : – Potential Tier2: ASCC, BNL, CNAF, GridKa, FNAL, IN2P3 and RAL ANL and U Chicago • Several sites provide Oracle production services for HEP and non-HEP applications – Significant deployment experience and procedures exist but can not be changed easily without affecting other site activities • MySQL is very popular in the developer community – Initial choice often made by s/w developers • Not always with full deployment picture in mind – MySQL used for production purposes in LHC • Not at very large scale, though • Expected to deployable with limited db administration resources • Several applications are bound to MySQL • Expect a significant role for both database flavors – To implement different parts of the LCG infrastructure LCG POOL, 3D and Oracle@CERN D.Duellmann, CERN 13 Situation on the Application Side • Databases are used by many applications in the LHC physics production chains – Project members from ATLAS, CMS and LHCb – Alice interested in replication for online / T0 • Currently many of these applications are run centralized – Several of these applications expect to move to a distributed model for scalability and availability reasons – Move to distributed mode can be simplified by a generic LCG database distribution infrastructure – Still, this will require some development work • Need to make key applications vendor neutral – DB abstraction layers become available in many foundation libraries – Applications which are only available for one DB vendor limit deployment • Distribution would profit from policy in the area LCG POOL, 3D and Oracle@CERN D.Duellmann, CERN 14 Distributed Databases vs. Distributed Caches • FNAL experiments deploy a combination of http based database access with web proxy caches close to the client – Performance gains • reduced real database access for largely read-only data • reduced transfer overhead compared to low level SOAP RPC based approaches – Deployment gains • Web caches (eg squid) are much simpler to deploy than databases and could remove the need for a local database deployment on some tiers • No vendor specific database libraries on the client side • “Firewall friendly” tunneling of requests through a single port • Expect cache technology to play a significant role towards the higher tiers which may not have the resources to run a reliable database service LCG POOL, 3D and Oracle@CERN D.Duellmann, CERN 15 Application s/w stack and Distribution Options APP client s/w RAL = relational abstraction layer web cache RAL network SQLite file web cache Oracle db & cache servers MySQL db file storage LCG POOL, 3D and Oracle@CERN D.Duellmann, CERN 16 Tiers, Resources and Level of Service • Different requirements and service capabilities for different tiers – Tier1 Database Backbone • High volume, often complete replication of RDBMS data • Can expect good network connection to other T1 sites • Asynchronous, possibly multi-master replication • Large scale central database service, local dba team – Tier2 • Medium volume, often only sliced extraction of data • Asymmetric, possibly only uni-directional replication • Part time administration (shared with fabric administration) – Higher Tiers and Laptop extraction • Support fully disconnected operation • Low volume, sliced extraction from T1/T2 • Need to deploy more than one replication/distribution technology – Each addressing specific parts of the distribution problem – But all together forming a consistent distribution model LCG POOL, 3D and Oracle@CERN D.Duellmann, CERN 17 Starting Point for a Service Architecture? T0 M O M - autonomous reliable service T1- db back bone - all data replicated - reliable service O T3/4 T2 - local db cache -subset data -only local service M O LCG POOL, 3D and Oracle@CERN M Oracle Streams Cross vendor extract MySQL Files Proxy Cache D.Duellmann, CERN 18 3D Data Inventory WG • Collect and maintain a catalog of main RDBMS data types – Select from catalog of well defined replication options • which can be supported as part of the service – Conditions and Collection/Bookkeeping data are likely candidates • Experiments and grid s/w providers fill a table for each data type which is candidate for storage and replication via the 3D service – Basic storage properties • Data description, expected volume on T0/1/2 in 2005 (and evolution) • Ownership model: read-only, single user update, single site update, concurrent update – Replication/Caching properties • Replication model: site local, all t1, sliced t1, all t2, sliced t2 … • Consistency/Latency: how quickly do changes need to reach other sites/tiers • Application constraints: DB vendor and DB version constraints – Reliability and Availability requirements • Essential for whole grid operation, for site operation, for experiment production, • Backup and Recovery policy – acceptable time to recover, location of backup(s) LCG POOL, 3D and Oracle@CERN D.Duellmann, CERN 19 3D Service Definition WG • DB Service Discovery – – • • Connectivity, firewalls and connection constraints Access Control - authentication and authorization – • – – Database server and client installation kits • Which database client bindings are required (C, C++, Java(JDBC), Perl, ..) ? Server and client version upgrades (eg security patches) • Are transparent upgrades required for critical services? Server administration procedures and tools • Need basic agreements to simplify shared administration • Monitoring and statistics gathering Backup and Recovery – – • Integration between DB vendor and LCG security models Installation and Configuration – • How does a job find a close by replica of the database it needs? Need transparent (re)location of services - eg via a database replica catalog Backup policy templates, responsible site(s) for a particular data type Acceptable latency for recovery Bottom line: service effort should not be underestimated! – – We are rather close to LHC startup and can only afford to propose models that have a good chance of working! Do not just hope for good luck; These services will be a critical part of the experiments’ infrastructure and should be handled accordingly! LCG POOL, 3D and Oracle@CERN D.Duellmann, CERN 20 Replacement CERN Physics Services • Current systems not scalable to initial exploitation phase of LHC – Disk server poor match for DB needs; Sun Cluster under-configured • Tests of Oracle 10g RAC on Linux, as proposed to PEB, promising • Main goals of replacement service: – – – – Isolation – 10g ‘services’ and / or physical separation Scalability - in both database processing power and storage Reliability – automatic failover in case of problems Manageability – significantly easier to administer than now • Timeline: Price Enquiries for front-end PCs, SAN infrastructure and SAN storage completed – Orders made, delivery November, pre-production early 2005? LCG POOL, 3D and Oracle@CERN D.Duellmann, CERN 21 SAN-based DB infrastructure ATLAS CMS LHCb Mid-range Linux PCs: dual power supply, mirrored system disk, with dual HBAs multiple GbitE (3) ALICE COMPASS HARP spare spare F/C switches: 2 x 64 ports Storage: 16 x 400GB disks LCG POOL, 3D and Oracle@CERN D.Duellmann, CERN 22 Oracle Contract / Distribution • Client run-time – Oracle 10g Instant Client • currently Linux & Windows; • Mac still pre-production; needs ‘PEB’ decision re any support (incl. local copy…) – 10g Early Adopters Release 2 (10.1.0.3) • Client developer (Software Developers’ Kit – SDK) – Interim solution based on CERN re-packaging – Official support expected with 10.1.0.3 • Expected around December (OracleWorld) (when 10g R2 will be announced) • Server distributions – Support issues should not be underestimated • See Oracle Security Alert + Data Volume, 3D scenarios, etc. – Oracle “Standard Edition One” will be evaluated • Client bundling with ATLAS s/w – – – – Oracle have agreed to treat Tier0+Tiern sites as equal parts of LCG No issues regarding use of Oracle client for LCG at sites used for LCG Access to distributions managed by IT-DB still needs to be controlled “Oracle Partner” status would not provide any “added value” LCG POOL, 3D and Oracle@CERN D.Duellmann, CERN 23 Oracle Security Alert • Severity 1 Security Alert issued August 31 – This is the highest security level • The vulnerability is high risk and requires little specialized knowledge to exploit. • Apply the patch and/or workaround to the affected products with the highest priority. • IT-DB began immediately to prepare plan for patching servers – Major effort, spanning ~4 weeks • Lessons: need to limit versions of Oracle s/w in use – Several versions of (8), 8i, 9i, 10g in use • Often dictated by constraints e.g. 3rd party s/w (non-Physics) – Need plan in place so can react quickly to future alerts • Propose: – a) time slot when ‘routine’ interventions can be performed – b) procedure for performing critical interventions when needed LCG POOL, 3D and Oracle@CERN D.Duellmann, CERN 27 Summary • The POOL persistency framework (http://pool.cern.ch) been integrated in three experiment s/w frameworks and has been successfully used in the 2004 Data Challlenges – Some ~400 TB of data store in POOL – POOL has been extended to support RDBMS data in multiple relational databases • The LCG 3D project (http://lcg3d.cern.ch) has together with the experiments and sites started to define and implement a distributed database service forTiers 0-2 – Several potential experiment applications and grid services exist but need to be coupled to the upcoming services – Difference in available T0/1/2 manpower resources will result in different level of service • Currently defining the key requirements and verify/adapt a proposed service architecture – Need to start pragmatic and simple to allow for first deployment in 2005 – A 2005 service infrastructure can only draw from already existing resources – Requirements in some areas will only become clear during first deployment when the computing models in this area firm up • Consolidation activities for Physics DB services at CERN – Oracle RAC on Linux investigated as main building block for LCG services – Pre-production service planned for mid-2005 LCG POOL, 3D and Oracle@CERN D.Duellmann, CERN 28