Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
CERN – European Organization for Nuclear Research Administrative Support - Internet Development Services CET and the quest for optimal implementation and maintenance efficiency with Data warehousing and J2EE Components Mikael Angberg, AS-IDS Outline Introduction to CET Maintenance and Implementation issues Quality Assurance Maintainability and Versatility Conclusions Questions CERN IDS Challenges facing CERN Today Build the worlds largest scientific instrument… Budget Staff With less budget.. 8.6 km fewer staff… CERN IDS for more scientists… with higher expectations... Challenges facing the CET Team Today Build the worlds coolest Financial Decision Support application… Budget Staff With a small budget.. CERN IDS few staff… for more financial managers… with higher expectations... CERN Expenditure Tracking CERN IDS CERN Expenditure Tracking Critical for CERN 1000+ “Financial” Managers Globally Distributed Wide range of user needs CERN IDS High Level System Requirements Quality Maintainability Versatility CERN Expenditure Tracking CERN IDS The “System Idea” User Interface Application (Java) Data Warehouse CERN IDS Business Logic Financial Data Data Warehouse Quality Data and Data Warehouse Quality o Capturing the “right” data o Validating the data o Error handling o Centralised Data o Managed complexity o Coding Standards o Code Inspections CERN IDS Data Warehouse Quality Centralize data One Place, One Format, No Redundancy Financial Data Sources Stores Purchase Finance CERN IDS Error Handling Process and Validation Extraction Loading Transform Data Warehouse Data Warehouse Quality Managed Complexity SUM..GROUP BY 1999 SUM..GROUP BY SUM..GROUP BY 2000 SUM..GROUP BY 2001 SUM..GROUP BY Partitioned Table SUM..GROUP BY Reduced number of DB Objects Parameterized Extraction CERN IDS Single Entry Point Application Quality Structured Development Process o Iterative… o Development o Testing o Deploy Coding Standards o Based on Sun’s Java Specification Code Inspections CERN IDS o Inspired by Fagan and NASA Quality Framework Benefits Ensured Data Consistency Extended data quality assurance through automated validation processes Greatly Reduced Database Maintenance Increased production code quality (less bugs) Higher developer productivity Increased transparency and knowledge transfer within and between development teams CERN IDS Maintainability and Versatility Objectives Satisfy a Global user community …consisting of more than 1000 people with different needs (and many requests) …by providing powerful analysis and reporting capabilities With a development team of maximum 5 people, minimum 1 CERN IDS - Maintainability - Versatility Facts and Dimensions The Dimensional Model Dimension Data Warehouse Time Dimension Location Measures / Facts Dimension Payments Commitments . Keys Order Partitioned Fact Table CERN IDS Dimension Supplier Maintainability and Versatility Managed Complexity SUM..GROUP BY 1999 SUM..GROUP BY SUM..GROUP BY 2000 SUM..GROUP BY 2001 SUM..GROUP BY CERN IDS Partitioned Table SUM..GROUP BY Aggregates and Query Rewrite Query Re-write SELECT SUM(sales) FROM raw_data; Dimension Tables Aggregate Tables / Materialized Views 1999 1999 2000 2000 2001 2001 Raw Data Table CERN IDS SUM..GROUP BY SUM..GROUP BY SUM..GROUP BY Scalability and maintained simplicity Partitioning <2000 2000 2001 2002 Partitioned Index Partitioned Table <2000 2000 2001 2002 Partitioned Index Partitioned Table CERN IDS Design Conclusions DWH The data warehouse structure allows : Access to any aggregate level Access to any range of data …transparently to the user Maintenance and Versatility perspective : “Simple” design – Automation of recurrent maintenance Flexible – Easy to tune and extend Scalable – Facilitates growing with preserved simplicity CERN IDS ART – the Java Reporting Framework “Increase maintainability, by applying existing design/code and documentation standards and ensure in-house knowledge of the product. “ “Use of the framework should be possible with knowledge of standard languages only such as Java, SQL, XML. “ User Interface Form Generation & Validations Three Main Components Report Generation HTML, Excel, PDF… CERN IDS Query Generation Data Retrieval & Preprecessing ART – the Java Reporting Framework Web-based Java components XML Templates Web report Input Objects XML Templates Input Validators Query Builder Report Generator CERN IDS Key “Success” Factors Development process integrating quality assurance Design assures data quality Robust yet Flexible Data Warehouse Design Simplicity - Low on maintenance Component based (Java) reporting framework CERN IDS Conclusions Users World-wide, highly reliable and available Financial Decision Support Developers Decrease in maintenance efforts Faster “time-to-market” Increased productivity and quality Development focuses on solving business problems – Stable Infrastructure CERN IDS Thank You For More Information Browse to: http://ais.cern.ch/ or Email: [email protected] CERN IDS Summary of CET Web-Based, multi-lingual 20 GB Data Warehouse - 360 GB Raw Data Over 1000 active users ~ 1000 Reports per day ~ 24 / 7 Availability 100% Automated Recurrent Maintenance No Ad Hoc Querying CERN IDS