Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Ideas on the LCG Application Architecture Application Architecture Blueprint RTAG 12th June 2002 P. Mato / CERN Application Domains Event Processing Applications – Trigger, Simulation, Reconstruction, Selection programs Mainly batch oriented. Interactive for development & debugging. Real-time. – Event/Detector display, data presentation programs Mainly interactive – Calibration and Alignment programs Batch and interactive Data Analysis Detector calibration Job configuration, submission, monitoring and control – Grid awareness 24/10/2001 LCG Application Architecture Mainly interactive 2 No Disjoint Domains For example, the LHCb requirements for interactive analysis: – Better [than PAW] integration with experiment framework » consistent with the analysis batch environment, use the same toolkits and experiment algorithms/tools » access the experiment data objects and allow browsing » integrated with event display » allow interactive reconstruction and simulation 24/10/2001 LCG Application Architecture 3 Architectural styles General categorization of systems [1] user-centric focus on the direct visualization and manipulation of the objects that define a certain domain data-centric focus upon preserving the integrity of the persistent objects in a system computation-centric focus is on the transformation of objects that are interesting to the system [1] G. Booch, “Object Solutions”, Addison-Wesley 1996 24/10/2001 LCG Application Architecture 4 Different style in different domains • The applications in the different domains may have different emphasis in: • Interactivity datacentric usercentric User Interface Interactivity Scripting Object store Database Data Integrity • Database • Computation Algorithms Computation • Elements of all three are present in all applications 24/10/2001 computation-centric LCG Application Architecture 5 Computation-centric: GAUDI Framework adequate for “all” event processing applications Algorithms process “event data” with the help of “services” and using “detector data”. Application Manager Message Service JobOptions Service Particle Prop. Service Other Services 24/10/2001 Converter Converter Converter Event Selector Event Data Service Persistency Service Data Files Transient Event Store Algorithm Algorithm Algorithm Detec. Data Service Transient Detector Store Persistency Service Data Files Histogram Service Transient Histogram Store Persistency Service Data Files LCG Application Architecture 6 User-centric: PAW Provides interactive graphical presentation and statistical or mathematical analysis, working on objects familiar to physicists like histograms, event files (Ntuples), vectors, etc. Is based on several components of the CERN Program Library. Easy to use and freely available No integration within experiments framework Offline analysis done in 2 separate steps “batch” and “interactive”, the user decides how much to do in each step 24/10/2001 LCG Application Architecture 7 The Challenge Many components are shared among the different applications in the different domains We would like to profit from the best of the existing solutions (implementations) for each component but not duplicate everything. >>> INTEGRATION <<< 24/10/2001 LCG Application Architecture 8 Component interface specifications Integration technologies Probably the best solution but unrealistic in our environment – Mainly for distributed computing – CORBA, DCOM (.NET), Web Services, JavaBeans Abstract interfaces Works well within a given Framework – C++ “in process” integration – Robust and efficient. A bit awkward for end-users. Scripting extension modules – Ideal for prototyping. Rapid application development Minimal coupling, (RAD). maximum flexibility. – Dependent of the scripting tool Lacks performance. 24/10/2001 LCG Application Architecture 9 For user-centric applications Interactive analysis, configuration, monitoring applications, etc. Software Bus Define a “bus” specifications equivalent to a hardware bus (e.g. VME) where modules implementing different functionality can be plugged easily. Grid Database GUI shell Bus Analysis Tools (Lizard) 24/10/2001 Experiment Framework (Gaudi) LCG Application Architecture math math 11 Python as a Software-Bus Python is an object-oriented scripting language commonly used in a wide variety of domains But, it could also be seen as framework where you can plug easily “extension modules” in binary form implemented using other languages. – Very easy and natural to interface to C++ classes (C++ API) – Python should only be the “glue” between modules developed in C++ or other languages – The interface (API) for Python extension modules is quite simple and at the same time flexible 24/10/2001 LCG Application Architecture 12 Module Communication Communication between two extension modules is always possible using Python primitive types (a) – Example: I would like to display an histogram of Gaudi It is more efficient to communicate using a reference to an interface Python (AIDA?) (b) – The objects in Python are only references Both ways are possible!! 24/10/2001 Python Primitives (a) (b) Plotter IH1D Lizard (C++) LCG Application Architecture IH1D* Gaudi (C++) 13 Why Python? Interpreted – However it is quite fast (byte code idea from Java) Dynamically typed – Not need to declare any variable Simple syntax – Emphasis by the authors to minimize typing Shells Powerful built-in types and modules Ideal for Scripting and Prototyping 24/10/2001 LCG Application Architecture 14 What is Feasible? Very rich set specialized generic modules LHC modules Gateways to other frameworks EDG API PVSS XML Several GUI toolkits Database GUI GUI Python JPE rootpython gaudipython Java Classes Root Classes Gaudi Framework 24/10/2001 LCG Application Architecture math math shell Very rich set of Python standard modules 15 Example Filling an Excel spreadsheet from a ROOT ntuple 24/10/2001 # Get the ntuple from the ROOT file import rootmodule hfile = TFile('hsimple.root') ntuple = gROOT.FindObject('ntuple') entries = ntuple.GetEntries() nvar = ntuple.GetNvar() tuple = ntuple.GetArgs() # Initialize Excel import win32com.client excel = win32com.client.Dispatch('Excel.Application') wbook = excel.Workbooks.Add() wsheet = wbook.WorkSheets.Add() wsheet.Name = ntuple.GetTitle() # Fill Excel sheet for i in xrange(500) : ntuple.GetEntry(i) for j in range(nvar) : wsheet.Cells(i+1,j+1).value = tuple[j] # Make Excel sheet visible excel.Visible = 1 LCG Application Architecture 16 Ideal Interactive Application Menu GUI Canvas (histograms, 3D graphics, etc.) Data Browser >>> execfile(‘mymacro’) >>> for c in cands : plot(c) >>> 24/10/2001 Scripting Window LCG Application Architecture 17 Where are the problems? The Software-Bus architecture is not applicable everywhere – I would not like to implement the LHCb 2 level trigger with that ! – Software-Bus + Abstract interfaces is a good compromise for user-centric applications (fast prototype) In addition to “standardize” to the Python extension API we need more – AIDA ?, interoperability between Boost/Swig? … Mixing several GUI toolkits, thread libraries, etc. may pose serious technical problems. 24/10/2001 LCG Application Architecture 18