Download AIDA, JAIDA and AIDAJNI: Data Analysis using - Indico

AIDA, JAIDA and AIDAJNI:
Data Analysis using
interfaces
DONSZELMANN, Mark (SLAC)
JOHNSON, Tony (SLAC)
TURRI, Massimiliano (SLAC)
SERBO, Victor (SLAC)
AIDA - Abstract Interfaces for
Data Analysis




AIDA is a standard set of interfaces for creating and manipulating
histograms, n-tuples and related data analysis objects, like Plotter, Fitter,
etc.
It has been created cooperatively by a group of developers working on
high-energy physics data analysis tools. The goal of the AIDA project is
to provide the user with a powerful set of interfaces which can be used
regardless of which analysis tool they are using.
Only define “protocol” for analysis objects, no internal details
 AIDA defines behavior, analysis tool provides implementation
AIDA defines XML format for representation and storage of analysis
objects.
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
Advantages

Use same code with any AIDA-compliant analysis tool.
A
User code
I
(e.g. GEANT4)
D
A


Analysis tool 1
Analysis tool 2
User needs to learn only one set of interfaces.
Interoperability of analysis tools.
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
AIDA Examples

IHistogram1D extends IHistogram
-
IAxis axis()
void fill(double x)
void fill(double x, double w)
double mean()
double rms()
double binEntries(int index)
double binError(int index)
….

IAxis
-

int bins()
double lowerEdge()
double upperEdge()
double binLowerEdge(int index)
double binUpperEdge(int index)
double binWidth(int index)
int coordinateToIndex(double x)
….
For more information on AIDA,
see: http://aida.freehep.org
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
Example code
IAnalysisFactory
ITree
IHistogramFactory
IFitFactory
IFitter
IPlotter
aF
tree
histF
fitF
fitter
plotter
IHistogram1D gaussHist
=
=
=
=
=
=
IAnalysisFactory.create();
aF.createTreeFactory().create();
aF.createHistogramFactory( tree );
aF.createFitFactory();
fitF.createFitter("Chi2");
aF.createPlotterFactory().create("Plot");
= histF.createHistogram1D("gaussHist",
"Gaussian Histogram",100,-5,5);
Random r = new Random();
for (int i=0; i<10000; i++) gaussHist.fill(r.nextGaussian());
IFitResult chi2Fit = fitter.fit(gaussHist,"g");
plotter.createRegions();
plotter.region(0).plot( gaussHist );
plotter.region(0).plot( chi2Fit.fittedFunction() );
plotter.show();
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
Current status





Current version is AIDA 3.2.1, stable
Ongoing work to improve interfaces
CVS repository, mailing lists, etc.
More details on AIDA Home Page:
http://aida.freehep.org
Interfaces are automatically generated by AID
from one source. Easy to add new languages



Java and C++ interfaces are currently generated
Python is under consideration now
Several AIDA implementations and analysis
tools
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
Implementations

There are several AIDA implementations and AIDAcompliant analysis tools:



PI (C++)
 http://cern.ch/pi
OpenScientist (C++)

http://www.lal.in2p3.fr/OpenScientist
 Guy Barrand, LAL, [email protected]
PAIDA (Python)
 http://paida.sourceforge.net
 Koji Kishimoto, Osaka University,
[email protected]
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
Implementations continued…



JAIDA (Java), part of FreeHEP Java library, SLAC Team
 http://java.freehep.org/jaida/index.html
AIDAJNI (C++ and Java), part of FreeHEP Java library, SLAC Team
 Allows C++ program to use JAIDA
 http://java.freehep.org/aidajni
This talk will focus on Java implementation of AIDA, but
short information about other implementations is
included below as well.
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
PI AIDA Implementation

PI project provides C++ implementations to all AIDA interfaces
except the Plotter

Two implementations exist for Histograms:



Use of Plug-in manager from SEAL




Dynamic loading of the PI components
User can choose at run time :


AIDA_Root_Histogram : wrapper around TH* histograms and TProfiles
AIDA_Native_Histogram : direct AIDA implementation (include Clouds)
implementation for Histogram or Fitter
storage type (ROOT, XML, HBOOK)
Interoperability with other AIDA implementations and other
analysis tools (ROOT, HippoDraw)
Python Bindings to PI AIDA classes (created using SEAL Dictionary)
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
Example of Interoperability

Plotting AIDA Histogram created using the PI
implementation in HippoDraw, ROOT and JAIDA
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
OpenScientist and AIDA
•
•
•
OpenScientist / Lab package is a C++ implementation of AIDA. http://www.lal.in2p3.fr/OpenScientist
OpenScientist-13.0, released for CHEP, is AIDA-3.2.1 compliant.
One new feature is OpenPAW (see poster). It is a PAW reimplementation over OpenScientist tools. It had
been done by using the AIDA interfaces as much as possible. Then :
•
OpenPAW could be seen as a PAW front end (façade) for AIDA.
•
AIDA could be seen as the C++ API to OpenPAW.
•
Standard pawex1 to pawex24 already emulated.
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
OpenScientist, AIDA and Panoramix
•
•
•
Panoramix is a visualization environment for LHCb (See oral presentation). It is already used
as a detector viewer and event display. But it could also be used as a “stat tool” viewer.
Panoramix uses OpenScientist tools. In particular the OpenScientist AIDA environment. BUT,
histograms used by LHCb are the AIDA compliant one managed by Gaudi (which is not
compliant to an AIDA::ITree).
One major trend in the next year(s) developments will be the integration within Panoramix of
all these AIDA pieces coming from various parties (GUI of Panoramix, AIDA::IPlotter done
with HEPVis / SoPlotter, AIDA histograms managed by Gaudi, the ones managed by the
AIDA::ITree of OSC / Lab, the ones of PI (managed by ?)). For interactivity, the author would
like to integrate a version of OpenPAW able to deal with all that.
• LHCb had been able to migrate this year its histograms to AIDA-3.2.1. This
version of AIDA is now supported by all implementations (OSC, PI, JAIDA).
Then LHCb / Panoramix is going to be an ideal place to put all that “in the
caldron” and have a big cooking to prepare “the magic potion”… (hoping that
some root will not spoil the mixture).
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
PAIDA: Python Implementation of AIDA

Pure Python (with zlib and Tkinter modules)







Implements AIDA 3.2.1
Check it out on: http://paida.sourceforge.net/
easy to install
run on many platforms
no need for any external library like Minuit etc.
no need for Fortran compiler on MacOSX any more
easy to create your own "pure Python" application
by embedding PAIDA
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
PAIDA continued…
Also has:
• Its own pure-Python fitter
• Plotting capabilities
• Save plots in PS format
• Save analysis objects to
XML file
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
JAIDA


JAIDA – Full implementation of AIDA in Java
JAIDA is part of FreeHEP Java library:
http://java.freehep.org

Current release of JAIDA is 3.2.3, based on
AIDA 3.2.1
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
JAIDA – Features

Fitting

support for multiple fitting engines (optimizers)



Support for multiple fitting methods


UNCMIN (pure Java)
Minuit (needs minuitAdapter native library)
LeastSquares, Chi2, BinnedMaximumLikelihood and
UnbinnedMaximumLikelihood
Stores





Read/Write support for AIDA XML format (with built-in gzip)
Read support for ASCII text files
Read support for HBOOK files (needs hbookAdapter native library)
Read support for Root files (write support hopefully coming soon)
Future plans to add:

SQL interface, binary store format …
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
JAIDA – Features continued…

Plotting

Two implementations of IAnalysisFactory



Multi-Threaded – plots update in real-time as they are filled


No need to call interact() or refresh()
Support for embedding plotter



BatchAnalysisFactory – Dummy plotter, ignore calls to IPlotter
AnalysisFactory – Full plotter functionality
In other Java GUI’s
In Java servlets for web based applications.
High quality graphics export in many formats including:

PDF, EPS, SVG, SWF, PNG, GIF, JPG
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
JAIDA, other details

Uses Java “service provider mechanism” for
dynamic loading of services at Runtime
JAIDA functionality can easily be extended by user
 Can be used to add support for new fitters, stores etc.
 Have defined “developer level” interfaces for
IOptimizer, IStore etc
Uses Java Expression Library (JEL) for runtime compilation
of expressions (for tuple evaluators and functions)
JAIDA is ideal for batch analysis and jobs that don’t
require GUI interactivity



Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
JAS3




For interactive analysis better to use JAS3: a
general purpose, open-source, interactive data
analysis tool
Internally uses JAIDA for analysis, but adds
GUI interactivity
Can easily expend JAS3 functionality by
writing custom “plugins”, see talk “Writing
Extension Modules (Plug-ins) for JAS3 ”
on September 30, 18:10
JAS3 Home Page: http://jas.freehep.org/jas3
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
JAS3 Overview

Design based on Application Shell, into which many
(optional) modules can be plugged

Highly customizable for different application domains




Modules can be updated independently of the shell


Possible to release bug fixes fast
Includes support for programming in several languages



HEP/Astrophysics/Other
Data analysis/Online Monitoring/GRID analysis
Experiment/User specific modules
Scripting: currently Pnuts and Python
Java (compiled)
Analysis (histograms, tuples, fitting, etc.) based on JAIDA
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
JAS3 Example
Can write and
run scripts,
Java code
Console allows
direct interaction
with scripting
language
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
AIDAJNI

“Glue” layer between C++ user code and
JAIDA. Part of FreeHEP Java library.
User
Application
(C++)

AIDAJNI
Layer
JAIDA
(Java)
Allows C++ program to use JAIDA
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
AIDAJNI continued…




AIDAJNI – Allows C++ program to use JAIDA
AIDAJNI is part of FreeHEP Java library:
http://java.freehep.org
Current release of AIDAJNI is 3.2.2, based on AIDA 3.2.1
Detailed instructions how to use AIDAJNI are included in
Geant4 Tutorial CD at SLAC. See Exercise #5 – “Analyzing
Geant4 Data with JAS/JAIDA”:
http://geant4.slac.stanford.edu/g4cd/July2004/Documentation/WorkshopExer
cises/Exercise5.html

and also in the AIDAJNI Release Notes:
http://java.freehep.org/aidajni/ReleaseNotes-3.2.2.html
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
Working with AIDAJNI

Before compiling C++ program user needs to:



Install JAIDA and AIDAJNI
Define JAIDA_HOME and AIDAJNI_HOME
environment variables
Run JAIDA and AIDAJNI setup scripts (come with
distributions)
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
Using JAIDA with C++
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
AIDATLD – Tag Library





Using the AIDA Tag Library you can easily insert live
plots into Java Server Pages (.jsp) files.
The AIDA Tag Library is designed to work in a container
supporting JSP 2.0 or greater (such as Tomcat 5.0.*)
HTML-like tags: <aida:tree>, <aida:plotter>,
<aida:style>, etc.
Examples/Demo and Documentation:
http://aidatld.freehep.org
Can be used to make data/plots accessible to the
collaboration quickly
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
AIDATLD Demo – plot from ROOT file
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
AIDATLD Source Example of JSP
.....
<%@taglib prefix="aida" uri="http://java.freehep.org/jsp/aida" %>
<%@taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %>
<html>
<head><title>Single plot of a histogram accessed via rootd</title></head>
<body>
<c:set var="rootDataURI" value="root://rh92.slac.stanford.edu/demo.root" />
<c:set var="histoPath" value="/h110" />
<aida:plotter>
<aida:region>
<aida:plot dataSourceURI="${rootDataURI}" plotObjectPath="${histoPath}" />
</aida:region>
</aida:plotter>
</body>
</html>
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
Conclusion







AIDA provides rich set of interfaces for data analysis
Ongoing effort to improve and extend AIDA
Several implementations of AIDA exist in C++, Java,
Python.
AIDAJNI allows to use JAIDA from the C++ program
Convenient way to do interactive and batch analysis with
JAIDA and JAS3
AIDATLD and JAIDA provide easy way to include live
plots into HTML pages
All products above use AIDA Interfaces
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland
More information

Here are some useful links:







AIDA Home Page: http://aida.freehep.org
AIDATLD – Tag Library: http://aidatld.freehep.org
FreeHEP Java Library: http://java.freehep.org
JAS3 Home Page: http://jas.freehep.org/jas3
Geant4 tutorial CD at SLAC:
http://geant4.slac.stanford.edu/g4cd/Welcome.html
Report bugs: http://bugs.freehep.org
Discussions, questions: http://forum.freehep.org
Victor Serbo, SLAC
27 September 2004, Interlaken, Switzerland