Download Function approximation and fitting. ALICE selected

Function approximation and fitting.
ALICE selected topics.
Marian Ivanov
17th
May 2016
1
Outlook
Non parametric regression in multiple dimensions
Space point distortion calibration
Tools
Possible application for ML in the calibration, reconstruction and
data compression
17th
May 2016
2
Non parametric regression
17th
May 2016
3
Non parametric regression
There are many cases when we need to represent, fit, store (calibration
and QA DBs) and use N-dimensional functions

N= 1, 2, 3, 4, 5, 6
Generic implementation for:




17th
May 2016
functional and error representation
fitting
persistence
basic QA (semi-automatic)
4
Use cases
Fast Monte Carlo parametrization
●
ND (q/pt  PID, zv)
Performance parametrization
●
ND (q/pt  PID, zv )
dEdx calibration

Bethe-Bloch() +4D relative correction
Space point distortion calibration - Run1:

3D distortion maps (r,,z)
Space point distortion calibration - Run2 and 3:

3D distortion maps (r,,z) changing in time
TPC correction for signal below threshold:

5D correction, currently O(10%) of reconstruction CPU time
TPC space point error parameterization

5D drift, angle(pad,time), Q, (local occupancy, shape)
Bremsstrahlung correction for electron tracking in ITS

17th
May 2016
3D with non-Gaussian errors
5
Motivation and Requirements
Analytical models are not sufficiently precise. N-dim correction has to be
applied
Underlying model too complex.

CPU consideration
Parameters of known functions are unknown and have to be fitted
Fast prototyping needed to consider multiple models

Flexibility of TFormula, but fast (as virtual function) and
persistent
Generic code

17th
May 2016
Avoid backward compatibility problems, orphan code - models
changing in time
6
Why generic solution?
Many use cases lead to non parametric regression model
Generic code




easier to maintain
faster to prototype
easier to profile and optimize
easier to keep models persistent
Ad hoc solutions


17th
May 2016
more flexible
no learning curve
7
Implementation: non parametric local regression
17th
May 2016
8
Implementation
Implemented as a local polynomial
regression with kernel smoother


The estimated function is smooth,
and the level of smoothness is set by
a single parameter.
Standard local regression computation at prediction time
[Image source: Hastie, Tibshirani, Friedman]
17th
May 2016
9
Implementation
Pre-computation of ND local regression on
the grid
Function evaluation using lookup
table

Code tested in real applications in up to 5D


17th
no curse of dimensionality problem
(major criticism of application of
approach above 2D)
interactive visualization currently under
development
May 2016
10
Interface and examples
Initialization:
– Layout defined by THn (regular grid)
– Robust option using median filter
MakeFit():
– Specify TTree for input. Define values, errors, coordinates and
kernel shapes on its variables.
17th
May 2016
11
Visualization and TFormula interface
Interface to TFormula through a static function
– TFormulaPrimitive as a pointer to the function to be tested in ROOT6
– Used for “interactive” visualization of the fit function projection and residuals
– AliNDLocal regression has to be registered to the list of known function
TPC
1/dEdx
17th
May 2016

12
Visualization and TTreeFormula interface
Interface to TTreeFormula through a static function.
– Used for “interactive” visualization of the fit function projection and residuals


1/dEdx
17th
May 2016
13
Space point distortion calibration
17th
May 2016
14
Interaction Rate Scan, pp LHC15l
Bulk distortions as expected for Ar – CO2 O(1 mm)
Excess of distortion in the hotspots on the sector edges and for floating
wires O(1-5) cm
17th
May 2016
1515
Outline of the correction method
TOF

TRD






ITS
Reconstruct TPC with large road-widths to not
loose TPC clusters attachment
16
Match to ITS and TRD/TOF with relaxed tolerances
Refit ITS-TRD-TOF part and interpolate to TPC as a
reference of true track at every pad-row
(good alignment is prerequisite!)
Collect Y, Z differences between distorted
clusters and reference points in sub-volumes
(voxels) of TPC
Extract 3D vector of distortion in every voxel
Create smooth parameterization (calibration DB
object) to use for correction during following
reconstruction
Do this procedure in short time intervals to follow
changed of Interaction rate (~20-40 min, choice
constrained by statistics)
Correction procedure proposed for RUN3 had to be commissioned and applied
for RUN2 data
●
●
17th
Mean distortion corrected
RMS worsening strongly reduced. Residual worsening due short
time range functuation O(0.01-0.1 s)
May 2016
16
Outliers removal algorithm
Outliers due reconstruction with open tolerances



17th
May 2016
Local median (mean) filters for space points to reject
random cluster
Track matching to reduce “fake” track matching with outer
detectors
Linear fits with MAD (Median absolute deviation) as a cost
function
17
Run2 LHC15o (5kHz) vs Run1 LHC11h (2.5kHz)
at high pT mean corrected data better than in Run1, except some regions but
larger RMS
Run 1 with standard CPass0/Cpass1 calibration (*)
RUN 2 corrections status as of the end of February 1st results from alternative approach (bigger granularity
of the maps, better outliers handling)
17th
May 2016
1818
Work in progress
17th
May 2016
19
Distortion fluctuation
TPC distortion fluctuation 20-30% significantly higher than intrinsic
resolution:

Additional effective RMS including correlation of distortion accounted
space point errors
Typical time scale of distortion fluctuation O(0.1 s)


Determined by drift time of ions
Fluctuation of the number of ions in region determining local distortion
Work in progress time dependent correction
●
●
●
17th
May 2016
Needs measurement of instantaneous luminosity
Coarse correction O(min) in place, uses trigger detectors
O(0.01-0.1s) correction using (digital) current under development
20
Challenge - Time dependent correction using currents
Update distortion maps with required time granularity
Full analytical solution CPU/GPU consuming
●
work in progress
I ↔  ↔  ↔ E ↔ 
Promising - linear approximation of delta correction around mean value

Procedure successful tested in 1D (time variation only in z direction) using the
transfer function fit (matrix fit using TLinearFitter with regularization)

(z) = A(z,z') I(z'))

Parameters of matrix A fitted using MC data

regular pattern obtained
Full 3D approximation needed. Possible?



17th
(x,y,z) = A(x,y,z,x',y',z') (x',y',z')I(x',y',z'))

epsilon map to be calibrated
Deep neural network?
Other ML alternative ?
May 2016
21
ML applications
17th
May 2016
22
ML application
Alternative generic functional representation
e.g dEdx calculation - alternative to the ND local regression based
correction

Very low momenta track finders - loop finder


for data compression
challenging task to find helix



strongly distorted tracks
unknown z position, t0 offset (t → z) to be fitted to estimate
distortion)
high density (cluster pile-up)
Current ↔ distortion calculation
17th
May 2016
23
Conclusion
17th
May 2016
24
Backup
17th
May 2016
25
Tools
Not sure it it belongs to this meeting?
MILIPEDDE for ALICE barrel tracker alignment
TLinearFitter used for fitting of the non linear distortion with
O(103) free parameters



Externsion in TStatToolkit - similar interface as for AliNDlocalRegression
Regularization
Priors
TTreeStream - cout like interface for TTree filling


Standardized approach for debug/analysis mode of runing
numerical code
used for dumping of the intermediate results into tree for further
analysis
Set of tools for robust fitting in TStatToolkit
Chebyshev polynomials for ND functional representation
17th
May 2016
26
TPC - calibration use cases
dEdx calibration - model precision:
–
–
Measured dEdx - truncated mean non proportional to the input
ionization
Measured yield O(5%) ==> dEdx calibration precision ~ 0.5 %
•
–
Relative correction - function of 4 parameters
•
•
17th
May 2016
Analytical correction not sufficiently precise
1/Q, track multiplicity
Dependence does not factorize
27
TPC distortions
The TPC was internally mechanically aligned to the 0.1 mm level
Biggest observed distortion in the bending plane due to the ExB effect



B field inhomogeneity – distortions up to 8 mm
E field nonlinearities due misalignments – distortions up to the 6 mm
E and B field main component misalignment – distortions up to 2 mm
Right plot - resulting space point correction map as used currently in the Alice
reconstruction

The ExB effect time dependent (pressure, temperature, gas composition) – parameters
updated on the run level
TPC space point correction framework developed - ALICE & STAR collaboration

02th
February 2011
Physical (numerical solution of the Poisson equation) and effective distortion models
TPC planning meeting
28
TPC Distortion/Alignment Fitting
Assumptions:
Space point distortion transformation commute (the order
of applying of corrections is not important)

Space point distortion can be approximated as a linear
combination of the “partial distortion” functions with given
parameter:

 = ki Ei

Space point distortion not directly observed. We define the
set of observables O.

 = ki Oei

Under given assumption the analytical (non iterative)
global minimization of distortion maps can be performed
solving the set of linear equations.

Assumptions were tested for the typical distortion in the
TPC, moreover the assumption were tested also for the
fitted parameters.

Numerical part based on the linear fitting package implemented in
the ROOT
Additional functionality implemented in the AliRoot (Alice
framework)




02th
February 2011
Input data observables and fit models from the tree
Possibility to add constrains
Possibility to check the the fit values (return value of the
FitPlaneConstrain can be used as a alias in tree)
Extraction of the partial fits
TPC planning meeting
Distributed computing
Calibration train
(Grid) filling of
residual
histograms
Merging
Creation of distortion
maps
Distortion models
Fitting
29
Example distortion fits - Field cage and Rod alignment
A side
Positive
C side
Negative
18 (rods) x 2 (IFC,OFC) x 2 (A side, C-side) + 2 rotated
clips x 2 (at the Resistor road)

Small misalignment ( ~ 0.1 mm ) leads to a
significant non linear distortion up to 6 mm
B field 0 data ( 4D histograms of residuals between the
line and space points ) used as a input for the
alignment and E field distortion calibration
3D Distortion map obtained from the track residual
histograms

02th
Linear fit with 796 parameters
February 2011
TPC planning meeting
30
TPC - calibration use cases
Space point distortion calibration - Run1:
–
–
3D distortion maps (r,phi,z)
Implemented as a linear combination of primitive distortions
•
fit using TLinearFitter with O(103) parameters
Space point distortion calibration - Run2 and 3:
–
17th
May 2016
3D distortion maps (r,phi,z) changing in time
31
Reconstruction use cases
TPC correction for signal below threshold
5D analytical correction (using several approximations) to be
replaced by non parametric function



currently O(10%) of reconstruction CPU time
Parameters: (position(pad,time), angle(pad,time), Qmax/threshold)
Option: fit analytical function or data driven parameterization

TPC space point error parameterization (5D)

17th
May 2016
drift, angle(pad,time), Q, (local occupancy, shape)
32
Reconstruction use cases
Bremsstrahlung correction for electron tracking in ITS
–
–
Standard Kalman filter assuming Gaussian errors, wrong
assumption for electron tracking in ITS
DNA (Dynamic noise adjustment) algorithm for ALICE ITS
tracker under preparation.
•
17th
May 2016
Significant improvement from a fast prototype with 3D non
parametric regression correction (deviation in 3 ITS layers
projection)
33
MC and analysis example
Fast MC:

tune on the MC




TPC response parameterization
TPC space point resolution
Matching efficiency parameterization
tuned on the Data
Analysis:

Performance parameterization (bias, pulls, efficiency, matching
eff.)
•
–
17th
May 2016
q/pt, vertex position, track multiplicity, PID
MC/data
34