Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
LHC-HCG meeting Matters of convention for “expected” results 1. Mean vs. median vs. Asimov 2. How do we define 1s and 2s bands ? Mingshui Chen Jan 27, 2011 Mingshui Chen ( University of Florida ) 1 A few examples Differential distributions of CL.95% upper limits (Bayesian with flat prior) for 1000 sets of outcomes 1. Large statistics nbkg=100, nsig=50, no systematics Discreteness of lines hardly seen 2. Single channel with small statistics 3. Multiple channels with small statistics nbkg=1, nsig=1, no systematics The discrete probabilities will be changing smoothly as one moves along r from small to large r two channel, no systematics nsig1=0.3 nbkg1=3 nsig2=0.5 nbkg2=1 The discrete probabilities will NOT be changing smoothly as one moves along r from small to large r, due to the dips Jan 27, 2011 Mingshui Chen ( University of Florida ) 2 Mean or Median or “Asimov” Do we show "mean" well defined, requires almost no conventions but statisticians do not like "mean" as it is not “preserved” under transformations of variables; but do we really care? or "median" requires a convention for highly discrete distributions or “Asimov data" ? “imaginary” (in general, non-integer) most probable experimental outcome requires a whole paragraph of explanations Jan 27, 2011 Mingshui Chen ( University of Florida ) 3 Conventions of bands A simple naïve convention ±1 and ±2 standard deviation (“two sided”) spread of the expectation obtained from a large number of toy experiments It becomes rather problematic for small statistics cases where bands must be asymmetric and even one-sided… Build up CL intervals by adding up probability-ranked possible experimental outcomes If p(r) changes smoothly, say first rising up and then falling down, then one would get a continuous interval. If probabilities are jumping up and down, such procedure would give disjoint sections as in the case of multiple channels with small statistics Build up CL intervals following “quantiles” using cumulative distributions see next slides Build up CL intervals following minimum-range principle see next slides We need a well-defined convention Jan 27, 2011 Mingshui Chen ( University of Florida ) 4 Convention of bands in LandS Throw 1000 toy experiments according to the background-only model (use bkgd systematic errors when non-zero) Evaluate r95% for each of the 1000 toy experiments Make differential and cumulative distributions of the obtained r95% values differential distribution of r95% cumulative distribution of r95% 0.977 0.841 0.5 (median) 0.159 0.023 average expected <r> Jan 27, 2011 Mingshui Chen ( University of Florida ) To define median/bands, use crossings of the percentile lines and the interpolation line between nearby pairs of physically possible values of r. The current interpolation choice is the Fermi function 5 Bands with different interpolations nbkg=1, nsig=1, no systematics, Bayesian with flat prior Fermi function interpolation - 2s = 3.00 - 1s = 3.00 median = 3.45 +1s = 4.73 +2s = 6.67 Jan 27, 2011 Mingshui Chen ( University of Florida ) Linear interpolation - 2s = 3.00 - 1s = 3.00 median = 3.46 +1s = 4.88 +2s = 6.72 Step function - 2s = 3.00 - 1s = 3.00 median = 4.11 +1s = 5.41 +2s = 6.78 6 Bands with minimum-range principle E.L. Crow and R.S. Gardner, Confidence intervals for the expectation of a Poisson variable, Biometrika 46 (1959), pp. 441–453. From the differential distribution of r95% , get all intervals which contain 68%/95% of possible limits e.g. on right plot, both ranges [4.1, 7.7] and [4.6, 9.4] contain 68% of possible limits Take the interval corresponding to minimum range Jan 27, 2011 Mingshui Chen ( University of Florida ) 7 An example with different conventions s = b = 10 / (m/100)^2 no systematics, Bayesian with flat prior "mean“ (blue solid) and "Asimov" (red solid) are the same for all three plots With interpolation Jan 27, 2011 Mingshui Chen ( University of Florida ) Step function Minimum-range principle 8 Summary: possible options to choose from "Typical" expected result : mean smooth (almost no convention) smooth Asimov (will it work for all methods?) median Fermi interpolation smooth (requires convention, e.g. 50% quintile) linear interpolation smooth step function jigged "Green"/"yellow" bands require convention, e.g. quantiles Fermi interpolation smooth (2.5%, 16%, 84%, 97.5%) linear interpolation smooth step function jigged minimum-range principle Jan 27, 2011 Mingshui Chen ( University of Florida ) jigged 9