Download Document

Chi Square Distribution (c2) and Least Squares Fitting
Problem: We have some measurements and would like some way to measure
how “good” these measurements really are.
Solution: Consider calculating the “c2” (“chi-square”)
Chi Square Probability Distribution Function (See Taylor Ch 8, 12, App. D or Barlow P106-8, 150-3)
l
Assume:
u We have a set of measurements {x1, x2, … xn}.
u We know the true value of each xi (xt1, xt2, … xtn).
u Obviously the closer the (x1, x2, … xn)’s are to the (xt1, xt2, … xtn)’s the better (or more accurate)
the measurements.
Can we get more specific? Can we put a number (or probability) on how well they agree?
l
l
Assume:
u The measurements are independent of each other.
u The measurements come from a Gaussian distribution.
u Let (s1, s2 ... sn) be the standard deviation associated with each measurement.
Consider the following two possible measures of the quality of the data:
n
n
xi  xti
d
R
 i
Both measures give zero
si
i 1
i 1 s i
when the measurements are
2
2
n
n
(x  x )
d
identical to the true values.
c 2   i 2 ti   i2
i 1
u
si
i 1
si
.
+d
y
R=0
.
-d
x
Which of the above gives more information on the quality of the data?
n Both R and c2 are zero if the measurements agree with the true value.
n R looks good because via the Central Limit Theorem as n   the sum  Gaussian. But
However, c2 is better!
Richard Kass/Sp12
P416 Lecture 6
1
One can show (derive) that the probability distribution function for c2 is exactly:
p ( c 2 , n) 
1
2 n / 2 1  c 2 / 2
2
[
c
]
e
0

c

n/2
2 G(n / 2)
This is called the "Chi Square" (c2) probability distribution function.
G is the “Gamma Function”.
G( x)  0 e t t x 1dt x  0
G(n  1)  n! (n  1,2,3...)
G(1 / 2)  
This is a continuous probability distribution that is a function of two variables:
c2 and n = number of degrees of freedom (DOF).
DOF=n = (# of data points) – (# of parameters calculated from the data points)
c2
The c2 pdf changes
its shape depending
the number of
degrees of freedom.
p(c2,n)
function for different
degrees of freedom (v=n)
For n  20, P(c2 > y) can be
approximated using a Gaussian pdf
with y = (2c2) 1/2 - (2n-1)1/2
c2
Richard Kass/Sp12
P416 Lecture 6
2
Example: We have 10 data points. How many degrees of freedom (n) do we have?
Let m and s be the mean and standard deviation of the data.
dof= # of degrees of freedom
If we calculate m and s from the 10 data points then n = 8.
If we know m and calculate s then n = 9.
If we know s and calculate m then n = 9.
max # of degrees of freedom = # of data points
If we know m and s then n = 10.
Example: We collected N events in an experiment and histogram the data
in m bins before performing a fit to the data points.
F We have m data points!
Example: We count cosmic ray events in 15 second intervals and sort the data into 5 bins:
Number of counts in 15 second intervals
0
1
2
3
4
Number of intervals with above counts
2
7
6
3
2
Although there were
36 cosmic rays in our
sample we have only
m=5 data points.
Suppose we want to compare our data with the expectations of a Poisson distribution (P):
P(0)=# intervals with 0 counts
e  m m
P( )  A
P(1)=# intervals with 1 counts
!
In order to do this comparison we need to know A, the total number of intervals (=20 for this example).
Since A is calculated from the data, we give up one degree of freedom: A=2+7+6+3+2=20
dof=m-1=5-1=4.
If we also calculate the mean (m) of our Possion from the data we lose another dof:
0  2  1 7  2  6  3  3  4  2
dof=m-1=5-2=3.
m
 1.8
2 7  63 2
Number of counts in 15 second intervals
0
1
2
3
4
Predicated Number of intervals with above counts
3.3
6.0
5.3
3.2
1.4
Richard Kass/Sp12
P416 Lecture 6
3
Like the Gaussian probability distribution, this integral can not be done in closed form:


1
2 n / 2 1  c 2 / 2
2
[
c
]
e
d
c
n/2
G(n / 2)
a2
P( c  a )   p(c , n)dc  
2
a
2
2
We must use a table to find the probability of exceeding a given c2 for a given number of dof.
Example: What’s the probability to have c2 10 with
the number of degrees of freedom n = 4?
Using Table D of Taylor or the graph on the right:
Taylor P292 - 3 : c~ 2  10 / 4  2.5 P(c2  10, n = 4) = 0.04.
Thus we say that the probability of getting a c2  10
with 4 degrees of freedom by chance is 0.04 (4%).
If instead of 4 degrees of freedom we only had
one degree of freedom then the probability of
getting a c2  10 with 1 degree of freedom by chance
is ~ 0.001 (0.1%). (Low probability of data being from a Poisson)
Example: Back to our cosmic ray example from p3.
We can calculate the probability that our data agrees
with the prediction from the Poisson using:
c 
2
data points

i 1
(measured i  predicted i ) 2
 1.03
predicted i
why is this in
the denominator??
 1.03, n = 3) ≈ 0.79.
Thus we say that the probability of getting a c2  1.03
with 3 degrees of freedom by chance is 0.79 (79%). Thus the data “looks” Poisson!
P(c2
Richard Kass/Sp12
P416 Lecture 6
4
Least Squares Fitting
is sometimes called c2 fitting
l Suppose we have n data points (xi, yi, si).
u Assume that we know a functional relationship between the points,
y  f (x,a,b...)
n Assume that for each yi we know xi exactly.
n The parameters a, b, … are constants that we wish to determine from out data.
u A procedure to obtain a and b is to minimize the following c2 with respect to a and b.

n [y  f (x ,a,b)]2
2
i
c  i
2
Least Squares Fitting (Taylor Ch 8)
i1
si
This is very similar to the Maximum Likelihood Method.
For the Gaussian case MLM and LS are identical.
Technically this is a c2 distribution only if the y’s are from a Gaussian distribution.

Since most of the time the y’s are not from a Gaussian we call it “least squares” rather than c2.
l Example: We have a function with one unknown parameter:
f (x,b) 1bx
Note: in this problem we can find a
Find b using the least squares technique.
closed form solution for b. However,
in general f(x,a,b) will be of the
u We need to minimize the following:
form that does not yield a closed
n [y  f (x ,a,b)]2
n [y 1 bx ]2
2
i
i
i
i
form solution. Some examples:
c 


2
2
n
i1
u

si
si
i1
To find the b that minimizes the above function, we do the following:
c 2  n [yi 1 bxi ]2 n 2[yi 1 bxi ]xi
 

0
2
2
b b i1
si
si
i1
n

yi xi
2
i1 s i
Richard Kass/Sp12
n

xi
2
i1s i
n

bxi2
2
i1 s i
0
P416 Lecture 6
f(x,a,b)=a*sin(bx)
f(x,a,b)=a+e-b
In these cases we must minimize c2
with respect to a & b
using a numerical procedure.
In labs 6 and 7 we will deal with
this situation.
5
n

Solving for b we find:
b
yi xi
2
i1 s i
n
n

xi
2
i1s i
xi2
 2
i1s i
Here each measured data point (yi) is allowed to have a different standard deviation (si).
The LS technique can be generalized to two or more parameters for simple and
complicated (e.g.
 non-linear) functions.
u One especially nice case is a polynomial function that is linear in the unknowns (ai):
f (x,a1...an )  a1  a2 x  a3 x 2  an x n1
n We can always recast problem in terms of solving n simultaneous linear equations.
We use the techniques from linear algebra and invert an n x n matrix to find the ai‘s!
l Example: Given the following data perform a least squares fit to find the value of b.

f (x,b) 1bx
u

u
x
1.0
2.0
3.0
4.0
y
2.2
2.9
4.3
5.2
s
0.2
0.4
0.3
0.1
Using the above expression for b we calculate:
b = 1.05
u
Next is a plot of the data points and the line from the least squares fit:
Richard Kass/Sp12
P416 Lecture 6
6
f (x,b) 1bx
b = 1.05

y
x
u
If we assume that the data points are from a Gaussian distribution,
we can calculate a c2 and the probability associated with the fit.
n [y 11.05x ]2 2.2  2.05 2 2.9  3.1 2 4.3  4.16 2 5.2  5.2 2
2
i
c  i
 
  
  
  
  1.04
2








0.2
0.4
0.3
0.1
si
i1
n

n
From Table D of Taylor:
The probability to get c2 1.04 for 3 degrees of freedom ≈ 80%.
We call this a "good" fit since the probability is close to 100%.
If however the c2 was large (e.g. 15),
the probability would be small (≈ 0.2% for 3 dof).
We would say this was a “bad” fit.
RULE OF THUMB
A “good” fit has c2 /dof ≤ 1
Richard Kass/Sp12
P416 Lecture 6
7
Some not so good things about the c2 distribution and test:
Given a set of data points two different functions can have the same value of c2.
The method does not specify a unique form of the solution or function
How many parameters should we fit for ??
We will get a different value for slope (b) if we fit 1+bx instead of a+bx
The value of c2 does not look at the order of the data points.
The c2 distribution ignores trends in the data points.
The distribution ignores the sign of differences between the data points and “true” values.
It only uses the square of the differences.
There are other distributions/statistical tests that are more powerful than c2 test:
“run tests” and “Kolmogorov test”
These tests do use the order of the points.
These tests can be harder to use or require more numerical processing than the c2 technique.
Richard Kass/Sp12
P416 Lecture 6
8