Download ada13

Robust estimation methods
• Robust estimation methods
are relatively insensitive to
mean X
“bad” data.
median XM
• Example: using median
rather than mean:
 N


2
• Mean X minimizes the sum X satisfies
 (Xi  A)  0
A i1

of squared errors (SSE):
• Median XM minimizes the

  N
XM satisfies
sum of absolute errors
 Xi  A  0
A i1

(SSE):
• Proof: use Heaviside step
1, x > 0
function:



1
–1
H(x)  0, x  0

1, x  0
H' (x)  2 (x)
How the median minimises SAE
• Need to show that the median minimises the
expected value of the sum of absolute errors:
X  X M  X  X M H X  X M 

X M
X  XM 

X M
X  XM H X  X M 
 H X  X M   X  X M H' X  X M 
 P(X  X M )  P(X  X M )
 0 when X M is the median.
0
(since X-XM=0
when H’ = 1 and
H’=0 for all other
values of X - XM)
Mean and median
• The median is less
sensitive to outliers than
the mean.
Mean
Median
• Although the median is
unbiased it is not a
minimum-variance
estimator.
• Note how standard
deviation of the median
varies with sample size in
comparison to standard
deviation of the mean.
Median
Mean
Evaluation of median without sorting
Xi  XM
Xi  XM
1
0
-1
• A useful application:
N
Since

i1
Xi  XM
 0, first make a guess at XM .
Xi  XM
N
Then estimate a new XM 

i1
N

i1
and iterate to convergence.
Xi
Xi  XM
,
1
Xi  XM
Median filtering and sigma-clipping
• Median filtering: take window of N points
• Replace central point by median of the N
points.
Window
• Sigma-clipping:
–
–
–
–
Fit all points by minimising 2
Set threshold K and check for outliers at ± K or more
Repeat fit omitting largest outlier
Iterate until set of rejected points converges.
Reject
Reject
Using 2min to reject models
• Suppose we fit M parameters to N data points:
2
 min

yi  f (xi ; P1 ...PM ) 2
2
  
 ~  N M
i

i1 
2
NM
N
 NM
 (  N2 M )  2(N  M)
• We use N-M because an N - parameter fit
should fit N points exactly.
• If model is good, then the best-fit 2min should
be:
2
 min
 N  M  2(N  M)
What if 2min is too high?
• Several possibilities to consider:
– Statistical fluke – use 2 distribution to estimate probability
– Wrong model – use 2 distribution to reject model
– Right model, but additional (nuisance) parameters not
correctly chosen:
x
– Error bars too small. Re-scale.
• The third possibility adds a constant to 2min .
• Can still use 2min +1 to set 1- confidence
intervals on parameter values.
x
Diagnosis of 2min ≠ N-M ±√2(N-M)
N–M
2=1
2min
N–M
2= 2min /(N–M) >1
2= 2min /(N–M) < 1
2min
N–M
ˆ

2min > N–M due to unimportant
parameters omitted from model
ˆ

2min > N–M due to underestimation of error bars
2min < N–M due to overestimation of error bars
If this happens
use these
values to estimate
errors on parameters