Download Likelihood based confidence intervals

Likelihood based confidence intervals
Christine Borgen Linander
A huge thank to a former colleague of mine Rune H B Christensen.
DTU Compute
Section for Statistics
Technical University of Denmark
[email protected]
August 20th 2015
© Christine Borgen Linander (DTU)
Likelihood based confidence intervals
Sensometrics Summer School
1 / 15
© Christine Borgen Linander (DTU)
Outline
We are interested in:
2
Examples
3
The likelihood function and confidence intervals
Examples revisited
5
Perspectives
© Christine Borgen Linander (DTU)
2 / 15
Motivation
Motivation
4
Sensometrics Summer School
Motivation
Outline
1
Likelihood based confidence intervals
1
Which sensory difference (d 0 ) is most supported by the data?
2
Which interval of sensory differences is supported by the data?
We usually answer those with:
1
The maximum likelihood estimate (MLE), d̂ 0
2
A confidence interval (CI) for d 0
. . . but there are many ways to compute the CI, and which is best?
Likelihood based confidence intervals
Sensometrics Summer School
3 / 15
© Christine Borgen Linander (DTU)
Likelihood based confidence intervals
Sensometrics Summer School
4 / 15
Motivation
Examples
Problems with standard CIs
John and Dorothy’s duo-trio experiment
The guessing probability is 1/2
They obtain 13 correct answers to 20 samples
Standard (Wald) 95% confidence intervals:
John analyzes the probability of a correct answer, pc :
p̂c = 0.65(0.11) and CI95% = [0.44; 0.86] which covers pc = 1/2
For binomial probability of a correct answer pc : p̂c ± 1.96 · se(p̂c )
For the Thurstonian δ: δ̂ ± 1.96 · se(δ̂) (Bi et al, 1997)
Dorothy analyzes the Thurstonian δ:
δ̂ = 1.42(0.63) and CI95% = [0.18; 2.66] (Bi et al, 1997) which does
NOT cover δ = 0
Problems and solution:
The standard CIs are incompatible and lead to contradictions
The standard CIs do not cover the values of δ or pc that are most
supported by the data
Which method is (most) correct?
CIs based on the likelihood function have better properties
Should we trust John or Dorothy?
How much evidence is there really in the data about a difference between
the products?
© Christine Borgen Linander (DTU)
Likelihood based confidence intervals
Sensometrics Summer School
5 / 15
© Christine Borgen Linander (DTU)
Examples
6 / 15
Properties of the likelihood function
Likelihood function
= density:
L(δ; x , n) = nx p x (1 − p)n−x , p = fpsy (δ)
The guessing probability is 1/3
They obtain 10 correct answers to 20 samples
Measures support of values of δ relative to δ̂
Peter analyzes the probability of a correct answer, pc :
p̂c = 1/2(0.11) and CI95% = [0.28; 0.72] which covers pc = 1/3
An objective way to measure information in the data about δ
Sally analyzes the Thurstonian δ:
δ̂ = 1.47(0.59) and CI95% = [0.32; 2.62] (Bi et al, 1997) which does
NOT cover δ = 0
duo−trio: x= 13 n= 20
1.0
Should we trust Peter or Sally?
The maximum likelihood
estimate (MLE)
0.8
Relative likelihood
Which method is (most) correct?
0.6
Likelihood CIs are given by
horizontal lines
0.4
Symmetric approximation
produces standard (Wald) CIs
proposed by Bi et al (1997)
0.2
How much evidence is there really in the data about a difference between
the products?
Likelihood based confidence intervals
Sensometrics Summer School
The likelihood function and confidence intervals
Peter and Sally’s triangle experiment
© Christine Borgen Linander (DTU)
Likelihood based confidence intervals
Sensometrics Summer School
7 / 15
0.0
0
1
^
δ
2
3
4
δ
© Christine Borgen Linander (DTU)
Likelihood based confidence intervals
Sensometrics Summer School
8 / 15
The likelihood function and confidence intervals
The likelihood function and confidence intervals
Properties of the likelihood function
Properties of the likelihood function
Likelihood function
= density:
L(δ; x , n) = nx p x (1 − p)n−x , p = fpsy (δ)
Likelihood function
= density:
L(δ; x , n) = nx p x (1 − p)n−x , p = fpsy (δ)
Measures support of values of δ relative to δ̂
Measures support of values of δ relative to δ̂
An objective way to measure information in the data about δ
An objective way to measure information in the data about δ
duo−trio: x= 13 n= 20
duo−trio: x= 13 n= 20
1.0
1.0
The maximum likelihood
estimate (MLE)
0.6
Likelihood CIs are given by
horizontal lines
0.4
0.2
Symmetric approximation
produces standard (Wald) CIs
proposed by Bi et al (1997)
95% limit
99% limit
0.0
0
1
2
3
0.6
Likelihood CIs are given by
horizontal lines
0.4
0.2
Symmetric approximation
produces standard (Wald) CIs
proposed by Bi et al (1997)
95% limit
99% limit
0.0
4
0
1
2
δ
3
4
δ
© Christine Borgen Linander (DTU)
Likelihood based confidence intervals
Sensometrics Summer School
8 / 15
© Christine Borgen Linander (DTU)
Likelihood based confidence intervals
Examples revisited
Sensometrics Summer School
John and Dorothy’s duo-trio example revisited (2)
“No difference” between the products has reasonably high likelihood
The symmetric approximations are inaccurate
An intermediate difference between products is most likely
Neither John’s nor Dorothy’s CIs are appropriate
A large difference between products is unlikely
Likelihood inference for δ and pc is compatible
duo−trio: x= 13 n= 20
duo−trio: x= 13 n= 20
0.8
0.8
0.8
0.4
0.2
95% limit
Relative likelihood
1.0
Relative likelihood
1.0
0.6
0.4
0.2
0.0
1
2
3
4
0.4
95% limit
0.5
0.6
0.7
Likelihood based confidence intervals
0.8
pc
0.9
John
0.8
0.6
0.4
0.2
95% limit
99% limit
99% limit
0.0
δ
© Christine Borgen Linander (DTU)
0.6
99% limit
0.0
0
Dorothy
0.2
95% limit
99% limit
duo−trio: x= 13 n= 20
1.0
Relative likelihood
duo−trio: x= 13 n= 20
1.0
0.6
8 / 15
Examples revisited
John and Dorothy’s duo-trio example revisited
Relative likelihood
The maximum likelihood
estimate (MLE)
Dorothy
0.8
Relative likelihood
Relative likelihood
0.8
1.0
0.0
0
1
2
3
4
0.5
0.6
0.7
δ
Sensometrics Summer School
9 / 15
© Christine Borgen Linander (DTU)
Likelihood based confidence intervals
0.8
pc
0.9
1.0
Sensometrics Summer School
10 / 15
Examples revisited
Examples revisited
Peter and Sally’s triangle example revisited (1)
Peter and Sally’s triangle example revisited (2)
“No difference” between the products has reasonably high likelihood
The likelihood curve tells the full story about the data
An intermediate difference between products is most likely
The likelihood curve illustrates the effect of confidence level
A large difference between products is unlikely
triangle: x= 10 n= 20
1.0
0.8
0.8
0.6
0.4
0.2
0.6
0.4
1
2
3
0.8
0.6
0.4
0.0
99% limit
4
95% limit
0.5
0.6
δ
0.7
0.8
0.9
0.4
0.2
95% limit
99% limit
0.0
0
0.4
0.6
99% limit
0.0
0
0.8
95% limit
99% limit
0.0
1.0
0.2
0.2
95% limit
triangle: x= 10 n= 20
1.0
Relative likelihood
1.0
Relative likelihood
triangle: x= 10 n= 20
Relative likelihood
Relative likelihood
triangle: x= 10 n= 20
1
2
3
4
0.4
0.5
0.6
δ
1.0
0.7
pc
0.8
0.9
1.0
pc
© Christine Borgen Linander (DTU)
Likelihood based confidence intervals
Sensometrics Summer School
11 / 15
© Christine Borgen Linander (DTU)
Examples revisited
Likelihood based confidence intervals
Sensometrics Summer School
12 / 15
Perspectives
Peter and Sally’s triangle example revisited (3)
Coverage probability
The symmetric approximation for δ is quite inaccurate
Sally’s CI is very misleading
Boyles (2008) showed that likelihood CIs have the best coverage
probability among common CIs for the binomial p.
triangle: x= 10 n= 20
triangle: x= 10 n= 20
1.0
Sally
0.6
0.4
0.2
95% limit
Peter
0.8
Relative likelihood
0.8
Relative likelihood
Coverage probability in % for the binomial p
with a nominal level of 95% (Boyles, 2008)
n
Standard Exact Likelihood
10
76.9
98.4
94.9
50
90.1
96.9
95.0
100
92.2
96.5
95.0
500
94.3
95.7
95.0
1.0
0.6
0.4
0.2
95% limit
99% limit
0.0
99% limit
0.0
0
1
2
3
4
0.4
0.5
0.6
δ
© Christine Borgen Linander (DTU)
Likelihood based confidence intervals
0.7
pc
0.8
0.9
1.0
Sensometrics Summer School
13 / 15
© Christine Borgen Linander (DTU)
Likelihood based confidence intervals
Sensometrics Summer School
14 / 15
Perspectives
Likelihood methods in discrimination testing
Likelihood — a common framework for:
Estimation
Testing
Confidence intervals
Maximum likelihood
Likelihood ratio test
Profile likelihood
Gracefully handle boundary cases
Likelihood methods extend to complex situations
© Christine Borgen Linander (DTU)
Likelihood based confidence intervals
Sensometrics Summer School
15 / 15