Download The Logic of Hypothesis Testing

TheLogicofHypothesisTesting
Howdoweknowifourdata
differsfromapopulation?
TheLogicofHypothesisTesting
Let’ssupposethemeanageforbabieswalkingis14months.
Wearegoingtotryanexperimentalinterventionthatseeksto
reducethatage.
Sowegeneratetwohypotheses:
1)Ourtreatmentdoesnotwork
2)Ourtreatmentworks
Letsthinkofthisnowintermsofpopulations:
PopulationOne,babieswhodidnotgetourtreatment
PopulationTwo,babieswhodidgetourtreatment
Andwecanquantifyadescriptivestatisticthatisrepresentative
ofourpopulation,themeanageatwhichtheybeginwalking
PopulationOne:meanageofwalking:μ1
PopulationTwo:meanageofwalking:μ2
Wetypicallyframethiswithinthecontextofthenulland
alternativehypotheses:
1)TheNullHypothesis:Ourtreatmentdoesnotwork:
2)μ1:TheAlternativeHypothesis:Ourtreatmentworks:
μ1=μ2
μ1≠μ2
x2
x1
So,let’ssaywerunthestudy,andweobtainthefollowingdata.
€
€
Howdoweknowifourtreatmentworked?
x2
x1
Here’stheweirdbit,wefirstwanttofindouttheprobabilityof
gettingourresultifthenullhypothesisistrue.
€
€
Wecallthisour“criticalvalue”
Theprobabilityofourresultsatwhichwewilldiscardthenull
hypothesisandacceptthealternativehypothesis.
x2
€
€
x1
0.05
Thepointatwhichasamplescoreis
soextremethatwediscardthenullhypothesis
Todothisweneedtoknowthesampling
distributiontoperformstatisticaltests
Thesamplingdistributionofthemean
providesallofthevaluesthemeancan
take,alongwiththeprobabilityofgetting
eachvalueifsamplingisrandomfromthe
null-hypothesispopulation
Considerthefollowing…
Imagineyouhaveapopulationthatconsistsof5
scores:
2,3,4,5,6
Ifyoutakesamplesofsize2,howmanydifferent
possiblesamplesarethereandwhatdothey
consistof?
Sample
1
2
3
4
5
6
7
8
9
10
11
12
13
2,2
2,3
2,4
2,5
2,6
3,2
3,3
3,4
3,5
3,6
4,2
4,3
4,4
Sample
14
15
16
17
18
19
20
21
22
23
24
25
4,5
4,6
5,2
5,3
5,4
5,5
5,6
6,2
6,3
6,4
6,5
6,6
Thesamplingdistributionofthemean
providesallofthevaluesthemeancan
take,alongwiththeprobabilityofgetting
eachvalueifsamplingisrandomfromthe
null-hypothesispopulation
Sample
x
Sample
1
2
3
4
5
6
7
8
9
10
11
12
13
2
2.5
3
3.5
4
2.5
3
3.5
4
4.5
3
3.5
4
14
15
16
17
18
19
20
21
22
23
24
25
2,2
2,3
2,4
2,5
2,6
3,2
3,3
3,4
3,5
3,6
4,2
4,3
4,4
x
4,5
4,6
5,2
5,3
5,4
5,5
5,6
6,2
6,3
6,4
6,5
6,6
4.5
5
3.5
4
4.5
5
5.5
4
4.5
5
5.5
6
Thesamplingdistributionofthemean
providesallofthevaluesthemeancan
take,alongwiththeprobabilityofgetting
eachvalueifsamplingisrandomfromthe
null-hypothesispopulation
Sample
x
Sample
1
2
3
4
5
6
7
8
9
10
11
12
13
2
2.5
3
3.5
4
2.5
3
3.5
4
4.5
3
3.5
4
14
15
16
17
18
19
20
21
22
23
24
25
2,2
2,3
2,4
2,5
2,6
3,2
3,3
3,4
3,5
3,6
4,2
4,3
4,4
x
4,5
4,6
5,2
5,3
5,4
5,5
5,6
6,2
6,3
6,4
6,5
6,6
4.5
5
3.5
4
4.5
5
5.5
4
4.5
5
5.5
6
Probabilityofgettingthemean
p ()=2.0
x
1/25
0.04
Probabilityofgettingthemean
p()=2.0
x
p()=2.5
x
p()=3.0
x
p()=3.5
x
p()=4.0
x
x
p()=4.5
p()=5.0
x
x
p()=5.5
x
p()=6.0
1/25
2/25
3/25
4/25
5/25
4/25
3/25
2/25
1/25
0.04
0.08
0.12
0.16
0.20
0.16
0.12
0.08
0.04
Samplingdistributionofthemean
x
p()
x
2
0.04
2.5
0.08
3
0.12
3.5
0.16
4
0.20
4.5
0.16
5
0.12
5.5
0.08
6
0.04
CharacteristicsoftheSamplingDistribution
oftheMean
1. Thesamplingdistributionofthemeanhasa
sx
ux
mean()andastandarddeviation()
Notethanisalsocalledthe
sx
standarderrorofthemean
CharacteristicsoftheSamplingDistribution
oftheMean
2. Themeanofthesamplingdistributionofthe
meanisthesameasthemeanofthe
population.
ux = u
CharacteristicsoftheSamplingDistribution
oftheMean
3. Thestandarddeviationofthesampling
distributionofthemeanis:
sx =
s
n
CharacteristicsoftheSamplingDistribution
oftheMean
4. Thesamplingdistributionofthemeanis
normallyshaped(usually)
CharacteristicsoftheSamplingDistribution
oftheMean
4. Thesamplingdistributionofthemeanis
normallyshaped(usually)
TheCentralLimitTheorem
REGARDLESSoftheshapeofthepopulationof
rawscores,thesamplingdistributionofthe
meanapproachesanormaldistributionas
samplesizeNincreases
TheBinomialDistribution
TheProblem…
Thereisnowaytoknowthesampling
distributionofthemeanforanydatathatwe
collect… sowhatdowecompareoursample
meanto???
Whatdowedo?
x2
x1
Wecouldjustusethenormaldistribution…
€
€
WHY?
1.Weknowthatthesamplingdistributionofthemeanforany
variableisnormalgivenareasonablesamplesize
2.Weknowthevaluesandprobabilitiesofthenormaldistribution
Wecouldcomparethedataagainstapopulationwithmeanof0andstandard
deviationof1– thenormaldistribution… bydoingthis,wegettheprobability
thatthemeanofourdatacomesfromthenormaldistribution.But…
But…
Thenormaldistributionisdeadtouseunlesswe
knowthestandarddeviationofthepopulation…
z=samplemean– populationmean
------------------------------------------populationstandarddeviation
Sowhatnow?
Whatdowedo?
Wecomputesomestatisticsfromourdata for
whichthereisaknownprobabilityofscores
givenaspecificsamplesize.
t=samplemean– populationmean
------------------------------------------standarddeviationofsample
x2
x1
Sowhatarewedoingwithz scores,t scores,Fratios,etc…
€
€
Wearecomputingastatisticwhichisoursamplesscoreon
thesamplingdistributionofthenullhypothesis…
AndWEASSUMEthesemagicaldistributionsapproximatethe
samplingdistributionofthemeanofourdata…
Asstated,todrawtheseconclusionsweneeda
teststatistictoevaluationagainsttheappropriate
samplingdistribution.
TestStatistic= varianceexplainedbymodel
-------------------------------------variancenotexplainedbymodel
TestStatistic= effect
-------error
Andthenwecancalculatetheprobabilityof
gettingtheteststatisticwehaveobtainedand
evaluatingitagainstalpha.
alpha=0.05
ifp<0.05thenwesayourteststatisticisdifferent
Ifp>0.05thenwesayourteststatisticisthesame
OtherThingsToThinkAbout
OneversusTwoTailedSignificance
TypeIandTypeIIErrors
TypeIError
Rejectthenullhypothesiswhenitistrue.
TypeIIError
Retainthenullhypothesiswhenitisfalse.
StateofReality
Decision
Hoistrue
Hoisfalse
RetainHo
Correct
Decision
TypeIError
α
TypeIIError
β
Correct
Decision
RejectHo
ControllingforthepossibilityofTypeIandType
IIerrorsisnoteasy...
Ifwereducealphatoreducethechanceofa
TypeIerror,weincreasethelikelihoodthatwe
aremakingaTypeIIerror!