Download lecture_3

Theory of tests
Thomas INGICCO
1 – Principle of tests (hypotheses)
2 – A first example with R
3 – Probabilities (law of random)
4 – Statistic and its law (test variable)
5 – Risk α and errors
6 – p-value
7 – Summary
E. Munch, The scream
Theory of tests
Theory of tests
Example 1
Let’s consider dices. I roll dices 4 times and obtain 4 times different results -> I
deduce that the dices are not fake.
So I am doubting now and want to check. So I roll the dices once again 4 times
and I obtain 4 times the same result -> I change my opinion and conclude
that the dices are fake.
How do I know the truth? I only have 8 rolls so a small sample for an infinite
population… -> I need a statistical test.
Theory of tests
Example 1
Let’s consider dices. I roll dices 4 times and obtain 4 times different results -> I
deduce that the dices are not fake.
So I am doubting now and want to check. So I roll the dices once again 4 times
and I obtain 4 times the same result -> I change my opinion and conclude
that the dices are fake.
How do I know the truth? I only have 8 rolls so a small sample for an infinite
population… -> I need a statistical test.
Example 2
I study two collections of lithic tools, one from northern Luzon, the other one
from Mindoro.
I sample 20 tools in each population. The mean length of the 20 artefacts in the
first population is 110 mm. In the second population it is 130mm.
May I deduce that the lithic tools from Mindoro are longer and then that the
prehistoric group are: 1) Different, 2) Have different abilities to knap,
3)Produce tools with different interests ?
If I do so, what is my risk to be wrong? -> I need a statistical test
What do we test and why do we test?
Investigation or inventory : All the operations aiming at collecting in a systematic
way, data or informations related to a group of individual or elements.
What do we test and why do we test?
Investigation or inventory : All the operations aiming at collecting in a systematic
way, data or informations related to a group of individual or elements.
Individual : Element = Statistical unit = Basal unit = what we sample
What do we test and why do we test?
Investigation or inventory : All the operations aiming at collecting in a systematic
way, data or informations related to a group of individual or elements.
Individual : Element = Statistical unit = Basal unit = what we sample
Population : Universe = Statistic assemblage = All the individuals
What do we test and why do we test?
Investigation or inventory : All the operations aiming at collecting in a systematic
way, data or informations related to a group of individual or elements.
Individual : Element = Statistical unit = Basal unit = what we sample
Population : Universe = Statistic assemblage = All the individuals
Sample : Part of the population that we observe
What do we test and why do we test?
Investigation or inventory : All the operations aiming at collecting in a systematic
way, data or informations related to a group of individual or elements.
Individual : Element = Statistical unit = Basal unit = what we sample
Population : Universe = Statistic assemblage = All the individuals
Sample : Part of the population that we observe
What do we test and why do we test?
Investigation or inventory : All the operations aiming at collecting in a systematic
way, data or informations related to a group of individual or elements.
Individual : Element = Statistical unit = Basal unit = what we sample
Population : Universe = Statistic assemblage = All the individuals
Sample : Part of the population that we observe
Probability
What do we test and why do we test?
Investigation or inventory : All the operations aiming at collecting in a systematic
way, data or informations related to a group of individual or elements.
Individual : Element = Statistical unit = Basal unit = what we sample
Population : Universe = Statistic assemblage = All the individuals
Sample : Part of the population that we observe
Probability
Statistic
What do we test and why do we test?
Investigation or inventory : All the operations aiming at collecting in a systematic
way, data or informations related to a group of individual or elements.
Individual : Element = Statistical unit = Basal unit = what we sample
Population : Universe = Statistic assemblage = All the individuals
Sample : Part of the population that we observe
Observation: Measured value for each sampled individual
The principle of test
First step :
• We define the hypotheses: H0 and H1.
H0 = Null hypothesis
H1 = Alternative hypothesis
Most of the time, H0 defines an absence of structure in the data.
Example 1: Chi2 Test
H0 = Independance between two qualitative variables
H1 = Dependance
Example 2: Comparison of a mean to a theoretical mean
H0 = Means are equal
H1 = Means are different
H1 = The observed mean is bigger than the theoretical mean
H1 = The observed mean is smaller than the theoretical mean
Example 3: Comparison of a mean to a theoretical mean
H0 = The difference between the means is of 5cm
H1 = The difference between the means is not of 5cm
The principle of test
Nul hypothesis H0 versus Alternative hypothesis H1
We start by considering that H0 is True.
We test H0:
1- If the test determines that H0 is False, then I accept H1 (we reject
H0)
2- If my test does not determine that H0 is False, I cannot reject
this hypothesis (we keep H0)
-> A theory is replaced by another one only if it can be rejected.
The principle of test
Second step :
• Once the hypotheses H0 and H1 defined, we determine the variable
of the test (the statistic) we need to reject H0.
It exists two categories of tests:
- Parametric tests: The distribution of probability of the measured variable in
the targeted population follows a law. The analysed data can be modelised
according to a known law. Then these tests will focus on the parameters of
this law (mean, variance,…).
- Non-parametric tests (or distribution free test): The distribution of the
measured variable does not matter. These tests do not focus on the
distribution parameters of the data. They are used for small samples (>10)
and/or for variables that do not follow a normal law.
The principle of test
Second step :
• Once the hypotheses H0 and H1 defined, we determine the variable
of the test (the statistic) we need to reject H0.
In the case of Chi2, the statistic is:
𝜒² =
(𝑂𝑖𝑗 − 𝐸𝑖𝑗 )²
𝑖, 𝑗
𝐸𝑖𝑗
In the case of the comparison of a mean to a theoretical mean, the statistic is:
𝑇=
𝑋 − 𝜇0
𝜎²/𝑛
The principle of test
Why to calculate a statistic?
-> Because we know the law of probability of this statistic under the
hypothesis H0, meaning if H0 is true.
This is this law of probability that would permit us to determine if «
there are some chances » that we obtained these data if H0 is true, or
in the opposite, if « there are very few chances » that we obtained
these data if H0 is true. In this second case, we will deduce that
according to our data, H0 is most probably False.
Let’s have an example !
How to make a test
Concretly, how to realize a test in R?
1- Determination of the hypotheses H0 and H1 which permits to clarify
the scientific question.
2- Choice of the test, search for the function in R.
3- Use the help for the test, so you know how to make the test and how
it works. In other words, you understand what you do, which is not the
case in Excel.
Example 1:
Question: Is the high of a ceramic correlated with the width of its
openning?
How to make a test
Ceram <read.table("K:/Cours/Philippines/Statistic
s-210/Data/Ceramics.txt",header=TRUE)
str(Ceram)
plot(Ceram$W.mouth,Ceram$H.rim)
Write the following instructions in R
window.
Take the time to understand the
instructions. Do not forget to use the
help in R and to comment the
meaning of each instruction.
How to make a test
Ceram <read.table("K:/Cours/Philippines/Statistic
s-210/Data/Ceramics.txt",header=TRUE)
str(Ceram)
plot(Ceram$W.mouth,Ceram$H.rim)
par(mfrow=c(1, 2))
hist(Ceram$W.mouth)
hist(Ceram$H.rim)
Write the following instructions in R
window.
Take the time to understand the
instructions. Do not forget to use the
help in R and to comment the
meaning of each instruction.
How to make a test
Ceram <read.table("K:/Cours/Philippines/Statistic
s-210/Data/Ceramics.txt",header=TRUE)
str(Ceram)
plot(Ceram$W.mouth,Ceram$H.rim)
par(mfrow=c(1, 2))
hist(Ceram$W.mouth)
hist(Ceram$H.rim)
apropos("test")
help.search("test")
?cor.test
Write the following instructions in R
window.
Take the time to understand the
instructions. Do not forget to use the
help in R and to comment the
meaning of each instruction.
How to make a test
Ceram <read.table("K:/Cours/Philippines/Statistic
s-210/Data/Ceramics.txt",header=TRUE)
str(Ceram)
plot(Ceram$W.mouth,Ceram$H.rim)
par(mfrow=c(1, 2))
hist(Ceram$W.mouth)
hist(Ceram$H.rim)
apropos("test")
help.search("test")
?cor.test
cor.test(Ceram$W.mouth,Ceram$H.rim)
Write the following instructions in R
window.
Take the time to understand the
instructions. Do not forget to use the
help in R and to comment the
meaning of each instruction.
Understand the result of a test
The P-value – a question of probabilities
I flip 50 times a coin. Is the coin fake?
We consider a statistic:
X= the number of times Heads is chosen in my experiment.
H0= the coin is not fake.
H1= the coin is fake.
If H0 is True, what should be the value of X? Meaning how many times
Heads should be obtained in 50 flips?
Is it 25?
Understand the result of a test
The P-value – a question of probabilities
I flip 50 times a coin. Is the coin fake?
We consider a statistic:
X= the number of times Heads is chosen in my experiment.
H0= the coin is not fake.
H1= the coin is fake.
If H0 is True, what should be the value of X? Meaning how many times
Heads should be obtained in 50 flips?
Is it 25?
-> NO. In 50 flips, I may obtain between 0 and 50 Heads, but, if the
coin is not fake, the probability (the chance) to obtain 0 Heads or 50
Heads is low, while the probability to obtain 23, 24, 25, 26, 27 Heads is
stronger.
Understand the result of a test
The P-value – a question of probabilities
Write the following instructions in R
window. We are going to estimate the law of
probability of X from 1000 experiments. One
experiment correspond to 50 flips.
Write the following instructions in R
window.
Take the tim to understand the
instructions. Do not forget to use the
help of R and to take notes on the
meaning of each instruction.
# statfun is a function that makes an experiment (50 flips of a coin) and returns
the number of Heads obtained.
coin <- c("T", "H")
statfun <- function(i){
simu <- sample(coin, 50, replace = TRUE, prob = c(0.5, 0.5))
# nbHeads <- table(simu)[1]
nbHeads <- table(simu)[2]
return(nbHeads)
}
res4 <- sapply(1:1000, statfun)
res4
Understand the result of a test
The P-value – a question of probabilities
table(res4)
names(table(res4))
as.numeric(names(table(res4)))
Head <- as.numeric(names(table(res4)))
Probability <- table(res4)/1000
plot(x, y, type="h")
x <- 0:50
y <- dbinom(x, 50, 0.5)
plot(x, y, type = "h")
Write the following instructions in R
window.
Take the tim to understand the
instructions. Do not forget to use the
help of R and to take notes on the
meaning of each instruction.
Binomial law
B(N,p)
B(50,0.5)
Number of selection
Probability of success
We will note that this law depends on two parameters.The probabilities of each
possible event from this plot are true only if p=0.5 and N=50. Meaning that the
probability to obtain Heads for every flip is of 0.5, so 50%, and the number of
flips is 50.
p=50 is True only if the coin is not fake.
k: each possible value within the discrete variable X.
f(k): frequency associated to each value = probability associated to k.
F(k): sum of the probabilities f(k) on the left or right of k regarding our interest.
F(k): it is the probability that X is upper or lower/equal to a value k.
F(k)upper = P(X>k)
F(30) upper = P(X>30)=0.118
cumsum(Probability)
sum(Probability[which(rownames(Probability)=="31"):length(Probability)])
k: each possible value within the continuous variable X.
f(k): probability distribution of X = probability density associated to k.
F(k): area under the curve f(k) on the left or right of k regarding our interest.
F(k): it is the probability that X is upper or lower/equal to a value k.
F(k)upper = P(X>k)
F(xk)lower - F(xj)lower = P(xk<xi≤xk)
Laws of probability
Binomial law
dbinom : probability f(k) of the variable X
pbinom : function of repartition of F(k) of the variable X
qbinom : give the value k of the variable X for a given value of F(k)
rbinom : generates random values for the variable X considering probabilities
Normal law
dnorm : probability f(k) of the variable X
pnorm : function of repartition of F(k) of the variable X
qnorm : give the value k of the variable X for a given value of F(k)
rnorm : generates random values for the variable X considering probabilities
Chi2 law
dchisq : probability f(k) of the variable X
pchisq : function of repartition of F(k) of the variable X
qchisq: give the value k of the variable X for a given value of F(k)
rchisq: generates random values for the variable X considering probabilities
Laws of probability
Binomial law
dbinom : probability f(k) of the variable X
pbinom : function of repartition of F(k) of the variable X
qbinom : give the value k of the variable X for a given value of F(k)
rbinom : generates random values for the variable X considering probabilities
Laws of probability
Binomial law
dbinom : probability f(k) of the variable X
pbinom : function of repartition of F(k) of the variable X
qbinom : give the value k of the variable X for a given value of F(k)
rbinom : generates random values for the variable X considering probabilities
Laws of probability
Binomial law
𝑃 𝑋 = 𝑘 = 𝐶𝑛
𝑘
× 𝑝𝑘 × (1 − 𝑝)𝑛−𝑘
X: random variable
P(X=k): function of repartition the variable X, so f(k)
𝑘
𝐶𝑛 : Combination of order k of the n elements
p: probability of the event 1 black ball for one selection, so 0 ≤ 𝑘 ≤ 𝑛
n: number of selection with replacement 1 ≤ 𝑛 ≤ +∞
K: number of black balls
Laws of probability
Binomial law
𝑃 𝑋 = 𝑘 = 𝐶𝑛
𝑘
× 𝑝𝑘 × (1 − 𝑝)𝑛−𝑘
X: random variable
P(X=k): function of repartition the variable X, so f(k)
𝑘
𝐶𝑛 : Combination of order k of the n elements
p: probability of the event 1 black ball for one selection, so 0 ≤ 𝑘 ≤ 𝑛
n: number of selection with replacement 1 ≤ 𝑛 ≤ +∞
K: number of black balls
The binomial law is grounded on two exclusive elements, « black » and « white » for
example when starting a chess game; or « boy » and « girl »; or « yes » or « no » when
you date a girl.
Each of these events is associated to a probability of appearance.
The binomial law gives the probability corresponding to many apparitions.
You have 50 balls in a bag: 10 black, 40 red.
The probability to have a random black ball after 1 selection is:
𝑝 = 𝑃(1 𝑏𝑙𝑎𝑐𝑘 1 𝑠𝑒𝑙𝑒𝑐𝑡𝑖𝑜𝑛)
If the selection is randomly done, then the probability is p=10/50=0.2
Laws of probability
Binomial law
𝑃 𝑋 = 𝑘 = 𝐶𝑛
𝑘
× 𝑝𝑘 × (1 − 𝑝)𝑛−𝑘
X: random variable
P(X=k): function of repartition the variable X, so f(k)
𝑘
𝐶𝑛 : Combination of order k of the n elements
p: probability of the event 1 black ball for one selection, so 0 ≤ 𝑘 ≤ 𝑛
n: number of selection with replacement 1 ≤ 𝑛 ≤ +∞
K: number of black balls
Let’s calculate the probability to have 2 black balls when selecting 4 balls with
replacement (knowing that the probability to get one black is of 0.2):
p=0.2; n=4; k=2
choose(n,k)*p^k*(1-p)^(n-k)
Laws of probability
Binomial law
𝑃 𝑋 = 𝑘 = 𝐶𝑛
𝑘
× 𝑝𝑘 × (1 − 𝑝)𝑛−𝑘
X: random variable
P(X=k): function of repartition the variable X, so f(k)
𝑘
𝐶𝑛 : Combination of order k of the n elements
p: probability of the event 1 black ball for one selection, so 0 ≤ 𝑘 ≤ 𝑛
n: number of selection with replacement 1 ≤ 𝑛 ≤ +∞
K: number of black balls
Let’s calculate the probability to have 2 black balls when selecting 4 balls with
replacement (knowing that the probability to get one black is of 0.2):
p=0.2; n=4; k=2
choose(n,k)*p^k*(1-p)^(n-k)
dbinom(k, n, p)
Laws of probability
Binomial law
𝑃 𝑋 = 𝑘 = 𝐶𝑛
𝑘
× 𝑝𝑘 × (1 − 𝑝)𝑛−𝑘
X: random variable
P(X=k): function of repartition the variable X, so f(k)
𝑘
𝐶𝑛 : Combination of order k of the n elements
p: probability of the event 1 black ball for one selection, so 0 ≤ 𝑘 ≤ 𝑛
n: number of selection with replacement 1 ≤ 𝑛 ≤ +∞
K: number of black balls
Let’s calculate the probability to have 2 black balls when selecting 4 balls with
replacement (knowing that the probability to get one black is of 0.2):
p=0.2; n=4; k=2
choose(n,k)*p^k*(1-p)^(n-k)
dbinom(k, n, p)
The probability P(X=2) is then of 15.36%
Laws of probability
Binomial law
𝑃 𝑋 = 𝑘 = 𝐶𝑛
𝑘
× 𝑝𝑘 × (1 − 𝑝)𝑛−𝑘
X: random variable
P(X=k): function of repartition the variable X, so f(k)
𝑘
𝐶𝑛 : Combination of order k of the n elements
p: probability of the event 1 black ball for one selection, so 0 ≤ 𝑘 ≤ 𝑛
n: number of selection with replacement 1 ≤ 𝑛 ≤ +∞
K: number of black balls
Let’s draw the function of repartition (of probability) of the law B(n=4, p=0.2):
p=0.2; n=4; k=0:4
plot(k, dbinom(k, n, p), pch=16, cex=2, xlim=range(0, 5), ylim=range(0, 0.5),
xlab=« Number of black balls k", ylab="Probability f(k)", cex.lab=1.5,
cex.axis=1.5, bty="l")
Understand the result of a test
The P-value – a question of probabilities
table(res4)
names(table(res4))
as.numeric(names(table(res4)))
x <- as.numeric(names(table(res4)))
y <- table(res4)/1000
plot(x, y, type="h")
x <- 0:50
y <- dbinom(x, 50, 0.5)
plot(x, y, type = "h")
Write the following instructions in R
window.
Take the tim to understand the
instructions. Do not forget to use the
help of R and to take notes on the
meaning of each instruction.
Binomial law
B(N,p)
B(50,0.5)
B(Number of selection, Probability of success)
We will note that this law depends on two parameters.The probabilities of each
possible event from this plot are true only if p=0.5 and N=50. Meaning that the
probability to obtain Heads for every flip is of 0.5, so 50%, and the number of
flips is 50.
p=50 is True only if the coin is not fake.
Laws of probability
Binomial law
𝑃 𝑋 = 𝑘 = 𝐶𝑛
𝑘
× 𝑝𝑘 × (1 − 𝑝)𝑛−𝑘
Let’s calculate the probability to have 2 black balls when selecting 4 balls with
replacement (knowing that the probability to get one black is of 0.2).
The question is to find the probability P(X ≥ 2).
We may then calculate the sum P(X = 2) + P(X = 3) + P(X = 4)
p=0.2; n=4
dbinom(2,n,p)+dbinom(3,n,p)+dbinom(4,n,p)
We may also exclude the probability P(X = 0) to have 0 black ball and the probability
P(X = 1) to have 1 black ball. The whole function of repartition being equal to 1:
p=0.2; n=4
1-dbinom(0,n,p)-dbinom(1,n,p)
Or we can directly calculate P(X ≥ 2), meaning F(1)upper=P(X>1)
pbinom(1,n,p, lower.tail=FALSE)
Laws of probability
Binomial law
𝑃 𝑋 = 𝑘 = 𝐶𝑛
𝑘
× 𝑝𝑘 × (1 − 𝑝)𝑛−𝑘
When we have 4 balls we replacement, to how many black balls corresponds the
probability p=0.1808:
p=0.2; n=4
qbinom(0.1808,n,p, lower.tail=FALSE)
When we have 4 balls we replacement, to how many black balls at least, corresponds the
probability p=0.15:
qbinom(0.15,n,p, lower.tail=FALSE)
qbinom(0.5,n,p)
p=0.2; n=4; k=0:4
plot(k, dbinom(k, n, p), pch=16, cex=2, xlim=range(0, 5), ylim=range(0, 0.5),
xlab="Number of black balls k", ylab="Probability f(k)", cex.lab=1.5, cex.axis=1.5,
bty="l")
abline(h=0.15,lwd=1,col="black")
abline(h=0.5,lwd=1,col="red")
Laws of probability
Binomial law
𝑃 𝑋 = 𝑘 = 𝐶𝑛
𝑘
× 𝑝𝑘 × (1 − 𝑝)𝑛−𝑘
When we have 4 balls we replacement, to how many black balls corresponds the
probability p=0.1808:
p=0.2; n=4
qbinom(0.1808,n,p, lower.tail=FALSE)
When we have 4 balls we replacement, to how many black balls at least, corresponds the
probability p=0.15:
qbinom(0.15,n,p, lower.tail=FALSE)
qbinom(0.5,n,p)
p=0.2; n=4; k=0:20
plot(k,dbinom(k,n,p), type="l", xlab=« Number of black balls k", ylab="Probability f(k)",
cex.lab=1.5, cex.axis=1.5)
p=0.2; n=20
lines(k,dbinom(k,n,p),lwd=3)
Laws of probability
Normal law
𝑓 𝑥 =
1
2𝜋𝜎 2
(𝑥−𝜇)2
−
𝑒 2𝜎2
f(x): Probability density
x: Continuous quantitative variable (−∞ < 𝑥 < +∞)
𝜇: Mean of the variable x
σ: Standrad deviation of the variable x
Laws of probability
Normal law
𝑓 𝑥 =
1
2𝜋𝜎 2
(𝑥−𝜇)2
−
𝑒 2𝜎2
f(x): Probability density
x: Continuous quantitative variable (−∞ < 𝑥 < +∞)
𝜇: Mean of the variable x
σ: Standrad deviation of the variable x
1) The normal is not normal in the way the other laws are abnormal.
2) The normal is very common but far to be the only law of probability
The normal distribution is immensely useful because of the central limit theorem, which
states that, under mild conditions, the mean of many random variables independently
drawn from the same distribution is distributed approximately normally.
Laws of probability
Normal law
𝑓 𝑥 =
1
2𝜋𝜎 2
(𝑥−𝜇)2
−
𝑒 2𝜎2
f(x): Probability density
x: Continuous quantitative variable (−∞ < 𝑥 < +∞)
𝜇: Mean of the variable x
σ: Standrad deviation of the variable x
Let’s consider the example of flakes size per layer in an archaeological site,
following a law N(70,10) and caluclate the density of probabilit of the value
60mm. It is f(60):
x=60; mu=70; sigma=10
1/(sqrt(2*pi)*sigma)*exp(-((x - mu)^2/(2*sigma^2)))
Laws of probability
Normal law
𝑓 𝑥 =
1
2𝜋𝜎 2
(𝑥−𝜇)2
−
𝑒 2𝜎2
f(x): Probability density
x: Continuous quantitative variable (−∞ < 𝑥 < +∞)
𝜇: Mean of the variable x
σ: Standrad deviation of the variable x
Let’s consider the example of flakes size per layer in an archaeological site,
following a law N(70,10) and caluclate the density of probability of the value
60cm. It is f(60):
x=60; mu=70; sigma=10
1/(sqrt(2*pi)*sigma)*exp(-((x - mu)^2/(2*sigma^2)))
With R:
dnorm(x,mu,sigma)
Laws of probability
Normal law
𝑓 𝑥 =
1
2𝜋𝜎 2
(𝑥−𝜇)2
−
𝑒 2𝜎2
f(x): Probability density
x: Continuous quantitative variable (−∞ < 𝑥 < +∞)
𝜇: Mean of the variable x
σ: Standrad deviation of the variable x
Let’s calculate the probability to select a 60mm long flake at least. The aim is to
determine the function of repartition F(60)upper=P(x ≥ 60):
x=60; mu=70; sigma=10
pnorm(x,mu,sigma, lower.tail=FALSE)
And the probability to select a 60mm long flake at the most. The aim is to
determine the function of repartition F(60)lower=P(x ≤ 60)
pnorm(x,mu,sigma)
Laws of probability
Normal law
𝑓 𝑥 =
1
2𝜋𝜎 2
(𝑥−𝜇)2
−
𝑒 2𝜎2
f(x): Probability density
x: Continuous quantitative variable (−∞ < 𝑥 < +∞)
𝜇: Mean of the variable x
σ: Standrad deviation of the variable x
Let’s calculate the threshold of sizes corresponding to 50% of the smallest flakes.
The aim is to determine the value of size xi that remains 50% of the area on the
left (so lower) of the curve, so F(xi)left=P(x ≤ xi)=0.5:
F=0.5; mu=70; sigma=10
qnorm(F,mu,sigma, lower.tail=TRUE)
Let’s do the same with a threshold of 2.5% of the longest flakes, so F(xi)right=P(x >
xi)=0.025:
F=0.025; mu=70; sigma=10
qnorm(F,mu,sigma, lower.tail=FALSE)
Laws of probability
Normal law
𝑓 𝑥 =
1
2𝜋𝜎 2
(𝑥−𝜇)2
−
𝑒 2𝜎2
f(x): Probability density
x: Continuous quantitative variable (−∞ < 𝑥 < +∞)
𝜇: Mean of the variable x
σ: Standrad deviation of the variable x
Let’s calculate the probability to select a flake between μ-1.96σ and μ+1.96σ . The
aim is to determine the function of repartition P(μ-1.96σ < x ≤ μ+1.96σ ) =P(x >
μ-1.96σ )-P(x > μ+1.96σ ) :
mu=70; sigma=10
pnorm(mu-1.96*sigma,mu,sigma, lower.tail=FALSE)pnorm(mu+1.96*sigma,mu,sigma, lower.tail=FALSE)
Or the other way around, P(μ-1.96σ < x ≤ μ+1.96σ ) =P(x ≤ μ+1.96σ )-P(x ≤ μ1.96σ ) :
mu=70; sigma=10
pnorm(mu+1.96*sigma,mu,sigma)- pnorm(mu-1.96*sigma,mu,sigma)
Laws of probability
Normal law
𝑓 𝑥 =
1
2𝜋𝜎 2
(𝑥−𝜇)2
−
𝑒 2𝜎2
f(x): Probability density
x: Continuous quantitative variable (−∞ < 𝑥 < +∞)
𝜇: Mean of the variable x
σ: Standrad deviation of the variable x
Remember, the 2.5% of the longest flakes mesure 89.6mm, so:
mu+1.96*sigma
Indeed, it corresponds to the value μ+1.96σ
Laws of probability
Chi2 law
𝑓 𝑥 =
1
𝜈
22 Γ(𝜈/2)
f(x): Probability density
x: Continuous variable (0 < 𝑥 < +∞)
𝜇: Number of df (0 < 𝜈 < +∞)
Γ : Gamma function
This law is used in many statistic tests.
×
𝜈
−𝑥
−1
𝑥2 × 𝑒 2
Laws of probability
Chi2 law
𝑓 𝑥 =
1
𝜈
22 Γ(𝜈/2)
×
𝜈
−𝑥
−1
𝑥2 × 𝑒 2
f(x): Probability density
x: Continuous variable (0 < 𝑥 < +∞)
𝜇: Number of df (0 < 𝜈 < +∞)
Γ : Gamma function
Degree of freedom df:
If
Then
𝑋 + 𝑌 = 12
𝑋=7
𝑌=5
There are two Random variable but only one Degree of freedom
Laws of probability
Chi2 law
𝑓 𝑥 =
1
𝜈
22 Γ(𝜈/2)
×
𝜈
−𝑥
−1
𝑥2 × 𝑒 2
f(x): Probability density
x: Continuous variable (0 < 𝑥 < +∞)
𝜇: Number of df (0 < 𝜈 < +∞)
Γ : Gamma function
Let’s calculate the probability density for the law 𝜒²4 for the value x=2:
x=2;df=4
1/(2^(df/2)*gamma(df/2)) *x^(df/2-1)*exp(-x/2)
In R:
x=2; df=4
dchisq(x,df)
Laws of probability
Chi2 law
𝑓 𝑥 =
1
𝜈
22 Γ(𝜈/2)
×
𝜈
−𝑥
−1
𝑥2 × 𝑒 2
f(x): Probability density
x: Continuous variable (0 < 𝑥 < +∞)
𝜇: Number of df (0 < 𝜈 < +∞)
Γ : Gamma function
Let’s calculate the probability that x≤2 on the law 𝜒²4. The aim is to determine
the function of repartition F(2)upper =P(x ≤2):
x=2; df=4
pchisq(x,df)
Let’s calculate the probability x>2 . so F(2)lower =P(x >2):
x=2; df=4
pchisq(x,df, lower.tail=FALSE)
Let’s calculate the threshold corresponding to 2.5% of the highest values. So
determining so F(xi)lower =P(x >xi)=0.025:
F=0.025; df=4
qchisq(F,df, lower.tail=FALSE)
Laws of probability
Chi2 law
𝑓 𝑥 =
1
𝜈
22 Γ(𝜈/2)
×
𝜈
−𝑥
−1
𝑥2 × 𝑒 2
f(x): Probability density
x: Continuous variable (0 < 𝑥 < +∞)
𝜇: Number of df (0 < 𝜈 < +∞)
Γ : Gamma function
Shape of the density curve:
Light line
x=seq(-0.1,10,0.01);df=1
plot(x, dchisq(x,df),type="l", ylim=range(0,1), xlab="Continuous variable x", ylab="Density
f(x)", cex.lab=1.5, cex.axis=1.5)
Thick line
df=2
lines(x, dchisq(x,df),lwd=3)
Dotted line
df=4
lines(x, dchisq(x,df), lty="39",lwd=2)
Long dotted line
df=8
lines(x, dchisq(x,df), lty="83",lwd=2)
Laws of probability
What is a law of probability?
These laws are probability laws of statistic. They are different from the
probability law of a measured variable.
The measured variable fluctuates according to the individuals while the
statistic variable fluctuates accoring to the sample.
For example, for a Student t test, if you want to compare means, your
sample must follow a normal law, while the statistic of the test follows a
Student law.
Understand the result of a test
The P-value – a question of probabilities
The aim of a statistical test is to conclude on a population while the observations
are only available for a sample of this population.
The value of the test varies based on the samples according to a probability
distribution.
The test forsees this fluctuation of the value of the test in only one case: the one
efined by the H0 hypothesis.
The aim of the test is then to determine if the value obtained is probable according
to the statistic distribution under H0. And for this it gives a p-value.
When the probability distribution under H0 is known, the p-value is nothing else
than a function of repartition of the law defined by the sample.
Understand the result of a test
The P-value – a question of probabilities
Let’s consider the example of flakes size in one layer of an archaeological site.
Question: Is the size of the obsidian flakes longer than the one of the andesite flakes that is
around 165mm.
A random sample of 50 obsidian flakes is selected that shows a mean of 165mm and a
standard deviation of 7mm.
The aim of the test is to conclude on the population of obsidian flakes for this archaeological
site although we sampled only a small part of this site.
H0: 𝜇𝑜𝑏𝑠𝑖𝑑𝑖𝑎𝑛 = 165𝑚𝑚
H1: 𝜇𝑜𝑏𝑠𝑖𝑑𝑖𝑎𝑛 > 165𝑚𝑚
The mean obsidian flake size is equal to andesite ones.
The mean obsidian flake size is strictly longer.
The formula is then M=m=168mm
With
M: calculated statistics
m: mean size in the sample
According to the test, M is a random variable which under H0 follows a normal law of
N(165,7/√50 = 0.99)
The function of repartition beyond M=168mm on the normal law N(165,0.99) is:
pnorm(168,165,0.99, lower.tail=FALSE)
Understand the result of a test
The P-value – a question of probabilities
The probability that M is above 168mm on N(165,0.99) is p=0.0012 what is low.
It means that if the random sample of 50 obsidian flakes comes from a population of
𝜇𝑜𝑏𝑠𝑖𝑑𝑖𝑎𝑛 = 165mm, the probability that m ≥ 168mm by sampling is p.
As we are gonna to see, we can then reject H0 with this p-value.
In conclusion, the sample does not permit to explain to distance from m to 165mm: m is
significatively longer than 𝜇𝑎𝑛𝑑𝑒𝑠𝑖𝑡𝑒= 165mm. It is then probable that is this archaeological site,
the 𝜇𝑜𝑏𝑠𝑖𝑑𝑖𝑎𝑛 > 𝜇𝑎𝑛𝑑𝑒𝑠𝑖𝑡𝑒
Understand the result of a test
The P-value – a question of probabilities
The 4 possible situations for a test of hypotheses:
Decision
Reality
H0
H1
H0
Right decision
1-α
Type I error
α
H1
Type II error
β
Right decision
1-β
Usually, we choose α= 5%, 1% or 0.1%. But those values do not have
any special meaning.
Understand the result of a test
The P-value – a question of probabilities
The 4 possible situations for a test of hypotheses:
Decision
Reality
H0
H1
H0
Right decision
1-α
Type I error
α
H1
Type II error
β
Right decision
1-β
Usually, we choose α= 5%, 1% or 0.1%. But those values do not have
any special meaning.
-> Do we have to minimize the risk α or the risk β ?
α means that we reject H0 while H0 is true.
β means that we reject H1 while H1 is true.
Understand the result of a test
The P-value – a question of probabilities
The 4 possible situations for a test of hypotheses:
Decision
Reality
H0: Innocent
H1: Guilty
H0: Innocent
Right decision
1-α
Type I error
α
H1: Guilty
Type II error
β
Right decision
1-β
Usually, we choose α= 5%, 1% or 0.1%. But those values do not have
any special meaning.
-> Do we have to minimize the risk α or the risk β ?
α means that we reject H0 while H0 is true.
β means that we reject H1 while H1 is true.
Understand the result of a test
The P-value – a question of probabilities
The 4 possible situations for a test of hypotheses:
Decision
Reality
H0: Innocent
H1: Guilty
H0: Innocent
Right decision
1-α
Type I error
Α = Weak
H1: Guilty
Type II error
β
Right decision
1-β
Usually, we choose α= 5%, 1% or 0.1%. But those values do not have
any special meaning.
-> Do we have to minimize the risk α or the risk β ?
α means that we reject H0 while H0 is true.
β means that we reject H1 while H1 is true.
Understand the result of a test
The P-value – a question of probabilities
The 4 possible situations for a test of hypotheses:
Decision
Reality
H0: Innocent
H1: Guilty
H0: Innocent
Right decision
1-α
Type I error
α
H1: Guilty
Type II error
Β = Weak
Right decision
1-β
Usually, we choose α= 5%, 1% or 0.1%. But those values do not have
any special meaning.
-> Do we have to minimize the risk α or the risk β ?
α means that we reject H0 while H0 is true.
β means that we reject H1 while H1 is true.
Understand the result of a test
The P-value – a question of probabilities
The 4 possible situations for a test of hypotheses:
Decision
Reality
H0: Innocent
H1: Guilty
H0: Innocent
Right decision
1-α
Type I error
Α =Middle
H1: Guilty
Type II error
Β =Middle
Right decision
1-β
Usually, we choose α= 5%, 1% or 0.1%. But those values do not have
any special meaning.
-> Do we have to minimize the risk α or the risk β ?
α means that we reject H0 while H0 is true.
β means that we reject H1 while H1 is true.
Understand the result of a test
The P-value – a question of probabilities
The 4 possible situations for a test of hypotheses:
Decision
Reality
H0: Innocent
H1: Guilty
H0: Innocent
Right decision
1-α
Type I error
Α =Middle
H1: Guilty
Type II error
Β =Middle
Right decision
1-β
Usually, we choose α= 5%, 1% or 0.1%. But those values do not have
any special meaning.
-> Do we have to minimize the risk α or the risk β ?
α means that we reject H0 while H0 is true.
β means that we reject H1 while H1 is true.
Minimizing the error α, increases the error β and then reduces the
power of the test.
Understand the result of a test
The P-value – a question of probabilities
In the case of a bilateral test, the technic consists in looking on upper and lower the
cacluclated X, to pick up the smallest value and to multiply by 2:
𝑝 = min 𝑃 𝑋 ≥ 𝑋𝑐𝑎𝑙𝑐 , 𝑃 𝑋 ≤ 𝑋𝑐𝑎𝑙𝑐
×2
Understand the result of a test
The P-value – a question of probabilities
In the case of the chi2, the p-value is the probability, if H0 is true, to obtain a value
of the statistic higher or equal to the observed value.
So what is the interest of the p-value?
-> to interpret the test and to conclude:
The risk α is the p-value. Or the risk α is the threshold to reject H0.
If p-value ≤ α, then I accept H1 (reject H0)
If p-value > α, then I accept H0
In the first case, the p-value of the test is significative, in the second case it is not.
By default, α is set to 5%
Summary and conclusion
Two hypotheses to define:
H0 is the nul hypothesis: the one we want to test
H1 is the alternative hypothesis (in the case we reject H0, H1 is considered as true)
The test depends of its statistic and its law of probability.
There are two types of error which are opposite one to the other one:
Type I error: conclude that H0 is false while in reality it is true
Type II error: conclude that H0 is true while in reality it is false
We always know the risk α as we define it.
If we know H1 with precision, then we can calculate β and the power of the test which
is equal to 1- β.
The P-value permits us to take our decision:
If it is inferior or equal to α, then I reject H0 and accept H1
If it is superior to α, then I accept H0