Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Theory of tests Thomas INGICCO 1 – Principle of tests (hypotheses) 2 – A first example with R 3 – Probabilities (law of random) 4 – Statistic and its law (test variable) 5 – Risk α and errors 6 – p-value 7 – Summary E. Munch, The scream Theory of tests Theory of tests Example 1 Let’s consider dices. I roll dices 4 times and obtain 4 times different results -> I deduce that the dices are not fake. So I am doubting now and want to check. So I roll the dices once again 4 times and I obtain 4 times the same result -> I change my opinion and conclude that the dices are fake. How do I know the truth? I only have 8 rolls so a small sample for an infinite population… -> I need a statistical test. Theory of tests Example 1 Let’s consider dices. I roll dices 4 times and obtain 4 times different results -> I deduce that the dices are not fake. So I am doubting now and want to check. So I roll the dices once again 4 times and I obtain 4 times the same result -> I change my opinion and conclude that the dices are fake. How do I know the truth? I only have 8 rolls so a small sample for an infinite population… -> I need a statistical test. Example 2 I study two collections of lithic tools, one from northern Luzon, the other one from Mindoro. I sample 20 tools in each population. The mean length of the 20 artefacts in the first population is 110 mm. In the second population it is 130mm. May I deduce that the lithic tools from Mindoro are longer and then that the prehistoric group are: 1) Different, 2) Have different abilities to knap, 3)Produce tools with different interests ? If I do so, what is my risk to be wrong? -> I need a statistical test What do we test and why do we test? Investigation or inventory : All the operations aiming at collecting in a systematic way, data or informations related to a group of individual or elements. What do we test and why do we test? Investigation or inventory : All the operations aiming at collecting in a systematic way, data or informations related to a group of individual or elements. Individual : Element = Statistical unit = Basal unit = what we sample What do we test and why do we test? Investigation or inventory : All the operations aiming at collecting in a systematic way, data or informations related to a group of individual or elements. Individual : Element = Statistical unit = Basal unit = what we sample Population : Universe = Statistic assemblage = All the individuals What do we test and why do we test? Investigation or inventory : All the operations aiming at collecting in a systematic way, data or informations related to a group of individual or elements. Individual : Element = Statistical unit = Basal unit = what we sample Population : Universe = Statistic assemblage = All the individuals Sample : Part of the population that we observe What do we test and why do we test? Investigation or inventory : All the operations aiming at collecting in a systematic way, data or informations related to a group of individual or elements. Individual : Element = Statistical unit = Basal unit = what we sample Population : Universe = Statistic assemblage = All the individuals Sample : Part of the population that we observe What do we test and why do we test? Investigation or inventory : All the operations aiming at collecting in a systematic way, data or informations related to a group of individual or elements. Individual : Element = Statistical unit = Basal unit = what we sample Population : Universe = Statistic assemblage = All the individuals Sample : Part of the population that we observe Probability What do we test and why do we test? Investigation or inventory : All the operations aiming at collecting in a systematic way, data or informations related to a group of individual or elements. Individual : Element = Statistical unit = Basal unit = what we sample Population : Universe = Statistic assemblage = All the individuals Sample : Part of the population that we observe Probability Statistic What do we test and why do we test? Investigation or inventory : All the operations aiming at collecting in a systematic way, data or informations related to a group of individual or elements. Individual : Element = Statistical unit = Basal unit = what we sample Population : Universe = Statistic assemblage = All the individuals Sample : Part of the population that we observe Observation: Measured value for each sampled individual The principle of test First step : • We define the hypotheses: H0 and H1. H0 = Null hypothesis H1 = Alternative hypothesis Most of the time, H0 defines an absence of structure in the data. Example 1: Chi2 Test H0 = Independance between two qualitative variables H1 = Dependance Example 2: Comparison of a mean to a theoretical mean H0 = Means are equal H1 = Means are different H1 = The observed mean is bigger than the theoretical mean H1 = The observed mean is smaller than the theoretical mean Example 3: Comparison of a mean to a theoretical mean H0 = The difference between the means is of 5cm H1 = The difference between the means is not of 5cm The principle of test Nul hypothesis H0 versus Alternative hypothesis H1 We start by considering that H0 is True. We test H0: 1- If the test determines that H0 is False, then I accept H1 (we reject H0) 2- If my test does not determine that H0 is False, I cannot reject this hypothesis (we keep H0) -> A theory is replaced by another one only if it can be rejected. The principle of test Second step : • Once the hypotheses H0 and H1 defined, we determine the variable of the test (the statistic) we need to reject H0. It exists two categories of tests: - Parametric tests: The distribution of probability of the measured variable in the targeted population follows a law. The analysed data can be modelised according to a known law. Then these tests will focus on the parameters of this law (mean, variance,…). - Non-parametric tests (or distribution free test): The distribution of the measured variable does not matter. These tests do not focus on the distribution parameters of the data. They are used for small samples (>10) and/or for variables that do not follow a normal law. The principle of test Second step : • Once the hypotheses H0 and H1 defined, we determine the variable of the test (the statistic) we need to reject H0. In the case of Chi2, the statistic is: 𝜒² = (𝑂𝑖𝑗 − 𝐸𝑖𝑗 )² 𝑖, 𝑗 𝐸𝑖𝑗 In the case of the comparison of a mean to a theoretical mean, the statistic is: 𝑇= 𝑋 − 𝜇0 𝜎²/𝑛 The principle of test Why to calculate a statistic? -> Because we know the law of probability of this statistic under the hypothesis H0, meaning if H0 is true. This is this law of probability that would permit us to determine if « there are some chances » that we obtained these data if H0 is true, or in the opposite, if « there are very few chances » that we obtained these data if H0 is true. In this second case, we will deduce that according to our data, H0 is most probably False. Let’s have an example ! How to make a test Concretly, how to realize a test in R? 1- Determination of the hypotheses H0 and H1 which permits to clarify the scientific question. 2- Choice of the test, search for the function in R. 3- Use the help for the test, so you know how to make the test and how it works. In other words, you understand what you do, which is not the case in Excel. Example 1: Question: Is the high of a ceramic correlated with the width of its openning? How to make a test Ceram <read.table("K:/Cours/Philippines/Statistic s-210/Data/Ceramics.txt",header=TRUE) str(Ceram) plot(Ceram$W.mouth,Ceram$H.rim) Write the following instructions in R window. Take the time to understand the instructions. Do not forget to use the help in R and to comment the meaning of each instruction. How to make a test Ceram <read.table("K:/Cours/Philippines/Statistic s-210/Data/Ceramics.txt",header=TRUE) str(Ceram) plot(Ceram$W.mouth,Ceram$H.rim) par(mfrow=c(1, 2)) hist(Ceram$W.mouth) hist(Ceram$H.rim) Write the following instructions in R window. Take the time to understand the instructions. Do not forget to use the help in R and to comment the meaning of each instruction. How to make a test Ceram <read.table("K:/Cours/Philippines/Statistic s-210/Data/Ceramics.txt",header=TRUE) str(Ceram) plot(Ceram$W.mouth,Ceram$H.rim) par(mfrow=c(1, 2)) hist(Ceram$W.mouth) hist(Ceram$H.rim) apropos("test") help.search("test") ?cor.test Write the following instructions in R window. Take the time to understand the instructions. Do not forget to use the help in R and to comment the meaning of each instruction. How to make a test Ceram <read.table("K:/Cours/Philippines/Statistic s-210/Data/Ceramics.txt",header=TRUE) str(Ceram) plot(Ceram$W.mouth,Ceram$H.rim) par(mfrow=c(1, 2)) hist(Ceram$W.mouth) hist(Ceram$H.rim) apropos("test") help.search("test") ?cor.test cor.test(Ceram$W.mouth,Ceram$H.rim) Write the following instructions in R window. Take the time to understand the instructions. Do not forget to use the help in R and to comment the meaning of each instruction. Understand the result of a test The P-value – a question of probabilities I flip 50 times a coin. Is the coin fake? We consider a statistic: X= the number of times Heads is chosen in my experiment. H0= the coin is not fake. H1= the coin is fake. If H0 is True, what should be the value of X? Meaning how many times Heads should be obtained in 50 flips? Is it 25? Understand the result of a test The P-value – a question of probabilities I flip 50 times a coin. Is the coin fake? We consider a statistic: X= the number of times Heads is chosen in my experiment. H0= the coin is not fake. H1= the coin is fake. If H0 is True, what should be the value of X? Meaning how many times Heads should be obtained in 50 flips? Is it 25? -> NO. In 50 flips, I may obtain between 0 and 50 Heads, but, if the coin is not fake, the probability (the chance) to obtain 0 Heads or 50 Heads is low, while the probability to obtain 23, 24, 25, 26, 27 Heads is stronger. Understand the result of a test The P-value – a question of probabilities Write the following instructions in R window. We are going to estimate the law of probability of X from 1000 experiments. One experiment correspond to 50 flips. Write the following instructions in R window. Take the tim to understand the instructions. Do not forget to use the help of R and to take notes on the meaning of each instruction. # statfun is a function that makes an experiment (50 flips of a coin) and returns the number of Heads obtained. coin <- c("T", "H") statfun <- function(i){ simu <- sample(coin, 50, replace = TRUE, prob = c(0.5, 0.5)) # nbHeads <- table(simu)[1] nbHeads <- table(simu)[2] return(nbHeads) } res4 <- sapply(1:1000, statfun) res4 Understand the result of a test The P-value – a question of probabilities table(res4) names(table(res4)) as.numeric(names(table(res4))) Head <- as.numeric(names(table(res4))) Probability <- table(res4)/1000 plot(x, y, type="h") x <- 0:50 y <- dbinom(x, 50, 0.5) plot(x, y, type = "h") Write the following instructions in R window. Take the tim to understand the instructions. Do not forget to use the help of R and to take notes on the meaning of each instruction. Binomial law B(N,p) B(50,0.5) Number of selection Probability of success We will note that this law depends on two parameters.The probabilities of each possible event from this plot are true only if p=0.5 and N=50. Meaning that the probability to obtain Heads for every flip is of 0.5, so 50%, and the number of flips is 50. p=50 is True only if the coin is not fake. k: each possible value within the discrete variable X. f(k): frequency associated to each value = probability associated to k. F(k): sum of the probabilities f(k) on the left or right of k regarding our interest. F(k): it is the probability that X is upper or lower/equal to a value k. F(k)upper = P(X>k) F(30) upper = P(X>30)=0.118 cumsum(Probability) sum(Probability[which(rownames(Probability)=="31"):length(Probability)]) k: each possible value within the continuous variable X. f(k): probability distribution of X = probability density associated to k. F(k): area under the curve f(k) on the left or right of k regarding our interest. F(k): it is the probability that X is upper or lower/equal to a value k. F(k)upper = P(X>k) F(xk)lower - F(xj)lower = P(xk<xi≤xk) Laws of probability Binomial law dbinom : probability f(k) of the variable X pbinom : function of repartition of F(k) of the variable X qbinom : give the value k of the variable X for a given value of F(k) rbinom : generates random values for the variable X considering probabilities Normal law dnorm : probability f(k) of the variable X pnorm : function of repartition of F(k) of the variable X qnorm : give the value k of the variable X for a given value of F(k) rnorm : generates random values for the variable X considering probabilities Chi2 law dchisq : probability f(k) of the variable X pchisq : function of repartition of F(k) of the variable X qchisq: give the value k of the variable X for a given value of F(k) rchisq: generates random values for the variable X considering probabilities Laws of probability Binomial law dbinom : probability f(k) of the variable X pbinom : function of repartition of F(k) of the variable X qbinom : give the value k of the variable X for a given value of F(k) rbinom : generates random values for the variable X considering probabilities Laws of probability Binomial law dbinom : probability f(k) of the variable X pbinom : function of repartition of F(k) of the variable X qbinom : give the value k of the variable X for a given value of F(k) rbinom : generates random values for the variable X considering probabilities Laws of probability Binomial law 𝑃 𝑋 = 𝑘 = 𝐶𝑛 𝑘 × 𝑝𝑘 × (1 − 𝑝)𝑛−𝑘 X: random variable P(X=k): function of repartition the variable X, so f(k) 𝑘 𝐶𝑛 : Combination of order k of the n elements p: probability of the event 1 black ball for one selection, so 0 ≤ 𝑘 ≤ 𝑛 n: number of selection with replacement 1 ≤ 𝑛 ≤ +∞ K: number of black balls Laws of probability Binomial law 𝑃 𝑋 = 𝑘 = 𝐶𝑛 𝑘 × 𝑝𝑘 × (1 − 𝑝)𝑛−𝑘 X: random variable P(X=k): function of repartition the variable X, so f(k) 𝑘 𝐶𝑛 : Combination of order k of the n elements p: probability of the event 1 black ball for one selection, so 0 ≤ 𝑘 ≤ 𝑛 n: number of selection with replacement 1 ≤ 𝑛 ≤ +∞ K: number of black balls The binomial law is grounded on two exclusive elements, « black » and « white » for example when starting a chess game; or « boy » and « girl »; or « yes » or « no » when you date a girl. Each of these events is associated to a probability of appearance. The binomial law gives the probability corresponding to many apparitions. You have 50 balls in a bag: 10 black, 40 red. The probability to have a random black ball after 1 selection is: 𝑝 = 𝑃(1 𝑏𝑙𝑎𝑐𝑘 1 𝑠𝑒𝑙𝑒𝑐𝑡𝑖𝑜𝑛) If the selection is randomly done, then the probability is p=10/50=0.2 Laws of probability Binomial law 𝑃 𝑋 = 𝑘 = 𝐶𝑛 𝑘 × 𝑝𝑘 × (1 − 𝑝)𝑛−𝑘 X: random variable P(X=k): function of repartition the variable X, so f(k) 𝑘 𝐶𝑛 : Combination of order k of the n elements p: probability of the event 1 black ball for one selection, so 0 ≤ 𝑘 ≤ 𝑛 n: number of selection with replacement 1 ≤ 𝑛 ≤ +∞ K: number of black balls Let’s calculate the probability to have 2 black balls when selecting 4 balls with replacement (knowing that the probability to get one black is of 0.2): p=0.2; n=4; k=2 choose(n,k)*p^k*(1-p)^(n-k) Laws of probability Binomial law 𝑃 𝑋 = 𝑘 = 𝐶𝑛 𝑘 × 𝑝𝑘 × (1 − 𝑝)𝑛−𝑘 X: random variable P(X=k): function of repartition the variable X, so f(k) 𝑘 𝐶𝑛 : Combination of order k of the n elements p: probability of the event 1 black ball for one selection, so 0 ≤ 𝑘 ≤ 𝑛 n: number of selection with replacement 1 ≤ 𝑛 ≤ +∞ K: number of black balls Let’s calculate the probability to have 2 black balls when selecting 4 balls with replacement (knowing that the probability to get one black is of 0.2): p=0.2; n=4; k=2 choose(n,k)*p^k*(1-p)^(n-k) dbinom(k, n, p) Laws of probability Binomial law 𝑃 𝑋 = 𝑘 = 𝐶𝑛 𝑘 × 𝑝𝑘 × (1 − 𝑝)𝑛−𝑘 X: random variable P(X=k): function of repartition the variable X, so f(k) 𝑘 𝐶𝑛 : Combination of order k of the n elements p: probability of the event 1 black ball for one selection, so 0 ≤ 𝑘 ≤ 𝑛 n: number of selection with replacement 1 ≤ 𝑛 ≤ +∞ K: number of black balls Let’s calculate the probability to have 2 black balls when selecting 4 balls with replacement (knowing that the probability to get one black is of 0.2): p=0.2; n=4; k=2 choose(n,k)*p^k*(1-p)^(n-k) dbinom(k, n, p) The probability P(X=2) is then of 15.36% Laws of probability Binomial law 𝑃 𝑋 = 𝑘 = 𝐶𝑛 𝑘 × 𝑝𝑘 × (1 − 𝑝)𝑛−𝑘 X: random variable P(X=k): function of repartition the variable X, so f(k) 𝑘 𝐶𝑛 : Combination of order k of the n elements p: probability of the event 1 black ball for one selection, so 0 ≤ 𝑘 ≤ 𝑛 n: number of selection with replacement 1 ≤ 𝑛 ≤ +∞ K: number of black balls Let’s draw the function of repartition (of probability) of the law B(n=4, p=0.2): p=0.2; n=4; k=0:4 plot(k, dbinom(k, n, p), pch=16, cex=2, xlim=range(0, 5), ylim=range(0, 0.5), xlab=« Number of black balls k", ylab="Probability f(k)", cex.lab=1.5, cex.axis=1.5, bty="l") Understand the result of a test The P-value – a question of probabilities table(res4) names(table(res4)) as.numeric(names(table(res4))) x <- as.numeric(names(table(res4))) y <- table(res4)/1000 plot(x, y, type="h") x <- 0:50 y <- dbinom(x, 50, 0.5) plot(x, y, type = "h") Write the following instructions in R window. Take the tim to understand the instructions. Do not forget to use the help of R and to take notes on the meaning of each instruction. Binomial law B(N,p) B(50,0.5) B(Number of selection, Probability of success) We will note that this law depends on two parameters.The probabilities of each possible event from this plot are true only if p=0.5 and N=50. Meaning that the probability to obtain Heads for every flip is of 0.5, so 50%, and the number of flips is 50. p=50 is True only if the coin is not fake. Laws of probability Binomial law 𝑃 𝑋 = 𝑘 = 𝐶𝑛 𝑘 × 𝑝𝑘 × (1 − 𝑝)𝑛−𝑘 Let’s calculate the probability to have 2 black balls when selecting 4 balls with replacement (knowing that the probability to get one black is of 0.2). The question is to find the probability P(X ≥ 2). We may then calculate the sum P(X = 2) + P(X = 3) + P(X = 4) p=0.2; n=4 dbinom(2,n,p)+dbinom(3,n,p)+dbinom(4,n,p) We may also exclude the probability P(X = 0) to have 0 black ball and the probability P(X = 1) to have 1 black ball. The whole function of repartition being equal to 1: p=0.2; n=4 1-dbinom(0,n,p)-dbinom(1,n,p) Or we can directly calculate P(X ≥ 2), meaning F(1)upper=P(X>1) pbinom(1,n,p, lower.tail=FALSE) Laws of probability Binomial law 𝑃 𝑋 = 𝑘 = 𝐶𝑛 𝑘 × 𝑝𝑘 × (1 − 𝑝)𝑛−𝑘 When we have 4 balls we replacement, to how many black balls corresponds the probability p=0.1808: p=0.2; n=4 qbinom(0.1808,n,p, lower.tail=FALSE) When we have 4 balls we replacement, to how many black balls at least, corresponds the probability p=0.15: qbinom(0.15,n,p, lower.tail=FALSE) qbinom(0.5,n,p) p=0.2; n=4; k=0:4 plot(k, dbinom(k, n, p), pch=16, cex=2, xlim=range(0, 5), ylim=range(0, 0.5), xlab="Number of black balls k", ylab="Probability f(k)", cex.lab=1.5, cex.axis=1.5, bty="l") abline(h=0.15,lwd=1,col="black") abline(h=0.5,lwd=1,col="red") Laws of probability Binomial law 𝑃 𝑋 = 𝑘 = 𝐶𝑛 𝑘 × 𝑝𝑘 × (1 − 𝑝)𝑛−𝑘 When we have 4 balls we replacement, to how many black balls corresponds the probability p=0.1808: p=0.2; n=4 qbinom(0.1808,n,p, lower.tail=FALSE) When we have 4 balls we replacement, to how many black balls at least, corresponds the probability p=0.15: qbinom(0.15,n,p, lower.tail=FALSE) qbinom(0.5,n,p) p=0.2; n=4; k=0:20 plot(k,dbinom(k,n,p), type="l", xlab=« Number of black balls k", ylab="Probability f(k)", cex.lab=1.5, cex.axis=1.5) p=0.2; n=20 lines(k,dbinom(k,n,p),lwd=3) Laws of probability Normal law 𝑓 𝑥 = 1 2𝜋𝜎 2 (𝑥−𝜇)2 − 𝑒 2𝜎2 f(x): Probability density x: Continuous quantitative variable (−∞ < 𝑥 < +∞) 𝜇: Mean of the variable x σ: Standrad deviation of the variable x Laws of probability Normal law 𝑓 𝑥 = 1 2𝜋𝜎 2 (𝑥−𝜇)2 − 𝑒 2𝜎2 f(x): Probability density x: Continuous quantitative variable (−∞ < 𝑥 < +∞) 𝜇: Mean of the variable x σ: Standrad deviation of the variable x 1) The normal is not normal in the way the other laws are abnormal. 2) The normal is very common but far to be the only law of probability The normal distribution is immensely useful because of the central limit theorem, which states that, under mild conditions, the mean of many random variables independently drawn from the same distribution is distributed approximately normally. Laws of probability Normal law 𝑓 𝑥 = 1 2𝜋𝜎 2 (𝑥−𝜇)2 − 𝑒 2𝜎2 f(x): Probability density x: Continuous quantitative variable (−∞ < 𝑥 < +∞) 𝜇: Mean of the variable x σ: Standrad deviation of the variable x Let’s consider the example of flakes size per layer in an archaeological site, following a law N(70,10) and caluclate the density of probabilit of the value 60mm. It is f(60): x=60; mu=70; sigma=10 1/(sqrt(2*pi)*sigma)*exp(-((x - mu)^2/(2*sigma^2))) Laws of probability Normal law 𝑓 𝑥 = 1 2𝜋𝜎 2 (𝑥−𝜇)2 − 𝑒 2𝜎2 f(x): Probability density x: Continuous quantitative variable (−∞ < 𝑥 < +∞) 𝜇: Mean of the variable x σ: Standrad deviation of the variable x Let’s consider the example of flakes size per layer in an archaeological site, following a law N(70,10) and caluclate the density of probability of the value 60cm. It is f(60): x=60; mu=70; sigma=10 1/(sqrt(2*pi)*sigma)*exp(-((x - mu)^2/(2*sigma^2))) With R: dnorm(x,mu,sigma) Laws of probability Normal law 𝑓 𝑥 = 1 2𝜋𝜎 2 (𝑥−𝜇)2 − 𝑒 2𝜎2 f(x): Probability density x: Continuous quantitative variable (−∞ < 𝑥 < +∞) 𝜇: Mean of the variable x σ: Standrad deviation of the variable x Let’s calculate the probability to select a 60mm long flake at least. The aim is to determine the function of repartition F(60)upper=P(x ≥ 60): x=60; mu=70; sigma=10 pnorm(x,mu,sigma, lower.tail=FALSE) And the probability to select a 60mm long flake at the most. The aim is to determine the function of repartition F(60)lower=P(x ≤ 60) pnorm(x,mu,sigma) Laws of probability Normal law 𝑓 𝑥 = 1 2𝜋𝜎 2 (𝑥−𝜇)2 − 𝑒 2𝜎2 f(x): Probability density x: Continuous quantitative variable (−∞ < 𝑥 < +∞) 𝜇: Mean of the variable x σ: Standrad deviation of the variable x Let’s calculate the threshold of sizes corresponding to 50% of the smallest flakes. The aim is to determine the value of size xi that remains 50% of the area on the left (so lower) of the curve, so F(xi)left=P(x ≤ xi)=0.5: F=0.5; mu=70; sigma=10 qnorm(F,mu,sigma, lower.tail=TRUE) Let’s do the same with a threshold of 2.5% of the longest flakes, so F(xi)right=P(x > xi)=0.025: F=0.025; mu=70; sigma=10 qnorm(F,mu,sigma, lower.tail=FALSE) Laws of probability Normal law 𝑓 𝑥 = 1 2𝜋𝜎 2 (𝑥−𝜇)2 − 𝑒 2𝜎2 f(x): Probability density x: Continuous quantitative variable (−∞ < 𝑥 < +∞) 𝜇: Mean of the variable x σ: Standrad deviation of the variable x Let’s calculate the probability to select a flake between μ-1.96σ and μ+1.96σ . The aim is to determine the function of repartition P(μ-1.96σ < x ≤ μ+1.96σ ) =P(x > μ-1.96σ )-P(x > μ+1.96σ ) : mu=70; sigma=10 pnorm(mu-1.96*sigma,mu,sigma, lower.tail=FALSE)pnorm(mu+1.96*sigma,mu,sigma, lower.tail=FALSE) Or the other way around, P(μ-1.96σ < x ≤ μ+1.96σ ) =P(x ≤ μ+1.96σ )-P(x ≤ μ1.96σ ) : mu=70; sigma=10 pnorm(mu+1.96*sigma,mu,sigma)- pnorm(mu-1.96*sigma,mu,sigma) Laws of probability Normal law 𝑓 𝑥 = 1 2𝜋𝜎 2 (𝑥−𝜇)2 − 𝑒 2𝜎2 f(x): Probability density x: Continuous quantitative variable (−∞ < 𝑥 < +∞) 𝜇: Mean of the variable x σ: Standrad deviation of the variable x Remember, the 2.5% of the longest flakes mesure 89.6mm, so: mu+1.96*sigma Indeed, it corresponds to the value μ+1.96σ Laws of probability Chi2 law 𝑓 𝑥 = 1 𝜈 22 Γ(𝜈/2) f(x): Probability density x: Continuous variable (0 < 𝑥 < +∞) 𝜇: Number of df (0 < 𝜈 < +∞) Γ : Gamma function This law is used in many statistic tests. × 𝜈 −𝑥 −1 𝑥2 × 𝑒 2 Laws of probability Chi2 law 𝑓 𝑥 = 1 𝜈 22 Γ(𝜈/2) × 𝜈 −𝑥 −1 𝑥2 × 𝑒 2 f(x): Probability density x: Continuous variable (0 < 𝑥 < +∞) 𝜇: Number of df (0 < 𝜈 < +∞) Γ : Gamma function Degree of freedom df: If Then 𝑋 + 𝑌 = 12 𝑋=7 𝑌=5 There are two Random variable but only one Degree of freedom Laws of probability Chi2 law 𝑓 𝑥 = 1 𝜈 22 Γ(𝜈/2) × 𝜈 −𝑥 −1 𝑥2 × 𝑒 2 f(x): Probability density x: Continuous variable (0 < 𝑥 < +∞) 𝜇: Number of df (0 < 𝜈 < +∞) Γ : Gamma function Let’s calculate the probability density for the law 𝜒²4 for the value x=2: x=2;df=4 1/(2^(df/2)*gamma(df/2)) *x^(df/2-1)*exp(-x/2) In R: x=2; df=4 dchisq(x,df) Laws of probability Chi2 law 𝑓 𝑥 = 1 𝜈 22 Γ(𝜈/2) × 𝜈 −𝑥 −1 𝑥2 × 𝑒 2 f(x): Probability density x: Continuous variable (0 < 𝑥 < +∞) 𝜇: Number of df (0 < 𝜈 < +∞) Γ : Gamma function Let’s calculate the probability that x≤2 on the law 𝜒²4. The aim is to determine the function of repartition F(2)upper =P(x ≤2): x=2; df=4 pchisq(x,df) Let’s calculate the probability x>2 . so F(2)lower =P(x >2): x=2; df=4 pchisq(x,df, lower.tail=FALSE) Let’s calculate the threshold corresponding to 2.5% of the highest values. So determining so F(xi)lower =P(x >xi)=0.025: F=0.025; df=4 qchisq(F,df, lower.tail=FALSE) Laws of probability Chi2 law 𝑓 𝑥 = 1 𝜈 22 Γ(𝜈/2) × 𝜈 −𝑥 −1 𝑥2 × 𝑒 2 f(x): Probability density x: Continuous variable (0 < 𝑥 < +∞) 𝜇: Number of df (0 < 𝜈 < +∞) Γ : Gamma function Shape of the density curve: Light line x=seq(-0.1,10,0.01);df=1 plot(x, dchisq(x,df),type="l", ylim=range(0,1), xlab="Continuous variable x", ylab="Density f(x)", cex.lab=1.5, cex.axis=1.5) Thick line df=2 lines(x, dchisq(x,df),lwd=3) Dotted line df=4 lines(x, dchisq(x,df), lty="39",lwd=2) Long dotted line df=8 lines(x, dchisq(x,df), lty="83",lwd=2) Laws of probability What is a law of probability? These laws are probability laws of statistic. They are different from the probability law of a measured variable. The measured variable fluctuates according to the individuals while the statistic variable fluctuates accoring to the sample. For example, for a Student t test, if you want to compare means, your sample must follow a normal law, while the statistic of the test follows a Student law. Understand the result of a test The P-value – a question of probabilities The aim of a statistical test is to conclude on a population while the observations are only available for a sample of this population. The value of the test varies based on the samples according to a probability distribution. The test forsees this fluctuation of the value of the test in only one case: the one efined by the H0 hypothesis. The aim of the test is then to determine if the value obtained is probable according to the statistic distribution under H0. And for this it gives a p-value. When the probability distribution under H0 is known, the p-value is nothing else than a function of repartition of the law defined by the sample. Understand the result of a test The P-value – a question of probabilities Let’s consider the example of flakes size in one layer of an archaeological site. Question: Is the size of the obsidian flakes longer than the one of the andesite flakes that is around 165mm. A random sample of 50 obsidian flakes is selected that shows a mean of 165mm and a standard deviation of 7mm. The aim of the test is to conclude on the population of obsidian flakes for this archaeological site although we sampled only a small part of this site. H0: 𝜇𝑜𝑏𝑠𝑖𝑑𝑖𝑎𝑛 = 165𝑚𝑚 H1: 𝜇𝑜𝑏𝑠𝑖𝑑𝑖𝑎𝑛 > 165𝑚𝑚 The mean obsidian flake size is equal to andesite ones. The mean obsidian flake size is strictly longer. The formula is then M=m=168mm With M: calculated statistics m: mean size in the sample According to the test, M is a random variable which under H0 follows a normal law of N(165,7/√50 = 0.99) The function of repartition beyond M=168mm on the normal law N(165,0.99) is: pnorm(168,165,0.99, lower.tail=FALSE) Understand the result of a test The P-value – a question of probabilities The probability that M is above 168mm on N(165,0.99) is p=0.0012 what is low. It means that if the random sample of 50 obsidian flakes comes from a population of 𝜇𝑜𝑏𝑠𝑖𝑑𝑖𝑎𝑛 = 165mm, the probability that m ≥ 168mm by sampling is p. As we are gonna to see, we can then reject H0 with this p-value. In conclusion, the sample does not permit to explain to distance from m to 165mm: m is significatively longer than 𝜇𝑎𝑛𝑑𝑒𝑠𝑖𝑡𝑒= 165mm. It is then probable that is this archaeological site, the 𝜇𝑜𝑏𝑠𝑖𝑑𝑖𝑎𝑛 > 𝜇𝑎𝑛𝑑𝑒𝑠𝑖𝑡𝑒 Understand the result of a test The P-value – a question of probabilities The 4 possible situations for a test of hypotheses: Decision Reality H0 H1 H0 Right decision 1-α Type I error α H1 Type II error β Right decision 1-β Usually, we choose α= 5%, 1% or 0.1%. But those values do not have any special meaning. Understand the result of a test The P-value – a question of probabilities The 4 possible situations for a test of hypotheses: Decision Reality H0 H1 H0 Right decision 1-α Type I error α H1 Type II error β Right decision 1-β Usually, we choose α= 5%, 1% or 0.1%. But those values do not have any special meaning. -> Do we have to minimize the risk α or the risk β ? α means that we reject H0 while H0 is true. β means that we reject H1 while H1 is true. Understand the result of a test The P-value – a question of probabilities The 4 possible situations for a test of hypotheses: Decision Reality H0: Innocent H1: Guilty H0: Innocent Right decision 1-α Type I error α H1: Guilty Type II error β Right decision 1-β Usually, we choose α= 5%, 1% or 0.1%. But those values do not have any special meaning. -> Do we have to minimize the risk α or the risk β ? α means that we reject H0 while H0 is true. β means that we reject H1 while H1 is true. Understand the result of a test The P-value – a question of probabilities The 4 possible situations for a test of hypotheses: Decision Reality H0: Innocent H1: Guilty H0: Innocent Right decision 1-α Type I error Α = Weak H1: Guilty Type II error β Right decision 1-β Usually, we choose α= 5%, 1% or 0.1%. But those values do not have any special meaning. -> Do we have to minimize the risk α or the risk β ? α means that we reject H0 while H0 is true. β means that we reject H1 while H1 is true. Understand the result of a test The P-value – a question of probabilities The 4 possible situations for a test of hypotheses: Decision Reality H0: Innocent H1: Guilty H0: Innocent Right decision 1-α Type I error α H1: Guilty Type II error Β = Weak Right decision 1-β Usually, we choose α= 5%, 1% or 0.1%. But those values do not have any special meaning. -> Do we have to minimize the risk α or the risk β ? α means that we reject H0 while H0 is true. β means that we reject H1 while H1 is true. Understand the result of a test The P-value – a question of probabilities The 4 possible situations for a test of hypotheses: Decision Reality H0: Innocent H1: Guilty H0: Innocent Right decision 1-α Type I error Α =Middle H1: Guilty Type II error Β =Middle Right decision 1-β Usually, we choose α= 5%, 1% or 0.1%. But those values do not have any special meaning. -> Do we have to minimize the risk α or the risk β ? α means that we reject H0 while H0 is true. β means that we reject H1 while H1 is true. Understand the result of a test The P-value – a question of probabilities The 4 possible situations for a test of hypotheses: Decision Reality H0: Innocent H1: Guilty H0: Innocent Right decision 1-α Type I error Α =Middle H1: Guilty Type II error Β =Middle Right decision 1-β Usually, we choose α= 5%, 1% or 0.1%. But those values do not have any special meaning. -> Do we have to minimize the risk α or the risk β ? α means that we reject H0 while H0 is true. β means that we reject H1 while H1 is true. Minimizing the error α, increases the error β and then reduces the power of the test. Understand the result of a test The P-value – a question of probabilities In the case of a bilateral test, the technic consists in looking on upper and lower the cacluclated X, to pick up the smallest value and to multiply by 2: 𝑝 = min 𝑃 𝑋 ≥ 𝑋𝑐𝑎𝑙𝑐 , 𝑃 𝑋 ≤ 𝑋𝑐𝑎𝑙𝑐 ×2 Understand the result of a test The P-value – a question of probabilities In the case of the chi2, the p-value is the probability, if H0 is true, to obtain a value of the statistic higher or equal to the observed value. So what is the interest of the p-value? -> to interpret the test and to conclude: The risk α is the p-value. Or the risk α is the threshold to reject H0. If p-value ≤ α, then I accept H1 (reject H0) If p-value > α, then I accept H0 In the first case, the p-value of the test is significative, in the second case it is not. By default, α is set to 5% Summary and conclusion Two hypotheses to define: H0 is the nul hypothesis: the one we want to test H1 is the alternative hypothesis (in the case we reject H0, H1 is considered as true) The test depends of its statistic and its law of probability. There are two types of error which are opposite one to the other one: Type I error: conclude that H0 is false while in reality it is true Type II error: conclude that H0 is true while in reality it is false We always know the risk α as we define it. If we know H1 with precision, then we can calculate β and the power of the test which is equal to 1- β. The P-value permits us to take our decision: If it is inferior or equal to α, then I reject H0 and accept H1 If it is superior to α, then I accept H0