Survey

Transcript

The issue A theory (a “box theory”) is formulated: the distribution(s) of the random variable(s) is (are) supposed to be known Refute! Yes Improve! A doubt...: Was the exp. welldesigned? KO Be humble! OK Test of significance No Predictions Experiment Observations Comparison via... Proving and disproving a theory … Every genuine test of a theory is an attempt to falsify it, or to refute it. Testability is falsifiability; [...] Confirming evidence should not count except when it is the result of a genuine test of the theory; and this means that it can be presented as a serious but unsuccessful attempt to falsify the theory. [...] One can sum up all this by saying that the criterion of the scientific status of a theory is its falsifiability, or refutability, or testability. Karl Popper, “Science: Conjectures and Refutations”, 1953 Tests of significance: an example A measurement, given by the sample mean x, is compared with the expected value, . If there is a difference between experiment and theory, i.e. between x and ... “... is the observed difference due to chance?” Key idea: if the observed value x is too many standard errors from its expected value , then it's hard to explain the result by chance. 0) Box theory and Experiment The box theory is the theory by means of which predictions (expected values) of random variables are made. The experiment is a set of measurements that produce values of the random variables. 1) Null and Alternative Hypothesis Null hypothesis H0: any observed difference is due to chance. Alternative hypothesis HA (H1): the observed difference is real. 2) Test statistic A statistic that measures the difference, suitably normalized, between the observed value, determined through the experiment, and the expected value, determined through the box theory. A test statistic is necessary in most cases. In the case of sequences of Bernoulli trials, true Pvalues (see next slide) are evaluated by directly using the binomial distribution. Ex. gr.: z, the difference, normalized on the standard error, between the observed value x and the expected value . 3) Pvalue Observed significance level = Pvalue = chance of getting a value of the test statistic as extreme as or more extreme than the observed one. For a supporter of H0, the Pvalue is the chance that things could go that way or worse! For an opponent of H0, the Pvalue is the chance that things could go that way or better! ! The P–value is not (ex.gr.) the probability of the null hypothesis being right! 3) Pvalue Convention related to the evidence against H0 and the box theory (levels of significance): P < 5 % ⇒ significant evidence against H0 ● P < 1 % ⇒ highly significant evidence against H0 ●