Download Framing and Testing Hypotheses

9/14/2016
Hypotheses
• What are they?
Framing and Testing Hypotheses
– Explanations that account for our observations
• Inferences
– We draw conclusions about causes based on our data
– ‘DATA’ do not make conclusions
– Previous discussions about ‘P‐values’
Hypothesis
• Manipulative Studies
– Well‐designed manipulative experiments allow us to be confident in our inferences
Hypotheses
• Scientific hypothesis must be testable
– Additional observations or results could cause us to modify or reject our hypothesis
• Observational or Correlative Studies
– Less confidence in inferences drawn from poorly designed experiments or studies in which variables of interest were not directly manipulated
• Good hypotheses should also generate novel predictions
Scientific Method
• Deduction
– Proceeds from the general case to the specific case
• All hares in Forests for the World (FFW) are Snowshoe Hares
• I sampled this Hare from FFW
• This hare is a Snowshoe Hare
– The conclusion must be logically true if the first two premises are correct ...
Scientific Method
• Induction
– Proceeds from the specific case to the general case
• All 25 hare observed were Snowshoe Hare
• All of these observations were made in FFW
• All hare in Forests for the World are Snowshoe Hare
– Although the conclusion is likely to be true, it may be false – confidence  with  n 1
9/14/2016
Statistics
• Statistics is an inductive process
– Always trying to draw general conclusions based on a specific, limited set of observations
Scientific Reasoning
• Both induction and deduction are used in all models of scientific reasoning
– Have different emphases
• Begin with observation that needs explanation
• Through induction develop a single hypothesis to explain it
• Once formed, the hypothesis (through deduction) generates further predictions
Scientific Reasoning
Scientific Reasoning
• Test predictions with additional observations
– Hypothesis supported or rejected
• Hypothesis  prediction  observation repeated ...
Hypothetico‐Deductive Method
Hypothetico‐Deductive Method
• Begins with an initial observation that you are trying to observe
• But uses multiple working hypotheses
• Goal is to falsify the hypotheses
• Falsification eliminates some alternatives
2
9/14/2016
Hypothetico‐Deductive Method
• Advantages
Hypothetico‐Deductive Method
• Disadvantages
– Consideration of multiple working hypotheses
– Highlights key predictive differences between hypotheses
– Emphasis on falsification leads to simple, testable hypotheses
Bayesian Inference
– Multiple working hypotheses may not always be available
– Must have the ‘correct’ hypothesis as one of the alternatives
Bayesian Inference
• The null hypothesis is the starting point for hypothesis testing
• Reject null hypothesis then move on to more complex hypotheses
• But where to start?
Bayesian Inference
• Do we start from a completely ‘null’ hypothesis?
• “There is no relationship between light and photosynthesis?”
Bayesian Inference
• Using a bit of knowledge of plant physiology we can formulate a more realistic initial hypothesis:
No Relationship
• The relationship should be non‐linear representing an upper limit on photosynthesis
No Relationship
3
9/14/2016
Bayesian Inference
• Data could then be used to test the support of this hypothesis
Information‐Theoretic Approach
Bayesian Inference
• Bayesian approach
– Use prior knowledge or information to generate and test hypotheses
– Could also be based on published literature
– Subjective?
– Compared to no prior information?
Approach?
• Simultaneously assess support for multiple, plausible hypotheses …
Testing Statistical Hypotheses
• The statistical null hypothesis H0 is usually one of ‘no pattern’
– No difference between groups
– No effect of one variable on another
Statistical Significance and P‐values
• “The control and treatment groups differed significantly from one another (P = 0.01)”
• What does this mean?
• The alternative hypothesis HA: the pattern exists
– Difference in means measured between groups
4
9/14/2016
Statistical Significance and P‐values
Statistical Null Hypothesis (H0)
• All statistical tests produce a test statistic, which is the numerical result of the test, and a probability value associated with that test
• “differences between groups are no greater than we would expect due to random variation”
• Statistical Null Hypothesis: the effect does not
operate
Statistical Alternative Hypothesis (HA)
Statistical Alternative Hypothesis (HA)
• With H0 stated, can propose one or more alternative hypotheses
• Strength of the tests and inference depend on how well the experiment is designed
• In most tests, HA is not explicitly stated because there are many alternate explanations
– Confounding factors?
The P‐value
The P‐value
• In many statistical analyses, we ask whether H0 can be rejected
• The P‐value measures the probability that the observed or more extreme differences would be found if the null hypothesis is true
• Specific to the test statistic (and therefore the test used)
• Therefore, small P‐values indicate that the result is improbable given the null hypothesis
• A large P‐value indicates that the observed result did occur given H0
• Thus the importance of meeting the assumptions of our test(s)
5
9/14/2016
The P‐value
• For comparing 2 groups, P‐values determined by
– The number of observations (n)
– The difference between the means of the samples
What is a significant P‐value?
• The operational critical value is P ≤ 0.05
• “Significant”
• Should we be rigid in this interpretation?
– The among of variation among individuals (s2)
What is a significant P‐value?
• Sets a standard or 1 in 20 (high) standard for testing
• This set point (α) determines the Type I error of a test
• Still not a safeguard against a poorly designed experiment
6