Download The Scientific Method

Practice & Communication of Science
The Scientific Method
@UWE_KAR
The Scientific Method

An evidence-based approach to determining
how the world works


how it works, not how it looks
ie observations used to propose mechanisms

“About thirty years ago there was much talk that
geologists ought only to observe and not theorise;
and I well remember some one saying that at this
rate a man might as well go into a gravel-pit and
count the pebbles and describe the colours. How
odd it is that anyone should not see that all
observation must be for or against some view if it is
to be of any service!”


Charles Darwin (1861)
Science provisional but used to defend views
The Scientific Method






General approach…
Theory(ies) generated to explain past
observations
Hypothesis (testable proposition) generated
to predict how the system will behave
Experiment – the hypothesis is tested
Analysis – did the system behave as
expected?
Refinement of theory – ie back to the
beginning…
The Scientific Method: Early Example

1796 - Edward Jenner and vaccination…

Theory(ies) generated to explain past observations


Hypothesis (testable proposition) generated to predict how
the system will behave


Infect a person with cowpox. Then try to infect the person with
smallpox.
Analysis – did the system behave as expected?


A person intentionally infected with cowpox won’t get ill after
subsequent exposure to smallpox
Experiment – the hypothesis is tested


It seems people who’ve had cowpox don’t get smallpox, so could cowpox
protect against smallpox?
The person did not develop smallpox!
Refinement of theory – ie back to the beginning…


Hmm, this might apply to diseases in general…
Smallpox recently eradicated (except in bio weapons labs! )
Science as an Evolutionary Process?

Evolution/Natural Selection arguably the most
important theory in biological science





Generate variation in population
Selection process selects ‘fittest’ individuals
Fittest variants go round the cycle again
Survivors embody ‘knowledge’ of their environment
Scientific Method’s central role in science




Theories/hypotheses
 Generate various explanations for phenomena
Experiments/analyses/conclusions/discussion/argument
 The selection process selects ‘fittest’ explanations
Refine theories/hypotheses
 Fittest explanations go round the cycle again
Surviving theories embody our knowledge of reality
Science as an Evolutionary Process?

Evolution uses the environment to select
individuals; science uses the scientific
community to select competing theories



Often opposing views – little ‘fence-sitting’
Peer-assessed publications/conferences the arena
‘Objective’, mechanistic reporting & decision-making


‘data suggest’ (not ‘my theory’), anonymous review
Evolution has a ‘direction’ arising from ‘random’
change; scientific progress also appears chaotic

“The stumbling way in which even the ablest of the scientists in every
generation have had to fight through thickets of erroneous observations,
misleading generalisations, inadequate formulations, and unconscious
prejudice is rarely appreciated by those who obtain their scientific
knowledge from textbooks.”
 JB Conant (1893-1978), Science and Common Sense
Science as an Evolutionary Process?

Evolution can occur in ‘bursts’ – punctuated
equilibrium; science can also undergo major
changes in brief periods of time

These are the ‘scientific revolutions’

or ‘paradigm shifts’


TS Kuhn, The Structure of Scientific Revolutions
(1970)
Replaces one conceptual worldview with another…




Geocentric  Heliocentric view of the heavens
Newtonian  Relativistic physics
Static earth  Plate Tectonics
Various ‘explanations’  Darwinian evolution
 Rather slow uptake – too revolutionary??
Science is ‘Just Theories’

Facts

individual truths known by observation/experience


Hypothesis

a testable proposition (but must be disprovable!)


apples, if dropped, fall to earth
apples fall due to a force, gravity, acting between
masses
Theory

a widely-accepted explanation accounting for a set
of observations


based on past facts/experiments, can make future
predictions
inverse square law; apples/earth; earth/moon
Science is ‘Just Theories’

Laws

sometimes, we have been so sure of our theories
we generalised them to ‘laws’




laws state things rather than explain ‘why’
eg Law of Conservation of Energy
a bit old-fashioned – suggests certainty
So, yes, science is ‘just theories’


but stronger because of it!
word ‘theory’ implies that things might be refined,
not that things are ‘wrong’


theory of evolution prime example of conflation of
‘theory’ with ‘wrong/unproven’
theory of gravity good enough for NASA
Role of Measurement in Science


Measurement permits greater detail and
consistency in our observation of the world
This typically involves the use of numerical
estimations…


"the
"the
"the
"the
sky
sky
sky
sky
is coloured"
is blue"
has colour in the range 455-492 nm"
has an absorption maximum of 472 nm“
We construct models to explain how the world
works (theories) and these derive from
numerical measurement

so theories, increasingly, tend to be mathematical
Mathematics and Science

The link between maths and science is both
profound and puzzling



Do gas molecules really know P1V1=P2V2?
“The Unreasonable Effectiveness of Mathematics in
the Natural Sciences.” E Wigner (1959)
Perhaps reality is mathematical


M Tegmark. Our Mathematical Universe (2014)
Regardless, maths, in the form of statistics, is
essential to becoming a practising scientist

permits the manipulation of measurements to…


describe them
draw inferences/extract meaning from their
comparison (hypothesis-testing)
The Problem with Measurement

Any act of measurement is accompanied by
uncertainty and error

Counting people in a room with absolute certainty









Perhaps
Perhaps
Perhaps
Perhaps
Perhaps
Perhaps
Perhaps
Etc, etc
they are moving around
someone is hiding
you run out of fingers
someone is pregnant
someone dies while you are counting
there is a waxwork dummy
someone is a shape-shifting alien
Not possible even with an ‘obvious’ measurement

even worse with complex systems!
The Problem with Complex Systems

Eg humans

We vary




We are ‘soft’


We all know a human when we meet one (?), but
there is huge variation in every parameter
We vary over time as well
We vary in our response to things
Can’t measure painfully or destructively! Ethics!
We can only measure comparatively few things
The Problem with Experiments

Eg we develop a drug that we think will lower
blood pressure…


We measure BP in a number of subjects before and
after the drug
Average BP changes


What are the possible explanations?




Unlikely to stay exactly the same
The drug affected the BP
Subjects’ blood pressures influenced by other things
Our measuring technique introduced variation
Can we decide, for certain, between the interesting
possibility (the drug) and other random effects?

No – never with absolute certainty!
The Problem with Certainty

Think what certainty about our BP-lowering
drug would require…

All subjects studied to have responded in the same
way


Us to know exactly what proportion of change was
due to random and measurement errors


How could we possibly know that without knowing
everything about everything?
All future subjects to respond similarly


How likely given that we vary so much?
How can we guarantee that?
Certainty is too ambitious

Best we can do is assign confidence to explanation
Replacing Certainty with Confidence

Think what confidence about our BP-lowering
drug would require…

All subjects studied to have responded in the same
way?


Us to know exactly what proportion of change was
due to random and measurement errors?


No. The odd rise in BP won’t undermine confidence
No. The bigger the average drop in BP & the smaller
the amount of variation in the effect, the more
confident we will be that the drug  BP change
That all future subjects respond similarly?

No. But repeating the experiment would be expected
to lead to same confident conclusion

In fact, confidence would actually increase!
The Central Role of Confidence

We can never be absolutely certain about the
outcome of experiments, only estimate a level
of confidence in our interpretation of results

Confidence expressed as a probability


Though confusingly, the probability refers to the
probability of nothing having happened!


0 – 1 or 0% – 100%
What is the probability of seeing a difference in BP
of this size if the drug didn’t work?
Because we can only ever assign confidence to
the outcome of our experiments, the same
applies to the theories we derive from them

Science is provisional and falsifiable – not absolute
The Central Role of Statistics

Statistics is the objective mechanism (the
‘judge’) which estimates confidence/probability


Statistics relies on an expectation that collected
data will vary/behave in a particular way
When an experiment is performed, the starting
assumption is that whatever you did will not affect
the way the data are distributed


the (in)famous Null Hypothesis (H0)
Statistical tests estimate the probability, p, that H0
is true

If p is low enough (typically 0.05) then H0 (nothing
happened) is rejected and replaced with the
Alternative Hypothesis, H1 (something happened!)
Statistics is a ‘Negative Friend’










Me – “Hey, statistics, I’ve developed a drug to lower BP – look at how my subjects’
BPs changed when I gave them the drug!”
Statistics – “You are fooling yourself and jumping to conclusions based on your own
expectations. I reckon the drop in mean BP you saw is down to random variation. I’ll
need some convincing to think otherwise!”
Me– “OK, maybe I’m being a bit premature. I guess there’s a chance I was unlucky
enough to end up with a set of readings through ‘random variation’ that make it look
like the drug works, even though it doesn’t. How much convincing do you need?”
Statistics – “I want a 1 in 20 or less chance of that ‘random’ explanation being true
before I’ll accept that it might be the drug altering the BP”.
Me– “Gosh, that’s quite a demanding level of proof!”
Statistics – “Well, you want confidence in any conclusion you reach, don’t you?”
Me– “I guess so. What factors are used to estimate the chance nothing happened?”
Statistics – “I might look at the difference between the mean values (you want to see
a big difference) the amount of variation is working out each mean (you want that to
be small) and the number of repeats (the larger the better).”
Me– “OK. Go for it.”
Statistics – “Sorry, it’s a 1 in 15 chance. You’ll have to accept that the drug didn’t
affect BP significantly. All is not lost, though. Perhaps you need to look at your
experimental design (drug dose, number of subjects, etc) and try again if you think
your drug really does affect BP”.









Summary
Science seeks to account for variation
Modern Science harnesses the Scientific
Method (experiment) rather than Scholasticism
(received wisdom)
The Scientific Method resembles evolution
Science recognises there is no absolute
knowledge (measurements incomplete/errors)
Science is fallible/provisional (ie it can change)
Yet scientific arguments don’t ‘sit on fences’
Science is fairly messy at its frontiers
Numerical measurement & maths is central
Statistics has vital role in the practice of science