Download Evaluation-Issues

Reliability and Validity
* How consistent the test is within itself - this
might be affected by different people collecting
the data differently in an observation or two
measures in the same test or tests which do not
match (IQ tests)
* How consistent the test is over time (for example
would Milgram’s test still be reliable today? –
the evidence, by the way, is that it is!)
Test Retest Method – Use the same participants (pps) and repeat the
same test after a period of time and correlate the results. If there is
a strong positive correlation, it is reliable.
Tighten your controls – are there any extraneous or confounding
variables that were not considered, that might have affected the
data – if there were then control for these. An example might be
personality in the Milgram experiment. He did not control for
personality and so it is possible his results showed how certain
personality types behave and not how everyone would behave.
Other sample controls that can be tightened is age of participants,
gender, culture, background and past experiment of the
participants etc.
If the study is observation then have more than one observer, collate
the data and if there is a strong positive correlation between
observers the data should be reliable. This is inter rater reliability
Internal – How properly the experiment was
conducted to produce truthful and accurate results
and therefore how likely it is to be measuring your
aim or hypothesis
External – Whether the results can be generalised
beyond the research situation to everyday life
situations
2. Ecological Validity…
…The extent to which the study’s results would occur
in a natural setting.
Internal Validity – check how the dependent variable is operationalised and
ensure this really does measure the hypothesis. For example does
gambling more than once a week on a fruit machine really make you a
regular gambler? Does a personality test really measure your personality?
Or – reduce demand characteristics by ensuring the participants do not know
the hypothesis and the aim of the researchers (Milgram)
Or -- reduce experimenter bias by ensuring the person who collects the data
does not know which group he is measuring (experimental or control) –
Maguire and Bandura did this!
Concurrent Validity – Correlate the findings of your experiment with
another measured. If there is a strong positive correlation, it is valid. For
example Baron Cohen did a measure of facial recognition to ensure it
correlated with the results of the eye test.
Ecological validity – ensure you research is carried out in a real or at least
realistic environment. Piliavin ensured high ecological validity and
Reicher and Haslam tried to recreate it in a an experimental setting.


Well Milgram was pretty reliable – he had a lot of controls - he
standardised his instructions, the situation and the procedure! He
has also repeated his experiment (so have others) and each time
very similar results have been found!
One thing Milgram did not ‘control’ for was the personality of his
participants. This leads us to the conclusion that by using a
volunteer sample it could be that his data was biased due to the
personality characteristics of his volunteers? Perhaps if he
measured their personalities first and then chose a selection of
different personalities or only measured one personality type he
may have obtained different results! For example perhaps
extroverts or very confident people are more likely to refuse to
obey and people who are more shy and timid are more likely to
obey?



There are two validity problems with Milgram:
1st internal validity – it could be said that
demand characteristics affected the data even
though he lied to the participants about the
nature of the hypothesis.
2nd ecological validity - the setting of a
university was not the type of setting where
you would expect to have to be thinking about
being obedient to an authority figure where
potential harm might be caused!



Point 1 – internal validity:
Milgram would have to reduce demand characteristics
and the only way he could do this would be not to let
people know they were taking part in an experiment the only option therefore would be to change his
method entirely.
One thing he did do to see how important the
university setting was on demand characteristics. He
moved his experiment down town to an office. Once
people did not know the experiment was for a
prestigious university department they were indeed
less obedient!


Point 2 –ecological validity
In order to make the Milgram experiment more valid
you would have to carry out the same or similar
experiment in a real life setting. Hoffman did this in
hospitals. He pretended to be a doctor on the phone to
nurses and ordered them to give dosages of medication
which was dangerous to patients without coming to
the ward and giving written authority. He found
without exception all the nurses were obedient to the
doctors (they were stopped before the medication was
given) even when they had suspicions that the
medication was dangerous.