Download Experiment, observation

Research strategies &
Methods of data collection
Experiment
Observation
Significance
• Statistical significance: the measured
effect or connection etc. is likely to truly
exist (it is not likely to be the consequence
of randomness).
• Practical significance: the effect is big
enough or the connection is strong enough
to be practically important?
Significance
• Type I error: rejecting a null hypothesis
that is true
– The significance level α is the probability of
making the wrong decision when the null
hypothesis is true.
• Type II error: failing to reject the null
hypothesis when it should be rejected.
Ways of investigation
(research strategies)
• Choosing a research strategy:
– Experiment
– Survey
– Archival and documentary research
– Case study (!)
– Action research (emergent and iterative;
solutions to real problems;
participative&collaborative; mixed knowledge)
– Grounded theory (reality is socially constructed;
developing explanations to social interactions;
inductive/abductive)
Experiments
• The most „natural science like” method
• Less frequently used in economics (with
the exception of experimental economics),
but is fairly accepted in management
• The idea is: if everything is kept constant
or under control except the one
experimental stimulus, than causality can
be identified and its impact measured
The classical experiment
• The dependent variable and the
independent variable are identified
• Pretesting and posttesting are conducted
• Experimental and control groups are given
Major types
• Laboratory (lab) experiment
• Natural experiment
The classical experiment
• Experimental and control groups formed
• Experimental group:
1. Pretest
2. Stimulus
3. Posttest
• Control group:
1. Pretest
2. No stimulus
3. Posttest
Assumptions of the classical
experiment
• The control and the experiental group are
identical (as similar as possible). Ways to
accomplish:
– Probability sampling
– Randomization
– Matching
• No other impact should be on the groups
• No bias from the researcher or from the
participants
Biases from the participant side
• Placebo-effect
• Hawthore-effect
Researcher bias
• Biased perception based on expectations
• Ways to avoid this:
– Rigorous and strict operationalization
– More objective measurement methods
– Measurement is based on tools and machines
– Training of the researchers
– Double blind experiments
Advantages
•
•
•
•
•
Causality is measurable
No need for representativeness
Repeatability
Inexpensive (relatively)
Scientific rigour
Disadvantages
• Artificial
• Natural experiments are rare
• Loose connection with complex, real
situations
Threats to internal validity
–
–
–
–
–
–
–
–
history
maturation
Testing effect
instrumentation
Statistical regression
Selection biases
Experimental mortality
demoralization
Threats to external validity
It is not reality: even the pretest can change
the situation.
A possible solution:
Solomon four group design
Preexperimental research
designs
• Not real experiments
• There are three posible violations
Observation
Definition
• Systematic viewing, recording, description,
analsys and interpretation of behavior
and/or processes
• Two traditional types:
– Participant observation
– Structured observation
• Two new, additional types:
– Internet mediated observation
– Videography
Participant observation,
researcher roles
Decision on role
•
•
•
•
•
•
Purpose of research
Status of the reseacher
Time
Degree of feeling suited to be a participant
Access
Ethics
Data collection
• Note making and recording
• Progressing data collection
– Descriptive observation
– Narrative account
– Focused observation
Data quality
• Observer error (misinterpreting), observer
drift (changing interpretetion)
• Observer bias (subjective view)
Informant verification can decrease this
bias.
• Observer effect.
Minimal interaction, habituation can help.
Advantages of participant
observation
Disadvantages of participant
observation
Structured observation
• High levelof peetermined structure.
• Aim is to quantify behavior (how often?
rather than why?).
Data collection
• The use of coding schedules
Data quality
• Informant error (not the normal output is
observed)
• Time error (untypical)
Advantages / disadvantages