Download Chapter 6

Chapter 6
Experiments in the Real World
Chapter 6
1
Example: The Case of Fickle Mice
• In the real world experiments don’t always go smoothly. Even
if they do we can’t always take a firm stand on the findings.
• Is our behavior coded into our genes?
 To find out, knock out a gene in one group of mice and compare their
behavior with a control group of normal mice.
 Mice have the same genetic makeup before the experiment, and each
mouse is randomly assigned to one group.
 Any difference during the experiment is then attributed to the
knocked-out gene.
• “No sooner has one group of researchers tied a gene to a
behavior when along comes the next study, proving that the
link is spurious or even that the gene in question has exactly
the opposite effect.” (published in the journal Science)
Chapter 6
2
Example: The Case of Fickle Mice
• To find out what goes wrong, scientists conducted
the same experiments with the same genetic strain
in three different labs (Oregon, Alberta, New York).
• The results were often different.
• It appears that very small differences in the lab
environments have big effects on the behavior of the
mice.
Chapter 6
3
Equal Treatment for All
• A sampler should know exactly what information she wants
and must compose questions that extract that information
from that sample.
• An experimenter must know exactly what treatments and
responses he wants information about, and he must construct
the apparatus needed to apply the treatments and measure
the responses.
• This is what is referred to as “designing an experiment”.
• The logic of a randomized comparative experiment assumes
that all the subjects are treated alike except for the
treatments that the experiment is designed to compare.
• Any other unequal treatment can cause bias.
Chapter 6
4
Example: Feeding Rats
Does a new cereal provide good nutrition?
• Compare the weight gains of young rats fed the new
product and rats fed a standard diet.
• Rats are placed in large racks of cages. Rats in upper
cages grow a bit faster than the ones in lower cages.
• If the experimenters put rats fed the new product at
the top, the experiment is biased in favor of the new
product.
• Solution: assign rats to the cages at random.
Chapter 6
5
Completely Randomized Design
In a completely randomized
experimental design, all the
experimental subjects are allocated
at random among all the treatments.
Chapter 6
6
Placebo and Experimenter Effects
• The problem:
– people may respond differently when they know
they are part of an experiment.
• The solution:
– use placebos, control groups, and double-blind
studies when possible.
Chapter 6
7
Double-Blind Experiments
The powerful placebo
• Want balding men to keep their hair? Give them a
placebo! One study found out that 42% of balding men
maintained or increased the amount of hair on their
heads when they took a placebo.
• Another study told 13 people who were very sensitive to
poison ivy that the stuff being rubbed on one arm was
poison ivy. It was a placebo, but all 13 broke out in rash.
• When the stuff rubbed on the other arm really was
poison ivy, but the subjects were told it was harmless,
and only 2 developed rash.
Chapter 6
8
Double-Blind Experiments
• Knowing that they are getting “just a placebo” might
weaken the placebo effect and bias the experiment
in favor of the other treatments.
• Also, if the doctors and other medical personnel
know if a certain subject is receiving “just a placebo”
they might expect less.
• Doctors’ expectations change how they interact with
patients and even the way they diagnose a patient’s
condition.
Chapter 6
9
Double-Blind Experiments
• In a double-blind experiment, neither the subjects nor the
people who work with them know which treatment each
subject is receiving.
Chapter 6
10
Refusals, Nonadherers, and Dropouts
• Sample surveys suffer from nonresponse due to failure to
contact some people selected for the sample and refusal of
others to participate.
• Subjects who participate but don’t follow the experimental
treatment, called nonadherers, can also cause bias.
• Experiments that continue over an extended period of time
also suffer dropouts, subjects who begin the experiment but
do not complete it.
Chapter 6
11
Can We Generalize?
• A well-designed experiment tells us that changes in the
explanatory variable cause changes in the response variable.
• It tells us that this happened for specific subjects in the
specific environment of this specific experiment.
• Can we generalize our conclusions from our little group of
subjects to a wider population?
 The first step is to make sure that our findings are statistically
significant, that they are too strong to occur just by chance.
 The treatments, the subjects, and/or the environment may not be
realistic.
Chapter 6
12
Can We Generalize?
• The problem:
 Lack of generalizability is due to:
 unrealistic treatments
 unnatural settings
 a sample that is not representative of the population
• The solution:
 Researchers should use natural settings with a properly chosen
sample.
• Good experiments combine statistical principles with
understanding of a specific field of study.
Chapter 6
13
Example: Center Brake Lights
• Cars sold in the U.S. since 1986 are required to have a center brake
light. This requirement was justified by randomized comparative
experiments with fleets of rental and business cars.
• The experiments showed that the third brake light reduced rearend collisions by 50%.
• After a decade in actual use, the Insurance Institute found only a
5% reduction in rear-end collisions.
• What happened? At the time the first experiment was carried out
most cars did not have center brake lights, so it caught the attention
of the drivers.
• Now that almost all the cars have the light, it no longer captures
their attention.
• The experiment’s conclusions did not generalize as well as
expected, because the environment changed.
Chapter 6
14
Experimental Design in the Real World
• A completely randomized design can have any
numbers of explanatory variables, and they might
interact in their effect on the response variable.
Chapter 6
15
Interacting Variables
• The problem:
The effect of an explanatory variable on a response
variable may vary over levels of other variables.
• The solution:
Measure and study potential interacting variables.
Does the relationship between explanatory and
response variables change for different levels of these
interacting variables?
If so, report results for different groups defined by the
levels of the interacting variables.
Chapter 6
16
Example: Effects of TV Advertising
• What are the effects of repeated exposure to an advertising
message? The answer may depend both on the length of the ad
and on how often it is repeated.
• In an experiment, all subjects viewed a 40-minute TV program that
included ads for a digital camera. Some subjects saw a 30-second
commercial; others, a 90-second version.
• The same commercial was repeated 1, 3, or 5 times during the
program.
• After viewing, all subjects answered questions about their recall of
the ad, their attitude toward the camera, and their intention to
purchase it (the response variables).
• Explanatory variables:
 length of the commercial, with 2 levels,
 Repetitions, with 3 levels.
Chapter 6
17
Chapter 6
18
Matched Pairs and Block Designs
• One common design that combines matching with
randomization is the matched pairs design, which
compares just two treatments.
• Assign one of the treatments to each subject
randomly.
• The order of treatments can influence the subject’s
response, so we randomize the order for each
subject.
Chapter 6
19
Example: Coke versus Pepsi
• Pepsi wanted to demonstrate that Coke drinkers prefer Pepsi when
they taste both colas blind.
• The subjects, all of whom said they were Coke drinkers, tasted both
colas from glasses without brand markings.
• Since the response may depend on which cola is tasted first, the
order of tasting should be chosen at random for each subject.
• When more than half the Coke drinkers chose Pepsi, Coke claimed
that the experiment was biased.
• Pepsi glasses were marked M and Coke glasses were marked Q.
Coke claimed that the results could just mean that people like the
letter M better than the letter Q.
• Matched pairs design is OK, but any distinction other than Coke vs
Pepsi should have been avoided.
Chapter 6
20
A block design combines the idea of creating equal groups
by matching with the principle of forming treatment groups
at random.
They control for the effect of some outside variables by
bringing them into the experiment to form the blocks.
Chapter 6
21
Example: Men, Women, and Advertising
• Women and men respond differently to advertising. An
experiment to compare the effectiveness of three TV
commercials for the same product will want to look at the
reactions of men and women, as well as assesses the overall
response to the ads.
• One could randomly assign subjects to three treatment
groups without regard to their sex.
• A better design considers women and men separately.
Randomly assign women to three groups, one to view each
commercial. Then separately assign men at random to three
groups.
Chapter 6
22
Chapter 6
23
Key Concepts
• Double-Blind Experiment
• Difficulties and Disasters
• Experimental Designs
– Completely Randomized Design
– Matched Pairs Design
– Block Design
Chapter 6
24
Exercise 6.13
• Comparing corn varieties. New varieties of corn with altered amino acid
content may have higher nutritional value than standard corn, which is low
in the amino acid lysine. An experiment compares two new varieties, called
opaque-2 and floury-2, with normal corn. The researchers mix corn-soybean
meal diets using each type of corn at each of three protein levels: 12%
protein, 16% protein, and 20% protein. They feed each diet to 10 one-day
old male chicks and record their weight gains after 21 days. The weight gain
of the chicks is a measure of the nutritional value of their diet.
a) What are the individuals and the response variable in this experiment?
b) How many explanatory variables are there? How many treatments? Use a
diagram to describe the treatments. How many experimental individuals
does the experiment require?
c) Use a diagram to describe a completely randomized design for this
experiment. (Don’t actually do the randomization.)
Chapter 6
25