Download ppt_w3-2 - Statistics

Chapter 3
Producing Data
Introduction to the Practice of
STATISTICS
SEVENTH
EDI T I O N
Moore / McCabe / Craig
Lecture Presentation Slides
Chapter 3
Producing Data
Introduction
3.1 Design of Experiments
3.2 Sampling Design
3.3 Toward Statistical Inference
3.4 Ethics
2
Introduction
Anecdotal Data
Available Data
Sample Surveys and Experiments
Observation vs. Experiment
3
Obtaining Data
Available data are data that were produced in the past for some
other purpose but that may help answer a present question
inexpensively. The library and the Internet are sources of available
data.
Beware of drawing conclusions from our own experience or
hearsay. Anecdotal evidence is based on haphazardly selected
individual cases, which we tend to remember because they are
unusual in some way. They also may not be representative of any
larger group of cases.
Some questions require data produced specifically to answer them.
This leads to designing observational or experimental studies.
4
Observation vs. Experiment
When our goal is to understand cause and effect, experiments are the
only source of fully convincing data.
The distinction between observational study and experiment is one of
the most important in statistics.
An observational study observes individuals and measures
variables of interest but does not attempt to influence the
responses. The purpose is to describe some group or situation.
An experiment deliberately imposes some treatment on
individuals to measure their responses. The purpose is to study
whether the treatment causes a change in the response.
In contrast to observational studies, experiments don’t just observe
individuals or ask them questions. They actively impose some treatment
in order to measure the response.
5
Confounding
Observational studies of the effect of one variable on another often fail
because of confounding between the explanatory variable and one or
more lurking variables.
A lurking variable is a variable that is not among the
explanatory or response variables in a study but that may
influence the response variable.
Confounding occurs when two variables are associated in
such a way that their effects on a response variable cannot be
distinguished from each other.
Well-designed experiments take steps to avoid confounding.
6
3.1 Design of Experiments
Experimental Units, Subjects, Treatments
Comparative Experiments
Bias
Principles of Experimental Design
Statistical Significance
Matched Pairs Design
Block Design
7
Individuals, Factors, Treatments
An experiment is a statistical study in which we actually do something
(a treatment) to people, animals, or objects (the experimental units)
to observe the response. Here is the basic vocabulary of
experiments.
The experimental units are the smallest collection of individuals to
which treatments are applied. When the units are human beings,
they often are called subjects.
The explanatory variables in an experiment are often called factors.
A specific condition applied to the individuals in an experiment is
called a treatment. If an experiment has several explanatory
variables, a treatment is a combination of specific values of these
variables.
8
Comparative Experiments
Experiments are the preferred method for examining the effect of one
variable on another. By imposing the specific treatment of interest and
controlling other influences, we can pin down cause and effect. Good
designs are essential for effective experiments, just as they are for
sampling.
A high school regularly offers a review course to prepare students for the SAT.
This year, budget cuts will allow the school to offer only an online version of the
course.
Students → Online Course → SAT Scores
Over the past 10 years, the average SAT score of students in the classroom
course was 1620. The online group gets an average score of 1780. That’s
roughly 10% higher than the long-time average for those who took the classroom
review course.
Is the online course more effective?
How would you know?
Are you certain the increase is due to the online course?
9
Comparative Experiments
Many laboratory experiments use a design like the one in the online
SAT course example:
Experimental
Units
In the laboratory environment, simple designs often work well.
Field experiments and experiments with animals or people deal
with more variable conditions.
Outside the laboratory, badly designed experiments often yield
worthless results because of confounding.
10
Randomized Comparative
Experiments
The remedy for confounding is to perform a comparative experiment
in which some units receive one treatment and similar units receive
another. Most well-designed experiments compare two or more
treatments.
Comparison alone isn’t enough. If the treatments are given to groups
that differ greatly, bias will result. The solution to the problem of bias is
random assignment.
In an experiment, random assignment means that
experimental units are assigned to treatments at random, that
is, using some sort of chance process.
11
Randomized Comparative
Experiments
In a completely randomized design, the treatments are
assigned to all the experimental units completely by chance.
Some experiments may include a control group that receives
an inactive treatment or an existing baseline treatment.
Group 1
Experimental
Units
Random
Assignment
Group 2
12
Randomization
One way to randomize an experiment is to rely on random digits to
make choices in a neutral way. We can use a table of random digits (like
Table B) or the random sampling function of a statistical software.
How to randomly choose n individuals from a group of N:
We first label each of the N individuals with a number (typically from 1 to
N, or 0 to N − 1).
A list of random digits is parsed into digits the same length as N (if N =
233, then its length is 3; if N = 18, its length is 2).
The parsed list is read in sequence and the first n digits corresponding to
a label in our group of N are selected.
The n individuals within these labels constitute our selection.
13
Principles of Experimental Design
Randomized comparative experiments are designed to give good
evidence that differences in the treatments actually cause the
differences we see in the response.
Principles of Experimental Design
1. Control for lurking variables that might affect the response, most
simply by comparing two or more treatments.
2. Randomize: Use chance to assign experimental units to
treatments.
3. Replication: Use enough experimental units in each group to
reduce chance variation in the results.
An observed effect so large that it would rarely occur by chance is
called statistically significant.
A statistically significant association in data from a well-designed
experiment does imply causation.
14
Cautions About Experimentation
The logic of a randomized comparative experiment depends on our
ability to treat all the subjects the same in every way except for the
actual treatments being compared.
In a double-blind experiment, neither the subjects nor those who
interact with them and measure the response variable know which
treatment a subject received.
The most serious potential weakness of experiments is lack of
realism. The subjects or treatments or setting of an experiment may
not realistically duplicate the conditions we really want to study.
15
Matched Pairs
A common type of randomized block design for comparing two
treatments is a matched pairs design. The idea is to create blocks by
matching pairs of similar experimental units.
A matched pairs design is a randomized blocked experiment
in which each block consists of a matching pair of similar
experimental units.
Chance is used to determine which unit in each pair gets each
treatment.
Sometimes, a “pair” in a matched pairs design consists of a
single unit that receives both treatments. Since the order of the
treatments can influence the response, chance is used to
determine which treatment is applied first for each unit.
16
Blocked Designs
Completely randomized designs are the simplest statistical designs for
experiments. But just as with sampling, there are times when the
simplest method doesn’t yield the most precise results.
A block is a group of experimental units that are known before the
experiment to be similar in some way that is expected to affect the
response to the treatments.
In a block design, the random assignment of experimental units to
treatments is carried out separately within each block.
Form blocks based on the most important unavoidable sources of variability
(lurking variables) among the experimental units.
Randomization will average out the effects of the remaining lurking variables
and allow an unbiased comparison of the treatments.
Control what you can, block on what you can’
’t control, and randomize
to create comparable groups.
17
3.2 Sampling Design
Population and Sample
Voluntary Response Sample
Simple Random Sample
Stratified Samples
Undercoverage and Nonresponse
18
Population and Sample
The distinction between population and sample is basic to statistics. To
make sense of any sample result, you must know what population the
sample represents.
The population in a statistical study is the entire group of
individuals about which we want information.
A sample is the part of the population from which we actually
collect information. We use information from a sample to draw
conclusions about the entire population.
Population
Collect data from a
representative Sample...
Sample
Make an Inference about
the Population.
19
How to Sample Badly
The design of a sample is biased if it systematically favors certain
outcomes.
Choosing individuals who are easiest to reach results in a
convenience sample.
A voluntary response sample consists of people who choose
themselves by responding to a general appeal. Voluntary
response samples show bias because people with strong opinions
(often in the same direction) are most likely to respond.
20
Simple Random Samples
Random sampling, the use of chance to select a sample, is the
central principle of statistical sampling.
A simple random sample (SRS) of size n consists of n
individuals from the population chosen in such a way that every
set of n individuals has an equal chance to be the sample
actually selected.
In practice, people use random numbers generated by a
computer or calculator to choose samples. If you don’t have
technology handy, you can use a table of random digits.
21
How to Choose a SRS
A table of random digits is a long string of the digits 0, 1, 2, 3, 4, 5,
6, 7, 8, 9 with these properties:
Each entry in the table is equally likely to be any of the 10 digits
0–9.
The entries are independent of each other. That is, knowledge
of one part of the table gives no information about any other
part.
How to Choose an SRS Using Table B
Step 1: Label. Give each member of the population a numerical label
of the same length.
Step 2: Table. Read consecutive groups of digits of the appropriate
length from Table B.
Your sample contains the individuals whose labels you find.
22
SRS Example
Use the random digits provided to select an SRS of four hotels.
01 Aloha Kai
02 Anchor Down
03 Banana Bay
04 Banyan Tree
05 Beach Castle
06 Best Western
07 Cabana
69051
08 Captiva
09 Casa del Mar
10 Coconuts
11 Diplomat
12 Holiday Inn
13 Lime Tree
14 Outrigger
15 Palm Tree
16 Radisson
17 Ramada
18 Sandpiper
19 Sea Castle
20 Sea Club
21 Sea Grape
22 Sea Shell
23 Silver Beach
24 Sunset Beach
25 Tradewinds
26 Tropical Breeze
27 Tropical Shores
28 Veranda
64817 87174 09517 84534 06489 87201 97245
69 05 16 48 17 87 17 40 95 17 84 53 40 64 89 87 20
Our SRS of four hotels for the editors to contact is: 05 Beach
Castle, 16 Radisson, 17 Ramada, and 20 Sea Club.
23
Other Sampling Designs
The basic idea of sampling is straightforward: take an SRS from the
population and use your sample results to gain information about the
population.
A probability is a sample chosen by chance. We must know what
samples are possible and what chance, or probability, each
possible sample has.
Sometimes there are statistical advantages to using more complex
sampling methods. One common alternative to an SRS involves
sampling important groups (called strata) within the population
separately. These “sub-samples” are combined to form one stratified
random sample.
To select a stratified random sample, first classify the population
into groups of similar individuals, called strata. Then choose a
separate SRS in each stratum and combine these SRSs to form
the full sample.
24
Cautions About Sample Surveys
Good sampling technique includes the art of reducing all sources of
error.
Undercoverage occurs when some groups in the population
are left out of the process of choosing the sample.
Nonresponse occurs when an individual chosen for the sample
can’t be contacted or refuses to participate.
A systematic pattern of incorrect responses in a sample survey
leads to response bias.
The wording of questions is the most important influence on
the answers given to a sample survey.
25
3.3 Toward Statistical
Inference
Parameters and Statistics
Sampling Variability
Sampling Distribution
Bias and Variability
Sampling from Large Populations
26
Parameters and Statistics
As we begin to use sample data to draw conclusions about a wider
population, we must be clear about whether a number describes a
sample or a population.
A parameter is a number that describes some characteristic of the
population. In statistical practice, the value of a parameter is not
known because we cannot examine the entire population.
A statistic is a number that describes some characteristic of a
sample. The value of a statistic can be computed directly from the
sample data. We often use a statistic to estimate an unknown
parameter.
Remember s and p: statistics come from samples and
parameters come from populations.
We write µ (the Greek letter mu) for the population mean and σ for the
population standard deviation. We write x (x-bar) for the sample mean
and s for the sample standard deviation.
27
Statistical Estimation
The process of statistical inference involves using information from a
sample to draw conclusions about a wider population.
Different random samples yield different statistics. We need to be able to
describe the sampling distribution of possible statistic values in order to
perform statistical inference.
We can think of a statistic as a random variable because it takes numerical
values that describe the outcomes of the random sampling process.
Therefore, we can examine its probability distribution using what we
learned in earlier chapters.
Population
Sample
Collect data from a
representative Sample...
Make an Inference
about the Population.
28
Sampling Variability
This basic fact is called sampling variability: the value of a statistic
varies in repeated random sampling.
To make sense of sampling variability, we ask, “What would happen if we
took many samples?”
Population
Sample
Sample
Sample
Sample
Sample
Sample
Sample
Sample
29
Sampling Distributions
If we measure enough subjects, the statistic will eventually get very
close to the unknown parameter.
If we took every one of the possible samples of a certain size, calculated
the sample mean for each, and graphed all of those values, we’d have a
sampling distribution.
The population distribution of a variable is the distribution of
values of the variable among all individuals in the population.
The sampling distribution of a statistic is the distribution of
values taken by the statistic in all possible samples of the same
size from the same population.
In practice, it’s difficult to take all possible samples of size n to obtain
the actual sampling distribution of a statistic. Instead, we can use
simulation to imitate the process of taking many, many samples.
30
Bias and Variability
We can think of the true value of the population parameter as the
bull’s-eye on a target and of the sample statistic as an arrow fired
at the target. Both bias and variability describe what happens
when we take many shots at the target.
Bias concerns the center of the
sampling distribution. A statistic used
to estimate a parameter is unbiased
if the mean of its sampling distribution
is equal to the true value of the
parameter being estimated.
The variability of a statistic is
described by the spread of its
sampling distribution. This spread is
determined by the sampling design
and the sample size n. Statistics from
larger probability samples have
smaller spreads.
31
Managing Bias and Variability
A good sampling scheme must have both small bias and small
variability.
To reduce bias, use random sampling.
To reduce variability of a statistic from an SRS, use a larger sample.
The variability of a statistic from a random sample does not depend
on the size of the population, as long as the population is at least 100
times larger than the sample.
32
Why Randomize?
The purpose of a sample is to give us information about a larger
population. The process of drawing conclusions about a population on
the basis of sample data is called inference.
Why should we rely on random sampling?
1.To eliminate bias in selecting samples from the list of
available individuals.
2.The laws of probability allow trustworthy inference about the
population.
Results from random samples come with a margin of
error that sets bounds on the size of the likely error.
Larger random samples give better information about the
population than smaller samples.
33
3.4 Ethics
Basic Data Ethics
Institutional Review Boards
Informed Consent
Confidentiality
Clinical Trials
Behavioral and Social Science Experiments
34
Basic Data Ethics
The most complex issues of data ethics arise when we collect data
from people.
Basic Data Ethics
The organization that carries out the study must have an
institutional review board that reviews all planned studies in
advance in order to protect the subjects from possible harm.
All individuals who are subjects in a study must give their
informed consent before data are collected.
All individual data must be kept confidential. Only statistical
summaries for groups of subjects may be made public.
35
Institutional Review Boards
The organization that carries out the study must have an institutional
review board that reviews all planned studies in advance in order to
protect the subjects from possible harm.
The purpose of an institutional review board is “to protect the rights
and welfare of human subjects (including patients) recruited to
participate in research activities.”
The institutional review board:
reviews the plan of study
can require changes
reviews the consent form
monitors progress at least once a year
36
Informed Consent
All subjects must give their informed consent before data are
collected.
Subjects must be informed in advance about the nature of a study
and any risk of harm it might bring.
Subjects must then consent in writing.
Who can’t give informed consent?
prison inmates
very young children
people with mental disorders
37
Confidentiality
All individual data must be kept confidential. Only statistical
summaries may be made public.
Confidentiality is not the same as anonymity. Anonymity prevents
follow-ups to improve non-response or inform subjects of results.
Separate the identity of the subjects from the rest of the data
immediately!
Example: Citizens are required to give information to the government
(tax returns, social security contributions). Some people feel that
individuals should be able to forbid any other use of their data,
even with all identification removed.
38
Clinical Trials
Clinical trials study the effectiveness of medical treatments on actual
patients—these treatments can harm as well as heal.
Points for a discussion:
Randomized comparative experiments are the only way to see
the true effects of new treatments.
Most benefits of clinical trials go to future patients. We must
balance future benefits against present risks.
The interests of the subject must always prevail over the interests
of science and society.
In the 1930s, the Public Health Service Tuskegee study recruited 399
poor blacks with syphilis and 201 without the disease in order to
observe how syphilis progressed without treatment. The Public
Health Service prevented any treatment until word leaked out and
forced an end to the study in the 1970s.
39
Behavioral and Social Science
Experiments
Many behavioral experiments rely on hiding the true purpose of the
study.
Subjects would change their behavior if told in advance what
investigators were looking for.
The “Ethical Principles” of the American Psychological Association
require consent unless a study merely observes behavior in a public
space.
40
Chapter 3
Producing Data
Introduction
3.1 Design of Experiments
3.2 Sampling Design
3.3 Toward Statistical Inference
3.4 Ethics
41