Download PollPosters - California State University, East Bay

Classroom Simulation:
The Margin of Error in a
Public Opinion Poll
JSM 2005 — Minneapolis
Bruce E. Trumbo
Eric A. Suess
Shuhei Okumura
California State University, East Bay
(formerly CSU Hayward)
[email protected]
[email protected]
Pedagogy of Probability
 Euclid formulated axioms of geometry
2300 years ago.
 Kolmogorov's axioms of probability: 1935
 Why the time gap? Probability is harder.
 “Practical experience” precedes formalism.
 You can draw a triangle with a stick in the
sand. Pyramid builders used geometry.
 Probability revealed in “large numbers.”
“Labs”: Gambling casinos began in 1600s.
Role of Simulation in Class
 Small-scale random experiments using
real objects can be used for introduction.
(Coins, dice, drawing beads from a bag.)
But time consuming, yield small-scale results.
 Computer simulation provides large-scale
results quickly.
 High-quality software necessary. Minitab,
SPSS, SAS, S-Plus, R are OK (not Excel).
R excellent for simulations and free, but not
point-and-click: www.r-project.org.
Presentations at Various Levels
 Basic statistics “service” course:
 Present as a slide show.
 No discussion of R programming.
 Prelude to discussion of normal (and possibly
normal approximation to binomial).
 Calculus prereq. statistics/probability course:
 Run multiple simulations in R.
 Discuss normal and binomial distributions.
 Perhaps introduce simple R code.
 Senior or first-year MS level:
 Also discuss Feller’s Arcsine Law.
Illustration: “Margin of Error”
in a Public Opinion Poll
Many polls quote a margin of sampling error
based on the number n of people sampled:
  2% for n = 2500
  3% for n = 1100
  4% for n = 625
For a candidate's “lead”: Double the margin of error.
What is the basis for these numbers?
 Assume a random sample.
 Ignore administrative / subjective issues
(nonresponse, phrasing of questions,
deceptive answers, etc.).
 Margin of error based on probability rules:
hard to prove at an elementary level.
 Simulation illustrates these rules.
Experimental, exploratory approach.
Simulating an Election Poll
To simplify, suppose:
 No undecided voters in population.
 53% of all voters favor Candidate “A”.
 47% of all voters favor Candidate “B”.
 Randomly sample 25 voters.
Can we detect from data on 25 subjects
that Candidate “A” is in the lead?
One Simulation Using R
> sample(0:1, 25, rep=T, prob=c(.47, .53))
[1] 1 0 1 1 0 0 1 0 0 1 0 0 0
[14] 0 1 0 1 0 1 1 1 0 1 1 0
Interpret 0s & 1s as candidate preferences
A B A A B B A B B A B B B
B A B A B A A A B A A B
Only 12 / 25 = 48% for “A”. Misleading result.
What can we conclude from this small poll?
Endpoint is 48%. What would other polls say?
Note: Here and beyond, vertical axis from 25% to 75% to help show detail.
Results Differ Widely. Next: On one graph, overlay 20 traces.
All 20 traces end between 40% and 70%, most above 50%.
Of 1000 traces, 377 (more than 1/3) end below 50%.
These give a false indication that P(A) may be less than 1/2.
A 25-subject poll can’t reliably show if “A” is winning.
Now Consider Polls with
n = 2500 Subjects
Major polling organizations
report such polls as having a
 2% margin of sampling error.
Intended Meaning:
If the true proportion for “A” is 53%,
then most 2500-subject polls (95%) will
have endpoints between 51% and 55%,
thus detecting that “A” is the favorite.
Short marks at right indicate 53%  2%. Reference line at 50%.
On “average” run: 19 out of 20 traces (95%) within margin of error.
95% of endpoints within 2% margin of error; only 3 in 1000 below 50%.
In the interval from 51% to 55% for Candidate “A” the peaked
curve has 95% probability; the flat curve only about 12%.
Feller's Arcsine Law
In above demos of margin of error and normal
distribution: P(A) = 53% and we focus on the
endpoint of each trace. Now change focus:
 Now P(A) = P(Heads) = 1/2 (think of a fair coin).
 For each of 10,000 traces, find the percentage of
steps where the trace is above 1/2.
Make a histogram of these percentages.
 Many students expect results to cluster around 1/2.
But actual cluster points are near 0 or 1.
 Histogram approximates BETA(1/2, 1/2), which
has an “arcsine” function as its CDF.
0.0
0.5
1.0
1.5
2.0
5000 Experiments Each With 10000 Tosses of a Fair Coin
0.0
0.2
0.4
0.6
0.8
1.0
Proportion of Time Head Count Exceeds Tail Count
Based on W. Feller (1957): Intro. to Probability Theory and Its Applications, Vol. I
[The "bathtub shaped" distribution is BETA(1/2, 1/2).]
In any one trace, either Heads or Tails is likely to stay fairly consistently in the lead.
References
Moore, David S.: Basic Practice of Statistics, 3rd ed., W. H. Freeman (2004) p479.
www.sci.csueastbay.edu/statistics/
Resources/Quiz/poll.htm
Feller, William: An intro. to probability theory and its applications, Vol. 1, 3 rd
ed., Wiley (1968)
Trumbo, Bruce E.: RU Simulating Column: “Polls on steroids,” STATS
magazine, Fall 2005. (Similar to parts of this poster presentation.)
www.r-project.org (Instructions for downloading and installing R.)
www.sci.csueastbay.edu/~btrumbo/JSM/pollsim
(R code, additional materials for this paper.)
Other S-Language Classroom Simulations at a Similar Level.
Trumbo, B. E.; Suess, E. A.; Schupp, C. W.: “Using R to compute probabilities
of matching birthdays,” Proc. JSM 2004. (Also see: STATS, Spring 2005.)
Trumbo, B. E., Suess, E. A; Fraser, C. M.: “Using Computer Simulation to
Investigate Relationships Between the Sample Mean and Standard Deviation,”
STATS, Fall 2001, pages 10-15.
Trumbo, B. E., Suess, E. A; Fraser, C. M.: “Contemporary statistical simulation methodology for undergraduates,” Proc. JSM 2000. (Also see: STATS, Winter 2000.)