Download You are probably familiar with the quote "There are three types of

April 26, 2006
STATISTICS AND THE LAW: SOME EXAMPLES AND THOUGHTS* by Dennis Gilliland,
Co-Director, CSTAT
Statistics as the science of data plays a role in virtually every human activity including Law. We
take this opportunity to give examples of the varied roles of statistics and probability in legal
settings and to offer comments on expert testimony.
You are probably familiar with the quote "There are three types of lies - lies, damn lies, and
statistics.” You may also be familiar with the declining prestige of the legal profession (The
Harris Poll® #69, September 8, 2005). The union of statistics and law may be a marriage made
in heaven or a playground for deviltry.
Some examples of statistics and probability in laws and regulations. Federal and state
agencies operationalize the intent of laws through administrative action and rules. These can be
far reaching and detailed descriptions of statistical methods for testing and monitoring of the
environment, product quality, safety, or even truth in advertising. For example, the Michigan
Public Service Commission requires a utility to employ ASTM sampling and test methodologies
to monitor the accuracy of its electricity meters. The Michigan Department of Environmental
Quality requires a paper mill to conduct specified tests of the effects of its effluent on water
quality, including the effect on the taste of fish. Before voiding the result of a contested election,
a court may require a probability, more than a possibility, of a reversed election result but for the
irregularities.
The Michigan Lottery Act 432.9 in Section (1) states that “The commissioner shall initiate,
establish and operate a state lottery…” In Section (2) the Act prohibits a lottery based on “skill.”
Rules employ the word random, and it is the ambiguous meaning of this word that the state later
used to defend itself following an error. The December 30, 1981 lottery to select five finalists
for a $1,000,000 drawing from a set 11,561 semi-finalists is described in Gilliland, Experiences
in Statistics (Kendall-Hunt, 1990, pp. 15 -17). The state executed a system that gave about sixty
times more probability of selection to some semi-finalists than other semi-finalists. Later the
commissioner of the lottery was quoted as saying “It is my belief that the drawing was fairly
conducted and that each of the contestants had a fair chance of winning.” In Bad Or, the
Dumbing of America (Random House Value Publishing, 1993, p. 194), Paul Fussell states that
“…..law is about little more than language precisely construed and effectively deployed.” We
1
might conclude in this lottery example that the lottery rules were neither precisely construed nor
effectively deployed.
Role of statistics in expert testimony. Since testimony often involves data and
information, it is not surprising that statistics and statistical inference show-up in court
testimony. Statistical testimony and those that proffer it should be judged against the
accepted practice in the professions (the Frye Rule of evidence resurrected) and the code
of ethics of the American Statistical Association (see www.amstat.org/profession/ ).
Under this code, an expert leaving out data that are unfavorable to his client would be in
violation of Section C. Responsibilities in Publications and Testimony, Part 5. Account
for all data considered in a study and explain the sample(s) actually used. (We expect
that a future issue of the CSTAT NEWS, Center for Statistical Training and Consulting,
Michigan State University, will feature an article on ethics in statistical analyses by
Professor Francis Dane, Finkbeiner Endowed Chair of Ethics, Saginaw Valley State
University. www.cstat.msu.edu )
Peter W. Huber in Galileo’s Revenge; Junk Science in the Courtroom (Basic Books, 1991, p. 17)
bemoans the loss of Frye. Huber documents damage resulting from the let-it-all-in approach to
expert testimony. (See Gilliland, The practice of statistics in legal proceedings, Proc. Social
Statistics Section, ASA 2001 Annual Meeting, Atlanta, 1 – 6 for three specific examples of
questionable testimony.)
The readers might be interested in an open letter dated 23 January 2002 from the Royal
Statistical Society to the Lord Chancellor regarding the use of statistical evidence in court cases
(www.therss.org.uk/archive/evidence/sc_letter.htlm). An unnamed expert had done one of those
weird probability calculations reminiscent of the Collins case. The egregious testimony was let
in though not tied to anything remotely sensible nor part of the science of statistics and
probability.
Attorneys may not favor the notion of a neutral expert, one anchored in the sea of science and
reviewed research. Paul Meier in Damned Liars and Expert Witnesses (J. Am. Statist. Assoc,
81, 1986, 269-276) gives us this quote: “John C. Shepherd of St. Louis, a distinguished trial
lawyer, who was President of the American Bar Association in 1984-1985, spoke to a conference
for lawyers on relations with the expert witnesses (Shepherd, 1973), and this is what he said: Many
people are convinced that the expert who really persuades a jury is the independent, objective,
nonarticulate type.... I disagree. I would go into a lawsuit with an objective, uncommitted,
independent expert about as willingly as I would occupy a foxhole with a couple of non-combatant
soldiers (pp. 21-22)."
Skepticism regarding the marriage of statistics and law will continue until the statistics
profession accepts professional standards and publicizes expert testimony. Until then, we can
expect some statisticians to blithely pick and choose procedures to please their sponsors or to get
statistical significance to please an editor. We end with the quote of remarks attributed to Justice
Justin J. Dewey by Arnold R. Brown in Lizzie Borden; The Legend, the Truth, the Final Chapter
(Rutledge Hill Press, 1991, p. 277). “Expert testimony constitutes a class of evidence which the
law requires you to subject to great scrutiny. It is a matter of frequent observation to see experts
2
of good standing expressing conflicting and irreconcilable views upon questions arising in a trial.
They sometimes manifest a strong bias or partisan spirit in favor of the party employing them.
They often exhibit a disposition to put forth theories rather than to verify or establish or illustrate
facts."
*Copyright © 2006 Dennis C. Gilliland
3