Download The Syllogism of Expert Testimony Edward J. Imwinkelried

The Syllogism of Expert Testimony
Edward J. Imwinkelried,
Professor of Law
University of California,
Davis, School of Law
29
I.
THE SYLLOGISM OF EXPERT TESTIMONY
The Potential Uses of Expert Witnesses
A.. Witnesses to observed facts – FRE 602
B. Witnesses expressing lay opinions – FRE 701
C.. Expert witnesses presenting a general lecture to the trier of fact – FRE 702 (“an
opinion or otherwise”)
D. Expert witnesses testifying to opinions derived by applying a general theory or
technique to the specific facts of the case – FRE 702
1. The witness’s qualification as an expert – FRE 702
The syllogism
2. The general theory or technique that the expert relies on – FRE 702(c)
3. The case-specific facts or data that the expert is evaluating – FRE 703
4. The application of the general theory or technique to the facts or data – FRE
702(d)
5. The final conclusion derived by applying the general theory or technique to the
case-specific facts and data – FRE 704
II. THE EXPERT’S MAJOR PREMISE: THE VALIDITY OF THE GENERAL THEORY OR
TECHNIQUE
A. The traditional view
The general acceptance standard under Frye v. United States, 293 F. 1013 (D.C.
Cir. 1923)
1 PAUL C. GIANNELLI, EDWARD J. IMWINKELRIED, ANDREA ROTH &
JANE CAMPBELL MORIARTY, SCIENTIFIC EVIDENCE § 1.06 (5th ed.
2012)
-----General acceptance as a proxy for empirical validity
Bert Black, Francisco J. Ayala & Carol Saffran-Brinks, “Science and the Law in
the Wake of Daubert: A New Search for Scientific Knowledge,” 72 TEXAS
LAW REVIEW 715 (1994)
-----The exemption for “soft” science
People v. McDonald, 37 Cal.3d 351, 208 Cal.Rptr. 236, 690 P.2d 709 (1984)(“a
30
machine”)
B. The trend toward the empirical standard
Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993)
The progeny of Daubert
General Electric Co. v. Joiner, 522 U.S. 136 (1997)
Kumho Tire Co. v. Carmichael, 526 U.S. 137 (1999)
Weisgram v. Marley Co., 528 U.S. 440 (2000)
The evolution of Frye into a distinct minority view
SCIENTIFIC EVIDENCE, supra, at §§ 1.-14-.16
But California’s continued adherence to Frye
People v. Leahy, 8 Cal.4th 587, 882 P.3d 321, 323, 34 Cal.Rptr.2d 663 (1994)
C. The recent Sargon decision
Sargon Enterprises, Inc. v. University of Southern California, 55 Cal.4th 747,
288 P.3d 1237, 149 Cal.Rptr.3d 614 (2012)
David L. Faigman & Edward J. Imwinkelried, “Wading into the Daubert Tide:
Sargon Enterprises, Inc. v. University of Southern California,” 64 HASTINGS
LAW JOURNAL 1665 (2013)
1. The background
a.
In re Lockheed Litig. Cases, 23 Cal.Rptr.3d 762 (Ct.App. 2005),
petition for review granted, 110 P.3d 289 ( Cal. 2005), petition for
dismissed, 83 Cal.Rptr.3d 478 (2007)
California Evidence Code § 801(b)
“If a witness is testifying as an expert, his testimony in the form of
an opinion is limited to such an opinion as is:
(b) Based on matter . . . perceived by or personally known to the
witness or made known to him at or before the hearing, whether or
not admissible, that is of a type that reasonably may be relied upon
by an expert in forming an opinion on the subject to which his
testimony relates, unless an expert is precluded by law from using
such matter as a basis for his opinion.”
An amicus brief filed by two academics
(1)–California Evidence Code § 802 as controlling
“A witness testifying in the form of an opinion may state on direct
the reasons for his opinion and the matters . . . upon which it is
based, unless he is precluded by law from using such reasons or
31
matter as a basis for his opinion.”
(2)–The existence of judicial power under § 802 to enunciate
uncodified restrictions on the theory or technique the expert
proposes relying on
California Evidence Code § 160: “‘Law’ includes constitutional,
statutory, and decisional law.”
(3)–The exercise of the power to authorize California trial judges
to conduct a circumscribed inquiry whether, as a matter of logic,
the reasons cited by the expert support the expert’s conclusion that
the theory or technique is valid
The dismissal of the case
b.
The article restating the position of the amicus brief
Edward J. Imwinkelried & David L. Faigman, “Evidence Code
Section 802: The Neglected Key to Rationalizing the California
Law of Expert Testimony,” 42 LOYOLA L.A. LAW REVIEW
427 (2009)
2. The Sargon decision itself
Edward J. Imwinkelried, “California Case Upsets Equilibrium in on
Expert Testimony,” NATIONAL LAW JOURNAL, May 20, 2013, at 16.
The scope of the decision:
–approving citations to Daubert, Joiner, and Kumho
–adopting the essential teachings of Daubert, Joiner, and Kumho
Daubert: the judge’s gatekeeping role to ensure the reliability
of the testimony
Kumho: the across-the-board application of the reliability
requirement to all types of expert testimony
Joiner: the evaluation of the validity of an expert’s analogical
reasoning
–endorsing the view that § 802 authorizes the enunciation of uncodified
restrictions on the types of theories and techniques that the expert may
rely on
32
–endorsing the view that § 802 authorizes the type of circumscribed
inquiry urged by the law review article
–but stopping short of authorizing the trial judge to pass on the credibility
of the foundational testimony under Evidence Code § 405, as the federal
trial judge may do under Federal Rule 104(a)
–and indicating in footnote six that Frye is still good law
III. THE EXPERT’S MINOR PREMISE: THE TRUSTWORTHINESS OF THE EVIDENCE
OF THE CASE-SPECIFIC FACTS THAT THE EXPERT APPLIES THE THEORY OR
TECHNIQUE TO
A.
The accepted methods of supplying the expert with the case-specific facts to the
evaluated
1. The expert’s personal observation of the fact
2. The hypothetical question
a. The requirement for the prior presentation of admissible evidence of
every element of the hypothesis
b. Special instructions to disregard the opinion if the jury rejects any
element of the hypothesis
3. Reliance on out-of-court reports if it is the reasonable (customary) practice of
the expert’s specialty to consider reports from such sources
a. The conventional wisdom that such reports are received for a
nonhearsay purpose under Federal Rule 801(c). The theory is mental
input, the effect on the state of mind of the hearer or reader. The expert’s
receipt of the report makes the expert’s opinion better grounded and
therefore more reasonable.
b. A limiting instruction under Federal Rule 105
c. No necessity for the earlier presentation of admissible evidence of the
secondhand fact
Advisory Committee Note, Federal Rule 703
B.
The impact of Williams v. Illinois, 132 S.Ct. 2221, 183 L.Ed.2d 89 (2012) on the
treatment of opinions based on out-of-court reports of secondhand facts
Edward J. Imwinkelried, “The Gordian Knot of the Treatment of Secondhand
33
Facts Under Federal Rule of Evidence 703 Governing the Admissibility of Expert
Opinions: Another Conflict Between Logic and Law,” 3 UNIVERSITY OF
DENVER CRIMINAL LAW REVIEW 1 (2013)
1. The Williams opinions
a.
Justice Alito’s plurality opinion
(1) The plurality’s position that the DNA report was nontestimonial
(2) The plurality’s position that the report was used for a
nonhearsay purpose
(3) The plurality’s concession that the falsity of an asserted
secondhand fact that is an essential premise of the opinion renders
the opinion irrelevant
(4) The plurality’s approval of a jury instruction that “[i]f the
statements in these questions are not supported by the proof, then
the answers to the questions are entitled to no weight . . . .”
b. Justice Thomas’ concurrence
(1) The justice’s position that the DNA report was non-testimonial
But for a different reason than the plurality
(2) The justice’s position that the report was used to prove the truth
of the assertion
c. Justice Kagan’s dissent
(1) The dissent’s position that the report was testimonial
(2) The dissent’s position that the report used to prove the truth of
the assertion
d. The upshot
(1) A formal holding that the statement was non-testimonial
The five: The plurality plus Justice Thomas
(2) A forceful dictum by five justices that reports received under
34
Federal Rule of Evidence 403 are received for the truth of the
assertion
The five: Justice Thomas plus the dissenters
2. A critical evaluation of the merits of the dispute between the plurality and the
other justices over the status of reports used under Rule 703
a. The five justices’ error in thinking that reports used under Rule 703
are necessarily received for the truth of the assertion
The legitimacy of a non-hearsay theory
b. The plurality’s concession that if a secondhand report is an essential
premise of the expert’s opinion, as a matter of logic, the falsity of the
report renders the opinion invalid and therefore irrelevant,
A conditional opinion
3. The fallout from Williams
a. The need for a a jury instruction that if a secondhand report that is an
essential premise of the expert’s opinion is false, the jury must reject the
opinion itself
The dissent goes beyond that, and the plurality concedes that.the
falsity of an essential secondhand report renders the opinion
invalid and irrelevant.
b. The question whether, in the longterm, Williams will blur the lines
between hypothetical questions and questions based on 703 secondhand
facts: As in the case of hypothetical questions, will the lower courts begin
demanding independent, admissible evidence of the truth of the
secondhand report? DAVID H. KAYE, DAVID E. BERNSTEIN &
JENNIFER L. MNOOKIN, THE NEW WIGMORE: EXPERT
EVIDENCE § 4.12.7, at 70-71 (2014 Cum.Supp.)
The logic
Versus
The law
35
The Advisory Committee Note, F.R.E. 703
------The committee’s observation that outside court,
experts often make “life-and-death decisions” on the basis
of such reports. The report is entitled to more weight than
an attorney’s assertion during opening statement.
-----The committee’s view that the proponent should not be
required to go to the length of spending the time to present
admissible evidence of the content of the report (“with[out]
the expenditure of substantial time in producing and
examining various authenticating witnesses”)
Oliver Wendell Holmes, Jr., “The life of the law has not been logic . . . .”
THE COMMON LAW 1 (1881)
IV. THE EXPERT’S FINAL CONCLUSION: EXPOSING MEASUREMENTS AS MERE
UNCERTAIN ESTIMATES–AND VIRTUALLY MEANINGLESS UNLESS
ACCOMPANIED BY A MATHEMATICAL INDICATION OF THE UNCERTAINTY SUCH
AS A CONFIDENCE INTERVAL
Edward J. Imwinkelried, “Forensic Metrology: The New Honesty About the Uncertainty
of Measurements in Scientific Analysis,” in PROCEEDINGS OF THE FOURTH
INTERNATIONAL CONFERENCE ON EVIDENCE LAW AND FORENSIC
SCIENCE 23 (Beijing, China 2013), to be reprinted in ___JOHN MARSHALL LAW
REVIEW ___(forthcoming 2014).
36
THE IMPORTANCE OF FORENSIC METROLOGY IN PREVENTING
MISCARRIAGES OF JUSTICE: INTELLECTUAL HONESTY ABOUT THE
UNCERTAINTY OF MEASUREMENT IN SCIENTIFIC ANALYSIS
Edward Imwinkelried*
* Edward L Barrett, Jr. Professor of Law, University of California, Davis; coauthor,
GIANNELLI, IMWINKELRIED, ROTH & CAMPBELL MORIARTY, SCIENTIFIC
EVIDENCE (5th ed. 2012). This article is based in part on the author=s presentation at the Fourth
International Conference on Evidence Law and Forensic Science, held in Beijing, China on July
20-21, 2013. Reprinted with the kind permission of China University of Political Science and
Law.
37
A[I]t would be unreasonable to conclude that the subject of scientific testimony must be >known=
to a certainty; arguably, there are no certainties in science.@
BJustice Blackmun, Daubert v. Merrell Dow Pharmaceuticals, Inc.1
I.
THE RISK THAT MISUNDERSTOOD STATISTICAL TESTIMONY CAN
CAUSE A MISCARRIAGE OF JUSTICE
At the end of the 19th century, Justice Holmes predicted that Athe man of the future is the
man of statistics.@2 History has proven Holmes right. Statistical testimony is now in widespread
use in American courtrooms:3
Today, complex statistical evidence is being introduced in American courts in a wide
variety of both civil and criminal matters. [A] LEXIS search of district court opinions
using the words, Astatistic,@ Astatistics,@ or Astatistical,@ turned up 608 examples in the
years 1960 to 1969; 2,786 cases from 1970 to 1979; 4,364 cases from 1980 to 1989; and
3,015 from 1900 through July 31, 1995.4
Although the courts are now receptive to statistical testimony, initially the courts were
skeptical about the ability of lay triers of fact to critically evaluate such testimony.5 The courts
feared that Astatistics are subject to a wide range of manipulation@6Bmanipulation that could
trigger a miscarriage of justice. The skeptics often pointed to the California Supreme Court=s
famous 1968 decision in People v. Collins7 as proof of that danger.
In Collins, two defendants, an Afro-American husband and his Caucasian wife, were
1
590 U.S. 579, 590 (1993).
2
Oliver Wendell Holmes, The Path of the Law, 10 Harv.L.Rev. 457, 469 (1897).
3
Jonathan Koehler, The Probity/Policy Distinction in the Statistical Evidence
Debate, 66 Tul.L.Rev. 141 (1991).
4
United States v. Shonubi, 895 F.Supp. 460, 514, 516-19 (E.D.N.Y. 1995),
vacated, 103 F.3d 1085 (2d Cir. 1997).
5
C. McCormick, Evidence ' 210 (6th ed. 2006).
6
United States v. State of Oregon, 782 F.Supp.2d 502, 513 (D.Or. 1991), aff=d, 28
F.3d 110 (9th Cir. 1994).
7
68 Cal.2d 319, 438 P.2d 33, 66 Cal.Rptr. 497 (1968)
38
charged with robbery. At trial, the prosecution had difficulty establishing the defendant=s identity
as the perpetrators. The victim did not identify the defendants, and one eyewitness gave
somewhat equivocal testimony. To corroborate the identifications, the prosecutor called a
mathematics instructor at a state college as a witness. During the witness=s direct examination,
the prosecutor listed several characteristics that witnesses had attributed to the perpetrators and
that the defendants happened to match. The prosecutor then asked the witness to assume a
probability for each characteristic:
party yellow automobile
man with mustache
girl with ponytail
girl with blond hair
Negro man with beard
interracial couple in car
1/10
1/4
1/10
1/3
1/10
1/1,000.8
The prosecutor then invited the witness to apply the product rule to this data. According to the
product rule, if individual probabilities are independent, the probability that they will concur is
the product of the multiplication of the individual probabilities.9 The witness complied and
testified to the figure of 1/12,000,000. In the prosecutor=s mind, the witness=s testimony
established
the probability there was but one chance in 12 million that any couple possessed the
distinctive characteristics of the defendants. Accordingly, under this theory, it was to be
inferred that there could be but one chance in 12 million that defendants were innocent . .
. .10
On appeal, the California Supreme Court reversed. The court found numerous flaws in
the foundation for the witness=s testimony:
$
There was no basis for the individual probabilities that the prosecutor asked the witness
to assume. The prosecutor had pulled the numbers out of thin air.11
$
Although the product rule may be applied only when the individual probabilities are truly
independent, there was obviously an inadequate showing of independence in the court
below.12 Again, one factor referred to a man with a moustache while another mentioned a
man with a beard.
8
438 P.2d at 37 n. 10.
9
1 Paul C. Giannelli, Edward J. Imwinkelried, Andrea Roth & Jane Campbell
Moriarty, Scientific Evidence ' 15.07[a], at 915-16 (5th ed. 2012).
10
438 P.2d at 37.
11
Id. at 38.
12
Id. at 39.
39
$
The prosecutor misstated the significance of the witness=s computation. Even if the
individual probabilities were accurate and the probabilities were independent, the
computation yielded only a random match probability.13 That computation answers the
question of how often you would have to dip into the relevant population to randomly
find another couple with the same set of characteristics?14 However, the prosecutor used
the figure as if it were a source probability.15 The latter probability addresses the
question of how probable it is that the defendants were the perpetrators.16 In order to
answer that question, the trier of fact would have to make an assumption about the size of
the relevant population.17 If only one person in a 1,000.000 in a population has a
characteristic but the relevant population size is 10,000,000, ten persons in the population
probably share the characteristic.
The court found that these errors were prejudicial, necessitating reversal. At the beginning of its
opinion, the court warned that mathematics is Aa veritable sorcerer in our computerized
society.@18 At the end of the opinion, the court asserted that A[u]ndoubtedly the jurors were
unduly impressed by the mystique of the mathematical demonstration but were unable to assess
its relevancy or value.@19
In a 2010 DNA case, McDaniel v. Brown,20 the United States Supreme Court struggled
with the question of whether a prosecutor=s misuse of a random match probability as a source
probability denied a defendant due process. McDaniel was a habeas proceeding rather than a
direct appeal. In McDaniel, the petitioner contended that in its arguments in the trial court the
government had committed A[t]he prosecutor=s fallacy . . . that the random match probability is
the same as the probability that the defendant was not the source of the DNA sample.@21 The
Court elaborated:
13
Id. at 40; 2 Paul C. Giannelli, Edward J. Imwinkelried, Andrea Roth & Jane
Campbell Moriarty, Scientific Evidence ' 18.04[c][3], at 105-06 (5th ed. 2012).
14
Id. at 105.
15
Id. at ' 18.04[c][3], at 106-08 (5th ed. 2012).
16
Id.
17
Id.
18
438 P.2d at 33..
19
Id. at 41.
20
McDaniel v. Brown, 558 U.S. 120 (2010).
21
Id. at 128.
40
In other words, if a juror is told the probability a member of the general population would
share the same DNA is 1 in 10,000 (random match probability), and he takes that to mean
there is only a 1 in 10,000 chance that someone other than the defendant is the source of
the DNA found at the crime scene (source probability), then he has succumbed to the
prosecutor=s fallacy. It is further error to equate source probability with probability of
guilt, unless there is no explanation other than guilt for a person to be the source of the
crime-scene DNA. This faulty reasoning may result in an erroneous statement that, based
on a random match probability of 1 in 10,000, there is a .01% chance the defendant is
innocent or a 99.99% chance the defendant is guilty.22
The AState concede[d] that [the prosecution witness] Romero had overstated [the] probative
value@ of the DNA evidence.23 Ultimately, though, the Court rejected the petitioner=s claim.
The Court found that there was ample other evidence to establish the petitioner=s guilt. The
Court concluded that together with the inculpatory non-DNA evidence in the record, a proper
statistical evaluation of the DNA match furnished Apowerful evidence of guilt.@24
The common denominator in these two highly publicized cases was the courts= concern
that there might have been a miscarriage of justice because the lay trier of fact could have
misused a numerical figure testified to by an expert. That concern is not limited to celebrated
cases such as Collins and McDaniel. For that matter, the concern is not confined to DNA cases.
The issue is virtually ubiquitous in cases involving scientific testimony. As Part II explains, the
same issue arises whenever an expert witness relies on a measurementBa common occurrence
when experts take the stand.
II.
THE EVOLUTION OF THE COURTS= APPROACH TO RECOGNIZING THE
UNCERTAINTY OF STATISTICAL TESTIMONY ABOUT MEASUREMENTS
IN SCIENTIFIC ANALYSIS
22
Id.
23
Id. at 182.
24
Id.
41
In many cases, forensic scientists rely on measurements as a basis for their opinions. In
the words of one scientist, Ascience is measurement. If I cannot make measurements, I cannot
study a problem scientifically.@25 Toxicologists measure the concentration of the toxic drug found
in human bodies to determine whether the level of toxin was high enough to have caused the
person=s death.26 Accident reconstruction experts measure the length of skidmarks27 and yaw
marks28 to determine the velocity of a vehicle. In speeding cases, prosecutors rely on radar
measurements of the velocity of the vehicle driven by the defendant.29 Drug analysts measure the
weight of seized contraband drugs to determine whether the weight was large enough to violate
criminal statutes.30 Measurements have long been critical in drunk driving prosecutions. Many
jurisdictions have statutes that erect a presumption of intoxication when the measured blood
alcohol concentration exceeds a prescribed threshold.31 Moreover, many countries and states
have gone farther and adopted per se statutes proscribing the operation of a vehicle by a person
with a specified blood alcohol concentration.32 Under these statutes, the prosecution need not
show that the operator was impaired or intoxication; proof that the driver had a certain blood
alcohol concentration such as 0.08% is sufficient. In some jurisdictions, the driver is subject to
an enhanced sentence if the blood alcohol concentration exceeds an elevated level such as
0.15%.33 In all these cases, the testimony about the forensic scientist=s measurement is usually
the most important evidence in the case.
25
William H. George, Musical Acoustics Today, New Scientist, Nov. 1962, at 257.
26
2 Paul C. Giannelli , Edward J. Imwinkelried, Andrea Roth & Jane Campbell
Moriarity, Scientific Evidence ' 20.05[d], at 351-53 (5th ed. 2012).
27
Id. at ' 27.06[a].
28
Id. at ' 27.06[b].
29
John Jendzurski & Nicholas G. Paulter, Calibration of Speed Enforcement Downthe-Road Radars, 114 Journal of Research of the National Institute of Standards and Technology
137 (2009).
30
SWGDRUG, Measurement Uncertainty for Weight Determinations in Seized
Drug Analysis (2010).
31
2 Paul C. Giannelli, Edward J. Imwinkelried, Andrea Roth & Jane Campbell
Moriarity, Scientific Evidence ' 22.01, at 473-76 (5th ed. 2012).
32
Id. at ' 22.01, at 361-62.
33
Ted Vosk, Measurement Uncertainty: Forensic Metrology and Result
Interpretation. Part Two: Legal Analysis, in Understanding DUI Scientific Evidence 255, 323
(2011 ed.).
42
In the past, forensic scientists testifying about such measurements have often presented
the court with a single point value.34 When the expert does so, there is a distinct possibility that
the trier of fact will treat the testimony as an exact value.35 In the words of Washington State
Commissioner Paul Moon, AIt has been this court=s experience since 1983 that juries it has
presided over place heavy emphasis on the numerical value of blood alcohol tests. If an expert
testifies that a particular blood alcohol content measurement is value A, without stating a
confidence interval, it is this court=s opinion that the evidence is being represented as an exact
value to the trier of fact.@36
The problem is that metrology, the science of measurement, tells that there is an
unavoidable, inherent element of uncertainty in every measurement.37 If the expert does not give
the trier of fact some sense of that uncertainty, the statement of the single point valueBa
number38-- is incomplete39 and therefore potentially misleading.40 The trier could place undue
trust41 in the testimony about the numerical value. That mistake might affect the trier=s ultimate
34
Ted Vosk, Measurement Uncertainty: Forensic Metrology and Result
Interpretation. Part One: Measurement Results and Interpretation, in Understanding DUI
Scientific Evidence 159, 220 (2011 ed.).
35
State of Washington v. Weimer, #7036A-09D (Snohomish Cty. Dist.Ct., Wash.
Mar. 23, 2010); Ted Vosk, Measurement Uncertainty: Forensic Metrology and Result
Interpretation. Part Two: Legal Analysis, in Understanding DUI Scientific Evidence 255, 311
(2011 ed.).
36
State of Washington v. Weimer, #7036A-09D (Snohomish Cty. Dist.Ct., Wash.
Mar. 23, 2010).
37
See Raghu Kacker, Klaus-Dieter Sommer & Rudiger Kessel, Evolution of
Modern Approaches to Express Uncertainty in Measurement, 44 Metrologia 513 (2007); Jesper
Kristiansen & Henning Willads Peterson, An Uncertainty Budget for the Measurement of
Ethanol in Blood by Headspace Gas Chromatography, 28 Journal of Analytical Toxicology 456
(2004).
38
Ted Vosk, Measurement Uncertainty: Forensic Metrology and Result
Interpretation. Part One: Measurement Results and Interpretation, in Understanding DUI
Scientific Evidence 159, 190, 225 (2011 ed.).
39
Id. at 191; Rod G. Gullberg, Measurement Uncertainty in Forensic Toxicology:
Its Estimation, Reporting and Interpretation, in Toxicity and Drug Testing 415, 415 (W. Acree
ed. 2012); Ted Vosk, Measurement Uncertainty: Forensic Metrology and Result Interpretation.
Part Two: Legal Analysis, in Understanding DUI Scientific Evidence 255, 230 (2011 ed.).
40
Id.; State of Washington v. Weimer, #7036A-09D (Snohomish Ct. Dist. Ct.,
Wash. Mar. 23, 2010); State of Washington v. Fausto and Ballow, #C076949 and 9Y6231062
(King Cty. Dist.Ct., Wash. Sep. 21, 2010).
41
Raghu Kacker, Klaus-Dieter Sommer & Rudiger Kessel, Evolution of Modern
43
decision,42 prompting the trier to either convict an innocent person or exonerate a guilty person.
Approaches to Express Uncertainty in Measurement, 44 Metrologia 513, 518 (2007).
42
State of Washington v. Fausto and Ballow, #C076949 and 9Y6231062 (King
Cty.Dist.Ct., Wash. Sep. 21, 2010).
44
However, metrology has not only demonstrated the inherent uncertainty of measurements.
It has also developed several methods of quantifying a measurement=s margin of error or
uncertainty.43 By using these methods, the testifying expert can put the trier in a much better
position to determine the appropriate evidentiary weight of the measurement.44 As we shall see
in Part IV, there are multiple approaches to quantifying the degree of error or uncertainty of a
measurement. The expert can draw on these approaches in order to prevent the jury from
ascribing inordinate weight to the testimony about the point value.
The purpose of this paper is to document the evolution of the law=s treatment of the
problem of uncertainty in forensic science and in particular uncertainty in measurements by
forensic scientists. Part I of this paper demonstrates that at first, many courts insisted that the
expert vouch for his or her opinion as a scientific or medical certainty. Part II points out that
later, as the courts began to realize the foolishness of such insistence, they permitted experts to
testify without opining that the point value was certainly exact. Part III shows that the evolution
of the law on this topic continues. Influenced by critiques from several prestigious scientific
organizations, a large number of courts now forbid the expert from couching his or her opinion as
a certainty. Finally, Part IV will describe an incipient judicial trend to require that in some way,
the expert quantity the degree of error or uncertainty in the expert=s measurement testimony.
This part discusses a number of recent cases as well as important developments in the field of
metrology. The paper concludes that this trend holds out the promise for more honest and open
cooperation between law and science.
A..
THE FIRST STAGE: THE ORIGINAL JUDICIAL INSISTENCE THAT
EXPERTS VOUCH FOR THEIR OPINIONS AS SCIENTIFIC OR MEDICAL
CERTAINTIES
In 1979, the International Association for Identification (I.A.I.) passed a resolution
making it professional misconduct for a a member latent print examiner to provide courtroom
testimony describing an identification as Apossible, probable, or likely@ rather than Acertain.@45 It
became a widespread practice at American trials for the expert=s proponent to invite the
43
Raghu Kacker, Klaus-Dieter Sommer & Rudiger Kessel, Evolution of Modern
Approaches to Express Uncertainty in Measurement, 44 Metrologia 513 (2007).
44
Ted Vosk, Measurement Uncertainty: Forensic Metrology and Result
Interpretation. Part One: Measurement Results and Interpretation, in Understanding DUI
Scientific Evidence 159, 218 (2011 ed.).
45
Nat=l Inst. Standards & Tech., Latent Print Examination and Human Factors:
Improving the Practice through a Systems Approach 350 (2012), citing I.A.I. Resolution VII.
Identification News 29 (Aug. 1979). See also Note, Daubert Rises: The (Re)applicability of the
Daubert Factors to the Scope of Forensics Testimony, 96 Minn.L.Rev. 1581, 1597 (2012)(AMost
forensic examiners are trained to either testify with absolute certainty or not at all. A firearmand-toolmark examiner testified that . . . >if two cartridge cases share the same magazine mark,
then one could say with one hundred percent certainty that the two cartridge cases had been
cycled through the same magazine.= [A] forensic shoe print analyst >offered a potential error rate
of zero for the method . . . .=@).
45
expert to testify that the expert=s opinion was a scientific or medical Acertainty.@ Eventually,
many courts elevated the practice to the status of a legal requirement.46 In some jurisdictions, it
was not merely that less certain testimony was legally insufficient to sustain a judgment in the
proponent=s favor; rather, if the expert refused to characterize his or her opinion in those terms,
the opinion was inadmissible as a matter of law.
That legal requirement reflected an idealized47 view of the physical world, which was
very widespread and shared by many, including judges and attorneys. According to this view,
the universe is orderly; and its phenomena are governed by invariable, determinist48 natural laws.
Those laws are not only mechanical and inexorable, but also discoverable by the scientific
method.49 A scientist could learn those natural laws by carefully applying the classic
Newtonian method of hypothesis formulation and empirical testing.
Newton had refined the scientific experimental technique in the late 17th and early 18th
centuries.50 The Age of Enlightenment began soon after.51 The supporters of the
Enlightenment were sanguine about the power of the human mind. One of the key events
triggering the Enlightenment was a translation of Newton=s Prinicipia into French with a preface
by Voltaire.52
46
State v. Holt, 246 N.E.2d 365 (Ohio 1969); Bert Black, The Supreme Court=s
View of Science: Has Daubert Exorcised the Certainty Demon?, 15 Cardozo Law Review 2129
(1994); Gregory Joseph, Less Than ACertain@ Medical Testimony, 1979 Medical Trial Technique
Quarterly 10 (1979); Gregory Joseph, Less Certain Medical Testimony, 14 Trial, Jan. 1978, at
51; Medical EvidenceBSufficiency of Expert=s Opinion, 17 Defense Law Journal 181 (1968).
47
Brief Amici Curiae of Physicians, Scientists, and Historians of Science in Support
of Petitioners at 11, Daubert v. Merrell Dow Pharmaceuticals, Inc. (U.S. No. 92-102).
48
Margaret G. Farrell, Daubert v. Merrell Dow Pharmaceuticals, Inc.: Epistemology
and Legal Process, 15 Cardozo Law Review 2183, 2194 (1994).
49
Anthony Z. Roisman, Conflict Resolution in the Courts: The Role of Science, 15
Cardozo Law Review 1945, 1950 (1994).
50
5 The Encyclopedia of Philosophy 489 (Paul Edwards ed. 1967).
51
Will Durant, The Story of Philosophy 199 (1961)
52
2 The Encyclopedia of Philosophy 519 (Paul Edwards ed. 1967).
46
AThe Age of Reason@ promoted the notion that armed with Newton=s systematic methodology,
the human intellect53 could identify and master the laws governing natural phenomena. Science
appeared to offer a Avision of total control.@54 Adherents of this view believed that proof of
scientific propositions was capable of attaining certainty and that scientific propositions could be
conclusively validated.55 In this simplistic universe, complete truth is attainable.56 In such a
universe, it is justifiable that the courts require that expert witnesses vouch for their opinions as
certainties.
B..
THE SECOND STAGE: THE COURTS= ABANDONMENT OF THE
REQUIREMENT THAT EXPERTS VOUCH FOR THEIR OPINIONS AS
CERTAINTIES
As Part I explained, the courts= original insistence that experts vouch for their opinions as
certainties reflected a particular, simplified view of the physical world. Eventually, new
scientific discoveries shattered that view.57 Those discoveries prompted the scientific
community to abandon the assumption
that the universe is entirely orderly. Quite to the contrary, they have confronted the
reality that the universe is at least partially chaotic and indeterminate. In 1927,
Heisenberg discovered Athat electrons could not be assigned a position in time and space
simultaneously, nor their future predicted by their present.@ His discovery led to the
53
Id.
54
Michael Crichton, Jurassic Park 313 (1990).
55
Brief Amici Curiae of Physicians, Scientists, and Historians of Science in Support
of Petitioners at 8, Daubert v. Merrell Dow Pharmaceuticals, Inc. (U.S. NO. 92-102).
56
Bert Black, The Supreme Court= View of Science: Has Daubert Exorcised the
Certainty Demon?, 15 Cardozo Law Review 2129, 2129-30 (1994).
57
Bert Black, A Unified Theory of Scientific Evidence, 56 Fordham L.Rev. 595,
616 (1988)(AAround the turn of the twentieth century, . . . advances in physiology and
psychology and the advent of the quantum and relativity theories in physics destroyed simple,
mechanistic certainty. Quantum theory tells us that certainty is a physical impossibility, relativity
that time is not absolute, and psychology that preconceptions color supposedly objective accounts
of the natural world@); Richard G. Halpern, Opening a New Door to Negotiation Strategy, 35
Trial, June 1999, at 22, 24 (Aphysicist Henri Poincare noticed that the orbits of certain planets did
not follow the paths predicted by the application of Newtonian physics. Poincare discovered that
for certain astronomical systems, typically those involving interactions among three or more
heavenly bodies, even the tiniest imprecisions in the measurement of a starting point would lead
to huge variations in the final resultBso huge that the measured prediction based on the principles
of Newtonian, or linear, physics would be no more accurate than a random position picked out of
a hat. He had discovered dynamical instability or chaos@).
47
formulation of the principle of uncertainty. That principle undermined the conception of
an Aorderly world in which [a] perfect understanding of cause and effect@ is possible.
[S]cience is now reconciled to the fact that some natural phenomena occur erratically.58
58
Edward J. Imwinkelried, Evidence Law Visits Jurassic Park: The Far-Reaching
Implication of the Daubert Court=s Recognition of the Uncertainty of the Scientific Enterprise, 81
Iowa Law Review 55, 60 (1995), citing Bert Black, The Supreme Court=s View of Science: Has
Daubert Exorcised the Certainty Demon?, 15 Cardozo Law Review 2129 (1994), Margaret G.
Farrell, Daubert v. Merrell Dow Pharmaceuticals, Inc.: Epistemology and Legal Process, 15
Cardozo Law Review 2183 (1994), and Brief Amici Curiae of Physicians, Scientists, and
Historians of Science in Support of Petitioners, Daubert v. Merrell Dow Pharmaceuticals, Inc.
(U.S. No. 92-102). See also Richard G. Halpern, Opening a New Door to Negotiation Strategy,
35 Trial, June 1999, at 22, 25 (AFirst stated by physicist Werner Heisenberg in 1927, the
Heisenberg uncertainty principle was . . . a major revelation . . . . The principle has three parts:
The act of observation changes any observed system irrevocably; precise measurement of any
system requires observation; because observation changes the system, accurate measurement is
impossible@).
48
Science not only reassessed its macrocosm view of the orderliness of the universe; it also
rethought the microcosm of the validation of individual scientific hypotheses. In some instances,
experts rely on deductive reasoning. If one assigns certain definitions to Atwo@ and Afour,@ one
can reason with inexorable deductive logic that two plus two equals four. However, in
investigational science exploring phenomena in the physical world, experts typically resort to
another mode of analysis, that is, inductive reasoning. Having observed specific instances of a
phenomenon, the scientist formulates a general hypothesis about the phenomenon and then
engages in controlled laboratory experimentation or systematic field observation to falsify or
validate the hypothesis.59 If the outcomes of numerous empirical tests of the hypothesis all
confirm the hypothesis, we can have increasing confidence in the validity of the hypothesis.60
However, we cannot regard as the hypothesis as Adefinitively confirmed because it is always
possible that an empirical test will some day demonstrate the theory to be incorrect.@61 Another
empirical test is always conceivable; and so long as that is true, a theoretical possibility of
falsification or disproof remains.62 In short, A[n]o amount of testing can establish that a
scientific theory is >true= in every conceivable circumstance.@63 When an expert relies inductive
reasoning to investigate the truth of an hypothesis, at most the hypothesis can be accepted
contingently64 or provisionally.65 Given the imperfect,66 incomplete67 state of our knowledge, the
59
Bert Black, Francisco J. Ayala & Carol Saffran-Brinks, Science and the Law in
the Wake of Daubert: A New Search for Scientific Knowledge, 72 Texas Law Review 715, 755
(1994).
60
Brief of the American Medical Association, American Medical
Association/Specialty Society Medical Liability Project as Amici Curiae in Support of
Respondent at 11, Daubert v. Merrell Dow Pharmaceuticals, Inc. (U.S. No. 92-102).
61
Id.
62
Bert Black, Francisco J. Ayala & Carol Saffran-Brinks, Science and the Law in
the Wake of Daubert: A New Search for Scientific Knowledge, 72 Texas Law Review 715, 764
(1994).
63
Clifton T. Hutchinson & Danny S. Ashby, Daubert v. Merrell Dow
Pharmaceuticals, Inc.: Redefining the Bases for Admissibility of Expert Scientific Testimony, 15
Cardozo Law Review 1875, 1885 (1994).
64
Bert Black, Francisco J. Ayala & Carol Saffran-Brinks, Science and the Law in
the Wake of Daubert: A New Search for Scientific Knowledge, 72 Texas Law Review 715,
Bruce S. Koukoutchos, Solomon Meets Galileo (And Isn=t Quite Sure What To Do With Him),
15 Cardozo Law Review 2237, 2253 (1994).
65
Brief Amici Curiae of Nicolas Bloembergen et al., at 9 n. 8, Daubert v. Merrell
Dow Pharmaceuticals, Inc. (U.S. No 92-102).
66
Brief of the Carnegie Commission on Science, Technology, and Government as
Amicus Curiae in Supprt of Neither Party at 22, Daubert v. Merrell Dow Phamaceuticals, Inc.
49
investigation can never validate the hypothesis as a certainty.68 As of July 2010, the
International Association for Identification changed its policy and expressly allowed members to
testify to qualified, uncertain opinions when such opinions are statistically defensible.69
(U.S. No. 92-102).
67
Bruce S. Koukoutchos, Solomon Meets Galileo (And Isn=t Quite Sure What To
Do With Him), 15 Cardozo Law Review 2237, 2253 (1994).
68
Vern R. Walker, The Siren Songs of Science: Toward a Taxonomy of Scientific
Uncertainty for Decisionmakers, 23 Connecticut Law Review 567 (1991).
69
Nat=l Inst. Standards & Tech., Latent Print Examination and Human Factors:
Improving the Practice through a Systems Approach (73 (2010), citing I.A.I. Resolution 2010-18,
dated July 16, 2010.
50
The changed views of the scientific community eventually had an impact on the legal
community. The courts gradually began relaxing the requirement that the expert affirm that he or
she could testify to their opinion as a scientific or medical certainty. In some jurisdictions, the
initial relaxation was selective. For example, although the court might insist that the expert
vouch for opinions about diagnosis70 or causation71 as a certainty, the courts accepted
prognosis72 opinions lacking that degree of certitude. Prognoses are futuristic predictions, and in
these jurisdictions it struck the courts as unrealistic to demand the same definiteness as in
opinions about past causation or present diagnosis.73 Other courts generally abandoned the
requirement.74 These courts reason that the indefiniteness of an opinion couched as a possibility
or probability reduces the weight of the testimony75 and renders the opinion more vulnerable to
discretionary exclusion,76 but they no longer enforce a rigid requirement automatically excluding
any opinion that is not phrased as a certainty.
70
Gregory Joseph, Less Than Certain Medical Testimony, 14 Trial, Jan. 1978, at 51-
71
Id. at 51.
72
Id. at 51-52.
73
Id. at 52.
52.
74
United States v. Oaxaca, 569 F.2d 518, 526 (9th Cir.), cert.denied, 439 U.S. 926
(1978); United States v. Spencer, 439 F.2d 1047, 1049 (2d Cir. 1971); Burke v. Town of
Walpole, 405 F.2d 66 (1st Cir. 2005); United States v. Glynn, 578 F.Supp.2d 567 (S.D.N.Y.
2008), cert.denied, 131 S.Ct. 2960, 180 L.Ed.2d 250 (2011); Huck v. State, 881 So.2d 1137,
1150 (Fla.Dist.Ct.App. 2004); State v. Boyer, 406 So.2d 143, 148 (La. 1981). .
75
In re Swine Flu Immunization Prods. Liability Litrigation, 533 F.Supp. 567, 578
(D.Colo. 1980).
76
Fed.R.Evid. 403, 28 U.S.C.A..
51
The Supreme Court itself recognized this insight. In Daubert, a large number of
scientists and scientific organizations submitted amicus curiae briefs.77 Many of those briefs
attempted to educate the members of the Court about the modern understanding of the limitations
of the scientific method. Drawing on several of those briefs,78 Justice Blackmun wrote: A[I]t
would be unreasonable to conclude that the subject of scientific testimony must be >known= to a
certainty; arguably there are no certainties in science.@79 That frank recognition by the Court has
prompted other courts to be more receptive to uncertain testimony such as statistical evidence. In
the final analysis, statistical testimony is overtly uncertain expert testimony.80 By way of
example, the expert concedes that she does not know the true value of a certain parameter of the
universe, but she has sampled the universe and computed a statistic that is an estimator of the
parameter. Several of the amicus briefs filed in the Daubert predicted that if the Court embraced
a more realistic view of the limits of the scientific method, the Court=s adoption of that view
would pave the way for the more liberal admissibility of a particular type of statistical testimony,
that is, testimony about the confidence intervals for point estimates.81 In the past, the courts
were often squeamish about explicitly acknowledging the uncertainty of testimony such as
scientific evidence.82 Some feared that doing so would undermine the public=s faith in the legal
system and their acceptance of the system=s outcomes as legitimate.83 However, once the
Supreme Court itself forthrightly conceded the inevitable uncertainty in scientific testimony, the
77
Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 581-82 *
(1993)(listing the amicus briefs).
78
Id. at 590.
79
Id.
80
Edward J. Imwinkelried, Evidence Law Visits Jurassic Park: The Far-Reaching
Implication of the Daubert Court=s Recognition of the Uncertainty of the Scientific Enterprise, 81
Iowa Law Review 55, 65 (1995).
81
Id. at 67 n. 131, citing Brief Amicus Curiae of Professor Alan R. Feinstein in
Support of Respondent at 3, 16, Daubert v. Merrell Dow Pharmaceuticals, Inc. (U.S. No. 92102), Brief Amici Curiae of Professors Kenneth Rothman et al. in Support of Petitioners at 7,
Daubert v. Merrell Dow Pharmaceuticals, Inc. (U.S. No. 92-102), and Brief of the United States
as Amicus Curiae Supporting Respondent at 6, Daubert v. Merrell Dow Pharmaceuticals, Inc.
(U.S. 92-102).
82
Daniel Shaviro, Statistical-Probability and the Appearance of Justice, 103 Harvard
Law Review 530, 534, 538, 547 (1989).
83
See generally Laurence H. Tribe, Trial by Mathematics: Precision and Ritual in
the Legal Process, 84 Harvard Law Review 1329 (1971); Charles Nesson, The Evidence or the
Event? On Judicial Proof and the Acceptability of Verdicts, 98 Harvard Law Review 1357
(1985).
52
lower courts, both federal and state, were less reluctant to make the same concession. Once they
had done so, it became clear that it was wrong-minded for the courts to continue to demand that
experts always vouch for their opinions as certainties. It was generally 84 permissible for the
expert to testify that the opinion he or she was asserting amounted to only a probability or
possibility.85
C.
THE THIRD STAGE: THE COURTS= PROHIBITION OF OPINIONS STATED
AS CERTAINTIES
As Part II explained, the concession of the inevitable uncertainty of investigational
science led to the courts= abandonment of the traditional rule that experts had to vouch for their
opinions as scientific or medical certainties. However, the logic of the concession leads beyond
the rejection of the traditional rule. If investigational science relying on inductive reasoning
cannot attain certainty, the courts should not only permit expressly uncertain expert testimony;
they also ought to forbid experts from testifying to purportedly certain opinionsBsuch opinions
are unwarranted as a matter of logic.
84
There are two caveats. The limited nature of the opinion might make the opinion
vulnerable to discretion exclusion under a statute such as Federal Rule of Evidence 403, and
without more an opinion worded as a mere possibility is legally insufficient to sustain the
proponent=s initial burden of production. Fed.R.Civ.P. 56, 28 U.S.C.A.. See Daubert v. Merrell
Dow Pharmaceuticals, Inc., 509 U.S. 579, 595-96 (1993).
85
Nat=l Inst. Standards & Tech., Latent Print Examination and Human Factors:
Improving the Practice through a Systems Approach 118 (2012)(Acourts do not normally demand
absolute certainty from scientists (or any other experts) . . . .@).
53
Within the past five years, two reports on forensic science, both coauthored by scientific
and legal authorities, drew this conclusion. Both reports attacked overstated expert opinions. In
2009, the National Research Council of the National Academies released Strengthening Forensic
Science in the United States: A Path Forward.86 The drafting committee included such members
as Judge Harry Edwards of the United States Court of Appeals for the District of Columbia
Circuit and Professor Constantine Gatsonsis, the Director of the Center for Statistical Sciences at
Brown University as well as two medical examiners and professors of biology, chemistry,
engineering, law, and physics. The report was highly critical of claims that Athe error rate for
fingerprint comparison [is] >essentially zero.=@87 The report stated flatly that the claim is Anot
scientifically plausible.@88 The report not only opposed the admission of expert opinions
couched as certainties; the report also went to the length of urging that A[a]ll results for every
forensic science method . . . indicate the uncertainty in the measurements that are made . . . .@89
The report formally recommended A[t]he development of quantifiable measures of uncertainty in
the conclusions of forensic analyses.@90 The report stated that A[a]s a general matter, laboratory
reports . . . should identify . . . the sources of uncertainty in the . . . conclusions along with
estimates of their scale . . . .@91 In particular, the report took the position that any measurement of
a subject=s blood alcohol concentration Aneed[s] to be reported, along with a confidence interval@
indicating the uncertainty of the measurement.92
86
Nat=l Research Council, Strengthening Forensic Science in the United States: A
Path Forward (2009).
87
Id. at 103. See Sandy Zabel, Fingerprint Evidence, 13 Journal of Law and Policy
143, 177 (2005)(Aexaminers have made assertions that fingerprints are >infallible,= that an
identification is >100 percent positive,= and that the >methodological error rate= for parts of the
process is zero@).
88
National Research Council, supra note 63, at 142. See also Sandy Zabel,
Fingerprint Evidence, 12 Journal of Law and Policy 177 (2005)(Ano scientific basis@).
89
National Research Council, supra note 63, at 184.
90
Id. at 190.
91
Id. at 186.
92
Id. at 117. See also David H. Kaye, David E. Bernstein & Jennifer L. Mnookin,
The New Wigmore: Expert Evidence 101 (2014 Cum.Supp.)(the English Law Commission
made a similar recommendation in a bill submitted to Parliament; the bill states that in deciding
the admissibility of expert testimony, the judge should consider “if the expert’s opinion relies on
the results of . . a . . . measurement . . . , whether the opinion takes proper account of matters,
such as the degree of precision or margin of uncertain, affecting the accuracy or reliability of
those results”).
54
The second report was published in 2012. In that year, the National Institute of Standards
and Technology=s Expert Working Group on Human Factors in Latent Print Analysis published
Latent Print Examination and Human Factors: Improving the Practice through a Systems
Approach.93 Like the National Research Council committee, the working group had a diverse
membership, including fingerprint examiners, computer scientists, cognitive researchers, law
professors, and statisticians. Like the National Research Council report, the N.I.S.T. report faults
experts= overstated claims that they could certainly attribute an impression to a single person Ato
the exclusion of all others in the world.@94 The report urged research into the development of a
probabilistic model for fingerprint testimony, similar to the model used in DNA cases.95 The
report observed that a Anumber of courts have curtailed the certainty with which a judgment of
individualization of toolmarks may be expressed.@96
93
Nat=l Inst. Standards & Tech., Latent Print Examination and Human Factors:
Improving the Practice through a Systems Approach (2012).
94
Id. at 72.
95
Id. at 85-86.
96
Id. at 120 n. 382, citing United States v. Willock, 696 F.Supp.2d 536 (D.Md.
2010), cert.denied sub nom. Smith v. United States, 132 S.Ct. 430, 181 L.Ed.2d 279 (2011);
United States v. Taylor, 663 F.Supp.2d 1170, 1180 (D.N.M. 2009), and Commonwealth v. Pytou
Heang, 458 Mass. 827, 942 N.E.2d 927 (2011).
55
In part inspired by critiques such as these two American reports,97 there is now a definite
judicial trend to forbid experts from testifying to purportedly certain opinions. If, as Justice
Blackmun declared in Daubert,98 it is impossible to attain certainty in investigational science, an
opinion couched as a certainty cannot qualify as reliable Ascientific . . . knowledge@ under Federal
Rule of Evidence 702; the available empirical data cannot support that knowledge claim. That
message has not been lost on the lower courts. As the N.I.S.T. report pointed out, several courts
have already forbidden purportedly certain opinions by toolmark experts.99 Similarly, in a 2010
opinion,100 a federal court prohibited a fingerprint examiner from testifying to such an opinion.
Furthermore, as we shall see in Part IV, in the same year a number of state courts have ruled that
a toxicologist may not describe a single point value for a subject=s blood alcohol concentration as
a certainty.101
D.
THE FOURTH STAGE: THE INCIPIENT JUDICIAL TREND TO INSIST
THAT THE EXPERT ACKNOWLEDGE AND PROVIDE A MEANINGFUL
QUANTITATIVE MEASURE OF THE UNCERTAINTY OF THE
MEASUREMENT
97
There is also a 2011 report from the English Law Revision Commission to the
same effect. David H. Kaye, David E. Bernstein & Jennifer L. Mnookin, The New Wigmore:
Expert Evidence ' 7.6.5, at 102-03 (2014 Cum.Supp.), citing Expert Evidence in Criminal
Proceedings in England and Wales para. 5.35(1) (2011). The commission submitted the report to
Parliament. The report urges Parliament to adopt legislation requiring the proponent of
testimony about a measurement to account for the Amargin of uncertainty.@
98
Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 590 (1993).
99
Note 70, supra.
100
United States v. Zajac, 748 F.Supp.2d 1327 (D. Utah 2010). See Paul Giannelli,
Fingerprints: Misidentifications, 20 Criminal Justice, Spr. 2005, at 50 (it is Aastounding@ that
experts still sometimes attempt to testify Athat there is a >zero error= rate in fingerprint
examination@); Tamara F. Lawson, Can Fingerprints Lie?: Re-weighing Fingerprint Evidence in
Criminal Jury Trials, 31 American Journal of Criminal Law 1, 7 (2003)(it is A[in]appropriate to
allow latent fingerprint expert witnesses to testify regarding their findings which are purported to
conclusively link an accused to a crime@); Note, Recognizing and Responding to a Problem with
the Admissibility of Fingerprint Evidence Under Daubert, 45 Jurimetrics Journal 41, 57
(2004)(the expert should not Atestify that a match conclusively ties a defendant to a print found at
the crime scene@; the state of the empirical research does not support Aan absolutist claim@).
101
State of Washington v. Fausto and Ballow, Case No. C076949 and 9Y6231062
(King Cty. Dist.Ct., Wash. Sep. 21, 2010); State of Washington v. Weimer, #7036A-09D
(Snohomish Cty. Dist.Ct., Wash. Mar. 23, 2010).
56
Measures of Error or Uncertainty That the Courts Have Already Approved
This paper began by reviewing the early judicial view that the expert must vouch for his
or her opinion as a scientific or medical certainty. Part III pointed out that today many courts not
only do not mandate such opinions; quite to the contrary, they forbid them. These courts
understandably preclude the expert from expressly misleading the trier of fact by claiming that
the expert=s conclusion is certainly correct. However, other courts have gone farther. Knowing
that a single measurement is inherently uncertain and that metrology recognizes several
quantitative measures of the margin of error or uncertainty, these courts demand102 that the expert
clarify the probative value of the measurement by proffering such a quantitative measure to
accompanying the testimony about the single point value. To date, these judicial demands have
taken two forms.
102
Rod G. Gullberg, Measurement Uncertainty in Forensic Toxicology: Its
Estimation, Reporting and Interpretation, in Toxicity and Drug Testing 415, 416 (W. Acree ed.
2012).
57
The Safety Margin or Band Gap Approach. The early cases opted for the safety margin
or band gap approach.103 In the 1940s the Swedish Supreme Court declared that in all drunk
driving prosecutions, the error or uncertainty of the blood alcohol concentration measurement
must be specified.104 Several American jurisdictions have followed suit. In evaluating the legal
sufficiency of the government=s evidence to sustain a conviction,105 courts in these states demand
that the inherent margin of error be deducted from the single point value. Suppose, for example,
that in a given jurisdiction, the law criminalizes operating a motor vehicle when the driver has a
blood alcohol concentration of 0.08% or more. Assume that in the instant case, the measurement
testimony is that the intoxilyzer reading indicated that the driver had a concentration of 0.081%.
However, suppose further that based on expert testimony, the court concludes that inherent
margin of error of the intoxilyzer is plus or minus 0.002%. On that set of assumptions, the
driver=s true concentration could be as low as 0.079%Bfalling below the 0.08% standard. After
considering all the evidence, the court would find the accused not guilty; the government=s
testimony is legally insufficient to support a finding that the accused=s blood alcohol
concentration was 0.08% or higher.
For its part, the intermediate appellate court in Hawaii has found that the inherent margin
of error for such blood alcohol readings is 0.0165%.106 The court stressed that as a matter of
interpretation, the trigger for the criminal prohibition is an actual blood alcohol concentration,
not a mere instrumental reading. The court reasoned that in order to present a legally sufficient
case, the government had to account for the margin of error in measurement.107 The court
elaborated:
In both of the cases at bar, the State has failed to establish a critical fact. The State
merely demonstrated that the reading of the breathalyzer machine was 0.10% for
Defendant Boehmer and 0.11% for Defendant Gogos. The inherent margin of error could
103
Ted Vosk, Measurement Uncertainty: Forensic Metrology and Result
Interpretation. Part One: Measurement Results and Interpretation, in Understanding DUI
Scientific Evidence 159, 206-07 (2011 ed.). See also State v. Finch, 291 Kan. 665, 244 P.3d
673 (2011)(despite the margin of error, the trial judge should not have granted the accused=s
motion for acquittal; however, the testimony about the error margin was a permissible factor for
the jury to consider during deliberation); State v. McGinley, 229 N.J.Super. 191, 550 A.2d 1305
(1988)(since the breathalyzer has a substantial error margin, the blood alcohol measurement must
be adjusted), overruled on other grounds, State v. Downie, 117 N.J. 450, 569 A.2d 242 (1990).
104
Ted Vosk, Measurement Uncertainty: Forensic Metrology and Result
Interpretation. Part One: Measurement Results and Interpretation, in Understanding DUI
Scientific Evidence 159, 214 (2011 ed.).
105
Ted Vosk, Measurement Uncertainty: Forensic Metrology and Result
Interpretation. Part Two: Legal Analysis, in id. at 255, 266.
106
State v. Boehmer, 1 Haw.App. 44, 613 P.2d 916 (1980).
107
Id. at 918-19.
58
put both defendants= actual blood alcohol level below the level necessary for the
presumption to arise. The failure of the prosecution to establish beyond a reasonable
doubt that the actual weight of alcohol in defendants= blood was at least 0.10% required
the trial judge to ignore [any presumption based on the test result].108
108
Id. at 918.
59
The Nebraska Supreme Court has reached a similar result.109 The court has stopped short
of announcing that the blood alcohol concentration measurement must always be adjusted for a
fixed margin of error.110 The need for an adjustment Ais dependent upon the credible evidence in
each case.@111 However, the court has reiterated that when the credible evidence establishes Athe
margin of error,@ that figure must be Adeduct[ed@ from the reading to Areduce[] the test result . . .
.@112
Unlike the Nebraska and Hawaii courts, the Iowa Supreme Court refused to require that
the trial judge deduct the figure representing the inherent margin of error from the single point
value for the blood alcohol concentration in determining whether the government has met its
burden of going forward.113 While the Iowa court rejected the argument that the margin of error
should be subtracted, the Iowa legislature found the argument persuasive. That legislature then
amended its statute to read:
The results of a chemical test may not be used as the basis for a revocation of a person=s
driver license or nonresident operating privilege if the alcohol or drug concentration
indicated by the chemical test minus the established margin of error inherent in the device
or method used to conduct the chemical test is not equal to or in excess of the level
prohibited . . . .114
In reporting its results, the Virginia Department of Forensic Science has adopted the same
practice.115
109
State v. Bjornsen, 201 Neb. 709, 271 N.W.2d 839 (1978).
110
State v. Babcock, 227 Neb. 649, 419 N.W.2d 527, 530 (1988).
111
Id. at 530.
112
Id.
113
Nugent v. Dept. of Transportation, 390 N.W.2d 125 (Iowa 1986).
114
Iowa C.A. ' 321J.12(6). See also State v. Schuck, 22 Ohio St.3d 296, 490 N.E.2d
596, 598 (1986); State v. Prestier, 7 Ohio Misc.2d 36, 455 N.E.2d 24, 38-39 (Mun.Ct. 1982).
115
Ted Vosk, Measurement Uncertainty: Forensic Metrology and Result
Interpretation. Part One: Measurement Results and Interpretation, in Understanding DUI
Scientific Evidence 159, 209 (2011 ed.).
60
The most recent decision in this vein is People v. Jabrocki,116 rendered in 2011 by a
Michigan trial court. In that case, District Judge Peter Wadel ruled that Ablood test results are not
reliable until the state police crime lab[oratory] calculates an uncertainty budget or error rate and
reports that calculation along with the blood test results.@ The court pointed out that the
standards of the American Society of Crime Laboratory Directors= Accreditation Board Arequires
the labs to develop an uncertainty budget for blood alcohol analysis.117 In addition, the court
quotes the 2009 National Academy of Sciences report on forensic science to the effect that A[a]ll
results of every forensic science method should indicate the uncertainty of the measurements that
are made . . . .@118 (In 2011, the English Law Commission made a similar recommendation.119)
Finally, the court cites the two Washington state decisions discussed in the ensuing paragraphs.
116
People of the State of Michigan v. Jeffrey James Jabrocki, 08-5461-FD (Dist.Ct.,
Mason Cty., May 6, 2011).
117
The court cites Jesper Kristiansen & Henning Willads Petersen, An Uncertainty
Budget for the Measurement of Ethanol in Blood by Headspace Gas Chromatography, 28 Journal
of Analytical Toxicology 456 (Sep. 2004). That article states that the uncertainty budget should
include:
----analytical standard uncertainty;
----biological variation (variation in both the density of blood and the matrix difference between
aqueous calibration samples and blood); and
-----traceability uncertainty related to the calibration standards.
Like the courts in some jurisdictions, the authors use the expression, a Asafety margin.@ Id. at
456, 462. Rod G. Gullberg, Measurement Uncertainty in Forensic Toxicology: Its Estimation,
Reporting and Interpretation, in Toxicity and Drug Testing 415, 428 (W. Acree ed.
2012)(discussing the concept of an uncertainty budget).
118
But see Christopher Boscia, Strengthening Forensic Alcohol Analysis in
California DUI Cases: A Prosecutor=s Perspective, 53 Santa Clara L.Rev. 733, 758-59
(2013)(Awhile little reported, NAS qualified its declaration about report requirements when it
said the reports >should describe, at a minimum, methods and materials, procedures, results, and
conclusions, and they should identify, as appropriate, the sources of uncertainty in the procedures
and conclusions along with estimates of their scale (to indicate the level of confidence in the
results). The NAS report did not list examples of when it would not be appropriate to report
sources of uncertainty in forensic science@).
119
David H. Kaye, David E. Bernstein & Jennifer L. Mnookin, The New Wigmore:
Expert Evidence ' 7.6.5, at 102, 107 (2014 Cum.Supp.)(the commission submitted a bill to
Parliament to require English courts to conduct a Daubert-style inquiry; one of the provisions of
the proposed bill reads that Aif the expert=s opinion relies on the results of the use of any method
(for instance, a test, measurement or survey),@ the judge should consider Awhether the opinion
takes proper account of matters, such as the degree of precision or margin of uncertainty,
affecting the accuracy or reliability of those results@; the report is Law Commission, Expert
Evidence in Criminal Proceedings in England and Wales para. 5.35(1)(2011), available at
http://www.justice.gov.uk/lawcommission/docs/lc325_Expert_Evidence_Report.pdf),
61
The confidence interval approach. While Hawaii, Iowa, and Nebraska have used the
safety margin approach to compensate for the uncertainty of measurements, Washington has
taken another approach.120 In Washington, the authorities administer two tests to each drunk
driving suspect and present testimony about the arithmetic mean or average of the two
measurements at trial.121 Two state courts have ruled that in order to convey a sense of the
uncertainty of the measurements to the trier, the prosecution expert must accompany that
testimony with an explanation of the confidence interval for the mean.
In March 2010 in State v. Weimer,122 Commissioner Paul Moon stated that testimony
about a single point value is misleadingly complete unless it is accompanied by evidence of the
confidence interval for the value. The commissioner cited the state version of Federal Rule of
Evidence 403 which empowers a trial judge to exclude an item of evidence if, in his or her
discretion, the judge believes that the probative dangers attending the evidence substantially
outweigh its probative value. As the commissioner explained, one of the recognized dangers
under Rule 403 is that the trier of fact will overvalue the item. Drawing on his previous
experience presiding over drunk driving trials, the commissioner concluded that if the trier does
not have the benefit of a quantified measure of the measurement=s uncertainty such as a
confidence interval, there is an intolerable risk that the jury will treat the point value as Aexact@
and attach undue weight to the value. Near the end of his opinion, Commissioner Moon tersely
asserted that A[t]o allow the test value into evidence without stating a confidence level violates@
Rules 403.
120
Rod G. Gullberg, Measurement Uncertain in Forensic Toxicology: Its Estimation,
Reporting and Interpretation, in Toxicity and Drug Testing 415, 417 (W. Acree ed. 2012)(there
are Amultiple ways@ to approach the problem of estimating uncertainty). To date, no firm
consensus has emerged. Id. at 451.
121
State of Washington v. Fausto and Ballow, Case No. C076949 and 9Y6231062
(King Cty.Dist.Ct. Sep. 21, 2010); Ted Vosk, Measurement Uncertainty: Forensic Metrology
and Result Interpretation. Part Two: Legal Analysis, in Understanding DUI Scientific Evidence
255, 312 (2011 ed.). Taking two measurements enables the analyst to determine whether the
subject was in the absorption or elimination phase at the time of the initial test. 2 Paul C.
Giannelli, Edward J. Imwinkelried, Andrea Roth & Jane Campbell Moriarity, Scientific
Evidence ' 22.04, at 532-33 (5th ed. 2012).
122
State of Washington v. Weimer, #7036A-09D (Snohomish Cty. Dist.Ct., Wash.
Mar. 23, 2010).
62
In September of the same year, a three-judge panel handed down its decision in State v.
Fausto and Ballow.123 The panel asserted that standing alone, the testimony about the mean
measurement paints a Afalse picture.@ In the panel=s view, without an indication of the
uncertainty of the mean, the jury is likely to misinterpret the single point value. The panel
bluntly stated that A[g]iven the inherent variability of measurement, a statement of a measurement
result is incomplete (perhaps even meaningless) without an accompanying statement of the
estimated uncertainty of measurement.@ The panel acknowledged that there are numerous ways
of calculating uncertainty. Indeed, the court expressly cited the Hawaii and Nebraska decisions
adopting the safety net approach requiring testimony about the inherent margin of error.
However, in the instant case the panel decided to require the presentation of testimony about a
bias-corrected mean124 and a confidence interval for the mean. On the one hand, the panel did
not make a sweeping ruling that the prosecution must always present testimony about the
confidence interval. On the other hand, the panel mandated that the government laboratory
provide the defense with the confidence interval to enable the defense to present that testimony to
the jury:
[A]bsent a confidence interval, a Afinal@ breath-alcohol measurement is only a Abest
estimate@ of a person=s breath-alcohol level. Given . . . the discovery rules and E[vidence]
R[ule] 702, the State must provide Defendants with a confidence interval for each
Defendant=s breath-alcohol measurement. Absent this information, a defendant=s breathalcohol measurement will be suppressed.
In all the Washington decisions, the court selected a confidence interval as the most
appropriate measure of the point value=s uncertainty. Before turning to the future of forensic
metrology in the next section, it is important to clarify the notion of a confidence interval. A
confidence interval gives the statistician a sense of the dispersion of the values.125
$ The initial step in computing a confidence interval is identifying the sample statistic to
construct the interval around.126
123
Case No. C076949 and 9Y6231062 (King Cty. Dist.Ct. Sep. 21, 2010).
124
In this context, a bias-corrected mean is the mean adjusted for systematic error.
Systematic errors are those that consistently result in the under or overestimation of the true
value. Ted Vosk, Measurement Uncertainty: Forensic Metrology and Result Interpretation, in
Understanding DUI Scientific Evidence 159, 168 (2011). Such errors remain constant or vary in
a predictable manner. Id. They can be caused by such factors as a calibration mistake or
environmental conditions. Id. A researcher can determine the bias by comparing the
measurements against a reference standard. Id. at 168-69. Having determined the bias, the
researchers corrects the point value to that extent. Id. at 169-70.
125
1 Paul C. Giannelli, Edward J. Imwinkelried, Andrea Roth & Jane Campbell
Moriarity, Scientific Evidence ' 15.02 (5th ed. 2012); Rod G. Gullberg, Measurement
Uncertainty in Forensic Toxicology: Its Estimation, Reporting and Interpretation, in Toxicity and
Drug Testing 415, 415 (W. Acree ed. 2012).
126
See generally Douglas A. Skoog, Donald M. West & F. James Holler,
63
$ The next step is determining the standard error or deviation for the statistic. In effect, the
standard deviation measures the average deviation from the average127 or mean:
Fundamentals of Analytical Chemistry 26-30 (5th ed. 1988); Donald P. Land & Edward J.
Imwinkelried, Confidence Intervals: How Much Confidence Should the Courts Have in
Testimony About a Sample Statistic?, 44 Criminal Law Bulletin 257 (Mar.-Apr. 2008); Edward
J. Imwinkelried, Expert Witness: A Look at Intervals, National Law Journal, May 19, 2008, at
13.
127
David H. Kaye & David A. Friedman, Reference Guide on Statistics, in Reference
Manual on Scientific Evidence 211, 298 (3d ed. 2011); Douglas A. Skoog, Donald M. West & F.
James Holler, Fundamentals of Analytical Chemistry 26-30 (5th ed. 1988).
64
The statistician takes the difference between each value and the mean (the average) and
then squares the differences, adds these squares, and divides by N (or N-1). Squaring the
value is critical. If the statistician simply took the differences and added them, the sum
would be zero; there would be some values higher than the mean and others lower. The
zero would give no insight into the dispersion, squaring the difference yields a positive
number. The standard deviation incorporates all the elements of the set in its
computation, and its magnitude is directly related to the amount of variation in the
data.128
$
The third step is the choice of a confidence coefficient:
The coefficient equals one minus the significance level for the test. Thus, suppose the
expert wanted a significance level of .05, at which an outcome has a probability of
occurring by chance 5% of the time or less. He or she would choose a coefficient of .95.
Researchers commonly use one of three coefficientsB.68, .95, or .99. [W]hen the values
are normally distributed, 68.26% should fall within one standard error of the sample
statistic, 95.44% within two standard errors, and 99.73% within three standard errors.129
$
At this point, the statistician is ready to compute the confidence interval. There are
numerous formulae for computing the interval. As a generalization, though, the formula
incorporates the sample statistic, the sample size, the standard error, and the coefficient.130 There
are various ways of expressing the interval. For instance, the expert might note:
95% CONF (67; 62, 72).
In this notation, 67 is the sample statistic, perhaps a mean. The remaining entries indicate that
with 95% confidence, the interval runs from a lower boundary of 62 to an upper boundary of 72.
It is important to appreciate both what a confidence interval means and what it does not mean.
By way of example, it does not mean that the interval certainly includes the true value of the
population parameter or even that there is a certain probability that the interval includes the true
value.131 What then does the interval denote? The interval represents the proportion of similarly
constructed intervals that should capture the true value:
128
1 Paul C. Giannelli, Edward J. Imwinkelried, Andrea Roth & Jane Campbell
Moriarity, Scientific Evidence ' 15.02[b], at 831-33 (5th ed. 2012).
129
Donald P. Land & Edward J. Imwinkelried, Confidence Intervals: How Much
Confidence Should the Courts Have in Testimony About a Sample Statistic?, 44 Criminal Law
Bulletin 257, 263 (Mar.-Apr. 2008).
130
Edward J. Imwinkelried, Expert Witness: A Look at Intervals, National Law
Journal, May 29, 2008, at 13.
131
Donald P. Land & Edward J. Imwinkelried, Confidence Intervals: How Much
Confidence Should the Courts Have in Testimony About a Sample Statistic?, 44 Criminal Law
Bulletin 257, 270-71 (Mar.-Apr. 2008).
65
It is not so much that we have confidence in the particular interval computed from the
initial sample. Rather, we can have a degree of confidence in the formula, method, or
process. Suppose, for instance, that the expert employed a .95 confidence coefficient. In
the long run on average, 95% of the confidence intervals based on similarly sized,
similarly drawn samples from the same population will capture or cover the true value of
the population parameter.132
If the interval is narrow with a high confidence coefficient, it is justifiable to attach great weight
to the interval.133
In mid-2013, the Washington Court of Appeals handed down its decision in Fausto and
Ballow.134 In the appellate court=s words, the district court judge had ruled that Abreath tests are
categorically inadmissible unless the State introduces a corresponding confidence interval.@135
However, the court affirmed the King County Superior Court=s reversal of that ruling.
Significantly, the court pointed out that on appeal, the prosecution did not Achallenge[]@ the trial
judge=s finding that Aevery measurement is >uncertain= . . . . Even the best instruments yield only
an estimate of the true value. Uncertainty indicates a range in which the true value of a
measurement is likely to occur.@136 For that matter, the prosecution did not Adispute that
measurement uncertainty is recognized in all sciences and uncertainty measurements may be
helpful to the trier of fact in some circumstances.@137 However, the court concluded that the
uncertainty of the BrAC test result went to its weight rather than its admissibility.138 The court
emphasized that the defense has the opportunity to Apresent uncertainty evidence challenging
BrAC test results.@139 Yet, in closing, the court made it clear that it was rejecting only a Ablanket@
rule excluding test results unaccompanied by a confidence interval.140 In dictum at the end of its
132
Id. at 267-68.
133
Id. at 270. See ATA Airlines, Inc. v. Federal Exp. Corp., 665 F.3d 882, 889-96
97th Cir. 2011)(a wide confidence interval).
134
State of Washington v. King County District Court West Division; King County
District Court, East Division; King County District Court, South Division; Ballow; and Fausto,
17 Wm.App. 630, 307 P.3d 765 (2013). See also Christopher Boscia, Strengthening Forensic
Alcohol Analysis in California DUI Cases: A Prosecutor=s Perspective, 53 Santa Cl.L.Rev. 733
(2013).
135
307 P.3d at 766.
136
Id. at 769.
137
Id.
138
Id. at 770.
139
Id.
66
opinion, the court added that in a given caseBperhaps one with a test result close to the legal limit
and a wide confidence intervalBAthe trial court@ could Aexercis[e] its discretion . . . to exclude a[]
. . . BrAC test result on a case-by-case basis.@ The intermediate appellate court=s opinion may
not be the last word on the issue. The individual defendants are now petitioning the Washington
Supreme Court to review the issue.
The Future
140
Id.
67
As we have seen, several American jurisdictions have already mandated the use of
quantitative measures of error or uncertainty by forensic experts to ensure that testimony about a
point value does not mislead the trier of fact. To date, the courts have approved the use of
margins of error and confidence intervals. However, those figures do not exhaust the
possibilities. As the Washington court noted in Fausto,141 there are several quantitative
measures of error or uncertainty.
The safety margin and confidence interval approaches illustrate the classical approach.142
In evaluating the point value, the challenge is identifying and accounting for all the sources of
error or uncertainty. There are two types of problems: systematic errors (bias) and random
errors.143 Systematic errors tend to be unidirectional and consistently cause the point value to
under or overestimate the true value.144 If an instrument is improperly calibrated, it might
underestimate each measurement; or the temperature in a controlled environment could result in
consistent overestimations.145 In contrast, random errors such as operator mistakes occur in an
unpredictable fashion.146 The analyst approaches the two types of error in different manners:
141
State of Washington v. Fausto and Bellow, Case No. C076949 and 9Y6231062
(King Cty.Dist.Ct., Wash. Sep. 21, 2010).
142
Charles Erlich, Rene Dybkaer & Wolfgang Woger, Evolution of Philosophy and
Description of Measurement (Preliminary Rationale for VIM3), 12 Accreditation and Quality
Assurance 201 (2007).
143
Raghu Kacker, Klaus-Dieter Sommer & Rudiger Kessel, Evolution of Modern
Approaches to Express Uncertainty in Measurement, 44 Metrologia 513, 515 (2007).
144
Douglas A. Skoog, Donald M. West & F. James Holler, Fundamentals of
Analytical Chemistry 13-14 (5th ed. 1988); Ted Vosk, Measurement Uncertainty: Forensic
Metrology and Result Interpretation. Part One: Measurement Results and Interpretation, in
Understanding DUI Scientific Evidence 159, 168 (2011 ed.).
145
Id.
146
Id. at 170.
68
$
The analyst attempts to identify and eliminate all systematic errors.147 It is virtually
impossible to know whether one has identified all the sources of systematic error.148 Often the
best approach is to compare the measurements in question against measurements with a reference
material.149 Based on that comparison, the analyst can compute a bias-corrected statistic.150
Otherwise, after making his or her best efforts to remove the sources of bias, the analyst has to
assume that all systematic error has been eliminated.151
$ The random error, though, is calculable.152 The analyst makes a large number of replicate
measurements to determine how they vary in an unpredictable manner.153 Given that set of
measurements, the analyst can compute a standard error or deviation154 (which, in turn, can be
employed to calculate a confidence interval).
However, there is no generally accepted method of combining systematic and random errors into
an overall error.155 Thus, under classical approach, the witness would testify to both a biascorrected statistic (to compensate for systematic error) and a separate confidence interval
(reflecting random error).
147
Charles Erlich, Rene Dybkaer & Wolfgang Woger, Evolution of Philosophy and
Description of Measurement (Preliminary Rationale for VIM3), 12 Accreditation Quality
Assurance 201, 203-05 (2007).
148
Id. at 206.
149
Ted Vosk, Measurement Uncertainty: Forensic Metrology and Results
Interpretation. Part One: Measurement Results and Interpretation, in Understanding DUI
Scientific Evidence 159, 168-69 (2011 ed.).
150
Id. at 169-70, 185.
151
Id. at 175, 179.
152
Charles Erlich, Rene Dybkaer & Wolfgang Woger, Evolution of Philosophy and
Description of Measurement (Preliminary Rationale for VIM3), 12 Accreditation and Quality
Assurance 201, 206.
153
Id. at 207; Ted Vosk, Measurement Uncertainty: Forensic Metrology and Results
Interpretation. Part One: Measurement Results and Interpretation, in Understanding DUI
Scientific Evidence 159, 170-71, 174, 176 (2011 ed.).
154
Id. at 171.
155
Id. at 178-79 (Athe two kinds of error are fundamentally different, requiring
different treatment@; Charles Erlich, Rene Dybkaer & Wolfgang Woger, Evolution of Philosophy
and Description of Measurement (Preliminary Rationale for VIMS), 12 Accreditation and
Quality Assurance 201, 206 (2007).
69
Yet, modern metrology is moving away from the classical approach.156 Uncertainty
analysis is replacing error analysis.157 Today one of the foundations for uncertainty analysis is
the 1995 Guide to the Expression of Uncertainty in Measurement (GUM).158 The International
Committee for Weights and Measures (CIPM), the International Bureau of Weights and
Measures (BIPM), and several national metrology institutes collaborated to develop the
recommendations forming the core of the GUM approach.159 The essence of the approach is
combining systematic and random effects160 to generate a coverage interval that includes the true
value of the measurand to a certain probability.161 Just as the classical approach attempts to
identify systematic and random errors, the GUM approach considers two types of uncertainty.
The two types differ in the manner in which their numerical values are estimated:162
$ Type A uncertainty can be determined by the statistical analysis of a series of replicate
measurements or observation.163 The analyst can use several statistical techniques such as the
calculation of a standard deviation to quantify the random effects.164
$ Type B uncertainty is determined Aby means other than the statistical analysis of series of
observations.@165 The analyst exercises judgment based on such considerations as his or her
personal experience with the limits of the measurement technique employed.166
156
Id. at 206; Appendix, Understanding DUI Scientific Evidence 459, 514-18 (2011
ed.).
157
Ted Vosk, Measurement Uncertainty: Forensic Metrology and Results
Interpretation. Part One: Measurement Results and Interpretation, in Understanding DUI
Scientific Evidence 159, 180 (2011 ed.).
158
Raghu Kacker, Klaus-Dieter Sommer & Rudiger Kessel, Evolution of Modern
Approaches to Express Uncertainty in Measurement, 44 Metrologia 513, 513 (2007).
159
Id.
160
Id. at 520; Charles Erlich, Rene Dydkaer & Wolfgang Woger, Evolution of
Philosophy and Description of Measurement (Preliminary Rationale for VIM3), 12 Accreditation
and Quality Assurance 201, 209, 211 (2007).
161
Id. at 210-12.
162
Id. at 181.
163
Id. at 211; Ted Vosk, Measurement Uncertainty: Forensic Metrology and Result
Interpretation. Part One: Measurement Results and Interpretation, in Understanding DUI
Scientific Evidence 159, 211 (2011 ed.).
164
Id. at 194.
165
Id.
166
Id. at 194-95 (AIt relies on scientific knowledge, experience, and judgment in the
context of all available information to create a priori probability density functions based on the
70
degree of belief about effects that are not characterized, at least not completely, through
measurement itself. The information relied on may include: [q]uantity values from authoritative
publications and handbooks, [q]uantity values from reference material certifications, [c]alibration
certificates, [m]anufacturer specifications, [a]ccuracy classifications of a verified measuring
instrument, [l]imits deduced through personal experience, [and e]xperience with, or general
knowledge of the behavior and property of relevant materials, methods, and instruments@)
71
Type A evaluations are sometimes called Aobjective@ since they rest on direct analysis of
measurement results while Type B evaluations are termed Asubjective@ because they rely on
Aresearcher judgment and other information.@167 However, it cannot be assumed that Type A
evaluations are necessarily more reliable; while a Type A evaluation might be based on a small
number of measurements, a Type B evaluation could be informed by extensive experience.168
167
Id. at 196.
168
Id.
72
In any event, after conducting both evaluations, the analyst treats the Type A and Type B
standard uncertainties alike169 and uses Bayesian techniques170 to combine171 the two evaluations.
The purpose of the combination is to describe the true state of knowledge about the overall
uncertainty of the point value.172 What is the degree of belief about the accuracy of the point
value?173 The analyst can use an uncertainty budget, listing all the types of Type A and B
uncertainties and their magnitudes.174 The end result is a combined standard uncertainty and
169
Id. at 198.
170
Charles Erlich, Rene Dybkaer & Wolfgang Woger, Evolution of Philosophy and
Description of Measurement (Preliminary Rationale for VIM3), 12 Accreditation and Quality
Assurance 201, 212 (2007); Ted Vosk, Measurement Uncertainty: Forensic Metrology and Result
Interpretation. Part One: Measurement Results and Interpretation, in Understanding DUI
Scientific Evidence 159, 181 (2011). The theorem, devised by the Reverend Thomas Bayes,
permits the combination of probabilities, namely, the use of a new input to revise a prior
probability to yield a posterior probability. 1 Paul C. Giannelli, Edward J. Imwinkelried, Andrea
Roth & Jane Campbell Moriarty, Scientific Evidence ' 15.07[a], at 916-19 (5th ed. 2012), citing
Thomas Bayes, An Essay Toward Solving a Problem in the Doctrine of Chances, 53 Phil.
Transactions Royal Society 370 (1963). Importantly, the theorem enables the statistician to
combine an a prior probability estimated by lay jurors with genetic evidence to establish the
posterior probability. In this context, the statistician can put the systematic effect and random
effect Aon the same footing.@ Vosk, supra, at 182.
171
Ted Vosk, Measurement Uncertainty: Forensic Metrology and Result
Interpretation. Part One Measurement Results and Interpretation, in Understanding DUI
Scientific Evidence 180, 182, 198 (2011 ed.)(ARegardless of whether Type A or Type B analysis
is engaged in, both are based on probability distributions, and the uncertainty so determined is
quantified as either a variance of standard deviation. When expressed as a standard deviation,
each quantified source constitutes a standard uncertainty. >The combined standard uncertainty of
a measurement result . . . is taken to represent the estimated standard deviation of the result=@);
Charles Erlich, Rene Dybkaer & Wolkgang Woger, Evolution of Philosophy and Description of
Measurement (Preliminary Rationale VIM3), 12 Accreditation and Quality Assurance 201, 209,
211 (2011 ed.).
172
Id. at 212; Raghu Kacker, Klaus-Dieter Sommer & Rudiger Kessel, Evolution of
Modern Approaches to Express Uncertainty in Measurement, 44 Metrologia 513, 517 (2007);
Ted Vosk, Measurement Uncertainty: Forensic Metrology and Result Interpretation. Part One:
Measurement Results and Interpretation, in Understanding DUI Scientific Evidence 159, 183,
186 (2011 ed.).
173
Id. at 181.
174
Id. at 197; Patrick T. Barone, DUI Defense: If Breath Alcohol Can=t Be Measured,
It Doesn=t Exist, 37 SADO Crim.Def. Newsletter, Nov. 2013, at 1, 10; Raghu Kacker, KlausDieter Sommer & Rudiger Kessel, Evolution of Modern Approaches to Express Uncertainty in
Measurement, 44 Metrologia 513, 521 (2007); Jesper Kristiansen & Henning Willads Petersen,
73
coverage or credible interval.175 With a specified level of confidence, 176 this Acoverage@177 or
Acredible@178 interval includes the true value. Thus, unlike a confidence interval,179 this coverage
interval includes the true value with a stated probability.180 Together, the best estimate and the
coverage interval completely characterize our knowledge of the quantity value sought.181
Although some commentators believe that it would be possible to use coverage intervals today,
others are of the view that the limited data with respect to breath alcohol analysis makes it
An Uncertainty Budget for the Measurement of Ethanol in Blood by Headspace Gas
Chromatography, 28 Journal of Analytical Toxicology 456 (Sep. 2004).
175
Charles Erlich, Rene Dybkaer & Wolfgang Woger, Evolution of Philosophy and
Description of Measurement (Preliminary Rationale for VIM3), 12 Accreditation and Quality
Assurance 201, 211 (2007); Raghu Kacker, Klaus-Dieter Sommer & Rudiger Kessel, Evolution
of Modern Approaches to Express Uncertainty in Measurement, 44 Metrologia 513, 519 (2007).
There is admittedly some confusion over the proper terminology. R. Willink, Coverage
Intervals and Statistical Coverage Intervals, 41 Metrologia L5, L6 (2004). For example, some
commentators appear to use the expression, Anominal coverage probability,@ to describe what
other statisticians refer to as the statistical confidence interval. Id.
176
Ted Vosk, Measurement Uncertainty: Forensic Metrology and Result
Interpretation. Part One: Measurement Results and Interpretation, in Understanding DUI
Scientific Evidence 159, 203 (2011 ed.).
177
Id.; Charles Erlich, Rene Dybkaer & Wolfgang Woger, Evolution of Philosophy
and Description of Measurement (Preliminary Rationale for VIM3), 12 Accreditation and Quality
Assurance 201, 211-12 (2007); Ted Vosk, Measurement Uncertainty: Forensic Metrology and
Result Interpretation. Part One: Measurement Results and Interpretation, in Understanding DUI
Scientific Evidence 159, 188, 203-04, 224 (2011 ed.).
178
Raghu Kacher, Klaus-Dieter Sommer & Rudriger Kessel, Evolution of Modern
Approaches to Express Uncerainty in Measurement, 44 Metrologia 513, 516 (2007); R. Willink,
Coverage Intervals and Statistical Coverage Intervals, 41 Metrologia L5, L6 (2004); E-mail
communication from Professor David H. Kaye (Nov. 12, 2012).
179
Ted Vosk, Measurement Uncertainty: Forensic Metrology and Result
Interpretation. Part One: Measurement Results and Interpretation, in Understanding DUI
Scientific Evidence 159, 204 (2011 ed.).
180
Id.; Charles Erlich, Rene Dybkaer & Wolfgang Woger, Evolution of Philosophy
and Description of Measurement (Preliminary Rationale for VIM3), 12 Accreditation and Quality
Assurance 201, 211-12 (2007).
181
Ted Vosk, Measurement Uncertainty: Forensic Metrology and Result
Interpretation. Part One: Measurement Results and Interpretation, in Understanding DUI
Scientific Evidence 159, 186 (2011 ed.).
74
inappropriate to use the Bayesian approach to estimate the uncertainty in such analyses.182
182
While some believe that shifting to the Bayesian approach is desirable, others
question whether doing so would be an improvement over the classical approach, at least when
used in clinical or forensic laboratories. The classical methods of determining uncertainty are set
out in numerous textbooks on analytical chemistry and have been employed for well over 100
years.
One proponent of continued use of the classical approach is Dr. G. Simpson. Dr.
Simpson is a retired American research chemist who has specialized in concentration
measurements with a beam of light. He has published several articles in scientific journals,
including Clinical Chemistry, dealing with the margin of error in breath test results. In a series of
emails to the author in January and February 2013, Dr. Simpson stated that in order to determine
the reliability of a breath test result, the amount of error in the result must be known. This can be
done only by conducting experiments on a large number of subjects where rebreathing tests are
conducted to establish the difference between the test result and the rebreathing test result. Dr.
Simpson points out that although there are available simulator data, few data compare subjects=
measured BrAC with the actual BrAC for a particular subject. In his words, AIf the
measurements have not been done, there is no way to establish the margin of error.@ In Dr.
Simpson=s view, clinical and forensic laboratories should eschew the Bayesian approach and
continue to rely on the classical approach.
Likewise, in February and March 2013 emails, a leading European toxicologist, Dr. Frank
Martens of Belgium, pointed out that in his country, all the laboratories continue to employ the
classical approach rather than Bayesian statistics.
75
The GUM approach is not the only alternative to classical error analysis.183 For instance,
there is a hybrid approach, sometimes called the Conventional Value Hybrid Approach.184 This
is a two-step procedure. First, the analyst calibrates a measurement standard by using a Ahighlevel@ measuring system. Next, the analyst performs a second measurement on the calibrated
measurement standard by using a Alower-level@ procedure. AAn example of the CHVA is the use
of a standard weight to verify the performance of a balance. The weight is the (calibrated)
measurement standard, and the balance is the lower-level measuring instrument used to obtain
the measured quantity value . . . . The knowable measurement error is the difference between the
indication and the conventional quantity value of the weight that is placed on the balance.@185
The International Electrotechnical Commission has developed still another approach.186 The
pragmatic IEC approach concentrates on the compatibility of measurement results rather than any
true value.187 The focus is on the question of whether the measurements are sufficiently
compatible for operational purposes.188 For operational purposes, it is often unnecessary to
know the true value.189
III.
CONCLUSION
As Part II.A noted, at one time many courts subscribed to the silly view that an expert
opinion was admissible only if the expert was prepared to vouch that the opinion was certainly
true. In a series of steps, the courts have retreated from that view. Initially, as Part II.B
explained, the courts took the small step of ruling that it was permissible for an expert to testify
to an opinion couched as a mere probability or possibility. The courts began to realize that
especially in investigational science when experts rely on inductive reasoning, in principle an
expert can never validate any hypothesis as a certainty. Of course, as Part II.C demonstrated, the
183
Jack Wallace, Ten Methods for Calculating the Uncertainty of Measurement, 50
Science and Justice 182 (2010). The American Society of Crime Laboratory Directors Adoes not
prescribe a specific formula@ or Aa specific approach to estimating uncertainty of measurement.@
Ted Vosk, Measurement Uncertainty: Forensic Metrology and Result Interpretation. Part One:
Measurement Results and Interpretation, in Understanding DUI Scientific Evidence 159, 213
(2011 ed.).
184
Charles Erlich, Rene Dybkaer & Wolfgang Woger, Evolution of Philosophy and
Description of Measurement (Preliminary Rationale for VIM3), 12 Accreditation and Quality
Assurance 201, 215-17 (2007).
185
Id. at 216-17.
186
Id. at 213.
187
Id.
188
Id.
189
Id.
76
logic of that realization led farther; the logic dictated that the courts forbid overstated opinions
phrased as certainties. Part II.D documents the denouement of this new line of authority: judicial
mandates that whenever practical, the expert give the trier of fact a quantitative measure of the
error or uncertainty of the measurements underlying his or her opinion.
That denouement represents progress in two important respects. First, this incipient trend
promotes honesty in the courtroom. It is axiomatic that measurements are inherently uncertain.
As the Washington cases emphasize, it is misleading to present the trier of fact with only a single
point value. There is a grave risk that without the benefit of qualifying testimony, a miscarriage
of justice can occur because the trier will mistakenly treat the point value as exact and ascribe
undue weight to the evidence. The antidoteBthe necessary qualification--is a quantitative
measure of the margin of error or uncertainty.
Secondly, this trend should foster a more cooperative relationship between law and
science. Scientists often complain that the law forces them to use artificial standards and
pressures them to testify in terms that make them uncomfortable. Psychiatrists frequently assert
that the law compels them to use definitions of insanity that have no counterpart in their
professional discipline.190 Likewise, experts who know that their measurements are uncertain
are ill at ease opining that their opinion is a scientific or medical certainty.191 Like any other
witness, the expert takes an oath to tell Athe whole truth;@192 and an expert with any familiarity
with metrology knows that his or her testimony about the single point value is far from Athe
whole truth.@ The law should permit and encourage experts to Aemploy[] in the courtroom the
same level of intellectual rigor that characterizes the practice of an expert in the relevant field.@193
In Daubert, Justice Blackmun acknowledged that Athere are important differences between the
quest for truth in the courtroom and the quest for truth in the laboratory.@194 However, there is
one point on which judges, attorneys, and lawyer ought to agree: No expert should be permitted
to mislead the trier of fact.195 There is an unavoidable uncertainty in every measurement
190
T. L. Clanon, Lois Shawer & Douglas Kurdys, Less Insanity in the Courts, 68
American Bar Association Journal 824, 824 (July 1982)(in court, psychiatrists and psychologists
Aare asked to testify on issues that have no scientific definitions . . . .@; psychiatrists and
psychologists are Adissatis[fied@ because the law relies on Aan obsolete concept . . . of the
relationship between mental illness and crime@; the disconnect is so extreme that A[m]any
[scrupulous] psychiatrists and psychologists refuse to participate in criminal trials . . . .@).
191
Rod Gullberg, Professional and Ethical Considerations in Forensic Breath Alcohol
Testing Programs, 5 Journal of the Alcohol Testing Alliance 22 (2006).
192
State of Washington v. Fausto and Ballow, No. CO76949 and 9Y6231062
(King.Cty.Dist.Ct., Wash. Sep. 21, 2012); Ted Vosk, Measurement Uncertainty: Forensic
Metrology and Result Interpretation. Part Two: Legal Analysis, in Understanding DUI Scientific
Evidence 255, 312 (2011 ed.).
193
Kumho Tire Co., Ltd. v. Carmichael, 526 U.S. 137, 152 (1999).
194
Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 596-97 (1993).
195
In the words of Federal Rule of Evidence 702(a), misleading testimony does not
77
underpinning an expert opinion, and it is intellectually dishonest for the expert to pretend
otherwise in court.
Ahelp the trier of fact to understand the evidence or to determine a fact in issue . . . .@
Furthermore, Federal Rule 403 recognizes that the risk of Amisleading the jury@ can justify the
exclusion of relevant, otherwise admissible evidence.
78