Download The Importance of Quantifying Uncertainty

The Importance of Quantifying Uncertainty: Examining the Effect of Audit Materiality
and Sensitivity Analysis Disclosures on Investors’ Judgments and Decisions
Aasmund Eilifsen
NHH Norwegian School of Economics
[email protected]
Erin L. Hamilton
University of Nevada, Las Vegas
[email protected]
William F. Messier, Jr.
University of Nevada, Las Vegas
and
NHH Norwegian School of Economics
[email protected]
Preliminary Version:
Do not quote without authors’ permission.
March 2017
We gratefully acknowledge feedback received from Stephen Perreault, Aaron Saiewitz, Jason
Smith, and workshop participants at the 2017 AAA Auditing Section Midyear Meeting and the
University of Nevada, Las Vegas. Professor Messier received financial support for this research
from the Kenneth and Tracy Knauss Endowed Chair in Accounting at UNLV and his Adjunct
Professor position at the Norwegian School of Economics.
The Importance of Quantifying Uncertainty: Examining the Effect of Audit Materiality
and Sensitivity Analysis Disclosures on Investors’ Judgments and Decisions
ABSTRACT
At present, accounts containing extreme levels of estimation uncertainty may be obscured
from investors given that they are reported using precise point estimates within financial
statements that receive an unqualified opinion from the auditors. The way in which investors
perceive these highly subjective estimates may depend on how transparently the level of
uncertainty underlying the estimate is disclosed. Our study examines two disclosures that should
help quantify the amount of uncertainty contained within a highly subjective fair value estimate: a
quantitative sensitivity analysis and the auditor’s quantitative materiality threshold. Using an
experiment with nonprofessional investors, we predict and find that participants who receive both
a quantitative sensitivity analysis and a materiality disclosure can differentiate between an estimate
with relatively low estimation uncertainty (i.e., low sensitivity) and one containing extreme
estimation uncertainty (i.e., high sensitivity). When materiality is not disclosed, however, investors
fail to recognize differences in reliability between the two levels of sensitivity, even though the
amount of imprecision in the low sensitivity condition represents a fraction of materiality, while
in the high sensitivity condition, this amount exceeds audit materiality multiple times over.
Keywords: Audit reporting, disclosures, estimates, estimation uncertainty, materiality,
sensitivity analysis.
Data Availability: The data is available from the authors.
I. INTRODUCTION
The extreme estimation uncertainty inherent in certain financial statement accounts has
become a substantial concern in recent years for auditors and standard setters alike (Christensen,
Glover and Wood 2012; PCAOB 2012; Glover, Taylor and Wu 2017).1 This estimation uncertainty
(or imprecision) in reported numbers has resulted from the increasing complexity of business
transactions and use of fair value measurements that often rely on management-derived models
that are extremely sensitive to small changes in subjectively-chosen inputs (e.g., Level 3 fair value
inputs) (Christensen et al. 2012; PCAOB 2012). In fact, some numbers recognized on the face of
the financial statements contain a level of estimation uncertainty that exceeds audit (planning)
materiality – sometimes by multiples of materiality (Christensen et al. 2012; Glover et al. 2017).2
At present, it is unclear whether investors recognize when financial statements contain
extreme levels of estimation uncertainty, given that all estimates, regardless of their imprecision,
appear as precise point estimates on the face of the financials. Understanding the amount of
uncertainty inherent in reported estimates is important, as prior archival research finds that less
reliable estimates lead to lower earnings persistence, and a failure on the part of investors to
anticipate this effect results in security mispricing (Richardson, Sloan, Soliman and Tuna 2005).
Accordingly, the purpose of our study is to examine whether investors are better able to evaluate
the reliability of uncertain estimates when this uncertainty is quantified. Specifically, we examine
1
Similar to Bell and Griffin's (2012, 148) description of a high-uncertainty estimate, we use the term extreme
estimation uncertainty to describe an estimate "for which it is reasonably possible that the level of inherent (i.e.,
irreducible) measurement uncertainty could be closely approaching or exceed the financial statement (and auditor's)
materiality threshold." Estimation uncertainty results from multiple sources, including the inability to precisely
measure one or more of the inputs into an estimation model at the date of the financial statements (i.e., unobservable
inputs) and historical volatility in model inputs (Christensen et al. 2012).
2
Planning materiality is an amount that the auditor establishes for the financial statements as a whole. This amount is
used to plan the nature, timing, and extent of audit procedures.
1
two disclosures that should help quantify the amount of uncertainty (or imprecision) in a highly
subjective fair value estimate: (1) a quantitative sensitivity analysis (QSA) and (2) the auditor’s
quantitative materiality threshold.
Over the past few years, regulators and standard setters worldwide have proposed or
adopted various accounting and auditing standards intended to enhance financial statement users’
understanding of complex and subjective aspects of financial reporting (e.g., IASB 2010, 2011;
FASB 2010a, 2011; FRC 2013; IAASB 2015; EC 2014; PCAOB 2013, 2016). The intent of such
increased disclosure is to improve users’ ability to make well-informed decisions based on
inherently uncertain financial statement information. One such disclosure is a sensitivity analysis,
which provides information about how a given estimate would vary if one or more of its
subjectively-chosen inputs were to change (IASB 2010). Sensitivity analyses can be either
quantitative or qualitative in nature (see Appendix A for an example of each). In 2010, the
Financial Accounting Standards Board (FASB 2010a) and International Accounting Standards
Board (IASB 2010) considered requiring a QSA for all Level 3 fair value measurements. In the
end, however, both the FASB and IASB decided to defer the decision on whether to require a
quantitative disclosure, and instead put into place a requirement that management provide a
narrative (i.e., qualitative) description of sensitivity (FASB 2011). Nonetheless, we believe a QSA
is more informative than a narrative disclosure, as it explicitly quantifies the amount of uncertainty
contained within a fair value estimate and should help investors differentiate between financial
statement items that are relatively reliable compared to relatively unreliable.
It is possible, however, that a QSA alone may not provide enough information to help
investors meaningfully evaluate the reliability of a subjective estimate. Although a QSA quantifies
the amount of uncertainty in an estimate, investors still must evaluate the significance of this
2
imprecision to the financial statements as a whole and to their own judgments and decisions. We
predict that the disclosure of a quantitative materiality threshold within the auditor’s report will
help investors interpret a QSA. A materiality threshold indicates the amount of imprecision or
inaccuracy (i.e., the level of misstatement) that auditors are willing to tolerate within the financial
statements. In this way, a materiality threshold may serve as a useful benchmark for evaluating the
severity of imprecision communicated within a QSA.
Accordingly, we predict that in the absence of a materiality disclosure, investors may
struggle to differentiate between a relatively precise (i.e., low sensitivity) fair value estimate and
one that contains an extreme level of estimation uncertainty (i.e., high sensitivity). When a
materiality threshold is disclosed; however, we expect investors will recognize when a reported
estimate contains a level of imprecision that exceeds audit materiality and will evaluate the item’s
reliability to be significantly lower compared to a more precise estimate (i.e., one in which the
range of uncertainty falls far below the auditor’s materiality threshold). Furthermore, when both
disclosures are absent and investors are only provided with a narrative description of sensitivity,
the inability to quantify the uncertainty contained in a highly subjective estimate is expected to
result in the lowest perceptions of financial statement reliability.
We test our predictions experimentally using a 2 × 2 + 1 between-subjects design in which
we manipulate the range of uncertainty indicated by a QSA (low sensitivity or high sensitivity)
and the presence or absence of a materiality threshold disclosure. We also include a control
condition in which participants only receive a narrative description of sensitivity, while a
materiality threshold and QSA are absent.3 Participants receive an audit report that describes a
3
The narrative description of sensitivity is held constant across all conditions and simply informs participants that
“even minor changes in (a subjectively chosen input) may result in significant changes in operating profit reported for
the period.”
3
highly subjective fair value estimate within a critical audit matter (CAM) disclosure.4 For some
participants, the auditor’s report also includes a disclosure of the auditor’s quantitative materiality
threshold. After reviewing the auditor’s report and an income statement that contains the line item
associated with the subjective fair value estimate, participants read a footnote disclosure related to
the estimate. In all conditions, a narrative description of sensitivity is provided. Additionally,
participants not assigned to the control condition receive a QSA that either indicates a relatively
low level of sensitivity (i.e., an amount of imprecision that falls far below the auditor’s materiality
threshold) or a relatively high level of sensitivity (that exceeds materiality by multiples of
materiality).
Consistent with our expectations, we find that when only a narrative description of
sensitivity is disclosed (i.e., the control condition), investors perceive the reliability of the fair
value estimate to be lower than when a QSA is also provided. Most importantly, we find that
investors’ ability to recognize differences in estimation uncertainty between the low and high
sensitivity conditions is improved when they are presented with both a QSA and a materiality
threshold disclosure. Without the disclosure of materiality to help investors interpret the QSA,
however, investors fail to recognize differences in reliability between the two levels of sensitivity,
even though the amount of imprecision in the high sensitivity condition is approximately 15 times
the amount indicated by the QSA in the low sensitivity condition. Surprisingly, we find that while
investors perceive the high sensitivity estimate to be relatively unreliable (when materiality is
disclosed), their willingness to invest in the company is not reduced relative to the low sensitivity
4
CAMs (also referred to as "key audit matters") are disclosures contained within the auditor’s report that “inform
investors and other financial statement users of matters arising from the audit that required especially challenging,
subjective, or complex auditor judgment” (PCAOB 2016, 3). Although CAM disclosures are unlikely to provide
investors with enough information to fully understand a CAM item, they have an attention-directing effect, prompting
investors to consider more carefully other disclosures that provide additional information about the item (e.g., related
footnote disclosures) (PCAOB 2016; Sirois, Bédard and Bera 2016).
4
condition. An additional experiment reveals that this finding results from an unconscious error in
reasoning on the part of our investor participants. When differences in estimation uncertainty are
made more salient to participants using a within-subjects design, we find that investors greatly
prefer to invest in a company whose financial statements contain a lower level of estimation
uncertainty.
Our study makes several important contributions to the accounting literature as well as to
accounting practice and policy. While several studies have examined how investors respond to
sensitivity analyses (e.g., Koonce, Lipe and McAnally 2005; Koonce, McAnally and Mercer 2005;
Elliott, Jackson and Smith 2008; Nelson and Rupar 2015), very little evidence exists regarding
factors that enhance the usefulness of the information contained within a sensitivity analysis
disclosure. Our results suggest that sensitivity analyses may be somewhat ineffective in helping
investors understand the amount of uncertainty contained in Level 3 fair value estimates, unless
they are quantitative in nature and supplemented with a quantitative materiality disclosure.
Furthermore, our study examines a context in which the subjectivity inherent in a reported estimate
is highlighted within a CAM disclosure. Given that CAMs are set to become a standard component
of the auditor’s report in much of the world, it is important for practitioners and standard setters to
begin to consider the types of information that will allow investors to understand the subjectivity
and complexity highlighted in CAM items. Our study serves such an objective.
Despite receiving strong support from investors (e.g., FASB 2010a; IASB 2010; PCAOB
Investor Advisory Group 2011; Singh and Peters 2015), standard setters have decided against
requiring the disclosure of QSAs and audit materiality (e.g., FASB 2010a, 2011; IASB 2011;
IAASB 2015), with very few exceptions.5 According to the FASB, preparers have argued that a
5
For example, a QSA is required when estimating the fair value of financial instruments (per IFRS 7) and a
quantitative materiality disclosure is required in the U.K. (FRC 2013).
5
QSA “would not be decision useful because the range of reasonably possible Level 3 fair values
would be extremely wide and, thus, would be meaningless” (PCAOB 2010, 45). Counter to this
argument, the results of our study suggest that QSA disclosures provide meaningful information
to investors that impacts their judgments and decisions, but only when a materiality threshold is
also disclosed. These findings should be informative for standard setters as they continue to
consider the types of disclosures that may help investors and other financial statement users
understand the most complex and subjective aspects of financial reporting.
The remainder of this paper proceeds as follows. In section II, we review related literature
and develop our hypotheses. The methodology and results are discussed in sections III and IV,
respectively. Finally, we conclude with a discussion in section V.
II. BACKGROUND AND HYPOTHESES
Extreme Estimation Uncertainty
Many financial statement accounts require estimation due to uncertainty surrounding the
measurement and valuation of items that are contingent on the outcome of future events (e.g.,
allowance for doubtful accounts, fair value measurement, goodwill impairment) (PCAOB 2003, ¶
1). Some estimates require complex valuation models that use uncertain and unobservable inputs
(e.g., Level 3 fair value inputs), which are highly subjective. As a result, the level of uncertainty
inherent in some estimates creates a range of reasonably possible values that exceeds auditors’
materiality thresholds, sometimes by multiples of materiality (Christensen et al. 2012; Cannon and
Bedard 2016; Glover et al. 2017). Christensen et al. (2012) use publicly available data to
demonstrate that a relatively small change to a fair value input (e.g., interest rates) has the potential
to change an estimate's reported value by as much as 50 times audit materiality.6 Because changes
6
Christensen et al. (2012) estimate audit materiality using the common materiality threshold of 5 percent of pre-tax
income to approximate the extent to which small changes in fair value inputs exceed materiality.
6
in fair value estimates often impact earnings, the net income figure reported by some companies
may also include an extreme level of estimation uncertainty.
Estimates containing a high degree of uncertainty can be problematic because they are
inherently unreliable due to their dependence on management’s judgment, which may be
inaccurate due to bias and/or the difficulty associated with predicting future conditions and events
(Lev, Li and Sougiannis 2010).7 The more subjective an estimate, the less reliable, and therefore,
the less useful it is to financial statement users (Richardson et al. 2005; Lev et al. 2010). Richardson
et al. (2005) provide empirical evidence that more subjective (and less reliable) accruals estimates
result in lower earnings persistence and that investors fail to anticipate this effect, leading to
security mispricing. Unfortunately, it is not always possible to reduce estimation uncertainty to an
immaterial amount (Bell and Griffin 2012). In the presence of extreme and irreducible estimation
uncertainty, the question becomes, how can management and auditors effectively communicate
the uncertainty contained within financial statement items, so that users can incorporate this
information into their decisions in a meaningful way?
As CAM disclosures become a standard component of the auditor’s report in much of the
world, identifying effective ways to communicate information about estimation uncertainty to
financial statement users is even more important. 8 The IAASB (2015) and PCAOB (2016)
anticipate that CAMs will frequently focus on financial statement areas involving significant
management judgment and high estimation uncertainty. By drawing attention to difficult and
7
Reliable financial statement information is that which is “free from material error or bias and can be depended upon
by users to represent faithfully that which it purports to represent” (FASB 2010b, ¶ BC3.22). While reliability was a
component of the original Conceptual Framework, it has since been replaced with the concept of faithful representation,
which the FASB believes is a clearer concept that still encompassed the original components of reliability (FASB
2010b).
8
CAM disclosures are now required by the IAASB (2015), European Commission (EC 2014), and the U.K.'s Financial
Reporting Council (FRC 2013). The PCAOB (2016) recently issued a revised proposal that would require auditors in
the U.S. to include CAM disclosures within the auditor's report.
7
subjective audit areas, CAMs may enable financial statement users to “analyze more closely any
related financial statement accounts and disclosures” (PCAOB 2013, p. A5-23). This attention
directing effect of CAMs, supported empirically in an experiment by Sirois et al. (2016), suggests
that CAMs may be most useful when they are supplemented with additional disclosures aimed at
helping users understand the uncertainty highlighted in the CAM. We focus on two disclosures
that are especially relevant for communicating the risk and uncertainty inherent in estimates
containing extreme estimation uncertainty: (1) quantitative sensitivity analyses (QSAs) and
quantitative auditor materiality thresholds.
Quantitative Sensitivity Analyses
One type of supplemental footnote disclosure that may help financial statement users
understand the amount of uncertainty contained within an account is the disclosure of a QSA. A
QSA computes the potential increase or decrease in a reported estimate (and/or the resulting effect
on earnings) that would result from a change in one or more of the inputs used to determine the
estimate (Linsmeier and Pearson 1997). For example, an estimate may be calculated using various
assumptions, such as future interest rates, commodity prices, etc. A QSA would indicate the effect
on earnings if one or more of these inputs were to change by a certain amount (e.g., a one percent
change in commodity prices would result in a $5 million change in reported earnings). While a
QSA is required for certain financial statement items (e.g., financial instruments measured at fair
value per IFRS 7),9 other items may only require a qualitative analysis or no sensitivity analysis
at all (see Appendix A for a comparison of a quantitative and qualitative sensitivity analysis). In
2010, the FASB (2010a) and IASB (2010) considered requiring a QSA for all Level 3 fair value
estimates. Although financial statement users were supportive of a quantitative disclosure and
9
Additionally, FASB ASC 715-20-50-1 requires public companies in the U.S. to disclose the sensitivity of
accumulated postretirement benefit obligations to a one percent change in assumed health care cost trend rates.
8
believe it provides “useful information that helps them to assess the subjectivity of an entity’s fair
value measurements categorized [as] Level 3” (IASB 2010, 16), both standard setting bodies
ultimately decided that a narrative description of sensitivity would suffice (FASB 2011).
Prior research finds that investors’ judgments and decisions are influenced by the
information contained within a QSA. For example, the format in which a QSA is presented (e.g.,
stating the impact on earnings in dollar versus percentage terms) appears to influence investors’
judgments regarding an item’s risk and reliability (Koonce, Lipe and McAnally 2005; Elliott et al.
2008; Nelson and Rupar 2015). The amount of loss communicated within a QSA also appears to
influence investors’ risk perceptions (Koonce, McAnally and Mercer 2005). Our study differs from
these previous studies in three distinct ways. First, our study examines the effect of a QSA in an
environment in which an uncertain estimate is highlighted within a CAM disclosure. When the
subjectivity inherent in a fair value estimate has been highlighted within a CAM, investors may
interpret a supporting QSA very differently than when such a CAM disclosure is absent (as has
been the case in prior literature).10 Second, our study compares the effect of a QSA to that of a
narrative description of sensitivity. This is an important comparison, given that standard setters
have debated whether a QSA provides meaningful information to investors beyond that contained
within a qualitative disclosure (FASB 2010a, 2011; IASB 2010). Finally, prior studies have not
examined how other management or auditor disclosures (e.g., disclosure of the auditor's
materiality threshold) may enhance the usefulness or understandability of a QSA.
When investors are aware that a point estimate reported on the face of the financial
statements contains estimation uncertainty (e.g., because this uncertainty has been highlighted
10
By highlighting a significant financial statement area, a CAM disclosure may cause investors to be more responsive
to information contained within a related footnote disclosure. In contrast, a CAM may cause investors to perceive an
item as generally unreliable, regardless of what may be indicated within a related QSA. Therefore, results obtained in
prior literature may not generalize to CAM environment.
9
within a CAM disclosure), they are likely to view the item as unreliable and desire additional
information to understand this uncertainty better. A narrative description of sensitivity is unlikely
to do much to satisfy investors' desire for additional information, as a qualitative sensitivity
analysis simply states that an estimate is highly sensitive to changes in underlying inputs. A QSA,
on the other hand, explicitly quantifies the amount of uncertainty in the reported estimate and,
therefore, should increase investors' perceptions of the item's reliability by providing an
informative supplement to the point estimate reported on the face of the financial statements.
Investors generally prefer financial disclosures which contain a level of precision that
matches (i.e., is congruent with) the level of uncertainty underlying the financial item itself,
referred to as the “congruence principle” (Du, Budescu, Shelly and Omer 2011). When a point
estimate is reported for a very uncertain item, investors believe the point estimate is inaccurate and
lacking in credibility. A range estimate, on the other hand, is perceived as more credible when
uncertainty is high because a range communicates that uncertainty exists and gives the parameters
of that uncertainty (Du et al. 2011). Given that all numbers reported on the face of the financial
statements are point estimates, which are likely to be viewed as unreliable in the case of highly
subjective estimates (Du et al. 2011), we examine whether supplementing a point estimate with a
QSA increases the perceived reliability of the reported amount.11 Furthermore, there is a general
belief within accounting that quantified disclosures are of higher quality because they are more
informative than disclosures that are purely qualitative (e.g., Botosan 1997; Kadous, Koonce, and
Towry 2010). Therefore, we expect that investors will perceive a highly subjective estimate as
11
Du et al. (2011) examined financial disclosures in the form of management earnings forecasts, which were either
reported as a point estimate or a range estimate. We extend Du et al. (2011) by examining whether supplementing a
point estimate on the face of the financial statements increases the perceived reliability of the reported estimate.
10
more reliable when it is supplemented with a QSA compared to when only a narrative description
of sensitivity is provided. Stated formally, our first hypothesis is as follows:
H1: Investors will judge the reliability of an uncertain estimate to be higher when
it is supplemented with a QSA compared to when it is not.
While a QSA is expected to reduce uncertainty, compared to when such an analysis is
absent, the degree to which uncertainty is reduced depends on the “sensitivity” of the financial
statement item. Consider an example in which a company reports a QSA indicating that a one
percentage point increase in an interest rate used to calculate an uncertain estimate will result in a
$50 million decrease in earnings. Compare this to an identical disclosure that only indicates a
potential earnings effect of $2 million. Obviously, the former example reduces uncertainty to a
much lesser extent than the latter example due to the high level of imprecision communicated by
the sensitivity analysis. Although investors prefer that estimates be communicated in a format that
is congruent with the uncertainty that exists in the environment, they also desire the
communication to be as precise as possible (e.g., a narrow range as compared to a wide range of
potential values) (Du et al. 2011). Therefore, we expect the level of sensitivity communicated by
a QSA to be negatively related to investors' perceptions regarding an estimate's reliability (i.e., less
sensitivity will result in increased perceptions of reliability). Stated formally, our second
hypothesis is as follows:
H2: In the presence of a QSA, investors will judge the reliability of an uncertain
estimate to be higher when the item is less sensitive to changes in underlying
inputs compared to more sensitive.
Auditors’ Quantitative Materiality Thresholds
Financial statement information is considered to be material if its omission or misstatement
is likely to influence the judgments of a reasonable investor (PCAOB 2010). In recent years,
several standard setting bodies have considered requiring the disclosure of quantitative materiality
11
thresholds within the auditor’s report. As a result, audit reports issued in the U.K. now include a
disclosure of the materiality threshold used for the financial statements as a whole (FRC 2013).12
The PCAOB, ASB, and IAASB, on the other hand, have decided against a materiality disclosure
requirement. However, auditors may choose to disclose materiality levels voluntarily (IAASB
2015), and in fact, PwC in Sweden and Finland has elected to do so for audits of financial
statements with years ending on or after December 31, 2016. While most standard setting bodies
have decided against a materiality disclosure requirement, investors generally support the
provision of such information (PCAOB 2011; Singh and Peters 2015).13
Much of the prior literature on materiality has focused on the various determinants of
materiality thresholds (Eilifsen and Messier 2015; see also Holstrum and Messier 1982 and
Messier, Martinov-Bennie, and Eilifsen 2005 for reviews of the materiality literature). Only a few
studies have considered the potential impact of disclosing materiality levels to the public. These
studies suggest that the disclosure of materiality may increase the accuracy of security prices
(Fisher 1990) and help to reduce the expectations gap that exists between auditors and the investing
public (Coram, Mock, Turner, and Tray 2011; Gray, Turner, Coram, and Mock 2011; Houghton,
Jubb, and Kend 2011). Our study extends prior research on materiality by examining how the
disclosure of a quantitative materiality threshold, in addition to a QSA disclosure, influences
investors’ perceptions regarding an amount recognized within the financial statements that
contains estimation uncertainty.
12
The EC (2014) requires a similar disclosure of the quantitative materiality threshold used for the overall financial
statements, but it is disclosed in an additional report provided to the audit committee rather than to the public.
13
The PCAOB’s Investor Advisory Group (2011) reports that 56 percent of investors surveyed agree that auditors
should be required to disclose materiality, while only 17 percent disagreed. Also, the CFA Institute reports that 82
percent of survey respondents support increased disclosure about materiality (Singh and Peters 2015).
12
As previously discussed, we expect the disclosure of a QSA to help quantify the amount
of uncertainty in an estimate. Once this uncertainty has been quantified, however, investors still
must assess how significant this imprecision (i.e., potential misstatement) is to the financial
statements as a whole and how this information might impact their judgments and decisions.14 The
disclosure of a quantitative materiality threshold should provide investors with a useful benchmark
for assessing the significance of a potential misstatement. Because materiality thresholds are
company-specific (i.e., are determined based on the quantitative and qualitative characteristics of
a given company), they make it possible to evaluate the severity of a misstatement in the context
of a given company's financial statements.
Context Theory suggests that the way in which a given cue is interpreted depends on the
context within which it is presented (Medin and Schaffer 1978; Medin, Goldstone, and Gentner
1993). While it may be difficult for investors to interpret a QSA, the disclosure of a quantitative
materiality threshold provides investors with a simple benchmark for evaluating the level of
uncertainty communicated by the QSA (i.e., puts the QSA "in context"). When a QSA suggests
that the imprecision in a reported estimate exceeds materiality (potentially by multiples of
materiality), investors are likely to perceive the estimate as very unreliable. In contrast, when a
QSA suggests that the level of imprecision falls below materiality, investors are likely to perceive
the related estimate as more reliable as it falls within a range of uncertainty considered
“acceptable” by the auditors. Stated formally, our third hypothesis is as follows:
H3: In the presence of a QSA, investors will judge the reliability of an uncertain
estimate to be highest (lowest) when the QSA indicates a range of potential
values that falls below (exceeds) the auditor’s disclosed materiality threshold.
14
Imprecision in reported estimates creates potential for material misstatement, as it indicates that an amount reported
in the financial statements may not be accurate. If this imprecision is large enough, it may suggest that the financial
statements (including net income) are materially misstated.
13
Given that the concept of materiality describes information or amounts expected to
influence the judgments and decisions of reasonable investors, we also examine how our variables
of interest influence investors' ultimate willingness to invest. When estimation uncertainty is
disclosed more transparently through a QSA and materiality disclosure, we expect investors to
recognize the risk that certain financial statement amounts (including net income) may be
materially misstated. As a result, we predict that when the uncertainty indicated within a QSA falls
below auditors' materiality threshold, investors will be more confident that the financial statements
are not materially misstated and will be more willing to invest. In contrast, when uncertainty
communicated within a QSA exceeds the materiality threshold, investors will doubt the reliability
of the financial statements, and in turn, will be relatively unwilling to invest. Stated formally, our
final hypothesis is as follows:
H4: In the presence of a QSA, investors will display the greatest (least)
willingness to invest in a company with financial statements containing an
uncertain estimate when the QSA indicates a range of potential values that
falls below (exceeds) the auditor’s disclosed materiality threshold.
III. METHOD
Participants
Participants in the experiment consisted of 193 graduate students enrolled in either a
Master’s in Accounting and Auditing program or a Master's in Economics and Business
Administration program at a large public business school in Bergen, Norway. 15 Like MBA
students, our participants also have the requisite coursework necessary to qualify as proxies for
nonprofessional investors (Elliott, Hodge, Kennedy, and Pronk 2007). Participants had completed
15
Two participants were determined to be outliers and were removed from the sample. One of the participants
completed the study in less than three minutes (median completion time was 20.4 minutes). The other participant's
response to the dependent variable (perceived reliability) was more than three standard deviations below the mean.
Including these participants does not change our study's inferences.
14
an average of 8.8 classes (during their bachelor’s and master’s programs) related to financial
accounting, auditing, finance, and valuation and had an average of 1.4 years of professional work
experience in the fields of accounting, finance, and/or business. Fifty percent of participants were
22-25 years old, 40 percent were 26-29, 8 percent were 30 or older, and 2 percent were under 22
years old. Forty-one percent of participants were female.16
Experimental Materials and Procedures
To examine our research hypotheses, we conducted an experiment that utilized a 2 × 2 + 1
between-subjects design. We manipulated the auditor’s disclosure of materiality in the audit report
(present versus absent) and management’s disclosure of a QSA (low sensitivity versus high
sensitivity). We also included a control condition in which both disclosures (materiality and a
QSA) were absent. In all conditions (including the control), participants are provided with a
narrative description of the estimate's sensitivity.
The experiment was conducted online using Qualtrics and was administrated in two
graduate-level courses by two of the study’s researchers. The study link was distributed to students
via email during class time and was completed in class by those students in attendance and outside
of class by those students not in attendance on the day of the study.
The experiment was completed in three stages. In the first stage, participants were
instructed to assume the role of a nonprofessional investor who is considering an investment in a
hypothetical fish farming company, King Fish Farming Group (KFFG). Participants received a
brief description of KFFG, before being provided with select information from the company’s
current year annual report. Specifically, participants viewed excerpts from the auditor’s report,
which included a standard unqualified opinion as well as a critical audit matter (CAM) disclosure.
16
None of the demographic variables differ significantly across conditions nor are they significant covariates in any
of the models tested.
15
In all conditions, the CAM disclosure described the high level of uncertainty associated with
estimating KFFG’s “adjustment of live fish inventory to fair value,” which is an income statement
account reflecting gains and losses associated with adjusting the company’s live fish inventory to
net realizable value.17 At the end of the CAM disclosure, participants were directed to, “See note
1 to the financial statements for the disclosure…related to the fair value of live fish.” (See
Appendix B for the full text of the CAM paragraph.) The audit report also included our
manipulation of the auditor’s materiality threshold disclosure (absent or present), which is
described in more detail in the next section of the paper.
The audit report was followed by a simplified income statement that included the operating
profit line item “adjustment of live fish inventory to fair value.” For each of the two years presented,
the magnitude of the adjustment was substantial relative to earnings but with opposite effects (i.e.,
the adjustment reflected a gain in the current year and a loss in the prior year). Therefore, even
though the company’s earnings increased by nearly 30 percent in the current year, this increase is
largely attributable to the substantial increase in the adjustment account, which represents a highly
subjective management estimate.
After viewing the company background, audit report, and income statement, participants
were asked for their initial perceptions regarding the reliability of KFFG’s “adjustment of live fish
inventory to fair value.” At this point in the study, participants had not yet received the footnote
17
Our experimental materials were based on the QSA disclosures of an actual fish farming company in Norway. Live
fish (a biological asset) are measured at fair value less cost to sell (i.e., net realizable value) in accordance with IAS
41 “Agriculture” and IFRS 13 “Fair Value Measurement.” Observable markets and transactions for the future sale of
live fish do not exist, so the valuation of fish inventory requires the establishment of an estimated fair value of the fish
in a hypothetical market. Therefore, fair value measurement is categorized as Level 3 in the fair value hierarchy.
Market prices for salmon and trout are extremely volatile due to seasonal variations in supply and demand, variations
in the quality and size of fish, and environmental factors, such as disease outbreaks and food scares. These factors, as
well as the three year production cycle of fish inventory, make it difficult to estimate future sales prices. Due to the
uncertain nature of estimating future fish prices, fish farming companies routinely disclose QSAs, which display the
potential effects on profit if the estimated fish prices used in the fair value model were to change.
16
disclosure related to the fair value adjustment account (and the accompanying QSA manipulation);
therefore, these initial reliability judgments serve as pre-treatment measures.
In the second stage of the study, participants received a footnote disclosure related to “live
fish inventory” and the associated income statement account (i.e., “adjustment of live fish
inventory to fair value”). The footnote included a description of how the estimated fair value of
live fish is determined, including how the estimate critically depends on estimated future market
prices of live fish (one of the primary inputs into management’s fair value model), and how even
minor changes in these estimated prices may result in significant changes in operating profit (i.e.,
a narrative description of sensitivity). (See Appendix C for the full text of the footnote). The
footnote disclosure contained our QSA manipulation (described in detail in the next section).
Finally, in stage three of the experiment, participants answered our dependent variables
and other post-experimental questions. Participants had access to the full case materials as they
answered these questions. Afterwards, participants responded to manipulation check and
demographic questions (access to case materials was prohibited in this final section of the
questionnaire).
Independent Variables
The first independent variable, Materiality Threshold, relates to a disclosure within the
auditor's report of the quantitative materiality threshold used during the audit of KFFG (Present
versus Absent). In all conditions, the audit report contained a brief definition of materiality, but
only participants in the Materiality Threshold – Present condition received the following sentence:
“We determined materiality for the Group to be NOK 44 million, which is approximately 5% of
profit before tax.”18 (See Appendix B for the full audit report.)
18
The Norwegian krone (NOK) is the currency of Norway. One U.S. dollar is equal to approximately 8.29 NOK.
17
The second independent variable, QSA, relates to the disclosure of a QSA within the "live
fish inventory" footnote (Low Sensitivity versus High Sensitivity). The QSA appeared as a table,
which quantified the amount by which operating profit would change if a critical input used in the
fair value estimate (i.e., estimated future market prices) had been higher or lower by 4% and by
20%. In the Low Sensitivity condition, both amounts (associated with a 4% and 20% change in
market prices) fall far below the auditor’s materiality threshold (i.e., are less than 5% of pretax
profit). In the High Sensitivity condition, however, both amounts exceed the auditor’s materiality
threshold. Specifically, a 4% change in market prices would cause operating profit to change by
more than 1.5 times materiality, and a 20% change in market prices would cause operating profit
to change by nearly 7.5 times materiality. (See Appendix C for the QSA disclosures.)
In the Control condition, participants did not receive a Materiality Threshold disclosure or
the table containing a QSA. However, all participants received a footnote disclosure that contained
a narrative description of the sensitivity of operating profit to changes in the estimated future
market price of fish. Holding the narrative description of sensitivity constant across all conditions
allows us to examine how the additional disclosure of a QSA (in the Low and High Sensitivity
conditions) affects investors' judgments and decisions relative to a Control condition in which only
a qualitative sensitivity analysis is provided.
Dependent Variables
There are two primary dependent variables used in the study: (1) perceived reliability and
(2) willingness to invest. To measure perceived reliability, participants indicated the reliability of
the income statement account, "adjustment of live fish inventory to fair value." 19 Reliability
19
To ensure participants had a clear understanding of what financial statement reliability represents, the following
definition was provided: “Reliability is the extent to which information is measured with little uncertainty, is verifiable,
and reflects a company’s business activities in an unbiased manner” (consistent with Frederickson, Hodge and Pratt
[2006] and Clor-Proell 2009).
18
perceptions were collected on an 11-point scale with endpoints labeled “1 – Not at all Reliable”
and “11 – Very Reliable.” Perceived reliability was collected both before and after participants
viewed the footnote disclosure related to live fish inventory (containing the QSA manipulation).
We then calculated a change score by subtracting participants’ initial reliability perceptions from
those collected after participants had viewed the footnote disclosure.20 This approach allows us to
more precisely measure the effect of a QSA disclosure while taking into account ex ante
differences in reliability perceptions across participants.
Our second dependent variable, willingness to invest, was measured using two questions.
The first question asked participants, “In your opinion, how attractive is KFFG as an investment?”
Responses were collected on an 11-point scale with endpoints labeled “1 – Very Unattractive” and
“11 – Very Attractive.” The second question asked, “Assuming you had NOK 50,000 to invest in
the common stock of one or more companies, how likely are you to invest a portion of this amount
in the common stock of KFFG?” Responses to this question were collected on an 11-point scale
with endpoints labeled “1 – Very Unlikely” and “11 – Very Likely.”21
IV. RESULTS
Manipulation Checks
We asked two questions to ensure participants attended to our manipulations. First,
participants were asked whether or not the footnote disclosure had indicated “the exact NOK
amount” by which operating profit would change if the estimated future market price of fish had
20
This approach of measuring reliability perceptions both before and after the treatment is consistent with prior studies
(e.g., Frederickson et al. 2006; Clor-Proell, Proell, and Warfield 2014) and helps control for investors’ ex ante beliefs
regarding the reliability of financial statement information, as such beliefs tend to be heterogeneous (Frederickson et
al. 2006). Although participants did view our Materiality Threshold manipulation prior to making their initial
reliability judgments, we do not find differences in pre-treatment reliability perceptions between the Materiality
Threshold conditions (p = 0.664).
21
As a follow-up to the “investment likelihood” question, we also asked participants the amount they would be willing
to invest in the common stock of KFFG (from NOK 0 to NOK 50,000); however, we do not find significant differences
between experimental conditions on this measure.
19
been higher or lower by 4% and by 20%. Out of 193 participants, 150 (77.7 percent) answered this
question correctly. Participants were then asked whether the auditors had disclosed the NOK
amount considered to be material during their audit of KFFG. One hundred seventy-seven
participants (91.7 percent) answered this question correctly. Analyses below are based on the 140
participants who correctly answered both manipulation check questions.22
Test of H1
H1 predicts that investors will perceive the reliability of an uncertain estimate to be higher
when it is supplemented with a QSA compared to when only a narrative description of sensitivity
is present. Panel A of Table 1 provides descriptive statistics for the two manipulated variables
(Materiality Threshold and QSA) and the Control condition. 23 Panel C provides the specific
contrast used to test H1. Results of this analysis show that investors perceive the reliability of the
fair value adjustment account (i.e., the uncertain estimate) to be higher in the presence of a QSA
disclosure (i.e., both the Low and High Sensitivity conditions) (mean = -0.63) compared to the
absence of such a disclosure (i.e., the Control condition) (mean = -1.21) (t = 1.74, p = 0.042, onetailed).24 Therefore, H1 is supported.25
22
Nearly all of the participants who failed a manipulation check question (87 percent) were in a condition in which
the disclosure was present, but the participant failed to recall seeing it. By only including in our analyses those
participants who correctly answered our manipulation check questions, we are able to rule out the possibility that our
results are due to differences between conditions in information acquisition (Frederickson et al. 2006). Overall, results
are similar (although weaker) when manipulation check failures are included in the analyses. Specifically, tests
associated with H1 are no longer significant, while results for H2, H3, and H4 are unchanged.
23
All p-values in the paper are two-tailed unless otherwise noted.
24
We find similar results (t = 1.55, p = 0.062, one-tailed) when this analysis is limited to the Materiality ThresholdAbsent condition only.
25
We also asked participants how difficult it was to evaluate the reliability of the uncertain estimate using information
disclosed by management and the auditors (on a scale ranging from "1 – Very Difficult" to "11 – Very Easy").
Participants indicated it was significantly easier to evaluate reliability when both a QSA and materiality threshold
were disclosed (mean = 3.55) compared to when they were not (mean = 2.43) (t = 2.49; p = 0.014, untabulated).
Participants also believed that the footnote disclosure about the uncertain estimate was significantly more informative
(on a scale ranging from "1 – Not at all Informative" to "11 – Very Informative") when it contained a QSA (mean =
5.90) compared to when it did not (mean = 4.61) (t = 2.24; p = 0.026, untabulated).
20
[Insert Table 1 here]
Test of H2
H2 predicts that investors will judge the reliability of an uncertain estimate to be higher
when the item is less sensitive to changes in underlying inputs compared to more sensitive. Panel
B of Table 1 reports a 2 x 2 ANOVA model (excluding the Control condition) that includes
Materiality Threshold and QSA as the independent variables and perceived reliability as the
dependent variable. The main effect of QSA is not significant in the ANOVA model (F = 1.43, p
= 0.118, one-tailed equivalent); therefore, H2 is not supported. Failing to find support for H2
suggests that in general, investors may struggle to recognize the level of uncertainty (i.e.,
sensitivity) indicated within a QSA. In H3, we examine whether a materiality threshold disclosure
helps investors differentiate between uncertain estimates that are relatively low, compared to high,
in sensitivity.
Test of H3
H3 predicts that investors will more easily recognize the difference in reliability between
the Low and High Sensitivity conditions in the presence of a materiality threshold disclosure. That
is, we expect that when a QSA is disclosed, reliability perceptions will be highest in the Materiality
Threshold–Present/Low Sensitivity condition (i.e., when the amount of sensitivity falls below the
materiality threshold), and lowest in the Materiality Threshold–Present/High Sensitivity condition
(i.e., when the amount of sensitivity exceeds the materiality threshold).
Figure 1 graphs the results associated with the Materiality Threshold x QSA interaction
predicted in H3. The ANOVA model presented in Panel B of Table 1 includes a significant
Materiality Threshold x QSA interaction (F = 4.82, p = 0.030). Simple effects tests indicate that
the form of this interaction is in line with our expectations. Specifically, the difference between
21
the Low and High Sensitivity conditions is significant in the Materiality Threshold–Present
condition (means of -0.11 and -1.18, respectively) (t = 2.40, p = 0.009, one-tailed), but not in the
Materiality Threshold–Absent condition (means of -0.80 and -0.48, respectively) (t = -0.71, p =
0.482). Therefore, H3 is supported. It would appear that in order for investors to recognize the
difference in reliability between an uncertain estimate that contains a relatively low level of
sensitivity and one containing a relatively high level of sensitivity, a QSA alone is insufficient.
Rather, the QSA must be supplemented with a disclosure of the auditor’s materiality threshold.
[Insert Figure 1 here]
Test of H4
H4 predicts that investors' willingness to invest in a company will be influenced by the
disclosure of a materiality threshold and a QSA. Specifically, we predict that when a QSA is
disclosed, willingness to invest will be highest in the Materiality Threshold–Present/Low
Sensitivity condition and lowest in the Materiality Threshold–Present/High Sensitivity condition.
Panel A of Table 2 provides descriptive statistics for both dependent variables related to the
willingness to invest (i.e., attractiveness and investment likelihood), and Panel B presents the
results of the ANOVA models associated with each dependent variable. Although the Materiality
Threshold x QSA interaction term is significant in the ANOVA model for investment likelihood
(F = 6.06, p = 0.015) and marginally significant for attractiveness (F = 2.89 p = 0.092), the
interaction is in the direction opposite of our prediction.
[Insert Table 2 here]
Simple effects indicate that when the auditor’s materiality threshold is disclosed, investors
perceive the company as more attractive when the QSA indicates High Sensitivity (mean = 6.89)
compared to when it indicates Low Sensitivity (mean = 6.18) (t = 1.75, p = 0.083, untabulated).
22
Similarly, investors in the Materiality Threshold–Present condition indicate that they are more
likely to invest in a company that discloses a QSA indicative of High Sensitivity (mean = 6.57)
compared to Low Sensitivity (mean = 5.29) (t = 2.43, p = 0.017, untabulated). The difference
between Low and High Sensitivity is not significant in the Materiality Threshold–Absent condition
(p > 0.200 for both attractiveness and investment likelihood, untabulated). These results are not in
line with our expectations. Therefore, H4 is not supported.
Supplemental Analysis
The results associated with H4 are surprising, given that participants perceive the fair value
adjustment account as significantly less reliable in the High Sensitivity condition compared to the
Low Sensitivity condition (when materiality is disclosed). Nonetheless, these reduced reliability
perceptions in the High Sensitivity condition do not discourage investors from investing in the
company; in fact, investors indicate a greater willingness to invest. To better understand this
unusual finding, we first examine its robustness to the removal of influential outliers. Using Cook's
Distance, we identify four participants who indicated an extreme value for the investment
attractiveness measure and seven participants who indicated an extreme value for investment
likelihood.26 When these outliers are removed from our sample, we find no significant differences
in participants' willingness to invest across experimental conditions.27 Therefore, it appears that
the results obtained in relation to H4, which were in the direction opposite of what we predicted,
are largely attributable to a small number of influential outliers. Nonetheless, we still fail to find
support for H4. As such, our next analysis examines the intentionality of participants’ investment-
26
Cook's Distance identifies influential data points by estimating the effect of removing a given observation from the
analysis. We considered values greater than the common threshold of 4/n to be influential outliers. The same
observations were identified as outliers when using studentized residuals greater than +/- 2.
27
Excluding these participants does not change our results related to perceived reliability. Specifically, results
associated with H1, H2, and H3 are unchanged when these outliers are excluded from our analyses.
23
related judgments. That is, we examine whether our finding that investors do not prefer to invest
in a company with lower, compared to higher, estimation uncertainty is conscious (intentional) or
results from unintentional bias.
In our experiment, the level of sensitivity (Low or High) was manipulated between-subjects.
In our post-experimental questionnaire, we included a within-subjects manipulation of
sensitivity.28 According to Kahneman and Tversky (1996), a between-subjects manipulation tests
a participant’s natural reasoning process, whereas a within-subjects manipulation makes the
independent variables more salient to participants, thus allowing them to identify and correct any
judgment biases or errors in reasoning that may exist. If we obtain different results related to H4
when sensitivity is manipulated within-subjects (compared to between-subjects), this would
suggest that the between-subjects results are unintentional (Kahneman and Tversky 1996; Libby,
Bloomfield and Nelson 2002).
In a real world investment context, investors rarely evaluate a company in isolation (as
participants did in our between-subjects manipulation). Rather, investors typically compare several
potential investments on a number of criteria to select the preferred investment. To simulate this
real-world investment context, in our post-experimental questionnaire, participants were told to
consider two firms (Firm A and Firm B) that are identical, except for the sensitivity of a fair value
estimate that appears on each firm’s income statement. Participants were then presented with a
QSA disclosure for each firm, where Firm A’s QSA was identical to the Low Sensitivity
manipulation from our original experiment, and Firm B’s QSA was identical to the High Sensitivity
manipulation. Thus, participants viewed both levels of our QSA manipulation simultaneously, and
28
The within-subjects manipulation was added to the very end of our experimental instrument during our second
round of data collection. A total of 74 participants viewed and responded to the within-subjects manipulation and are
included in the analyses reported in this section.
24
could decide which company they preferred. 29 Appendix D provides the full within-subjects
manipulation. We then asked participants the following two questions regarding their willingness
to invest in Firm A (Low Sensitivity disclosure) or Firm B (High Sensitivity disclosure): (1) “In
your opinion, which firm is a more attractive investment?” and (2) “Assuming you had NOK
50,000 to invest in the common stock of Firm A or Firm B, which firm are you more likely to
invest in?” We find that for both questions, a significantly greater proportion of participants
preferred Firm A (75.7 percent) compared to Firm B (13.5 percent) (χ2 = 56.57; p < 0.001), with
10.8 percent indicating no preference.
The results obtained on a within-subjects basis (i.e., a greater willingness to invest in a
company with a QSA indicative of low sensitivity compared to high sensitivity, as predicted in
H4) are inconsistent with results obtained on a between-subjects basis. This suggests that the
between-subjects results obtained in our original experiment were unintentional. Specifically,
participants are aware that companies with financial statements containing a lower level of
estimation uncertainty are preferable for investment purposes; however, they are not able to access
this information in their natural reasoning process (i.e., when evaluating only one company in
isolation) (Kahneman and Tversky 1996; Libby et al. 2002). 30 Given that most investment
decisions are not made in isolation, we believe our within-subjects experiment provides important
insights regarding how QSA and materiality disclosures influence investment decisions by
29
We also provided participants with the auditor’s quantitative materiality threshold for both companies, which was
the same amount provided in the Materiality Threshold-Present condition in our main experiment. We chose to
provide a materiality threshold because H4 predicts that investors’ willingness to invest will be highest (lowest) in
Low (High) Sensitivity condition, when materiality is also disclosed.
30
As part of our within-subjects manipulation, we also asked participants, “Which firm has a more reliable fair value
estimate?” We find that a significantly greater proportion of participants (χ2 = 110.42; p < 0.001) perceive Firm A’s
estimate as more reliable than Firm B’s (67.6 percent and 2.7 percent, respectively), which is consistent with our
findings on a between-subjects basis and suggests that the between-subjects results related to perceived reliability are
consistent with participants’ intentions.
25
allowing investors to more easily compare the level of estimation uncertainty faced by different
companies.
V. DISCUSSION AND CONCLUSION
Over time, financial statements have begun to incorporate a greater amount of estimation
uncertainty. As a result, investors often face the difficult task of making judgments and decisions
based on inherently imprecise information. Although standard setters and regulators worldwide
have considered mandating a number of disclosures to help users understand the uncertainty in
audited financial statements, one concern is that individuals may still fail to understand and/or use
this information once it is disclosed (Clor-Proell et al. 2014). To address this concern, we
conducted a study that examined whether nonprofessional investors recognize differences in
estimation uncertainty more easily in the presence of disclosures that help to quantify this
uncertainty.
We found that investors generally perceive an estimate as more reliable when it is
supplemented with a sensitivity analysis that is quantitative in nature, as opposed to qualitative (as
was the case in our Control condition). At present, companies are only required to disclose QSAs
for certain fair value estimates (e.g., financial instruments), but not for others (e.g., inventory). Our
results suggest that a QSA may make the task of evaluating highly subjective fair value estimates
less difficult for investors by explicitly quantifying the level of uncertainty contained within them.
Most importantly, when a QSA suggested that an estimate reported on the face of the
financial statements contained extreme estimation uncertainty (i.e., high sensitivity), investors
assessed the estimate’s reliability to be lower, compared to investors who viewed a QSA indicative
of low sensitivity, but only when the auditor’s materiality threshold was also disclosed. In the
absence of a materiality disclosure, investors’ reliability perceptions did not differ regardless of
26
the level of uncertainty indicated within the QSA. This finding is particularly interesting given the
fact that only one regulatory body, the U.K.’s FRC, requires the public disclosure of audit
materiality. Throughout the rest of the world, regulators and standard setters have considered, but
ultimately decided against, requiring auditors to disclose their materiality thresholds. Nonetheless,
our results suggest that for nonprofessional investors to recognize differences in estimation
uncertainty disclosed in sensitivity analyses, not only must this disclosure be quantitative in nature,
but also supplemented with a disclosure of materiality.
Finally, counter to our expectations, investors' willingness to invest was not deterred when
the presence of extreme estimation uncertainty was disclosed most transparently (i.e., in the
presence of a materiality threshold disclosure and a QSA indicative of high, as opposed to low,
sensitivity). Even though investors perceived reliability to be significantly lower in this condition,
relative to the low sensitivity condition, these perceptions did not appear to “flow through” and
influence investment decisions. However, when differences in estimation uncertainty were made
more salient with a within-subjects manipulation, we found results that were more consistent with
our expectations, in that participants indicated a greater preference to invest in a company with a
QSA indicative of low, compared to high, sensitivity.
Our finding that investment decisions were not responsive to the level of estimation
uncertainty contained within financial statements is consistent with findings in the archival
literature. Specifically, Richardson et al. (2005) find that security prices are not sufficiently
reduced when financial statements contain highly uncertain estimates, even though such estimates
are associated with lower earnings persistence. Our results suggest that this finding is likely driven
in part by investors’ failure to recognize when estimates are indicative of low reliability. However,
even when we provided participants with additional disclosures that helped them recognize the
27
presence of a low reliability estimate, their willingness to invest was not deterred. It was not until
differences in estimation uncertainty were made more salient using a within-subjects design that
participants' investment decisions reflected a strong preference for financial statements containing
lower levels of estimation uncertainty.
Although our study provides important insights, it is also subject to certain limitations.
First, participants in our study were provided with much less information than they might normally
consider when making investment decisions. Although this approach may limit the generalizability
of our results, it helps to ensure that participants acquire the pieces of information that are most
important to our study, thus allowing us to examine how this information is used once it has been
acquired (Clor-Proell et al. 2014). Second, we examined only two types of uncertainty disclosures
in our study. It is possible that other types of disclosures may also help investors recognize
differences in estimation uncertainty. This is a question for future research to explore.
Finally, we tested our hypotheses in an environment in which an uncertain fair value
estimate was highlighted in a CAM disclosure. Although the inclusion of CAMs within the audit
report will soon be the norm in much of the world, it is uncertain whether our results would hold
in the absence of a CAM disclosure that highlights the uncertain estimate. Interestingly, the revised
PCAOB (2016) proposal related to CAMs includes the requirement that CAMs must be material.
Although the fair value adjustment account in our study was highly material in all instances, the
level of imprecision underlying the estimate was only material in the High Sensitivity condition.
Our results suggest that investors are unlikely to differentiate between material and immaterial
CAM items unless materiality is also disclosed. As one member of the PCAOB Standing Advisory
Group (2016, 37) commented, “Once you make [materiality] a gating issue, I think we deserve to
know how wide the gate is.”
28
APPENDIX A: Quantitative versus Qualitative (Narrative) Sensitivity Analyses
Example of a Quantitative Sensitivity Analysis (QSA)
The following example was taken from Starbuck's 10-K for the year ended October 2, 2016:
Example of a Qualitative (or Narrative) Sensitivity Analysis
The following example was taken from Best Buy's 10-K for the year ended January 30, 2016:
Property and Equipment Impairments
Our impairment loss calculations require management to make assumptions and to apply judgment in
order to estimate fair values, including estimating cash flows and useful lives and selecting a discount
rate that reflects the risk inherent in future cash flows. If actual results are not consistent with our
estimates and assumptions we may be exposed to impairments that could be material.
29
APPENDIX B: Auditor's Report Containing the Materiality Threshold Manipulation
Below is the auditor's report provided to all participants. The sentence surrounded by brackets
represents the Materiality Threshold manipulation, which was only present in the Materiality
Threshold–Present condition.
IMPORTANT EXCERPTS FROM THE INDEPENDENT AUDITOR’S REPORT
Audit Opinion
We have audited the financial statements of King Fish Farming Group (the Group). In our opinion, the
accompanying financial statements present fairly, in all material respects, the financial position of the
Group at December 31, 20X2, and its financial performance and its cash flows for the year then ended in
accordance with International Financial Reporting Standards (IFRSs) as adopted by the EU.
Key Audit Matters
Key audit matters are those matters that, in our professional judgment, were of most significance in our
audit of the consolidated financial statements of the current period. One of the key audit matters in our
audit of the Group relates to the estimation of the fair value of live fish inventory, as described below.
Adjustment of Live Fish Inventory to Fair Value
The Group values its inventory of live fish at fair value less costs to sell. Active markets for the future
sale of live fish do not exist. Therefore, the Group estimates the fair value of live fish using a model that
reflects the Group’s own assumptions and projections.
The valuation of live fish and measurement of profitability critically depend on the current estimate of
salmon and trout prices at the expected time of sale. Historically, salmon and trout prices have been
subject to large fluctuations. The uncertain nature of market prices does not allow future sales prices to be
estimated with certainty.
The fair value estimate of live fish is considered to be one of the most judgmental audit risks, as small
changes in the assumptions used in the valuation (e.g., future sales prices) could have a significant impact
on the Group’s profitability. See note 1 to the financial statements for the disclosure of the accounting
policies and management’s estimation process related to the fair value of live fish.
Our Application of Materiality
We define materiality as the magnitude of misstatement in the financial statements that makes it probable
that the economic decisions of a reasonably knowledgeable person would be changed or influenced. We
use materiality both in planning the scope of our audit work and in evaluating the results of our work.
We determined materiality for the Group to be NOK 44 million, which is approximately 5% of profit
before tax.
Bergen, 20 March 20X3
Big 4 accounting firm
Knut Johansen
State Authorized Public Accountant (Norway)
30
APPENDIX C: Footnote Disclosure Containing the QSA Manipulation
Below is the footnote disclosure related to live fish inventory. All participants received this
disclosure. However, only participants in the Low Sensitivity and High Sensitivity conditions
(i.e., those not in the Control condition) received the final section of the disclosure, titled "The
sensitivity of operating profit to changes in estimated future fish prices," which contained the
QSA manipulation.
NOTE 1. Live fish inventory (Extract)
The following table provides a reconciliation of the beginning and ending balances for live fish inventory.
(All figures in NOK 1 000)
Beginning balance of live fish inventory at 01.01.20X2
Increase due to inventory costs added during the year
Reduction due to sale of fish inventory
Adjustment of live fish inventory to fair value (profit impact)
Ending balance of live fish inventory at 31.12.20X2
3 082 219
4 860 324
-4 783 276
383 156
3 542 423
The Company recognizes live fish inventory at fair value less costs to sell. Active markets for the future
sale of live fish do not exist, so the calculation of fair value is carried out using a valuation model (level 3
in the valuation hierarchy) where the relevant inputs (e.g., future market prices) are primarily
unobservable and uncertain.
In 20X2, the Company recorded a gain of NOK 383 million to adjust live fish inventory to fair value, which
directly impacts operating profit reported for the period. The amount of this adjustment critically depends
on the estimate of future salmon and trout prices at the expected time of sale. Because salmon and trout
prices are subject to reasonably large fluctuations, even minor changes in the estimated market price of fish
used in our valuation model may result in significant changes in operating profit reported for the period.
The sensitivity of operating profit to changes in estimated future fish prices
The table below shows the amount by which operating profit for 20X2 would change if the estimated
future market price of fish used in our valuation model had been higher or lower by 4% and by 20%.
Low Sensitivity Manipulation:
Change in operating profit (in NOK 1
000)
4% price change
20% price change
+/- 4 380
+/- 21 900
4% price change
20% price change
+/- 71 936
+/- 328 549
High Sensitivity Manipulation:
Change in operating profit (in NOK 1
000)
31
APPENDIX D: Within-Subjects Manipulation of Low Sensitivity versus High Sensitivity
Consider two firms (Firm A and Firm B) that are identical, except for the sensitivity of a fair
value estimate that appears on each firm's income statement. Below is a sensitivity analysis for
each firm that shows the amount by which operating profit would change if an important input
used in the company's fair value model had been higher or lower by 4% and by 20%.
Firm A's Sensitivity Analysis (in NOK 1 000):
4% change in a fair
value model input
20% change in a fair
value model input
+/- 4 380
+/- 21 900
4% change in a fair
value model input
20% change in a fair
value model input
+/- 71 936
+/- 328 549
Change in operating profit (in NOK 1
000)
Firm B's Sensitivity Analysis (in NOK 1 000):
Change in operating profit (in NOK 1
000)
Assuming that the auditors for both Firm A and Firm B determined materiality to be NOK 44
million (reflecting 5% of profit before tax), please answer the following questions:
In your opinion, which firm is a more attractive investment?
□ Firm A is more attractive.
□ Firm B is more attractive.
□ Firm A and Firm B are equally attractive.
Assuming you had NOK 50,000 to invest in the common stock of Firm A or Firm B, which firm
are you more likely to invest in?
□ I am more likely to invest in Firm A.
□ I am more likely to invest in Firm B.
□ I am equally likely to invest in Firm A or in Firm B.
32
REFERENCES
Bell, T. B., and J. B. Griffin. 2012. Commentary on auditing high-uncertainty fair value estimates.
Auditing: A Journal of Practice & Theory 31 (1): 147-155.
Botosan, C.A. 1997. Disclosure level and the cost of equity capital. The Accounting Review
Vol. 72 (No. 3): 323-49.
Cannon, N., and J. C. Bedard. 2016. Auditing challenging fair value measurements: Evidence from
the field. The Accounting Review forthcoming.
Christensen, B. E., S. M. Glover, and D. A. Wood. 2012. Extreme estimation uncertainty in fair
value estimates: Implications for audit assurance. Auditing: A Journal of Practice &
Theory 31 (1): 127 – 146.
Christensen, B. E., S. M. Glover, and C. J. Wolfe. 2014. Do critical audit matter paragraphs in the
audit report change nonprofessional investors' decision to invest? Auditing: A Journal of
Practice & Theory 33 (4): 71–93.
Clor-Proell, S. M. 2009. The Effects of Expected and Actual Accounting Choices on
Judgments and Decisions. The Accounting Review 84 (5): 1465–1493.
Clor-Proell, S. M., C. A. Proell, and T. D. Warfield. 2014. The effects of presentation salience and
measurement subjectivity on nonprofessional investors' fair value judgments.
Contemporary Accounting Research 31 (1): 45-66.
Coram, P., T. Mock, J. Turner, and G. Gray. 2011. The communicative value of the auditor’s
report. Australian Accounting Review 21 (3): 235–252.
Du, N., D. V. Budescu, M. K. Shelly, and T. C. Omer. 2011. The appeal of vague financial
forecasts. Organizational Behavior and Human Decision Processes 114 (2): 179-189.
Eilifsen, A. and W. F. Messier, Jr. 2015. Materiality guidance of the major public accounting firms.
Auditing: A Journal of Practice & Theory 34 (2): 3-26.
Elliott, W. B., F. D. Hodge, J. J. Kennedy, and M. Pronk. 2007. Are M.B.A. students a good proxy
for nonprofessional investors? The Accounting Review 82 (1): 139-168.
Elliott, W. B., K. E. Jackson, and S. D. Smith. 2008. The influence of sensitivity disclosures on
investor judgments. Working paper, University of Illinois at Urbana-Champaign.
European Commission (EC). 2014. Regulation (EU) No 537/2014 of the European Parliament and
of the Council of 16 April 2014 on Specific Requirements Regarding Statutory Audit of
Public-Interest Entities and Repealing Commission Decision 2005/909/EC. Official
Journal of the European Union. Strasbourg, France: EC.
33
Financial Accounting Standards Board (FASB). 2010a. Fair Value Measurements and Disclosures
(Topic 820), Improving Disclosures about Fair Value Measurements. Norwalk, CT: FASB.
Financial Accounting Standards Board (FASB). 2010b. Conceptual Framework for Financial
Reporting. Chapter 3, Qualitative Characteristics of Useful Financial Information.
Statement of Financial Accounting Concepts No. 8. Norwalk, CT: FASB.
Financial Accounting Standards Board (FASB). 2011. Fair Value Measurement (Topic 820),
Amendments to Achieve Common Fair Value Measurement and Disclosure Requirements
in U.S. GAAP and IFRS. Accounting Standards Update No. 2011-4. Norwalk, CT: FASB.
Financial Reporting Council (FRC). 2013. Consultation Paper: Revision to ISA (UK and Ireland)
700 - Requiring the Auditor's Report to Address Risks of Material Misstatement,
Materiality and a Summary of the Audit Scope. London, England: Financial Reporting
Council.
Fisher, M. H. 1990. The effects of reporting auditor materiality levels publicly,
privately, or not at all in an experimental setting. Auditing: A Journal of Practice &
Theory 9 (Supplement): 184-223.
Frederickson, J. R., F. D. Hodge, and J. H. Pratt. 2006. The evolution of stock option accounting:
Disclosure, voluntary recognition, mandated recognition, and management disavowals.
The Accounting Review 81 (5): 1073-1093.
Glover, S. M., M. H. Taylor, and Y. Wu. 2017. Current practices and challenges in auditing fair
value measurements and complex estimates: Implications for auditing standards and the
academy. Auditing: A Journal of Practice & Theory 36 (1): 63-84.
Gray, G. L., J. L. Turner, P. J. Coram, and T. J. Mock. 2011. Perceptions and misperceptions
regarding the unqualified auditor’s report by financial statement preparers, users, and
auditors. Accounting Horizons 25 (4): 659–684.
Holstrum, G. L., and W. F. Messier 1982. A Review and Integration of Empirical
Research on Materiality, Auditing: A Journal of Practice & Theory 9 (Supplement):
184-223.
Houghton, K. A., C. Jubb, and M. Kend. 2011. Materiality in the context of audit: The real
expectations gap. Managerial Auditing Journal 26 (6): 482–500.
International Accounting Standards Board (IASB). 2010. Exposure Draft: Measurement
Uncertainty Analysis Disclosure for Fair Value Measurements (Limited Re-Exposure of
Proposed Disclosure). London, England: International Accounting Standards Committee
Foundation.
34
International Accounting Standards Board (IASB). 2011. Fair Value Measurement. International
Financial Reporting Standard (IFRS) 13. London, UK: IASB.
International Auditing and Assurance Standards Board (IAASB). 2015. International Standard on
Auditing 701: Communicating Key Audit Matters in the Independent Auditor’s Report.
New York, NY: International Federation of Accountants.
Kachelmeier, S. J., J. J. Schmidt, and K. Valentine. 2017. The disclaimer effect of disclosing
critical audit matters in the auditor’s report. Working Paper, University of Texas at Austin.
Kadous, K., L. Koonce and K. L. Towry. 2010. Quantification and persuasion in managerial
judgment. Contemporary Accounting Research 22 (3): 643-686.
Kahneman, D., and A. Tversky. 1996. On the reality of cognitive illusions. Psychological Review
103 (3): 582-588.
Koonce, L., M. G. Lipe, and M. L. McAnally. 2005. Judging the risk of financial instruments:
Problems and potential remedies. The Accounting Review 80 (3): 871-895.
Koonce, L., M. L. McAnally, and M. Mercer. 2005. How do investors judge the risk of financial
items? The Accounting Review 80 (1): 221-241.
Lev, B., S. Li, and T. Sougiannis. 2010. The usefulness of accounting estimates for predicting cash
flows and earnings. Review of Accounting Studies 15 (4): 779-807.
Libby, R., R. Bloomfield, and M. W. Nelson. 2002. Experimental research in financial accounting.
Accounting, Organizations and Society 27 (8): 775-810.
Linsmeier, T. J., and N. D. Pearson. 1997. Quantitative disclosures of market risk in the SEC
release. Accounting Horizons 11 (1): 107-135.
Medin, D. L., R. L. Goldstone, and D. Gentner. 1993. Respects for similarity. Psychological
Review 100 (2): 254-278.
Medin, D. L., and M. M. Schaffer. 1978. Context theory of classification learning. Psychological
Review 85: 207-238.
Messier Jr., W. F., N. Martinov-Bennie, and A. Eilifsen 2005. A review and integration
of empirical research on materiality: two decades later. Auditing: A Journal of
Practice & Theory 24 (2): 153-187.
Nelson, M. W., and K. K. Rupar. 2015. Numerical formats within risk disclosures and the
moderating effect of investors' concerns about management discretion. The Accounting
Review 90 (3): 1149-1168.
Public Company Accounting Oversight Board (PCAOB). 2003. Auditing Accounting Estimates.
PCAOB Interim Auditing Standards AU Section 342. Washington, D.C.: PCAOB.
35
Public Company Accounting Oversight Board (PCAOB). 2010. Consideration of Materiality in
Planning and Performing the Audit. Auditing Standard No. 11. Washington, D.C.:
PCAOB.
Public Company Accounting Oversight Board (PCAOB). 2011. PCAOB Release No. 2011-003:
Concept Release on Possible Revisions to PCAOB Standards Related to Reports on Audited
Financial Statements. Washington, D.C.: PCAOB.
Public Company Accounting Oversight Board (PCAOB). 2012. Auditing the Future. June 7.
Washington,
DC:
PCAOB.
Available
http://pcaobus.org/News/Speech/Pages/06072012_HansonAICPA.aspx.
at:
Public Company Accounting Oversight Board (PCAOB). 2013. PCAOB Release No. 2013-005:
Proposed Auditing Standards - The Auditor's Report on an Audit of Financial Statements
When the Auditor Expresses an Unqualified Opinion. Washington, D.C.: PCAOB.
Public Company Accounting Oversight Board (PCAOB). 2016. PCAOB Release No. 2016-003:
Proposed Auditing Standards - The Auditor's Report on an Audit of Financial Statements
When the Auditor Expresses an Unqualified Opinion. Washington, D.C.: PCAOB.
Public Company Accounting Oversight Board (PCAOB) Investor Advisory Group. 2011. PCAOB
Investor Advisory Group Meeting: March 16, 2011. Washington, D.C. Available at:
https://pcaobus.org//News/Events/Documents/03162011_IAGMeeting/Role_Of_The_Au
ditor.pdf
Public Company Accounting Oversight Board (PCAOB) Standing Advisory Group. 2016.
PCAOB Standing Advisory Group Meeting: May 18, 2016. Washington, D.C. Available
at: https://pcaobus.org/Rulemaking/Docket034/SAG-Transcript-ARM-proposal-05-1816.pdf
Richardson, S. A., R. G. Sloan, M. T. Soliman, and I. Tuna. 2005. Accrual reliability, earnings
persistence and stock prices. Journal of Accounting and Economics 39 (3): 437-485.
Singh, M., and S. J. Peters. 2015. Materiality: Investor Perspectives. Charlottesville, VA: CFA
Institute.
Sirois, L., J. Bédard, and P. Bera. 2016. The informational value of key audit matters in the
auditor’s report: Evidence from an eye-tracking study. Working Paper, HEC Montreal.
36
FIGURE 1
Results of H3:
Interaction of Materiality Threshold and QSA on Perceived Reliability
2
PerceivedReliability
1.5
1
0.5
QSA:
LowSensitivity
0
HighSensitivity
-0.5
-1
-1.5
-2
Absent
MaterialityThreshold
Present
____________________________________________________________
Figure 1 depicts the results of the test of H3. The graph shows the interaction of Materiality
Threshold (Absent, Present) and QSA (Low Sensitivity, High Sensitivity) on investors’ reliability
perceptions. Perceived reliability was measured by asking participants to indicate the reliability of
an uncertain estimate reported on the income statement using an 11-point scale with endpoints
labeled “1 – Not at all Reliable” and “11 – Very Reliable.” This question was asked both before
and after participants received the QSA manipulation, and a change score was then calculated by
subtracting pre-treatment reliability perceptions from post-treatment reliability perceptions.
______________________________________________________________________________
37
TABLE 1
Effect of Materiality Threshold and QSA Disclosures on the
Perceived Reliability of an Uncertain Estimate
Panel A: Mean (Standard Deviation) of Perceived Reliabilitya
Materiality Threshold - Absent
No Sensitivity
Low
High
("Control")
Sensitivity
Sensitivity
(n = 28)
( n = 25)
( n = 31)
-1.21
-0.80
-0.48
(1.23)
(1.85)
(1.84)
Materiality Threshold – Present
Low
High
Sensitivity
Sensitivity
( n = 28)
( n = 28)
-0.11
-1.18
(1.42)
(1.52)
Panel B: ANOVA Table – Perceived Reliabilityb
Source
Materiality
QSA
Materiality*QSA
Error
df
1
1
1
108
SS
0.00
3.97
13.40
300.53
F
0.00
1.43
4.82
p-value
0.998
0.235
0.030
Panel C: Planned Contrast and Simple Effects
Comparison
Control < Low & High Sensitivity (H1)
Materiality-Present: Low > High Sensitivity
Materiality-Absent: Low = High Sensitivity
df
137
108
108
t
1.74
2.40
0.71
p-value
0.042*
0.009*
0.482
_______________________________________
Table 1 reports descriptive statistics (Panel A) and results of an ANOVA with perceived reliability
as the dependent variable and Materiality Threshold ("Materiality") and QSA (i.e., quantitative
sensitivity analysis) as independent variables (Panel B). Panel C provides planned contrasts and
simple effects. P-values are two-tailed unless otherwise noted.
a
Perceived reliability was measured by asking participants their perceptions regarding the
reliability of an uncertain estimate using an 11-point scale with endpoints labeled “1 – Not at all
Reliable” and “11 – Very Reliable.” This measure was elicited before and after participants
received the QSA manipulation. A change score was then calculated.
b
The "Control" condition was dropped for purposes of the 2x2 ANOVA model.
*One-tailed test.
_____________________________________________________________________________
38
TABLE 2
Effect of Materiality Threshold and QSA Disclosures on Willingness to Invest
Panel A: Mean (Standard Deviation) of Attractiveness and Investment Likelihooda
Materiality Threshold - Absent
No
Low
High
Sensitivity
Sensitivity Sensitivity
("Control")
( n = 25)
( n = 31)
(n = 28)
Attractiveness
6.82
6.40
6.13
(1.81)
(1.08)
(1.59)
Investment
Likelihood
6.07
(2.39)
6.08
(1.38)
Materiality Threshold - Present
Low
High
Sensitivity
Sensitivity
( n = 28)
( n = 28)
5.52
(1.96)
6.18
(1.91)
6.89
(1.37)
5.29
(2.48)
6.57
(1.89)
Panel B: ANOVA Tables – Attractiveness and Investment Likelihoodb
Dependent Variable
Attractiveness
Source
Materiality
QSA
Materiality*QSA
Error
df
1
1
1
108
SS
2.05
1.37
6.76
252.27
F
0.88
0.59
2.89
p-value
0.351
0.446
0.092
Investment
Likelihood
Materiality
QSA
Materiality*QSA
Error
1
1
1
108
0.47
3.63
23.81
424.15
0.12
0.92
6.06
0.729
0.339
0.015
_______________________________________
Table 2 reports descriptive statistics (Panel A) and results of ANOVAs with willingness to invest
(i.e., attractiveness and investment likelihood) as the dependent variable and Materiality Threshold
("Materiality") and QSA (i.e., quantitative sensitivity analysis) as independent variables (Panel B).
P-values are two-tailed.
a
Attractiveness of the company as an investment was measured on an 11-point scale with
endpoints labeled “1 – Very Unattractive” and “11 – Very Attractive.” Investment likelihood was
measured on an 11-point scale with endpoints labeled “1 – Very Unlikely” and “11 – Very Likely.”
b
The "Control" condition was dropped for purposes of the 2x2 ANOVA model.
_____________________________________________________________________________
39