Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
The Importance of Quantifying Uncertainty: Examining the Effect of Audit Materiality and Sensitivity Analysis Disclosures on Investors’ Judgments and Decisions Aasmund Eilifsen NHH Norwegian School of Economics [email protected] Erin L. Hamilton University of Nevada, Las Vegas [email protected] William F. Messier, Jr. University of Nevada, Las Vegas and NHH Norwegian School of Economics [email protected] Preliminary Version: Do not quote without authors’ permission. March 2017 We gratefully acknowledge feedback received from Stephen Perreault, Aaron Saiewitz, Jason Smith, and workshop participants at the 2017 AAA Auditing Section Midyear Meeting and the University of Nevada, Las Vegas. Professor Messier received financial support for this research from the Kenneth and Tracy Knauss Endowed Chair in Accounting at UNLV and his Adjunct Professor position at the Norwegian School of Economics. The Importance of Quantifying Uncertainty: Examining the Effect of Audit Materiality and Sensitivity Analysis Disclosures on Investors’ Judgments and Decisions ABSTRACT At present, accounts containing extreme levels of estimation uncertainty may be obscured from investors given that they are reported using precise point estimates within financial statements that receive an unqualified opinion from the auditors. The way in which investors perceive these highly subjective estimates may depend on how transparently the level of uncertainty underlying the estimate is disclosed. Our study examines two disclosures that should help quantify the amount of uncertainty contained within a highly subjective fair value estimate: a quantitative sensitivity analysis and the auditor’s quantitative materiality threshold. Using an experiment with nonprofessional investors, we predict and find that participants who receive both a quantitative sensitivity analysis and a materiality disclosure can differentiate between an estimate with relatively low estimation uncertainty (i.e., low sensitivity) and one containing extreme estimation uncertainty (i.e., high sensitivity). When materiality is not disclosed, however, investors fail to recognize differences in reliability between the two levels of sensitivity, even though the amount of imprecision in the low sensitivity condition represents a fraction of materiality, while in the high sensitivity condition, this amount exceeds audit materiality multiple times over. Keywords: Audit reporting, disclosures, estimates, estimation uncertainty, materiality, sensitivity analysis. Data Availability: The data is available from the authors. I. INTRODUCTION The extreme estimation uncertainty inherent in certain financial statement accounts has become a substantial concern in recent years for auditors and standard setters alike (Christensen, Glover and Wood 2012; PCAOB 2012; Glover, Taylor and Wu 2017).1 This estimation uncertainty (or imprecision) in reported numbers has resulted from the increasing complexity of business transactions and use of fair value measurements that often rely on management-derived models that are extremely sensitive to small changes in subjectively-chosen inputs (e.g., Level 3 fair value inputs) (Christensen et al. 2012; PCAOB 2012). In fact, some numbers recognized on the face of the financial statements contain a level of estimation uncertainty that exceeds audit (planning) materiality – sometimes by multiples of materiality (Christensen et al. 2012; Glover et al. 2017).2 At present, it is unclear whether investors recognize when financial statements contain extreme levels of estimation uncertainty, given that all estimates, regardless of their imprecision, appear as precise point estimates on the face of the financials. Understanding the amount of uncertainty inherent in reported estimates is important, as prior archival research finds that less reliable estimates lead to lower earnings persistence, and a failure on the part of investors to anticipate this effect results in security mispricing (Richardson, Sloan, Soliman and Tuna 2005). Accordingly, the purpose of our study is to examine whether investors are better able to evaluate the reliability of uncertain estimates when this uncertainty is quantified. Specifically, we examine 1 Similar to Bell and Griffin's (2012, 148) description of a high-uncertainty estimate, we use the term extreme estimation uncertainty to describe an estimate "for which it is reasonably possible that the level of inherent (i.e., irreducible) measurement uncertainty could be closely approaching or exceed the financial statement (and auditor's) materiality threshold." Estimation uncertainty results from multiple sources, including the inability to precisely measure one or more of the inputs into an estimation model at the date of the financial statements (i.e., unobservable inputs) and historical volatility in model inputs (Christensen et al. 2012). 2 Planning materiality is an amount that the auditor establishes for the financial statements as a whole. This amount is used to plan the nature, timing, and extent of audit procedures. 1 two disclosures that should help quantify the amount of uncertainty (or imprecision) in a highly subjective fair value estimate: (1) a quantitative sensitivity analysis (QSA) and (2) the auditor’s quantitative materiality threshold. Over the past few years, regulators and standard setters worldwide have proposed or adopted various accounting and auditing standards intended to enhance financial statement users’ understanding of complex and subjective aspects of financial reporting (e.g., IASB 2010, 2011; FASB 2010a, 2011; FRC 2013; IAASB 2015; EC 2014; PCAOB 2013, 2016). The intent of such increased disclosure is to improve users’ ability to make well-informed decisions based on inherently uncertain financial statement information. One such disclosure is a sensitivity analysis, which provides information about how a given estimate would vary if one or more of its subjectively-chosen inputs were to change (IASB 2010). Sensitivity analyses can be either quantitative or qualitative in nature (see Appendix A for an example of each). In 2010, the Financial Accounting Standards Board (FASB 2010a) and International Accounting Standards Board (IASB 2010) considered requiring a QSA for all Level 3 fair value measurements. In the end, however, both the FASB and IASB decided to defer the decision on whether to require a quantitative disclosure, and instead put into place a requirement that management provide a narrative (i.e., qualitative) description of sensitivity (FASB 2011). Nonetheless, we believe a QSA is more informative than a narrative disclosure, as it explicitly quantifies the amount of uncertainty contained within a fair value estimate and should help investors differentiate between financial statement items that are relatively reliable compared to relatively unreliable. It is possible, however, that a QSA alone may not provide enough information to help investors meaningfully evaluate the reliability of a subjective estimate. Although a QSA quantifies the amount of uncertainty in an estimate, investors still must evaluate the significance of this 2 imprecision to the financial statements as a whole and to their own judgments and decisions. We predict that the disclosure of a quantitative materiality threshold within the auditor’s report will help investors interpret a QSA. A materiality threshold indicates the amount of imprecision or inaccuracy (i.e., the level of misstatement) that auditors are willing to tolerate within the financial statements. In this way, a materiality threshold may serve as a useful benchmark for evaluating the severity of imprecision communicated within a QSA. Accordingly, we predict that in the absence of a materiality disclosure, investors may struggle to differentiate between a relatively precise (i.e., low sensitivity) fair value estimate and one that contains an extreme level of estimation uncertainty (i.e., high sensitivity). When a materiality threshold is disclosed; however, we expect investors will recognize when a reported estimate contains a level of imprecision that exceeds audit materiality and will evaluate the item’s reliability to be significantly lower compared to a more precise estimate (i.e., one in which the range of uncertainty falls far below the auditor’s materiality threshold). Furthermore, when both disclosures are absent and investors are only provided with a narrative description of sensitivity, the inability to quantify the uncertainty contained in a highly subjective estimate is expected to result in the lowest perceptions of financial statement reliability. We test our predictions experimentally using a 2 × 2 + 1 between-subjects design in which we manipulate the range of uncertainty indicated by a QSA (low sensitivity or high sensitivity) and the presence or absence of a materiality threshold disclosure. We also include a control condition in which participants only receive a narrative description of sensitivity, while a materiality threshold and QSA are absent.3 Participants receive an audit report that describes a 3 The narrative description of sensitivity is held constant across all conditions and simply informs participants that “even minor changes in (a subjectively chosen input) may result in significant changes in operating profit reported for the period.” 3 highly subjective fair value estimate within a critical audit matter (CAM) disclosure.4 For some participants, the auditor’s report also includes a disclosure of the auditor’s quantitative materiality threshold. After reviewing the auditor’s report and an income statement that contains the line item associated with the subjective fair value estimate, participants read a footnote disclosure related to the estimate. In all conditions, a narrative description of sensitivity is provided. Additionally, participants not assigned to the control condition receive a QSA that either indicates a relatively low level of sensitivity (i.e., an amount of imprecision that falls far below the auditor’s materiality threshold) or a relatively high level of sensitivity (that exceeds materiality by multiples of materiality). Consistent with our expectations, we find that when only a narrative description of sensitivity is disclosed (i.e., the control condition), investors perceive the reliability of the fair value estimate to be lower than when a QSA is also provided. Most importantly, we find that investors’ ability to recognize differences in estimation uncertainty between the low and high sensitivity conditions is improved when they are presented with both a QSA and a materiality threshold disclosure. Without the disclosure of materiality to help investors interpret the QSA, however, investors fail to recognize differences in reliability between the two levels of sensitivity, even though the amount of imprecision in the high sensitivity condition is approximately 15 times the amount indicated by the QSA in the low sensitivity condition. Surprisingly, we find that while investors perceive the high sensitivity estimate to be relatively unreliable (when materiality is disclosed), their willingness to invest in the company is not reduced relative to the low sensitivity 4 CAMs (also referred to as "key audit matters") are disclosures contained within the auditor’s report that “inform investors and other financial statement users of matters arising from the audit that required especially challenging, subjective, or complex auditor judgment” (PCAOB 2016, 3). Although CAM disclosures are unlikely to provide investors with enough information to fully understand a CAM item, they have an attention-directing effect, prompting investors to consider more carefully other disclosures that provide additional information about the item (e.g., related footnote disclosures) (PCAOB 2016; Sirois, Bédard and Bera 2016). 4 condition. An additional experiment reveals that this finding results from an unconscious error in reasoning on the part of our investor participants. When differences in estimation uncertainty are made more salient to participants using a within-subjects design, we find that investors greatly prefer to invest in a company whose financial statements contain a lower level of estimation uncertainty. Our study makes several important contributions to the accounting literature as well as to accounting practice and policy. While several studies have examined how investors respond to sensitivity analyses (e.g., Koonce, Lipe and McAnally 2005; Koonce, McAnally and Mercer 2005; Elliott, Jackson and Smith 2008; Nelson and Rupar 2015), very little evidence exists regarding factors that enhance the usefulness of the information contained within a sensitivity analysis disclosure. Our results suggest that sensitivity analyses may be somewhat ineffective in helping investors understand the amount of uncertainty contained in Level 3 fair value estimates, unless they are quantitative in nature and supplemented with a quantitative materiality disclosure. Furthermore, our study examines a context in which the subjectivity inherent in a reported estimate is highlighted within a CAM disclosure. Given that CAMs are set to become a standard component of the auditor’s report in much of the world, it is important for practitioners and standard setters to begin to consider the types of information that will allow investors to understand the subjectivity and complexity highlighted in CAM items. Our study serves such an objective. Despite receiving strong support from investors (e.g., FASB 2010a; IASB 2010; PCAOB Investor Advisory Group 2011; Singh and Peters 2015), standard setters have decided against requiring the disclosure of QSAs and audit materiality (e.g., FASB 2010a, 2011; IASB 2011; IAASB 2015), with very few exceptions.5 According to the FASB, preparers have argued that a 5 For example, a QSA is required when estimating the fair value of financial instruments (per IFRS 7) and a quantitative materiality disclosure is required in the U.K. (FRC 2013). 5 QSA “would not be decision useful because the range of reasonably possible Level 3 fair values would be extremely wide and, thus, would be meaningless” (PCAOB 2010, 45). Counter to this argument, the results of our study suggest that QSA disclosures provide meaningful information to investors that impacts their judgments and decisions, but only when a materiality threshold is also disclosed. These findings should be informative for standard setters as they continue to consider the types of disclosures that may help investors and other financial statement users understand the most complex and subjective aspects of financial reporting. The remainder of this paper proceeds as follows. In section II, we review related literature and develop our hypotheses. The methodology and results are discussed in sections III and IV, respectively. Finally, we conclude with a discussion in section V. II. BACKGROUND AND HYPOTHESES Extreme Estimation Uncertainty Many financial statement accounts require estimation due to uncertainty surrounding the measurement and valuation of items that are contingent on the outcome of future events (e.g., allowance for doubtful accounts, fair value measurement, goodwill impairment) (PCAOB 2003, ¶ 1). Some estimates require complex valuation models that use uncertain and unobservable inputs (e.g., Level 3 fair value inputs), which are highly subjective. As a result, the level of uncertainty inherent in some estimates creates a range of reasonably possible values that exceeds auditors’ materiality thresholds, sometimes by multiples of materiality (Christensen et al. 2012; Cannon and Bedard 2016; Glover et al. 2017). Christensen et al. (2012) use publicly available data to demonstrate that a relatively small change to a fair value input (e.g., interest rates) has the potential to change an estimate's reported value by as much as 50 times audit materiality.6 Because changes 6 Christensen et al. (2012) estimate audit materiality using the common materiality threshold of 5 percent of pre-tax income to approximate the extent to which small changes in fair value inputs exceed materiality. 6 in fair value estimates often impact earnings, the net income figure reported by some companies may also include an extreme level of estimation uncertainty. Estimates containing a high degree of uncertainty can be problematic because they are inherently unreliable due to their dependence on management’s judgment, which may be inaccurate due to bias and/or the difficulty associated with predicting future conditions and events (Lev, Li and Sougiannis 2010).7 The more subjective an estimate, the less reliable, and therefore, the less useful it is to financial statement users (Richardson et al. 2005; Lev et al. 2010). Richardson et al. (2005) provide empirical evidence that more subjective (and less reliable) accruals estimates result in lower earnings persistence and that investors fail to anticipate this effect, leading to security mispricing. Unfortunately, it is not always possible to reduce estimation uncertainty to an immaterial amount (Bell and Griffin 2012). In the presence of extreme and irreducible estimation uncertainty, the question becomes, how can management and auditors effectively communicate the uncertainty contained within financial statement items, so that users can incorporate this information into their decisions in a meaningful way? As CAM disclosures become a standard component of the auditor’s report in much of the world, identifying effective ways to communicate information about estimation uncertainty to financial statement users is even more important. 8 The IAASB (2015) and PCAOB (2016) anticipate that CAMs will frequently focus on financial statement areas involving significant management judgment and high estimation uncertainty. By drawing attention to difficult and 7 Reliable financial statement information is that which is “free from material error or bias and can be depended upon by users to represent faithfully that which it purports to represent” (FASB 2010b, ¶ BC3.22). While reliability was a component of the original Conceptual Framework, it has since been replaced with the concept of faithful representation, which the FASB believes is a clearer concept that still encompassed the original components of reliability (FASB 2010b). 8 CAM disclosures are now required by the IAASB (2015), European Commission (EC 2014), and the U.K.'s Financial Reporting Council (FRC 2013). The PCAOB (2016) recently issued a revised proposal that would require auditors in the U.S. to include CAM disclosures within the auditor's report. 7 subjective audit areas, CAMs may enable financial statement users to “analyze more closely any related financial statement accounts and disclosures” (PCAOB 2013, p. A5-23). This attention directing effect of CAMs, supported empirically in an experiment by Sirois et al. (2016), suggests that CAMs may be most useful when they are supplemented with additional disclosures aimed at helping users understand the uncertainty highlighted in the CAM. We focus on two disclosures that are especially relevant for communicating the risk and uncertainty inherent in estimates containing extreme estimation uncertainty: (1) quantitative sensitivity analyses (QSAs) and quantitative auditor materiality thresholds. Quantitative Sensitivity Analyses One type of supplemental footnote disclosure that may help financial statement users understand the amount of uncertainty contained within an account is the disclosure of a QSA. A QSA computes the potential increase or decrease in a reported estimate (and/or the resulting effect on earnings) that would result from a change in one or more of the inputs used to determine the estimate (Linsmeier and Pearson 1997). For example, an estimate may be calculated using various assumptions, such as future interest rates, commodity prices, etc. A QSA would indicate the effect on earnings if one or more of these inputs were to change by a certain amount (e.g., a one percent change in commodity prices would result in a $5 million change in reported earnings). While a QSA is required for certain financial statement items (e.g., financial instruments measured at fair value per IFRS 7),9 other items may only require a qualitative analysis or no sensitivity analysis at all (see Appendix A for a comparison of a quantitative and qualitative sensitivity analysis). In 2010, the FASB (2010a) and IASB (2010) considered requiring a QSA for all Level 3 fair value estimates. Although financial statement users were supportive of a quantitative disclosure and 9 Additionally, FASB ASC 715-20-50-1 requires public companies in the U.S. to disclose the sensitivity of accumulated postretirement benefit obligations to a one percent change in assumed health care cost trend rates. 8 believe it provides “useful information that helps them to assess the subjectivity of an entity’s fair value measurements categorized [as] Level 3” (IASB 2010, 16), both standard setting bodies ultimately decided that a narrative description of sensitivity would suffice (FASB 2011). Prior research finds that investors’ judgments and decisions are influenced by the information contained within a QSA. For example, the format in which a QSA is presented (e.g., stating the impact on earnings in dollar versus percentage terms) appears to influence investors’ judgments regarding an item’s risk and reliability (Koonce, Lipe and McAnally 2005; Elliott et al. 2008; Nelson and Rupar 2015). The amount of loss communicated within a QSA also appears to influence investors’ risk perceptions (Koonce, McAnally and Mercer 2005). Our study differs from these previous studies in three distinct ways. First, our study examines the effect of a QSA in an environment in which an uncertain estimate is highlighted within a CAM disclosure. When the subjectivity inherent in a fair value estimate has been highlighted within a CAM, investors may interpret a supporting QSA very differently than when such a CAM disclosure is absent (as has been the case in prior literature).10 Second, our study compares the effect of a QSA to that of a narrative description of sensitivity. This is an important comparison, given that standard setters have debated whether a QSA provides meaningful information to investors beyond that contained within a qualitative disclosure (FASB 2010a, 2011; IASB 2010). Finally, prior studies have not examined how other management or auditor disclosures (e.g., disclosure of the auditor's materiality threshold) may enhance the usefulness or understandability of a QSA. When investors are aware that a point estimate reported on the face of the financial statements contains estimation uncertainty (e.g., because this uncertainty has been highlighted 10 By highlighting a significant financial statement area, a CAM disclosure may cause investors to be more responsive to information contained within a related footnote disclosure. In contrast, a CAM may cause investors to perceive an item as generally unreliable, regardless of what may be indicated within a related QSA. Therefore, results obtained in prior literature may not generalize to CAM environment. 9 within a CAM disclosure), they are likely to view the item as unreliable and desire additional information to understand this uncertainty better. A narrative description of sensitivity is unlikely to do much to satisfy investors' desire for additional information, as a qualitative sensitivity analysis simply states that an estimate is highly sensitive to changes in underlying inputs. A QSA, on the other hand, explicitly quantifies the amount of uncertainty in the reported estimate and, therefore, should increase investors' perceptions of the item's reliability by providing an informative supplement to the point estimate reported on the face of the financial statements. Investors generally prefer financial disclosures which contain a level of precision that matches (i.e., is congruent with) the level of uncertainty underlying the financial item itself, referred to as the “congruence principle” (Du, Budescu, Shelly and Omer 2011). When a point estimate is reported for a very uncertain item, investors believe the point estimate is inaccurate and lacking in credibility. A range estimate, on the other hand, is perceived as more credible when uncertainty is high because a range communicates that uncertainty exists and gives the parameters of that uncertainty (Du et al. 2011). Given that all numbers reported on the face of the financial statements are point estimates, which are likely to be viewed as unreliable in the case of highly subjective estimates (Du et al. 2011), we examine whether supplementing a point estimate with a QSA increases the perceived reliability of the reported amount.11 Furthermore, there is a general belief within accounting that quantified disclosures are of higher quality because they are more informative than disclosures that are purely qualitative (e.g., Botosan 1997; Kadous, Koonce, and Towry 2010). Therefore, we expect that investors will perceive a highly subjective estimate as 11 Du et al. (2011) examined financial disclosures in the form of management earnings forecasts, which were either reported as a point estimate or a range estimate. We extend Du et al. (2011) by examining whether supplementing a point estimate on the face of the financial statements increases the perceived reliability of the reported estimate. 10 more reliable when it is supplemented with a QSA compared to when only a narrative description of sensitivity is provided. Stated formally, our first hypothesis is as follows: H1: Investors will judge the reliability of an uncertain estimate to be higher when it is supplemented with a QSA compared to when it is not. While a QSA is expected to reduce uncertainty, compared to when such an analysis is absent, the degree to which uncertainty is reduced depends on the “sensitivity” of the financial statement item. Consider an example in which a company reports a QSA indicating that a one percentage point increase in an interest rate used to calculate an uncertain estimate will result in a $50 million decrease in earnings. Compare this to an identical disclosure that only indicates a potential earnings effect of $2 million. Obviously, the former example reduces uncertainty to a much lesser extent than the latter example due to the high level of imprecision communicated by the sensitivity analysis. Although investors prefer that estimates be communicated in a format that is congruent with the uncertainty that exists in the environment, they also desire the communication to be as precise as possible (e.g., a narrow range as compared to a wide range of potential values) (Du et al. 2011). Therefore, we expect the level of sensitivity communicated by a QSA to be negatively related to investors' perceptions regarding an estimate's reliability (i.e., less sensitivity will result in increased perceptions of reliability). Stated formally, our second hypothesis is as follows: H2: In the presence of a QSA, investors will judge the reliability of an uncertain estimate to be higher when the item is less sensitive to changes in underlying inputs compared to more sensitive. Auditors’ Quantitative Materiality Thresholds Financial statement information is considered to be material if its omission or misstatement is likely to influence the judgments of a reasonable investor (PCAOB 2010). In recent years, several standard setting bodies have considered requiring the disclosure of quantitative materiality 11 thresholds within the auditor’s report. As a result, audit reports issued in the U.K. now include a disclosure of the materiality threshold used for the financial statements as a whole (FRC 2013).12 The PCAOB, ASB, and IAASB, on the other hand, have decided against a materiality disclosure requirement. However, auditors may choose to disclose materiality levels voluntarily (IAASB 2015), and in fact, PwC in Sweden and Finland has elected to do so for audits of financial statements with years ending on or after December 31, 2016. While most standard setting bodies have decided against a materiality disclosure requirement, investors generally support the provision of such information (PCAOB 2011; Singh and Peters 2015).13 Much of the prior literature on materiality has focused on the various determinants of materiality thresholds (Eilifsen and Messier 2015; see also Holstrum and Messier 1982 and Messier, Martinov-Bennie, and Eilifsen 2005 for reviews of the materiality literature). Only a few studies have considered the potential impact of disclosing materiality levels to the public. These studies suggest that the disclosure of materiality may increase the accuracy of security prices (Fisher 1990) and help to reduce the expectations gap that exists between auditors and the investing public (Coram, Mock, Turner, and Tray 2011; Gray, Turner, Coram, and Mock 2011; Houghton, Jubb, and Kend 2011). Our study extends prior research on materiality by examining how the disclosure of a quantitative materiality threshold, in addition to a QSA disclosure, influences investors’ perceptions regarding an amount recognized within the financial statements that contains estimation uncertainty. 12 The EC (2014) requires a similar disclosure of the quantitative materiality threshold used for the overall financial statements, but it is disclosed in an additional report provided to the audit committee rather than to the public. 13 The PCAOB’s Investor Advisory Group (2011) reports that 56 percent of investors surveyed agree that auditors should be required to disclose materiality, while only 17 percent disagreed. Also, the CFA Institute reports that 82 percent of survey respondents support increased disclosure about materiality (Singh and Peters 2015). 12 As previously discussed, we expect the disclosure of a QSA to help quantify the amount of uncertainty in an estimate. Once this uncertainty has been quantified, however, investors still must assess how significant this imprecision (i.e., potential misstatement) is to the financial statements as a whole and how this information might impact their judgments and decisions.14 The disclosure of a quantitative materiality threshold should provide investors with a useful benchmark for assessing the significance of a potential misstatement. Because materiality thresholds are company-specific (i.e., are determined based on the quantitative and qualitative characteristics of a given company), they make it possible to evaluate the severity of a misstatement in the context of a given company's financial statements. Context Theory suggests that the way in which a given cue is interpreted depends on the context within which it is presented (Medin and Schaffer 1978; Medin, Goldstone, and Gentner 1993). While it may be difficult for investors to interpret a QSA, the disclosure of a quantitative materiality threshold provides investors with a simple benchmark for evaluating the level of uncertainty communicated by the QSA (i.e., puts the QSA "in context"). When a QSA suggests that the imprecision in a reported estimate exceeds materiality (potentially by multiples of materiality), investors are likely to perceive the estimate as very unreliable. In contrast, when a QSA suggests that the level of imprecision falls below materiality, investors are likely to perceive the related estimate as more reliable as it falls within a range of uncertainty considered “acceptable” by the auditors. Stated formally, our third hypothesis is as follows: H3: In the presence of a QSA, investors will judge the reliability of an uncertain estimate to be highest (lowest) when the QSA indicates a range of potential values that falls below (exceeds) the auditor’s disclosed materiality threshold. 14 Imprecision in reported estimates creates potential for material misstatement, as it indicates that an amount reported in the financial statements may not be accurate. If this imprecision is large enough, it may suggest that the financial statements (including net income) are materially misstated. 13 Given that the concept of materiality describes information or amounts expected to influence the judgments and decisions of reasonable investors, we also examine how our variables of interest influence investors' ultimate willingness to invest. When estimation uncertainty is disclosed more transparently through a QSA and materiality disclosure, we expect investors to recognize the risk that certain financial statement amounts (including net income) may be materially misstated. As a result, we predict that when the uncertainty indicated within a QSA falls below auditors' materiality threshold, investors will be more confident that the financial statements are not materially misstated and will be more willing to invest. In contrast, when uncertainty communicated within a QSA exceeds the materiality threshold, investors will doubt the reliability of the financial statements, and in turn, will be relatively unwilling to invest. Stated formally, our final hypothesis is as follows: H4: In the presence of a QSA, investors will display the greatest (least) willingness to invest in a company with financial statements containing an uncertain estimate when the QSA indicates a range of potential values that falls below (exceeds) the auditor’s disclosed materiality threshold. III. METHOD Participants Participants in the experiment consisted of 193 graduate students enrolled in either a Master’s in Accounting and Auditing program or a Master's in Economics and Business Administration program at a large public business school in Bergen, Norway. 15 Like MBA students, our participants also have the requisite coursework necessary to qualify as proxies for nonprofessional investors (Elliott, Hodge, Kennedy, and Pronk 2007). Participants had completed 15 Two participants were determined to be outliers and were removed from the sample. One of the participants completed the study in less than three minutes (median completion time was 20.4 minutes). The other participant's response to the dependent variable (perceived reliability) was more than three standard deviations below the mean. Including these participants does not change our study's inferences. 14 an average of 8.8 classes (during their bachelor’s and master’s programs) related to financial accounting, auditing, finance, and valuation and had an average of 1.4 years of professional work experience in the fields of accounting, finance, and/or business. Fifty percent of participants were 22-25 years old, 40 percent were 26-29, 8 percent were 30 or older, and 2 percent were under 22 years old. Forty-one percent of participants were female.16 Experimental Materials and Procedures To examine our research hypotheses, we conducted an experiment that utilized a 2 × 2 + 1 between-subjects design. We manipulated the auditor’s disclosure of materiality in the audit report (present versus absent) and management’s disclosure of a QSA (low sensitivity versus high sensitivity). We also included a control condition in which both disclosures (materiality and a QSA) were absent. In all conditions (including the control), participants are provided with a narrative description of the estimate's sensitivity. The experiment was conducted online using Qualtrics and was administrated in two graduate-level courses by two of the study’s researchers. The study link was distributed to students via email during class time and was completed in class by those students in attendance and outside of class by those students not in attendance on the day of the study. The experiment was completed in three stages. In the first stage, participants were instructed to assume the role of a nonprofessional investor who is considering an investment in a hypothetical fish farming company, King Fish Farming Group (KFFG). Participants received a brief description of KFFG, before being provided with select information from the company’s current year annual report. Specifically, participants viewed excerpts from the auditor’s report, which included a standard unqualified opinion as well as a critical audit matter (CAM) disclosure. 16 None of the demographic variables differ significantly across conditions nor are they significant covariates in any of the models tested. 15 In all conditions, the CAM disclosure described the high level of uncertainty associated with estimating KFFG’s “adjustment of live fish inventory to fair value,” which is an income statement account reflecting gains and losses associated with adjusting the company’s live fish inventory to net realizable value.17 At the end of the CAM disclosure, participants were directed to, “See note 1 to the financial statements for the disclosure…related to the fair value of live fish.” (See Appendix B for the full text of the CAM paragraph.) The audit report also included our manipulation of the auditor’s materiality threshold disclosure (absent or present), which is described in more detail in the next section of the paper. The audit report was followed by a simplified income statement that included the operating profit line item “adjustment of live fish inventory to fair value.” For each of the two years presented, the magnitude of the adjustment was substantial relative to earnings but with opposite effects (i.e., the adjustment reflected a gain in the current year and a loss in the prior year). Therefore, even though the company’s earnings increased by nearly 30 percent in the current year, this increase is largely attributable to the substantial increase in the adjustment account, which represents a highly subjective management estimate. After viewing the company background, audit report, and income statement, participants were asked for their initial perceptions regarding the reliability of KFFG’s “adjustment of live fish inventory to fair value.” At this point in the study, participants had not yet received the footnote 17 Our experimental materials were based on the QSA disclosures of an actual fish farming company in Norway. Live fish (a biological asset) are measured at fair value less cost to sell (i.e., net realizable value) in accordance with IAS 41 “Agriculture” and IFRS 13 “Fair Value Measurement.” Observable markets and transactions for the future sale of live fish do not exist, so the valuation of fish inventory requires the establishment of an estimated fair value of the fish in a hypothetical market. Therefore, fair value measurement is categorized as Level 3 in the fair value hierarchy. Market prices for salmon and trout are extremely volatile due to seasonal variations in supply and demand, variations in the quality and size of fish, and environmental factors, such as disease outbreaks and food scares. These factors, as well as the three year production cycle of fish inventory, make it difficult to estimate future sales prices. Due to the uncertain nature of estimating future fish prices, fish farming companies routinely disclose QSAs, which display the potential effects on profit if the estimated fish prices used in the fair value model were to change. 16 disclosure related to the fair value adjustment account (and the accompanying QSA manipulation); therefore, these initial reliability judgments serve as pre-treatment measures. In the second stage of the study, participants received a footnote disclosure related to “live fish inventory” and the associated income statement account (i.e., “adjustment of live fish inventory to fair value”). The footnote included a description of how the estimated fair value of live fish is determined, including how the estimate critically depends on estimated future market prices of live fish (one of the primary inputs into management’s fair value model), and how even minor changes in these estimated prices may result in significant changes in operating profit (i.e., a narrative description of sensitivity). (See Appendix C for the full text of the footnote). The footnote disclosure contained our QSA manipulation (described in detail in the next section). Finally, in stage three of the experiment, participants answered our dependent variables and other post-experimental questions. Participants had access to the full case materials as they answered these questions. Afterwards, participants responded to manipulation check and demographic questions (access to case materials was prohibited in this final section of the questionnaire). Independent Variables The first independent variable, Materiality Threshold, relates to a disclosure within the auditor's report of the quantitative materiality threshold used during the audit of KFFG (Present versus Absent). In all conditions, the audit report contained a brief definition of materiality, but only participants in the Materiality Threshold – Present condition received the following sentence: “We determined materiality for the Group to be NOK 44 million, which is approximately 5% of profit before tax.”18 (See Appendix B for the full audit report.) 18 The Norwegian krone (NOK) is the currency of Norway. One U.S. dollar is equal to approximately 8.29 NOK. 17 The second independent variable, QSA, relates to the disclosure of a QSA within the "live fish inventory" footnote (Low Sensitivity versus High Sensitivity). The QSA appeared as a table, which quantified the amount by which operating profit would change if a critical input used in the fair value estimate (i.e., estimated future market prices) had been higher or lower by 4% and by 20%. In the Low Sensitivity condition, both amounts (associated with a 4% and 20% change in market prices) fall far below the auditor’s materiality threshold (i.e., are less than 5% of pretax profit). In the High Sensitivity condition, however, both amounts exceed the auditor’s materiality threshold. Specifically, a 4% change in market prices would cause operating profit to change by more than 1.5 times materiality, and a 20% change in market prices would cause operating profit to change by nearly 7.5 times materiality. (See Appendix C for the QSA disclosures.) In the Control condition, participants did not receive a Materiality Threshold disclosure or the table containing a QSA. However, all participants received a footnote disclosure that contained a narrative description of the sensitivity of operating profit to changes in the estimated future market price of fish. Holding the narrative description of sensitivity constant across all conditions allows us to examine how the additional disclosure of a QSA (in the Low and High Sensitivity conditions) affects investors' judgments and decisions relative to a Control condition in which only a qualitative sensitivity analysis is provided. Dependent Variables There are two primary dependent variables used in the study: (1) perceived reliability and (2) willingness to invest. To measure perceived reliability, participants indicated the reliability of the income statement account, "adjustment of live fish inventory to fair value." 19 Reliability 19 To ensure participants had a clear understanding of what financial statement reliability represents, the following definition was provided: “Reliability is the extent to which information is measured with little uncertainty, is verifiable, and reflects a company’s business activities in an unbiased manner” (consistent with Frederickson, Hodge and Pratt [2006] and Clor-Proell 2009). 18 perceptions were collected on an 11-point scale with endpoints labeled “1 – Not at all Reliable” and “11 – Very Reliable.” Perceived reliability was collected both before and after participants viewed the footnote disclosure related to live fish inventory (containing the QSA manipulation). We then calculated a change score by subtracting participants’ initial reliability perceptions from those collected after participants had viewed the footnote disclosure.20 This approach allows us to more precisely measure the effect of a QSA disclosure while taking into account ex ante differences in reliability perceptions across participants. Our second dependent variable, willingness to invest, was measured using two questions. The first question asked participants, “In your opinion, how attractive is KFFG as an investment?” Responses were collected on an 11-point scale with endpoints labeled “1 – Very Unattractive” and “11 – Very Attractive.” The second question asked, “Assuming you had NOK 50,000 to invest in the common stock of one or more companies, how likely are you to invest a portion of this amount in the common stock of KFFG?” Responses to this question were collected on an 11-point scale with endpoints labeled “1 – Very Unlikely” and “11 – Very Likely.”21 IV. RESULTS Manipulation Checks We asked two questions to ensure participants attended to our manipulations. First, participants were asked whether or not the footnote disclosure had indicated “the exact NOK amount” by which operating profit would change if the estimated future market price of fish had 20 This approach of measuring reliability perceptions both before and after the treatment is consistent with prior studies (e.g., Frederickson et al. 2006; Clor-Proell, Proell, and Warfield 2014) and helps control for investors’ ex ante beliefs regarding the reliability of financial statement information, as such beliefs tend to be heterogeneous (Frederickson et al. 2006). Although participants did view our Materiality Threshold manipulation prior to making their initial reliability judgments, we do not find differences in pre-treatment reliability perceptions between the Materiality Threshold conditions (p = 0.664). 21 As a follow-up to the “investment likelihood” question, we also asked participants the amount they would be willing to invest in the common stock of KFFG (from NOK 0 to NOK 50,000); however, we do not find significant differences between experimental conditions on this measure. 19 been higher or lower by 4% and by 20%. Out of 193 participants, 150 (77.7 percent) answered this question correctly. Participants were then asked whether the auditors had disclosed the NOK amount considered to be material during their audit of KFFG. One hundred seventy-seven participants (91.7 percent) answered this question correctly. Analyses below are based on the 140 participants who correctly answered both manipulation check questions.22 Test of H1 H1 predicts that investors will perceive the reliability of an uncertain estimate to be higher when it is supplemented with a QSA compared to when only a narrative description of sensitivity is present. Panel A of Table 1 provides descriptive statistics for the two manipulated variables (Materiality Threshold and QSA) and the Control condition. 23 Panel C provides the specific contrast used to test H1. Results of this analysis show that investors perceive the reliability of the fair value adjustment account (i.e., the uncertain estimate) to be higher in the presence of a QSA disclosure (i.e., both the Low and High Sensitivity conditions) (mean = -0.63) compared to the absence of such a disclosure (i.e., the Control condition) (mean = -1.21) (t = 1.74, p = 0.042, onetailed).24 Therefore, H1 is supported.25 22 Nearly all of the participants who failed a manipulation check question (87 percent) were in a condition in which the disclosure was present, but the participant failed to recall seeing it. By only including in our analyses those participants who correctly answered our manipulation check questions, we are able to rule out the possibility that our results are due to differences between conditions in information acquisition (Frederickson et al. 2006). Overall, results are similar (although weaker) when manipulation check failures are included in the analyses. Specifically, tests associated with H1 are no longer significant, while results for H2, H3, and H4 are unchanged. 23 All p-values in the paper are two-tailed unless otherwise noted. 24 We find similar results (t = 1.55, p = 0.062, one-tailed) when this analysis is limited to the Materiality ThresholdAbsent condition only. 25 We also asked participants how difficult it was to evaluate the reliability of the uncertain estimate using information disclosed by management and the auditors (on a scale ranging from "1 – Very Difficult" to "11 – Very Easy"). Participants indicated it was significantly easier to evaluate reliability when both a QSA and materiality threshold were disclosed (mean = 3.55) compared to when they were not (mean = 2.43) (t = 2.49; p = 0.014, untabulated). Participants also believed that the footnote disclosure about the uncertain estimate was significantly more informative (on a scale ranging from "1 – Not at all Informative" to "11 – Very Informative") when it contained a QSA (mean = 5.90) compared to when it did not (mean = 4.61) (t = 2.24; p = 0.026, untabulated). 20 [Insert Table 1 here] Test of H2 H2 predicts that investors will judge the reliability of an uncertain estimate to be higher when the item is less sensitive to changes in underlying inputs compared to more sensitive. Panel B of Table 1 reports a 2 x 2 ANOVA model (excluding the Control condition) that includes Materiality Threshold and QSA as the independent variables and perceived reliability as the dependent variable. The main effect of QSA is not significant in the ANOVA model (F = 1.43, p = 0.118, one-tailed equivalent); therefore, H2 is not supported. Failing to find support for H2 suggests that in general, investors may struggle to recognize the level of uncertainty (i.e., sensitivity) indicated within a QSA. In H3, we examine whether a materiality threshold disclosure helps investors differentiate between uncertain estimates that are relatively low, compared to high, in sensitivity. Test of H3 H3 predicts that investors will more easily recognize the difference in reliability between the Low and High Sensitivity conditions in the presence of a materiality threshold disclosure. That is, we expect that when a QSA is disclosed, reliability perceptions will be highest in the Materiality Threshold–Present/Low Sensitivity condition (i.e., when the amount of sensitivity falls below the materiality threshold), and lowest in the Materiality Threshold–Present/High Sensitivity condition (i.e., when the amount of sensitivity exceeds the materiality threshold). Figure 1 graphs the results associated with the Materiality Threshold x QSA interaction predicted in H3. The ANOVA model presented in Panel B of Table 1 includes a significant Materiality Threshold x QSA interaction (F = 4.82, p = 0.030). Simple effects tests indicate that the form of this interaction is in line with our expectations. Specifically, the difference between 21 the Low and High Sensitivity conditions is significant in the Materiality Threshold–Present condition (means of -0.11 and -1.18, respectively) (t = 2.40, p = 0.009, one-tailed), but not in the Materiality Threshold–Absent condition (means of -0.80 and -0.48, respectively) (t = -0.71, p = 0.482). Therefore, H3 is supported. It would appear that in order for investors to recognize the difference in reliability between an uncertain estimate that contains a relatively low level of sensitivity and one containing a relatively high level of sensitivity, a QSA alone is insufficient. Rather, the QSA must be supplemented with a disclosure of the auditor’s materiality threshold. [Insert Figure 1 here] Test of H4 H4 predicts that investors' willingness to invest in a company will be influenced by the disclosure of a materiality threshold and a QSA. Specifically, we predict that when a QSA is disclosed, willingness to invest will be highest in the Materiality Threshold–Present/Low Sensitivity condition and lowest in the Materiality Threshold–Present/High Sensitivity condition. Panel A of Table 2 provides descriptive statistics for both dependent variables related to the willingness to invest (i.e., attractiveness and investment likelihood), and Panel B presents the results of the ANOVA models associated with each dependent variable. Although the Materiality Threshold x QSA interaction term is significant in the ANOVA model for investment likelihood (F = 6.06, p = 0.015) and marginally significant for attractiveness (F = 2.89 p = 0.092), the interaction is in the direction opposite of our prediction. [Insert Table 2 here] Simple effects indicate that when the auditor’s materiality threshold is disclosed, investors perceive the company as more attractive when the QSA indicates High Sensitivity (mean = 6.89) compared to when it indicates Low Sensitivity (mean = 6.18) (t = 1.75, p = 0.083, untabulated). 22 Similarly, investors in the Materiality Threshold–Present condition indicate that they are more likely to invest in a company that discloses a QSA indicative of High Sensitivity (mean = 6.57) compared to Low Sensitivity (mean = 5.29) (t = 2.43, p = 0.017, untabulated). The difference between Low and High Sensitivity is not significant in the Materiality Threshold–Absent condition (p > 0.200 for both attractiveness and investment likelihood, untabulated). These results are not in line with our expectations. Therefore, H4 is not supported. Supplemental Analysis The results associated with H4 are surprising, given that participants perceive the fair value adjustment account as significantly less reliable in the High Sensitivity condition compared to the Low Sensitivity condition (when materiality is disclosed). Nonetheless, these reduced reliability perceptions in the High Sensitivity condition do not discourage investors from investing in the company; in fact, investors indicate a greater willingness to invest. To better understand this unusual finding, we first examine its robustness to the removal of influential outliers. Using Cook's Distance, we identify four participants who indicated an extreme value for the investment attractiveness measure and seven participants who indicated an extreme value for investment likelihood.26 When these outliers are removed from our sample, we find no significant differences in participants' willingness to invest across experimental conditions.27 Therefore, it appears that the results obtained in relation to H4, which were in the direction opposite of what we predicted, are largely attributable to a small number of influential outliers. Nonetheless, we still fail to find support for H4. As such, our next analysis examines the intentionality of participants’ investment- 26 Cook's Distance identifies influential data points by estimating the effect of removing a given observation from the analysis. We considered values greater than the common threshold of 4/n to be influential outliers. The same observations were identified as outliers when using studentized residuals greater than +/- 2. 27 Excluding these participants does not change our results related to perceived reliability. Specifically, results associated with H1, H2, and H3 are unchanged when these outliers are excluded from our analyses. 23 related judgments. That is, we examine whether our finding that investors do not prefer to invest in a company with lower, compared to higher, estimation uncertainty is conscious (intentional) or results from unintentional bias. In our experiment, the level of sensitivity (Low or High) was manipulated between-subjects. In our post-experimental questionnaire, we included a within-subjects manipulation of sensitivity.28 According to Kahneman and Tversky (1996), a between-subjects manipulation tests a participant’s natural reasoning process, whereas a within-subjects manipulation makes the independent variables more salient to participants, thus allowing them to identify and correct any judgment biases or errors in reasoning that may exist. If we obtain different results related to H4 when sensitivity is manipulated within-subjects (compared to between-subjects), this would suggest that the between-subjects results are unintentional (Kahneman and Tversky 1996; Libby, Bloomfield and Nelson 2002). In a real world investment context, investors rarely evaluate a company in isolation (as participants did in our between-subjects manipulation). Rather, investors typically compare several potential investments on a number of criteria to select the preferred investment. To simulate this real-world investment context, in our post-experimental questionnaire, participants were told to consider two firms (Firm A and Firm B) that are identical, except for the sensitivity of a fair value estimate that appears on each firm’s income statement. Participants were then presented with a QSA disclosure for each firm, where Firm A’s QSA was identical to the Low Sensitivity manipulation from our original experiment, and Firm B’s QSA was identical to the High Sensitivity manipulation. Thus, participants viewed both levels of our QSA manipulation simultaneously, and 28 The within-subjects manipulation was added to the very end of our experimental instrument during our second round of data collection. A total of 74 participants viewed and responded to the within-subjects manipulation and are included in the analyses reported in this section. 24 could decide which company they preferred. 29 Appendix D provides the full within-subjects manipulation. We then asked participants the following two questions regarding their willingness to invest in Firm A (Low Sensitivity disclosure) or Firm B (High Sensitivity disclosure): (1) “In your opinion, which firm is a more attractive investment?” and (2) “Assuming you had NOK 50,000 to invest in the common stock of Firm A or Firm B, which firm are you more likely to invest in?” We find that for both questions, a significantly greater proportion of participants preferred Firm A (75.7 percent) compared to Firm B (13.5 percent) (χ2 = 56.57; p < 0.001), with 10.8 percent indicating no preference. The results obtained on a within-subjects basis (i.e., a greater willingness to invest in a company with a QSA indicative of low sensitivity compared to high sensitivity, as predicted in H4) are inconsistent with results obtained on a between-subjects basis. This suggests that the between-subjects results obtained in our original experiment were unintentional. Specifically, participants are aware that companies with financial statements containing a lower level of estimation uncertainty are preferable for investment purposes; however, they are not able to access this information in their natural reasoning process (i.e., when evaluating only one company in isolation) (Kahneman and Tversky 1996; Libby et al. 2002). 30 Given that most investment decisions are not made in isolation, we believe our within-subjects experiment provides important insights regarding how QSA and materiality disclosures influence investment decisions by 29 We also provided participants with the auditor’s quantitative materiality threshold for both companies, which was the same amount provided in the Materiality Threshold-Present condition in our main experiment. We chose to provide a materiality threshold because H4 predicts that investors’ willingness to invest will be highest (lowest) in Low (High) Sensitivity condition, when materiality is also disclosed. 30 As part of our within-subjects manipulation, we also asked participants, “Which firm has a more reliable fair value estimate?” We find that a significantly greater proportion of participants (χ2 = 110.42; p < 0.001) perceive Firm A’s estimate as more reliable than Firm B’s (67.6 percent and 2.7 percent, respectively), which is consistent with our findings on a between-subjects basis and suggests that the between-subjects results related to perceived reliability are consistent with participants’ intentions. 25 allowing investors to more easily compare the level of estimation uncertainty faced by different companies. V. DISCUSSION AND CONCLUSION Over time, financial statements have begun to incorporate a greater amount of estimation uncertainty. As a result, investors often face the difficult task of making judgments and decisions based on inherently imprecise information. Although standard setters and regulators worldwide have considered mandating a number of disclosures to help users understand the uncertainty in audited financial statements, one concern is that individuals may still fail to understand and/or use this information once it is disclosed (Clor-Proell et al. 2014). To address this concern, we conducted a study that examined whether nonprofessional investors recognize differences in estimation uncertainty more easily in the presence of disclosures that help to quantify this uncertainty. We found that investors generally perceive an estimate as more reliable when it is supplemented with a sensitivity analysis that is quantitative in nature, as opposed to qualitative (as was the case in our Control condition). At present, companies are only required to disclose QSAs for certain fair value estimates (e.g., financial instruments), but not for others (e.g., inventory). Our results suggest that a QSA may make the task of evaluating highly subjective fair value estimates less difficult for investors by explicitly quantifying the level of uncertainty contained within them. Most importantly, when a QSA suggested that an estimate reported on the face of the financial statements contained extreme estimation uncertainty (i.e., high sensitivity), investors assessed the estimate’s reliability to be lower, compared to investors who viewed a QSA indicative of low sensitivity, but only when the auditor’s materiality threshold was also disclosed. In the absence of a materiality disclosure, investors’ reliability perceptions did not differ regardless of 26 the level of uncertainty indicated within the QSA. This finding is particularly interesting given the fact that only one regulatory body, the U.K.’s FRC, requires the public disclosure of audit materiality. Throughout the rest of the world, regulators and standard setters have considered, but ultimately decided against, requiring auditors to disclose their materiality thresholds. Nonetheless, our results suggest that for nonprofessional investors to recognize differences in estimation uncertainty disclosed in sensitivity analyses, not only must this disclosure be quantitative in nature, but also supplemented with a disclosure of materiality. Finally, counter to our expectations, investors' willingness to invest was not deterred when the presence of extreme estimation uncertainty was disclosed most transparently (i.e., in the presence of a materiality threshold disclosure and a QSA indicative of high, as opposed to low, sensitivity). Even though investors perceived reliability to be significantly lower in this condition, relative to the low sensitivity condition, these perceptions did not appear to “flow through” and influence investment decisions. However, when differences in estimation uncertainty were made more salient with a within-subjects manipulation, we found results that were more consistent with our expectations, in that participants indicated a greater preference to invest in a company with a QSA indicative of low, compared to high, sensitivity. Our finding that investment decisions were not responsive to the level of estimation uncertainty contained within financial statements is consistent with findings in the archival literature. Specifically, Richardson et al. (2005) find that security prices are not sufficiently reduced when financial statements contain highly uncertain estimates, even though such estimates are associated with lower earnings persistence. Our results suggest that this finding is likely driven in part by investors’ failure to recognize when estimates are indicative of low reliability. However, even when we provided participants with additional disclosures that helped them recognize the 27 presence of a low reliability estimate, their willingness to invest was not deterred. It was not until differences in estimation uncertainty were made more salient using a within-subjects design that participants' investment decisions reflected a strong preference for financial statements containing lower levels of estimation uncertainty. Although our study provides important insights, it is also subject to certain limitations. First, participants in our study were provided with much less information than they might normally consider when making investment decisions. Although this approach may limit the generalizability of our results, it helps to ensure that participants acquire the pieces of information that are most important to our study, thus allowing us to examine how this information is used once it has been acquired (Clor-Proell et al. 2014). Second, we examined only two types of uncertainty disclosures in our study. It is possible that other types of disclosures may also help investors recognize differences in estimation uncertainty. This is a question for future research to explore. Finally, we tested our hypotheses in an environment in which an uncertain fair value estimate was highlighted in a CAM disclosure. Although the inclusion of CAMs within the audit report will soon be the norm in much of the world, it is uncertain whether our results would hold in the absence of a CAM disclosure that highlights the uncertain estimate. Interestingly, the revised PCAOB (2016) proposal related to CAMs includes the requirement that CAMs must be material. Although the fair value adjustment account in our study was highly material in all instances, the level of imprecision underlying the estimate was only material in the High Sensitivity condition. Our results suggest that investors are unlikely to differentiate between material and immaterial CAM items unless materiality is also disclosed. As one member of the PCAOB Standing Advisory Group (2016, 37) commented, “Once you make [materiality] a gating issue, I think we deserve to know how wide the gate is.” 28 APPENDIX A: Quantitative versus Qualitative (Narrative) Sensitivity Analyses Example of a Quantitative Sensitivity Analysis (QSA) The following example was taken from Starbuck's 10-K for the year ended October 2, 2016: Example of a Qualitative (or Narrative) Sensitivity Analysis The following example was taken from Best Buy's 10-K for the year ended January 30, 2016: Property and Equipment Impairments Our impairment loss calculations require management to make assumptions and to apply judgment in order to estimate fair values, including estimating cash flows and useful lives and selecting a discount rate that reflects the risk inherent in future cash flows. If actual results are not consistent with our estimates and assumptions we may be exposed to impairments that could be material. 29 APPENDIX B: Auditor's Report Containing the Materiality Threshold Manipulation Below is the auditor's report provided to all participants. The sentence surrounded by brackets represents the Materiality Threshold manipulation, which was only present in the Materiality Threshold–Present condition. IMPORTANT EXCERPTS FROM THE INDEPENDENT AUDITOR’S REPORT Audit Opinion We have audited the financial statements of King Fish Farming Group (the Group). In our opinion, the accompanying financial statements present fairly, in all material respects, the financial position of the Group at December 31, 20X2, and its financial performance and its cash flows for the year then ended in accordance with International Financial Reporting Standards (IFRSs) as adopted by the EU. Key Audit Matters Key audit matters are those matters that, in our professional judgment, were of most significance in our audit of the consolidated financial statements of the current period. One of the key audit matters in our audit of the Group relates to the estimation of the fair value of live fish inventory, as described below. Adjustment of Live Fish Inventory to Fair Value The Group values its inventory of live fish at fair value less costs to sell. Active markets for the future sale of live fish do not exist. Therefore, the Group estimates the fair value of live fish using a model that reflects the Group’s own assumptions and projections. The valuation of live fish and measurement of profitability critically depend on the current estimate of salmon and trout prices at the expected time of sale. Historically, salmon and trout prices have been subject to large fluctuations. The uncertain nature of market prices does not allow future sales prices to be estimated with certainty. The fair value estimate of live fish is considered to be one of the most judgmental audit risks, as small changes in the assumptions used in the valuation (e.g., future sales prices) could have a significant impact on the Group’s profitability. See note 1 to the financial statements for the disclosure of the accounting policies and management’s estimation process related to the fair value of live fish. Our Application of Materiality We define materiality as the magnitude of misstatement in the financial statements that makes it probable that the economic decisions of a reasonably knowledgeable person would be changed or influenced. We use materiality both in planning the scope of our audit work and in evaluating the results of our work. We determined materiality for the Group to be NOK 44 million, which is approximately 5% of profit before tax. Bergen, 20 March 20X3 Big 4 accounting firm Knut Johansen State Authorized Public Accountant (Norway) 30 APPENDIX C: Footnote Disclosure Containing the QSA Manipulation Below is the footnote disclosure related to live fish inventory. All participants received this disclosure. However, only participants in the Low Sensitivity and High Sensitivity conditions (i.e., those not in the Control condition) received the final section of the disclosure, titled "The sensitivity of operating profit to changes in estimated future fish prices," which contained the QSA manipulation. NOTE 1. Live fish inventory (Extract) The following table provides a reconciliation of the beginning and ending balances for live fish inventory. (All figures in NOK 1 000) Beginning balance of live fish inventory at 01.01.20X2 Increase due to inventory costs added during the year Reduction due to sale of fish inventory Adjustment of live fish inventory to fair value (profit impact) Ending balance of live fish inventory at 31.12.20X2 3 082 219 4 860 324 -4 783 276 383 156 3 542 423 The Company recognizes live fish inventory at fair value less costs to sell. Active markets for the future sale of live fish do not exist, so the calculation of fair value is carried out using a valuation model (level 3 in the valuation hierarchy) where the relevant inputs (e.g., future market prices) are primarily unobservable and uncertain. In 20X2, the Company recorded a gain of NOK 383 million to adjust live fish inventory to fair value, which directly impacts operating profit reported for the period. The amount of this adjustment critically depends on the estimate of future salmon and trout prices at the expected time of sale. Because salmon and trout prices are subject to reasonably large fluctuations, even minor changes in the estimated market price of fish used in our valuation model may result in significant changes in operating profit reported for the period. The sensitivity of operating profit to changes in estimated future fish prices The table below shows the amount by which operating profit for 20X2 would change if the estimated future market price of fish used in our valuation model had been higher or lower by 4% and by 20%. Low Sensitivity Manipulation: Change in operating profit (in NOK 1 000) 4% price change 20% price change +/- 4 380 +/- 21 900 4% price change 20% price change +/- 71 936 +/- 328 549 High Sensitivity Manipulation: Change in operating profit (in NOK 1 000) 31 APPENDIX D: Within-Subjects Manipulation of Low Sensitivity versus High Sensitivity Consider two firms (Firm A and Firm B) that are identical, except for the sensitivity of a fair value estimate that appears on each firm's income statement. Below is a sensitivity analysis for each firm that shows the amount by which operating profit would change if an important input used in the company's fair value model had been higher or lower by 4% and by 20%. Firm A's Sensitivity Analysis (in NOK 1 000): 4% change in a fair value model input 20% change in a fair value model input +/- 4 380 +/- 21 900 4% change in a fair value model input 20% change in a fair value model input +/- 71 936 +/- 328 549 Change in operating profit (in NOK 1 000) Firm B's Sensitivity Analysis (in NOK 1 000): Change in operating profit (in NOK 1 000) Assuming that the auditors for both Firm A and Firm B determined materiality to be NOK 44 million (reflecting 5% of profit before tax), please answer the following questions: In your opinion, which firm is a more attractive investment? □ Firm A is more attractive. □ Firm B is more attractive. □ Firm A and Firm B are equally attractive. Assuming you had NOK 50,000 to invest in the common stock of Firm A or Firm B, which firm are you more likely to invest in? □ I am more likely to invest in Firm A. □ I am more likely to invest in Firm B. □ I am equally likely to invest in Firm A or in Firm B. 32 REFERENCES Bell, T. B., and J. B. Griffin. 2012. Commentary on auditing high-uncertainty fair value estimates. Auditing: A Journal of Practice & Theory 31 (1): 147-155. Botosan, C.A. 1997. Disclosure level and the cost of equity capital. The Accounting Review Vol. 72 (No. 3): 323-49. Cannon, N., and J. C. Bedard. 2016. Auditing challenging fair value measurements: Evidence from the field. The Accounting Review forthcoming. Christensen, B. E., S. M. Glover, and D. A. Wood. 2012. Extreme estimation uncertainty in fair value estimates: Implications for audit assurance. Auditing: A Journal of Practice & Theory 31 (1): 127 – 146. Christensen, B. E., S. M. Glover, and C. J. Wolfe. 2014. Do critical audit matter paragraphs in the audit report change nonprofessional investors' decision to invest? Auditing: A Journal of Practice & Theory 33 (4): 71–93. Clor-Proell, S. M. 2009. The Effects of Expected and Actual Accounting Choices on Judgments and Decisions. The Accounting Review 84 (5): 1465–1493. Clor-Proell, S. M., C. A. Proell, and T. D. Warfield. 2014. The effects of presentation salience and measurement subjectivity on nonprofessional investors' fair value judgments. Contemporary Accounting Research 31 (1): 45-66. Coram, P., T. Mock, J. Turner, and G. Gray. 2011. The communicative value of the auditor’s report. Australian Accounting Review 21 (3): 235–252. Du, N., D. V. Budescu, M. K. Shelly, and T. C. Omer. 2011. The appeal of vague financial forecasts. Organizational Behavior and Human Decision Processes 114 (2): 179-189. Eilifsen, A. and W. F. Messier, Jr. 2015. Materiality guidance of the major public accounting firms. Auditing: A Journal of Practice & Theory 34 (2): 3-26. Elliott, W. B., F. D. Hodge, J. J. Kennedy, and M. Pronk. 2007. Are M.B.A. students a good proxy for nonprofessional investors? The Accounting Review 82 (1): 139-168. Elliott, W. B., K. E. Jackson, and S. D. Smith. 2008. The influence of sensitivity disclosures on investor judgments. Working paper, University of Illinois at Urbana-Champaign. European Commission (EC). 2014. Regulation (EU) No 537/2014 of the European Parliament and of the Council of 16 April 2014 on Specific Requirements Regarding Statutory Audit of Public-Interest Entities and Repealing Commission Decision 2005/909/EC. Official Journal of the European Union. Strasbourg, France: EC. 33 Financial Accounting Standards Board (FASB). 2010a. Fair Value Measurements and Disclosures (Topic 820), Improving Disclosures about Fair Value Measurements. Norwalk, CT: FASB. Financial Accounting Standards Board (FASB). 2010b. Conceptual Framework for Financial Reporting. Chapter 3, Qualitative Characteristics of Useful Financial Information. Statement of Financial Accounting Concepts No. 8. Norwalk, CT: FASB. Financial Accounting Standards Board (FASB). 2011. Fair Value Measurement (Topic 820), Amendments to Achieve Common Fair Value Measurement and Disclosure Requirements in U.S. GAAP and IFRS. Accounting Standards Update No. 2011-4. Norwalk, CT: FASB. Financial Reporting Council (FRC). 2013. Consultation Paper: Revision to ISA (UK and Ireland) 700 - Requiring the Auditor's Report to Address Risks of Material Misstatement, Materiality and a Summary of the Audit Scope. London, England: Financial Reporting Council. Fisher, M. H. 1990. The effects of reporting auditor materiality levels publicly, privately, or not at all in an experimental setting. Auditing: A Journal of Practice & Theory 9 (Supplement): 184-223. Frederickson, J. R., F. D. Hodge, and J. H. Pratt. 2006. The evolution of stock option accounting: Disclosure, voluntary recognition, mandated recognition, and management disavowals. The Accounting Review 81 (5): 1073-1093. Glover, S. M., M. H. Taylor, and Y. Wu. 2017. Current practices and challenges in auditing fair value measurements and complex estimates: Implications for auditing standards and the academy. Auditing: A Journal of Practice & Theory 36 (1): 63-84. Gray, G. L., J. L. Turner, P. J. Coram, and T. J. Mock. 2011. Perceptions and misperceptions regarding the unqualified auditor’s report by financial statement preparers, users, and auditors. Accounting Horizons 25 (4): 659–684. Holstrum, G. L., and W. F. Messier 1982. A Review and Integration of Empirical Research on Materiality, Auditing: A Journal of Practice & Theory 9 (Supplement): 184-223. Houghton, K. A., C. Jubb, and M. Kend. 2011. Materiality in the context of audit: The real expectations gap. Managerial Auditing Journal 26 (6): 482–500. International Accounting Standards Board (IASB). 2010. Exposure Draft: Measurement Uncertainty Analysis Disclosure for Fair Value Measurements (Limited Re-Exposure of Proposed Disclosure). London, England: International Accounting Standards Committee Foundation. 34 International Accounting Standards Board (IASB). 2011. Fair Value Measurement. International Financial Reporting Standard (IFRS) 13. London, UK: IASB. International Auditing and Assurance Standards Board (IAASB). 2015. International Standard on Auditing 701: Communicating Key Audit Matters in the Independent Auditor’s Report. New York, NY: International Federation of Accountants. Kachelmeier, S. J., J. J. Schmidt, and K. Valentine. 2017. The disclaimer effect of disclosing critical audit matters in the auditor’s report. Working Paper, University of Texas at Austin. Kadous, K., L. Koonce and K. L. Towry. 2010. Quantification and persuasion in managerial judgment. Contemporary Accounting Research 22 (3): 643-686. Kahneman, D., and A. Tversky. 1996. On the reality of cognitive illusions. Psychological Review 103 (3): 582-588. Koonce, L., M. G. Lipe, and M. L. McAnally. 2005. Judging the risk of financial instruments: Problems and potential remedies. The Accounting Review 80 (3): 871-895. Koonce, L., M. L. McAnally, and M. Mercer. 2005. How do investors judge the risk of financial items? The Accounting Review 80 (1): 221-241. Lev, B., S. Li, and T. Sougiannis. 2010. The usefulness of accounting estimates for predicting cash flows and earnings. Review of Accounting Studies 15 (4): 779-807. Libby, R., R. Bloomfield, and M. W. Nelson. 2002. Experimental research in financial accounting. Accounting, Organizations and Society 27 (8): 775-810. Linsmeier, T. J., and N. D. Pearson. 1997. Quantitative disclosures of market risk in the SEC release. Accounting Horizons 11 (1): 107-135. Medin, D. L., R. L. Goldstone, and D. Gentner. 1993. Respects for similarity. Psychological Review 100 (2): 254-278. Medin, D. L., and M. M. Schaffer. 1978. Context theory of classification learning. Psychological Review 85: 207-238. Messier Jr., W. F., N. Martinov-Bennie, and A. Eilifsen 2005. A review and integration of empirical research on materiality: two decades later. Auditing: A Journal of Practice & Theory 24 (2): 153-187. Nelson, M. W., and K. K. Rupar. 2015. Numerical formats within risk disclosures and the moderating effect of investors' concerns about management discretion. The Accounting Review 90 (3): 1149-1168. Public Company Accounting Oversight Board (PCAOB). 2003. Auditing Accounting Estimates. PCAOB Interim Auditing Standards AU Section 342. Washington, D.C.: PCAOB. 35 Public Company Accounting Oversight Board (PCAOB). 2010. Consideration of Materiality in Planning and Performing the Audit. Auditing Standard No. 11. Washington, D.C.: PCAOB. Public Company Accounting Oversight Board (PCAOB). 2011. PCAOB Release No. 2011-003: Concept Release on Possible Revisions to PCAOB Standards Related to Reports on Audited Financial Statements. Washington, D.C.: PCAOB. Public Company Accounting Oversight Board (PCAOB). 2012. Auditing the Future. June 7. Washington, DC: PCAOB. Available http://pcaobus.org/News/Speech/Pages/06072012_HansonAICPA.aspx. at: Public Company Accounting Oversight Board (PCAOB). 2013. PCAOB Release No. 2013-005: Proposed Auditing Standards - The Auditor's Report on an Audit of Financial Statements When the Auditor Expresses an Unqualified Opinion. Washington, D.C.: PCAOB. Public Company Accounting Oversight Board (PCAOB). 2016. PCAOB Release No. 2016-003: Proposed Auditing Standards - The Auditor's Report on an Audit of Financial Statements When the Auditor Expresses an Unqualified Opinion. Washington, D.C.: PCAOB. Public Company Accounting Oversight Board (PCAOB) Investor Advisory Group. 2011. PCAOB Investor Advisory Group Meeting: March 16, 2011. Washington, D.C. Available at: https://pcaobus.org//News/Events/Documents/03162011_IAGMeeting/Role_Of_The_Au ditor.pdf Public Company Accounting Oversight Board (PCAOB) Standing Advisory Group. 2016. PCAOB Standing Advisory Group Meeting: May 18, 2016. Washington, D.C. Available at: https://pcaobus.org/Rulemaking/Docket034/SAG-Transcript-ARM-proposal-05-1816.pdf Richardson, S. A., R. G. Sloan, M. T. Soliman, and I. Tuna. 2005. Accrual reliability, earnings persistence and stock prices. Journal of Accounting and Economics 39 (3): 437-485. Singh, M., and S. J. Peters. 2015. Materiality: Investor Perspectives. Charlottesville, VA: CFA Institute. Sirois, L., J. Bédard, and P. Bera. 2016. The informational value of key audit matters in the auditor’s report: Evidence from an eye-tracking study. Working Paper, HEC Montreal. 36 FIGURE 1 Results of H3: Interaction of Materiality Threshold and QSA on Perceived Reliability 2 PerceivedReliability 1.5 1 0.5 QSA: LowSensitivity 0 HighSensitivity -0.5 -1 -1.5 -2 Absent MaterialityThreshold Present ____________________________________________________________ Figure 1 depicts the results of the test of H3. The graph shows the interaction of Materiality Threshold (Absent, Present) and QSA (Low Sensitivity, High Sensitivity) on investors’ reliability perceptions. Perceived reliability was measured by asking participants to indicate the reliability of an uncertain estimate reported on the income statement using an 11-point scale with endpoints labeled “1 – Not at all Reliable” and “11 – Very Reliable.” This question was asked both before and after participants received the QSA manipulation, and a change score was then calculated by subtracting pre-treatment reliability perceptions from post-treatment reliability perceptions. ______________________________________________________________________________ 37 TABLE 1 Effect of Materiality Threshold and QSA Disclosures on the Perceived Reliability of an Uncertain Estimate Panel A: Mean (Standard Deviation) of Perceived Reliabilitya Materiality Threshold - Absent No Sensitivity Low High ("Control") Sensitivity Sensitivity (n = 28) ( n = 25) ( n = 31) -1.21 -0.80 -0.48 (1.23) (1.85) (1.84) Materiality Threshold – Present Low High Sensitivity Sensitivity ( n = 28) ( n = 28) -0.11 -1.18 (1.42) (1.52) Panel B: ANOVA Table – Perceived Reliabilityb Source Materiality QSA Materiality*QSA Error df 1 1 1 108 SS 0.00 3.97 13.40 300.53 F 0.00 1.43 4.82 p-value 0.998 0.235 0.030 Panel C: Planned Contrast and Simple Effects Comparison Control < Low & High Sensitivity (H1) Materiality-Present: Low > High Sensitivity Materiality-Absent: Low = High Sensitivity df 137 108 108 t 1.74 2.40 0.71 p-value 0.042* 0.009* 0.482 _______________________________________ Table 1 reports descriptive statistics (Panel A) and results of an ANOVA with perceived reliability as the dependent variable and Materiality Threshold ("Materiality") and QSA (i.e., quantitative sensitivity analysis) as independent variables (Panel B). Panel C provides planned contrasts and simple effects. P-values are two-tailed unless otherwise noted. a Perceived reliability was measured by asking participants their perceptions regarding the reliability of an uncertain estimate using an 11-point scale with endpoints labeled “1 – Not at all Reliable” and “11 – Very Reliable.” This measure was elicited before and after participants received the QSA manipulation. A change score was then calculated. b The "Control" condition was dropped for purposes of the 2x2 ANOVA model. *One-tailed test. _____________________________________________________________________________ 38 TABLE 2 Effect of Materiality Threshold and QSA Disclosures on Willingness to Invest Panel A: Mean (Standard Deviation) of Attractiveness and Investment Likelihooda Materiality Threshold - Absent No Low High Sensitivity Sensitivity Sensitivity ("Control") ( n = 25) ( n = 31) (n = 28) Attractiveness 6.82 6.40 6.13 (1.81) (1.08) (1.59) Investment Likelihood 6.07 (2.39) 6.08 (1.38) Materiality Threshold - Present Low High Sensitivity Sensitivity ( n = 28) ( n = 28) 5.52 (1.96) 6.18 (1.91) 6.89 (1.37) 5.29 (2.48) 6.57 (1.89) Panel B: ANOVA Tables – Attractiveness and Investment Likelihoodb Dependent Variable Attractiveness Source Materiality QSA Materiality*QSA Error df 1 1 1 108 SS 2.05 1.37 6.76 252.27 F 0.88 0.59 2.89 p-value 0.351 0.446 0.092 Investment Likelihood Materiality QSA Materiality*QSA Error 1 1 1 108 0.47 3.63 23.81 424.15 0.12 0.92 6.06 0.729 0.339 0.015 _______________________________________ Table 2 reports descriptive statistics (Panel A) and results of ANOVAs with willingness to invest (i.e., attractiveness and investment likelihood) as the dependent variable and Materiality Threshold ("Materiality") and QSA (i.e., quantitative sensitivity analysis) as independent variables (Panel B). P-values are two-tailed. a Attractiveness of the company as an investment was measured on an 11-point scale with endpoints labeled “1 – Very Unattractive” and “11 – Very Attractive.” Investment likelihood was measured on an 11-point scale with endpoints labeled “1 – Very Unlikely” and “11 – Very Likely.” b The "Control" condition was dropped for purposes of the 2x2 ANOVA model. _____________________________________________________________________________ 39