Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
MEASURING THE COST OF POOR CHOICES AMONG RISKY PROSPECTS Daniel McFadden 1 Schaeffer Center for Health Policy and Economics, USC Department of Economics, UC Berkeley December 10, 2013 INTRODUCTION The economic arguments that competitive markets are Pareto-efficient, and that self-organized markets are the standard to beat in applied mechanism design, assume that consumers are rational in their expectations and preference maximization. Tests of revealed preference, and laboratory and field experiments from behavioral economics, indicate that in some circumstances many consumers are not perfectly rational, at least in a narrow neoclassical sense. A natural question to ask is how much damage is done to individual welfare and to the efficiency of resource allocation by imperfectly rational consumers. Some economic evidence supports the idea that most harm from faulty decision-making is selfinflicted, and that consumers learn through market discipline to be self-protective. Studies in financial markets suggest that “noise” traders may harm themselves, but do not degrade resource allocation when rational counter-parties can through leverage offset and correct their positions. Evidence from laboratory experiments is that “irrationality” often errs on the side of caution, where refusals to trade provide satisficing status quo protection against losses. However, there appears to have been little systematic study of the consequences of faulty consumer decision-making. This paper takes a first step, asking how to quantify deviations from perfect rationality of consumer behavior in choice among risky prospects in insurance markets, and how to relate these deviations to losses in individual welfare. I concentrate on insurance markets for three reasons. First, these are markets where expectations are important, and faulty perceptions can be critical. Second, the primary purpose of insurance is to transfer income between states of nature, so that core utility 1 This research is supported by the Behavioral and Social Research program of the National Institute on Aging (grants P01AG033559 and RC4AG039036), and by the Presidential Fund of USC. I am indebted to Florian Heiss, Kevin Murphy, Kenneth Train, and Joachim Winter for useful comments, and to Bo Zhou for computational assistance. 1 should be dominated by expected net benefits, and idiosyncratic tastes for contract terms and providers should be relatively unimportant. Third, consumer choice among uncertain prospects seems to bring out glaring departures from rationality, distortions in expectations, hyperbolic discounting, ambiguity and loss aversion, superstition, and procrastination; see Kahneman and Tversky (1979) and Kahneman (2011). FIGURE 1. The Elements of Decision-Making How can choices among risky prospects break bad? Start from Figure 1, a schematic of consumer decision-making. The “Chicago” model of rationality insists that perceptions be consistent with objective statistical evidence and public facts, without influence from private experience and memory, that the decision-maker have a well-defined, stable utility for outcomes that is not influenced by affect and attitudes, context, or “animal spirits”, and that the process of decision-making be utility-maximization. This model does not question tastes, “de gustibus non e disputantum”, and does not recognize limits of reasoning capacity as a problem for decisionmaking. As one moves toward behavioral views of decision-making, perceptions become subjective, and may be statistically inconsistent (e.g., prospect theory) or excessively sensitive to personal experience and memory (e.g., representativeness, prominence, and primacy/recency effects). Utility may be influenced by affect, context, and “animal spirits”, making it stochastic, 2 or even chaotic, and the idea that risky prospects are evaluated in terms of their expected utility may break down. Finally, heuristics, analogies, satisficing, and other shortcuts may replace utility maximization in the decision-making process. There are three possible reactions to behavioral evidence against the “Chicago” model of consumer behavior. The first is to discard the concept of utility as a factor in choice and a measure of well-being, as in Danny Kahneman’s summation: “Economists have preferences, psychologists have attitudes”. The second is to simply expand the definition of “rationality” to include and excuse any observed choice behavior, the classical view summarized by Taussig (1912): “An article can have no value unless it has utility. No one will give anything for an article unless it yield him satisfaction. Doubtless people are sometimes foolish, and buy things, as children do, to please a moment’s fancy; but at least they think at the moment that there is a wish to be gratified. Doubtless, too, people often buy things which, though yielding pleasure for the moment, or postponing pain, are in the end harmful. But here ... we must accept the consumer as the final judge. The fact that he is willing to give up something in order to procure an article proves once for all that for him it has utility – it fills a want.” The third is to take the “benevolently paternalistic” view of Thaler and Sunstein (2008) that an intelligent observer can ascribe utilities to individuals, detect behavior that is inconsistent with reasonable self-interest (e.g., errant perceptions, erratic decision processes, and even expected utility maximizing behavior with inappropriate utility functions), and determine when “nudges” are justified to “buck up” decision-making and reduce the likelihood that people harm themselves or degrade market efficiency. There are difficulties with all three of these reactions. The first two effectively preclude pursuit of a consumer-side parallel to the theory of industrial organization that would study whether consumer conduct in markets is consistent with efficient allocation, and what market rules promote healthy conduct. The third response is disturbing to people who fear Orwellian erosion of consumer sovereignty. And it is true that while I think I could improve your choices, I am skeptical that you could improve mine. However, the third response can be viewed more benignly as an extension of social judgments that already establish limits on behavior, such as laws that mandate primary education and forbid recreational drug use, or consumer protection legislation that provides transparency and limits deception. The approach of this paper is in the spirit of ThalerSunstein, starting with an assumption that people have, or can be imputed to have, core utilities that are sufficiently stable and context-free to provide a basis for welfare judgments, but also make 3 decisions cluttered by errant perceptions and careless optimization that do not necessarily maximize or reveal core preferences. Key questions are then what can be learned about core utility from observed behavior, and what market interventions can improve core consumer welfare. How can the quality of choices among risky prospects be evaluated? The man who buys a lottery ticket feels he made a good choice if it wins, and a bad choice if it loses. However, this ex post evaluation is neither a good predictor of future payoffs nor a reliable metric for judging whether it was ex ante rational to buy the ticket. More useful is an answer to the question: “Using statistically realistic expectations, how did expected utility from actual choices compare with expected utility that would have been obtained from a benchmark decision-making rule that used only information available at the time of choice?” If an answer can be phrased in terms of consumer surplus foregone, or excess cost of obtaining the benchmark outcome, then performance relative to the benchmark can be used as a measure of welfare loss, and used to guide the design of managed markets. The most rigorous neoclassical benchmark decision rule is “expected utility maximization with rational expectations”, but of more practical interest are heuristics and decision aids promoted by market managers that “nudge” consumers toward “boundedly rational” decisionmaking. SOME CONSUMER THEORY Suppose a consumer faces a binary choice between self-insuring against a specific casualty loss and buying an insurance contract that provides complete coverage in the event of the loss. What are the implications of neoclassical consumer theory if individuals have core utilities, but faulty perceptions? A first issue is entanglement of perceptions and tastes for risky prospects: Do consumer’s choices among lottery tickets reflect risk preferences, or misperceptions of the probabilities of payoffs? Prospect Theory, the Kahnemann and Tversky (1979) summary of empirical regularities in choice experiments, notes that people are inconsistent in the editing process that defines their “status quo”, the perception of small probabilities, and the manipulation of the probabilities of events. Another element of their summary is representation of choice as a maximand of “expected” value function using consumers’ errant probabilities and a valuation function that has a “fourfold pattern of risk attitudes”, as depicted in Figure 2, with loss aversion, risk aversion for modest gains, risk affinity for modest losses, and reversed risk attitudes for large gains or large losses. 4 Figure 2. The Valuation Function from Prospect Theory Losses Gains However, a characterization that makes these effects entirely a result of errant perceptions has consumers maximizing stable core utilities with mild risk aversion that are insensitive to the status quo, combined with superstitious perceptions that conservatively underestimate the moderate probabilities of small gains or losses from the status quo, with more underestimation for gains, but overestimate or ignore very small probabilities of large gains or losses. This interpretation is a best case for economic analysts of errant consumers, as the core utility foundations for the welfare calculus remain intact, and the analyst’s task is reduced to clearing away the effects of faulty perceptions and optimization errors to recover core preferences. However, a deeper question is whether ambiguity in distinguishing perceptions and preferences makes it necessary to resort to attribution of core preferences by a social planner, or recovery of core utility through experiments that directly reveal perceptions in various contexts. Proceeding with the assumption that core utility is meaningful, consider a consumer facing a voluntary decision between insuring completely at a premium C against a casualty loss, or selfinsuring and facing an uncertain loss L. Let G denote the objective CDF for L. Suppose the 5 consumer has a core CARA utility function of disposable income with a risk attitude parameter . For a fully rational consumer, the utility of buying the insurance contract is [1 – e-(y-C)]/, where y denotes income, and the expected utility of self-insuring is [1 – ∫ e−(y−L) 𝐺𝐺(𝑑𝑑𝑑𝑑)]/. Let m(t) = log ∫ etL 𝐺𝐺(𝑑𝑑𝑑𝑑) denote the cumulant generating function of the CDF G, with an expansion m(t) = t+ 2t2/2 + O(t3), where is the mean and 2 is the variance of L. The fully rational consumer buys the insurance contract if [1 – e-(y-C)]/ > [1 – e-(y-m()/)]/, or in other words if the premium C is less than the certainty-equivalent actuarial value R m()/ = + 2/2 + O(2) of the income loss when self-insured, C < R. Now suppose consumers have errant perceptions that selfinsuring results in a certainty-equivalent expected income loss Q, and buy insurance when C < Q. The values (R,Q) are heterogeneous in the population; let H(R,Q) denote their CDF. The table below gives the possible outcomes and consequences: Condition Choice Optimal Choice? Penalty Probability 1. Q ≤ C & R ≤ C Self-insure Yes 0 H(C,C) 2. Q > C & R ≤ C Buy No R-C H(C,) – H(C,C) 3. Q ≤ C & R > C Self-insure No C-R H(,C) – H(C,C) 4. Q > C & R > C Buy Yes 0 1 - H(C,) - H(,C) + H(C,C) Rows 2 and 4 are conditions where the consumer buys insurance; the population share buying insurance is 1 – H(,C). Rows 3 and 4 are conditions under which buying insurance is optimal; the share optimally insured is 1 – H(C,). When consumers self-insure, rows 1 and 3, their ex ante perception was a net certainty-equivalent income gain C – Q > 0. These consumers experience ex post net income shocks C – L relative to buying insurance, but these measures of regret are confounded by luck and provide unreliable evidence to correct perceptions. The mean of C – L among self-insurers is C – E(R|Q>C), while they expect C – E(Q|Q>C). Then, E(Q-R|Q>C) is a mean surprise to the self-insured, but one they will not generally learn since Q’s are not reliably reported public information. A benchmark for the value of insurance is the mean expected certainty-equivalent income loss if no one insures, 6 (1) − ∫0 𝑅𝑅𝐻𝐻(𝑑𝑑𝑑𝑑, ) = − ∫0 �1 − 𝐻𝐻(𝑅𝑅, )�𝑑𝑑𝑑𝑑. Rows 2 and 3 are conditions where faulty perceptions lead to sub-optimal choices. Consumers in row 2 incur a mean certainty-equivalent income penalty (2) 𝐶𝐶 ∫𝑄𝑄=𝐶𝐶 ∫𝑅𝑅=0(𝑅𝑅−𝐶𝐶)𝐻𝐻(𝑑𝑑𝑑𝑑,𝑑𝑑𝑑𝑑) H(C,) – H(C,C) = 𝐶𝐶 ∫𝑅𝑅=0(𝑅𝑅−𝐶𝐶)𝐻𝐻(𝑑𝑑𝑅𝑅,)−𝐻𝐻(𝑑𝑑𝑅𝑅,𝐶𝐶)) H(C,) – H(C,C) =− 𝐶𝐶 ∫𝑅𝑅=0(𝐻𝐻(𝑅𝑅,)−𝐻𝐻(𝑅𝑅,𝐶𝐶))𝑑𝑑𝑑𝑑 H(C,) – H(C,C) . Consumers in row 3 incur similarly a mean certainty-equivalent income penalty (3) 𝐶𝐶 ∫𝑄𝑄=0 ∫𝑅𝑅=𝐶𝐶(𝐶𝐶−𝑅𝑅)𝐻𝐻(𝑑𝑑𝑅𝑅,𝑑𝑑𝑑𝑑) H(,C) – H(C,C) = ∫𝑅𝑅=𝐶𝐶(𝐶𝐶−𝑅𝑅)𝐻𝐻(𝑑𝑑𝑑𝑑,𝐶𝐶) H(,C) – H(C,C) = − ∫𝑅𝑅=𝐶𝐶(𝐻𝐻(,𝐶𝐶)−𝐻𝐻(𝑅𝑅,𝐶𝐶))𝑑𝑑𝑅𝑅 H(,C) – H(C,C) . To illustrate the calculation, consider enrollment by seniors in 2008 in the Medicare Part D drug insurance program. This insurance exchange offers a standard contract at a heavily subsidized, competitively determined premium. Let T denote a consumer’s total drug bill in 2008, a random variable at the time of an enrollment decision at the time of open enrollment at the end of 2007, and let T-1 denote the 2007 total drug bill, which we assume was known to the consumer at the time an enrollment decision for 2008 was made. For this illustration, we ignore other information available to consumers when they make their enrollment choices, such as age, gender, and health conditions; this is empirically reasonable since the correlation of total drug bills across years is about 0.8. A Part D standard plan has a premium C and a benefit schedule B(T) = 0.25min(2325,max(0,T-275)) + 0.95max(0,T-5726.75). 2 Make the assumption, consistent with considerable evidence, that the consumer is essentially neutral to risks in drug needs, and assume the limiting risk parameter = 0. Then the certainty equivalents (R,Q) that enter the model above are simply expected losses relative to the partial insurance provided by the Part D standard plan; R = E(B(T )| T-1), and it is optimal to buy insurance when its actuarial value exceed the premium, R > C. I estimate H(R,|T-1) and the density f(T-1) of T-1 from a sample of 991,930 seniors whose enrollment in a Part D plan is “mandatory” and hence not selected by enrollment choice, the result of inclusion in a retiree or union health plan, or in Medicaid or Medicare Advantage coverage. I 2 This calculation ignores the presence of actuarially equivalent and enhanced alternatives to the standard plan that are offered at an unsubsidized additional premium, and ignores premium variations by region and low-income subsidies. 7 estimate H(R|Q>C,T-1) and the density f(T-1|Q>C) from a sample of 1,926,035 seniors who voluntarily enrolled in a stand-alone Part D drug plan in 2008. The population share 1 - H(,C) of voluntary decision-makers who enroll is a number that can be inferred from surveys or from Medicare market statistics. Then, 1 – H(,C|T-1) = (1 – H(,C))f(T-1|Q>C)/f(T-1) is the share of voluntary decision-makers who enroll, given T-1. Cautions about the realism of this example are that it does not take into account penalties for late enrollment that make enrollment attractive even if it is first-year actuarially unfair, it uses a calibration step is used to reconcile the somewhat different distributions of health status and drug needs between the populations of mandatory and volunteer enrollees, and it calibrates the average premium to match the statistic that 76 percent of seniors with an voluntary choice buy Part D insurance. The model then establishes that it is optimal for 95.6 percent of consumers to buy insurance, 1.8 percent optimally self-insure, 2.6 percent buy insurance when it is a poor choice, 22.2 percent self-insure when this is a poor choice, and 73.4 percent buy insurance and it is optimal. The mean penalty per capita for the small share of people making a poor choice to buy insurance is $50, while the mean penalty for the relatively large share of people making a poor choice to self-insure is $172, or about 13.7 percent of their mean total drug bill. Thus, poor choices in this example lead to an economically significant loss in demand for insurance, and an economically significant loss in individual welfare. BENCHMARK ASSESSMENT OF EX ANTE EXCESS COST OF POOR INSURANCE PLAN CHOICES In this section, I describe a benchmark method for ex ante assessment of the quality of insurance contract choices, with an application to plan choices of volunteer enrollees in the Medicare Part D drug insurance exchange. The application is drawn from Heiss, Leive, McFadden, and Winter (2013). Plan choices are made during an open-enrollment period at the end of each year for the contract that will be in place during the coming year, before actual health conditions and drug needs in the coming year are realized. Useful evaluations of the quality of plan choice have to be correspondingly ex ante, based on expected utility in the year ahead, given information available to the decision-maker at the time of choice. In overview, we assess whether people could have achieved higher expected utility than they did if instead of their actual decision rule, they had used a benchmark rule that required only information they had available at the time of their actual plan choice. A leading benchmark, and 8 the focus of our empirical analysis, is the recommendation of the Medicare’s “Plan Finder”, an internet tool that ranks the cost of plans in the coming year when given a person’s medicine cabinet for the current year. This decision rule is available to any senior who can access the Medicare website, personally or through a helper. Other possible benchmark rules are naive heuristics such as “choose a minimum premium plan”, and the fully rational decision rule that maximizes statistically realistic expected utility conditioned on information available at the time of choice. At the time of plan choice in an open enrollment period at the end of year t-1 consumers have a vector of information, denoted Xt-1, that includes their age, gender, health conditions, a list of their current drugs and doses which we term their medicine cabinet (MC), their pharmacy total drug bill for the year (T), and if they have one, their current Part D plan and its premium (C) and realized out-of-pocket (OOP) cost. Each consumer is assumed to have sufficient information, from the Medicare Plan Finder or otherwise, to determine the formulary and benefit design mapping for each available Plan k in year t that gives the OOP for any given medicine cabinet MC; we denote this mapping OOP = FBD(MC,k,t). We as analysts can also construct FBD from plan attributes and drug prices. Given Xt-1, there is a statistically realistic conditional density fR(MC | Xt-1) of year t medicine cabinets. Assume that consumers have CARA utility functions Ukt = (1 – exp(-α(y – Ckt – OOPkt)))/α, where y is discretionary income, Ckt is the (known) insurance premium Ckt for Plan k in year t, and OOPkt = FBD(MC,k,t)) is the (uncertain) cost out-of-pocket cost that will be realized in year t if this plan is chosen. Rational expected utility is then (4) EUkt = fR(MC | k,Xt-1) (1 – exp(-α(y – Ckt – FBD(MC,k,t)))) /α = (1 – exp(-α(y – Ckt – mR(α | k,Xt-1)))/α, where mR(z | k,Xt-1) = log ∑ exp(z FBD(MC,k,t)) fR(MC | k,Xt-1) is the conditional cumulant generating function for plan k. When α is small, mR(α | k,Xt-1)/α ≈ R(k,Xt-1), the conditional mean OOP cost for plan k in year t. Then expected utility is a decreasing function of Ckt + R(k,Xt-1). Let kA(Xt-1) denote the consumer’s actual plan choice, and IAt(Xt-1) = 𝐶𝐶𝑘𝑘𝐴𝐴(𝑋𝑋𝑡𝑡−1 ),𝑡𝑡 + 𝑅𝑅 (k A (Xt−1 ) , 𝑋𝑋𝑡𝑡−1 ) denote the associated expected cost. Let kB(Xt-1) denote a benchmark rule plan choice, and IBt(Xt-1) = 𝐶𝐶𝑘𝑘𝐵𝐵 (𝑋𝑋𝑡𝑡−1 ),𝑡𝑡 + 𝑅𝑅 (k B (Xt−1 ) , 𝑋𝑋𝑡𝑡−1 ) denote its associated expected cost. 9 We judge ex ante quality of actual choice compared to a benchmark rule by comparing IAt(Xt-1) and IBt(Xt-1). As noted earlier, we assume is near zero and higher moments can be ignored. 3 We use two criteria, the excess expected cost IAt(Xt-1) - IBt(Xt-1), and an indicator for whether the consumer’s actual choice has an expected cost greater than that of the benchmark rule, IAt(Xt-1) > IBt(Xt-1). Making these assessments for an individual requires evaluation of the realistic ex ante conditional expected OOP cost R(k,Xt-1), a difficult inference problem. However, this individual’s ex post realized OOP cost in plan k , OOPtk determined by her realized medicine cabinet, by definition satisfies E(OOPtk | k,Xt-1) = R(k,Xt-1). Then, averaging over the population, we have (5) Share with overspending = 𝑬𝑬𝑋𝑋 𝑡𝑡−1 1(IAt(Xt-1) > IBt(Xt-1)) = E 1(CAt + OOPAt > CBt + OOPBt), (6) Average overspending = 𝑬𝑬𝑋𝑋 𝑡𝑡−1 ( IAt(Xt-1) – a IBt(Xt-1)) = E (CAt + OOPAt - CBt - OOPBt). The final equalities are simply ex post unconditional expectations over the population that are estimated by their empirical equivalents, year t averages over a representative sample of voluntary decision-makers. The distinction between ex ante and ex post performance metrics comes from whether or not the benchmark is feasible, since for a given benchmark the estimate of expected excess spending in the ex ante calculation is identical to the average of regret in an ex post calculation. If the benchmark is feasible, using only information the consumer has and requiring only practical calculation, then it provides a useful ex ante standard against which to judge actual choice behavior. On the other hand, if the benchmark is infeasible, for example the perfect foresight rule that requires the consumer to know exactly what drugs needs will be in the coming year, it is potentially quite misleading, as even a consumer who makes a fully rational expectedutility maximizing choice can have large wistful regret relative to the perfect foresight benchmark when uncertainty is high. However, differences in average ex post excess spending, relative to the perfect foresight benchmark, between actual choice and a feasible benchmark choice will coincide 3 Heiss et all (2013) estimate a discrete choice model for plan choice as a function of premium, expected OOP cost, and expected OOP variance, and find that premium and expected OOP cost are weighted similarly, with somewhat more weight on expected OOP cost, but that the coefficient of OOP variance is volatile and often inconsistent with risk aversion. The leading interpretation is that enrollees are unable to assess variance, and show some preference for higher variance plans because these are associated with desirable plan features such as elimination of initial deductions. Another possibility is that Part D plans fall in a region where underestimates of probabilities of modest losses lead to apparent risk affinity, as in prospect theory. 10 with ex ante average excess spending. (This result depends on linearity, and does not hold for calculations of shares with excess spending.) We carry out the calculations described above on a sample of 1,138,105 voluntary enrollees in Part D in 2008 whose plan choices are not circumscribed by qualification into low-income support programs, restricted by employer or union based health plans, or by enrollment in Medicare Advantage plans; details of the sample are given in Heiss et al (2013). Construction of the FBD mapping and determination of OOPkt for each enrollee in each available plan was a major undertaking described in Hess et al (2013). To avoid selection effects, we use our calculated value for OOP even when it is observed in the plan actually chosen. When FBD incorporates therapeutic substitution, replacing drugs in a medicine cabinet by the least costly therapeutic equivalent in the formulary of a plan under examination, we find that 80.3 percent of actual plan choices in 2008 had higher cost than the Plan Finder benchmark rule, and that the average excess cost was $314, 25.8 percent of the mean OOP cost $1214 of voluntary choosers. We conclude that poor plan choice is economically significant and results in substantially higher excess drug costs relative to the Plan Finder decision rule. We also considered other benchmarks, the naïve “minimize premium” rule, and our version of a “rational expectations” rule that first estimated R(k,Xt-1) by a “method of sieves”, and then chose a plan that minimized Ckt + R(k,Xt-1). We found that 61.6 percent of actual choices are worse than the minimize premium rule, with higher mean excess drug cost of $121. We also found that our version of a rational expectations rule performed only slightly better than Plan Finder: 80.6 percent of actual choices did worse than the rational expectations rule, and the mean excess cost relative to this rule was $349. To summarize, even simple decision rules such as “choose the lowest premium plan” or “choose the plan suggested by Plan Finder” would have generated considerably lower ex ante expected spending than the actual plan choices in the Part D market. More sophisticated rules, such as our version of a rational decision rule that conditions on a small set of observable end-ofprior-year health and drug use characteristics, would have brought moderately larger savings. It is hard to reconcile these monetary losses with the implicit decision costs associated with using a tool such as Plan Finder, or with true idiosyncratic tastes for specific insurers and benefit designs. In this application, there seems to be a strong case for nudging consumers toward lower cost plans, both for their own benefit, and to sharpen their choices so that poor plans are pressured to reduce their premiums or exit the market. Heiss et al (2013) suggest, for example, that Medicare take a 11 lesson from Geico’s popular advertisement for its car insurance, and tell consumers that “15 minutes with Plan Finder could save you 15 percent or more on your prescription drugs.” . References Heiss, F.; Leive, A.; McFadden, D.; Winter, J. (2013) “Plan Selection from Part D: Evidence from administrative data,” Journal of Health Economics, 32, 1325–1344. Kahneman, Daniel. Thinking, Fast and Slow. New York: Farrar, Strauss and Giroux, 2011. Kahneman, Daniel, and Amos Tversky. "Prospect Theory: An Analysis of Decision Under Risk". Econometrica. XLVII (1979): 263-291. McFadden, D. (1999) “Computing Willingness to Pay in Random Utility Models,” in J. Moore, R. Riezman, and J. Melvin (eds.), Trade, Theory and Econometrics: Essays in Honour of John S. Chipman, Routledge. 12