Download Evaluating Frameworks That Provide Value Measures for

VALUE IN HEALTH 20 (2017) 185–192
Available online at www.sciencedirect.com
journal homepage: www.elsevier.com/locate/jval
Evaluating Frameworks That Provide Value Measures for Health
Care Interventions
Jeanne S. Mandelblatt, MD, MPH1,2, Scott D. Ramsey, MD, PhD3, Tracy A. Lieu, MD, MPH4,
Charles E. Phelps, PhD5,*
1
Department of Oncology, Georgetown University Medical Center, Washington, DC, USA; 2Lombardi Comprehensive Cancer Center,
Washington, DC, USA; 3Fred Hutchinson Cancer Research Center, Seattle, WA, USA; 4Division of Research, Kaiser Permanente
Northern California, Oakland, CA, USA; 5University of Rochester, Rochester, NY, USA
AB STR A CT
The recent acceleration of scientific discovery has led to greater
choices in health care. New technologies, diagnostic tests, and
pharmaceuticals have widely varying impact on patients and populations in terms of benefits, toxicities, and costs, stimulating a
resurgence of interest in the creation of frameworks intended to
measure value in health. Many of these are offered by providers and/
or advocacy organizations with expertise and interest in specific
diseases (e.g., cancer and heart disease). To help assess the utility of
and the potential biases embedded in these frameworks, we created
an evaluation taxonomy with seven basic components: 1) define the
purpose; 2) detail the conceptual approach, including perspectives,
methods for obtaining preferences of decision makers (e.g., patients),
and ability to incorporate multiple dimensions of value; 3) discuss
inclusions and exclusions of elements included in the framework, and
whether the framework assumes clinical intervention or offers
alternatives such as palliative care or watchful waiting; 4) evaluate
data sources and their scientific validity; 5) assess the intervention’s
effect on total costs of treating a defined population; 6) analyze
how uncertainty is incorporated; and 7) illuminate possible
conflicts of interest among those creating the framework. We apply
the taxonomy to four representative value frameworks recently
published by professional organizations focused on treatment of
cancer and heart disease and on vaccine use. We conclude that
each of these efforts has strengths and weaknesses when evaluated
using our taxonomy, and suggest pathways to enhance the utility
of value-assessing frameworks for policy and clinical decision
making.
Keywords: cost-effectiveness, multi-attribute decision analysis, value
frameworks.
Introduction
to evaluate exemplar value frameworks to illustrate strengths,
gaps, and potential concerns. We intend for this synthesis to
stimulate discussion about best practices and use of value
metrics in policy and clinical decisions. We derived these principles in our synthesis of standard practice for the reporting of
CEAs [23–25], recent International Society for Pharmacoeconomics and Outcomes Research summaries of multicriteria decision
analysis (MCDA) reporting guidelines [26], standard medical
journal requirements for conflict of interest disclosures [27],
standard Cochrane review methods [28], and our own analysis.
Although we draw many of these ideas from others, assembling
them to evaluate models that estimate the value of health care
is novel.
The explosion of scientific knowledge, discovery, and technology
began a new era of health care, wherein deployment of health
care interventions depends on personal, lifestyle, genetic, and
molecular disease factors. Although these advances have the
potential to improve health, they increase the complexity of
analyses to determine value at reasonable and affordable costs
[1–8].
Faced with new health care interventions and escalating costs
of care, many professional organizations have developed frameworks to measure and/or rank the value of novel interventions
[3,7,9–22]. Some of these new frameworks use components of
cost-effectiveness analysis (CEA), whereas others create de novo
value measures. It remains unclear how such frameworks will
guide clinical practice policy, and whether they offer particular
advantages over traditional approaches such as CEA.
First, we summarize a taxonomy of principles—expanding
upon cost-effectiveness theory—that provides an approach to
assess different value frameworks. Next, we apply this taxonomy
Copyright & 2017, International Society for Pharmacoeconomics and
Outcomes Research (ISPOR). Published by Elsevier Inc.
Value Taxonomy
Value frameworks can assist in public health prioritization,
practice guidelines, formulary or related resource expenditure
decisions (e.g., purchase of equipment or coverage of a specific
* Address correspondence to: Charles E. Phelps, 30250 South Highway 1, Gualala, CA 95445.
E-mail: [email protected].
1098-3015$36.00 – see front matter Copyright & 2017, International Society for Pharmacoeconomics and Outcomes Research (ISPOR).
Published by Elsevier Inc.
http://dx.doi.org/10.1016/j.jval.2016.11.013
186
VALUE IN HEALTH 20 (2017) 185–192
Table 1 – Proposed taxonomy for evaluation of frameworks assessing the value of health care interventions.
Category
Purpose
Conceptual approach, perspective,
and preferences
Intervention components and
comparators
Data sources
Economics/costs
Uncertainty and identification of
important gaps
Conflicts of interest
Component details
Is the purpose defined and are the elements of the framework consistent with the purpose?
Is the value structure of the framework clear and transparent? If it requires assumptions, do they
have general and clinical face validity?
What perspective does the framework use (e.g., societal, patient, provider, payer)? Is the perspective
made explicit or embedded in the framework?
Does the framework include standard methods of measuring value, such as cost-effectiveness
analysis?
Does the framework consider multiple attributes of medical interventions such as individual
preferences, equity and the distribution of health care benefits and costs, issues involving
externalities (such as contagious diseases), and the value of scientific breakthrough?
If the framework includes multiple dimensions of value, are the weights for each component
population-based or expert-provided, and are the methods for eliciting such weights both clearly
stated and methodologically sound?
Does the framework allow individual patient preferences? If so, are they elicited with methods
known to be free of bias and to produce reliable results?
Does the framework include all components and consequences of the intervention, or merely a
portion of those (e.g., drug acquisition costs)?
Does the framework aggregate or disaggregate such things as toxicity or other side effects of
intervention?
Does the framework assume as a baseline that some intervention will be provided, or does it allow
for “watchful waiting” or “palliative care” as an option?
Are clinical and other data derived from expert opinion, population surveys, or other sources? Are
the sources made clear, and is the process replicable?
What is the effect of the intervention on the total cost of treating a defined population, including
whether inclusion of the intervention will increase, decrease, or leave unchanged the total costs of
care for that population?
Is uncertainty in data about costs or effects considered in conclusions about value? Are there gaps in
knowledge about aspects of care that could change the value rating?
Does the sponsoring organization have a financial stake in the process, and if so, is this declared?
Does the framework bias the results in favor of the sponsor’s financial position?
drug), and patient-physician decision making about a specific
care choice. Table 1 describes our taxonomy to evaluate these
value frameworks. We intend for this to apply to these different
purposes, but specific elements may be more or less salient for
different uses of the value framework. The following section
highlights the implications of these varying purposes when using
this taxonomy to evaluate value frameworks.
Conceptual Approach, Perspective, and Preferences in
Measuring Value
Value frameworks have evolved, in part, to provide greater
standardization and transparency in health care decision making
[29]. Furthermore, formal frameworks can minimize biases of
intuitive decision making from issues of framing of the problem,
risk analysis, intertemporal trade-offs, and others [30].
We began our taxonomy in the formal framework of CEA.
CEAs most commonly use the societal perspective, but this
perspective usually does not correspond to that held by individual participants in health care decisions—patients, providers,
payers, public policy decision makers, or owners of resources
used to create health care interventions—nor does it consider
community well-being or other important policy issues such as
the distribution of benefits and costs across various subsets of the
population (among other omissions).
No single perspective can represent the interests of all participants in value-based decisions. For example, physicians generally
advise patients using a patient perspective, but may introduce the
societal perspective in their role as the steward of societal
resources [31] or, in some cases, their own professional organization’s or personal financial situation. Thus, we believe that
multiple perspectives should be considered, varying by individual
or organization assessing the intervention, consistent with the
most recent recommendations for reporting CEAs [24,25].
CEA can also be useful in developing value frameworks
because it provides data on intermediate events of importance
to judging the value of an intervention, such as toxicity rates
and overall survival, measured in life-years saved, qualityadjusted life-years, or disability-adjusted life-years [23–25,32,33].
Compared with the existing care standard, the ratio of added
costs to added gains in health benefits—the incremental costeffectiveness ratio—stands as one value criterion. Nevertheless,
no consensus exists on cutoff points to determine the “acceptability” of a given incremental cost-effectiveness ratio, and both
illness severity and the number of people affected can influence
“cutoff point” choices.
CEA also does not typically capture less tangible factors such
as the fear associated with diseases such as HIV, cancer, degenerative neurological conditions, or mental illness, because these
do not represent distinct health outcome states for which utility
measurements exist. In concept, CEA could incorporate such
issues, but in many cases, the data requirements would be
impossibly complex. Full CEA evaluation requires populationbased utility-level estimates for every state of nature in the
model. Capturing such things as fear of the diseases, contagion
effects, restrictions on freedom, and modification of religious
rites (all arising in response to the Ebola virus) provides examples
of issues that CEA cannot meaningfully capture.
Although grounded in CEA, our taxonomy extends beyond
this foundation. CEA does not generally consider many other
attributes of decision making that affect the use of health
care interventions. For example, CEA cannot address the
VALUE IN HEALTH 20 (2017) 185–192
distribution of health burdens that might fall particularly
heavily on disadvantaged populations (e.g., sickle cell anemia
in African Americans), or whether some aspects of a medical
intervention have important moral or ethical components
(e.g., abortion after detection of a fetal genetic mutation with
profound consequences).
Applying CEA for decision making brings another issue to the
foreground: a new technology may pass standard “cutoff”
requirements but then overwhelm the available budget—the
issue of “affordability.” CEA methods are at present not equipped
to deal with this issue.
MCDA tools offer one approach to formally include these
“other issues” into the value framework. These tools include
such approaches as the analytic hierarchy process [34] and the
multi-attribute utility theory [35,36], both used in recent health
care decision processes [37–40].
These models bring to the foreground an issue that does not
arise in CEA: How does one balance among competing goals in a
value model? MCDA tools use various methods to elicit weights
attached to each (potentially) competing goal. With a single
decision maker, these methods work well in general [34,36], but
the best methods to combine individual into group preferences
remain elusive. If group estimates are not derived from relevant
population samples, our framework suggests that analysts
should provide good justification and clear logic for using alternative sources.
Our taxonomy proposes that value frameworks incorporate
preferences, examine multiple perspectives, and are transparent.
In addition, we suggest that value frameworks should consider
distribution and ethical issues and other pertinent factors by
using MCDA methods.
Accounting for All Components of the Health Condition and
Intervention
Value frameworks should consider many aspects of each disease
process and all short-term and downstream benefits and harms
of interventions. For example, in cancer, including only one cost
component, such as drug acquisition costs for a new treatment,
would bias the overall assessment by ignoring other costs, such
as those of drug administration, supportive care, and other
aspects of therapy. In the treatment of HIV, cancer, and other
conditions, drugs may have significant toxicity that can affect
their value. Screening tests such as mammography can save
lives, but also carry with them meaningful rates of false-positive
results and overdiagnosis. These issues should enter a full value
assessment, not only in health status outcomes and costs but
also in emotional burdens created [41,42].
Some value frameworks treat all harms as equal in consequence to patients or other decision makers. For example,
when the intervention is a drug treatment, rates of all side
effects might be used in comparing the value of two therapies.
This creates biases when the intensities and/or durations of
the side effects differ, or when some effects have greater
consequence than do others to individual patients (e.g.,
nausea/vomiting vs. fatigue vs. increased risk of iatrogenic
death). In best practice, our taxonomy recommends that
individual preferences on these issues inform the overall
assessments of alternative therapies, and that when infeasible,
population-based estimates serve as an alternative.
Present value frameworks tend to focus on active therapy
(e.g., drugs, surgery, and/or invasive tests). More recently, for
certain diseases, such as early, nonaggressive prostate cancers,
with invasive and potentially toxic standard-care therapies,
alternatives now include no therapy (active surveillance). In the
advanced disease setting, palliative interventions are being
increasingly considered. If value frameworks do not explicitly
187
consider these less aggressive approaches (i.e., only comparing
novel therapies with standard therapy), the resulting decision
may be biased toward therapeutic action, possibly conflicting
with patient preferences. Therefore, we suggest that value frameworks compare all feasible options for patients, not just those
using therapeutic intervention.
What are the sources of and process for use of data in the
framework? Does this process conform to accepted methods for
evidence evaluation?
Several accepted methods and hierarchies evaluate the quality
of evidence about health care intervention effects. Randomized
controlled trials generally provide stronger evidence than
do other methods (e.g., historical comparisons, case control
studies, and others), and sample sizes and research design in
all studies affect precision and accuracy of parameter estimates [28].
In cases in which good-quality, empirically based evidence
on key issues in the value framework is not available, expert
judgment may provide the only realistic option. Using expertbased estimates creates biases when the experts have a professional or financial stake in choices being evaluated (e.g.,
profit from sales of a device or drug, or enhanced reimbursement for delivery of a new agent [see later for further discussion]). When possible, valuation models should use experts
not in conflict of interest situations and should clearly identify
when any such conflicts exist.
Does the framework include the full economic consequences of
all intervention components?
As in standard CEA, our taxonomy recommends that value
frameworks include the full cost of all the components and
consequences of an intervention. For example, in frameworks
to evaluate drug treatments, economic impacts would include
drug costs, provider time and delivery costs, staff time, facilities
and equipment overhead and costs, costs of treatment-related
side effects and supportive care medications required, patient
time and travel costs, and costs of all downstream events until
death from the disease or other causes. To the extent that
frameworks exclude portions of these costs, they result in biased
recommendations.
Many value frameworks have different perspectives and
include different costs. Patient-perspective frameworks, for
example, might normally include co-payments made by patients
only. Payer-centric frameworks may focus only on the costs and
savings to the health care provider. These approaches can lead to
suboptimal value-based decisions from a societal perspective.
Costs also need to be considered in the context of
extant budgets. For example, hepatitis C medications have
cost-effectiveness ratios within the range of other accepted
interventions, but still have high absolute costs that may
overwhelm pre-established budgets.
Projected budget impact has played a key role in some health
care system decisions about these medications, mostly appearing
as an independent discussion of budget impact in addition to
standard CEA [17]. We believe that this approach misses a key
point: budgets and acceptable levels for cost-effectiveness ratios
cannot be considered independently. Introducing a new (and
expensive) treatment adds budget pressure, which invariably will
raise the value of incremental resources (what economists call
the “shadow price”). The proper solution will likely involve reassessment of the existing budget and re-examination of interventions that cannot compete (in a cost-effectiveness paradigm)
with the newly tightened budget situation.
188
VALUE IN HEALTH 20 (2017) 185–192
How does the framework account for uncertainty in effect or cost
data?
Many analytic methods incorporate effects of uncertainty on the
conclusions of a value recommendation. A key aspect of characterizing uncertainty relates to identification of gaps in knowledge
or evidence about aspects of care that could change the value
conclusions. Our taxonomy does not specify a particular method
to identify the impact of uncertainty, but rather that uncertainty
and gaps in data be explicitly evaluated.
Conflicts of Interest
Professional specialty organizations often have the best knowledge about a particular set of health care interventions, and thus
are well placed to develop value frameworks, but may also have
conflicts of interest in these issues. Value assessments may be
subject to bias if financial incentives favor more (and costlier)
interventions in certain delivery settings (e.g., fee-for-service vs.
managed care or accountable care organizations). These issues
extend also to the use of facilities owned in part or whole, directly
or indirectly, by decision-making providers.
Beyond financial incentives, the purpose of specialist organizations is to advocate for professional authority and action,
which may further bias these sponsors toward interventions
rooted in their specialties (such as chemotherapy, which is rooted
in oncology, compared with palliative care, with separate specialty affiliations). Thus, we should consider whether the value
framework’s sponsoring organization has a professional stake in
the process, and if so, whether the framework potentially biases
the results in favor of the sponsor’s position.
Examples of Recent Frameworks
Value frameworks have recently emerged from different medical
specialty groups representing oncology [9,16–22], cancer screening [10], radiation oncology [11,12], cardiology [7], surgery [13],
pediatrics [3], and infectious disease and vaccines [14]. To
demonstrate the application of our taxonomy, we selected four
exemplar value frameworks spanning several specialties
(Table 2).
Oncology
The oncology frameworks largely focus on the value of new
drugs, driven by the more rapid escalation of drug costs relative
to other components of cancer care, and the impact of drug costs
on patient co-payments [31,43,44]. In other disease specialties,
new treatment paradigms, expensive drugs, and high-priced
technology have also stimulated the development of value
frameworks.
The American Society of Clinical Oncology’s Task Force on
Value in Cancer Care [18,19] and the National Comprehensive
Cancer Center Network [16] both developed value assessments
for systemic therapy drugs. Comments about this work recognize
that the intended use centers on patient-based treatment decisions rather than on a societal perspective. These frameworks
followed many elements recommended in our taxonomy, but
they omit several key value components. First, they focus only on
clinical outcomes related to survival and drug costs. Second, by
limiting the framework to clinical trial evidence about episodes of
drug treatment, they also do not permit assessment of the value
of a given therapy in the general population [45]. Third, the
framework excludes other important costs of care, such as future
need for radiation, repeat surgery, and palliative care. Fourth,
many relevant outcomes needed to determine value from a
societal or patient perspective are excluded, including the impact
of therapy on quality of life [46], perhaps as a result of omission
of such data in the original clinical trials of efficacy [47]. The
revised framework of the American Society of Clinical Oncology
[19] endeavors to capture all types of toxicity, but can succeed
only to the extent lower grade adverse events are reported.
Next, the method used to assign value points is both arbitrary
and relies primarily on opinions of providers rather than samples
of patients or their families. Finally, there is no accounting for
uncertainty in the underlying data or the result, giving a false
sense of precision when comparing among treatments.
Other oncology frameworks include DrugAbacus [48], the
European Society for Medical Oncology’s Magnitude of Clinical
Benefit Scale [22], and the pan-Canadian Oncology Drug Review
[20,21]. These frameworks seek to inform policy rather than
clinical decision making. They too focus primarily on drugs, with
DrugAbacus evaluating cost of drug development and production
versus actual retail pricing. The pan-Canadian Oncology Drug
Review is the only value framework in oncology that includes a
societal perspective, patient stakeholders, and components of a
CEA framework including quality-adjusted life-years; considers
evidence beyond trials; and determines how the short-term
budget impact of new interventions affects the Canadian health
care budget.
These oncology frameworks were developed by oncologists
(with other researchers) who have professional and financial
stakes in the drugs they prescribe, and so the possibility of
conflict of interest arises. A recent review of oncology drug value
frameworks was supported by a pharmaceutical company. The
authors declared that they owned stock or consulted for this
company [2].
Cardiology
The American College of Cardiology and the American Heart
Association developed another recent value framework [7] to
inform their practice guidelines. This framework rates both the
quality of evidence and the value of care, with the latter based on
a CEA framework. On the basis of CEA results, they developed a
score for value from low to high using the World Health
Organization’s CHOICE (CHOosing Interventions that are CostEffective) project, including the use of cutoff values of 1 to 3 times
the per capita gross domestic product [49]. Notably, the writing
committee had fewer than 50% of members with industry
relationships, and all these relationships were published.
Although the primary data resources for this evaluation come
from independent entities, we note that when committee members are the providers of therapies, they may have inherent
conflicts of interest that can cause deliberate or subconscious
bias. In general, the American College of Cardiology-American
Heart Association framework robustly addresses almost all the
key components of our taxonomy but could be improved by more
explicit definitions of intervention components, and approaches
to uncertainty.
Vaccines
Frameworks for the evaluation of vaccines against infectious
diseases present many complexities that do not arise with
other medical interventions, including the need to capture
spread of disease in the population; changes in population’s
demographic characteristics because of aging and migration;
the natural history of immunity, herd protection, and community well-being; the need to model the population equilibrium; the lack of postmarket surveillance for adverse events;
and the ability of the vaccine to prevent disease or diseaserelated effects (e.g., postherpetic neuralgia) [6,50,51]. These
factors present unique challenges in defining the perspective,
individual preferences versus the effects of treatment of one
person on unaffected persons (i.e., herd effects), and time
horizon for a value framework [39]. Despite this complexity,
Table 2 – Examples of application of proposed taxonomy for evaluation of frameworks assessing the value of health care interventions.
Protocol element
Purpose
Stated purpose
References existing methods of
assessing value, allowing
comparison across diseases
National Comprehensive
Cancer Network [16]
American Heart AssociationAmerican College of
Cardiology [7]
European Vaccine Working
Group [33]
Patient-physician drug treatment
decisions; separate framework
for adjuvant vs. metastatic Rx
Yes; will need to develop format
for use in practice
Patient-physician drug treatment
decisions for surgery, radiation,
and drugs
Yes; included with online
treatment guidelines
Inform practice guidelines and
performance measures
Public health policy decisions
Yes
Yes
No
No
Yes
Yes
Bonus points subjectively assigned
Scoring system subjective/expert
opinion
NA
Yes
Implies clinical decision-making
perspective
Implies clinical decision-making
perspective
Societal, with separate analyses
with patient perspectives for
clinical decisions; consider
subgroups
Uses costs per QALY to grade value
low to high value using WHOCHOICE framework; includes
uncertain value category
Societal with long-term
population outcomes
De novo method: net health
benefit score and costs per drug
treatment course. “Bonus
points” and toxicity included in
net benefits score for a new
regimen vs. control therapy
Points for benefit are assigned on
the basis of categories of median
survival
Does not allow comparisons across
multiple available regimens
Considers multidecision
Preferences included in metastatic
attributes (preferences,
framework only, but based on
equity, distribution,
expert designation of “bonus
externalities)
points” for symptom palliation,
and quality of life, longer life
with metastases
Intervention components and comparators
All components and
Only clinical benefit as gain in
comparators considered
survival, major toxicity, and
costs of drugs and supportive
care for drug delivery; patient
co-pay for drugs included; other
effects and costs excluded;
update to framework includes
lower grade toxicities
De novo method: evidence block
score from 1 to 5 on the basis of
subjective assessment of
benefits
Uses dynamic agent-based
models and costs per QALY
Does not allow comparisons across
multiple available regimens
Value categories can be based on
societal cutoff points or a
percent of GDP
May consider single or multiple
vaccines
Unknown
Considers preferences,
distribution, and budget impacts
Explicitly considers use of multiattribute theory to assist with
decisions
Effectiveness, safety (i.e., major
toxicity), quality of evidence,
consistency of evidence, and
affordability
Implicit in CEA framework, but not
specified; framework used as a
broad template
Very broad horizon and effects
considered in a CEA framework
189
continued on next page
VALUE IN HEALTH 20 (2017) 185–192
Framework consistent with
purpose
Approach
Clear and transparent
conceptual framework
Do assumptions have
empirically supported
clinical basis?
Is the perspective made
explicit
American Society of Clinical
Oncology [19]
190
Table 2 – continued
Protocol element
Toxicity/side effects
considered
Considers no treatment as a
comparator (e.g., “watchful
waiting” or “palliative care”)
Data sources
Trials or observational studies
vs. expert opinion; quality of
data
Budget impact assessed
Uncertainty and information
gaps considered
Conflicts of interest
Sponsoring organization has
financial stake? Declared?
National Comprehensive
Cancer Network [16]
Examines only one drug and one
control drug
Yes, but not in costs for
hospitalization etc.
No
No direct comparisons among
treatment modalities
For some regimens
American Heart AssociationAmerican College of
Cardiology [7]
European Vaccine Working
Group [33]
Implicit but not specified
Yes
For some treatment options
(surgery, radiation, drugs)
Unknown
NA
RCT and expert opinion
RCT and expert opinion
Specifies US data; data rated from
A to C on the basis of a formal
review of the evidence quality;
includes randomized trials and
observational data
Vaccine trials using intention-totreat results
Only drug costs; drug delivery and
supportive care for drug delivery
Includes patient co-pays for drugs
but not time costs
Drug cost, supportive care, drug
administration, and toxicity
costs; costs ranked subjectively
on an “affordability scale” from
1 to 5 (inexpensive to very
expensive)
Budget impact not considered
Not considered
Includes all costs in CEA
framework
Includes all costs in CEA
framework
Unknown/Unstated
Not explicitly considered; does
include an unknown category
for value
Explicitly addressed
o50% of writing committee
required to not have conflicts;
relationships with drug or
device companies allowed and
disclosed online
Unknown/Unstated
Yes
Budget impact not considered
Not considered
Not explicitly addressed
Disclosures included in journal
article
Potential conflicts because value
primarily determined by
oncologists who deliver (and
bill) care
Yes; No
Not explicitly addressed
Potential conflicts because value
primarily determined by
oncologists who deliver (and
bill) care
Yes; No
No; No
Explicitly addressed
Disclosure included
Employees of drug or vaccine
companies excluded; all
conflicts of relationships
declared
CEA, cost-effectiveness analysis; CHOICE, CHOosing Interventions that are Cost-Effective; GDP, gross domestic product; NA, not applicable; QALY, quality-adjusted life-year; RCT, randomized
controlled trial; WHO, World Health Organization.
VALUE IN HEALTH 20 (2017) 185–192
Economics/costs
All costs stemming from the
intervention included
American Society of Clinical
Oncology [19]
VALUE IN HEALTH 20 (2017) 185–192
the European guidelines for vaccine value assessment consider
virtually all the components in our taxonomy [52]. These
guidelines explicitly address conflicts of interest: the developers excluded employees of drug or vaccine companies and
required members to disclose potential conflicts.
Conclusions
Although the movement to examine the value of health care is
rapidly gaining prominence, no consensus exists on the value
framework’s structure and components. Wider implementation
of value frameworks may be possible by further use of data from
clinical trials and information made available through electronic
records [46,53]. It will also be useful to include education about
value frameworks in medical and public health training [54,55].
Our taxonomy could support these efforts and extend previous
overviews [2,25] by including a comprehensive set of metrics
grounded in CEA.
Source of financial support: This work was funded by a grant from
the National Cancer Institute (grant no. U01CA183081) at the
National Institutes of Health. This work was also supported in
part by National Cancer Institute (grant nos. R35 CA197289 and
U01 CA152958).
R EF E R EN CE S
[1] Lawler M, French D, Henderson R, et al. Shooting for the moon or flying
too near the sun? Crossing the value Rubicon in precision cancer care.
Public Health Genomics 2016;19:132–6.
[2] Chandra A, Shafrin J, Dhawan R. Utility of cancer value frameworks for
patients, payers, and physicians. JAMA 2016;315:2069–70.
[3] Swank MF, Brennan LK, Gentry D, Kemner AL. Using frameworks to
diagram value in complex policy and environmental interventions to
prevent childhood obesity. J Public Health Manag Pract 2015;21(Suppl. 3)
S116–20.
[4] Rogowski W, Payne K, Schnell-Inderst P, et al. Concepts of
“personalization” in personalized medicine: implications for economic
evaluation. Pharmacoeconomics 2015;33:49–59.
[5] Committee on Valuing Community-Based, Non-Clinical Prevention
Programs; Board on Population Health and Public Health Practice;
Institute of Medicine. An Integrated Framework for Assessing the Value
of Community-Based Prevention. Washington, DC: National Academies
Press, 2012.
[6] Pronk NP, Hernandez LM, Lawrence RS. An integrated framework for
assessing the value of community-based prevention: a report of the
Institute of Medicine. Prev Chronic Dis 2013;10:120323.
[7] Anderson JL, Heidenreich PA, Barnett PG, et al. ACC/AHA statement on
cost/value methodology in clinical practice guidelines and performance
measures: a report of the American College of Cardiology/American
Heart Association Task Force on Performance Measures and Task Force
on Practice Guidelines. Circulation 2014;129:2329–45.
[8] Porter ME. What is value in health care? N Engl J Med 2010;363:2477–81.
[9] de Souza JA, Seiwert TY. A value framework in head and neck cancer
care. Am Soc Clin Oncol Educ Book 2014;e304–9.
[10] Harris RP, Wilt TJ, Qaseem A. A value framework for cancer screening:
advice for high-value care from the American College of Physicians.
Ann Intern Med 2015;162:712–7.
[11] Thaker NG, Pugh TJ, Mahmood U, et al. Defining the value
framework for prostate brachytherapy using patient-centered outcome
metrics and time-driven activity-based costing. Brachytherapy 2016;
15:274–82.
[12] Teckie S, McCloskey SA, Steinberg ML. Value: a framework for radiation
oncology. J Clin Oncol 2014;32:2864–70.
[13] Yount KW, Turrentine FE, Lau CL, Jones RS. Putting the value
framework to work in surgery. J Am Coll Surg 2015;220:596–604.
[14] Barnighausen T, Bloom DE, Cafiero-Fonseca ET, O’Brien JC. Valuing
vaccination. Proc Natl Acad Sci U S A 2014;111:12313–9.
[15] Mitera G, Earle C, Latosinsky S, et al. Choosing wisely Canada cancer
list: ten low-value or harmful practices that should be avoided in
cancer care. J Oncol Pract 2015;11:e296–303.
[16] National Comprehensive Cancer Network. NCCN Clinical Practice
Guidelines in Oncology (NCCN Guidelines®) with NCCN Evidence
Blocks. Available from: http://www.nccn.org/evidenceblocks/pdf.
[Accessed August 1, 2016].
191
[17] Institute for Clinical and Economic Review. Methods: the ICER
approach. Available from: http//:icer-review.org/methodology/
icers-methods/. [Accessed August 1, 2016].
[18] Schnipper LE, Davidson NE, Wollins DS, et al. American Society of
Clinical Oncology statement: a conceptual framework to assess the
value of cancer treatment options. J Clin Oncol 2015;33:2563–77.
[19] Schnipper LE, Davidson NE, Wollins DS, et al. Updating the American
Society of Clinical Oncology value framework: revisions and reflections
in response to comments received. J Clin Oncol 2016;34:2925–34.
[20] Cheung MC, Sabharwal M, Chambers A, et al. Multiple dimensions of
value: evaluative frameworks for new cancer therapies. J Clin Oncol
2016;34:1428–9.
[21] Hoch JS, Sabharwal M. Informing Canada’s cancer drug funding
decisions with scientific evidence and patient perspectives: the PanCanadian Oncology Drug Review. Curr Oncol 2013;20:121–4.
[22] Cherny NI, Sullivan R, Dafni U, et al. A standardised, generic, validated
approach to stratify the magnitude of clinical benefit that can be
anticipated from anti-cancer therapies: the European Society for
Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS).
Ann Oncol 2015;26:1547–73.
[23] Gold MR, Siegel JE, Russell LB, Weinstein MC,editors. Cost-Effectiveness
in Health and Medicine. New York, NY: Oxford University Press, 1996.
[24] Neumann PJ, Sanders GD, Russell LD, et al., editors. Cost-Effectiveness
in Health and Medicine (2nd ed.). New York, NY: Oxford University
Press, 2017.
[25] Sanders GD, Neumann PJ, Basu A, et al. Recommendations for conduct,
methodological practices, and reporting of cost-effectiveness analyses:
second panel on cost-effectiveness in health and medicine. JAMA
2016;316:1093–103.
[26] Marsh K, IJzerman M, Thokala P, et al. Multiple criteria decision
analysis for health care decision making—emerging good practices:
report 2 of the ISPOR MCDA Emerging Good Practices Task Force. Value
Health 2016;19:125–37.
[27] International Committee of Medical Journal Editors. Standard
statistical evaluation tools for medical research. Available from: http://
www.icmje.org/recommendations/browse/roles-and-responsibilities/
author-responsibilities–conflicts-of-interest.html. [Accessed October
10, 2016].
[28] Cochrane Training. Cochrane handbook for systematic reviews of
interventions. Available from: http://training.cochrane.org/handbook.
[Accessed October 10, 2016].
[29] Madhavan G, Phelps CE, Colwell RR, et al. In plain view: a transparent
systems approach for enhancing health policy decisions. Bridge
2016;46:5–10.
[30] Kahneman D. Thinking, Fast and Slow. New York, NY: Farrar, Strass
and Giroux, 2011.
[31] Schnipper LE, Bastian A. New frameworks to assess value of cancer
care: strengths and limitations. Oncologist 2016;21:654–8.
[32] World Health Organization. Metrics: disability-adjusted life year
(DALY). Available from: http://www.who.int/healthinfo/
global_burden_disease/metrics_daly/en/. [Accessed August 1, 2016].
[33] Malone DC, Berg NS, Claxton K, et al. International Society for
Pharmacoeconomics and Outcomes Research comments on the
American Society of Clinical Oncology value framework. J Clin Oncol
2016;34:2936–7.
[34] Saaty TL. Decision making with the analytic hierarchy process. Int J
Serv Sci 2008;1:83–98.
[35] Edwards W, Barron FH. SMARTS and SMARTER: improved simple
methods for multiattribute utility measurement. Organ Behav Hum
Decis Process 1994;60:306–25.
[36] Barron FH, Barrett BE. Decision quality using ranked attribute weights.
Manag Sci 1996;42:1515–23.
[37] Schmidt K, Aumann I, Hollander I, et al. Applying the analytic
hierarchy process in healthcare research: a systematic literature
review and evaluation of reporting. BMC Med Inform Decis Mak
2015;15:112.
[38] Madhavan G, Sangha K, Phelps C, et al., editors. Ranking Vaccines: A
Prioritization Framework, Phase I: Demonstration of Concept and a
Software Blueprint. Washington, DC: National Academies Press, 2012.
[39] Madhavan G, Sangha K, Phelps C, et al., editors. Ranking Vaccines: A
Prioritization Software Tool, Phase II: Prototype of a Decision-Support
System. Washington, DC: National Academies Press, 2013.
[40] Madhavan G, Phelps C, Rappuoli R, et al., editors. Ranking Vaccines:
Applications of a Prioritization Software Tool, Phase III: Use Case
Scenarios and Data Framework. Washington, DC: National Academies
Press, 2014.
[41] Nelson HD, Fu R, Cantor A, et al. Effectiveness of breast cancer
screening: systematic review and meta-analysis to update the 2009 U.S.
Preventive Services Task Force recommendation. Ann Intern Med
2016;164:244–55.
[42] Nelson HD, Pappas M, Cantor A, et al. Harms of breast cancer screening:
systematic review to update the 2009 U.S. Preventive Services Task
Force recommendation. Ann Intern Med 2016;164:256–67.
192
VALUE IN HEALTH 20 (2017) 185–192
[43] Henry J; Kaiser Family Foundation. 2015 Employer health benefits
survey. Available from: http://kff.org/report-section/
ehbs-2015-summary-of-findings/. [Accessed May 18, 2016].
[44] Shankaran V, Jolly S, Blough D, Ramsey SD. Risk factors for financial
hardship in patients receiving adjuvant chemotherapy for colon
cancer: a population-based exploratory analysis. J Clin Oncol
2012;30:1608–14.
[45] Angelis A, Kanavos P. Critique of the American Society of Clinical
Oncology value assessment framework for cancer treatments: putting
methodologic robustness first. J Clin Oncol 2016;34:2935–6.
[46] Basch E. Toward a patient-centered value framework in oncology.
JAMA 2016;315:2073–4.
[47] Li J, Asabere A, Kelly S, et al. Application of ASCO value framework
evaluations of net health benefit for oncology drugs launched in the
United States between 2013 and 2015. Value Health 2015;18:A489.
[48] Memorial Sloan Kettering Cancer Center. Evidence driven drug pricing
project. Available from: http://www.drugabacus.org/. [Accessed August
1, 2016].
[49] World Health Organization. Choosing interventions that are costeffective project. Available from: http://www.who.int/choice/
cost-effectiveness/en/. [Accessed August 1, 2016].
[50] Ultsch B, Damm O, Beutels P, et al. Methods for health
economic evaluation of vaccines and immunization decision
frameworks: a consensus framework from a European
Vaccine Economics Community. Pharmacoeconomics
2016;34:227–44.
[51] Ultsch B, Damm O, Beutels P, et al. Methods for health economic
evaluations of vaccines—results from an international expert
(workshop). Value Health 2014;17:A552.
[52] Ultsch B, Damm O, Perleth M, Wichmann O. Health economics in
immunization decision-making—results from a systematic literature
research and a stakeholder symposium in Germany. Value Health
2015;18:A569.
[53] Fayanju OM, Mayo TL, Spinks TE, et al. Value-based breast cancer care:
a multidisciplinary approach for defining patient-centered outcomes.
Ann Surg Oncol 2016;23:2385–90.
[54] Patel MS, Davis MM, Lypson ML. The VALUE framework: training
residents to provide value-based care for their patients. J Gen Intern
Med 2012;27:1210–4.
[55] Phelps C, Madhavan G, Rappuoli R, et al. Strategic planning in
population health and public health practice: a call to action for higher
education. Milbank Q 2016;94:109–25.