Download Becoming ADEPT - Unit 5b

Becoming ADEPT - Unit 5b - Economic Evaluations
1. Learning Objectives
Having successfully completed this unit, you will be able to:

Describe the four main types of economic evaluation and their essential features

Recognise the terminology associated with economic evaluations

Describe the main strategies associated with searching for economic evaluations

Use a filter to search for economic evaluations more efficiently

Be familiar with alternative sources and resources for economic studies
2. Introduction
Twenty years ago it would have seemed very strange to worry about the cost
implications of a particular diagnostic intervention or treatment. Concerns about
health care were centred almost exclusively on clinical effectiveness (whether an
intervention works), not cost effectiveness (whether an intervention is worth doing).
This was the era of, for example, heart transplantation. In the intervening years we
have seen the growth in an ageing population, the development of numerous high cost
low volume procedures and the political supremacy of interventionist, “cost counting”
governments. Now cost considerations are not merely the concern of the health
authorities (“the purchasers”) but also of hospital administrations and even of ward
managers or practice managers. Such considerations have led to the development of
an increasingly more sophisticated range of tools for evaluating potential costs and
benefits of possible interventions against available resources.
You may be saying to yourself, “I don’t usually get asked for ‘economic evaluations’
as such. What is the relevance of these to my literature searching?” Consider instead
how many times you get asked for information on whether it is worth introducing a
new way of doing things, or what the benefits are of a particular course of action.
What is economic evaluation?
“Economic evaluation is a technique that was developed by economists to assist
decision making when choices have to be made between several courses of action. In
essence, it entails drawing up a balance sheet of the advantages (benefits) and
disadvantages (costs) associated with each option so that choices can be made”.
(Robinson R. Economic evaluation and health care. What does it mean? BMJ. 1993
Sep 11; 307(6905): 670-3).
Three important points should be made at this juncture:
1. Economic evaluation shares, with evidence based medicine, a concern with
supporting decision making.
2. Costs is used here in its more general sense (E.g. “What will it cost me?”) rather
than being specifically about money. A particular course of action may cost you
money but it may equally cost you pain, inconvenience, anxiety or
embarrassment. The challenge here would be how to place a value on such
intangibles.
3. The concept of a balance sheet is common to all types of economic evaluations.
Where they differ is in the methods that they use (the “currency”, if you like) to
construct this balance sheet.
4. Strictly speaking, a study that merely reports costs is not an economic evaluation,
as it doesn’t attempt to evaluate costs against benefits. However, often you may
find that your users are also interested in studies that provide baseline information
about the cost of an intervention or service. This type of study is called a cost
analysis or a cost study. You will find that the strategies that are suggested later
in this Unit will retrieve the straightforward cost analysis/cost study as well as
economic evaluations. The distinction between these is best handled by the reader
rather than at the search strategy stage. On the NHS Economic Evaluations
Database (NHS EED), produced by the NHS Centre for Reviews and
Dissemination at the University of York, those studies which are just cost analyses
are identified as such and will not be assigned a structured abstract.
The four main methods currently used for economic evaluation (sometimes called
economic appraisal) are:

Cost-minimisation analysis

Cost-effectiveness analysis

Cost-utility analysis1

Cost-benefit analysis2
Cost-minimisation
Cost minimisation is the simplest form of economic evaluation. You should have
evidence that the outcomes of two or more alternative procedures are the same. You
are interested in the one that costs the least. You therefore do not have to worry about
the benefit side of the “balance sheet” and only collect information on the costs. A
comparison of day surgery with inpatient treatment of hernias or haemorrhoids is
suggested as being suitable for a cost-minimisation study. Evidence, preferably from a
clinical trial, would show similar clinical outcomes so which is the least costly
1
In North America health economists view the cost-utility analysis as a particular type of cost
effectiveness analysis. Studies in the North American literature, or indeed some of those indexed on
the MEDLINE database, identified as cost effectiveness analyses may, on closer inspection, be costutility analyses.
2
Cost-benefit analysis is sometimes used, incorrectly, as a generic term describing any type of
economic evaluation. As such, a study referred to as a cost-benefit analysis may be one of the other
forms of analysis. For these reasons, as well as the caveat regarding cost analyses mentioned, above it
is simplest to think in terms of a single retrieval strategy for all types of economic study.
option? You do this kind of economic evaluation whenever you are choosing whether
to buy English Golden Delicious or French Golden Delicious apples at the
supermarket.
Cost-effectiveness
Cost-effectiveness is the next easiest economic evaluation. You know that the
outcomes of the alternative procedures or programmes will vary. Nevertheless these
outcomes may all be expressed in common natural units (E.g. Life years gained,
number of readmissions, positive cases detected). For example, there is a range of
treatments for control of hypertension, all with different outcomes or side effects.
However, all may be measured primarily in terms of the reductions in diastolic blood
pressure (mm Hg) that they achieve. An example from your supermarket would be to
compare a number of different fruits in terms of the amount of calories per penny
spent. Assuming that number of calories is the only aspect you’re interested in, you
would be able to rank your choices in terms of cost per unit of outcome.
Cost-utility analysis
Cost effectiveness information may be useful in clarifying choices between treatments
for the same condition. It is limited, however, when one is trying to assist choices
between different diseases, or between programmes where units of outcome vary.
(E.g. If you are a purchaser wishing to compare a hypertension prevention programme
with a cervical cancer screening programme how would you compare reductions in
blood pressure with number of diagnosed cases of cancer of the cervix?). In addition,
a cost-effectiveness analysis would not even help you to arrive at an overall single
summary estimate of a number of outcomes for the same condition if all are measured
in different ways (e.g. mortality and morbidity).
In such circumstances you would want to agree a single subjective level of well-being
(known as a utility) that can be applied to different states of health or disease.
Imagine in a GP’s surgery if, instead of having 20 people all saying “I’m not feeling
very well” he/she had a subjective measure of distinguishing a 0.01 headache from a
9.00 cardiac arrest.
To measure utility a number of quality of life scales have been developed. Quality
adjusted life years (QALYs) combine a utility-based measure of quality of life with a
quantitative measure of life years to obtain a single measure of lifetime utility. So, for
example, you could compare the benefits of treatment for a common but non-life
threatening condition (such as varicose veins) with a treatment for a more rare but
serious condition such as congestive heart failure. Obviously you would need to
collect a great deal of supporting data about what it feels like to live with these
conditions. To return to our supermarket analogy if, instead of looking only at
calorific value you literally wanted to compare apples and oranges across a number of
dimensions, (E.g. Taste, texture, calories, carbohydrates etc.) you could compile data
from a variety of sources (nutrition textbooks, the systematically recorded preferences
of your household members etc) and come up with a single measure (the FRUTY?). A
pound of apples may be worth 5 FRUTYs whereas a pound of oranges may only rate
2 FRUTYs. You could then make your purchasing decision expressed in terms of a
universal unit, namely cost per FRUTY gained.
Cost-benefit analysis
Our final measure, cost-benefit analysis, has the potential for confusion, as many
people use the term inappropriately as a synonym for economic evaluation. However,
strictly speaking, the term is restricted to those forms of evaluation that are used to
place a monetary value on benefits or outcomes. (In other words to use monetary
values on the benefits side of the balance sheet as well as on the costs side.) There is a
whole body of literature, mostly originating from the Health Economics Research
Unit (HERU) in Aberdeen, that looks at “willingness to pay”, for example. An
example would be “What percentage of your household’s total income would you be
willing to pay regularly for a complete cure for arthritis?” To conclude our
supermarket analogy, you could ask your children, “How much of your pocket money
would you be prepared to give me each week for me to buy strawberries instead of
apples?”
Health economics is currently a growth area within the United Kingdom.
Organisations to watch out for are:

Centre for Health Economics, University of York (CHE). Produce a distinctive
yellow A4 series of reports which are available online.

Health Economics Research Unit, University of Aberdeen (HERU). Produce an
irregular series of pale green A5 reports. These are available on the internet.

Health Economics Research Group, Brunel University (HERG). Produce a blue
A4 report series, texts of some of these are available on the internet.

Office of Health Economics, London (OHE). Produce a number of disease
specific booklets, a newsletter and other publications.

School of Health and Related Research, University of Sheffield (ScHARR). The
Sheffield Health Economics Group within ScHARR produce a Discussion Paper
Series as well as the ScHARR Occasional Paper Series and the Trent Institute
Guidance Notes for Purchasers.
Elsewhere, the major academic contributors are in Australia and Canada:

Centre for Health Economics Policy and Analysis (CHEPA), McMaster
University, Canada

Centre for Health Economics Research and Evaluation (CHERE), Australia.
Whilst much work has been done in the United States by the mighty RAND
Corporation.
3. Searching for Economic Evaluations
The methodological filters that have so far been developed for economic evaluations
have not been tested in the same way as the filters for the other types of study covered
in this programme. However, a number of librarians, including Julie Glanville at NHS
CRD at the University of York and Andrew Booth at ScHARR, University of
Sheffield, have been involved in developing optimal strategies for either compiling
the NHS Economic Evaluation Database (NHS EED) or for conducting systematic
reviews. (Similar strategies have been developed by Sassi, Archard and McDaid (see
Recommended Reading). And the InterTASC Information Specialists' Sub-Group
(ISSG), a group of information professionals that supports research groups within
England providing technology assessments to the National Institute for Clinical
Excellence (NICE), is also currently developing economic evaluation filters.)
Unfortunately, there is no publication type associated with economic evaluations.
However, as well as a designated subheading (/economics) there are a number of
MeSH terms that can be used:
ECONOMICS
Costs and Cost Analysis
Cost Allocation
Cost-Benefit Analysis3
Cost Control
Cost Savings
Cost of Illness
Cost Sharing
Deductibles and Coinsurance
Medical Savings Accounts
Health Care Costs
Direct Service Costs
Drug Costs
Employer Health Costs
Hospital Costs
Health Expenditures
Capital Expenditures
Relevant materials may be retrieved using a strategy comprised of the following three
sorts of components:
For reasons explained above, the MeSH term “Cost-Benefit Analysis” will not only identify cost
benefit analyses, due to the inappropriate use of the term.
3
1. Generic
Economic evaluation$
Economic analys$
Pharmacoeconomi$
Health economi$
2. Subheading
/economics in MeSH
3. Specific
exp "Costs and Cost Analysis"/
cost benefit$
cost containment
cost effective$
cost minimi$
cost utility$
N.B. The subheading “economics” can be used either with interventions (diagnostic
or therapeutic) or with diseases. Obviously the distinction between these for the
purposes of indexing can be somewhat arbitrary. For this reason it is best to search for
economics as a “floating subheading” (Ovid: ec.fs).
Ovid query syntax
1. (economic (evaluation$ OR analy$)) OR pharmacoeconomi$ OR (health
economi$)
2. (cost benefit$.tw.) OR (cost containment$.tw.) OR (cost effective$) OR (cost
minimi$) OR (cost utilit$)
3. exp "Costs and Cost Analysis"/
4. ec.fs.
5. OR/1-4
N.B. the above strategy avoids the very general search terms “economi$” and “cost or
costs” as, generally speaking, these will make the search too sensitive. However, if
you are searching for a very specific disease or condition and do not retrieve any
materials using the above then you can use these two additional expressions to widen
your coverage:
Ovid query syntax
1. (economic evaluation$ OR economic analy$ OR pharmacoeconomi$ OR health
economi$).tw
2. cost benefit$.tw OR cost containment$.tw OR cost effective$.tw OR cost
minimi$.tw OR cost utilit$.tw
3. exp "Costs and Cost Analysis"/
4. ec.fs.
5. economi$.tw OR cost.ti,ab OR costs.ti,ab
6. OR/1-5
Our “one-line strategy” for economic evaluations follows. You will find this a useful
strategy to have to hand if your reader just wants an entrée into the cost issues of a
particular condition or intervention.
exp "Costs and Cost Analysis"/ OR ec.fs.
Here is a worked example that shows the usefulness of this one-line filter:
You are approached by a Senior Registrar in Public Health from your local
purchasing authority. Your Trust’s Pharmacy and Therapeutics committee has been
trying to decide on formulary guidelines for the use of streptokinase or tissue
plasminogen activator (tPA) in the treatment of acute myocardial infarction (MI).
Members of the committee have been arguing for weeks about whether the added
expense of tPA is worth it. The committee has reached an impasse and has asked the
Director of Public Health for some outside help to reach a good decision. Knowing
that the hospital faces pressure to keep costs down, the DPH wants some evidence
about this question to present to the next committee meeting later this week. The
Senior Registrar asks you to help him find out for his boss if a formal economic
analysis that compares thrombolytic agents for acute MI has been done so that he can
brief her for the Committee meeting.
Adapted from User Guides to the Medical Literature
You break the request down into the following components:
PATIENT
Patients with acute myocardial infarction
INTERVENTION
tissue plasminogen activator (tPA) (alteplase)
OUTCOME
Costs associated with reduced risk of stroke
COMPARISON
Streptokinase
Combining the Patient-Intervention-Outcome-Comparison model, your brief search
strategy might look as follows:
Ovid query syntax
MYOCARDIAL INFARCTION/
AND
THROMBOLYTIC THERAPY/ OR streptokinase.mp OR exp FIBRINOLYTIC
AGENTS/
AND
exp PLASMINOGEN ACTIVATORS/ OR exp Tissue Plasminogen Activator/
AND
exp "Costs and Cost Analysis"/ OR ec.fs
AND
limit to english language
Results
Using the brief search strategy, you find 109 references on Medline (1966 – August
2004) which include the following key article:
Title
Early thrombolysis for the treatment of acute myocardial infarction: a systematic review and economic
evaluation. [Review] [130 refs]
Author
Boland A. Dundar Y. Bagust A. Haycox A. Hill R. Mujica Mota R. Walley T. Dickson R.
Institution
Liverpool Reviews and Implementation Group, New Medical School, Liverpool, UK.
Source
Health Technology Assessment (Winchester, England). 7(15):1-136, 2003.
Abstract
BACKGROUND: Coronary heart disease (CHD) is a major cause of morbidity and mortality in the UK
accounting for around 125,000 deaths a year. Acute myo-cardial infarction (AMI) affects an estimated 274,000
people each year. Of these, approximately 50% (137,000) die within 30 days of AMI and over half these deaths
occur prior to reaching hospital or other medical assistance. The development and introduction of new
pharmacological agents has made it necessary to review the clinical and cost-effectiveness of older and newer
agents used for early thrombolysis. Those reviewed in this document include streptokinase, alteplase, reteplase and
tenecteplase. OBJECTIVES: To examine the clinical and cost-effectiveness of available drugs for early
thrombolysis in the treatment of AMI in hospital and pre-hospital settings. RESULTS: In the hospital setting
definitive conclusions on efficacy are that streptokinase is as effective as non-accelerated alteplase, that
tenecteplase is as effective as accelerated alteplase, and that reteplase is at least as effective as streptokinase. There
seem to be significant differences between drugs in incidence of stroke, with streptokinase having the lowest rate.
However, streptokinase causes more allergic reactions than other drugs. The research failed to identify any studies
conducted in the pre-hospital setting that compared the effectiveness of different drugs. However, there is no
reason to believe that the effectiveness of a drug will be altered by administration in the pre-hospital setting. Given
the similarity in outcome between the drugs, cost-effectiveness becomes largely determined by the acquisition
costs of the drugs. This conclusion was robust to a range of variations in assumptions. Streptokinase was therefore
the most cost-effective drug.
MeSH Subject Headings
Cost-Benefit Analysis; Drug Evaluation; Emergency Treatment / ec [Economics]; Fibrinolytic Agents / ae
[Adverse Effects]; Fibrinolytic Agents / cl [Classification]; *Fibrinolytic Agents / ec [Economics]; *Fibrinolytic
Agents / tu [Therapeutic Use]; Hospitalization / ec [Economics]; Human; *Myocardial Infarction / dt [Drug
Therapy]; Myocardial Infarction / ec [Economics]; Randomised Controlled Trials; Recombinant Proteins / ae
[Adverse Effects]; Recombinant Proteins / ec [Economics]; Recombinant Proteins / tu [Therapeutic Use];
Streptokinase / ae [Adverse Effects]; Streptokinase / ec [Economics]; Streptokinase / tu [Therapeutic Use];
*Thrombolytic Therapy / ec [Economics]; Time Factors; Tissue Plasminogen Activator / ae [Adverse Effects];
Tissue Plasminogen Activator / ec [Economics]; Tissue Plasminogen Activator / tu [Therapeutic Use]; Treatment
Outcome.
To pick up more references, try using the 6-line strategy mentioned earlier in this unit.
You will retrieve about 180 references. What do you notice about the relevance of the
search results?
N.B. Although we have concentrated on the MEDLINE database for the purposes of
this ADEPT module, you will find that the NHS EED database would be an obvious
choice for economic evaluations. Not only would it give increased journal coverage
beyond that of MEDLINE, but it would also give assessments and summaries of
published economic studies.
4. Exercise
Time allotted: No more than half an hour.
Cost effectiveness of lithotripsy versus open cholecystectomy for gallstones.
During the monthly gastroenterology audit meeting in your Trust a debate has arisen over the
treatment of choice for gallstones. A new consultant, recently arrived from London, has been
describing his previous hospital’s use of lithotripsy. His new colleagues are not convinced,
though they are interested in getting information not only on how effective lithotripsy is
compared with open cholecystectomy but also on the relative costs when all cost factors have
been taken into account.
Not wishing to have blood spilled on the new postgraduate centre carpet, the clinical tutor, a
gastroenterologist himself, asks for your assistance. “Do me a favour, my dear librarian. Find
me an article that will settle the argument by proving once and for all that we don’t need one
of these new-fangled contraptions”. Forgetting the PGMC carpet , your first inclination is to
respond to his patronising tone by performing an informal cholecystectomy using the library
date stamp. However, with your future employment prospects in view, you agree to conduct a
MEDLINE search on the topic. But, of course, you cannot guarantee that the results of the
search will necessarily support the clinical tutor’s desired conclusion!
By now you’re beginning to get the hang of these annoying little “anatomy” grids so,
in no time at all you have the following completed to (almost) perfection!:
Break down the above enquiry into the following components
PATIENT/
POPULATION/
CONDITION
INTERVENTION
Patients with
Extracorporeal
gallbladder stones shock-wave
who require
lithotripsy
elective treatment
(i.e. not emergency
admissions)
OUTCOME
COMPARISON
Cost benefits
Open
Cholecystectomy
Quality of life?
Relief of pain?
Relief of other
symptoms
Complications?
Readmissions?
(Possibly also largebulk versus smallbunk stones)
Now try to match the above components to the corresponding MeSH terminology
PATIENT/
POPULATION/
CONDITION
INTERVENTION
OUTCOME
COMPARISON
Cholelithiasis
Lithotripsy
Treatment Outcome
Cholecystectomy
Quality of Life
Comparative-Study
Cost-Benefit
Analysis
Health Status
Biliary Tract
Diseases
Costs and Cost
Analysis
You are probably wondering why you are having such an easy life in this module. We
have not only demonstrated how to break down the question, but we have even
translated this into possible MeSH terms. There must be a catch somewhere! Before
you worry about that use some or all of the above terms to construct your economic
evaluation search strategy. Write your strategy in the box below.
My sample search strategy for cost effectiveness of lithotripsy versus cholecystectomy is:-
Special surprise bonus question
A new feature has recently been added to Ovid. It enables users to run economics- and
cost-related queries very quickly – not to mention other sorts of queries too
(diagnostic queries, therapeutic queries, etc.). Have you discovered it yet?
And now for the catch! You will have noticed in both the sample search on
myocardial infarction and in the exercise question that outcome-related terms have
appeared frequently (quality of life, treatment outcome, etc.). Staff at the now-defunct
UK Clearing House for Information on Health Outcomes, Nuffield Institute for
Health, University of Leeds, have devised the following strategy for searching for
outcomes information.
Treatment outcome methodological filter for Medline (OVID)
Developed by Brettle, A, and Grant, MJ, at the UK Clearing House for Information on
Health Outcomes, Nuffield Institute for Health, University of Leeds.

1. health status indicators/

2. outcome and process assessment (health care)/

3. outcome assessment (health care)/

4. quality of life/

5. (outcome and measure$).tw.

6. (health and outcome$).tw.

7. 1 or 2 or 3 or 4 or 5 or 6
Don’t worry! You are not going to have to do more exercises for these filters.
However, you will find it helpful to have them available for ready reference whenever
you are looking specifically for information on the outcome of a particular treatment
and how to measure it.
If you have got access to the Internet there is one last exercise for you to perform. Log
in to the NHS Economic Evaluations Database (NEED)
(http://www.york.ac.uk/inst/crd/nhsdhp.htm). How many articles are there on
lithotripsy? How many on cholecystectomy? Are there any on both?
Can you see that the NHS Economic Evaluations Database has an important part to
play in the identification of economic evaluations but also in critically appraising
them and in defining into which of the four categories of study types they fall?
If you have not got access to the Internet you can spend the final moments of your
time reading about the Office of Health Economics database, HEED. It is unlikely
that your organisation will be willing to fork out the hefty annual subscription (for 5
copies of the CDROM per month) but you should, at least, be aware of its contents:
Health Economic Evaluations Database
HEED contains information on studies of cost-effectiveness and other forms of
economic evaluation, with objective analysis of key articles. HEED now has more
than 19,000 bibliographic references in total, about half of which are reviewed
articles. Each month clinical and economics databases are searched, combined with
hand searches of the leading journals containing evaluation literature. Working papers
from academic institutions and references from reviewed and other papers are also
included.
Reviews are undertaken of the key literature by researchers in academic institutions
according to a structured format. Entries are also coded according to ICD-9 and other
codes. The full detailed report includes details of the study question, key results and,
subject to permission from the publisher, the abstract in full.
HEED has been set up with finance from more than 20 companies in the USA, Europe
and Japan. It was launched in April 1995.
Source: OHE News and http://www.ohe-heed.com/
5. Summary
In this unit, we have looked briefly at the four main types of economic evaluations.
We have learnt that economic evaluations are those cost studies where an attempt is
made to arrive at a balance sheet weighing up both costs and benefits. We have also
seen that, although there is no single methodological filter for economic evaluations,
an optimal search strategy will include generic textwords (e.g. Economic evaluation),
specific methodologies (E.g. cost minimization/minimisation) and the \economics
subheading.
In addition to the above strategies for optimising retrieval from MEDLINE we have
seen that there are a number of other important sources for economic evaluations.
These include the grey literature from UK and international academic institutions and
the NHS EED (NHS Economic Evaluation Database) and HEED (Health Economic
Evaluation Database). A documented problem is the tendency for studies of clinical
effectiveness (randomised controlled trials) to be published separately from economic
evaluations. It will often be helpful, therefore, to obtain source references from the
bibliography of the economic study in order to give the reader a complete picture of
the benefit side of the “balance sheet”.
6. Recommended reading

CRD Report No. 6. Making Cost-Effectiveness Information Accessible: The NHS
Economic Evaluation Database Project. CRD Guidance for Reporting Critical
Summaries of Economic Evaluations. (£7.50 inc. p&p).

Gagliardi, A, and Helik, T. “Finding information on pharmacoeconomics: the
latest trend in health care information.” Bibliotheca Medica Canadiana 1995; 16
(4): 149-151.

Drummond, MF; Richardson, WS; O'Brien, BJ; Levine, M, and Heyland, D.
Users' guides to the medical literature. XIII. “How to use an article on economic
analysis of clinical practice. A. Are the results of the study valid?” EvidenceBased Medicine Working Group. JAMA. 1997, May 21; 277(19): 1552-7

McKibbon, A, et al. Evidence-Based Principles and Practice. 1999. [In particular,
see chapter eight, “Secondary Publications: Economic Analyses”.]

O'Brien, BJ; Heyland, D; Richardson, WS; Levine, M, and Drummond, MF.
Users' guides to the medical literature. XIII. “How to use an article on economic
analysis of clinical practice. B. What are the results and will they help me in
caring for my patients?” Evidence-Based Medicine Working Group. JAMA 1997,
Jun 11; 277(22):1802-1806

Robinson, R. “Economic evaluation and health care. What does it mean?” BMJ.
1993, Sep 11; 307(6905): 670-3

Robinson, R. “Costs and cost-minimisation analysis.” BMJ. 1993, Sep 18;
307(6906): 726-8

Robinson, R “Cost-effectiveness analysis.” BMJ. 1993, Sep 25; 307(6907): 793-5

Robinson, R. “Cost-utility analysis.” BMJ. 1993, Oct 2; 307(6908): 859-62

Robinson, R. “Cost-benefit analysis.” BMJ. 1993, Oct 9; 307(6909): 924-6

Sassi, F; Archard, L, and McDaid, D. “Searching Literature Databases for Health
Care Economic Evaluations: How Systematic Can We Afford to Be?” Medical
Care. 2002, May; 40(5): 387-94