Download Training on cost-effectiveness analysis of results

Cost-Effectiveness
Analysis of Results-Based
Financing in Zimbabwe
and Zambia
Nov 3, 2016, World Bank
Headquarters, Washington, DC
Donald S Shepard, PhD
Wu Zeng, MD, PhD
Brandeis University, Waltham, MA
Schema for cost-effectiveness analysis
(CEA) for RBF programs
Inputs
Administrator’s cost
Other donors’ cost
Provider’s cost
(User’s cost)
Intermediate results
Component outcomes
Cost-effectiveness outcomes
Cost
Incremental cost
Household survey
Facility survey
HMIS data
Quality score card
Effects on coverage
Effects on quality
Incremental lives saved,
DALYs or QALYs
Incremental cost-effectiveness ratio
(ICER)
Toolkit
Web:
http://documents.worldbank.org/curated/
en/2015/09/25069701/costeffectiveness-analysis-results-basedfinancing-programs-toolkit
 Costs
 Financial costs
 Government and donor
perspective
 Effectiveness—coverage
 Impact evaluation with
districts compared
 Lives Saved Tool (LiST)
software
 Literature and country data
 Effectiveness--quality
 Facility surveys
 Exit interviews
 Expert opinion (Delphi
panel)
Incremental cost-effectiveness ratio (ICER)
Evaluate cost-effectiveness by the ICER, the price of one unit of
good health. The lower the better!
𝐷𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒 𝑖𝑛 𝑐𝑜𝑠𝑡𝑠
𝐼𝐶𝐸𝑅 =
𝐷𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒 𝑖𝑛 ℎ𝑒𝑎𝑙𝑡ℎ 𝑜𝑢𝑡𝑐𝑜𝑚𝑒𝑠
Numerator: Added costs of RBF (difference in costs between
RBF and control districts)
Denominator: Added effectiveness or health outcomes
(difference in health outcomes between RBF and controls),
often expressed as quality-adjusted life years, QALYs
Both quantity (coverage) and quality contribute
CEA of RBF in Zimbabwe
Annual operating costs of RBF program
$2.04
$2.04
RBF subsidy payments, $7,045,211
$0.41
$0.41
Staff costs, $1,434,096
$0.20
$0.10
General administration*, $699,311
Capacity building (meetings, workshops, training)*, $426,644
$0.12
$0.06
HQ support costs*, $372,845
$0.11
$0.05
Transport costs, $299,412
$0.09
$0.09
Capital items for Cordaid*, $298,745
$0.09
$0.04
Supplies / equipment for facilities*, $128,209
The number after each category on the left is
the current aggregate annual cost for the
intervention districts (population 3.46 million).
For categories marked with asterisks, half of the
current costs were considered start-up
expenses and would be reduced in a mature
program. The labels on the right are per capita
costs
Current per capita costs
Mature per capita costs
$0.04
$0.02
Total, $10,704,473
$0.00
$2.82
$0.50
$1.00
$1.50
$2.00
$2.50
Per capita annual operating cost
$3.00
$3.09
$3.50
Aggregate costs from Nat Pharm (US$)*
Group
Pre-period Post-period
(Jan 2011- (Apr 2012Mar 2012)
Jun 2014)
Spending
/year (Pre)
Spending/
year (Post) Net difference
Control
6,771,163
33,466,940
5,416,930
14,874,196
Intervention
6,062,025
29,478,515
4,849,620
13,101,562
Adjusted
Intervention
Difference
13,316,434
Population
2,229,897
-1.63%
Spending
/capita
$6.67
3,461,010
$3.79
3,461,010
$3.85
$0.06
*Source: Nat Pharm data base of drugs distributed representing 92,000 orders to 354 control and 359
intervention customers. Due to the substantial change in expenditure per year, we computed the
difference in differences based on the ratios of aggregate expenditure. We calculated the pre-period
ratio of intervention to control (0.8953). We computed the “adjusted intervention” by applying this ratio to
the control spending/year (post), and computed the net difference by comparing actual and adjusted
intervention values.
Financial net costs of current RBF per
capita per year (USD)
Cost components
Cost
Incremental RBF operational costs
$3.09
Incremental costs at World Bank headquarters
Subtotal
Net costs of consumables from Nat Pharm
$0.10
Less Health Transition Fund payments to control
districts (no administrative costs included)
Subtotal
Net cost
Subtotal
$3.19
-$0.06
-$0.81
-$0.87
$2.32
Impact of RBF in quantity and quality of
services
 Quantity of care
 Institutional delivery: 13.4%
 Postpartum care: 13.3%
 Quality of care
Quality indicator
Endline
RBF Con
-trol
0.87 0.83
0.75 0.68
DIDs
Pvalue
Relative
DIDs
Vaccination
Institutional Delivery
Baseline
RBF
Control
0.87
0.89
0.73
0.75
0.06**
0.10***
0.009
0.001
6.90%
12.90%
Ante-natal care
Post-natal care
0.72
0.71
0.79
0.75
0.10***
0.15***
0.000
0.000
13.70%
20.00%
0.75
0.77
Note: DID denotes difference-in-differences.
0.72
0.65
Effectiveness (RBF vs. control): quality & quantity
 Calculated lives and QALYs saved for selected services with RBF and projected from controls:
Maternal deaths and lives saved
Year
RBF Control
Deaths in children under five and lives saved
Lives
saved
Year
RBF Control
Lives
saved
2012
416
416
0
2012
9,705
9,705
0
2013
370
414
44
2013
8,613
9,345
732
2014
Total
365
1,151
393
1,223
28
72
2014
8,136
8,875
739
26,454 27,925
1471
Total
 Quantitative results
 Annual number of lives saved is 772, i.e., (72+1,471)/2,
 Equivalent to 18,288 QALYs gained
 With population of 3.46 million in RBF districts, RBF generates 528 QALYs/100,000 population/year
 Site visits suggested coaching strengthened RBF
Contributions of quality and quantity
QALYs gained per 100,000 population per
year
Projected health impacts
600
Relative shares of quality and quantity
528
500
400
350
Quality
improvement,
34%
300
200
178
Quantity
(Coverage
improvem
ent), 66%
100
0
Quantity
Quality
(Coverage improvement
improvement)
Overall
(effective
coverage
improvement)
Cost-effectiveness results: RBF vs. control
Incremental per capita costs: US $2.32
QALY impacts (per 100,000 population per year)
Quantity (coverage) alone: 350 QALYs
Quality and quantity (effective coverage): 528 QALYs
ICERs ($/QALY gained)
Quantity (coverage) impacts alone: $2.32/0.00350 = $663
Combined coverage and quality impacts: $2.32/0.00528 = $439
CEA of RBF in Zambia
Study design
Incentivized services (indicators) and unit
prices
No Indicator
Unit Price
US$
0·20
1
Curative consultation
2
Institutional delivery by skilled birth attendant
6·40
3
Antenatal care (ANC) - prenatal and follow up visits
1·60
4
Postnatal care visit (PNC)
3·30
5
Full immunization of children under one year
2·30
6
Pregnant women receiving 3 doses of malaria intermittent preventive treatment (IPTp)
1·60
7
Family planning (FP) users of modern contraceptive methods
0·60
8
Pregnant women counselled and tested for HIV
1·80
9
HIV+ pregnant women given niverapine (NVP) and zidovudine (AZT)
2·00
Results
12.00
HQ costs
Incremental cost per capita ($)
10.00
Program costs
MSL costs
8.00
6.00
4.00
2.00
0.00
HQ costs
Program costs
MSL costs
RBF vs INP
0.22
5.90
0.57
RBF vs CON
0.33
8.65
0.97
INP vs CON
0.10
2.75
0.40
Program costs (RBF + input financing) and
distribution of incentives
Pregnant women
counselled and
tested for HIV
9.3%
Equipment
14.6%
Pregnant
women given
NVP and AZT
0.2%
Curative
consultations
30.0%
Operational costs
7.6%
M&E
0.9%
Meetings/
Workshops
2.2%
Trainings
6.9%
Modern FP
methods
28.5%
Incentive
payment
51.4%
Third dose of IPTp
3.7%
Full vaccination
6.3%
Consultancy
costs
16.3%
PNC
6.6%
Institutional
deliveries
ANC
14.0%
1.4%
Coverage and quality of key maternal and
child health services at baseline and endline
Baseline
Services
RBF
INP
Endline
CON
RBF
INP
DIDs
CON
RBF vs
INP
RBF vs
CON
INP vs
CON
Coverage of key maternal and child services
Ins Del
68·3%
56·4%
70·9%
80·8%
74·3%
71·2%
-5·4%
12·2%**
17·6%***
ANC
97·5%
96·2%
96·3%
98·9%
99·0%
99·1%
-1·4%
-1·4%
0·0%
PNC
70·3%
56·0%
76·4%
82·4%
73·8%
80·7%
-5·7%
7·8%*
13·5%***
BCG
95·6%
97·8%
97·6%
100·0%
99·5%
95·6%
2·7%
6·4%*
3·7%*
DPT
97·1%
95·2%
95·8%
98·6%
97·6%
91·8%
-0·9%
5·5%*
6·4%*
HIB
82·5%
88·3%
81·8%
97·9%
88·7%
78·1%
15·0%***
19·1%***
4·1%
IPT
92·0%
92·4%
95·1%
98·0%
96·1%
98·1%
2·3%
3·0%**
0·7%
FP∆
6·5%
9·9%
7·7%
34·0%
15·6%
15·7%
21·8%**
19·5%**
-2·3%
Quality index of key maternal and child services
Ins Del
65·5%
66·8%
67·0%
73·5%
74·1%
71·9%
0·7%
3·1%
2·4%
ANC
66·9%
69·1%
68·6%
75·0%
77·2%
73·8%
0·0%
2·9%
2·8%
PNC
66·7%
68·4%
68·3%
74·1%
76·6%
73·4%
-0·8%
2·3%
3·0%
Vaccination
78·7%
80·7%
81·7%
81·2%
80·0%
80·4%
3·2%
3·8%
0·6%
FP
77·7%
78·6%
80·6%
81·6%
77·6%
74·8%
4·9%
9·7%
4·8%
QALYs gained from the RBF program in
comparison with controls
Population
RBF vs INP
RBF vs CON
INP vs CON
Mid-point (lower bound-upper bound)
Mid-point (lower bound-upper bound)
Mid-point (lower bound-upper bound)
QALYs gained
(unadjusted for
quality)
QALYs gained
QALYs gained
(adjusted for quality) (unadjusted for quality)
QALYs gained
(adjusted for quality)
QALYs gained
(unadjusted for
quality)
QALYs gained
(adjusted for quality)
Pregnant women
237 (216-302)
302 (237-345)
475(425-539)
604(539-626)
237(176-302)
302(237-345)
Children under 5
5 088(3 733-6 015)
6 300(4 826-7 323)
11 816(10 480-13 100)
14 574(13 195-15 953)
6728(5 171-8 131)
8 274(6 704-9 843)
Total
5 325 (3 948-6 317)
6 602(5 064-7 688) 12 291(10 905-13 639)
15 178(13 734-16 579)
6 966(5 347-8 433)
8 576(6 942-10 188)
Incremental cost-effectiveness ratios
Cost/life saved (US$)
Cost/QALY gained (US$)
Mid-point (lower bound- upper
bound)
Mid-point (lower bound-upper
bound)
RBF vs INP (unadjusted)
39 621 (33 388 - 53 381)
1 674 (1 411 - 2 258)
RBF vs INP (quality adjusted)
31 952 (27 514 - 41 657)
1 350 (1 163 - 1 761)
RBF vs CON (unadjusted)
25 553 (23 024 - 28 767)
1 079 (972 - 1 216)
RBF vs CON (quality adjusted)
20 689 (18 945 - 22 865)
874 (800 - 966)
INP vs CON (unadjusted)
14 786 (12 211 - 19 235)
624 (515 - 813)
INP vs CON (quality adjusted)
12 280 (10 110 - 14 837)
507 (427 - 626)
Comparison
Discussion
Discussion: Interpretation of RBF in Zimbabwe
Reference: Zimbabwe’s 2012 GDP/capita was $980
If ICER < GDP/capita, program highly cost-effective (WHO)
ICER of current RBF
 Improved coverage alone: $663 or 0.68x GDP/capita, highly cost-
effective
 Including quality gains: $439 or 0.45x GDP/capita, highly costeffective
Mature RBF program would lower cost by 9.0% to $2.11 per
capita
DALYs per100
population per
year
704
Discussion: program maturity (Zimbabwe
as an example)
528
0
Phase I period
Estimated phase in
Phase I period
Calculated average
Phase I period
Mature program
Calculated average is 528 DALYs vs. mature program (704)
Potential increase for mature program: 33%
Projected ICER of mature RBF program
Projected cost per capita $2.11
Projected impact is 704 QALYs per 100,000
population per year
Projected ICER is $300
i.e. $2.11 / (704 / 100,000) or 0.31 x GDP/capita
Comparison with Zambia RBF
Reference: GDP/capita $1,759 (2013)
ICER of RBF
 Compared to Input-financing: $1,350 or 0.77 GDP/capita, highly cost-effective
 Compared to pure control: $874 or 0.50 GDP/capita, highly cost-effective
ICER of input financing
 Compared to pure control: $507 or 0.29 GDP/capita, highly cost-effective
Comparison with other maternal-child health programs
Reproductive health vouchers in Uganda (African Strategies
for Health, 2015)
$302 / QALY or 0.59 x GDP/capita ($510)
Simulated maternal community-based health insurance in
Uganda (African Strategies for Health, 2015)
$298 / QALY or 0.58 x GDP/capita ($510)
RBF is among the very highly cost-effective interventions
Potential refinements to RBF
1. Use both penalties and rewards
Human nature: people work hard to avoid penalties
2. Establish a threshold and pay only for activities above it
e.g. antenatal care: pay only for incremental coverage over 90%
3. Pay for improvements over last year’s average as well as attainment
e.g. Last year’s average 4; this quarter 5; improvement 1
4. Pay a fixed dollar amount for remoteness
Current incentives as % of volume are too small
5. Combine RBF with more formative supervision and demand side
Helps providers learn to improve quantity and quality
Acknowledgments
Ministry of Finance, Zimbabwe
Ministry of Health, Zambia
Ministry of Health and Child Care,
World Bank, Zambia
Zimbabwe
World Bank, Zimbabwe
Cordaid, Zimbabwe
World Bank Headquarters
Financial support
 World Bank Health Results
Innovation Trust Fund
Contacts
Donald S Shepard, PhD
Wu Zeng, MD, PhD
[email protected]
[email protected]
+1 781 736 3975
+1 781 736 3888
www.brandeis.edu/~shepard
www.brandeis.edu