Download Bo Sjö - International Initiative for Impact Evaluation

Bo Sjo
Development per Aid Dollar
Data Envelopment Analysis applied to
aid efficiency
Bo Sjö
Email: [email protected]
March 2009
The Aid Efficiency Literature
• Focus on economic growth: “Does aid cause
economic growth in the long run?”
• If not, aid becomes a matter of income distribution
only
• Two conclusions:
– “+1 % in poor countries” taking a correct definition and
condition on correct variables
– “Significance is the result of data mining, remove or add
countries, extends sample and the significant results are
gone”
The typical aid efficiency regression:
Outcome = factors that drives this outcome
+ special factors (dummies)
+ initial level of outcome
+ aid
+ aid  policy
+ randomness
•
•
•
•
Special factors ex: HIV/AIDS frequency, Africa, size, etc
Aid flow measured in previous period
“Aid  policy”, ‘policy’ is supposed to enhance the effects of aid
Outcome is typically the change (improvement) in whatever you
chose to study, most frequently economic growth.
Critique against the growth literature
• Aid has more objectives than economic growth
• Poverty is more than lack of economic resources
–
–
–
–
–
Democracy
Health
Education
Gender issues
Lack of capacities etc..
Limitations of the regression approach
• Only one outcome at a time
• Need to know “factors that drive this outcome”,
• Need to know the technology or the functional form
of whatever drives the outcome in order to say
something about
• the effects of aid,
• and what actually works.
An alternative to regression - DEA
• There are specific problems related to the aid
efficiency that DEA can handle, both on macro and
sector levels.
• Aid has multiple goals and multiple output/outcomes
• A straightforward measure of efficiency that can be
used in decision making
• The ‘technology’ behind aid  development is
complex.
• Think ODA  Development
DEA
• A general definition of efficiency:
• Efficiency = Output/Input
• Relative ranking of aid efficiency across countries
rather than absolute measures
• Calculate efficiency scores for aid efficiency with
multiple outputs
– A ranking of ‘decision making units’ between 0 and 1.0,
where 1 most efficient. All those that get 1.0 represent the
best practice frontier.
DEA
•
•
•
•
•
•
i: number of units
E = Outputs /Inputs
w: J outputs (y)
v: K inputs (x)
Max E
Subj. to E between 1
and 0
J
Ei 
w
j 1
K
i
v
k 1
j
yi
j
k k
i i
x
Pros and Cons of DEA
• Sensitive to outliers but quite forgiving regarding
“technology” behind E = output/input.
– Thus, filter for outliers, work with averages.
• Number of inputs and outputs are limited (of course)
• But, No need to impose the same weights on
different outputs, or the different inputs.
• Inputs can be substituted for each other
• Outputs can be substituted for each other
• “Every unit is put in their best light”
Aid Efficiency - ODA
• E = (Improvements in Development) / Total
ODA p.p
• Development: 4 common indices
• ODA at time t, outcome time t+1
• A pure accounting exercise: to capture
“Development per aid dollar”
A major point in this analysis
I do not have to know the technology. I can assume
that the amount of total ODA to a country was
optimally composed cross sectors to have the
biggest impact wherever possible.
Data issues I
• ODA countries
– A huge number of countries has officially received aid
– Adjust for humanitarian aid
– Filter out too rich, too small, too little ODA, extreme changes
in outputs, etc. to create a more homogeneous sample
without outliers. Around 60 countries left
• Four broad development indices:
•
•
•
•
Poverty/ Economy: PPP-adjusted GDP
Health: Under 5 morality rate
Education: Net primary school enrolment
Democracy & governance: Voice and Accountability index
Data issues II
• Outcomes are measured as relative changes
between periods.
• Net primary school enrolment requires additional
data sources and some interpolation
2 samples, 2 periods and 2 set-ups
• Samples: “All donor countries” and Sweden
• Long period (10+10 year): Total ODA received
1985-1995, outcomes 1996-2005.
• Short period (5+5 year): Total ODA received 19972001, outcomes 2002-2006.
Set-up 1: 4 outcomes /ODA
• A clear inverse relation between ODA received and
development. The more aid the less the country
develops compared to other ODA receiving
countries
• There are some big exceptions, but not more than
randomness could create
• Holds for both “All donor countries” and Sweden
• “Why are you allocating aid in the way you do?”
Set-up 1: Conclusions
1)
2)
3)
4)
Lack of aid is not a restriction for development
Aid could be negative for development – not tested
Donors could be super-selective, – not tested
The problems of finding aid efficiency is not solved
simply by saying that aid has different objectives
5) Aid efficiency must be identified on the margin, or
in a context
Set-up 2: Efficiency against Need and Ability
• In each period ODA is given, and must be allocated
across countries to receive expected outcomes.
• Evaluate against two balancing factors;
– Need for Aid (GDP)
– Ability or Capacity (governance, performance )
• E = outcomes / Input
• 3 Inputs: Total ODA, Mean PPP-adjusted GDP, and
Mean V&A index, 1997-2001
• 2 Outcomes: Improvements in PPP-GDP and V&A,
2002-2006
•
Results – There are differences
Top countries
Bangladesh
Burundi
Colombia
Ethiopia
Mozambique
Ruwanda
Bottom Countries
Bolivia
Top
countries
B Jamaica
Namibia
Mauritius
Botswana
Set-up 2: Results for Sweden
• The bottom list. Similar
results “all donors” and
Sweden.
• Sweden 51 countries in the
sample, 1997-2006
• The bottom list. Compared
to the development in other
countries, and how much
ODA these received the
outcome was relatively bad.
38
Zambia
0.536
176.55
39
Albania
0.532
9.87
40
Ghana
0.531
21.29
41
Mongolia
0.462
11.95
42
Ecuador
0.439
16.29
43
India
0.439
201.78
44
Peru
0.424
24.92
45
Guatemala
0.421
92.81
46
Honduras
0.398
64.70
47
El Salvador
0.375
52.52
48
Nicaragua
0.366
211.28
49
Philippines
0.352
56.77
50
Bolivia
0.342
152.71
51
Namibia
0.334
115.23
Set-up 2: Result the top - Sweden
•
•
•
•
Top 13 countries
First 5 are the relatively most
efficient (1.0)
Yes, Zimbabwe is there.
Conclusion: In relation to the
initial conditions, the allocation
of money was optimal with
respect to the outcome.
Remember the amount of aid is
fixed – and it must be allocated
across countries given initial
relative conditions.
No
Country
CRS
SUMODA
1
Burundi
1.000
14.72
2
Rwanda
1.000
42.62
3
Sudan
1.000
10.51
4
Togo
1.000
3.37
5
Zimbabwe
1.000
171.40
6
Nigeria
0.957
3.83
7
Ethiopia
0.926
213.86
8
Angola
0.907
116.26
9
Mali
0.876
7.93
10
Guinea-Bissau
0.869
38.22
11
Viet Nam
0.865
303.39
12
Mozambique
0.828
391.14
13
Morocco
0.826
4.40
IDA’s CPR and DEA ?
• Is IDA’s Country Performance Index predicting the results in
Set-up 2?
• Answer not really. Further research …
• What about Sweden’s aid to poor performing countries? How
is it motivated? Are there differences is modalities, areas etc.?
Here is a basis for asking questions.
• What about Sweden's land ‘focusation’ from 2007?
• Sweden picked a little bit more countries at the top, and
excluded many at the bottom
• But, still long term development partnership countries might be
a mixed bunch. More questions about differences in policies
etc.
• Has Sweden clearly identified Need and Ability? And the
consequences there-off?
Set-up 2: Conclusions
• It is relevant to ask “How did you choose to allocate
your ODA budget across countries, ex ante?”
• And, “How can we judge that the allocation was
good or bad, ex post?”
• Remember: The link between the amount of aid and
development is inverse.
• Aid is allocated across countries on expected
outcome, and initial conditions. (Do we know
which?)
•
Model 2 Conclusions
• Mozambique and Viet Nam have done well, and
received large amount of aid. Easy to motivate.
• But, Zimbabwe did not so good? Or did it?
• In relation to other countries development, the null
that the relatively large amount of aid was well
balanced cannot be rejected by the results here.
• Is this situation captured by the Government’s letter
of instructions to Sida?
Final
•
•
•
•
•
•
•
DEA is a very ‘soft’ way of measuring development efficiency, no a
priori weights reflecting minimum or acceptable standards or
benchmarks.
All results are based in “constant returns to scale”. Relative efficiency
is measured on a straight line.
Switch to “variable returns to scale” and the differences are less
pronounced. (The scale is reduced). This means that a more
theoretically based, “cause and effects” model might is necessary.
Say, aid is another source of capital that improves productivity in the
economy.
If we are satisfied with the setup of the DEA model.
This exercise leads to a basis for asking deeper questions about the
allocation of aid budgets.
It is possible to ask further question not only about the allocation, but
about the efficiency ranking. What exogenous factors might explain
the ranking which are not captured by the “ability” variable.
These factors are important for aid agencies’ accountability reporting.
If there are systematic factors that predict aid outcomes, these needs
to be accounted for in evaluation and in donors’ performance.