Download Evaluation Office - Asian Development Bank

Evaluation Office
Evaluation for Environmentally
Sustainable Development
Rob D. van den Berg
Shanghai, October
2007
Evaluation Office
What is the GEF?



The Global Environment Facility is an independent
financial mechanism for multilateral environmental
agreements that helps developing countries protect the
global environment while promoting sustainable
development. 32 countries contribute to the GEF trust
fund.
GEF partnerships unite international organizations,
governments, NGOs, scientists and the private sector.
177 countries are members of the GEF.
Evaluation Office
Projects funded by the GEF
Since 1991, the GEF has funded more than 2000
projects in over 151 countries.
Over $ 6 billion in GEF grants.
>$20 billion in co-financing.
Evaluation Office
Global Environment Facility

Ultimate goal is to achieve global
environmental benefits:
–
–
–
–
–
–
Reducing the risks of climate change
Stemming biodiversity loss
Preventing ozone depletion
Safeguarding international waters
Eliminating Persistent Organic Pollutants
Preventing land degradation
Evaluation Office
IAs/EAs
UNDP
CBD
Donor
Replenishment
Group
Evaluation
Office
UNEP
STAP
WB
UNFCC
Assembly
POPs
ADB
NGOs
AfDB
Council
CCD
CEO/Chair
GEF Secretariat
Multilateral
Fund of
Montreal
Protocol
EBRD
FAO
IDB
IFAD
International
Waters
UNIDO
Evaluation Office
GEF links to Multilateral Environmental
Agreements



The designated financial mechanism for the
– Convention on Biological Diversity (CBD)
– Convention on Climate Change (UNFCCC)
– POPs Convention
A designated financial mechanism for the
– Convention on Combating Desertification
(UNCCD)
The GEF collaborates closely with other treaties
and agreements to reach common goals
(International Waters, Montreal Protocol)
Evaluation Office
GEF “Implementing Agencies”
UNDP
UNEP
World Bank
Technical
assistance/
capacity building
projects
Global regional/
and transboundary
projects
Investment
projects
Evaluation Office
GEF “Executing Agencies”

FAO

UNIDO

IFAD

African Development Bank

Asian Development Bank

European Bank for Reconstruction and
Development

Inter-American Development Bank
Evaluation Office
Key Government
Department
Political Focal Point
Operational Focal Point
Implementing Agencies
Other Government
Departments
Executing Agencies
(Global Conventions)
Stakeholders
• NGOs
• Private Sector
• Civil Society
• Other Institutions
1 / 04
Evaluation Office
Six Focal Areas of the GEF






Biodiversity
Climate Change
International Waters
Ozone Depletion (only countries in transition)
Land Degradation
Persistent Organic Pollutants – POPs
Evaluation Office
Current GEF Portfolio
(in millions of US dollars, January 2006)
$157
$156
$181
$516
$2,200
$858
Biodiversity
Climate Change
International Waters
Multi-focal Area
Ozone
POPs
Land Degradation
$2,055
Total GEF
Total Co-Financing
TOTAL
$6,126.72
$20,225.00
$26,351.72
Evaluation Office
Resource Allocation
Framework (RAF)





In Climate Change and Biodiversity: 70% of the funds to go to
individual countries – 25% to groups and 5% to Global and
Regional Projects
Allocation based on past performance (WB index) and on potential
for global environmental benefits
RAF will be extended to other focal areas in GEF-5
Mid-term evaluation of RAF upcoming
Questions and problems noted:
–
–
–
Slow start of RAF
Countries with low capacity will not be able to access GEF sufficiently
Indexes for benefits not sufficiently cognizant of vulnerability and of
marine resources
Evaluation Office
Unrealistic expectations


GEF does not have the means to directly
influence global environmental problems
Example in Climate Change:
–
–

in order to change trends, reductions of 110-150
billion metric tons of GHG are needed per year
currently the GEF has contributed less than 1% over
12 years…
In many focal areas, the GEF has a catalytic
role…
Evaluation Office
Less money in real terms
3,500
GEF1
3,000
GEF2
GEF3
2,500
GEF4
2,000
1,500
1,000
500
0
Replenishment
New money
In '94 dollars
Evaluation Office
GEF Evaluation Office
Evaluation Office
The M & E pyramid…
enablin g environment
oversight
STAP
advice
COUNCIL
Overa ll
P erformance
S tudy
Thematic and
cross-cutting
evaluations,
impact assessments,
country portfolio reviews
APR
Port folio and program review s
Focal ar ea indicators
Ann ual PIR, Proj ects -at-risk systems,
Sup ervi sion
Pr ojec t ind icat ors
Mo nito ring
Mid -term evaluations
M&E pol icy
Evaluation Office
The GEF Evaluation Office
Mission statement:
“Enhancing global environmental benefits
through excellence, independence and
partnership in monitoring and evaluation”
We carry out our work with:
 Impartiality
 Professionalism
 Transparency
Evaluation Office
Five Evaluation Criteria
1 (2)
1.
Relevance – the extent to which the activity is suited to
local and national development priorities and
organizational policies, including changes over time.
2.
Effectiveness: The extent to which an objective has been
achieved or how likely it is to be achieved.
3.
Efficiency: The extent to which results have been
delivered with the least costly resources possible. Also
called cost-effectiveness or efficacy.
Evaluation Office
Five Evaluation Criteria
2 (2)
4.
Results: The positive and negative, and foreseen and
unforeseen, changes to and effects produced by a
development intervention. In GEF terms, results include
direct project outputs, short- to medium term outcomes,
and longer-term impact including global environmental
benefits, replication effects and other, local effects.
5.
Sustainability: The likely ability of an intervention to
continue to deliver benefits for an extended period of time
after completion. Projects need to be environmentally as
well as financially and socially sustainable.
Evaluation Office
Monitoring and Evaluation Policy


Purpose of M&E in the GEF:
– Promote accountability: results, effectiveness,
processes and performance
– Promote learning, feedback and knowledge
sharing as basis for decision making on all
levels
The Policy is based on existing norms and
practices for M&E; it ensures that M&E in the GEF
aspires to the highest international standards
Evaluation Office
Minimum requirements
1.
2.
3.
Project Design: All projects will include a
concrete and fully budgeted M&E plan
Application of Project M&E: Project
monitoring and supervision will include
implementation of the M&E plan
Project Evaluation: Each Full Sized Project
(and Medium Sized Project) will be evaluated
at end of implementation
Evaluation Office
Use of evaluations
Evaluation
Use
Follow-up (MAR)
OPS3
Replenishment
OPS4
Local benefits study
Project preparations and oversight
APR
Biosafety
Better focus on countries needs
New GEF biosafety
strategy
CPE Costa Rica
Better interaction with countries –
CR: better RAF strategy
In RAF
Annual Performance
Report
Improvements terminal evaluations,
M&E planning and oversight
APR
Joint Activity Cycle
Evaluation
Redesign of activity cycle
Council in June
Incremental Costs
Reformulation of ICA
Council in June
Evaluation Office
GEF Evaluation Office
Work Program 2007-10
2007-08









Capacity Development
Evaluation
Annual Performance Report
Annual Report on Impact
RAF mid-term review
4 Country Portfolio
Evaluations in Africa
Catalytic Role of the GEF
GEF Focal Areas
Evaluation of Partnership
and umbrella projects
Small Grants Programme
2009-10





Country Portfolio
Evaluations
Annual Performance
Report
Annual Report on Impact
Six Focal Area evaluations
Fourth Overall
Performance Study
Evaluation Office
Development of results indicators




Results: outputs, outcomes, impacts
Outputs: the products, capital goods and services
which result from a development intervention; may also
include changes resulting from the intervention which
are relevant to the achievement of outcomes
Outcomes: the likely or achieved short-term and
medium-term effects of an intervention’s outputs
Impacts: positive and negative, primary and secondary
long-term effects produced by a development
intervention, directly or indirectly, intended or
unintended.
Evaluation Office
Output and outcome

Output indicators are usually non-problematic and part
of the regular monitoring, and final outputs are reported
in the terminal evaluation
–

Outcomes: indicators should be chosen carefully to
measure achievement of desired outcomes
–

Indicators should be “SMART”
Focal area task forces have worked on “tracking tools” and
indicators to be used throughout their portfolios
GEF Evaluation Office supports development of
indicators
Evaluation Office
GEF “SMART” performance indicators:





Specific: Capture the essence of the desired result by clearly and
directly relating to achieving an objective, and only that objective.
Measurable: Specify indicators unambiguously so that all parties
agree on what the system covers and there are practical ways to
measure the indicators and results.
Achievable and Attributable: Identify what changes are anticipated
as a result of the intervention and whether the result(s) are realistic.
Attribution requires that changes in the targeted developmental issue
can be linked to the intervention.
Relevant and Realistic: Establish levels of performance that are
likely to be achieved in a practical manner, and that reflect the
expectations of stakeholders.
Time-bound, Timely, Trackable, and Targeted: Track progress in a
cost-effective manner at desired frequency for a set period, with clear
identification of the particular stakeholder group to be impacted by the
project or program.
Evaluation Office
Impact of the GEF




Impact of GEF interventions is usually beyond time
horizon
Sometimes also beyond the physical boundaries of an
intervention
Impact will have to be established in collaboration with
others (UNEP, scientific community, WRI, IUCN, etc.)
But if no impact indicators are adopted, no
collaboration will be possible and no impact can be
attributed to the GEF when it finally occurs