Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
UNFCCC Expert Workshop on Monitoring and Evaluating Capacity-Building in Developing Countries Rio de Janeiro, Brazil, 6-7 November, 2008 Methodologies of monitoring and evaluating capacity development Makoto Kato Japan [email protected] 1 Outline • Defining Capacity, and Capacity Development • Methodologies and Tools for Monitoring and Evaluating Capacity Development • Comparison? M&E at national level? • Uniqueness of Capacity Development in Climate Change area 2 Defining Capacity, and Capacity Development Capacity: Developing countries’ ability to cope with challenges by their own organization and actions. Capacity development: Process in which such capacity is being strengthened and sustained at individual, institutional, and society levels as a whole. Source: JICA 2006 3 Capacity at different levels (Capacity Development must be comprehensive) Consensus in Burden sharing by different stakeholders New Waste Segregation Rules Society Level Policy Goals toward Waste Reduction Institutional Level Awareness raised about Waste Reduction Individual Level Strengthened Management of Waste Control Authority Administrators’ Capacity Enhancement Source: JICA 2006 4 Different Approaches to Capacity Development (Capacity Development should be country-driven and sustainable) Level of Capacity necessary to cope with challenges External Capital Transfer Not Sustainable Existing Capacity Approach A: Transfer of capacity from external player (Filling a Gap with external capital transfer) External Input (Incl. Knowledge Sharing) More Sustainable Existing Capacity Approach B: Developing capacity with external facilitation(building capacity under the ownership of host countries/communities) Source: JICA 2006 5 Key Drivers of Capacity Development • • • • • Ownership Enabling Environment Incentives Leadership Knowledge 6 Methodologies and Tools for Monitoring and Evaluating Capacity-Building (1) Flow of M&E Conceptual Figure of Project Cycle Management(PCM) Plan Evaluation Identification Evaluation Formulation Implementation / Monitoring Preparation Implementation 7 Methodologies and Tools for Monitoring and Evaluating Capacity-Building (2) Where indicators are used: Format of Project Design Matrix (PDM) Narrative Summary Indicators Means of Verification Important Assumption - - - Overall Goal Vertical Logical Relationship Project Objective Outputs Activities Inputs Pre-conditions 8 Methodologies and Tools for Monitoring and Evaluating Capacity-Building (3) How Overalls Goals/project objectives and Indicators are made? Problem Tree Effect Cause Core Problem Methodologies and Tools for Monitoring and Evaluating Capacity-Building (3) How Overalls Goals/project objectives and Indicators are made? Objective Tree Ends Core Objective Feasibility Means Expected Situations Methodologies and Tools for Monitoring and Evaluating Capacity-Building • Evaluation is heavily dependent on “Target Setting” (based on capacity assessment) Target in different levels: Overall Goal, Objectives of Efforts (eg. Policy/Programme/Project objectives) • Indicators are useful only if/when they are elaborated with means of verification, important assumption and precondition (used in PDM) • M&E are properly done by Project Owners (and contracted external evaluators) 11 Can we compare? Same type Projects in 2 different countries Country A Governance Structure Project X Development Priority Country B Governance Structure Development Priority Existing Capacity Existing Capacity Other attributes (eg, Size) Other attributes (eg, Size) Project Y Comparison is only useful for drawing lessons from “Good Practices” (Simple replication does not happen) 12 Can we compare? Same type Projects in the same country Country C Governance Structure Development Priority Project Q (Internal Factors) • Willingness of Stakeholders •Entry point of Efforts etc. Existing Capacity Other attributes (eg, Size) Project P (Internal Factors) • Willingness of Stakeholders •Entry point of Efforts etc. Easier to compare the two, since the external factors are the same or similar. But still internal factors of projects should be taken into account. 13 Can we compare? M&E at National Level Accuracy of Comparison (Evaluation) • Project level>Programme level>Policy level> Regional/International Network (At higher level, quantified indicators may omit useful qualified information) Methods of M&E • We use the same M&E methods(some variety), but indicators are more tailor-made Pre-condition of Defining Indicators • Capacity Assessment (jointly done by D-ing+D-ed countries) is crucial • Assessment(BAU) Monitoring(Indicators) Evaluation (Indicators) must be coherent, and reflect a specific context. 14 Uniqueness of Capacity Development in Climate Change area Narrative Summary Indicators Overall Goal - Means of Important Assumption • ThisVerification Area is already decided by 2/CP7. • Fill in one of the 15 scopes e.g. (promotion of) CDM - Project Objective Outputs Activities Inputs Pre-conditions Entry points of such efforts are different by host countries. Host countries chose such entry point, jointly with our Agencies. 15 Conclusion from our practice • A single methodology (with variation) for M&E is used. • Indicators are used and functions in specific context (within PDM), and simple aggregation of evaluation results is strictly avoided. • Indicators are useful to interpret lessons from Good Practices. • Evaluation at larger level employs more qualitative/narrative way. • Entry points of efforts are different by countries, and it affects selection of indicators. • For climate change, we approach both from “Overall Goal” and from the bottom side of PDM( and it still works). 16