Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Development of a Monitoring and Evaluation System to Support Managing for Development Result (MfDR) By Mr. John Fargher – Australian Team Leader VAMESP II, Mrs. Nguyen Linh Chi – Monitoring and Evaluation Officer – VAMESP II and Mr. Cao Manh Cuong – Foreign Economic Relations Department Ministry of Planning and Investment, Vietnam 1. The National ODA Monitoring System in Vietnam The national monitoring system is a system that provides monitoring data from PMUs to Donors and Line Agencies, then to Government ODA management agencies. There are three level of the system that are linking together. The first level of the system is the Aligned Monitoring Tool (AMT) - a progress reporting tool the government developed to meets its needs for information on disbursement, process and performance of public investments. In this way the major Donors, accounting for 80% of ODA in Vietnam, have aligned with the Government of Vietnam public investment monitoring system. The second level of the system is to build the Portfolio Monitoring Tool (PMT). The PMT is used to capture data from the AMT and compare progress between projects. The PMT uses a small number of benchmarking indicators for disbursement and process as well as a small number of leading sector outcome indicators of performance that are directly linked to the national 5 year plan (the Social and Economic Development Plan 2006-2010 - SEDP). Benchmarking can be performed between projects, across donors and across locations or sectors, depending on the nature of the portfolio of investments being monitored. The PMT is a powerful tool for portfolio management, benchmarking among projects, sectors and regions. The third level is the National Monitoring Tool (NMT) which is a decision support tool that summarises the monitoring data from all public investments as aggregate and benchmarked indicators. These data are designed to support MPI in its obligations to report quarterly to the Prime Minister of contribution of public investment to achieving national strategic outcomes set out in the SEDP. These three key tools are designed to support the work needed to establish and operate a national monitoring system for public investment. The road map towards the national monitoring system is presented schematically in Chart 1. Chart 1 - The road map towards the national monitoring system 1 Effectiveness and efficiency of national M&E system Monitoring supports management for development results Institutionalised feedback & benchmarking Profession of M&E practitioners in Vietnam National data flows from monitoring system Demand for monitoring data to support management Tools support use of institutionalised format GoV and donors agree common set of outcome indicators GoV and Donor data needs aligned in institutionalised format 2006 1.1 2008 2010 Aligned Monitoring Tool (AMT) Since 2004, the Government of Vietnam has worked with the 5 major ODA banks in the country (World Bank, JBIC, ADB, KfW of Germany and AfD of France) to develop and trial a aligned monitoring format. The Ministry of Planning and Investment has led this initiative with 5 other Ministries and 7 Provinces. Most recently, national government agencies have reached agreement on how they can adapt the harmonized monitoring format to meet their needs, and so have a Government of Vietnam monitoring format with which donors can align. AMT is currently being used by more than 40 large ODA projects in Vietnam to monitor the following aspects of public investment implementation in progress, disbursement, procurement (Bidding, contract implementation and payment), safeguards ( Land clearance, Environment), activities to date, and output deliveries AMT enables variance from plan analysis. AMT also provide the automatic rating results for ranking project performance. 1.2 Portfolio Monitoring Tool (PMT) The PMT is designed to monitor the public investment portfolio of a Line Agency. It imports and aggregates data from one or more AMTs – data collected and reported by PMUs - in the portfolio. The PMT then analyses these data and answers performance and management questions. The PMT provides the following functions via a userfriendly interface: automatically import data from AMTs reported by PMUs; stores the list of projects in the portfolio; monitors the progress of projects; produces preformatted Reports, Charts, and Queries; 2 maintains an archive of documents received from projects; automatically export data to other formats such as XML; and benchmarks the progress and performance of projects using predefined indicators. For monitoring performance for a portfolio of investments and activities, the PMT uses the following functions: Extracts data from aligned monitoring tool – project data is extracted from the disbursement sheets (disbursement rate), procurement sheets (process monitoring) and activity sheets (performance monitoring) of the aligned monitoring tool into a database in each national or line agency. Determines the variance from plan – by comparing the planned and actual results the PMT determines the variance from plan for selected performance monitoring indicators and expresses actual performance as a % of planned performance. Tabulates results - the PMT tabulates results for specific process and performance indicators. Performance indicators are set out below. Assigns a rating score to variance – the PMT then assigns a rating score to the performance into one of pre-assigned criteria (0-50% actual as % of plan; 51-75% actual as % of plan, 76-100% actual as % plan and actual ahead of plan). Assigns overall rating - the PMT then produces a table of results organised by rank as an output. The table can be disaggregated by type of process (bid approval, land clearance or contract approval), type of progress (implementation and delivery of goods, works or services activities or outputs), as well as by location or donor. Charts the results – if required, tabular results can be presented as charts or graphs to illustrate results and highlight performance that is either significantly delayed or ahead of plan. PMT can be used either at line agency level for portfolio management, benchmarking or at project level in case the project has many provincial project management units (PPMU) 1.3 National Monitoring Tool (NMT) The national monitoring tool is based on an aligned format. It is also the national monitoring database. This will be finalised by the end of year 2006 and will be a database that enables benchmarking of performance, process and disbursement monitoring data for all investments in Vietnam. This part of the system will be used to prepare feedback to line agencies and management units to support management for development results. It will also contribute to the development of annual reports to Government and donors. 2. Proposed project rating system using data from the monitoring system To make most effective use of data from the AMT, a project rating system is proposed that uses analyses of the AMT data for quick assessment of the performance of a project as well as benchmarking among different projects. The following indicators for rating projects have been proposed and are currently under development: 1) Disbursement; 2) Bidding efficiency; 3) Activity Efficiency; 4) Contract Payments; 5) Resettlement Efficiency; 6) Resettlement Achievement Rate; 7) Environmental Management Efficiency; 8) Performance Indicators 3 A rating derived from the above indicators can be produced directly from the data gathered in the AMT. The principles for selecting indicators in the rating system are: benchmarking indicators, collect data once, simple and believable, quality assurance. The proposed project rating system is a tool to help identify public investment implementation that can provide lessons learned for continuous improvement. For activities ahead of plan, good practices can be shared with similar investments elsewhere. For under-performing activities, higher authority management inputs can be focused to reduce delays and help improve performance. The rating system focuses on disbursement rate and also on performance of public investments. The rating system is a practical tool for making effective use of monitoring data from many investments to support the decision making needed in management for development results. 3. Linking on ground activities and national system 3.1 Local and national monitoring systems Local public investment monitoring systems focus on disbursement, and variance from plan for activities and outputs delivered from national target programs as well as local investment programs. This complements national monitoring systems well and the sensibly connect at the Provincial Line Agency level on this measures. Process monitoring – for example procurement or social and environmental safeguards – are not routinely monitored at commune or district levels. Where there is a provincial project management unit, process monitoring is undertaken. However, as project management is integrated into existing government institutions, consistent with Hanoi Core Statement commitments, this may be something that needs to be developed further for national target programs or large ODA projects. Linkages between local and national monitoring systems are presented schematically in Chart 2. 4 3.2 Options for integrating monitoring systems Considering large, multi-province ODA projects such as Rural Transport 3 or a national public investment program such as P135 or NTPII, we have identified three options for integrating monitoring data reporting and feedback between all levels of government, using the tools and systems successfully piloted by VAMESP II. These options are summarised in Chart 3. Option 1 (Complex but thorough) Option 2 (Comprehensive Option 3 (Practical) All options assume that formats are adapted to be consistent with GSO (Decree 31) and national government agency requirements (eg Decree 17 for ODA) so that CPC and DPC monitoring work can report once to meet the needs of different stakeholders. 5 6