Download 1. The National ODA Monitoring System in Vietnam

Development of a Monitoring and Evaluation System
to Support Managing for Development Result (MfDR)
By Mr. John Fargher – Australian Team Leader VAMESP II,
Mrs. Nguyen Linh Chi – Monitoring and Evaluation Officer – VAMESP II
and Mr. Cao Manh Cuong – Foreign Economic Relations Department
Ministry of Planning and Investment, Vietnam
1.
The National ODA Monitoring System in Vietnam
The national monitoring system is a system that provides monitoring data from PMUs to
Donors and Line Agencies, then to Government ODA management agencies. There are
three level of the system that are linking together.
The first level of the system is the Aligned Monitoring Tool (AMT) - a progress reporting
tool the government developed to meets its needs for information on disbursement,
process and performance of public investments. In this way the major Donors,
accounting for 80% of ODA in Vietnam, have aligned with the Government of Vietnam
public investment monitoring system.
The second level of the system is to build the Portfolio Monitoring Tool (PMT). The PMT
is used to capture data from the AMT and compare progress between projects. The
PMT uses a small number of benchmarking indicators for disbursement and process as
well as a small number of leading sector outcome indicators of performance that are
directly linked to the national 5 year plan (the Social and Economic Development Plan
2006-2010 - SEDP). Benchmarking can be performed between projects, across donors
and across locations or sectors, depending on the nature of the portfolio of investments
being monitored. The PMT is a powerful tool for portfolio management, benchmarking
among projects, sectors and regions.
The third level is the National Monitoring Tool (NMT) which is a decision support tool that
summarises the monitoring data from all public investments as aggregate and
benchmarked indicators. These data are designed to support MPI in its obligations to
report quarterly to the Prime Minister of contribution of public investment to achieving
national strategic outcomes set out in the SEDP.
These three key tools are designed to support the work needed to establish and operate
a national monitoring system for public investment. The road map towards the national
monitoring system is presented schematically in Chart 1.
Chart 1 - The road map towards the national monitoring system
1
Effectiveness and efficiency of national M&E system
Monitoring supports management for development results
Institutionalised feedback & benchmarking
Profession of M&E practitioners in Vietnam
National data flows from monitoring system
Demand for monitoring data to support management
Tools support use of institutionalised format
GoV and donors agree common set of outcome indicators
GoV and Donor data needs aligned in institutionalised format
2006
1.1
2008
2010
Aligned Monitoring Tool (AMT)
Since 2004, the Government of Vietnam has worked with the 5 major ODA banks in the
country (World Bank, JBIC, ADB, KfW of Germany and AfD of France) to develop and
trial a aligned monitoring format. The Ministry of Planning and Investment has led this
initiative with 5 other Ministries and 7 Provinces. Most recently, national government
agencies have reached agreement on how they can adapt the harmonized monitoring
format to meet their needs, and so have a Government of Vietnam monitoring format
with which donors can align.
AMT is currently being used by more than 40 large ODA projects in Vietnam to monitor
the following aspects of public investment implementation in progress, disbursement,
procurement (Bidding, contract implementation and payment), safeguards ( Land
clearance, Environment), activities to date, and output deliveries
AMT enables variance from plan analysis. AMT also provide the automatic rating results
for ranking project performance.
1.2
Portfolio Monitoring Tool (PMT)
The PMT is designed to monitor the public investment portfolio of a Line Agency. It
imports and aggregates data from one or more AMTs – data collected and reported by
PMUs - in the portfolio. The PMT then analyses these data and answers performance
and management questions. The PMT provides the following functions via a userfriendly interface:
 automatically import data from AMTs reported by PMUs;
 stores the list of projects in the portfolio;
 monitors the progress of projects;
 produces preformatted Reports, Charts, and Queries;
2



maintains an archive of documents received from projects;
automatically export data to other formats such as XML; and
benchmarks the progress and performance of projects using predefined indicators.
For monitoring performance for a portfolio of investments and activities, the PMT uses
the following functions:
 Extracts data from aligned monitoring tool – project data is extracted from the
disbursement sheets (disbursement rate), procurement sheets (process monitoring)
and activity sheets (performance monitoring) of the aligned monitoring tool into a
database in each national or line agency.
 Determines the variance from plan – by comparing the planned and actual results
the PMT determines the variance from plan for selected performance monitoring
indicators and expresses actual performance as a % of planned performance.
 Tabulates results - the PMT tabulates results for specific process and performance
indicators. Performance indicators are set out below.
 Assigns a rating score to variance – the PMT then assigns a rating score to the
performance into one of pre-assigned criteria (0-50% actual as % of plan; 51-75%
actual as % of plan, 76-100% actual as % plan and actual ahead of plan).
 Assigns overall rating - the PMT then produces a table of results organised by rank
as an output. The table can be disaggregated by type of process (bid approval, land
clearance or contract approval), type of progress (implementation and delivery of
goods, works or services activities or outputs), as well as by location or donor.
 Charts the results – if required, tabular results can be presented as charts or
graphs to illustrate results and highlight performance that is either significantly
delayed or ahead of plan.
PMT can be used either at line agency level for portfolio management, benchmarking or
at project level in case the project has many provincial project management units (PPMU)
1.3
National Monitoring Tool (NMT)
The national monitoring tool is based on an aligned format. It is also the national
monitoring database. This will be finalised by the end of year 2006 and will be a
database that enables benchmarking of performance, process and disbursement
monitoring data for all investments in Vietnam. This part of the system will be used to
prepare feedback to line agencies and management units to support management for
development results. It will also contribute to the development of annual reports to
Government and donors.
2.
Proposed project rating system using data from the monitoring system
To make most effective use of data from the AMT, a project rating system is proposed
that uses analyses of the AMT data for quick assessment of the performance of a project
as well as benchmarking among different projects.
The following indicators for rating projects have been proposed and are currently under
development: 1) Disbursement; 2) Bidding efficiency; 3) Activity Efficiency; 4) Contract
Payments; 5) Resettlement Efficiency; 6) Resettlement Achievement Rate; 7)
Environmental Management Efficiency; 8) Performance Indicators
3
A rating derived from the above indicators can be produced directly from the data
gathered in the AMT. The principles for selecting indicators in the rating system are:
benchmarking indicators, collect data once, simple and believable, quality assurance.
The proposed project rating system is a tool to help identify public investment
implementation that can provide lessons learned for continuous improvement. For
activities ahead of plan, good practices can be shared with similar investments
elsewhere. For under-performing activities, higher authority management inputs can be
focused to reduce delays and help improve performance. The rating system focuses on
disbursement rate and also on performance of public investments. The rating system is
a practical tool for making effective use of monitoring data from many investments to
support the decision making needed in management for development results.
3.
Linking on ground activities and national system
3.1
Local and national monitoring systems
Local public investment monitoring systems focus on disbursement, and variance from
plan for activities and outputs delivered from national target programs as well as local
investment programs. This complements national monitoring systems well and the
sensibly connect at the Provincial Line Agency level on this measures.
Process monitoring – for example procurement or social and environmental safeguards
– are not routinely monitored at commune or district levels. Where there is a provincial
project management unit, process monitoring is undertaken. However, as project
management is integrated into existing government institutions, consistent with Hanoi
Core Statement commitments, this may be something that needs to be developed
further for national target programs or large ODA projects.
Linkages between local and national monitoring systems are presented schematically in
Chart 2.
4
3.2
Options for integrating monitoring systems
Considering large, multi-province ODA projects such as Rural Transport 3 or a national
public investment program such as P135 or NTPII, we have identified three options for
integrating monitoring data reporting and feedback between all levels of government,
using the tools and systems successfully piloted by VAMESP II. These options are
summarised in Chart 3.

Option 1 (Complex but thorough)

Option 2 (Comprehensive

Option 3 (Practical)
All options assume that formats are adapted to be consistent with GSO (Decree 31) and
national government agency requirements (eg Decree 17 for ODA) so that CPC and DPC
monitoring work can report once to meet the needs of different stakeholders.
5
6