Download 1 - rapc.gov.md

Experts Group within the Government Office
(Public Administration Reform Coordination Unit)
METHODOLOGY OF MONITORING AND
EVALUATION OF THE CENTRAL PUBLIC
ADMINSITRATION REFORM IN THE
REPUBLIC OF MOLDOVA
Chisinau, 2006
1
Table of Contents
I. METHODOLOGY OF MONITORING AND EVALUATION OF THE CENTRAL
PUBLIC ADMINISTRATION REFORM.................................................................................. 3
1. Monitoring and Evaluation Objectives .................................Error! Bookmark not defined.
2. Monitoring and Evaluation Mechanism and Concept ...........Error! Bookmark not defined.
3. Monitoring and Evaluation Framework ................................Error! Bookmark not defined.
4. Monitoring and Evaluation Process. Systematic Reporting. Calendar of Monitoring and
Evaluation Activities .............................................................................................................. 7
5. Evaluation Procedure of the CPA Reform Succes ................Error! Bookmark not defined.
6. Plan of Performance Monitoring and Evaluation of CPA Reform Results Error! Bookmark
not defined.
II. PLAN OF APPLICATION OF CPA REFORM EVALUATION AND MONITORING
METHODS AND INSTRUMENTS ...............................................Error! Bookmark not defined.
ATTACHMENTS........................................................................................................................ 22
Attachment 1. Indicators of Measuring CPA Reform Result.................................................... 22
Attachement 2. Indicators of Measuring CPA Reform Performances in the Republic of
Moldova ............................................................................................................................... 23
Attachment 3. Central Public Administration Reform Evaluation and Monitoring Scheme ..... 29
Attachment 4. Schedule of main events for 2006 ...................................................................... 30
2
I. METHODOLOGY OF MONITORING AND EVALUATION OF THE
CENTRAL PUBLIC ADMINISTRATION REFORM
1.
Monitoring and Evaluation Objectives
The main objective of the monitoring and evaluation of the central public administration
reform (CPA reform) progress has been defined as the efficient CPA reform implementation
according to the Government Decision No.1402 of December 30, 2005.
The objectives of the CPA reform monitoring are:
 ensuring the correlation of the activities stipulated in the Implementation Plan of the
Central Public Administration Reform Strategy in the Republic of Moldova with the
objectives and expected results of the CPA reform as a whole;
 identification of the progress, results and challenges in the CPA reform implementation.
The objectives of the CPA reform evaluation are:
 ensuring the fact that the implementation of the CPA reform is efficient, meets the
deadlines and has a positive impact;
 identifying if the objectives of the Central Public Administration Reform Strategy in the
Republic of Moldova are achieved.
2. Monitoring and Evaluation Mechanism and Concept
Monitoring is an on-going process of collecting and analyzing information used for
measuring the progress of the Central Public Administration Reform Strategy in the Republic of
Moldova according to the objectives and expected results. Monitoring provides regularly the
interested persons with information about the deviations from the Implementation Plan of the
CPA Reform Strategy and from the action plans of the ministries and other central administrative
authorities coordinated with the Public Administration Reform Coordination Unit (PAR Unit)
and helps identify problems hampering the CPA reform implementation.
Evaluation shows the importance, implementation progress and results of the CPA reform as
a whole, highlights the most important results and determines the CPA reform impact, provides
information about the application of the best practices by the persons involved in the CPA reform
implementation, helps identify the corrective actions when necessary.
The Monitoring and Evaluation System of the CPA reform implementation includes the
following elements:
 CPA Reform Monitoring and Evaluation Mechanism – describes the CPA reform
monitoring and evaluation forms, methods and instruments;
 Monitoring and Evaluation Framework – describes the performance indicators for the
purpose of monitoring the CPA reform implementation according to the components,
objectives and actions of the CPA Reform Strategy; indicators of CPA reform result
evaluation (impact and capacities); indicators for determining the success of the CPA
reform implementation;
 Monitoring and evaluation process – describes the coordination mechanism and methods
of implementing monitoring and evaluation actions;
 The calendar of the monitoring and evaluation activities is a schedule of the monitoring
and evaluation activities for the period of the reform implementation
(month/quarter/year).
The Monitoring Mechanism includes office and field monitoring and drawing up
monitoring reports. The office monitoring includes preparation of two types of documents with a
3
specific format: the document accompanying the monitoring visit in the field and the quarterly
and annual monitoring reports. The document accompanying the field monitoring visit includes
information about the implementation phase of the actions stipulated in the action plans of the
ministries and other central administrative authorities, the identified problems, other information
which will be discussed during the monitoring visit in the field.
Field monitoring is carried out for the purpose of analyzing the way the reported activities of
the central public administration authorities (CPAA) have been implemented. The date of the
monitoring visit is agreed upon with the CPAA. After carrying out the monitoring visit a
monitoring report is compiled according to the unified requirements, containing conclusions and
recommendations. This report on the monitoring visit is sent to the concerned CPAA, which is
obliged to draw up, in case of necessity, a plan of actions (measures) for implementing the
recommendations, to set the timeframe and to appoint the persons in charge of this. The data
provided by the field monitoring are compared with the data possessed by the PAR Unit.
The monitoring is done by PAR Unit, which does both office and field monitoring together
with the working groups within CPAA which monitor the implementation of the own action
plans.
The suggested monitoring mechanism includes three types of monitoring reports: the field
monitoring report per each visit at CPAA, the quarterly monitoring report (report on CPA
reform progress) based on the information provided by CPAA and the annual monitoring report
based on the information provided by CPAA for the quarter IV and data provided by the
evaluation of the results, applying different evaluation instruments and studies, and by the
evaluation of the CPA reform success.
The monitoring is accompanied by the evaluation of the CPA reform results: the annual
evaluation which will be done by means of research, questionnaires, evaluation forms and result
indicators; the final evaluation and post-evaluation. Each type of evaluation ends up with an
evaluation report, respectively the annual evaluation report, the final evaluation report and the
post-evaluation report.
The monitoring and evaluation mechanism includes the following monitoring and evaluation
instruments (techniques):
 The Implementation Plan of the Central Public Administration Reform Strategy – in
operational terms describes the strategy in details per each component, objective and
action;
o Action plans of ministries and other central public authorities – in operational
terms describe in details the actions implemented by the ministries and other CPA
authorities according to the actions of the Implementation Plan of the CPA
Reform Strategy;
 Quarterly reports on the progress of central public administration reform in the
Republic of Moldova – reflect the progress, results and challenges arisen in the
implementation process of the Implementation Plan of the CPA Reform Strategy and
solutions for eliminating identified drawbacks;
o Quarterly (and annual) information (reports) on the CPA reform progress at
territorial level – reflects the progress, results and challenges arisen in the
implementation process of the CPA reform at the level of CPA authority
(territorial);
 Annual monitoring report – reflects the progress, results and challenges of the CPA
reform implementation as a whole, and the solutions to eliminate the identified
drawbacks;
 Reports on the evaluation of the achievement of certain objectives (the most important
ones or the ones delayed) of the CPA Reform Strategy, which will be highlighted as a
result of monitoring the CPA reform as a whole for a certain period (annually, as a rule),
4
at the end of the CPA reform implementation period, as well as after three years from the
implementation of the CPA reform;
 Field monitoring visits at ministries and other central administrative authorities, and
other interested persons involved in the CPA reform implementation – are carried out in
order to make sure that the results of the CPA reform implementation are reflected in the
actions and measures applied by the persons in charge; the field monitoring visit ends
with a Field Monitoring Report.
 Research, questionnaires, evaluation forms and other ways of getting information –
which will provide information about the CPA reform progress, as well as about the
results and the problems identified in the CPA reform implementation process, about the
quality of the delivered training services and of other informational events (study visits,
conferences, round tables) etc.;
 Meetings of the Governmental Commission for CPA Reform – examine the reports
and analyze the achievements obtained as a result of the implementation of the major
actions;
 Meetings of the Consultative and Supervisory Committee for CPA reform –
coordinate and monitor the planned actions at each reform implementation phase;
 The annual meetings – examine the Annual Report on the CPA Reform Progress, as
well as the CPA reform progress as compared to the objectives and actions of the CPA
Reform Strategy, the problems identified in the CPA implementation process, the
formulation of the conclusions and the updating of the Implementation Plan for the
following year. The formal documents are not issued.
These monitoring and evaluation instruments and methods will be applied by the PAR Unit
in order to show that the CPA reform progress corresponds to the objectives and actions of the
CPA Reform Strategy in the Republic of Moldova; in order to determine the impact of the CPA
reform on the capacity building of the central public administration authorities.
The methodology of assessing the impact of the CPA reform as a whole. The evaluation of
the CPA reform as a whole is carried out by combining the multilateral analysis with the analysis
of the documents, by consulting the concerned persons and by means of field visits at the central
public administration authorities and the concerned persons, and by determining the feedback.
The evaluation of the CPA reform as a whole is done with the help of questionnaires, research,
field visits and interviews for getting information.
The evaluation of the CPA reform as a whole is focused on the determination of the:
 importance,
 degree of implementation,
 success of the CPA reform, consisting of:
o CPA reform impact on the building of an efficient and democratic governance;
o contribution of the CPA reform implementation to the capacity building of the
central public administration;
o degree of CPA reform success.
According to this methodology, the indicators of measuring the CPA reform impact as a whole
and of measuring the CPA capacities will be applied and the progress of CPA reform
implementation as a whole will be evaluated.
Using the monitoring and evaluation concept described above, the following sections
describe the elements of the Monitoring and Evaluation System of the central public
administration reform:
 Monitoring and Evaluation Framework;
5


Monitoring and evaluation process, systemic reporting and calendar of the monitoring
and evaluation activities within CPA reform;
Method of evaluation of the CPA reform success as a whole, applying the numerical
rating.
3. Monitoring and Evaluation Framework
The monitoring and evaluation system is based on the setting of a number of indicators which
measure the progress (performances) and the results achieved in the CPA reform implementation
process. The monitoring and evaluation framework includes the following components:
 The initial value of the performance indicators (by January 1, 2006) used to describe the
state of things before introducing the CPA Reform Strategy;
 The target value which describes the value of the indicator which has to be achieved by
the end of the implementation period of the CPA Reform Strategy;
 CPA reform progress impact indicators;
 The target value of the impact indicators which reflects the value of the indicator which
has to be achieved at the end of the period of CPA Reform Strategy implementation;
 Capacities of information gathering: information sources, methods and instruments of
collecting primary data, formulas of calculation secondary data; persons involved in data
collecting.
Two types of indicators will be applied to monitoring and evaluation:
 Performance indicators measuring the degree of implementation of the actions provided
in the Implementation Plan of the CPA Reform Strategy and of the CPA reform as a
whole for a definite period (quarter/year) as compared with the initial value of the
indicators at the beginning of the CPA reform implementation;
 Result indicators which reveal the impact of the CPA reform on the beneficiaries,
society and citizens.
Only some of the actions provided in the Implementation Plan of the CPA Reform Strategy
can be numerically measured. That’s why the system of indicators of monitoring and evaluation
includes quantitative and qualitative indicators. The quantitative indicators are calculated by
applying calculation methods (formulas). The qualitative indicators determine if the actions are
implemented according to the requirements or quality standards and identify the ways of
eliminating the unsatisfactory performances. The progress in consolidating the institutional
framework of the central public administration authorities, the quality of the training programs,
study visits, conferences and other informative events can be measured by converting the
qualitative descriptions into quantitative information, using numerical rating.
Each performance indicator is characterized by the following elements:
a) The source of information (bodies of the central public administration which provide
data) and persons in charge of data collecting;
b) Frequency of data collecting;
c) Primary data collecting methods: questionnaires, evaluation forms, etc.;
d) Secondary data calculation methods and formulas;
e) Place where data are kept.
For the purpose of elaborating the framework of CPA reform progress monitoring and CPA
reform impact evaluation and for the purpose of identifying if the CPA reform implementation
6
corresponds to the objectives and actions of the CPA Reform Strategy, the following actions
must be taken:
 To select the performance indicators for measuring the CPA reform progress;
 To determine the initial value of the performance indicators at the beginning of the CPA
reform implementation (on January 1, 2006);
 To determine the target value of the performance indicators which has to be achieved by
the end of the CPA reform implementation period;
 To select the indicators of measuring the result of CPA reform implementation (impact
and CPA capacities);
 To determine the target value of the result indicators which has to be achieved by the end
of the CPA reform implementation period;
 To elaborate the Monitoring and Evaluation Mechanism which includes the planning
methods, types of reports, methods and techniques of collecting quantitative and
qualitative information, meeting forms, etc.
The impact of the central public administration reform as a whole is evaluated by means of
an approach based on results and is measured by applying impact indicators of reform
implementation and CPA capacity indicators. The evaluation framework, which includes the
list of impact indicators and the description of the indicators, is laid out in Attachment 1.
The progress of the central public administration reform as a whole is monitored by applying
the approach based on results and is measured by applying performance indicators. The
monitoring framework, which includes the list of performance indicators used to monitor the
progress of CPA reform implementation per component and objective of CPA Reform Strategy,
and the description of each indicator, is laid out in Attachment 2. The data about the factual
value of the monitoring indicators received from ministries and other central administrative
authorities must be kept in digital records to which the PAR Unit consultants (Experts Group
within the Government Office) should have access.
The primary data used to monitor the progress and to evaluate the CPA reform result will be
received from ministries and other CPA authorities or collected by the persons in charge of
monitoring from the persons involved in the CPA reform implementation, using monitoring and
evaluation instruments (techniques) (field monitoring visits, evaluation forms, research,
questionnaires, etc.)
4. Monitoring and Evaluation Process. Systematic Reporting.
Calendar of Monitoring and Evaluation Activities
The monitoring process is determined by the Reform Strategy of the Central Public
Administration in the Republic of Moldova approved through Government Decision no. 1402 of
December 30, 2005. The monitoring and evaluation process specifies the institutional
responsibilities of the PAR Unit, ministries and other central administrative authorities and of
other involved persons, ensures the participation of the involved persons and of the civil society,
introduces transparency and identifies the problems and good practices in the CPA reform
implementation.
The monitoring and evaluation process imposes the following requirements:
 Delimitation of responsibilities in the field of monitoring and evaluation;
 Elaboration of the calendar of monitoring and evaluation activities regarding the
application of monitoring and evaluation instruments and the presentation of reports
during CPA reform implementation;
 Requirements to be met by reports on monitoring and evaluation of the CPA reform
implementation and the frequency of reporting.
7
Responsibilities. According to the Government Decision no.1402 of December 30, 2005 and
CPA Reform Strategy, the Public Administration Reform Coordination Unit (Experts Group
within the Government Office):
 monitors the central public administration reform implementation;
 provides the necessary methodological assistance to ministries and other central
administrative authorities in the reform implementation process;
 informs regularly the Governmental Commission (GC) on central public administration
reform about the reform progress;
 coordinates the support provided by donors during the whole CPA reform implementation
period, as well as ensures the management of the financial means of donors (Trust Fund).
The main responsibilities of the persons in charge of the monitoring and evaluation field are
included in the functions and terms of reference. Three persons in the PAR Unit have
responsibilities in the field of monitoring and evaluation of the CPA reform implementation: one
main consultant in the monitoring and coordination of the implementation of the Implementation
Plan of the CPA Reform Strategy, one consultant in monitoring and evaluation of the CPA
reform as a whole, one consultant in monitoring and evaluation of the CPA reform at territorial
level.
In order to achieve the monitoring and evaluation objectives, besides the above mentioned
persons from the PAR Unit, some persons from ministries and other central administrative
authorities are also involved in the monitoring and evaluation process, and according to the
Government Decision no.1402 of December 30, 2005 are responsible for ”providing information
required by the persons in charge of ” the implementation of the Implementation Plan of the CPA
Reform Strategy and provision of the „information about the implementation of the
Implementation Plan of the Central Public Administration Reform Strategy in the Republic of
Moldova (for the component they are responsible for)”. Working groups are set up in ministries
and other central administrative authorities; their members are responsible for the CPA reform
implementation, the appointed deputy minister is responsible for the CPA reform implementation
within the ministry and other central public authorities.
PAR Unit consultants consolidate and generalize the information received from ministries
and other central administrative authorities and draw up reports and informative notes. According
to the Government Decision no.1402 of December 30, 2005 and CPA Reform Strategy, PAR
Unit:
 informs „regularly the Governmental Commission for Central Public Administration
Reform about the progress of the reform” and presents „systematic reports.... which will
include as well solutions to eliminate the identified drawbacks”.
According to the CPA Reform Strategy, the permanent monitoring of the reform
implementation process is „done by the Supervisory Committee...with the view to identifying the
achievements and actions set forth by the stakeholders”.
The responsibilities of the stakeholders classified according to monitoring and evaluation
instruments are presented in table 1.
Table 1. Main Responsibilities and Timeframe for Presenting CPA Reform Monitoring and
Evaluation Documents
Monitoring
Instruments
CPA Reform
Strategy
1
PAR Unit / consultants
Elaborates, updates annually
and submits the
Responsibilities of the stakeholders
Ministries and other
Consultative and
CAB1
Supervisory
Committee (SC)
Coordinate the
Coordinates the
Implementation Plan
planned actions.
Governmental
Commission (GC),
Government
Approves the
Implementation
CAB – central administrative bodies
8
Implementation
Plan
CPAA Action
Plans
Quarterly reports
on the progress
of the reform
Quarterly
information
about CPA
reform progress
at territorial
level.
Annual report on
the monitoring of
the CPA reform.
Annual report on
the evaluation of
the CPA reform
progress
Field visits at
CPAA and other
persons involved
in CPA reform
implementation
Studies,
questionnaires,
evaluation forms
etc.
Meetings of the
Consultative and
Supervisory
Committee
Meetings of the
Governmental
Implementation Plan to the SC
and GC by October 1, this
year.
Provides methodological
assistance to CPAA in the
elaboration of the action plans.
Elaborates reports according
to the unified requirements
within 21 days from the
finalization of the reporting
period, suggests solutions for
eliminating identified
drawbacks and presents them
to the SC, GC, Parliament and
Presidency.
Ensures methodological
assistance to ministries and
other CAB.
Elaborates the report based on
the information provided by
CPAA and on the annual
evaluation report, submits it to
SC, GC, Parliament and
Presidency within 21 days
from the end of the reporting
period.
Identifies the objectives of the
strategy which are to be
evaluated. Elaborates the
annual evaluation report and
presents it to SC and GC by
December 25 this year.
Visits each CPAA at least
once in three months and
checks the results and actions.
Visits other persons involved
in CPA reform
implementation, when
necessary.
Organizes the gathering of
information provided by
CPAA and the evaluation of
the feedback about the reform
implementation. Suggests and
provides grounds for the
corrective actions.
Organizes meetings and
distributes materials.
Organizes meetings and
distributes materials.
according to the
procedure in force.
Plan.
Elaborate own action
plans and submit them
(paper and digital
variants) to PAR Unit by
September 1 this year.
Not applicable
May be informed
Approves the
CPAA action plans
Ensures the
communication
and coordination
of the actions
among the
stakeholders.
Approves the
results.
Elaborate and present to
PAR Unit information
about CPA reform
progress at territorial level
within 7 days from the
finalization of the
reporting period in
written and electronic
form.
Not applicable
Not applicable
Not applicable
Examines the
report and
ensures the
coordination of
actions.
Identifies the
results of the
implementation of
major actions,
takes decisions on
the reform policy.
Provide information
required from persons in
charge of the evaluation
of the CPA reform results.
Examines the
report and
ensures the
coordination of
actions.
Provide information
required by the persons in
charge of the field
monitoring.
Is informed.
Approves the
results, takes policy
decisions in the
field of the CPA
reform, when
necessary.
Is informed.
Provides information
required by the persons in
charge.
Monitors and
coordinates the
implementation
of the corrective
actions.
May be informed
about the results.
Takes decisions on
the corrective
actions.
Not applicable
Monitors and
coordinates the
reform
implementation.
Not applicable
Is informed about
the identified
problems
Not applicable
Analyzes the
results of the
9
Commission
Annual meetings
implementation of
the major actions.
The meetings are of participatory nature and are attended by donors and other persons involved in the
CPA reform implementation. The meeting takes place in January at the end of the reporting year.
The training programs and other informational events (study visits, conferences, round tables,
etc.) are evaluated at the end of the training program (the last day).
The annual meeting takes place in the fourth quarter of the year in order to formulate
conclusions and recommendations for the updating of the Implementation Plan for the following
year.
The CPA reform monitoring and evaluation process has two phases, each including several
activities (see chart 1):
 Phase I. Actions of monitoring the reform implementation at the level of ministries and
other central administrative authorities carried out by persons in charge within these
central public administration authorities with the methodological assistance of the PAR
Unit consultants, including:
o Activity 1.1. Determining the efficiency and degree of implementation of the action
plans of ministries and other central public administrative authorities;
o Activity 1.2. Determining the factual values of the performance and result
indicators of CPA reform implementation within ministries and other central
administrative authorities;
o Activity 1.3. Identifying problems and measures for eliminating problems
hampering reform implementation within central public authorities.
 Phase II. Actions of monitoring and evaluation of CPA reform as a whole undertaken
by the PAR Unit consultants, including:
o Activity 2.1. Determining the efficacy and the degree of implementation of the
Implementation Plan of the Central Public Administration Reform Strategy;
o Activity 2.2. Determining the performance and result indicators of CPA reform
implementation as a whole;
o Activity 2.3. Monitoring costs of CPA reform implementation (the using of
resources from the state budget and Trust Fund).
o Activity 2.4. Determining the achievements and identifying the problems arisen in
the implementation of the CPA reform as a whole, elaborating measures to solve
these problems;
o Activity 2.5. Identifying the objectives of the CPA Reform Strategy which will
undergo evaluation (of major importance or the achievement of which is delayed).
The sequence and links between the actions in the CPA reform monitoring and evaluation
process are shown in chart 1.
The calendar of monitoring and evaluation activities. The scheme (schedule) of
monitoring and evaluation activities, presenting main types of activities, reports and events is
shown in Attachment 3. In case if certain main events are frequent, a monthly schedule may be
used as part of the general scheme (one example is given in Attachment 4).
According to CPA reform monitoring and evaluation methodology, ministries and other
central administrative authorities must present quarterly information about the CPA reform
progress within seven calendar days after the end of the reporting period. Data about the factual
value of the performance indicators and result indicators (presented annually) will be part of the
information presented annually by ministries and other central administrative authorities to PAR
Unit.
10
The frequency of drawing up monitoring reports on CPA reform implementation progress is
determined through Government Decision no.1402 of December 30, 2005 according to which the
Reports on central public administration reform progress in the Republic of Moldova must be
drawn up quarterly. The PAR Unit consultants will draw up quarterly and annual reports (quarter
IV) on CPA reform progress within twenty one calendar days from the end of the reporting
period and will submit them to the Supervisory Committee, Governmental Commission,
Parliament and Presidency.
According to the monitoring and evaluation methodology, the CPA reform as a whole will be
the subject of evaluation carried out by the PAR Unit consultants during the annual period
(quarter IV), at the end of the CPA reform implementation period (quarter 3, 2008 for getting
information for the Final Report) and after a period of about three years from the end of the
technical assistance provided by donors for the reform implementation with the view to
determining the sustainable impact of the CPA reform.
Chart 1. The Monitoring and Evaluation Process of the CPA Reform
Phase I
Phase II
Activity 1.1
Activity 2.1
Activity 1.2
Activity 2.2
Activity 1.3
Activity 2.3
Activity 2.4
Activity 2.5
Requirements to reporting. PAR Unit presents the quarterly reports on central public
administration reform progress in the Republic of Moldova, based on the evaluation of the CPA
reform implementation progress as compared to the objectives and actions provided in the
Reform Strategy and included in the Strategy Implementation Plan. The implementation progress
of the CPA reform as a whole is determined based on the analysis of the degree of
implementation of the actions stipulated in the Annual Implementation Plan of the CPA Strategy
and on the degree of implementation of actions within CPAA, while at identifying the annual
progress of the CPA reform implementation the value of performance indicators is taken into
account too. The reports on the CPA reform progress will include the identified problems and
solutions to eliminate the identified drawbacks.
The annual monitoring reports „on progress of central public administration reform in the
Republic of Moldova” (quarter IV) will include, according to the monitoring and evaluation
11
methodology, the evaluation of results and achievements of the CPA reform. The procedure of
determining the achievements is described in section 5.
The monitoring methodology foresees the determination of the contents, outline and
frequency of reports. The outline of the Quarterly and Annual Monitoring Report „on progress of
central public administration reform in the Republic of Moldova” will be determined by PAR
Unit (see the Application Plan of Monitoring Methods and Instruments).
Ministries and other central administrative bodies, in charge of the implementation of the
actions of the reform, present quarterly to PAR Unit information about the reform
implementation within the authority, specifying the achievements and explaining the failure to
carry out certain actions planned according to the procedure determined by the PAR Unit.
The evaluation report on the implementation of the CPA reform as a whole or on the
achievement of some of the objectives of the strategy must meet the unified requirements set by
PAR Unit.
Different materials on CPA reform implementation, including informative notes, service
notes, appeals, letters, comments, memoirs, orders, directives, etc. will be provided at the request
of the stakeholders of the CPA reform or when necessary.
5. Evaluation Procedure of the CPA Reform Success
The success of the CPA Reform Strategy implementation is evaluated annually applying the
Numerical Rating consisting of four numerical rates: 0– unsuccessful, 1 – less then successful, 2
– successful, 3 – highly successful2. According to the present Methodology of Monitoring and
Evaluation of CPA Reform as a whole, the numerical rating must be applied, while the progress
is evaluated annually based on the evaluation of the following components:
A. General efficacy and degree of implementation of the Implementation Plan of the CPA
Reform Strategy (Action 2.1);
B. General performances of the CPA reform within the central public administration
authorities and of the CPA reform as a whole (Action 2.2);
C. Efficacy of the CPA reform implementation costs (Action 2.3).
The evaluation methodology of the success of implementation of the CPA Reform Strategy
and Strategy Implementation Plan is further described.
First of all, the General Efficacy and the degree of implementation of the Implementation
Plan of the CPA Reform Strategy are monitored per component and action of the strategy
reflected in the Strategy Implementation Plan in order to make sure that the actions stipulated in
the Implementation Plan are carried out qualitatively and in time (or are protracted) (Action 2.1).
The criteria and rates of monitoring the implementation of the actions stipulated in the
Implementation Plan are shown in table 2.
Table2. Efficiency Criteria and Implementation Rates of the Actions Stipulated in the
Implementation Plan
Efficiency Criteria
Rate Description
Rate Value
(Score)
Timely implementation of all the actions stipulated in the
Implementation Plan and reaching a 90% rate of presented documents.
Highly efficient
3
Timely implementation of at least 75% of the actions foreseen in the
Implementation Plan and reaching a rate of at least 50% of presented
Efficient
2
2
The Numerical Rating System has been determined according to the systems applied by UNDP, World Bank and
other international organizations.
12
documents
Timely implementation of at least 50% of the actions foreseen in the
Implementation Plan and reaching a rate of at least 25% of presented
documents.
Timely implementation of less than 50% of the actions foreseen in the
Implementation Plan and reaching a rate of less than 25% of presented
documents.
Less efficient
1
Inefficient
0
The general numerical rates of evaluation of General Efficacy and of the degree of
implementation of the CPA Reform Strategy as a whole are determined by summing the
numerical rates of the four components of the Strategy. The general numerical rates of evaluation
of Efficacy and of the degree of implementation of each component of the Strategy reflected in
the Implementation Plan are determined by summing the numerical implementation rates of all
the respective actions. The evaluation criteria and numerical rates of the General Efficacy and of
the degree of implementation of the Implementation Plan of the CPA Reform Strategy are
presented in table 3 per each component of the Strategy and as a whole.
Table 3. General Efficacy and Degree of Implementation of the Actions Stipulated in the
Implementation Plan
Criterion
Share
A. General Efficacy and Degree of Timely
Implementation
N/A
Component I
Central Public Administration Reorganization
%
Action 1
%
...
Component II
Decision Making Process
%
Action 1
%
...
Component III
Human Resources Management
%
Action 1
%
...
Component IV
Public Finances Management
%
Action 1
%
...
Component V
Participation and Communication
%
Rating
Rate Value
Highly relevant,
Relevant,
Partly relevant,
Irrelevant
Highly efficient,
Efficient,
Less efficient,
Inefficient
Highly efficient,
Efficient,
Less efficient,
Inefficient
3
2
1
0
3
2
1
0
3
2
1
0
Highly efficient,
Efficient,
Less efficient,
Inefficient
Highly efficient,
Efficient,
Less efficient,
Inefficient
3
2
1
0
3
2
1
0
Highly efficient,
Efficient,
Less efficient,
Inefficient
Highly efficient,
Efficient,
Less efficient,
Inefficient
3
2
1
0
3
2
1
0
Highly efficient,
Efficient,
Less efficient,
Inefficient
Highly efficient,
Efficient,
Less efficient,
Inefficient
3
2
1
0
3
2
1
0
Highly efficient,
Efficient,
Less efficient,
Inefficient
3
2
1
0
13
Action 1
%
Highly efficient,
Efficient,
Less efficient,
Inefficient
3
2
1
0
...
Component VI
Monitoring the Implementation Process
%
Action 1
%
...
%
Highly efficient,
Efficient,
Less efficient,
Inefficient
Highly efficient,
Efficient,
Less efficient,
Inefficient
Highly efficient,
Efficient,
Less efficient,
Inefficient
3
2
1
0
3
2
1
0
3
2
1
0
The evaluation of the General Efficacy and of the degree of CPA reform implementation will
help determine the deviations from the Implementation Plan and take corrective actions, when
necessary.
Secondly, the General Performances of the CPA reform as a whole are evaluated. The
criteria and numerical rates of conversion of performance indicators per each component and
objective of the CPA Reform Strategy into numerical rates per each component of the Strategy
and as a whole are presented in table 4.
Table4. Evaluation of the General Performances of the CPA Reform Implementation
Criterion
B. General Performances
Ministry of Economy and Trade
Ministry of Finances
Rating Description
Highly relevant,
Relevant,
Partly relevant,
Irrelevant
Highly relevant,
Relevant,
Partly relevant,
Irrelevant
Highly relevant,
Relevant,
Partly relevant,
Irrelevant
Rate value
3
2
1
0
3
2
1
0
3
2
1
0
...
(according to the list of authorities set according to art.24 of
Law no. 64-XII of May 31, 1990 on the Government)
The relevance of the CPA reform implementation will be determined after evaluating the
general performance of the CPA reform implementation. In case of partial relevance or nonrelevance of the CPA reform as a whole or of some components and objectives of the strategy the
reasons and delays will be identified and corrective actions and measures will be taken.
Thirdly, the costs of the CPA reform implementation are monitored to make sure that the
reform is implemented within the limits of the approved budget (monitoring indicator: degree of
execution of the budget calculated as the ratio between the factual and planned expenses).
Fourthly, the success of the CPA reform as a whole is determined (Action 2.4 from chart 1)
by calculating the general numerical rate as the weighted average of the numerical rates of the
components:
A. General Efficacy and degree of implementation of the Implementation Plan of the CPA
Reform Strategy (Action 2.1);
B. General performances of the CPA reform implementation within the central public
administration authorities and as a whole (Action 2.2);
C. Efficacy of CPA reform implementation costs (Action 2.3).
14
The criteria, shares and numerical rates by components mentioned above which apply to
evaluating the general numerical rate of the success of the CPA reform as a whole are shown in
table 5.
Table 5. Criteria, Share and Evaluation Rates of the Success of the CPA Reform as a whole
Criteria
Share
A. General Efficacy and degree of
implementation of the actions
stipulated in the Implementation Plan
B. General Performance of the CPA
reform implementation
45%
C. Overall CPA reform Budget cost
(from the state budget and Trust
Fund)
CPA REFORM GENERAL
EVALUATION
(Weighted average of A, B and C)
20%
35%
Rate Description
Highly efficient,
Efficient,
Less efficient,
Inefficient
Highly relevant,
Relevant,
Partly relevant,
Irrelevant
Highly efficient,
Efficient,
Less efficient,
Inefficient
Numerical Value of
Rates
3
2
1
0
3
2
1
0
3
2
1
0
Highly Successful (HS): The quarterly/annual overall weighted
average (OWA/A) is > 2.5 and none of the 3 criteria has a rate less
than 2; otherwise the rating would be downgraded by one level.
Successful (S): OWA/A is between 1.6 ≤ S ≤ 2.5and none of the 3
criteria has a rate less than 1; otherwise the rating would be
downgraded by one level.
Less then successful (LS): OWA/A is between 0.6 ≤P S ≤ 1.6
and number of criteria receiving a rate of less than 1 should not
exceed 2; otherwise the lowest rating would be downgraded by
one level.
Unsuccessful (U): OWA/A is < 0.6.
The evaluation of the CPA reform implementation by applying the numerical rating method
is done annually, for each year of the implementation period.
Conclusion. The evaluation of the success of the implementation of the CPA reform as a
whole, which includes the general efficacy and degree of CPA reform implementation, the
general performance of reform implementation and the efficiency of usage of financial resources
for CPA reform implementation helps determine the objectives and components which are
achieved and implemented successfully or less successfully. The success evaluation will give the
possibility to identify the objectives and components of the CPA Reform Strategy and the actions
of the Implementation Plan the achievement and implementation of which was less successful or
even a failure, to reveal problems which hamper the implementation of the CPA reform.
Respectively, measures will be taken to eliminate the identified problems and, if necessary,
corrective actions will be taken too.
15
6. Plan of Performance Monitoring and Evaluation of CPA Reform
Results
The plan of performance monitoring and evaluation of CPA reform results and success is shown
in table 2.
Table 2. Plan of Monitoring and Evaluation of CPA Reform in the Republic of Moldova
OBJECTIVE OF THE GOVERNMENT:
CENTRAL PUBLIC ADMINISTRATION REFORM
STRATEGIC OBJECTIVES:
Performance Indicators:
I.
Central Public Administration Reorganization
PI1, PI2
II.
Optimization of the Decision Making Process
PI3, PI4, PI5
III.
Improvement of the Human Resource Management
PI6, PI7, PI8, PI9
IV. Improvement of the Public Finance Management
V.
Participation and Communication
VI. Implementation of the Strategy Implementation Plan
VII. Monitoring PAR Unit Activity
PI10, PI11
PI12
PI13
PI14, PI15
EXPECTED RESULTS :
I.
Result Indicators:
The competences, functions and organizational structure of specialized CPA bodies will be directed towards and
focused more on the implementation of the priorities of the Government, eflected in the basic strategic documents.
The capacities of the Governmental structures responsible for the fulfillment of these tasks are to be strengthened.
R1
Most of the institutions (services, offices, agencies) which are currently under direct subordination of the
Government will be placed under the authority of ministries or will be reorganized taking into account the
applicable results of the functional review.
Overlapping of functions will be eliminated, ensuring at the same time the cooperation of a number of ministries in
the priority areas of Government.
Regulatory functions will be separated from the service provision functions.
A unique mechanism will be used to coordinate all the activities on European integration and to adjust the current
legal framework to the acquis communitaire.
II. The activity and resources of Government and ministries will be directed mostly to the development of the
necessary policies, monitoring and control, as well as to delegation of functions of sector policy implementation and
those dealing with the micro-management of other legal entities.
R2
An appropriate legal framework will be adopted and implemented, which will clearly define the CPAA typology,
main competences and mutual relations regarding monitoring, control and clear reporting arrangements.
III. The implementation of the new personnel policy will have a great impact on the whole civil service activity and will
result in:

building a professional staff of civil servants;

obtaining stability and loyalty of civil servants;

ensuring transparency in civil service in order to cooperate with the civil society and consolidate the state.
IV. Using financial resources according to the approved budget.
EXPECTED SUCCESS
R3
R4
Numerical Rates for Success Measuring:
Highly Successful:
2,5<HS≤3
Successful:
1,6≤S≤2,5
Less then successful:
0,6<LS<1,6
Unsuccessful:
0,6<U
16
The plan of monitoring and evaluation of the CPA reform in the Republic of Moldova has
been elaborated according to the CPA Reform Monitoring and Evaluation Concept, included in
this monitoring and evaluation methodology and includes:
 Monitoring achieved performances, which reveal the progress of the CPA reform
implementation applying performance indicators (PI), which measure the way the reform
achieves its objectives;
 Evaluation of the achieved results (R) of CPA reform implementation which defines the
objectives the Government and Central Public Administration wish to achieve;
 Evaluation of the success (S) of CPA reform implementation which defines the way of
measuring the success applying the numeric rating.
The evaluation indicators of results expected to be achieved while implementing the CPA
reform and the CPA reform performance monitoring indicators have been elaborated for each
CPA reform component. Further one can find the list of result and performance indicators per
each component of CPA Reform Strategy and the interactions between these two groups of
indicators.
I. Central Public Administration Reorganization
R1. Increased capacity of central public administration.
PI1. Number of CPAA (out of the total number), which have been reorganized at the level of
institution, department (section, service) and civil servant, according to the new requirements.
PI2. Number of drafts of normative acts approved by the Government at once (out of the total
number).
II. Optimization of the Decision Making Process
R.2. Increase of the planning and reporting system efficiency.
PI3. The share of drafts of normative acts approved by the Government, which have been drafted
by applying live consultancy mechanism.
PI4. Share of planning documents (concept, strategy, program, plan) of CPAA elaborated
according to the unified requirements.
PI5. Number of CPAA (out of the total number), which apply the manual and electronic record
keeping system of documents.
III. Improvement of the Human Resources Management
R.3. Efficient system of human resources management.
PI6. Share (in %) of civil servants employed through open competition.
PI7. Efficacy of knowledge transfer to civil servants at training programs.
PI8. Share of civil servants who have benefited of at least 5 training days (internal and external
training) and the number of CPAA which have improved the professional skills of civil
servants (in the fields: sector programs, application of informational technologies, studying
international languages, etc.), according to the Training Plan and Program.
PI9. Comparisons between public and private sectors as far as the remuneration concerns.
IV. Improvement of the Public Finances Management
R4. Efficiency increase of the sector programs.
PI10. Number of articles from the Law on State Budget which have been modified during the
year.
PI11. Number of sector programs (out of the total number), which involve annual reporting on
the achievement of the objectives and efficiency of the use of financial resources.
17
V. Participation and Communication
PI12. Progress in the development of the website and quality of reflecting activities within CPA
reform.
VI. Monitoring of the Implementation Process
PI13. Degree of implementation of the Strategy Implementation Plan.
VII: Monitoring PAR Unit Activity
PI14. Efficacy of the CPA reform implementation process coordination.
PI15. Degree of meeting the needs of PAR Unit in human resources and logistics (equipment,
soft, techniques).
II. PLAN OF APPLICATION OF CPA REFORM
EVALUATION AND MONITORING METHODS AND
INSTRUMENTS
The plan of application of CPA reform evaluation and monitoring methods and instruments
includes actions and methodological work procedures necessary for the implementation of the
monitoring and evaluation methods and instruments specified in the CPA Reform Evaluation and
Monitoring Methodology (see table 6).
18
Table 6. Plan of Application of CPA Reform Evaluation and Monitoring Methods and Instruments
No.
Monitoring and Evaluation
Methods and Instruments
1.
CPA Reform Strategy
Implementation Plan
2.
3.
Action plans of ministries
and other central public
authorities
Quarterly (and annual)
Information (reports) about
the implementation of the
CPA reform at territorial
level
Activities
Expected Results
Timeframe
1) Elaboration of the monitoring pattern of the
implementation of the Reform
Implementation Plan
1) Monitoring pattern of the
implementation of the Reform
Implementation Plan
1) August 14-18, 2006
1)
Elaboration of the unified requirements for
the action plans of CPAA (module for the
action plan and the document accompanying
it)
2) Meeting with the working groups in order
to clarify the unified requirements to the
action plans of the CPAA
3) Spreading „good practices” among
ministries and other CPA authorities.
4) Application of the benchmarking method:
identification of the significant CPAA
activities, comparative analysis with the
„good practices” of reform implementation
within CPAA
1) Unified requirements which will
apply to the Action Plans of
CPAA (module for the action
plan and the document
accompanying it)
2) 1 meeting with the working
groups from CPAA
1) September 18-29, 2006
4) Comparative analysis
(benchmarking) of the results
achieved by different CPAA in
the field of CPA reform
implementation
4) December 16-31, 2006
1) Formulation of the unified requirements
applied to the information provided by
CPAA
2) Elaboration of a Model of Information
Reception from CPAA
3) Testing the system of performance
indicators and result indicators within the
pilot CPAA
4) Organization of a meeting with the working
groups for the purpose of discussing the
application of the CPA reform performance
and result indicators
1) Unified requirements which will
apply to the information provided
by CPAA
2) Pattern of reception from CPAA
1) August 14-30, 2006
3)
„Good practices” in the field of
CPA reform implementation
3) Initial value of performance
indicators at the date of January
1, 2006
4) Meetings of CPAA working
groups regarding the application
of the performance and result
indicators. One meeting held at
each CPAA at their request
Person in
Charge
Consultant in
M&E
Consultant in
M&E-
2) October 09-13, 2006
3) December 1-31, 2006
Consultant in
communication
Consultant in
M&E
Consultant in
M&E
2) December 1-31, 2006
3) December, 1-15
4) September 18-29, 2006
19
No.
Activities
Expected Results
1) Formulation of the unified requirements
applied to the quarterly reports on CPA
reform progress
2) Drawing up the report on reform progress
for the third quarter of 2006.
3) Elaboration of the table of performance
indicators of reform implementation
4) Analysis of the documents presented by
CPAA, carried out by PAR Unit (office
monitoring) and drawing up the report on
monitoring for quarter III, 2006
5) Testing performance indicators of CPA
reform progress
6) Calculation of the initial value of
performance indicators of CPA reform as a
whole based on the data provided by CPAA
7) Testing the algorithm of evaluating the
implementation success of the CPA reform
as a whole
1) Unified requirements which will
apply to the quarterly reports on
CPA reform progress
2) Report on CPA reform progress for
quarter III, 2006
3) Table of performance indicators of
reform implementation
4) Monitoring report on the CPA
reform progress for quarter III, 2006
7) Data about the evaluation of the
implementation success of the CPA
reform as a whole
6) December 9-31, 2006
Annual report on CPA
reform monitoring (reveal
the running phase of the
CPA reform process)
1) Formulation of the unified requirements
which will apply to the annual report on
CPA reform monitoring
2) Holding meetings and organizing know-how
transfers among persons involved in the
CPA reform implementation
1) Unified requirements applied to the
annual report on CPA reform
monitoring
2) Meetings and know-how transfer
among persons involved in the CPA
reform implementation
1) August 14-25, 2006
6.
Report on the evaluation of
the CPA reform progress
1)
1) Unified requirements applied to
the annual evaluation reports
1) August 14-25, 2006
Consultant in
M&E
7.
Field monitoring visits at
ministries and other central
administrative authorities
1) August 21-30, 2006
Consultant in
M&E
4.
5.
Monitoring and Evaluation
Methods and Instruments
Quarterly reports on the
CPA reform progress
Formulation of the unified requirements
which will apply to the annual evaluation
reports
5) Final set of indicators of monitoring
the CPA reform progress
6) Initial value of performance
indicators of CPA reform as a whole
1) Formulation (using the office monitoring) of 1) Requirements applied to the
the requirements which will apply to the
document accompanying the visit
document accompanying the field
and the report on field monitoring
monitoring visit and the report on the field
visit
monitoring visit at CPAA
2) Elaboration of the documents accompanying 2) Documents accompanying visits,
the monitoring visits, carrying out field
field monitoring visits, reports on
Timeframe
1) August 14-25, 2006
Person in
Charge
Consultant in
M&E
2) October 9-20, 2006
3) December 1-31, 2006
4) Within 21 days from the
end of the reporting
period
5) September 25 – October
6, 2006
7) December 18-31, 2006
Consultant in
M&E
2) Periodically
2) Permanently
20
No.
Monitoring and Evaluation
Methods and Instruments
Activities
visits, drawing up reports on the CPA
reform field monitoring visits
8.
Research, questionnaires,
evaluation forms and other
ways of getting information.
Meetings of the
Consultative and
Supervisory Committee for
CPA reform
Expected Results
Timeframe
Person in
Charge
field monitoring of CPA reform
1) Elaboration of evaluation forms of
knowledge transfer to civil servants.
 Training programs
 Study visits
 CPA reform website
2) Implementation of the evaluation forms
3) Elaboration of the analytical material
regarding the methodical materials and
experience of other countries in the
monitoring and evaluation of the central
public administration reform.
1) Evaluation forms of knowledge
and information transfer to civil
servants
1) August 21-30, 2006
2) Getting feedback from the
beneficiaries of know-how
3) Analytical study regarding the
methodical materials and experience
of other countries in monitoring and
evaluation of the central public
administration reform
2) Permanently
Consultant in
M&E
3) November 1-30, 2006
9.
Annual meetings
1) Organizing and holding the annual meeting
1) Annual meetings
1) Periodically
10.
Methodological work
procedures of monitoring
and evaluation
1) Elaboration of the Work Procedure Guide
regarding the functioning of the working
groups within CPAA
2) Formulation of the requirements which will
apply to the Electronic Newsletter „News of
CPA Reform” and „CPA Reform
Messenger”
3) Formulation of proposals of improving the
Action Plan of the PAR Unit.
1) Guide to Work Procedures on the
functioning of the working groups
within CPAA.
2) Requirements applied to the
Newsletter and Messenger of the
CPA reform
3) Clear proposals of improving the
Action Plan of the PAR Unit
1) September 4-8, 2006
Consultant in
economy
Consultant in
M&E
2) August 28 – September
11, 2006
Consultant in
communication
3) August 14-18, 2006
Consultant in
M&E
21
ATTACHMENTS
Attachment 1. Indicators of Measuring CPA Reform Result
Objectives of the
Strategy
CPA Capacity and Impact Indicators
Indicators / Result
Data
Target Value at the End of
the Reform
Implementation Process
I. Central public
administration
reorganization
II. Optimization of the
decision making process
R1. Increased capacity of central
public administration
CPAA have achieved at least 80%
of the objectives
R.2. Increase in the planning and
reporting system efficiency
III. Improvement of the
human resources
management
R.3. Efficient system of human
resources management
IV. Improvement of the
public finances management
R4. Increase in the sector programs
efficiency
Timely presentation, according to
the unified standards, of the
planning documents and reports.
Civil servants have clearly
defined responsibilities and civil
servants’ performances are
evaluated yearly.
All the sector programs have
achieved at least 90% of the
objectives.
22
Attachment 2. Indicators of Measuring CPA Reform Performances in the Republic of Moldova
Objectives
1
I. Central
Public
Administration
Reorganization
II. Decision
Making Process
Indicators by
Components and
Objectives of CPA
Reform Strategy
Primary Frequency Primary Data Collecting and Secondary Data Calculation Method
Information
Sources
Initial
Value,
01.01.2006
Target Value at
the End of CPA
Reform
Implementation
Process
Factual Value
of the
Indicator
2
PI1. Number of
CPAA (out of the
total number) which
have been
reorganized at the
level of institution,
department (section,
service) and civil
servant, according to
the new
requirements.
3
CPAA
4
Annually
5
Primary data: Answer „Yes/No” at the level of institution,
department (sector, service) and civil servant?”.
Secondary data: PAR Unit calculates the weighted average (PI1)
in % of the „Yes” answers for all the CPAA to the question „Has
the CPAA been reorganized?”.
6
N/A
7
All the CPAA
will have been
reorganized by
January 1, 2008
8
CPAA:
„Yes” or „No”
answer
PAR Unit:
PI1;
PI2. Number of drafts
of normative acts
approved by
Government at once
(out of the total
number)
Government Annually
Office
Primary data: The total number of drafts of normative acts laid
before Government, including the number of normative acts
approved by the Government at once.
Secondary data: PAR Unit calculates the share of the normative
acts approved at once of the total number of drafts of normative
acts laid before Government:
PI2 (%) = N1/N*100.
Where: N1 –number of normative acts approved by Government
at once; N - total number of drafts of normative acts laid before
Government.
...
Approval of at
least 75% of
normative acts
laid before
Government (by
December 1,
2008).
PAR Unit:
PI2 (%);
PI3. Share (of the
total number) of the
drafts of normative
acts approved by
Government and
Government Annually
Office
Primary data: Total number of drafts of normative acts approved
by the Government, including the number of normative acts
drafted by using the live consultancy mechanism.
Secondary data: PAR Unit calculates the share of the normative
acts drafted by using the live consultancy mechanism in the total
...
Application of
the live
consultancy
mechanism to at
least 90% of all
PAR Unit:
PI3 (%);
23
Objectives
III. Human
Resources
Management
Indicators by
Components and
Objectives of CPA
Reform Strategy
Primary Frequency Primary Data Collecting and Secondary Data Calculation Method
Information
Sources
drafted by applying
the live consultancy
mechanism.
number of normative acts approved by Government:
PI3 (%) = N1/N*100.
Where: N1 number of normative acts drafted by using the live
consultancy mechanism and approved by the Government; N total number of drafts of normative acts approved by Government.
PI4. Share of
planning documents
(concept, strategy,
program, plan) of
CPAA elaborated
according to the
unified requirements
CPAA
PI5. Number of
CPAA (out of the
total number) which
use the manual and
electronic record
keeping system of
documents
APAP,
MID
PI6. Percentage of
civil servants
employed through
open competition
CPAA
Annually
Annually
Annually
Primary data: Number of all planning documents (concept,
strategy, program, plan) of CPAA, including those drafted
according to the unified requirements.
Secondary data: 1) Each CPAA calculates the share in % (PI4n)
of planning documents drafted according to the unified
requirements in the total number of planning documents as the
ratio of the number of documents elaborated according to the
unified requirements (NDRn) to the total number of planning
documents (TNDn):
PI4n=NDRn/TNDn*100.
2) PAR Unit calculates the weighted average (PI4) of the
documents drafted according to the unified requirements of all the
planning documents applying the formula:
PI4=(∑ NDRn)/( ∑TNDn)*100
Primary data: „Yes/No” answer to the question „Does the CPAA
use the manual and electronic record keeping system of
documents?”
Secondary data: PAR Unit calculates the weighted average (PI5)
in % of „Yes” answers for all the CPAA
Primary data: Number of civil servants (out of the total number)
employed through open competition.
Secondary data: 1) Each CPAA calculates the percentage of civil
servants employed through open competition:
PI6n (%) = NECn/TNSn*100,
Where: PI6n –percentage of civil servants employed through open
Initial
Value,
01.01.2006
Target Value at
the End of CPA
Reform
Implementation
Process
Factual Value
of the
Indicator
the elaborated
normative acts
(by December 1,
2008)
0
...
...
Starting with
January 1, 2008
all the CPAA
will elaborate
planning
documents
according to the
unified
requirements
CPAA:
NDRn;
TNDn;
PI4n;
Starting with
January 1, 2008
all the CPAA
will apply the
manual and
electronic system
of documents’
record keeping.
Starting with
January 1, 2007
civil servants
will be employed
through open
competition.
CPAA:
„Yes” or „No”
answer;
PAR Unit:
NDR;
TND;
PI4;
PAR Unit:
PI5 (%);
CPAA:
NECn;
TNSn;
PI6n (%);
PAR Unit:
24
Objectives
Indicators by
Components and
Objectives of CPA
Reform Strategy
PI7. Efficacy of
knowledge transfer to
civil servants at
training programs
PI8. Share of the civil
servants who have
benefited of at least 5
days of training
(internal or external)
Primary Frequency Primary Data Collecting and Secondary Data Calculation Method
Information
Sources
APAP,
CPAA,
Beneficiarie
s of training
courses
APAP,
CPAA
Last day
of the
training
course
competition in CPAA „n”, NECn –total number of civil servants
employed through open competition, TNSn –total number of
employees in CPAA with the order number „n”, n – order number
of CPAA.
2) PAR Unit calculates the total number of civil servants,
including those employed through open competition; percentage
of civil servants employed through open competition in the
reporting period:
PI6 (%) = ∑NECn/∑TNSn*100,
Where: PI6 –percentage of civil servants employed through open
competition in CPA; NEC –total number of civil servants
employed through open competition in CPAA, TNS – total
number of employees per CPAA.
Primary data: Collecting data from the beneficiaries of training by
applying the monitoring mechanism „Evaluation form of the civil
servants – beneficiaries of training services”.
Secondary data:1) Academy of Public Administration (APAP)
and each CPAA, when implementing internal training programs,
evaluate the efficacy of knowledge transfer to civil servants per
each person, sub-total per each training program (PI7n,i) and total,
applying the formula:
PI7n=(∑PI7n,i)/I,
Where „i” is the type of the training program, „I” is the total
number of training programs.
2) PAR Unit evaluates the efficacy of knowledge transfer to civil
servants, sub-total per each training program (PI7i) and total (PI7),
applying the formulas:
PI7i=(∑PI7n,i)/N,
PI7= (∑PI7i)/I.
Primary data: Number of civil servants (out of the total number)
having benefited from at least 5 days of training (internal and
external).
Secondary data: 1) Each CPAA calculates the share of civil
servants who have benefited from at least 5 days of training
Initial
Value,
01.01.2006
Target Value at
the End of CPA
Reform
Implementation
Process
Factual Value
of the
Indicator
NEC;
TNS;
PI6 (%).
N/A
...
High efficiency
of the training
programs
(Efficiency
criteria are
stipulated in the
„Evaluation form
of the civil
servants –
beneficiaries of
training
services”)
APAP:
Total number
of civil servant
per CPAA that
benefit from
training;
CPAA:
PI7n,i;
PI7n;
All the civil
servants will
have benefited
from at least 5
days of training
APAP:
NSIn Number of
civil servants
per CPAA
PAR Unit:
PI7i;
PI7;
25
Objectives
Indicators by
Components and
Objectives of CPA
Reform Strategy
Primary Frequency Primary Data Collecting and Secondary Data Calculation Method
Information
Sources
and number of CPAA
which have
developed the
professional skills of
civil servants (in the
fields: sector
programs, application
of informational
technologies,
studying international
languages, etc.,
implementation
according to the
Training Plan and
Program)
IV. Public
Finances
Management
Initial
Value,
01.01.2006
(internal and external) of the total number of civil servants:
PI8n (%)=NSIn/TNSn*100
Where PI8n –ordine share per CPAA with the number order „n”,
NSIn – number of civil cervants from CPAA of „n” type that have
benefited from at least 5 days of training (internal and external),
TNSn –total number of civil servants from „n”CPAA.
2) PAR Unit determines the number of CPAA (N’) which have
developed the skills of civil servants out of the total number N; it
is calculated as the sum of CPAA which have achieved the target
value of the performance indicators.
3) PAR Unit calculates the share of civil servants which have
benefited from at least 5 days of training (internal and external) of
the total number of civil servants:
PI8 (%)=∑NS/∑TNS*100
Where PI8 – weighted average, NS – number of civil servants
from all the CPAA that have benefited from at least 5 days of
training, TNS –total number of civil servants from all CPAA.
PI9. Comparisons
between the public
and private systems
as far as the
remuneration
concerns
PI10. Number of
articles from the Law
on State Budget
which have been
modified during the
year
National
Bureau of
Statistics
Annually
PAR Unit,
Official
Monitor
Annually
PI11. Number of
sector programs (out
MF, MET,
CPAA
Annually
Target Value at
the End of CPA
Reform
Implementation
Process
Factual Value
of the
Indicator
(internal and
external) by the
end of the CPA
reform
implementation
process
who have
benefited form
at least 5 days
of training
(internal and
external);
PI8n;
(31.12.2008).
PAR Unit:
PI8;
Primary data: The average monthly salary in the private and
public sectors for 2006, 2007 and 2008.
Secondary data: PAR Unit calculates the ratio (PI) of the average
monthly salary in the public sector (PuS) to the average monthly
salary in the private sector (PrS) for 2006, 2007 and 2008:
PI9=PuS/PrS
Primary data: Amendments of the Law on State Budget published
in the Official Monitor.
Secondary data: Calculation of the number of articles from the
Law on State Budget which have been modified during the year
and comparison with the number of amendments introduced the
previous year.
...
Primary data: Answer „Yes/No” to the question „Is the annual
reporting on the achievement of the objectives and efficiency of
...
...
NBS:
PuS;
PrS;
Decrease (or
elimination) in
number of the
amendments
introduced to the
Law on State
Budget during
the year.
All the sector
programs have
PAR Unit:
PI9;
PAR Unit:
PI10
MF and MET:
List of sector
26
Objectives
Indicators by
Components and
Objectives of CPA
Reform Strategy
Primary Frequency Primary Data Collecting and Secondary Data Calculation Method
Information
Sources
VI. Monitoring
the
Implementation
Process
01.01.2006
use of financial resources applied?” per each sector program.
Secondary data: PAR Unit determines weighted average (PI11)
as the arithmetical mean of CPAA per each sector program.
of the total number of
programs) which
involve annual
reporting on the
achievement of
objectives and on the
efficiency of the use
of financial
resources.
V. Participation
and
Communication
Initial
Value,
Target Value at
the End of CPA
Reform
Implementation
Process
Factual Value
of the
Indicator
had a functioning
mechanism for 3
years since the
start of the CPA
reform
programs;
CPAA:
„Yes” or „No”
answers;
PAR Unit:
PI11;
(till 31.12.2008).
PI12. Progress in the
development of the
website and quality
of reflecting activities
within CPA reform.
PI13. Degree of
implementation of
the Strategy
Implementation Plan.
PAR Unit
Quarterly
Primary data: Evaluation form placed on the website.
Secondary data: PAR Unit evaluates the efficiency (PI12) of the
website by applying the numerical rating (0-lack, 3-high quality)
CPAA
Quarterly
PI14. Efficacy of
coordination of the
CPA reform
PAR Unit
Primary data: State of things regarding the implementation of
actions stipulated in the Action Plan of the CPAA.
Secondary data: 1) Each CPAA calculates the degree of
implementation of the actions stipulated in its Action Plan (per
each action and on the whole):
PI13n (%) = NAIn/TNAn,
Where: PI13n - degree of implementation of the Action Plan of the
CPAA with the order number „n”, NAIn –number of actions
implemented in „n” CPAA, TNAn - total number of actions
stipulated in the action plan of „n” CPAA, n – CPAA order
number.
2) PAR Unit calculates the general degree of implementation of
the Implementation Plan (PI13):
PI13(%) = (∑PI13n)/N,
Where: PI13 - degree of implementation of the Implementation
plan in %. N – number of CPAA
Primary data: „Yes/No” answer to the question „Have at least
90% of the actions stipulated in the Action Plan of the PAR Unit
been timely and qualitatively implemented?”
Quarterly
N/A
High quality (3
points) achieved
during one year
since the launch
of the website.
All CPAA have
achieved
annually 80% of
the objectives.
PAR Unit:
PI12;
Timely and
quality
implementation
PAR Unit:
PI14.
CPAA:
PI13n (%);
PAR Unit:
PI13(%);
27
Objectives
Indicators by
Components and
Objectives of CPA
Reform Strategy
Primary Frequency Primary Data Collecting and Secondary Data Calculation Method
Information
Sources
Initial
Value,
01.01.2006
implementation
process.
PI15. Degree of
meeting the needs of
the PAR Unit in
human resources and
logistics (equipment,
soft, techniques)
PAR Unit
Annually
Primary data: Evaluation form of the needs of PAR Unit, taking
into account the experience of other countries and of PAR Unit
consultants in the organization of the model work place in the
Government Office.
Secondary data: Average degree of meeting the needs of PAR
Unit (PI15) is calculated as the arithmetical mean of the degrees of
meeting all the needs (numerical rates from 0 to 3)(PI15j):
PI15=(∑PI15j)/J,
Where: j – type of need (professional skills of the PAR Unit
consultants; work technologies and instruments; equipment), J –
total number of groups of needs (at least three). The Radar
Diagram will be applied to present the results.
...
Target Value at
the End of CPA
Reform
Implementation
Process
of at least 90%
of the actions
stipulated in the
Action Plan of
the PAR Unit.
Meeting
integrally the
needs (maximal
numerical rate,
3)
Factual Value
of the
Indicator
PAR Unit:
PI15;
Abbreviations:
CPAA – Central Public Administration Authorities.
APAP –Academy of Public Administration under the President of the Republic of Moldova
MID – Ministry of Informational Development.
N/A – Not Applicable
28
Attachment 3. Central Public Administration Reform Evaluation and Monitoring Scheme
Monitoring techniques / Reports
Quarter
Strategy Implementation Plan
CPAA Action Plans
Evaluation and Monitoring Methodology
Quarterly reports on CPA reform progress
Annual Report on CPA reform monitoring (quarter IV)
Final Report3
Quarterly Information Provided by CPAA
Annual Evaluation Report
Final Evaluation Report
Post – evaluation, 3 years after the finalization of the
technical assistance
Financial Reports (Trust Fund)
Audit 5 (Trust Fund)
Main Events:
Field Monitoring Visits
Workshops
Conferences
Round Tables
Surveys
Studies
Personnel Training Programs
Study visits
Annual Meeting
Executor
1
PAR Unit
CPAA
PAR Unit
PAR Unit
PAR Unit
PAR Unit
CPAA
PAR Unit
PAR Unit
Government
OAS4
OAS
PAR Unit
4
*
*
*
*
*
*
1
*
*
*
2007
2 3
*
*
*
*
*
*
4
*
*
1
*
*
*
2008
2 3
*
4
+ 3 years
1 2 3
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
PAR Unit
3
At least one month before the end of the CPA reform implementation.
4
OAS –Office of Administrative Support for assisting the Government in strategy implementation.
Audit is carried out once a year.
5
2006
2 3
*
*
*
*
*
*
*
*
*
*
*
*
*
*
29
Attachment 4. Schedule of main events for 2006
Events6
Field monitoring visit– permanently
1. Ministry of Economy and Trade
2.
...
30.
Office Monitoring:
Documents accompanying field visits
Annual and quarterly monitoring
Receiving quarterly information from CPAA
Annual evaluation: CPA reform evaluation
study
Analytical study of the international
experience in CPA reform monitoring and
evaluation
Workshops
Study visits
Latvia
Latvia
Training programs:
...
Meetings of Governmental Commission
Meetings of the Consultative Commission
6
Person in charge
Consultant in M&E
Jan Feb
Mar
Apr
May June July
Aug
-
Sept
-
Consultant in M&E
Consultant in M&E
Consultant in M&E
Procurement of
consultancy services
Consultant in M&E
Oct
-
Nov
-
Dec
-
*
*
*
*
*
*
Consultant in
communication
Consultant in human
resources
*
*
*
Consultant in human
resources
Consultant in economy
Consultant in economy
*
*
*
*
*
The events are identified in the Implementation Plan of the CPA Reform Strategy.
30