Download 1.3. Impact and results indicators

Pillar 5 Monitoring and Supervision Functions of the Romanian Public Procurement
System to enhance competition in the public procurement market
This Pillar contains a main text (Strategy), a table with the specific actions (Action plan) and
annexes. The annexes are complementing documents. They must be read in the light of the
Strategy and Action plan and cannot amend or contradict the actions as set out in the
Strategy and Action plan. Those texts always prevail over the annexes.
1.1 State of Play
The major objective of any public procurement system within the EU Member States is to
deliver “value for money” in an efficient manner for the use of public funds. Performance
measurement at system level and use of adequate tools and mechanisms provide the basis
for assessing whether the procurement system ultimately delivers in accordance with the
main objectives set.
Public procurement is the main instrument to satisfy the needs of the contracting authorities
financed from Romanian budget: in 2013, 13.3 % of Romania's GDP1 is advertised via
www.e-licitatie.ro2.
Although the Romanian public procurement system is theoretically guided by principles such
as transparency, competitiveness and economic efficiency for the benefit of society as a
whole, the current organisation of the monitoring and supervision of the system needs
substantial improvement and development.
The following paragraphs reflect the main deficiencies impeding the capacity of the PP
system to efficiently monitor the ability of the public procurement framework to effectively
regulate the economic activity and market reply:
i.
1
The current monitoring function is limited to providing sets of statistics such as the
number of contract notices, the types of procedures or contracting authorities, which do
not allow for a consistent description of the PP market, but rather express only one
dimension of the market. The key stakeholders involved are narrowly focusing on their
own organisation and own activity-based indicators. Although some indicators are
EUR 142 245 million – calculated using the exchange rate (1 EUR - 4,419 RON) – source http://www.insse.ro/cms/ - published GDP
for 2013 RON 628 581.3 million, current prices using expenditures approach
2
EUR 18 945 million - calculated using the exchange rate (1 EUR - 4,419 RON), and RON 83 716 million, as published by ANRMAP
on the webpage: http://www.anrmap.ro/sites/default/files/documente/documente-2546.pdf
collectable, from SEAP publications, they are not proactively processed. Statistics are
reported, mostly on the type of procedures used, without being able to support policy
development purposes. Moreover, the maturity of the current monitoring practices is low
– focused on activity and not on performance.
In consequence, monitoring is underdeveloped; neither effective economic monitoring
(market knowledge), nor effective monitoring of irregular behaviour or of PP policy
objectives is ensured under the current system.
ii. Implementation of adequate public procurement key performance indicators and
baselines at national level is still a challenge for Romania. KPIs relevant for the
Romanian public procurement system are either not yet well-defined, or
broadly/commonly accepted by the key stakeholders, or they encompass different
interpretations and viewpoints.
iii. Currently, the SEAP system does not include data fields to capture all key
information that can be used by the stakeholders of the Romanian public
procurement system to further correlate and analyse the strength and quality of the
competition. For example, key statistics and indicators specific for the activity of the
non-national bidders/tenderers are not possible to be produced with accuracy based
on SEAP data. This aspect is an indication of the overall credibility and attractiveness
of the Romanian public procurement system towards foreign economic operators and
investors.
Due to lack of integration of information along the procurement process, it is currently very
difficult to assess the actual influence of various factors on the degree of competition and to
monitor the end to end procurement process. This stems also from a deficient monitoring
function at the public procurement system level.
Indeed, according to the Single Market Scoreboard3 the overall performance of Romania
in PP is below average, scoring unsatisfactory for 2 out of the 3 dimensions (bidder
participation, accessibility, efficiency of the procedure).
3
http://ec.europa.eu/internal_market/scoreboard/performance_per_policy_area/public_procurement/index_en.htm
Chart no: Bidder participation in Romania, compared to other EU member states, based on
notices published in the Tenders Electronic Daily (TED) for the period 2011-2013
Currently, in terms of market reply, according to the open data exported from SEAP4, for
2014 (for all the procurement procedures published in SEAP, under or above the thresholds
for publication in the Official Journal), the average number of offers received by procedure is
7, 54, as presented below:
i.
Average number of offers received by procedure
Type of procedure
Competitive Dialogue
Request for quotation (open
procedure under EU thresholds)
Open procedure
Restricted procedure
Restricted accelerated procedure
Negotiation
Accelerated negotiated
Negotiation without publication
Grand Total
ii.
Average number of
received offers
3.00
Number of
procedures
8
8.05
8.57
2.59
2.45
3.90
1.01
4.48
7.54
8861
5049
35
6
418
71
3880
18328
Average number of offers received by type of contract
Contract
Type
Supply
Works
Services
Grand
Total
Average number of
received offers
8.45
4.88
3.85
Number of procedures
9685
3894
4749
7.54
18328
Moreover, due to the high level of utilisation of lowest price criteria (for 97% of the
procedures in 2014) and as reflected by substantial differences between estimated price and
contractual price, the current Romanian market conditions determine economic operators to
compete strongly on prices with visible negative effects for ensuring sustainable investments
and value for money.
Apart from the deficiencies at the level of the monitoring function, the supervision function
in the Romanian PP system is limited to ex-post controls of procedures further to the
specific notices/intimations sent by economic operators.
4
http://data.gov.ro/dataset/achizitii-publice-2007-2014-contracte
1.2 Way Forward
1.2.1 Actions: scope and expected impact
1.2.1.1 Re-designing and re-dimensioning the monitoring function
The monitoring function needs in-depth reinforcing as it represents a key instrument
providing knowledge on the market, with a view to enable the Romanian authorities to take
adequate measures to increase transparency, competitiveness and integrity of the public
procurement system through:
i. Policy and legislative initiatives
ii. Tailored guidance to contracting authority
iii. Corrective actions undertaken by responsible institutions.
Therefore, a re-dimensioning and a re-design of the function are necessary including all
market actors, proper tools and sound performance indicators, aimes especially at gaining a
reliable picture of the market reply on the following aspects:
i.
Market size and characteristics: number of procedures, types of procedures, types of
contracting authorities, procedures value types; length of procedures; final price of
the contract;
ii. Intensity of competition: number of offers received and rejected per type of
procedures/contracts/sectors, etc.;
iii. Monitoring economic activities: volume of procurement as compared to the GDP, level
of prices, access of SMEs, penetration of foreign bidders;
iv. Efficiency of public procurement: average period for planning, preparation, awarding,
implementation, final value of contracts etc.;
v. Identifying potential suspicious/irregular practices: high selection or exclusion rates
for the same economic operator by the same contracting authority, oligopoly
tendency, single offer, red flag companies.
In order to achieve a functional and efficient monitoring system, the following measures will
be taken:
i.
Set up of relevant monitoring indicators to be automatically extracted from the
national electronic system;
ii. Set up a dedicated department within the National Agency for Public Procurement,
provided with adequate resources;
iii. Ensure procedures to connect the monitoring function with the policy making and with
the supervision function.
The monitoring function should also incorporate the following levels:
i.
ii.
Romanian Public Procurement – At this level, indicators should monitor the
effectiveness of the public procurement procedures carried out in Romania, together
with the procurement practices (procedures, strategies followed by contracting
authorities). Monitoring should allow policy makers to assess the performance of the
system as a whole in relation to the objectives and goals established via the policy
making function;
Contracting Authority - performance of the contracting authority’s operations
regarding their ability to efficiently managed procurement procedures and reach the
assigned objectives;
1.2.1.2 Establishing the indicators to be considered within the monitoring function
Meaningful indicators and baselines can contribute to a systematic and comprehensive
practical approach in assessing the overall performance of the Romanian public procurement
system.
Therefore, on the basis of the analysis of the indicators already available in SEAP, the
following set of indicators shall be developed even where this will require the adjustment of
the interface with the CA. All these indicators should be aggregated at national level, as well
as at regional level, so as to provide a detailed picture of the overall performance of the
Romanian public procurement system:
A. Market size and characteristics
5
Description
Generic KPI indicator
Actual
situation
Future
situation5
Number of procedures
Number
possible
possible
Type of procedures
Number and % of
all procedures
possible
possible
Type of contracting authorities
Number and % of
all contracting
authorities
possible
possible
Value of procurement advertised via each
and all type of procedures
Value and
percentage
possible
possible
Described with reference available as of June 2015, future situation describes the situation once SICAP will be
implemented and improved.
Value of procurement contracted via each
and all type of procedures
Value and
percentage
possible
possible
Weight of the use of each award criterion
in a procedure
% of each award
criteria used on
each type of
procedures
possible
possible
Weight of the use of each award criterion
in a procedure - per type of sector –
assimilated with group of CPV code
% of each award
criteria used in
each sector
possible
possible
Use of lowest price ./. most economically
advantageous tender
Number and % of
all procedures
possible
possible
Most commonly used quality criteria,
possibly groups to be defined
Number and % of
all MEAT
procedures
possible
possible
B. Intensity of competition
Description
Generic KPI indicator
Actual
situation
Future
situation6
Average number of offers received per each
type of procedure
Number
Possible
possible
Average number of offers received per
category of procurement (specific types of
works, services and supplies, based on the
main CPV group)
Number
Possible
possible
Rejection rate of bidders by types of contracts
(works, supply, services)
%
Possible
possible
Rejection rate of bidders by type of procedure
%
Rejection rate of bidders per type of sector –
assimilated with group of CPV code
%
Possible
possible
6
Described with reference available as of June 2015
C. Monitoring economic activities
Description
Generic KPI indicator
Actual
situation
Future
situation7
Division of contracts into lots
% of procedures
with lots out of
all total
procedures
%
N/A
possible
N/A
N/A
Openness (participation of tenderers from the
outside EU) per type of procedure and
contracts
% of contractors
from EU market
(other than
Romania)
% of contractors
from EU market
(other than
Romania)
% of contractors
from outside EU
market
Requires
adjustment
of interface
with the CA
Requires
adjustment
of interface
with the CA
possible
Openness (award of contracts to tenderers
from the Rest of the world) per type of
procedure and contracts
% of contractors
from outside EU
market
Participation of SMEs and award of contracts
to SMEs
Openness (participation of tenderers from
other Member States) per type of procedure
and contracts
Openness ( award of contracts to tenderers
from other MS) per type of procedure and
contracts
N/A
Requires
adjustment
of interface
with the CA
possible
D. Efficiency of public procurement (including contract implementation)
Description
Generic KPI indicator
Time span between the publication of the Number of days
procurement opportunity and the signing of a
contract for each type of procedure (including
also other distributions than average)
Duration of the review procedure - measured Number of days
by the number of days that elapse between
the receipt of an appeal and the decision
(judgment) adopted by the review body
7
8
Described with reference available as of June 2015
Described with reference available as of June 2015
Actual
situation
Future
situation8
possible
possible
N/A
Requires
adjustment of
interface with
the CA
Outcome of review procedure (how many
procedures are annulled, suspended… etc)
Number and %
N/A
Requires
adjustment of
interface with
the CA
Requires
adjustment of
interface with
the CA
Requires
adjustment of
interface with
the CA
Number of internal administrative appeals and
their outcome (how many appeals result in a
change of position of the administration)
Number and %
N/A
Contract duration: planned versus achieved
%
N/A
Level of use of electronic means for all phases
of the procurement cycle from publication of
contract notices to the award of contracts.
% of total
procedures and
total value
Contract value versus actual value
%
N/A
Description
Generic KPI indicator
Actual
Future
situation situation9
Conflict of interests identified for each type of
procedure
Number
N/A
Conflict of interests identified for each sector - Number
assimilated with group of CPV code
N/A
Contracts awarded by the same contracting
authority to the same economic operator by
type of procedure
possible
Requires
adjustment of
interface with
the CA
E. Irregular/problematic behaviour
9
Described with reference available as of June 2015
Number
Requires
adjustment of
interface with
the CA
Requires
adjustment of
interface with
the CA
possible
Contracts awarded by the same contracting
authority to the same economic operator by
sector - assimilated with group of CPV code
Number
N/A
possible
Contracts awarded by contracting authorities
based on single offer
Number
N/A
possible
Whenever applicable, the above indicators will be exploited according to statistical
standards, considering the average performance, as well as standard deviations, with a view
to identifying possible shortcomings.
1.2.1.3 Connecting the monitoring function with the policy making function and with
the supervision function
An important role of the monitoring function will be also to provide data necessary for the
Policy Department within ANAP to check policy compliance of the Romanian public
procurement system. Currently, SEAP allows monitoring of Public procurement as part of
GDP and of e-procurement as part of total procurement over a certain period.
In addition, indicators will be extended in the new SICAP system to allow monitoring the
share of green procurement, energy efficiency, innovation and sustainable procurement, in
compliance with the requirements of the EU Directives.
The data generated by SEAP/future SICAP system shall be collected and analysed
periodically by the specialised unit within ANAP. Quarterly reports will be elaborated and
submitted to the Policy Unit and to the Supervision Unit within ANAP in order to allow, on
case by case basis, to undertake the necessary measures. As far as needed, specialised
reports, focusing on specific concerns (such as an assessment of the market concerning
public procurements in some regions / sectors) may also be issued:
i. Policy and legislative initiatives
ii. Tailored guidance to contracting authority
iii. Corrective actions undertaken by responsible institutions.
Therefore the re-design of the monitoring function should lead to an increase of
responsiveness of governmental stakeholders to market indicators, allowing a qualitative
process of improving the transparency, efficiency, and openness of the system.
In addition, Annual Reports will be issued by ANAP on the functioning of the Romanian
public procurement system and will be published on ANAP website in order to inform all
partners on the main data, analysis, problems and follow-up actions.
To increase transparency, all these reports (quarterly and annual) and all the indicators
identified above shall be made available to the public, allowing monitoring from civil society
and companies. In addition, peer review will not be limited to the analysis of reports
published by ANAP. The data collected on SICAP will be available to anybody for analysis.
Moreover, all awarded contracts (without annexes) and all substantial contract modifications
shall be published in SICAP to allow peer review and monitoring of civil society. Only a
limited number of contracts may not be made public (national security and defence, breach
of Intellectual property rights etc.)
1.2.1.4 Establishing the supervision function
According to the legislation in force, the supervision function is currently developed at the
process level, referring to ex-post checks performed by ANRMAP and triggered by
notifications of potential irregularities in the procedure. The present function shall be also
maintained in the future institutional set up, in order to ensure quick reaction to potential
cases of irregular behaviour.
In addition, the supervision function will be developed as a systematic observation of the
public procurement system conducted in a coherent way in order to assess how the system
functions and develops over time and to establish whether the targets have been achieved.
Consequently, procedures shall be in place to ensure that supervision will rely on the
monitoring function and on the ex-ante and ex-post control activities revealing in real time
systemic problems or irregularities.
Also inter institutional mechanisms will be in place to ensure that when systemic dysfunctions
are revealed using various red-flags, ANAP will define the nature of the problem and refer
the issues to the competent authorities for corrective actions.
The future specialised Unit within ANAP shall check the functioning of the Romanian public
procurement at two levels: 1) overall national level 2) indicators enabling to spot
inefficiencies and irregularities at the level of public buyers/contracting authorities.
ANAP shall identify problematic contracting authorities and make information available
through the annual report. Moreover it shall take corrective measures.
The corrective actions will address a wide range of systemic problem and will imply:
i.
Cooperation with the Competition Council for identified competition failures, on the
basis of the existing Action Plan to fight against collusive practices;
ii. Submission to the audit and control bodies of data on sectors / type of contracting
authorities, which have been identified as more risky (e.g. lower competition, higher
rates of excluded/non-selected bidders, number of contract modifications). These
inputs will be processed to be reflected in the elaboration of the annual risk
assessment and the corresponding audit and control strategies for the institutions
concerned. They will then also be taken into account for updating the sample
methodology underlying control activities.
iii. Submission to the Policy Unit within ANAP a regular analysis of main shortcomings
that can be solved through policy initiatives or additional guidance to stakeholders,
etc.
A dedicated Supervision Unit will be set up within ANAP in order to ensure the adequate
administrative capacity for the processing of the data and for initiating measures meant to
correct the deficiencies in the public procurement system and to ensure its transparency and
efficiency.
1.3. Impact and results indicators
The monitoring mechanism described under the chapter “Risks, governance and
performance monitoring” is directly applicable for the monitoring of indicators and the
expected results described under this headline.
With regard to Pillar 5, the following actions will be taken, which will be assessed against the
below indicators of impact:
Summary of proposed actions, lead and contributing institutions, completion timescales and expected impacts
Identified
issue/trigger for
the action
Fragmented
monitoring
function
Proposed Action
Re-dimensioning and
re-design of the
monitoring function,
through
-
Lead
ANAP
Contributors
AADR,
Competition
Council, ANI
KPI
No indicators on
the market
structure, market
dynamics or
public
procurement
policy
New system for
monitoring
designed and
implemented
All Indicators described
in the pillar are in place
as part of the
monitoring function
Expected impacts
Increased degree of
market knowledge,
decision based on
figures
September 2015
Design December
2015
-
Source of
Information
Monitoring
function fully
operational
Dedicated Unit in
ANAP has
sufficient staff
and they are
sufficiently
skilled.
Set up of a
dedicated unit
within ANAP on
monitoring(with
x persons)
Deadline
ANAP
organigram
ANAP web page
IT infrastructure
June 2016
ANAP
AADR, InterMinisterial
committee, All
stakeholders
All indicators
described in the
strategy are put
in place and are
measured.
July 2016
Reports issued
by ANAP
Up to date information
on policies implemented
using PP as a tool.
12
Not appropriate
connection
between
monitoring and
supervision
function
Clear and documented
connection monitoringpolicy and monitoringsupervision function
ANAP
AADR,
Competition
Council, ANI,
Court of
Account, other
regulatory,
legal bodies
Information
provided to all
relevant
stakeholders in
charge with
supervision
(direct access in
SICAP )
July 2016
Reports issued
by ANAP
Increased speed in
reaction to the market
signals or market status
Lack of
transparency and
access to Public
procurement
information
Quarterly and Annual
reports including the
indicators described in
this pillar will be
published for public
access
ANAP
AADR, InterMinisterial
committee, All
stakeholders
quarterly and
July 2016
annual
publication of
complete reports
(including all
indicators), made
accessible to the
public
Reports issued
by ANAP
Increased transparency
of public procurement
Publication of all
awarded contracts in
SEAP/SICAP for
access to the public
AADR
ANAP
All awarded
contracts are
published in
SEAP/SICAP
Information
avalable on
SEAP/SICAP
Increased transparency
of public procurement
March 2016
13
Underdeveloped
supervision
function
Set up of a dedicated
Unit for Supervision
ANAP
Procedures to secure
follow-up on basis of
information provided by
the monitoring
function.
Implementation of
corrective actions.
ANI,
Competition
Council, audit
and control
bodies
Dedicated Unit in
ANAP has
sufficient staff
and they are
sufficiently
skilled.
September 2015
July 2016
ANAP
organigram
Compliant public
procurement system
Report issued by
ANAP
Number of
corrective
actions initiated
Number of
problematic CAs
identified
Publication of contract
modifications leading to
price increase above
5%
AADR
ANAP
All awarded
contracts are
published in
SEAP/SICAP
March 2016
Information
available on
SEAP/SICAP
Strengthened external
control defining more
accurate sample of
verified contract
modifications
Publication after
contract termination, of
all contract
modifications which
occurred
AADR
ANAP
All awarded
contracts are
published in
SEAP/SICAP
March 2016
Information
available on
SEAP/SICAP
Increased transparency
of public procurement
14