Download PUBLIC SECTOR SUSTAINABILITY IN

PERFORMANCE MEASUREMENT IN THE PUBLIC SECTOR AT NATIONAL
AND LOCAL LEVELS – SOME APPLICATIONS TO SLOVENIA
Aleksander Aristovnik
Faculty of Administration, University of Ljubljana
Abstract
The main purpose of the paper is to review and to discuss different conceptual and methodological issues related to
the performance measurement in public sector at national and local levels. In particular, a composition of Public
Sector Performance (PSP) and Public Sector Efficiency (PSE) indicators as well as Free Disposal Hull (FDH) and
Data Envelopment Analysis (DEA) techniques are presented and then applied to selected emerging economies in
Eastern Europe with a special focus on Slovenia and with some methodological suggestions for measuring
(in)efficiency at local levels. The paper employs indicators relating to efficiency/effectiveness at the national as well
as at the local levels in a number of major policy areas: education, health care and infrastructure. The results show
that efficiency and effectiveness in Slovenia differs significantly at national (in comparisons to other selected
countries). However, in general, a relatively average performance in most countries is accomplished with too many
inputs so that efficiency is quite low. In other words, the selected governments (including the Slovenian government)
could use less resources to attain the same outcomes if they were fully efficient. In addition, the paper points out
some deficiencies relating to the employed techniques and definitions. Indeed, the applications of presented
techniques are hampered by lack of suitable data to apply those techniques and the precise definition of inputs,
outputs and outcomes which may significantly influence the empirical results.
1.
Introduction
Measuring performance has been increasingly important in public sector recently. In particular, at the time of
increased pressures ob public expenditures, stemming from demographic trends and globalization, the improvement
of the efficiency and effectiveness of public spending features high on the political agenda. However, for many
reasons, both political and technical, performance measurements have become an integral part of relatively few
governments' management or decision-making systems yet. The threat of privatization and spending cutbacks, made
without due consideration for impact of these changes in the future, has certainly helped increase the interest. In
addition, several other factors led to the recent focus on performance measurement such as the pervasive
dissatisfaction with government employees’ unresponsiveness to the public, the dynamics of Wagner’s Law, which
systematically increases the relative size of government, and hence puts pressure on the public finances, and the
implementation of New Public Management paradigm. But the introduction of performance indicators into public
management has been also carrying both a potential for greater effectiveness and substantial risk. It is thus necessary
to unbundle the concept of performance, and review the country- and sector-specific conditions that make for public
sector reform success or failure. The key determinant of success or failure is whether the changes were realistic,
introduced gradually, and consistent with both the methodological complexity of the topic and the specific country
realities (especially administrative capacity and the governance regime).
The purpose of the paper is to discuss and review some previous researches on the performance and efficiency of
public sector as well as different conceptual and methodological issues related to the performance measurement in
public sector. In particular, a definition and measurements of efficiency and effectiveness will be discussed.
Additionally, a composition of Public Sector Performance (PSP) and Public Sector Efficiency (PSE) indicators as
well as Data Envelopment Analysis (DEA) is presented and then applied to Slovenia and selected EU countries.
However, the focus of the paper is not on how to cut public expenditures, but rather more on investigating potential
reserves to increase the value for money of public spending, i.e. how to make the most of limited public resources.1
1
Note, however, that it is not only public expenditure but also tax regulatory policies that affect the efficiency of the public sector. While
expenditure is a relatively good proxy of the tax burden, we ignore the composition of tax revenue and other characteristics of tax system.
1
The paper is organized as follows. In the next section we present a brief literature review of measuring public
spending efficiency. Section 3 shows a theoretical background and empirical results of public sector performance
(PSP) and efficiency (PSE) indicators as well as Free Disposal Hull (FDH) and Data Envelopment Analysis (DEA)
analyses. This section also presents empirical applications of the presented techniques which are applied to Slovenia
and selected countries. Section 4 points out some methodological approaches for measuring public sector
performance and efficiency at local levels. Final section provides main concluding remarks.
2.
A Brief Literature Review of Public Spending Efficiency
The debate of the role of the public sector has shifted in recent years towards empirical assessments of the efficiency
and usefulness of public sector activities. A growing academic literature has been investigating the stabilisation,
allocation and distribution effects of public expenditure (see e.g. Tanzi and Schuknecht (2000); Afonso et al. (2005,
2006, 2008), Sánchez and Bermejo (2007), Mandl et al. (2008), etc.) as well as the role of rules and institutions, and
the scope for privatising public sector activities (see e.g. Rodrik (2000), Strauch and von Hagen (2000), Persson and
Tabellini (2001), Drake and Simper (2001), Gwartney (2002)). Most studies conclude that public spending could be
much smaller and more efficient than today. However, for this to happen, governments should adopt better
institutions and should transfer many non-core activities to the private sector.
Previous studies on the performance and efficiency of the public sector that applied non-parametric methods find
significant divergence of efficiency across countries. Studies include notably Fakin and Crombrugghe (1997) for the
public sector, Gupta and Verhoeven (2001) for education and health in Africa, Clements (2002) for education in
Europe, St. Aubyn (2003) for education spending in the OECD, Afonso et al. (2005, 2006) for public sector
performance expenditure in the OECD and in emerging markets, Afonso and St. Aubyn (2005, 2006a, 2006b) for
efficiency in providing health and education in OECD countries. De Borger and Kerstens (1996), and Afonso and
Fernandes (2006) find evidence of spending inefficiencies for the local government sector. Additionally, Afonso et
al. (2008) assess the efficiency of public spending in redistributing income. Most studies apply the Data
Envelopment Analysis (DEA) method while Afonso and St. Aubyn (2006a) undertook a two-step DEA/Tobit
analysis, in the context of a cross-country analysis of secondary education efficiency.
Composite indicators are very often by-products of efficiency measurements since they are constructed to serve as
input or output indicator.2 Afonso et al. (2005) computed composite indicators of public sector performance and
public sector efficiency for 23 industrialised (OECD) countries. Indicators suggest notable but not extremely large
differences in the public sector performance across countries (with a few exceptions). Countries with the highest
values for sub-indicators include Switzerland (administration and infrastructure), Japan (education), Iceland (health),
Austria (distribution), Norway (economic stability) and Luxembourg (economic performance). Countries such as
Luxembourg, Japan, Norway, Austria and the Netherlands report high total performance indicators. Looking at
country groups, small governments (industrialised countries with public spending below 40% of GDP in 2000) on
balance report better economic performance than big governments (public spending above 50% of GDP) or medium
sized governments (spending between 40 and 50% of GDP). Big governments feature more even income distribution
whereas small ones perform better especially in the administrative, stability and economic performance domains.
These results are consistent with those found in Tanzi and Schuknecht (2000). When comparing the main economic
players of today, it is noteworthy that the US and particularly Japan report above-average performance in most subindices and for the total PSP measure. By contrast, the EU (weighted average) performs below average. Additonally,
efficiency indicators are computed weighing performance by the amount of relevant public expenditure. They found
significant differences in public sector efficiency across countries. Netherlands, Australia, Finland and Luxembourg
show the best values for overall efficiency. Looking at country groups, small governments post the highest efficiency
amongst industrialised countries. Differences are considerable as small governments on average post a 40% higher
scores than big ones.
2
The problem with such composite indicators is twofold. First, the choice of the sub-indicators used is highly subjective and can therefore reflect
prejudice. Second, the weighting of each sub-indicator is a complex issue, as there may be trade-offs or complementarities between the different
aspects measured. Afonso et al. (2005), however, point out that in most cases giving alternative weights to the sub-indicators does not change the
results very much.
2
Other authors (e.g. SCP/CERP (2004)) have tried to improve on the work by Afonso et al. (2005). The countryclusters resulted are very similar. Southern European countries present low general and educational performance,
new EU member states show low general performance but high educational one, and the Northern European and
Anglo-Saxon countries with high scores in both items (although the differences among countries in the educational
performance are high; e.g. Luxembourg with a high macroeconomic score but fairly poor results for the effectiveness
of its education system). It should be noted that, though there is considerable correlation between public sector
performance in the different areas, it is by no means perfect. Countries that do well in several respects also produce
poorer performances in other areas. Finland, for example, records high scores for many policy areas, but has a low
score for other, such as crime. In the other side, Poland does badly in many areas, but does well on economic growth,
income distribution and its school drop-out rate. Similarly, Afonso et al. (2006) computed these indicators for the
new EU member states and emerging markets. They found that expenditure efficiency across new EU member states
is rather diverse, especially compared to the group of top performing emerging markets in Asia. From the analysis of
composite public sector performance and efficiency scores they found that countries with lean public sectors and
public expenditure ratios not far from 30% of GDP tend to be most efficient.
3.
Assessing Efficiency in Public Sector
The measurement of efficiency generally requires: (a) an estimation of costs; (b) an estimation of output; and (c) the
comparison between the two. Applying this concept to the spending activities of governments, we can say that public
expenditure is efficient when, given the amount spent, it produces the largest possible benefit for the country’s
population.3 Often efficiency is defined in a comparative sense: the relation between benefits and costs in country X
is compared with that of other countries. This can be done for total government expenditure, or for expenditure
related to specific functions such as health, education, poverty alleviation, building of infrastructures and so on. If in
country X the benefits exceed the costs by a larger margin than in other countries, then public expenditure in country
X is considered more efficient. However, the measurement of public efficiency is relatively complicated as
comparison and measurement of both costs and benefits may be difficult. Deficient budgetary classifications, lack of
reliable data, difficulties in allocating fixed costs to a specific function, and failure to impute some value to the use of
public assets used in the activity can also hamper the determination of real costs.4
Figure 1: Conceptual Framework of Efficiency and Effectiveness
Source: Mandl et al. (2008, p. 3).
Figure 1 illustrates the link between input, output and outcome, the main components of efficiency and effectiveness
indicators.5 The monetary and non-monetary resources deployed (i.e. the input) produce an output. For example,
education spending (input) affects number of students completing a grade (output). The input-output ratio is the most
3
The word benefit is used because economists often make a distinction between output and outcome.
More about measuring costs and efficiency of public spending see Drake and Simper (2001) and Afonso et al. (2006).
5
Another relevant issue for the analysis of public spending inefficiencies is the fact that public expenditure financing must rely on distortional
taxation. This implies that both direct and indirect costs are relevant when estimating the economic impacts of inefficiency in public services
provision. Afonso and Gaspar (2007), in simple numerical exercises, with a calibrated model, found that indirect costs, associated with excess
taxation burden, amplify the cost of inefficiency by between 20 and 30 per cent.
4
3
basic measure of efficiency.6 However, compared to productivity measurement, the efficiency concept incorporates
the idea of the production possibility frontier, which indicates feasible output levels given the scale of operations.
The greater the output for a given input or the lower the input for a given output, the more efficient the activity is.
Productivity, by comparison, is simply the ratio of outputs produced to input used.
On the other hand effectiveness relates the input or the output to the final objectives to be achieved, i.e. the outcome.
The outcome is often linked to welfare or growth objectives and therefore may be influenced by multiple factors
(including outputs but also exogenous 'environment' factors). The effectiveness is more difficult to assess than
efficiency, since the outcome is influenced political choice. The distinction between output and outcome is often
blurred and output and outcome are used in an interchangeable manner, even if the importance of the distinction
between both concepts is recognized. For example, the outputs of a health system are often measured in terms of the
number of operations performed or days spent in a hospital. The final outcome, however, could be how many
patients got well enough to return to an active life. Thus, the effectiveness shows the success of the resources used in
achieving the objectives set.
3.1.
3.1.1.
Methods for Measuring Public Sector Performance and Efficiency
Public Sector Performance and Efficiency Indicators
The measurement of public sector performance (PSP) (defined as the outcome of public sector activities) and
efficiency (PSE) (defined as the outcome relative to the resources employed) is still very limited. Afonso et al.
(2005) provides a proxy for measuring public sector performance and efficiency. 7 Their paper compares the
performance of the public sector and relates it to resource use. These authors use indicators relating to effectiveness
and in a number of major policy areas: education, health care and infrastructure. In addition, they draw on indicators
of the quality of public administration, based on survey data. Finally, the authors operationalize the conventional
functions of government: distribution, stabilization and allocation (Musgrave and Musgrave, 1984, p. 184). These
indicators are aggregated by means of unweighted totalling of standardised component scores. Performance is then
related to resource use on two levels: in each concrete policy area, and for the public sector as a whole.
Figure 2 displays the composition of PSP indicators.8 As to the “opportunity indicators”, administrative performance
of government is measured as a composite of the following indices: corruption, red tape, quality of the judiciary, and
the size of the shadow economy. The education indicator contains secondary school enrolment and the OECD
educational attainment indicators in order to measure both the quantity and quality of education. The health
performance indicator contains infant mortality and life expectancy. The public infrastructure indicator contains a
measure of the communication and transport infrastructure quality. All these indicators change slowly so that
observations every 10 years provide a good impression of changes over time except in the case of public
infrastructure where period averages have been used.
Afonso et al. (2005) distinguished public sector performance (PSP), defined as the outcome of public policies, from
public sector efficiency (PSE), defined as the outcome in relation to the resources employed. Assume that public
sector performance (PSP) depends on the values of certain economic and social indicators (I). If there are i countries
and j areas of government performance which together determine overall performance in country i, PSPi, (where
PSPij = f(Ik)) we can then write:
PSPi 
n
 PSP
(I)
ij
j 1
6
When measuring efficiency, a distinction can be made between technical and allocative efficiency. Technical efficiency measures the pure
relation between inputs and outputs taking the production possibility frontier into account. On the other hand, allocative inefficiency occurs if the
distribution of particular public sector outputs is not in accordance with personal preferences (Bailey, 2002, p. 119).
7
Some other authors and papers have tried to improve and supplement on the work by Afonso et al. (2005), such as SCP/CERP (2004), Sánchez
and Bermejo (2007), Afonso et al. (2006, 2008), etc.
8
The primary data for performance sub-indicators is based on World Bank (WDI and World Development Report) and IMF (WEO) data sources.
4
However, public sector performance must be set in relation to the inputs used to gauge their efficiency. In order to
get some values of efficiency of the state, public sector efficiency (PSE) is composed, taking into account the
expenditure related to each selected sub-indicator. In this respect, the public sector performance (PSP) indicator is
weighted by the relevant category of public expenditure (PEX) as follows 9:
PSEi 
PSPi

PEX i
PSPij
n
 PEX
j 1
(II)
ij
Figure 2: Public Sector Performance (PSP) Indicator
Source: Afonso et al. (2005, p. 10).
9
The input measures for opportunity indicators (see Figure 2) are public consumption, health expenditure and education as proxy for inputs to
produce administrative, health and education outcomes, respectively. On the other hand, the inputs for standard “Musgravian” indicators are
transfers/subsidies and total spending as proxies for input to affect income distribution and economic stabilization/economic efficiency,
respectively. For some caveats of such approach see Afonso et al. (2005).
5
3.1.2.
Other Parametric and Non-Parametric Methods
An alternative approach is based on the concept of efficiency frontier (productivity possibility frontier). There are
multiple techniques to calculate or estimate the shape of the efficiency frontier. Most investigations aimed at
measuring efficiency are based either on parametric or nonparametric methods. The main difference between the
parametric and the non-parametric approach is that parametric frontier functions require the ex-ante definition of the
functional form of the efficiency frontier.
A very common parametric approach is the Stochastic Frontier Analysis (SFA). It is a statistical method to fit the
frontier and is based on econometric methods. This approach assumes a specific functional form for the relationship
between input and output. The advantage of this method is that it is able to cover the effects of exogenous shocks, i.e.
nondiscretionary factors. The model can specify the equations based on such assumptions (Mandl et al., 2008, p. 9).
On the other hand, the non-parametric approach constructs an efficiency frontier using input/output data for the
whole sample following a mathematical programming method. 10 This frontier provides a benchmark by which the
efficiency performance can be judged. This technique is therefore primary data-driven. Among the different nonparametric methods the Free Disposal Hull (FDH) technique imposes the fewest restrictions. 11 It follows a stepwise
approach to construct the efficiency frontier. Along this production possibility frontier one can observe the highest
possible level of output/outcome for a given level of input. Conversely, it is possible to determine the lowest level of
input necessary to attain a given level of output/outcome. This allows identifying inefficient producers both in terms
of input efficiency and in terms of output/outcome efficiency (Afonso et al., 2003, p. 18).
An alternative non-parametric technique that has recently started to be applied to public expenditure analysis is Data
Envelopment Analysis (DEA).12 DEA approach is based on a linear combination of input and outputs in order to
specify the efficiency frontier. Convexity of the set of input-output combinations is assumed since this method
constructs an envelope around the observed combinations. According to DEA methodology, the general relationship
can be given by the following function for each country i (Afonso et al., 2006, p. 21):
Yi=f(Xi), i=1,...,n
(III)
where we have Yi – a composite indicator reflecting our output measure; Xi – spending or other relevant inputs in
country i. If Yi
f (xi), it is said that country i exhibits inefficiency. For the observed input level, the actual output
is smaller than the best attainable one and inefficiency can then be measured by computing the distance to the
theoretical efficiency frontier.
3.2.
Empirical Evidence of Public Spending Efficiency in Slovenia and Some Selected EU Member States
From the empirical results indicators suggest notable but not extremely large differences in the public sector
performance across countries (with a few exceptions) (see Table 1). Starting with the overall PSP indicator, the best
performers seem to be Cyprus, Ireland and Malta. Interestingly, Slovenia is just behind Malta and in front of Portugal
and Greece which post a broadly average result. When comparing the best performers in Afonso et al. (2006) with
those from Afonso et al (2003) (23 OECD countries), the results confirm that most of the new EU member countries
show lower public sector performance than developed OECD countries. 13 However, some of the new EU member
countries (such as Cyprus, Malta and Slovenia) are already on the average performance level of the “old”
industrialized countries.
Countries with the highest values for sub-indicators include Estonia (administration), Hungary (human capital),
Slovakia (distribution), Greece (economic stability) and Cyprus (economic performance). Slovenia as one of the
most developed new EU member states shows relatively good performance on human capital and stability, and lags
10
For an overview of non-parametric techniques see Simar and Wilson (2003).
FDH analysis was first proposed by Deprins et al. (1984).
12
DEA analysis, originating from Farrell’s (1957) seminal work was originally developed and applied to firms that convert inputs into outputs
(e.g. Coelli et al. (1998) for a number of applications).
13
Indeed, it is noteworthy that the US and particularly Japan report above-average for the total PSP measure and the EU (weighted average)
performs below average (for more see Afonso et al. (2005)).
11
6
behind on administration. Indeed, it is interesting that the new EU member states are particularly good performer on
human capital and income distribution (both probably show a heritage from a socialist system) and relatively weak
on administration, economic performance and stability (last two probably due to turbulent transition process).
Table 1: Comparison of Public Sector Performance (PSP) in Selected EU Countries (2001/2003)
Country
Opportunity
Indictors
Administration
Education
Health
Distribution
“Musgravian”
Indicators
Stability
Public
Sector
Performance
(PSP)
Economic
performance
Score (Rank)
Score (R)
Score(R)
Score (R)
Score (R)
Score (R)
Score
(R)
Bulgaria
0.80
(13)
1.09 (9)
0.99 (12)
1.17
(4)
0.06 (15)
0.31 (15)
0.74 (15)
Cyprus
n. a.
1.12 (5)
1.04 (1)
n.a.
1.59 (3)
1.54
(1)
1.33
(1)
Czech R.
1.00
(8)
1.14 (2)
1.02 (7)
1.19
(3)
0.74 (11)
0.74 (13)
0.97
(9)
Estonia
1.25
(1)
1.11 (7)
0.99 (12)
1.00 (12)
0.57 (12)
0.88
(6)
0.97
(9)
Greece
0.95
(10)
1.04 (12)
1.04 (1)
1.07 (10)
1.67 (1)
0.76 (12)
1.09
(5)
Hungary
1.09
(5)
1.16 (1)
1.00 (10)
1.21
(2)
0.97 (8)
0.88
(6)
1.05
(6)
Ireland
1.17
(2)
1.11 (7)
1.03 (4)
1.02 (11)
1.64 (2)
1.47
(2)
1.24
(2)
Latvia
1.03
(7)
0.98 (14)
0.98 (15)
1.08
(8)
0.76 (10)
0.88
(6)
0.95 (12)
Lithuania
0.98
(9)
1.12 (5)
1.00 (10)
1.08
(8)
0.37 (13)
0.84
(9)
0.90 (13)
Malta
1.11
(3)
1.03 (13)
1.04 (1)
n.a.
1.45 (4)
1.12
(3)
1.15
(3)
Poland
0.92
(12)
1.08 (10)
1.01 (8)
1.09
(7)
0.83 (9)
0.81
(10)
0.96 (11)
Portugal
1.11
(3)
0.88 (15)
1.03 (4)
0.98 (13)
1.30 (6)
0.91
(5)
1.04
(7)
Romania
0.63
(14)
1.13 (3)
0.98 (15)
1.10
(6)
0.18 (14)
0.63 (14)
0.78 (14)
Slovakia
0.95
(10)
1.07 (11)
1.01 (8)
1.28
(1)
1.09 (7)
0.77 (11)
1.03
(8)
Slovenia
1.07
(6)
1.13 (3)
1.03 (4)
1.14
(5)
1.35 (5)
0.99
(4)
1.12
(4)
Note: Data sources - World Bank (WDI and World Development Report), Global Competitiveness Report, IMF (WEO Database), 2003/2004.
Source: Afonso et al. (2006, p. 31), own calculations.
Indicators of Public Sector Efficiency (PSE) are computed weighing performance by the amount of relevant public
expenditure (see Table 2). One can find significant differences in public sector efficiency across countries. Ireland,
Cyprus and Greece show the best values for overall efficiency. In this respect, the results for measuring public sector
efficiency show an accentuation of the findings for public sector performance. This suggests that more public
spending often has relatively low returns as regards improved performance (which is consistent with the findings of
Afonso (2003) for OECD countries). Most low performers, range between 0.8 and 0.9 and Cyprus is the only new
EU member country with an average PSE score. Slovenia, in this respect, significantly lags behind Ireland and even
Cyprus and shows approximately same efficiency performance as most of the new EU member countries (see Figure
3).
Table 2: Comparison of Public Sector Efficiency (PSE) in Selected EU Countries (2001/2003)
Country
Opportunity
Indictors
Administration
Education
Health
Distribution
“Musgravian”
Indicators
Stability
Public
Sector
Performance
(PSE)
Economic
performance
Score (Rank)
Score (R)
Score(R)
Score (R)
Score (R)
Score (R)
Score
(R)
Bulgaria
0.79
(12)
1.49 (2)
1.00 (4)
1.01
(9)
0.06 (15)
0.29
(15)
0.77
(15)
Cyprus
n. a.
0.92 (10)
1.66 (1)
n.a.
1.44 (2)
1.39
(2)
1.08
(2)
Czech R.
0.76
(13)
1.31 (4)
0.66 (14)
1.04
(8)
0.66 (11)
0.66
(11)
0.85
(10)
Estonia
1.09
(2)
0.83 (12)
0.91 (6)
1.21
(3)
0.57 (12)
0.87
(5)
0.91
(5)
Greece
0.97
(3)
1.32 (3)
0.83 (9)
0.83
(10)
1.23 (3)
0.56
(14)
0.96
(3)
Hungary
0.83
(8)
1.12 (7)
0.75 (11)
1.05
(6)
0.70 (9)
0.63
(13)
0.85
(10)
Ireland
1.36
(1)
1.18 (6)
0.84 (8)
1.44
(1)
1.79 (1)
1.61
(1)
1.37
(1)
Latvia
0.82
(10)
0.79 (13)
1.14 (2)
1.11
(5)
0.75 (8)
0.87
(5)
0.91
(5)
Lithuania
0.83
(8)
0.88 (11)
0.90 (7)
1.27
(2)
0.40 (13)
0.90
(3)
0.86
(8)
Malta
0.92
(4)
0.99 (8)
0.68 (12)
n.a.
1.16 (4)
0.90
(3)
0.78
(14)
Poland
0.89
(7)
0.98 (9)
0.97 (5)
0.80
(13)
0.69 (10)
0.68
(9)
0.83
(12)
Portugal
0.92
(4)
0.71 (14)
0.66 (14)
0.90
(11)
1.01 (6)
0.71
(8)
0.82
(13)
Romania
0.69
(14)
1.53 (1)
1.03 (3)
1.05
(6)
0.20 (14)
0.68
(9)
0.86
(8)
Slovakia
0.82
(10)
1.23 (5)
0.77 (10)
1.18
(4)
0.90 (7)
0.64
(12)
0.92
(4)
Slovenia
0.91
(6)
n.a.
0.68 (12)
0.81
(12)
1.15 (5)
0.84
(7)
0.88
(7)
Note: Data sources - World Bank (WDI and World Development Report), Global Competitiveness Report, IMF (WEO Database), 2003/2004.
Source: Afonso et al. (2006, p. 33), own calculations.
7
When taking into account sub-indicators, it can be found that differences in efficiency are much more pronounced
than in performance across countries (see Table 2). New EU member countries with the highest values for efficiency
sub-indicators include Estonia (administration), Romania (human capital), Cyprus (health, economic stability and
performance) and Lithuania (distribution). Slovenia in this respect shows relatively poor efficiency performance
especially in the field of health, distribution and administration. Generally, a relatively average performance (PSP) of
the new EU member states is related with relatively high level of inputs, reflecting public sector inefficiency (low
PSE) in the region.
Figure 3:
Public Sector Performance (PSP) and Efficiency (PSE) in Slovenia and Selected EU Countries
1,5
CYP
Performance
IRL
MLT
P OR
1
SLO
HUN
CZE
P OL
LTU
GRE
SVK
LAT
EST
ROM
BUL
0,5
0,6
0,8
1
1,2
1,4
1,6
Efficiency
Source: Afonso et al. (2006, p. 31-33), own calculations.
When applying Free Disposal Hull (FDH) technique (excluding Singapore) Ireland and Cyprus are seen as efficient
(see Figure 4). Here an average public spending as percentage of GDP in 2001-2005 period measures the input and
as output we use PSP indicators presented before. One can see that some countries come very close to the frontier
(e.g. Estonia and Lithuanian) while the other countries are further and therefore less efficient. The figure also shows
that the great majority of the countries under consideration are well inside the production possibility frontier. They
mostly report a much higher ration of public expenditure-to-GDP than Ireland, but nevertheless often report lower
PSP indicators.
A DEA approach largely confirms the findings of the PSE composite indicators reported in Table 3 and FDH
approach presented in Figure 4. By using PSP composite indicator as output measure and the government
expenditure as a ration of GDP as input measure the results show that Slovenia could use about 57 per cent less
resources for the attained output. According to this score, Slovenia is even less efficient than Bulgaria, Czech R. and
Latvia. Moreover, from an output perspective, Slovenia presents slightly better ranking as for the level of its input
obtains around 73 per cent of the output it should be delivered. The top efficiency performers in the selected group of
the EU countries are Cyprus (input oriented score) and Ireland (output oriented score), but both significantly lag
behind the most efficient country, i.e. Singapore.
According to the presented results, significant challenges remain in the area of public sector performance
improvements for Slovenia in the near future. As already mentioned, the results suggest relatively significant
inefficiency in social spending (in health care and distribution) and public administration. In the health care,
inefficiencies are mainly related to relatively high spending (around 9 percent of GDP in recent years, which is above
EU-15 and OECD averages), rather than weak outputs. The efficiency and effectiveness of health care spending
could be improved by further reform with combination of different measures such as increasing the role of the
private sector in the provision of health care services, shifting resources to more affordable outpatient care,
rationalization of the hospital network etc. Moreover, despite social protection has been relatively effective in
reducing poverty, there is room to improve targeting in order to avoid double-dipping. Last but not least, Slovenian
8
public administration has a number of deficiencies (such as the lack of criteria for efficiency and motivational
elements, lack of the criteria for employment and promotion, lack of citizen participation, openness and transparency
of public administration, etc) that must be addressed in a further course of administrative reform in order to improve
its efficiency/effectiveness. To these difficulties with the organization and evaluation of work processes and the
coordination of activities, unsatisfactory openness towards citizen participation should be added. Although egovernment is developing, much need to be done in the near future to improve public administration performance in
Slovenia.
Figure 4:
Efficiency Frontier with the FDH Methodology (Selected EU Countries, excluding Singapore)
2
SGP
1,5
Performance
CYP
IRL
1
MLT
GRE SLO
HUN
P OR
SVK
LATP OL CZE
EST
LTU
ROM
BUL
0,5
10
20
30
40
50
60
T otal Spending Expenditure (in % of GDP)
Source: Afonso et al. (2006, p. 31), own calculations.
Table 3: Comparison of Efficiency in Selected EU Countries by PSE and DEA Methods
Public Sector Efficiency (PSE)
DEA Analysis
Input Oriented
Output Oriented
Score
Rank
Score
Rank
Score
Rank
Bulgaria
0.77
15
0.461
7
0.483
15
Cyprus
1.08
2
0.489
4
0.867
1
Czech R.
0.85
10
0.439
8
0.637
9
Estonia
0.91
5
0.489
4
0.632
10
Greece
0.96
3
0.369
14
0.713
5
Hungary
0.85
10
0.355
15
0.687
6
Ireland
1.37
1
0.576
1
0.813
2
Latvia
0.91
5
0.486
6
0.624
12
Lithuania
0.86
8
0.535
2
0.588
13
Malta
0.78
14
0.408
11
0.753
3
Poland
0.83
12
0.412
10
0.627
11
Portugal
0.82
13
0.385
13
0.678
7
Romania
0.86
8
0.528
3
0.509
14
Slovakia
0.92
4
0.406
12
0.674
8
Slovenia
0.88
7
0.431
9
0.731
4
Note: Countries included in the analysis but not included in the table: Brazil, Chile, Korea, Mauritius, Mexico, Singapore, South Africa,
Thailand and Turkey.
Country
Source: Afonso et al. (2006, p. 33-37), own calculations.
9
4. Methodological Approach for Measuring Public Sector Performance and Efficiency at Local Levels
In order to measure performance and efficiency at local levels a one input/one output approach is frequently
employed. Accordingly, to proxy for the municipal resources consumption (input) induced by the provision of
municipal services, for each municipality could be used its total per capita expenditures registered on municipal
accounts for a particular year. As a result, a suggested spending efficiency analysis does not distinguish technical
from allocative efficiency. However, as the measurement of the latter requires price information, while the former
only requires quantity data (Lovell, 2000), selecting per capita municipal spending gives at least the guarantee that
all inputs are considered in the analysis (De Borger and Kerstens (2000)). Additionally, this variable is a more
realistic municipal input measure (see Fisher (1996), De Borger and Kerstens (2000), Hayes, Razzolini and Ross
(1998), Afonso and Fernandes (2003)) if we acknowledge the reduced margin of manoeuvre of the municipal
authorities to influence current expenditure choices, mainly those concerning municipal personnel salaries.
Concerning municipal outputs, the analysis usually focuses on global municipal performance stemming from the
municipal provision of specific services (e.g., waste collection, road lighting, etc.). In this context, we can
constructed a composite indicator of municipal performance (for instance Total Municipal Performance index). Thus,
using that municipal performance indicator as an output measure and municipal per capita expenditure as the input
measure, different non-parametric (FHD, DEA, etc.) methods could be applied to the suggested data set (see table 4).
However, as the approach is confronted with the difficulty of directly measuring some of the municipal production
results, some performance indicators are surrogate measures of municipal demand. For instance, the indicators such
as »Total resident population« and »Centrality index« are used to capture the needs of local authorities to provid
common basic administrative services, factors which are beyond the control of individual municipal authorities (see
Athanassopoulos (1995)). The selection of indicators is based upon two general arguments implied within the
proposed analysis. First, municipalities with similar demand for homogeneous services should also have similar
performance (see Eeckaut, Tulkens and Jamar (1993) and Athanassopoulos (1995)). Second, performance of
municipal governments can be measured in terms of the improvement of observable factors directly controlled by
municipal governments during the time period under consideration. In Table 4 proposed performance indicators used
to quantitatively proxy the results of individual municipal services provision are presented. As suggested by several
authors (see, for example, De Borger and Kerstens (1996) and Afonso and Fernandes (2003)), to quantify a single
municipal performance indicator all values of each subindicator mentioned in Table 4 can be normalised by setting
the average equal to 1. Then, each sub-indicator relative to the overall average can be calculated, giving them an
equal weight. Finally, these sub-indicators are grouped under four five broader categories, giving to each subindicator an equal weight in order to compute a single municipal performance indicator.
Previous empirical analyses show that municipalities in the EU countries could be generally characterised as being
relatively inefficient (see for instance De Borger and Kerstens (1996), Afonso and Fernandes (2003)). Indeed, on
average, roughly the same level of local output could be achieved with substantially fewer resources, i.e., that local
performance could be improved without necessarily increasing municipal spending. Hence, previous empirical
results support the general argument that more spending does not necessarily translate into better local living
standards. One may hypothesise that the actual decentralisation process must be endowed of a rationale that
promotes the improvement of spending efficiency based on comparison of municipalities with clearly identified best
practices benchmarks.
10
Table 4: Selected Output/Outcome Indicators for Different Groups of Municipal Services
Source: Fisher (1996), Afonso and Fernandes (2003).
11
5. Conclusion
Tight budgets and demanding citizens put governments under increasing pressure to show that they are providing
good value for money. Providing information about public sector performance can satisfy the public’s need to know,
and could also be a useful tool for governments to evaluate their performance. In this respect, the aim of the paper
was to discuss and review different conceptual and methodological issues related to the performance measurement in
public sector. In recent years, the debate of the role of the public sector has shifted significantly towards empirical
assessments of the efficiency and usefulness of its activities. In particular, a composition of Public Sector
Performance (PSP) and Public Sector Efficiency (PSE) indicators as well as Free Disposal Hull (FDH) and Data
Envelopment Analysis (DEA) was presented and applied to Slovenia. Slovenia as one of the most developed new EU
member states shows relatively good performance on human capital and stability, and lags behind on public
administration. In addition, when measuring public sector effectiveness, Slovenia significantly lags behind Ireland
and even Cyprus and shows approximately same efficiency performance as most of the new EU member countries.
The results also show that Slovenia could use about 57 per cent less resources for the attained output. However,
according to this score, Slovenia is even less efficient than Bulgaria, Czech R. and Latvia. Moreover, from an output
perspective, Slovenia presents slightly better ranking as for the level of its input obtains around 73 per cent of the
output it should be delivered. All in all, the empirical results show that Slovenia has relatively low efficiency in
public sector, even in comparisons to some new EU member states (such as Cyprus). Accordingly, there is a room to
improve key social indicators while containing public spending. In this respect, Slovenia has a great potential for
increased efficiency and effectiveness in public spending, especially in the filed of public administration, health care
and distribution.
Furthermore, the current paper also points out the need for public performance measurement at local levels. Indeed,
number of empirical researches found substantial inefficiency at the local levels. These results support the general
argument that more spending does not necessarily translate into better local living standards. Therefore, the empirical
framework could be extended to the Slovenian case in the near future in order to access the (in)efficiency at the
municipal level in comparison with the other EU member states. On the basis of the results it would be possible to
argue that a decentralisation process in Slovenia should be endowed of a rationale that promotes the improvement of
spending efficiency based on comparison of the municipalities with clearly identified best practices benchmarks.
Nevertheless, at least three caveats should be pointed out when we measure performance of public sector. Firstly, the
applications of presented techniques are hampered by lack of suitable data to apply those techniques. Quality data are
needed because the techniques available to measure efficiency are sensitive to outliers and may be influenced by
exogenous factors. Indeed, substantial inefficiency may be simply a reflection of environmental factors (such as
climate, socio-economic background, etc.). This also suggests applying a combination of techniques to measure
efficiency and effectiveness. Secondly, the precise definition of inputs, outputs and outcomes may significantly
influence the results. Finally, it seems important to bear in mind that by using a non-parametric approach, and in
spite of DEA being an established and valid methodology, differences across countries are not statistically assessed,
which can be considered as a limitation of such methodology.
12
References
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
Afonso, A. and V. Gaspar. 2007. Dupuit, Pigou and cost of inefficiency in public services provision. Public
Choice, 132 (3-4), 485-502.
Afonso, A. and S. Fernandes. 2006. Local Government Spending Efficiency: DEA Evidence for the Lisbon
Region. Regional Studies 40 (1), 39-53.
Afonso, A. and S. Fernandes. 2003. Efficiency of Local Government Spending: Evidence for the Lisbon Region.
Available at SSRN: http://ssrn.com/abstract=470481 or DOI: 10.2139/ssrn.470481
Afonso, A. and St. Aubyn. 2005. Non-parametric Approaches to Education and Health Efficiency in OECD
Countries. Journal of Applied Economics, 8 (2), 227-246.
Afonso, A. and St. Aubyn. 2006a. Cross-country Efficiency of Secondary Education Provision: a Semiparametric Analysis with Non-discretionary Inputs. Economic Modelling, 23 (3), 476-491.
Afonso, A. and St. Aubyn. 2006b. Relative Efficiency of Health Provision: a DEA Approach with Nondiscretionary Inputs”. ISEG-UTL, Department of Economics Working Paper nº 33/2006/DE/UECE.
Afonso, A., Schuknecht L. and V. Tanzi. 2005. Public Sector Efficiency: An International Comparison. Public
Choice, 123 (3-4), 321-347.
Afonso, A., Schuknecht L. and V. Tanzi. 2006. Public Sector Efficiency: Evidence for New EU Member States
and Emerging Markets,” European Central Bank, Working Paper Series 581, European Central Bank: Frankfurt.
Afonso, A., Schuknecht L. and V. Tanzi. 2008. Income distribution determinants and public spending efficiency.
Working Paper Series 861, European Central Bank: Frankfurt.
Athanassopoulos, A. 1995. Goal Programming & Data Envelopment Analysis (GoDEA) for Target-Based
Multilevel Planning: Allocating Central Grants to the Greek Local Authorities, European Journal of
Operational Research, 87 (3), 535-550.
Bailey, S. J. 2002. Public Sector Economics: Theory, Policy and Practice. London: Palgrave Macmillan.
Clements, B. 2002. How Efficient is Education Spending in Europe? European Review of Economics and
Finance, 1, 3-26.
Coelli T., Rao, D. and G. Battese. 2002. An Introduction to Efficiency and Productivity Analysis. 6th edition,
Massachusetts, Kluwer Academic Publishers.
De Borger, B., Kerstens, K., Moesen, W. and Vanneste, J. 1994. Explaining differences in productive efficiency:
An application to Belgian Municipalities, Public Choice, 80, 339-358.
De Borger, B. and K. Kerstens. 1996. Cost efficiency of Belgian local governments: A comparative analysis of
FDH, DEA, and econometric approaches. Regional Science and Urban Economics, 26, 145-170.
De Borger, B. and Kerstens, K. 2000. What Is Known about Municipal Efficiency? in: Blank, Jos L. T. (ed),
Public Provision and Performance: contributions from efficiency and productivity measurement, Amsterdam,
North-Holland, 299-330.
Deprins, D., Simar L. and H. Tulkens. 1984. Measuring labor-efficiency in post offices; in: Marchand, M.;
Pestieau, P. and Tulkens, H. (Eds.), The performance of public enterprises: concepts and measurement.
Amsterdam: North-Holland.
Drake, L. and R. Simper. 2001. The Economic Evaluation of Policing Activity: An Application of a Hybrid
Methodology. European Journal of Law and Economics, Vol. 12.
Eeckaut, P., Tulkens, H. and Jamar, M. A. 1993. Cost Efficiency in Belgian Municipalities, in: Fried, H.; Lovell,
C. and Schmidt, S. (eds), The Measurement of Productive Efficiency – Techniques and Applications. New York,
Oxford University Press, 300-334.
Fakin. B. and A. de Crombrugghe. 1997. Fiscal adjustment in transition economies: social transfers and the
efficiency of public spending: a comparison with OECD countries. Policy Research Working Paper 1803.
Washington, DC: World Bank.
Farrell, M. 1957. The Measurement of Productive Efficiency. Journal of the Royal Statistical Society Series A
(General), 120 (3), 253-281.
Fisher, R. 1996. State and local Public Finance. USA, Irwin.
Gupta, S. and M. Verhoeven. 2001. The efficiency of government expenditure Experiences from Africa. Journal
of Policy Modelling, 23, 433-467.
Gwartney, J., Lawson, R., Park, W., Wagh, S., Edwards, C. V. y de Rugy. 2002. Economic freedom of the
world: 2002 Annual Report. Fraser Institute, Vancouver.
Hayes, K.; Razzolini, L. and Ross, L. 1998. Bureaucratic choice and nonoptimal provision of public goods:
Theory and evidence, Public Choice, 94, 1-20.
13
26. Lovell, C. 2000. Measuring Efficiency in the Public Sector, in: Blank, Jos L. T., (ed), Public Provision and
Performance. Amsterdam, North-Holland.
27. Mandl, U., Dierx A. and F. Ilzkovitz. 2008. The effectiveness and efficiency of public spending. European
Commission, Economic Papers 31, Februar.
28. Musgrave, R.A. and P.B. Musgrave. 1984. Public Finance in Theory and Practice. Fourth edn., McGrawHill.
29. Persson, T. and G. Tabellini. 2001. Political institutions and policy outcomes: What are the stylised facts?,
Mimeo
30. Rodrick, D. 2000. Institutions for high-quality growth: What they are and how to acquire them. NBER Working
Paper nº 7540.
31. Sánchez, A. M. and L. R. Bermejo. 2007. Public Sector Performance and Efficiency in Europe: The Role of
Public R&D. Working Papers 01/2007, Institute of social and Economic Analysis: Madrid.
32. SCP/CERP. 2004. Public Sector Performance. An international comparison of education, health care, law and
order and public administration. Social and Cultural Planning Office, The Hague.
33. Simar, L. and P. Wilson. 2003. Efficiency analysis: the statistical approach, lecture notes.
34. St. Aubyn, M. 2003. Evaluating Efficiency in the Portuguese Education Sector. Economia, 26, 25-51.
35. Strauch, R. and J. von Hagen. 2000. Institutions, politics and fiscal policy. Kluwer Academics Publishers,
Boston.
th
36. Tanzi, V. and L. Schuknecht. 2000. Public spending in the 20 century. A global perspective. Cambridge
University Press, Cambridge.
37. Wallis, J. and Oates, W. 1988. Decentralization in the Public Sector: An Empirical Study of State and Local
Government, in: Rosen, H. (ed), Fiscal Federalism: Quantitative Studies. Chicago: University of Chicago Press.
14