Download Setting the Context for Leadership Performance Measurement

Results Based Management
Workshop
Setting context for Leadership
Performance Measurement
Ging Wong
Andrew Graham
Objective
Set the Project in context of Results
Based Management developments in
Canada and elsewhere
Provide an overview of the linkages of
results, accountability, performance
measurement and risk
Suggest ways that the Project can
uniquely support the effective transfer of
relevant and useful elements of RBM
School of Policy Studies
ACCOUNTABILITY
RISK
3
PERFORMANCE
MEASUREMENT
School of Policy Studies
Flexibility and Control
Tension in management thinking and practice
between structure/accountability and
initiative/flexibility.
Challenge is how to create a balance that recognizes
public sector accountabilities that create very high
levels of expectation with the need to get the job
done, i.e. meet the objectives of the program.
4
School of Policy Studies
Flexibility and Control
“While flexibility is becoming essential, the
public is not willing to forsake accountability
to achieve it. Confronted with antiquated
systems of governance, managers must
exercise leadership, not to in an ‘anything
goes’ manner. In short mangers face the
challenge of developing an alternative form of
accountability that allows for greater
flexibility of action.”
Feldman and Khademian
5
School of Policy Studies
Flexibility and Control
Public Sector accountability is complex and
varied
It is changing with the role of public servants
changed, becoming more involved
Understanding accountability in the public
sector means understanding that there is a
continuum of accountabilities: from
operational to policy to political
6
School of Policy Studies
Flexibility and Control
Reality that there are significant ‘gear
slippages’ in public sector accountability in
which the day-to-day at times takes on a
much greater significance than so-called
higher level accountabilities for policy
outcomes
In the public sector it is more likely that a
failed program can run really well and a
successful one can founder on details
7
School of Policy Studies
Defining the Concept of
Accountability….
The basic notion of accountability is
that those acting on behalf of
another person or group report
back to the person or group or are
responsible to them in some way.
- Owen E. Hughes, Public Management and
Administration
8
School of Policy Studies
Defining the Concept of
Accountability….
“Accountability is about evaluating performance,
meeting legitimate standards, fulfilling legitimate
commitments, and holding responsible those who fail
to meet the standards. The right to judge government
performance flows naturally from the role of citizen,
as does the right to sanction those who fail to meet
the standards.”
Janice Gross Stein, The Cult of Efficiency
9
School of Policy Studies
Defining the Concept of
Accountability….
•From a management perspective, accountability
runs the full gambit of activities, from the specific
to the general
•In the public sector, this means from detailed
administration to accountability for program
outcomes
•At no point is this either easy or simple.
•There are no fire walls between the detailed and
the general.
School of Policy Studies
10
The Accountability Continuum
“ Public mangers do not have the luxury of
separating out expectations for flexible
leadership from demands for strict
accountability in the form of structures or
guarantees that check and limit
management action. They must grapple with
both.”
- Feldman and Khadamian
11
School of Policy Studies
The Accountability Continuum
INTERNAL ACCOUNTABILITY
EXTERNAL ACCOUNTABILITY
MANAGEMENT
COEFFICENTS
DATA
12
PROGRAM
COEFFICIENTS
INFORMATION
KNOWLEDGE
School of Policy Studies
Sustained accountability through
results based management
Key trend in governments in
Canada
Federal government and
provinces have adopted various
methodologies
School of Policy Studies
13
Based on simple model of organizational
learning and adaptation
School of Policy Studies
14
RBM leads to consideration of key
elements of a management framework
Effective statement of goals and
outcomes
Performance measurement systems
School of Policy Studies
15
RBM leads to consideration of key
elements of a management framework
The role of monitoring and control in
execution
School of Policy Studies
16
RBM leads to consideration of key
elements of a management framework
The role of evaluation in determining
outcomes and the validity of the of the
outcome expectations
Means to develop organizational
learning across complex systems in
both implementing and adapting
policies (case studies, etc)
School of Policy Studies
17
desirable properties of performance measuresPerformance measures have a number of desirable properties. Most of these can be dispensed with fairly
quickly to focus on three key problems:relevance and balance
Performance measures such as those embodied in the IDTs are undoubtedly relevant. There
are, however, two problems of balance. First is that the measures may not span an agency’s portfolio. For example, the health targets in DFID’s PSA
exclude health activities outside of the top ten recipients. And the targets as a whole do not apply to activities in middle-income countries. The second
problem of balance is that, for reasons discussed below, it is better to have a set of indicators spanning the range of the log-frame from inputs to
outcomes.measures known, understood and trusted
Performance measures used in development are usually clearly defined and well understood, at least
by those who work directly with them. There has been some tension between top-down setting of targets and a more participatory approach. This has also
been a factor behind the promotion of the Millennium Development Goals (MDGs) as rivals to the IDTs. Data quality can be more problematic. The IDTs
include maternal mortality, for which data are notoriously unreliable. The measure was included in DFID’s first PSA, but when it came to reporting DFID
told the Treasury that data were unavailable. Country coverage for most indicators is patchy. But the more serious problem is the timeliness of the data,
discussed below.affected and attributable
Changes in performance measures should be affected by the activities of the organisation and the extent of the
effect measurable (attributable). However, attribution becomes harder to establish as we move through the log-frame from inputs to outcomes. As we saw
above, USAID has abandoned claiming that changes in economic outcomes can be attributed to its work.achievable
Targets should be achievable, but not
too easily. Most analyses suggest that the IDTs will not be met, although there is regional variation in performance. But this is not the same as saying that
they could not be met if aid and government resources were better focused toward meeting them.linked to existing management systems
Organisations
will already have in place management information systems. The rise of results-based measures was in part a response to the fact that existing systems
focused on inputs and internal activities, such as spending and staffing, rather than achieving outcomes. But new results-oriented systems should not be
separate or parallel to these existing systems. There are two further problems here: aggregation (can the various measures be added up to summarise
performance for regions, sectors and the agency as a whole?) and alignment (do the data tell us about impact on the outcomes of interest?)
School of Policy Studies
18
three key problemsThe above discussion identifies three problem areas. These
problems are less severe (though present) when using RBM at the project level,
becoming more intense as we attempt to measure results at country level or for the
whole portfolio.data availability
Data become available only with a time lag, this
being particularly so for developmental outcome measures like infant mortality and
income-poverty. DFID’s PSA covering 2001-2004 includes amongst its targets 'a
decrease in the average under-5 mortality rate from 132 per 1,000 live births in 1997
to 103 on the basis of data available in 2004'. Implicit here is the recognition that
data available in 2004 will probably be for 2001, namely the start of the period of the
current PSA. How can results of current strategies be judged on the basis of results
achieved before the strategy was put in place?attribution
It is very difficult to
attribute changes in developmental outcomes to the activities of an individual
agency. This problem is encountered where agencies work together, as in sector
programmes or providing budget support, or for agency performance at the country
level. One agency’s support to public sector reform in an African country (providing
the funds to pay for retrenchment packages) had as the outcome indicator: maternal
mortality. It is difficult to imagine what sensible use could have been made of
maternal mortality data (even with no time lag) to judge the performance of this
programme.integration with existing management information systems
All donor
agencies have some sort of monitoring and evaluation system at the project and
programme level which should provide a basis for both feedback at the project level
and 'feed-up' to management. But how do these systems tackle the problems of
aggregation and alignment
School of Policy Studies
19