Download Lessons learnt from drugs and alcohol payment by results pilot

Drug and alcohol recovery pilots
Lessons learnt from drugs and alcohol
payment by results pilot
Contents
1. Introduction
2. Lessons learnt
2.1 Local area single assessment and referral system (LASARS)
2.2 Deciding outcomes and setting payment structures
2.3 Resources
2.4 Contracts
2.5 System change
2.6 Data
3. Summary
2
1. Introduction
In its 2010 Drug Strategy, the government announced it would pilot payment
by results in the drug and alcohol sector. Service providers would be paid a
proportion of their contract based on specific successful outcomes. Eight
areas took part in the pilot:
- Oxfordshire
- Stockport
- West Kent
- Wigan
- Wakefield
- Lincolnshire
- Bracknell Forest
- Enfield
A national model for the pilots emerged from discussions between
government departments, local authorities and relevant stakeholders in the
drug and alcohol sector. Each area then adapted the model to fit local needs.
In April 2012, eight unique drugs and alcohol recovery payment by results
pilots (DAR PbR) were launched.
3
1. Introduction
This document is intended for use by commissioners and providers of drug
and alcohol services who are considering using a PbR method. The aim of the
document is to highlight the key areas where additional attention may need
to be focused in order to optimise the PbR approach.
This document draws on the comments of the eight pilots during the twoyear implementation phase and their experiences of payment by results. It
also looks at some of the wider lessons on commissioning drug and alcohol
services.
An independent evaluation of the DAR PbR pilot programme has been
commissioned and will report in spring 2015. This document does not preempt those findings.
4
1. Introduction
The three key main messages from the DAR PbR pilots are:
1. The PbR pilots sharpened providers’ focus on achieving sustained recovery
for drug and alcohol users and helped to encourage innovation
2. Commissioners and providers have had to spend a lot of time monitoring
performance under PbR and it has been an administratively burdensome
system
3. Partnership working is essential: understanding what the data shows,
sharing this information and negotiating any necessary action are all vital
if to PbR is to work
Seven of the eight pilot areas will continue with the PbR approach after the
pilots have ended.
5
2.1. Lessons learnt: Local area single assessment and
referral systems (LASARS)
One bit of the pilot system we will definitely keep when the
pilot is over is the LASARS function
We have seen a big
increase in activity
but we don’t know
why yet. It could be
due to more focus on
assessments,
chasing
appointments or a
clearer pathway into
treatment
WHAT THE
PILOT
AREAS SAY
Our LASARS function is
more resource intensive
than we had originally
anticipated
The LASARS complexity setting function influences the amount of money a provider can
receive so it is essential that clients are accurately assessed. Commissioners need to closely
audit the function so that providers don’t suffer as a result of poor LASARS performance
6
2.1. Lessons learnt: Local area single assessment and
referral systems (LASARS)
LASARS assess the complexity of users (e.g. heroin users require more input
to achieve recovery outcomes than cannabis users), which forms the basis of
the PbR tarrifs (the amount paid for an outcome for a certain complexity of
user).
The pilots used a number of different LASARS models: those run by
independent providers, those embedded in commissioning teams, and those
run by providers. The latter model requires independent audit to ensure
providers do not game the system by exaggerating user complexity.
Independent LASARS must function effectively if providers are to receive
accurate payments.
7
2.1. Lessons learnt: Local area single assessment
and referral systems (LASARS)
Some LASARS models do only the necessary tasks of assessing complexity and
setting tariffs; others employ skilled professionals (eg, social workers, nurses)
to do in-depth individual assessments.
Each model has its pros and cons, but those using the more basic model
found it easier to recruit and retain staff.
LASARS are also resource intensive, so decisions on design need careful
consideration.
8
2.2. Lessons learnt: Deciding outcomes and setting
payment structures
PbR in drug and alcohol recovery services can be complex. We are glad we
decided to allocate a fairly low proportion of the overall contract price to outcome
payments. It was big enough to focus providers attention but small enough to
help build relationships with our provider
We needed to change
tariffs after year one. We
had to decide whether to
pay outcomes on the
basis of year of entry to
treatment or year of
achievement of outcome.
This is complex but there
are benefits to deciding
this early on
WHAT THE
PILOT
AREAS SAY
Setting tariffs
takes a long time.
Longer than you
might think
We wanted to have a balance of interim and
final outcome measures. However, this has
meant we have a large number of outcomes
which makes the process complicated
9
2.2. Lessons learnt: Deciding outcomes and setting
payment structures
While some PbR schemes pay out for activity, the DAR PbR pilots pay for
specific outcomes.
However, genuine outcomes can take a long time to achieve. So to secure an
ongoing income for providers, some pilots allocated a lower proportion
(~30%) of the contract price to PbR, while others paid an attachment fee for
each service user engaged.
A large number of potential outcomes add complexity to the system and do
not necessarily secure extra funding for providers.
What’s more, capturing outcome indicators that can be clearly measured, and
building them into the contract and tender, is complex and time-consuming.
10
2.2. Lessons learnt: Deciding outcomes and setting
payment structures
If the proportion of the contract paid out on a PbR basis is to increase over
time (i.e. year on year), how the tariffs are divided needs to be considered up
front. For example, are they paid based on the year users enter treatment or
the year they leave?
It is also necessary to be able to change tariffs, but this can be complex.
Providers need to develop their understanding of which interventions are
likely to deliver outcomes.
11
2.3. Lessons learnt: Resources needed
We found the DAR PbR pilot resource intensive for both our providers and
commissioners and it required managing a lot of data
LASARS are
still a new
concept and
some models
require a
considerable
amount of
resource
WHAT THE
PILOT
AREAS SAY
Have management and
monitoring systems
ready well in advance
Given that providers’ payments are dependent on reported outcome
achievement, providers need to ensure that data is robustly
recorded and systematically reported, while commissioners need to
be confident that they understand the outcome data and can
answer providers’ queries and challenges
12
2.3. Lessons learnt: Resources needed
It takes time and cooperation among service providers, service users and
commissioners to develop a successful PbR model. Compromises are needed
to find a model that works for everybody. Many pilot areas recommend a 12month ‘shadow PbR’ that sets performance benchmarks ahead of actually
linking payments to outcomes.
Most pilots found the PbR method was data heavy. Even with additional
support, some had to hire additional people to analyse the data.
Many providers took longer than expected to manage the data collection. It
was important for them to get this right, as inaccurate recording could lead to
under-payment and a lot of work to correct the situation.
At the same time, commissioners had to monitor performance closely to
ensure the correct tariffs were set and outcomes were accurately recorded.
13
2.4. Lessons learnt: Contracts
We worked closely with our provider to design a flexible contract
Ensure contracts are
flexible enough to allow
both sides to exit before it
gets to the point where a
provider does not get paid
because outcomes haven’t
been met. This protects the
organisations and the
service users
WHAT THE
PILOT
AREAS SAY
It looks like our provider may
achieve better results than
we anticipated so we are
glad we placed a cap on our
contract
We forgot to factor in the pension costs when TUPEing staff across. This has
made the contract more expensive than initially anticipated
14
2.4. Lessons learnt: Contracts
Good relationships between commissioners and providers are key. Both need
to trust the data-monitoring process and to be prepared to negotiate as new
information comes to light.
Flexible contracts with regular break clauses give both sides the chance to
make changes to payments or to exit if things aren’t working out.
As in any contract, consider carefully how you will address break clauses and
TUPE issues, to ensure continuity of service provision to a group of highly
vulnerable individuals.
Tariffs are based on estimates of success. If success is greater than
estimated, it could put pressure on budgets – payment caps may be needed.
15
2.5 Lessons learnt: System change
We experienced a shift in focus towards recovery from our Drug and Alcohol service
providers. It is unclear whether it was the wider changes to the system or the PbR
element that provided this shift
We included service users
every step of the way in our
service redesign
WHAT THE
PILOT
AREAS SAY
You can expect to see a
dramatic dip in successful
completions and other
outcomes for at least six
months after a big change to
the system
We wish we had introduced changes to the system before rather than in parallel
with moving to PbR commissioning. Introducing the two together was too great
a change
16
2.5 Lessons learnt: System change
The National Treatment Agency set up drug system change pilots (DSCPs) in
2009 to see if more user-led, outcome-focused approaches could improve the
way drug treatment and related services were delivered. NatCen evaluated
these pilots, with many of their findings reflected in the lessons of the DAR
PbR pilots:
1.
Good partnership working is vital to effective system change. Treatment
providers and partner agencies who are actively engaged and committed
are crucial elements of successful pilots.
2.
Strong leadership and workforce development are important foundations
for system change.
3.
DSCPs spoke of the “strain of change” associated with significant
upheavals in the treatment system and delays in getting the new systems
fully functioning. As a result, the full impact of the pilots and the return
on the investment will not be visible for some time
17
2.6 Lessons learnt: Data
Accurate data is the most important element of PbR. It is how providers get paid
Sometimes our data is
telling us that things aren’t
going well but neither we
nor our providers can
understand why yet
WHAT THE
PILOT
AREAS SAY
It has taken longer than we
anticipated to iron out the
collection and interpretation
of the data
Our provider needs to have faith in the data we are using to evaluate their
performance
18
2.6. Lessons learnt: Data
Because PbR pays out on results, it relies on accurate data to assess how far a
provider has helped users achieve recovery outcomes.
There will always be some variation when measuring outcomes. Much is
explained by known factors (e.g. the mix of users, or seasonal variations in
types of crime). There are also unknown variations – other potential factors
that may affect outcomes.
This unexplained variance is called ‘noise’ and is typically a symptom of small
scale projects, such as the drug and alcohol recovery pilots. While it is not
possible to adjust measurements for noise, its size can be estimated.
19
2.6. Lessons Learnt: Data
Noise has been found in the drug and alcohol outcome data, meaning the
data may not necessarily reflect providers’ input – they could sometimes be
paid when their work has not led to improved user outcomes, and sometimes
not be paid when their work has led to improved outcomes.
There is a risk that the level of Noise could affect providers propensity to
innovate if they are not confident that extra effort results in extra reward.
20
2.6. Lessons learnt: Data – impact of unexplained
variance
Low unexplained variance
Effective provider
Ineffective
provider
High unexplained variance
8/10
High unexplained variance
3/10
Chance works in provider’s favour
Chance works against provider
Low unexplained variance
Outcome measure:
will they get paid based on
their actual performance?
21
2.6. Lessons learnt: Data – mitigating unexplained
variance
The impact of unexplained variance on providers is largely determined by the
size of the client-base. Larger cohorts (such as those in other PbR initiatives:
e.g. Transforming Rehabilitation and the Work Programme) mitigate against
unexplained variance.
However, DAR services commissioned at local authority level have much
smaller client-bases, so the unexplained variance falls outside the acceptable
range for the outcome data to be a robust reflection of provider input.
Allocating a smaller proportion of overall contract value to specified
outcomes (e.g. up to 20%) will reduce the financial impact of unexplained
variance.
22
3. Summary
While it is too soon to draw conclusions, the pilots report seeing benefits to
PbR when commissioning recovery services.
They have also faced challenges, which is to be expected when testing a new
approach.
Their feedback may help commissioners and providers who are designing
their own PbR systems.
23
3. Summary
The three key messages the pilots shared are:
1. PbR has sharpened providers’ focus on achieving sustained recovery for
drug and alcohol users and helped encourage innovation.
2. Commissioner and provider staff have to spend a lot of time monitoring
performance under PbR and it has been administratively burdensome.
3. Partnership working is essential: understanding what the data shows,
sharing this information and negotiating any necessary action are all vital
if to PbR is to work.
Seven of the eight pilot areas will continue with the PbR approach after the
pilots have ended.
24