Download Report on outcomes data retreat held at the University of Edinburgh

Meaningful AND Measurable
Exploring the use of personal outcomes information in health
and social care
Data Retreat Report
Background
A two day residential data retreat was held at the University of Edinburgh in March 2012. Core
funding was obtained from the University of Edinburgh College Knowledge Exchange fund by Ailsa
Cook, supplemented by funding from the Community Care Benchmarking Network (CCBN), The
Public Policy Network at the University of Edinburgh and by the Institute for Research and
Innovation in Social Services (IRISS). The overall aim of the retreat was to provide an opportunity for
practitioners, managers, information specialists, policy makers and researchers to work together to
explore the practical and conceptual issues underpinning the capture, analysis and use of
information on service user and carer outcomes in health and social care settings. The intention was
to achieve this overall aim by addressing the following objectives:





Begin to establish a shared language for discussing the capture, analysis and use of
information on service user and carer outcomes
Consider the conceptual and practical issues underpinning the capture, analysis and use of
information on service user and carer outcomes.
Identify and share current practice in this area.
Identify gaps and areas where capacity and skills need to be addressed
Develop an action plan for addressing these issues
Most of the available time was devoted to group work, allowing individuals from diverse
organisations to work collaboratively to address the identified objectives. Gordon Smith from the
CCBN facilitated an exercise which involved delegates working in pairs, role-playing outcomes
focused conversations between a service user and a practitioner, who recorded the outcomes in a
pro-forma. For most of the groupwork exercises, groups recorded their key messages on flipcharts
which were used to feed back to plenary sessions, allowing key themes to emerge. The flipcharts
were taken away after the event, so that the key themes could be collated, as set out in pages 2 to 4.
There were also a limited number of inputs, as follows:
Measuring outcomes: imperatives, challenges and strategies, Emma Miller, University of Strathclyde
Observations from a sociological perspective, Richard Freeman, University of Edinburgh
To what extent is personal experience measurable? Lessons from research into patient experience
and quality of life, Pam Ramsay, University of Edinburgh
Current practice in the capture, analysis and use of outcomes, Bridey Monger and Nigel Henderson,
Penumbra
1
Day 1 focused largely on setting the context and sharing current practice and issues, while Day 2
involved pinning down the issues in order to develop an action plan. At lunchtime on day 2,
delegates were joined by members of the Joint Improvement Team and the Care Inspectorate, with
a view to discussing key recommendations with representatives from national bodies. The
organising group, who co-facilitated the event, consisted of Ailsa Cook, University of Edinburgh,
Gordon Smith, CCBN, Ellen Daly, IRSS and Emma Miller, University of Strathclyde. A full list of
delegates is available at the end of the report.
Overarching messages
A central finding from the data retreat was that it is not possible to look at issues around the capture
and use of outcomes information in isolation. It was evident from the discussions that participants
viewed wider issues of culture, systems, practice and policy as inseparable from the more specific
issues around outcomes data. Key messages from these discussions included:
•
To realise the benefits of an outcomes approach, it is essential to start from the person not
from system priorities
•
There is considerable confusion in the use of language surrounding the outcomes agenda.
Different agencies need to develop a shared understanding of the language and use it
consistently
•
Good quality conversations are essential to support an outcomes focus, from frontline
practice, through to policy level
•
Practitioners need continued support through positive leadership, and outcomes focused
supervision – ‘consistent reinforcement of the dialogues throughout the organisation’
•
The culture change involved is significant, including the shift from focusing on deficits to
focusing on capacities and strengths
•
Because no one organisation can achieve the changes alone, collaboration between
agencies, including local authorities, the NHS, third sector and regulatory bodies, is key. A
network is essential and participating organisations were viewed as having a key role in this
•
The commitment to collaboration will be tested by the currently competitive environment
•
Although there are policy drivers which reference outcomes, there are also policy barriers,
particularly the prevailing performance culture, with a focus on targets not outcomes
One approach or many?
There was ongoing discussion as to practicality and desirability of developing a core set of methods
and tools for capturing outcomes data that could be implemented across organisations. Participants
were keen to learn from good practice elsewhere, but simultaneously wanted to ensure that
systems and processes were sensitive to their local context. There was agreement that whilst
methods for capturing , analysing and reporting on outcomes data needed to be specifically tailored
to local contexts there was potential to share a common overarching approach to the task, including
2
the philosophy and assumptions underpinning this work. There is also scope to share the use of
specific tools, which could be applied in different contexts.
Core themes
By the end of the first day, it was possible to identify five key themes which related directly to the
capture, analysis and use of outcomes information. On the second morning, participants elected to
participate in a working group on one of the five key themes identified: scale measures, recording
outcomes, a shift in thinking from attribution to contribution, using qualitative and quantitative data
and using outcomes information. These themes are briefly described below, with a summary of key
discussion points from the two days, and the recommendations put forward for each of them.
Scale measures
Visual displays such as spidergrams can be appealing to both service users and practitioners
Words can be preferable to numbers for different reasons: numerical scales can disrupt the
conversation; it can be more challenging for people with cognitive impairments to manage
numerical scales, and numbers can be more difficult to interpret in general –‘what does a 3 mean? ‘
Need to ensure that the person is not judged or made to feel a failure if they do not ‘improve’
Need to promote consistent use amongst practitioners
Need to consider the appropriateness of each scale to the client group. Some client groups are likely
to achieve significant progress on outcomes within a short time but others may be very different
Need to consider that outcomes are not just about measuring change! They can also be about
avoiding or slowing deterioration while retaining some quality of life
Recommendations
•
Clarify the purpose of different scale measures
•
Guidance is needed on the purpose and use of scale measures
Recording
It should not be assumed that practitioners will easily to adapt to an outcomes focus, when systems
have focused on processes, deficits and tasks for so long
There was broad agreement that there is no such thing as the perfect tool. However, there were
different views about whether a few key tools should be selected and promoted to all agencies, or
whether it was preferable to develop tools in-house, albeit through adapting existing models
Part of the challenge for the public is that many people don’t know where to start when asked about
what is important to them. Low expectations influence both identification of outcomes and
measurement: ‘Do people know what they want? Is it limited to what they know?’
Recommendations
3
•
A glossary of terms would help to develop a common language
•
Upskill staff in outcomes focused conversations, including through the modelling of
outcomes focused conversations in supervision, and continued sharing of practice examples
Attribution/contribution
Need to move from the futile attempt to establish clear cause and effect with outcomes, or
attribution, to understanding how different individuals and agencies contribute towards outcomes
The concept of contribution is more person–centred and enabling, and less paternalistic, because it
allows consideration of the person’s contribution towards their outcomes, not just services
Need to ensure that everyone is working towards the same outcomes, while developing a better
understanding of the role that different parties play, which allows services to articulate their role
More honesty and openness is needed about limits and possibilities of what can be achieved in
contributing towards outcomes, instead of all agencies trying to claim they can do it all
Recommendations
•
Articulate and promote a better understanding of contribution rather than attribution,
particularly in the context of greater integration
•
Focus on outcomes to support a shift from output focused targets
Qualitative and quantitative data
There is a need for confidence building with qualitative data
Need to triangulate outcomes data from different sources
Need to explore how Talking Points fits with other outcomes tools
Efforts should be made to feed outcomes information back into the organisation – represented for
practitioners in a clear and understandable way
Recommendations
•
Need to make a business case for a co-ordinated approach to triangulating data, through
gathering existing evidence
•
Need pathfinders based working collaboratively towards a national framework
Use of information
Need to investigate what is working and what is not with outcomes, while paying attention to the
specific context within which the data are collected
Develop an understanding of what works for whom in what circumstances
Need to look at outcomes longitudinally including development of pathways
4
It needs to be more about learning and development than performance
Conversations about outcomes need to be taking place all the way up the tree
Information needs to be taken to planning groups and the role of commissioning is critical
Survey data is inconsistent with outcomes data from conversations and any use of surveys in the mix
needs to take this into account
Recommendations
•
There should be an overarching framework under which outcomes are recorded and
measured – like SHANARI/GIRFEC in children’s services
•
Make better use of existing networks to address the identified issues, such as the CCBN and
IRISS, with information shared across partnerships and providers
Evaluation
After the event 14 participants provided feedback via survey monkey. The findings from these
respondents were overwhelmingly positive with everyone reporting that attending the event had led
to new insights, that they had developed connections and networks helpful to their work and that
they will use the learning in their work. Analysis of the open ended responses highlighted strong
support for holding a similar event in the future and the importance of hearing from a range of
perspectives, including research. Respondents identified a range of ways in which they were taking
this work forward and there was strong consensus as to the need for a co-ordinated response to the
issues discussed at the data retreat going forward.
Next steps
The data retreat brought together a range of individuals who have been working on the outcomes
agenda for some time. While the event involved revisiting many of the challenges involved , there
was also a sense of progress and a shared sense of direction, as one group recorded on their
flipchart: ‘The level of consensus in the room is very encouraging.’
There are a number of processes in place to build on this momentum and to take forward the
recommendations from the retreat.

The Community Care Outcomes Community of Practice is already established and will be
used to promote ongoing sharing of the learning from participants at the data retreat and
more widely. Participants are encouraged to join the community of practice (if they are not
already members) and participate in the discussions on this forum. The community of
practice is accessed through the Local Government Association Knowledge Hub
https://knowledgehub.local.gov.uk/web/khub . If you are not already a member you can
register from this web link and then use the search function to find the Community Care
Outcomes Group.
5

To take forward some of the more complex aspects of this agenda, Ailsa Cook and Emma
Miller are applying for funding from the Economic and Social Research Council for a
knowledge exchange project. The project will involve close working with people in
partnerships and provider organisations to explore the extent to which research approaches
and techniques can be used in practice settings to support this work. If you are keen to get
involved in this work or would like more information please contact Ailsa or Emma,
([email protected], [email protected] ).

IRISS are embarking on a specific project to support the use of outcomes information. This
project will include a particular focus on recording and the use of data visualisation tools. For
more information please contact Ellen Daly ([email protected]).

The Scottish Community Care Benchmarking Network will provide a co-ordinating function,
to ensure that developments in this area made by a wide range of stakeholders are joined
up. They are in the process of developing a project plan and will share this more widely in
early May. For more information or if you are keen to feed into this process please contact
Gordon Smith ([email protected]).

Penumbra are leading a small working group to look at issues relating to the use of scale
measures in more detail. For more information on this contact Bridey Monger
([email protected]).
6
Delegates
First name
Kimberley
Nick
Karen
Nicola
Kevin
Chris
Patricia
Jane
Ailsa
Eleanor
Ellen
Fidelma
Richard
Hamish
Richard
Julie
Barbara
Trisha
Nigel
Sara
Rikke
Kathleen
Aileen
Anne-Sofie
Margaret
Anne-Marie
Sandra
Stuart
Ailsa
Hugh
Ann
Rosie
Emma
Eileen
Bridey
Philip
Greg
Victoria
Ian
Robin
Susan
Marcia
Pam
Winona
Gordon
Andrew
Alison
Surname
Alexander
Andrews
Barrie
Beech
Beveridge
Bruce
Clapham
Cluness
Cook
Cunningham
Daly
Eggo
Fowles
Fraser
Freeman
Gardner
Graham
Hall
Henderson
Hitchin
Iversholt
Kennedy
Kenny
Laegran
Laird
MacDonald
Mackay
Mackenzie
McAllister
McCann
McGuigan
McLoughlin
Miller
Moir
Monger
Morgan-Klein
Muldoon
Murray
Nicol
Paterson
Polding-Clyde
Ramsay
Ramsay
Samet
Smith
Thompson
Upton
Organisation
Scottish Community Care Benchmarking Network
City & County of Swansea
Healthcare Improvement Scotland
Health Intelligence, NHS Grampian
East Renfrewshire Council
Joint Improvement Team
East Renfrewshire Community Health & Care Partnership
Shetland Islands Council
University of Edinburgh
Dept of Health and Social Care, City of Edinburgh Council
IRISS
Social Work Services, Dumfries and Galloway Council
Care Inspectorate
Midlothian Council
Politics and International Relations
VOCAL
NHS National Services Scotland
Aberlour Child Care Trust
Penumbra
Communities & Wellbeing, Midlothian Council
IRISS
Planning and Performance Team, Inverclyde CHCP
Community Nursing, Edinburgh Community Health Partnership
CHSS Knowledge Exchange Office, UoE
Highland Council
Alzheimer Scotland
North Lanarkshire Housing & Social Work Services
Care Inspectorate (Social Care and Social Work Improvement Scotland)
Social Work Dept, Dundee City Council
Joint Improvement Team Action Group
The Richmond Fellowship Scotland
VOCAL
University of Strathclyde
Joint Improvement Team Action Group
Penumbra
Social Work Performance & Information, Falkirk Council
Aberdeenshire Council
Turning Point Scotland
Social Work Services, Glasgow City Council
Moray Council
Care Inspectorate (Social Care and Social Work Improvement Scotland)
Care Inspectorate (Social Care and Social Work Improvement Scotland)
SHSS/Nursing Studies, UoE
Joint Improvement Team Action Group
Scottish Community Care Benchmarking Network
School of Social and Political Science
Stirling Council
7