Download Click here to - Shift Consulting Group Pty Ltd

IPSP Outcomes Reporting Framework
What you need to know and what you need to do
Measuring and monitoring outcomes: Why are
we doing this?
Measuring and monitoring outcomes enables us to identify:
• If/how the IPSP leads to changes in thinking and practice, and
the extent of this change
• If/how program elements work together to produce outcomes
• The types of support and professional development that
produce the best outcomes
• Opportunities for a nationally consistent approach, while
retaining flexibility to be regionally responsive
• Most importantly, opportunities for the continuous
improvement of the program.
Elements of the Framework
The IPSP Service Process Logic (ie: the “program logic”)
• The service process logic is drawn from IPSP guidelines and
summarises how IPSP is theorised to work in terms of:
• Inputs (forms of investment in IPSP)
• Processes (accountability and compliance; internal IPSP
management; IPSP planning and delivery)
• Outputs (IPSP services delivered)
• Impacts (short-medium term changes)
• Outcomes (longer-term changes)
NOTE: the logic also notes external variables that influence the
achievement of outcomes
Elements of the Framework
The IPSP Service Process Logic (ie: the “program logic”)
• At an overarching level, understanding the service process
logic allows us to ask questions about IPSP in terms of:
• the implementation of IPSP (with a focus on continuous
improvement in service delivery)
• the logic of IPSP (with a focus on continuous improvement
in the design of IPSP in the future)
Elements of the Framework
The IPSP Performance Indicator Framework
• The IPSP Performance Indicator Framework outlines the
indicators of program equity, effectiveness and efficiency
• Indicators include:
• Output indicators (indicators of accessibility, appropriateness
and quality of the services delivered)
• Impact indicators
• Outcome indicators
NOTE: Not all indicators can be easily measured
Elements of the Framework
Data Collection Instruments (across all IPSP elements – PSC,
ISA, IPSU)
Layer
Data Collection Instrument
Timeline
Planning
Needs Analysis Survey (All program elements)
April-May 2014
Needs Analysis Consultation (All program elements)
Undertaken by IPSP providers
Reflective Conversations (All program elements)
On-going July 2014 – June 2016
Initial Feedback Survey: Inclusion Support (ISA)
On-going July 2014 – June 2016
Initial Feedback Survey: Professional Development (PSC & IPSU)
On-going July 2014 – June 2016
Follow-up Feedback Survey (PSC & ISA)
On-going July 2014 – June 2016
Annual Outcomes Focus Groups (All program elements)
March 2014 & 2015
Annual Outcomes Survey (All program elements)
September 2014 & 2015
Formative
Feedback
Summative
Reporting
IPSU Data collection
1. Initial feedback survey
2. Reflective Conversations
Approach
Our focus here is to approach this in ways that:
• are culturally appropriate (recognising that
surveys are a mainstream cultural
instrument)
• are practical and do not detract from service
delivery capacity
• assist with building capacity for engaging
with the National Quality Frameworks
Your Outcomes Reporting Toolkit
Shift Consulting has developed a toolkit for
providers in consultation with the IPSU Outcomes
working group which includes:
• Guidelines for implementation
• This PowerPoint
• Information for educators fact sheet
• Reflective conversations resource for educators
• Hard copy survey
• Reflective conversation data sheets
IPSU Feedback Surveys
• Initial feedback is collected through a short
survey.
• The survey can be administered:
• On-line
• Off-line (eg: via an IPSU ipad)
• Paper-based (and answers entered by
the IPSU into the survey platform)
IPSU feedback survey
Is to be undertaken as follows:
• With services on your first visit after July 2014
• With Coordinators/educators when they are new
Links to surveys
Online version
https://www.surveygizmo.com/s3/1700022/IPSUFeedback
Offline version
http://www.surveygizmo.com/s3/offline/index.php?__slug=www.surveygi
zmo.com/s3/1700022/IPSUFeedback&sLanguage=Auto
How to get the survey data into our
system: Online mode
Survey undertaken by educators in online mode
• They can do it alone of with your assistance
• Requires internet connection
They are
given a link
by you
They go
online and
fill out the
survey
Their
completed
survey is in
the system
How to get the survey data into our
system: Offline mode
Survey undertaken by educators with your assistance in
offline mode
• They can do it alone with your assistance
• Does not require internet connection but does require laptop,
ipad or smartphone
They are
given a link
by you or
you sit
together
They follow
the link and
fill out the
survey
You later
upload the
survey into
the system
How to get the survey data into our
system: hard copy
Survey undertaken by educators by filling out a hard copy
form
• They can do it alone
• Is handed to you at the conclusion of the support visit
• Later, you enter the survey into the system (offline mode
quickest)
You provide
hard copy
survey
They fill out
the survey
and hand it
back to you
You later
enter the
survey into
the system
Why Reflective Conversations?
The data gathered through the ‘reflective
conversation’ instrument provides us (IPSP) with a
means to build knowledge and understanding of the
process of change in the IPSP context.
Obviously, in the course of facilitating a
conversation, “change” becomes contextualized with
reference to the specific service environment,
children, families and educators.
Reflective Conversations
A reflective conversation is
a guided conversation that
encourages educators to:
Implement
plans &
strategies
Plan for
positive
change
Reflect on
change
Identify
barriers
Identify
strategies
Reflective Conversations Documenting
reflections (targeted at educators)
• The key to conducting a good reflective conversation is
active listening and responsively guiding the
conversation in ways that encourage critical reflection,
problem solving and planning.
• Documenting key reflections and ideas along the way
provides the facilitator and the educator(s) with a
framework for drawing links between ideas and critically
reflecting on these over time.
• Documenting the conversation is modelling the same
process as an educator might use to document
children’s learning
Reflective Conversations
Where educators begin
• How a conversation about change occurs and the depth
of reflection that will occur is influenced by where
educators begin.
• An educator’s “level of reflection” will influence how the
reflective conversation will take place.
• The role of a professional facilitating a reflective
conversation is to assist the educator to reflect more
deeply and grow their reflection and understanding
Bain’s Critical Reflection Framework
Level 3:
RECONSTRUCTING
Level 2:
ELABORATING
Level 1: REACTING
•Focuses on own feelings
•Offers straight forward
description
•Seeks basic
understanding
•Compares self with
others
•Considers alternatives
•Seeks deeper
understanding
•Draws new conclusions
about practice
•Explores relationships
between theory and
practice
•Drives own learning
(Miller, 2011, p. 6)
Reflective Conversations
Levels of Reflection
Level 1: REACTING
• Focuses on own feelings: Commenting on feelings
related to an event
• Offers straight forward description: Providing a
straight description with no added observation or
insights
• Seeks basic understanding of what occurred and why
Reflective Conversations
Levels of Reflection
Level 2: ELABORATING
• Compares self with others: Comparing one’s reaction to
other people’s responses and perspectives
• Considers alternatives: Analysing an event by asking
questions and considering alternatives
• Seeks deeper understanding of an event by relating it to
current literature and theories
Reflective Conversations
Levels of Reflection
Level 3: RECONSTRUCTING
• Draws new conclusions about practice: Drawing revised
and new conclusions about own practices and the practices
of others
• Explores relationships between theory and practice:
Ongoing exploration of relationships between practice,
literature and theory
• Drives own learning: Planning further learning on the
basis of your reflections.
Reflective Conversations
A frame of reference for facilitators
• Conversations flow… sometimes people offer up information
without prompting… at other times they need significant
support to draw out the details.
• There is a set of questions that can provide a facilitator with a
frame of reference to support a holistic conversation.
• The key is that the facilitator must keep an ear out for the what
information has been offered up and where they need to ask
questions and encourage further reflection.
• The documentation tools provided can assist with this
process.
Reflective Conversations
Reference Questions – Initial Visit (from July
2104)
• What are the most significant learnings you have
drawn from today?
• Are there any ways in which you have changed the
way you think about [topic]?
• How do you think you will continue to use what you
have learned from today?
Reflective Conversations
Reference Questions – Follow-up visit
What is the most significant change that has taken place as a
result of receiving this support? Think about (encourage reflection
on)
• The way you think about your role/approach/practice?
• Your relationships with others such as your team members,
children and families?
• The practices that you use in relation to working with team
members, children and families?
• Have you experienced any barriers to implementing change in
your practice?
• How have or could these be overcome?
• How do you think you will continue to use what you have
learned in the future?
How to get the Reflective
conversations data into our system:
Utilise existing methods of engaging in reflective
conversations with educators. This is NOT a structured
interview and the questions are there to guide your
conversation.
• Document the conversation in terms of the reflective questions
and themes outlined in the datasheet
Gather data
using existing
reflective
conversations
strategies
Transfer
information to
reflective
conversations
data sheet
Forward to
Shift
Consulting as
required (3
monthly)
Got any questions?
Please forward any questions through the IPSU IPSP Outcomes
working group – members are Chris, Kate and Peter.