Download DTES-SGS-Evaluation-Framework-DRAFT-v11

DTES Second Generation Strategy
Evaluation Framework
DRAFT – NOT FOR DISTRIBUTION
Purpose of the evaluation framework
– Give an overview of the strategy at a level that can
be measured
– System performance measurement
– Quality improvement
– Demonstrates logical link:
Inputs
Activities
Outputs
Outcomes
– Identify gaps in measurement
– Prioritize activities that will most likely get us to
our ultimate goal
Current state of health in DTES
– Poor health outcomes for DTES population
– High volumes of ED visits and hospitalizations of
at-risk people from DTES
– Care is fragmented, uncoordinated, inaccessible
for some
– Housing is unstable
– Access to nutritious food is challenging
Goal of DTES SGS
• Improving health outcomes for the population
in the DTES
DTES Second Generation Strategy Theory of Change
Long term outcomes
Ultimate goal
Intermediate outcomes
Indirect result of outputs
Immediate outcomes
Outputs
Direct results of outputs
The type, volume, and quality of products and services
Activities
The work we are doing to produce the outputs
Inputs
The resources required to undertake the activities
Inputs
Activities
Outputs
Clientreported
quality of life
Overall health
improvement
Intermediate
Immediate
Outcomes
Long Term
DTES Second Generation Strategy Theory of Change
Health issues/risk identified and
addressed before complications occur
At-Risk people
engaged into care
Unbroken attachment to
care and treatment retention
Patient experience of care:
• Trauma-informed
• Culturally competent
• Harm reduction
• Recovery-orientation
Strengthen
relationships and
partnerships
Low Threshold
Addiction Clinic
•
•
Social Determinants of Health
People get access to nutrition
People are appropriately housed
Coordination
Access &
Accessibility
New models of care:
• Low barrier addiction care model
• MH substance use drop-in model
• Integrated care model
New clinic sites
Integrated care
Strategies:
• Food & nutrition strategy
• Housing strategy
• Peers strategy
Nutritious food
services
Appropriate
housing services
Capacity building:
• Trauma-informed care
• Cultural competency
• Best pain management practices
• Harm reduction
• Recovery-orientation
Integrated care teams
Housing contracts
Embedded peers
Food contracts
Clientreported
quality of life
Overall health
improvement
Intermediate
Immediate
Outcomes
Long Term
DTES Second Generation Strategy Theory of Change
Focus of
System
Performance
Measurement
Health issues/risk identified and
addressed before complications occur
At-Risk people
engaged into care
Unbroken attachment to
care and treatment retention
•
•
Social Determinants of Health
People get access to nutrition
People are appropriately housed
Inputs
Activities
Outputs
Focus of
Quality
Improvement
Patient experience of care:
• Trauma-informed
• Culturally competent
• Harm reduction
• Recovery-orientation
Strengthen
relationships and
partnerships
Low Threshold
Addiction Clinic
Coordination
Access &
Accessibility
New models of care:
• Low barrier addiction care model
• MH substance use drop-in model
• Integrated care model
New clinic sites
Integrated care
Strategies:
• Food & nutrition strategy
• Housing strategy
• Peers strategy
Nutritious food
services
Appropriate
housing services
Capacity building:
• Trauma-informed care
• Cultural competency
• Best pain management practices
• Harm reduction
• Recovery-orientation
Integrated care teams
Housing contracts
Embedded peers
Food contracts
Inputs
Activities
Outputs
Clientreported
quality of life
Overall health
improvement
Intermediate
Immediate
Outcomes
Long Term
DTES Second Generation Strategy Theory of Change
Focus of
System
Performance
Measurement
Health issues/risk identified and
addressed before complications occur
At-Risk people
engaged into care
Patient experience of care:
• Trauma-informed
• Culturally competent
• Harm reduction
• Recovery-based
Strengthen
relationships and
partnerships
Low Threshold
Addiction Clinic
People are appropriately
housed/ increased housing
tenure
People get access
to food
Unbroken attachment to
care and treatment retention
Coordination
Access &
Accessibility
New models of care:
• Mental health & addiction care
• MH substance use drop-in model
• Integrated care model
• Pain management best practices
New clinic sites
Food services
Appropriate
housing services
Integrated care
Strategies:
• Food & nutrition strategy
• Housing strategy
• Peers strategy
Training:
• Trauma-informed care
• Cultural competency
• Best pain management practices
• Harm reduction
• Recovery-based
Integrated care teams
Housing contracts
Embedded Peers
Food contracts
Performance Measurement
• Peer reviewed, validated methods,
generalizability of results
• Give validity to the innovations of the SGS
– VCH is partnering with external researchers at
UBC and SFU to conduct population-based
analysis of key health outcomes and some outputs
of the SGS
Long Term
Outcome
Measure
Indicator
definition
Data
source
Overall health
improvement
Increase in number of
clients with HONOS
score in target range
EMR HONOS,
PARIS HONOS
Clinically significant
improvement from
baseline
VCHRI project
Oct 2017 –
March 2018
Reduction in the rate
of ED admissions,
hospital admissions,
death
VCHRI project
Oct 2017 –
March 2018
Improved functional
assessment in 4 key
areas: physical health,
substance use, mental
health hospitalization;
adherence to
medications
TBD by data
reference group
Chronic disease
indicators per
guideline care
EMR – TBD by
data reference
group
Clients with a Q-LES-Q
score in target range
EMR HONOS,
PARIS HONOS TBD
Client-reported
quality of life
Reporting
format/
Time
frame
Dashboard /
Quarterly
SGS Innovation
• Integrated care teams (ICTs) and clinics, and
expanded mobile care.
• Peer navigators
• Shared treatment continuums
• Dedicated ICT for women
• Enhanced partnerships with private clinics
• Cultural competence and trauma-informed
practice
• Peers at drop-ins, tenant support workers
linking to care teams
• Connect with private clinics
• Address service gaps for women and children
• Managed alcohol
• Strategic plan for harm reduction
• Best practices for pain management
• Overdose training
• Address social determinants of health:
housing & food
• Washrooms policy
Intermediate
Outcome
Measure
Indicator
definition
Data source Reporting SGS Innovation
format /
Time
frame
Health
issues/risk
identified and
addressed
before
complications
occur
Preventable hospitalizations:
Number of in-patient acute
care hospitalizations for
conditions where
appropriate ambulatory care
may prevent or reduce the
need for admission to
hospital within VCH care.
ED and Acute data
in Decision
Support
Ambulatory care sensitive
condition acute admissions
Acute data in DS
Dashboard /
Quarterly
- CTAS 4 and 5 level ED
visits by known clients
- ED visit rate
- Hospitalization rate
- Acute length of stay
ED and Acute data
in Decision
Support
Dashboard /
Quarterly
# who stabilized on adequate
methadone
LTAC - TBD
# who stabilized on adequate
methadone/ suboxone
LTAC - TBD
# days reduced illicit opiate
use
LTAC - TBD
• Integrated care teams (ICTs) and clinics,
and expanded mobile care.
• Peer navigators
• Shared treatment continuums
• Dedicated ICT for women
• Enhanced partnerships with private
clinics
• Cultural competence and traumainformed practice
• Peers at drop-ins, tenant support
workers linking to care teams
• Connect with private clinics
• Address service gaps for women and
children
• Managed alcohol
• Strategic plan for harm reduction
• Best practices for pain management
• Overdose training
• Address social determinants of health:
housing & food
• Washrooms policy
Immediate
Outcome
Measure
Indicator definition
Data
source
Reporting
format /
Time frame
SGS Innovation
At-Risk people
engaged into
care
Counts of both total volume and
unique clients
VCHRI project
Oct 2017 – March
2018
Service engagement: No. of
individuals known to have specified
conditions I = (1,…n) who are
engaged in optimal care.
VCHRI project
Oct 2017 – March
2018
•
•
•
•
•
•
•
Peer navigation
Drop-ins
Trauma-informed practice
Cultural competence
Dedicated ICT for women
Overdose training
Washrooms policy
EMR
Dashboard /
Quarterly
•
•
•
•
Peer navigation
Integrated care teams
Mobile health services
Address gap in care for
women and children
Trauma-informed practice
Cultural competence
Dedicated ICT for women
Washrooms policy
Connect with private clinics
Service engagement: # of new
patients in program who initiated on
methadone or suboxone
Unbroken
attachment to
care and
treatment
retention
Retention / Internal attachment:
Clients who in the past 14 months
have had at least 4 visits to the
clinic/program (regular appointments
– evenly distributed) and have a Care
Plan documented, especially patients
with certain health conditions
People get
access to
nutritious food
Food access/nutrition: TBD
Service contracts that align
with SGS food strategy
Appropriate
housing
Appropriateness / stability of housing
(tenure): To be developed
• Housing models designed
for appropriateness
• Create efficient / effective
system for access to
housing
•
•
•
•
•
Clientreported
quality of life
Overall health
improvement
Intermediate
Immediate
Outcomes
Long Term
DTES Second Generation Strategy Theory of Change
Health issues/risk identified and
addressed before complications occur
At-Risk people
engaged into care
Unbroken attachment to
care and treatment retention
•
•
Social Determinants of Health
People get access to nutrition
People are appropriately housed
Inputs
Activities
Outputs
Focus of
Quality
Improvement
Patient experience of care:
• Trauma-informed
• Culturally competent
• Harm reduction
• Recovery-orientation
Strengthen
relationships and
partnerships
Low Threshold
Addiction Clinic
Coordination
Access &
Accessibility
New models of care:
• Low barrier addiction care model
• MH substance use drop-in model
• Integrated care model
New clinic sites
Integrated care
Strategies:
• Food & nutrition strategy
• Housing strategy
• Peers strategy
Nutritious food
services
Appropriate
housing services
Capacity building:
• Trauma-informed care
• Cultural competency
• Best pain management practices
• Harm reduction
• Recovery-orientation
Integrated care teams
Housing contracts
Embedded peers
Food contracts
Quality Improvement - Output
measurement
• Changes in outputs happen sooner than
outcomes
– Making sure we are on track to achieve improved
health outcomes
– Opportunity to fail fast and make course
corrections
Output
Measure:
Quality of
care
Indicator Definition
Data source
Reporting
format /
time frame
SGS innovation
Accessibility
Same day service: Clients who
were referred to a certain
service/team/program and
were engaged in
treatment/service that same
day (clinical assessment, case
management assessment,
intervention, etc.)
EMR and PARIS
Dashboard
Same day service at point
of request
Wait times to 1st contact:
Clients with an open referral
during the reporting period
and who have been contacted
by the treatment
team/program
EMR and PARIS
Dashboard
Wait times from service
request to first contact
Cultural
competence
Patient experience of
culturally competent care
Core competencies survey
Core competency training
Traumainformed care
Patient experience of traumainformed care
Core competencies survey
Core competency training
Harm reduction
Patient experience of harm
reduction care
Core competencies survey
Core competency training
Recovery-based
care
Patient experience of
recovery-based care
Core competencies survey
Core competency training
Output
Measure
Indicator definition
Data
source
Reporting
format
SGS
Innovation
Access
Client volumes: Clients with an open referral during
the reporting period
EMR and PARIS
Dashboard
Number of clients turned away
TBD
• Low Threshold
Addiction Clinic
• Integrated Care
Teams
Care
coordination
Care coordination: % of clients with a new care
plan or care plan update in last month
EMR and PARIS
- TBD
Dashboard
Care coordinator
Integration
Integration: To what extent do you think that the
array of services offered to DTES clients, at this
time, is integrated in a way that best serves client
needs?
VCHRI project
Data should be
available Jan 2017
Peer navigators,
ICTs
Integration: # peers connected to drop-ins
Contract
deliverable
Food
services
TBD
Contract
deliverable
Food contract
(Schedule A)
Housing
spaces
TBD
Contract
deliverable TBD
TBD
Gaps
• Measurement of access to nutritious food
• Measures appropriate housing
• Patient experience of attachment, engagement,
self-management of health
• Patient experience of accessibility, coordination &
integration
• Benchmarks
• Program evaluation of some work streams
• Process evaluation of the overarching project
Balanced scorecard
Area
Measures
System
Yes!
Neighbourhood
No
Staff
No
Clients
Some
Questions for the group
• Given the gaps – where would you prioritize?
– Appetite for program evaluation?
– Appetite for process evaluation?
– Setting benchmarks?
• Where do we want to be 18 months from now?
• What do we want to know 18 months from now?
– Where are our opportunities for shared
monitoring and evaluation with the community?