Download Presentation

ATD 2016 International
Conference & Exposition
The Premier Conference for TD Professionals
Session SU404
Moving Healthcare Talent
Development Solutions to a
Pay-for-Value Model
Timothy R. Brock, PhD
The Institute 4 Worth Performance
Learning Objectives
• CONDUCT a needs assessment to establish and align the “current
state” quantitative and qualitative metrics of stakeholders at
different five levels required to implement a successful pay-for-value
talent development project, program, or performance improvement
initiative.
• WRITE results-focused objectives with the key stakeholders that
establish the “required state” measures to close the pay-for-value
gaps at all five of these levels.
• MEASURE the success of achieving each objective to make the
necessary process improvement adjustments required to meet the
driving stakeholder pay-for-value needs.
Do you agree with this statement?
There is a greater demand
within your organization
to show value for investments
in all talent development
programs and projects.
What is value?
The value imperative
Imperative
to show the
value
of
expenditures
in terms that
resonate
with all
stakeholders.
The new healthcare value imperative
All healthcare systems
are working
toward building
internal systems
that not only
deliver better quality
but also prove them.
Shifting paradigms
Activity Based
Pay-For-Service
No business need for the program
Results Based
Pay-For-Value
Program linked to specific business needs
No assessment of performance issues
Assessment of performance effectiveness
No specific measurable objectives
Specific objectives for behavior and
business impact
No effort to prepare program participants
to achieve results
Results expectations communicated to
participants
No effort to prepare the work environment Environment prepared to support transfer
to support transfer
No efforts to build partnerships with key
managers
Partnerships established with key
managers and clients
No measurement of results or benefit-cost Measurement of results and benefit-cost
analysis
analysis
Planning and reporting is input focused
Planning and reporting is outcome focused
Executive View of Value
Measure
Current
Measure
Should
Measure
Measure
Importance
1. Inputs
Last year, 78,000 employees received
formal learning.
94%
85%
6
2. Efficiency
Formal learning costs $2.15 per hour of
learning consumed.
78%
82%
7
3. Reaction
Employees rated our training very high,
averaging 4.2 out of 5.
53%
22%
8
32%
28%
5
11%
61%
4
8%
96%
1
4%
74%
2
40%
44%
3
4. Learning
5. Application
6. Impact
7. ROI
8. Awards
92% of participants increased
knowledge and skills
At least 78% of employees are using the
skills on the job.
Our programs are driving our top 5
business measures in the organization.
Five ROI studies were conducted on
major programs yielding an average of
68% ROI.
Our learning and development program
won an award from the ATD.
Hospital now use ROI to define value
“…hospitals typically measure
ROI
from a business perspective—
cost, revenues or operating
efficiencies—
but many benefits of clinical
applications fall into
quality and safety realms
that do not easily translate into
dollars.”
Page, D. (2010). IT’s return on investments is tricky to pin down. Hospital & Health Networks. Retrieved from http://www.medsphere.com/news/medspherein-the-news/681-its-return-on-investment-is-tricky-to-pin-down
Pay-for-Value has patient care ROI implications
"We should
measure
clinical ROI
in terms of
measurable impact
on patient care.”
Typical reaction to ROI
The ROI calculation
BCR =
Program Benefits
Program Costs
(Program Benefits – Program Costs)
ROI =
Net Program Benefits
Program Costs
X 100
Try it!
BCR =
ROI =
$500,000
$200,000
= 2.50:1
$500,000 - $200,000
$200,000
= 1.5 X 100 = 150%
A. BCR = 2.50; ROI = 250%
B. BCR = 1.50; ROI = 150%
C. BCR = 2.50; ROI = 150%
D. BCR = 3.50; ROI = 250%
It’s that easy!
But Where Do
The Numbers
Come From?
Value + ROI = Measurable Impact
To create ROI,
we must
develop a
“value measuring
methodology.”
Best Practice Committee. (2002). The Value of IT investments: It’s not just return on investments.
Retrieved from http://www.cio.gov/documents/thevalueof_it_investments.pdf
Comprehensive, Credible Evaluation System
Source: Phillips, P. P., The Bottomline on ROI. 2nd Edition, HRDQ Press, 2011.
Clinical Evaluation Framework
A comprehensive measurement and evaluation
process that generates six types of measures:
1. Reaction and Planned Action
Perceptions
2. Learning
KSAs
3. Application
Job Behaviors
4. Business Impact
Results
5. Return on Investment
Calculation
6. Intangible Measures
Can’t $$$
This balanced approach to measurement includes a
technique to isolate the effect of the program or solution.
Source: Measuring ROI in Healthcare: Tools and Techniques to Measure the Impact and ROI in Healthcare Improvement Projects and Programs, 2013, V. V. Buzachero, J.
Phillips, P. P. Phillips, and Z. L. Phillips.
The Value Isolation Issue
The business case would be
easier to
assess
if interventions were
implemented with
strong evaluation designs
that could
isolate
intervention effects.
Source: Lurie, N., S. A. Sommers, et al. (2008). "Challenges to Using A Business Case For Addressing Health Disparities." Health Affairs 27(2): 334-338.
The New Pay-for-Value Imperative
The ability to
connect the dots
and show
bottom line accountability
while maintaining
quality
and
efficiencies
will be the key to survival
in the healthcare industry.
Inputs
Reaction & Planned Action
Learning
Application & Implementation
Isolate the Effects of the Program
Impact
ROI
Intangible Benefits
Source: Measuring ROI in Healthcare: Tools and Techniques to Measure the Impact and ROI in Healthcare Improvement Projects and Programs, 2013, V. V. Buzachero, J.
Phillips, P. P. Phillips, and Z. L. Phillips.
A comprehensive unit-based safety program
Issue
Reduce central line blood infections in intensive care units occurring at a
group of hospitals located at Birmingham, AL metro area
Program
Implement a new set of procedures to reduce the number of infections
• Involved a checklist system that set up specific steps
• Used by doctors, nurses, and technicians
Organizational Change Opportunity
Culture change required
ROI Opportunity
Needed various levels of data beyond the traditional monitoring of
infections, length of stay, and costs associated with these infections.
Source: Measuring ROI in Healthcare: Tools and Techniques to Measure the Impact and ROI in Healthcare Improvement Projects and Programs, 2013, V. V.
Buzachero, J. Phillips, P. P. Phillips, and Z. L. Phillips.
10-Step ROI Methodology (ROI Institute)
Capture
Costs of the
Program
Data Collection
Level 4
Business Impact
Level 3
Application &
Implementation
Level 2
Learning & Confidence
Level 1
Reaction &
Planned Action
Develop/
Review
Objectives
of Program
Collect Data
After Program
Implementation
Collect Data
During Program
Implementation
Develop
Evaluation Plans
and Baseline
Data
Data Analysis
Isolate the
Effects of
the Program
Convert
Data to
Monetary
Value
Reporting
Calculate the
Return on
Investment
Generate
Impact
Study &
Report
Results
Level 5
ROI
Identify
Intangibles
(Potential
Benefits)
Evaluation Planning
Source: Measuring ROI in Healthcare: Tools and Techniques to Measure the Impact and ROI in Healthcare Improvement Projects and Programs, 2013, V. V. Buzachero, J.
Phillips, P. P. Phillips, and Z. L. Phillips.
Pay-for-Value Alignment Model
Objectives
Needs
Evaluation
Start Here
End Here
Payoff Needs 5
Business Needs 4
Performance Needs 3
Initial Analysis
Learning Needs 2
Preference Needs 1
Business Alignment and
Forecasting
ROI
Objective
5 ROI Results
Business
Objective
4 Business Results
Performance
Objective
3 Performance Results
Evaluation
Learning
Objective
2 Learning Results
Reaction
Objective
Input
Input Needs 0 Objective
1 Reaction Results
0 Input Results
ICU Central Line
Infection Reduction
Program
The ROI Process Methodology
Key Pay-for-Value Questions
Payoff
Is this problem worth solving?
Impact
What are the specific measures?
Application
What is occurring or not occurring on the
job that influences the Impact measure?
Learning
What knowledge and skills are required
to support the job performance need?
Reaction
How should the program be structured
and perceived?
Input
Who are the participants?
Pay-for-Value Alignment Model
Objectives
Needs
Evaluation
Start Here
End Here
Payoff Needs 5
Business Needs 4
Performance Needs 3
Initial Analysis
Learning Needs 2
Preference Needs 1
Business Alignment and
Forecasting
ROI
Objective
5 ROI Results
Business
Objective
4 Business Results
Performance
Objective
3 Performance Results
Evaluation
Learning
Objective
2 Learning Results
Reaction
Objective
Input
Input Needs 0 Objective
1 Reaction Results
0 Input Results
ICU Central Line
Infection Reduction
Program
The ROI Process Methodology
Pay-for-Value Chain of Impact - Needs
Needs
Payoff
Business
Pay-for-Value Metrics
 Patient Safety
 Quality Care




Number of central line bloodstream infections
Mortality rate
Days in hospital
ICU Costs
Performance
 Inefficient procedures, inconsistent compliance
 Lack of accountability
Learning
 Consistent use of new procedures to standard
 Peer communication accountability
Preference
 Use physical simulation to refine, validate, and practice new procedure
 Value program as:
 Necessary
 Important
 Feasible
 Practical
Input
 All doctors, nurses, and technicians (participants) in the intensive care
units are involved
Pay-for-Value Alignment Model
Objectives
Needs
Evaluation
Start Here
End Here
Payoff Needs 5
Business Needs 4
Performance Needs 3
Initial Analysis
Learning Needs 2
Preference Needs 1
Business Alignment and
Forecasting
ROI
Objective
5 ROI Results
Business
Objective
4 Business Results
Performance
Objective
3 Performance Results
Evaluation
Learning
Objective
2 Learning Results
Reaction
Objective
Input
Input Needs 0 Objective
1 Reaction Results
0 Input Results
ICU Central Line
Infection Reduction
Program
The ROI Process Methodology
Matching Objectives with Levels – An Exercise
Identify the
Pay-for-Value Chain of Impact
Need Level
of an Objective
1
Reaction
2
Learning
3
Application
4
Impact
5
ROI
Question 1
Apply the new procedures using the
checklist during a discreet event
simulation with 100% accuracy.
1
Reaction
2
Learning
3
Application
4
Impact
5
ROI
Question 2
Achieve a 4 out of 5 rating on the
appropriateness of the new speak up
policy to ensure checklist compliance.
1
Reaction
2
Learning
3
Application
4
Impact
5
ROI
Question 3
Track daily use of checklist and speak
up events to monitor application
trends and integration.
1
Reaction
2
Learning
3
Application
4
Impact
5
ROI
Question 4
Reduce central line bloodstream
infections by 50% in six months.
1
Reaction
2
Learning
3
Application
4
Impact
5
ROI
Question 5
Achieve a 30% ROI from ICU central
line infection reduction program within
12 months after implementation.
1
Reaction
2
Learning
3
Application
4
Impact
5
ROI
Question 6
Reduce ICU Costs by 3%.
1
Reaction
2
Learning
3
Application
4
Impact
5
ROI
Pay-for-Value Chain of Impact - Objectives
Type
ROI
Impact
Application
Learning
Preference
Input
Needs-Driven Objectives
 ROI Objective is 25%




Central line bloodstream infections will be reduced by 50% in six months
Mortality rate reduced by 5%
Days in hospital reduced by 2%
ICU Costs reduced by 3%
 Checklist will be monitored
 The use of new procedures will be observed
 Extent of “speak up” conversations will be collected
 All participants must demonstrate knowledge of the checklist and new
procedures
 Participants must practice “speak up” conversations with colleagues and visitors
 All participants must perceive the value of the program as:
 Necessary
 Important
 Feasible
 Practical
 Involve all doctors, nurses, and technicians (participants) providing patient care
at the intensive care units
Setting the ROI Objective
• Set the value with other investments
15%
• Set the value slightly above other investments
25%
• Set at break even
0%
• Set at client expectations
30%
Pay-for-Value Alignment Model
Objectives
Needs
Evaluation
Start Here
End Here
Payoff Needs 5
Business Needs 4
Performance Needs 3
Initial Analysis
Learning Needs 2
Preference Needs 1
Business Alignment and
Forecasting
ROI
Objective
5 ROI Results
Business
Objective
4 Business Results
Performance
Objective
3 Performance Results
Evaluation
Learning
Objective
2 Learning Results
Reaction
Objective
Input
Input Needs 0 Objective
1 Reaction Results
0 Input Results
ICU Central Line
Infection Reduction
Program
The ROI Process Methodology
Pay-for-Value Chain of Impact – Evaluation
Type
Needs-Driven Evaluation
ROI
 Calculate ROI
Impact
Monitor for 6 months:
 Number of central line bloodstream infections
 Mortality rate
 Days in hospital
 ICU Costs
Application
 3-month questionnaire to check frequency use
 Supervisors monitor work
Learning
 Practice clinical and conversation skills to standard with feedback during
physical simulation
Reaction
 Reaction questionnaire at the end of physical simulation refinement,
validation, & practice
 Commit to implementation Action Plan
Input
 All doctors, nurses, and technicians (participants) in the intensive care
units involved
Pay-for-Value Alignment Model
Start Here
Objectives
Needs
Evaluation
End Here
Initial Analysis
Payoff Needs 5
5 ROI Results
Quality Care
Patient Safety
Calculate ROI
ROI of 25%
Business Needs 4
Number of Infections
Mortality Rate
Days in Hospital
ICU Costs Performance Needs
3
Inefficient, inconsistent, procedures
Lack of accountability
Demonstrate competence
 Applying checklist procedures
 Using “speak up” technique
Preference Needs 1
 Favorable rating (4 out of 5)
on program’s need,
relevance, & practicality
 Commitment to Action Plan
Use physical simulation to refine, validate, and
practice
Value program as necessary, important, feasible,
Input Needs
and practical
Business Alignment and
Forecasting
Monitor for 6 months:
 Number of Infections
 Mortality Rate
 Days in Hospital
 ICU Costs
 Monitor checklist compliance
 Observe procedure in practice
 Collect frequency of “speak up” conversation
Learning Needs 2
Consistent use of procedures to standard
Peer communication accountability
4 Business Results
 Reduce central line infections by 50%
 Reduce mortality rate by 5%
 Reduce hospital days by 2%
 Reduce ICU costs by 3%
0
3 Performance Results
3-month questionnaire to check
frequency use
Supervisors monitor work
2 Learning Results
Practice clinical and conversation
skills to standard with feedback
during physical simulation
1 Reaction Results
Reaction questionnaire at the end of
physical simulation validate & practice
Commit to implementation Action Plan
0 Input Results
ICU Central Line
Infection Reduction
Program
The ROI Process Methodology
Evaluation
Pay-for-Value Alignment Model
Objectives
Need
Payoff
Business
Performance
Learning
Preferences
Evaluation
 Quality Care
 Patient Safety
 ROI Objective is 25%
 Calculate ROI/BCR




 Central line bloodstream infections will
be reduced by 50% in six months
 Mortality rated reduced by 5%
 Days in hospital reduced by 2%
 ICU Costs reduced by 3%
Monitor for 6 months:
 Number of Infections
 Mortality Rate
 Days in Hospital
 ICU Costs
 Insufficient,
inconsistent
procedures
 Lack of
accountability
 Checklist will be monitored
 The use of new procedures will be
observed
 Extent of “speak up” conversations will
be collected
 3-month questionnaire
to check frequency use
 Supervisors monitor
work
 Consistent use of
procedures to
standard
 Peer communication
accountability
 All participants must demonstrate
knowledge of the checklist and new
procedures
 Participants must practice “speak up”
conversations with colleagues and
visitors
 Practice clinical and
conversation skills to
standard with feedback
during physical
simulation
 Use physical
simulation to refine,
validate, and practice
 Value program as
necessary,
important, feasible,
and practical
 All participants must perceive the value
of the program as:
 Necessary
 Important
 Feasible
 Practical
 Reaction questionnaire
at the end of physical
simulation validate &
practice
 Commit to
implementation Action
Plan
Number of Infections
Mortality Rate
Days in Hospital
ICU Costs
ICU Central Line Infection Reduction Program
ROI
Impact
Application
Learning
Reaction
Level 4 & 5 Selection Criteria
Benchmarking*
Cost of the program
52%
Importance of program to strategic objectives
50%
Executive interest in the evaluation
48%
Visibility of the program
45%
Linkage of program to operational goals and issues
29%
Life cycle of the program
14%
Investment of time required
7%
Size of target audience
6%
*Survey of Users, N = 235
Source: Phillips, P. P., The Bottomline on ROI. 2nd Edition, HRDQ Press, 2011.
Recommended Targets
5%
Level 5 - ROI
Level 4 – Business Impact
10%
Level 3 - Application
30%
Level 2 - Learning
60%
Level 1 - Reaction
100%
Source: Phillips, P. P., The Bottomline on ROI. 2nd Edition, HRDQ Press, 2011.
Healthcare System Example
Program (ERACS Pathway)
Enhanced Recovery After Colorectal Surgery
Business Impact
Reduced length of patient hospital stay
Reduced complications
Improved patient comfort
Increased provider engagement (improved
retention)
Attracted more providers wanting to adopt
solution
Enhanced image—seen as innovative
Healthcare System Example
Program
Sexual Harassment Prevention Workshop
Business Impact
Turnover Reduction
Complaint Reduction
Job Satisfaction
Absenteeism
Stress Reduction
Recruiting
Sprint/Nextel Example – Diversity Program
Program
All-Inclusive Workforce Program (AIW)
Business Impact
Attrition Rate
Employee Satisfaction
Communication
Cooperation & Teamwork
Diversity Mix
Verizon – Complaint Resolution
Program
Customer Service Training
Business Impact (Target)
Customer Complaint Escalation
Process Improvement Tool – Fund, Fix, or Fold
• Just because a program has a negative ROI does not mean
the program is discontinued
• Must understand what happened so we can make changes to
make it positive in the future to drive business value
• ROI Institute global benchmarking studies indicate that 20%30% of programs have a negative ROI with 10% of those
being discontinued
• It is rare when ROI is the only factor
• Intangibles can offset a negative ROI
•
Ownership, engagement, commitment, communication, teamwork,
cooperation, etc.
• If it can’t produce any business value, then we have a datadriven case to let it die in peace
The Pay-for-Value Payoff
 Align projects to healthcare organization needs
 Show contributions of selected projects
 Earn respect of senior management/administrators
 Build staff morale
 Justify/defend budgets
 Improve support for projects
 Enhance design and implementation processes
 Identify inefficient projects that need to be redesigned
or eliminated
 Identify successful projects that can be implemented in
other areas
 Earn a “seat at the table”
Source: Measuring ROI in Healthcare: Tools and Techniques to Measure the Impact and
ROI in Healthcare Improvement Projects and Programs, 2013, V. V. Buzachero, J. Phillips,
P. P. Phillips, and Z. L. Phillips.
It’s that easy!
References
• Best Practice Committee. (2002). The Value of IT Investments: It’s Not
Just Return on Investments. Retrieved from
http://www.cio.gov/documents/thevalueof_it_investments.pdf
• Buzachero, V.V., Phillips, J., Phillips, P. P., Phillips, Z. L., Measuring
ROI in Healthcare: Tools and Techniques to Measure the Impact and
ROI in Healthcare Improvement Projects and Programs, 2013.
• Lurie, N., S. A. Sommers, et al. (2008). "Challenges to Using A
Business Case For Addressing Health Disparities." Health Affairs 27(2):
334-338.
• Page, D. (2010). IT’s Return on Investments is Tricky to Pin Down.
Hospital & Health Networks. Retrieved from
http://www.medsphere.com/news/medsphere-in-the-news/681-itsreturn-on-investment-is-tricky-to-pin-down
Your feedback helps ATD continue to provide top-notch educational
programs that help you stay on top of a changing profession.
Evaluations forms for this session are available via the mobile app
and at the following link: http://www.atdconference.org/attendees.