Download Scrap Learning—Your Programs Are Not As

Scrap Learning—Your Programs Are Not
As Good as You Think They Are
Training Industry Conference & Exposition 2015
May 7, 2015
8:30 ET
John R. Mattox, II, Ph.D.
Senior Consultant, Talent Solutions
Doing More with Less
New Learning
Environment
Business
Demands
Constrained
Resources
OTJ / JIT
Rapid deployment
Tightening budgets
Social Learning
Customize everything
Expanding responsibilities
Gamification
Minimize learners’ time
Lack of manager support
Increased Complexity in
L&D Solution Set
Employee Development Critical
for Business Outcomes
Heads of L&D Reporting Increased
Complexity of L&D Solutions
Line Leaders Identifying Employee Development
as Critical to Achieving Business Outcomes
93%
Agree or
Strongly Agree
86%
Agree or
Strongly Agree
Source: CEB 2012 L&D Team Capabilities Survey
2
© 2014 CEB. All rights reserved.
Falling Short of Expectations
CEOs Expect More Insights
CEOs Opinions Regarding Information on Return on Investment in Human Capital
Information is Important
Receives Sufficient Information
0%
50%
100%
Source: PwC 2012 Annual Global CEO Survey
CFOs Lack Confidence in
HR / L&D Spend
Line Leaders Not Satisfied with
Effectiveness of L&D
CFOs that feel HR / L&D are spending
the right amounts in the right places
Line Leaders Reporting Satisfaction with the Overall
Effectiveness of the L&D Function
12%
Confident or
Highly Confident
23%
Agree or
Strongly Agree
Source: CEB 2014 CEB Overhead Cost Management Survey
Source: CEB 2011 L&D Team Capabilities Survey
3
© 2014 CEB. All rights reserved.
Fundamental Disconnect
Top 5 L&D Metrics
Reported to Business
Top 3 Business
Questions for L&D
1.
Training Expense per Employee
2.
Satisfaction with Training
3.
Training Hours per FTE
4.
External Vendor Expense
5.
L&D Cost per FTE
1.
Results: To what degree will a
learning program improve a
specific business outcome?
2.
Value: What will be the return on
the learning investment?
3.
Application: How can we
increase application of new skills
on the job?
Source: How Executives View Learning Metrics
by Patti and Jack Phillips, CLO Magazine, Dec. 2010
Source: CEB Corporate Leadership Council Analytics Survey, 2013
4
© 2014 CEB. All rights reserved.
Predictive Stats: Exercise
Which aspects of training predict business impact?
5
© 2014 CEB. All rights reserved.
Audience Input
Scrap
Learning that is delivered but
= not applied back on the job
Learning
What causes scrap?
6
© 2014 CEB. All rights reserved.
Scrap Learning Root Causes
Ineffective
delivery
Content not
directly relevant
Low learner
motivation
Content quality
issues
Wrong learners
attend
No opportunity to
apply
Examples don’t
connect
Misalignment
with priorities
Low organizational
support
Insufficient
practice
Delivered at
wrong time
Insufficient time to
apply
Inadequate
support materials
Learners already
know info
Lack of manager
support
Direct L&D
Control
Business
Environment
7
© 2014 CEB. All rights reserved.
Scrap Learning is Rampant
Scrap Learning by Industry
Accommodation and Food Services
Aerospace & Defense
Banking
Business Services
Chemicals
Computer Hardware
Computer Software
Consumer Products Manufacturers
Energy & Utilities
Financial Services
Government
Health Care
Industrial Manufacturing
Insurance
Pharmaceuticals
Retail
Transportation Services
41%
43%
41%
43%
53%
48%
49%
46%
48%
41%
50%
48%
47%
47%
45%
37%
46%
0%
Source: CEB 2014 Training Effectiveness Dashboard, N=27,095
10%
20%
30%
40%
Scrap Learning
8
© 2014 CEB. All rights reserved.
50%
60%
70%
80%
Impactful Training
90%
100%
Make an Impact
Scrap Learning
Performance Improvement
Typical Company
45%
MTM Client Average
within 6 Months
38%
MTM Client Average
33%
MTM Top 25%
MTM Top 5%
6%
8%
11%
18%
20%
10%
29%
Source: CEB Analysis 2014
9
© 2014 CEB. All rights reserved.
Reduce Scrap, Increase Performance
>300 organizations
>18 million evaluations
40%
Average Organization Measuring Scrap
33% Scrap Learning
11% Performance Increase
30%
20%
Average Organization
45% Scrap Learning
6% Performance Increase
10%
0%
60%
Source: CEB Analysis 2014
50%
40%
30%
20%
Scrap Learning Rate
10
© 2014 CEB. All rights reserved.
10%
0%
Performance Gain Due to Learning
= Learning Organization
Financial Ramifications
Annually, for an organization of 10,000 employees:
Average Organization
Waste in Learning Budget
Due to Scrap
Value of Performance
Improvement Due to Learning
$5.4 Million
$36 Million
45%
scrap1
X $1,195 spend per
10,000 employees
learner2
X
6% performance improvement3 X $60,000
average salary X 10,000 employees
Average Organization
Measuring Scrap
$3.9 Million
$60 Million
33% scrap3 X $1,195 spend per learner2 X
10,000 employees
10% performance improvement3 X $60,000
average salary X 10,000 employees
Unrealized Gains for
Average Organization
$1.5 Million
in Waste
$24 Million
in Opportunity Costs
1CEB
2014 Training Effectiveness Dashboard
2ASTD 2013 State of the Industry Report
3CEB Analysis 2014
Next month, suboptimal
learning will cost $2.1 Million
11
© 2014 CEB. All rights reserved.
Profile of the Metrics That Matter Top 25%
½
3x
400%
Amount of
Scrap
Learning
Performance
Gain Due
to Learning
Higher
Learning
ROI
Attributes
 Using Metrics That Matter for 2+ years
 Leverage recommended best practices
 Focus on continuous improvement
 Scale measurement to enterprise over time
Source: CEB Analysis 2014
12
© 2014 CEB. All rights reserved.
Sample
Organizations
A New Way Forward
Reducing scrap learning and improving impact requires a fundamentally different
approach to measurement that pinpoints what is working, what is not working, and why
Current State
Future State
Benefit
Metrics
Volume, Cost,
Satisfaction
Efficiency, Effectiveness,
Outcomes
Comprehensive view of learning
impact
Source
Learner
Learner, Instructor,
Manager, Business data
Triangulate perspectives to uncover
gaps and demonstrate value
Timing
After learning
After learning and
On-the-Job
Identify drivers of scrap that occur
outside of training
Benchmarks
Internal
Internal and External
Prioritize improvements based on
comparison to competitors
Process
Manually
intensive
Highly automated
Scale measurement to be
consistent and have significant
impact across the enterprise
13
© 2014 CEB. All rights reserved.
Scrap learning and Manager Support
Robert Brinkerhoff, Ph.D. focuses on the influence of managers and the
support they can provide before and after training.
Job
Performance
Manager
engagement /
support extends the
“length” of training?
Organization
14
© 2014 CEB. All rights reserved.
Manager Support
Manager support comes both before and after training
Key steps include:
Evaluate learner
readiness
Training
Get involved after
training
Follow up on
expectations
Expectation setting
Provide performance
resources
15
© 2014 CEB. All rights reserved.
Learner Readiness
Sending the right person to the right training is essential. However, only
21% of organizations pre-evaluate learners “most of the time” or “all of
the time” prior to sending them to training.
60%
Do you pre-assess learners to
determine their readiness for
training?
50%
50%
40%
30%
29%
20%
16%
10%
5%
0%
No, none of the time
Yes, some of the time
Yes, most of the time
16
© 2014 CEB. All rights reserved.
Yes, all of the time
Expectation Setting
Setting learning expectations can often increase attention and learning.
Yet, 75% of companies indicate that managers set post-training
performance expectations with learners less than 25% of the time.
70%
61%
How often do you set expectations
prior to training?
60%
50%
40%
30%
20%
16%
14%
9%
10%
0%
0%
1 to 25%
51 to 75%
17
© 2014 CEB. All rights reserved.
76% to 100%
Manager Involvement
While 52% of managers appear to have some level of involvement—encouraging
application of training–there is a large minority (44%) that does not. Scrap learning can
be reduced by getting managers involved.
"I have little involvement in how my employees use what they
learned back on the job.”
44%
“I will encourage my employee to use the training they take
back on the job.”
41%
52%
“I will hold my employees accountable for how they use the
training back on the job.”
11%
Other
4%
0%
5%
10% 15% 20% 25% 30% 35% 40% 45% 50%
18
© 2014 CEB. All rights reserved.
Follow up on Expectations
Managers require follow up actions in various forms. As the complexity of the required
action increases, the percentage of managers who require the action decreases. The
most complex requirement “measurement of a business result change” was least often
selected at 13%.
Other
35%
Require a summary debrief to the manager or team to share what was learned
35%
Require demonstration of the learning within a reasonable time frame
32%
Provide a specific program or project to use the training
19%
Require an action plan be created to outline how the training will be used on the job
16%
Require measurement of a business result change within a reasonable time frame
13%
0%
5%
10%
15%
20%
25%
30%
Other: 61% of “other” comments said managers do “nothing”. Another 24% indicated follow
up varies by manager or was inconsistent.
19
© 2014 CEB. All rights reserved.
35%
40%
Provide Resources
Managers support learners with a variety of resources—most often (25%) by publicly
recognizing successful application of training. It is safe to say that scrap learning could be
reduced if any and all of these resources were applied more often.
Other
43%
Managers publicly recognize and celebrate successful application of training on the job
25%
Managers formally observe and provide feedback to learners within 90 days after training
22%
Managers re-prioritize a learner’s daily tasks after learning to emphasize use of training
21%
Time is specifically set aside by managers to allow learners to try new concepts learned
16%
Money is allocated by managers to learners to fund new ideas implemented on the job
9%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
Other: 67% of “other” comments indicated that managers did not actively support learners
after training. Another 16% indicated that manager support was inconsistent.
20
© 2014 CEB. All rights reserved.
50%
Manager Support: Summary
 Here are the general estimates of manager support for selected
activities
 Results indicate that most managers are not substantially involved with
supporting direct reports before or after training
21%
Evaluate learner
readiness
Training
Get involved after
training
25%
52%
35%
Follow up on
expectations
Expectation setting
25%
Provide performance
resources
21
© 2014 CEB. All rights reserved.
Manager Support and Performance
• Correlations between
“Transfer of Learning” and
other factors
• All are statistically significant
• Based on responses from
managers, when they are
engaged, they contribute
17.5% to training transfer
and performance on the job.
Source: Training Industry Quarterly Magazine
http://www.cedmaeurope.org/newsletter%20articles/TrainingOutsourcing/Manager%20Engagement
%20-%20Reducing%20Scrap%20Learning%20(Oct%2010).pdf
•
•
N = 93,806 ** N = 1,286
0.419 * 0.419 = .175 or 17.5%
22
© 2014 CEB. All rights reserved.
Making Your Programs Better
The road to improvement is filled with measurement
• Evaluate programs using an appropriate model (e.g., Kirkpatrick)
• Use standard processes and tools (e.g., Metrics That MatterTM)
• Compare results to benchmarks
• Measure scrap and other factors that lead to performance
improvement
23
© 2014 CEB. All rights reserved.
Thank You
John R. Mattox, II, Ph.D.
Senior Consultant, CEB Metrics That Matter
615 714 7299
[email protected]
24
© 2014 CEB. All rights reserved.