Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Scrap Learning—Your Programs Are Not As Good as You Think They Are Training Industry Conference & Exposition 2015 May 7, 2015 8:30 ET John R. Mattox, II, Ph.D. Senior Consultant, Talent Solutions Doing More with Less New Learning Environment Business Demands Constrained Resources OTJ / JIT Rapid deployment Tightening budgets Social Learning Customize everything Expanding responsibilities Gamification Minimize learners’ time Lack of manager support Increased Complexity in L&D Solution Set Employee Development Critical for Business Outcomes Heads of L&D Reporting Increased Complexity of L&D Solutions Line Leaders Identifying Employee Development as Critical to Achieving Business Outcomes 93% Agree or Strongly Agree 86% Agree or Strongly Agree Source: CEB 2012 L&D Team Capabilities Survey 2 © 2014 CEB. All rights reserved. Falling Short of Expectations CEOs Expect More Insights CEOs Opinions Regarding Information on Return on Investment in Human Capital Information is Important Receives Sufficient Information 0% 50% 100% Source: PwC 2012 Annual Global CEO Survey CFOs Lack Confidence in HR / L&D Spend Line Leaders Not Satisfied with Effectiveness of L&D CFOs that feel HR / L&D are spending the right amounts in the right places Line Leaders Reporting Satisfaction with the Overall Effectiveness of the L&D Function 12% Confident or Highly Confident 23% Agree or Strongly Agree Source: CEB 2014 CEB Overhead Cost Management Survey Source: CEB 2011 L&D Team Capabilities Survey 3 © 2014 CEB. All rights reserved. Fundamental Disconnect Top 5 L&D Metrics Reported to Business Top 3 Business Questions for L&D 1. Training Expense per Employee 2. Satisfaction with Training 3. Training Hours per FTE 4. External Vendor Expense 5. L&D Cost per FTE 1. Results: To what degree will a learning program improve a specific business outcome? 2. Value: What will be the return on the learning investment? 3. Application: How can we increase application of new skills on the job? Source: How Executives View Learning Metrics by Patti and Jack Phillips, CLO Magazine, Dec. 2010 Source: CEB Corporate Leadership Council Analytics Survey, 2013 4 © 2014 CEB. All rights reserved. Predictive Stats: Exercise Which aspects of training predict business impact? 5 © 2014 CEB. All rights reserved. Audience Input Scrap Learning that is delivered but = not applied back on the job Learning What causes scrap? 6 © 2014 CEB. All rights reserved. Scrap Learning Root Causes Ineffective delivery Content not directly relevant Low learner motivation Content quality issues Wrong learners attend No opportunity to apply Examples don’t connect Misalignment with priorities Low organizational support Insufficient practice Delivered at wrong time Insufficient time to apply Inadequate support materials Learners already know info Lack of manager support Direct L&D Control Business Environment 7 © 2014 CEB. All rights reserved. Scrap Learning is Rampant Scrap Learning by Industry Accommodation and Food Services Aerospace & Defense Banking Business Services Chemicals Computer Hardware Computer Software Consumer Products Manufacturers Energy & Utilities Financial Services Government Health Care Industrial Manufacturing Insurance Pharmaceuticals Retail Transportation Services 41% 43% 41% 43% 53% 48% 49% 46% 48% 41% 50% 48% 47% 47% 45% 37% 46% 0% Source: CEB 2014 Training Effectiveness Dashboard, N=27,095 10% 20% 30% 40% Scrap Learning 8 © 2014 CEB. All rights reserved. 50% 60% 70% 80% Impactful Training 90% 100% Make an Impact Scrap Learning Performance Improvement Typical Company 45% MTM Client Average within 6 Months 38% MTM Client Average 33% MTM Top 25% MTM Top 5% 6% 8% 11% 18% 20% 10% 29% Source: CEB Analysis 2014 9 © 2014 CEB. All rights reserved. Reduce Scrap, Increase Performance >300 organizations >18 million evaluations 40% Average Organization Measuring Scrap 33% Scrap Learning 11% Performance Increase 30% 20% Average Organization 45% Scrap Learning 6% Performance Increase 10% 0% 60% Source: CEB Analysis 2014 50% 40% 30% 20% Scrap Learning Rate 10 © 2014 CEB. All rights reserved. 10% 0% Performance Gain Due to Learning = Learning Organization Financial Ramifications Annually, for an organization of 10,000 employees: Average Organization Waste in Learning Budget Due to Scrap Value of Performance Improvement Due to Learning $5.4 Million $36 Million 45% scrap1 X $1,195 spend per 10,000 employees learner2 X 6% performance improvement3 X $60,000 average salary X 10,000 employees Average Organization Measuring Scrap $3.9 Million $60 Million 33% scrap3 X $1,195 spend per learner2 X 10,000 employees 10% performance improvement3 X $60,000 average salary X 10,000 employees Unrealized Gains for Average Organization $1.5 Million in Waste $24 Million in Opportunity Costs 1CEB 2014 Training Effectiveness Dashboard 2ASTD 2013 State of the Industry Report 3CEB Analysis 2014 Next month, suboptimal learning will cost $2.1 Million 11 © 2014 CEB. All rights reserved. Profile of the Metrics That Matter Top 25% ½ 3x 400% Amount of Scrap Learning Performance Gain Due to Learning Higher Learning ROI Attributes Using Metrics That Matter for 2+ years Leverage recommended best practices Focus on continuous improvement Scale measurement to enterprise over time Source: CEB Analysis 2014 12 © 2014 CEB. All rights reserved. Sample Organizations A New Way Forward Reducing scrap learning and improving impact requires a fundamentally different approach to measurement that pinpoints what is working, what is not working, and why Current State Future State Benefit Metrics Volume, Cost, Satisfaction Efficiency, Effectiveness, Outcomes Comprehensive view of learning impact Source Learner Learner, Instructor, Manager, Business data Triangulate perspectives to uncover gaps and demonstrate value Timing After learning After learning and On-the-Job Identify drivers of scrap that occur outside of training Benchmarks Internal Internal and External Prioritize improvements based on comparison to competitors Process Manually intensive Highly automated Scale measurement to be consistent and have significant impact across the enterprise 13 © 2014 CEB. All rights reserved. Scrap learning and Manager Support Robert Brinkerhoff, Ph.D. focuses on the influence of managers and the support they can provide before and after training. Job Performance Manager engagement / support extends the “length” of training? Organization 14 © 2014 CEB. All rights reserved. Manager Support Manager support comes both before and after training Key steps include: Evaluate learner readiness Training Get involved after training Follow up on expectations Expectation setting Provide performance resources 15 © 2014 CEB. All rights reserved. Learner Readiness Sending the right person to the right training is essential. However, only 21% of organizations pre-evaluate learners “most of the time” or “all of the time” prior to sending them to training. 60% Do you pre-assess learners to determine their readiness for training? 50% 50% 40% 30% 29% 20% 16% 10% 5% 0% No, none of the time Yes, some of the time Yes, most of the time 16 © 2014 CEB. All rights reserved. Yes, all of the time Expectation Setting Setting learning expectations can often increase attention and learning. Yet, 75% of companies indicate that managers set post-training performance expectations with learners less than 25% of the time. 70% 61% How often do you set expectations prior to training? 60% 50% 40% 30% 20% 16% 14% 9% 10% 0% 0% 1 to 25% 51 to 75% 17 © 2014 CEB. All rights reserved. 76% to 100% Manager Involvement While 52% of managers appear to have some level of involvement—encouraging application of training–there is a large minority (44%) that does not. Scrap learning can be reduced by getting managers involved. "I have little involvement in how my employees use what they learned back on the job.” 44% “I will encourage my employee to use the training they take back on the job.” 41% 52% “I will hold my employees accountable for how they use the training back on the job.” 11% Other 4% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 18 © 2014 CEB. All rights reserved. Follow up on Expectations Managers require follow up actions in various forms. As the complexity of the required action increases, the percentage of managers who require the action decreases. The most complex requirement “measurement of a business result change” was least often selected at 13%. Other 35% Require a summary debrief to the manager or team to share what was learned 35% Require demonstration of the learning within a reasonable time frame 32% Provide a specific program or project to use the training 19% Require an action plan be created to outline how the training will be used on the job 16% Require measurement of a business result change within a reasonable time frame 13% 0% 5% 10% 15% 20% 25% 30% Other: 61% of “other” comments said managers do “nothing”. Another 24% indicated follow up varies by manager or was inconsistent. 19 © 2014 CEB. All rights reserved. 35% 40% Provide Resources Managers support learners with a variety of resources—most often (25%) by publicly recognizing successful application of training. It is safe to say that scrap learning could be reduced if any and all of these resources were applied more often. Other 43% Managers publicly recognize and celebrate successful application of training on the job 25% Managers formally observe and provide feedback to learners within 90 days after training 22% Managers re-prioritize a learner’s daily tasks after learning to emphasize use of training 21% Time is specifically set aside by managers to allow learners to try new concepts learned 16% Money is allocated by managers to learners to fund new ideas implemented on the job 9% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% Other: 67% of “other” comments indicated that managers did not actively support learners after training. Another 16% indicated that manager support was inconsistent. 20 © 2014 CEB. All rights reserved. 50% Manager Support: Summary Here are the general estimates of manager support for selected activities Results indicate that most managers are not substantially involved with supporting direct reports before or after training 21% Evaluate learner readiness Training Get involved after training 25% 52% 35% Follow up on expectations Expectation setting 25% Provide performance resources 21 © 2014 CEB. All rights reserved. Manager Support and Performance • Correlations between “Transfer of Learning” and other factors • All are statistically significant • Based on responses from managers, when they are engaged, they contribute 17.5% to training transfer and performance on the job. Source: Training Industry Quarterly Magazine http://www.cedmaeurope.org/newsletter%20articles/TrainingOutsourcing/Manager%20Engagement %20-%20Reducing%20Scrap%20Learning%20(Oct%2010).pdf • • N = 93,806 ** N = 1,286 0.419 * 0.419 = .175 or 17.5% 22 © 2014 CEB. All rights reserved. Making Your Programs Better The road to improvement is filled with measurement • Evaluate programs using an appropriate model (e.g., Kirkpatrick) • Use standard processes and tools (e.g., Metrics That MatterTM) • Compare results to benchmarks • Measure scrap and other factors that lead to performance improvement 23 © 2014 CEB. All rights reserved. Thank You John R. Mattox, II, Ph.D. Senior Consultant, CEB Metrics That Matter 615 714 7299 [email protected] 24 © 2014 CEB. All rights reserved.